This page gives you a brief introduction on writing your own Magi Agent. It is designed to give you sample code, briefly explain it, then show you the pieces needed to run it. After reading this page you should be able to write and run a basic Magi Agent. Further details and more advanced agent information can be found in the Magi Agent Library document.
An agent runs in two modes: a threaded mode and a process mode. In the process mode, the Magi Daemon runs the agent in a process space separate from itself and communicates with the agent via a pipe or a socket. In the threaded mode, the Magi Daemon loads python codes directly, and runs the agent in a thread in its own process space.
In most cases you will want to use the Orchestrator and an AAL file to run and coordinate your agent actions. In order to get the basic agent control (via remote procedure calls), you’ll need to derive your agent from the DispatchAgent base class.
The DispatchAgent implements the simple remote procedure call (RPC) mechanism used by the orchestrator (and available through the Magi python API). This allows any method in a class derived from DispatchAgent to be invoked in an AAL (or by a MagiMessage if using the Magi python interface directly). You almost always want to derive your agent from DispatchAgent. The DispatchAgent code simply loops, parsing incoming messages, looking for an event message. When it finds one, it attempts to call the method specified in the message with the arguments given in the message. You needn’t worry about message handling or parsing when you derive from DispatchAgent, you can simply write your code and call that code from AALs.
To write and execute your agent you need three things: the Agent code, an interface description file (IDL), and an AAL file. The Agent code implements the Agent. The IDL file describes the Agent function and specifies things the Magi Daemon needs to know to load and execute the Agent code. (Among these is the location of the agent code and the execution mode.)
For this document, we will create a simple agent which creates a single file on a host. The agent is called FileCreator. For this example, the agent would run in threaded mode. The entire agent implementation of FileCreator is:
from magi.util.agent import DispatchAgent, agentmethod
from magi.util.processAgent import initializeProcessAgent
# The FileCreator agent implementation, derived from DispatchAgent.
class FileCreator(DispatchAgent):
def __init__(self):
DispatchAgent.__init__(self)
self.filename = '/tmp/newfile'
# A single method which creates the file named by self.filename.
# (The @agentmethod() decorator is not required, but is encouraged.
# it does nothing of substance now, but may in the future.)
@agentmethod()
def createFile(self, msg):
'''Create a file on the host.'''
# open and immediately close the file to create it.
open(self.filename, 'w').close()
# the getAgent() method must be defined somewhere for all agents.
# The Magi daemon invokes this mehod to get a reference to an
# agent. It uses this reference to run and interact with an agent
# instance.
def getAgent():
agent = FileCreator()
return agent
# In case the agent is run as a separate process, we need to
# create an instance of the agent, initialize the required
# parameters based on the received arguments, and then call the
# run method defined in DispatchAgent.
if __name__ == "__main__":
agent = FileCreator()
initializeProcessAgent(agent, sys.argv)
agent.run()
This is the complete agent code. It has a single method, createFile, which creates the file “/tmp/newfile” when called. Every agent to run in threaded mode needs to implement a method named getAgent which returns an instance of the agent to run. The Magi Daemon uses this method to get an instance of the agent to run as a local thread and communicate with the agent instance. Also, in order for an agent to run as a separate process, it should be runnable as a standalone process, which can be done by creating an insatnce of the agent, initilize it based on the input parameters, and then run it.
The IDL file that goes along with the agent implementation is presented below. Note that the execution mode is “thread” and the inheritance is specified as “DispatchAgent”. When you run this, you need to specify the name of your implementation file. This IDL assumes the implementation file above is in the local directory and is named “FileCreator.py”. Note that it lists methods and internal variables that the author wants exposed to external configuration. In this case, we expose the variable “filename” that represents the path of the file to be created. The agent implementation has a default value set for the filename variable. Later we will show how to set this outside of the agent implementation.
name: FileCreator
display: File Creator Agent
description: This agent creates a file on the test node.
execute: thread
mainfile: FileCreator.py
inherits:
- DispatchAgent
methods:
- name: createFile
help: Create the file
args: []
variables:
- name: filename
help: the full path of the file to create
type: string
Given the Agent implementation and IDL above and the AAL fragment below, the method createFile will be called on all test nodes associated with myAgentGroup in the complete AAL file.
eventstreams:
myStream:
- type: event
agent: myFileCreators
method: createFile
args: { }
Magi agents are usually contained to a single directory. Create a local directory named “FileCreator” and copy the agent implementation code above to the file “FileCreator/FileCreator.py”. Now copy the IDL above to a file named “FileCreator/FileCreator.idl”. The file and directory can be named anything, but if you deviate from the naming scheme given, make sure the mainfile setting in the IDL and the code setting in your AAL (below) matches your naming scheme.
Use the sample AAL file below, replacing the “PATH” string with the full path to your “FileCreator” directory. Note: the PATH you use must the NFS mounted location on the test nodes. Also replace the the “NODES” string with the comma-separated list of nodes on your testbed on which you want to run the agent.
streamstarts: [main]
groups:
myFileCreatorGroup [NODES]
agents:
myFileCreators:
group: myFileCreatorGroup
# (note: the "path" argument is the agent directory. The
# direcory must contain an IDL and agent implementation.)
path: PATH/FileCreator
execargs: []
eventstreams:
main:
- type: event
agent: myFileCreators
method: createFile
args: { }
(Note that YAML files, which the AAL is, cannot have tabs, so if you are cutting and pasting this, make sure to remove possibly inserted tabs.)
Because your agent code is on an NFS-mounted filesystem, Magi Daemon on any of the experiment nodes can read the code directly. If this is not the case, Magi supplies a script, agentTool, to copy your agent code to the nodes on the testbed.
Once you have your AAL file, IDL file, and Agent code ready, you run the Magi Orchestrator to run the event streams in your AAL file - and thus your agent code. Give the AAL file and the control node in your experiment to the Magi Orchestrator like below (where $control is the fully qualified domain of your experiment’s control node, i.e. controlNode.myExperiment.myProject on DETER):
magi_orchestrator --control $control --events myEvents.aal -o run.log -v
This command runs the Orchestrator. It connects to the $control node, runs the events in the myEvents.aal file, writes verbose output to run.log. On standard out, you should see Orchestrator output like the following:
stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__
stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete.
stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup
stream initialization : done : trigger AgentLoadDone myFileCreators complete.
stream initialization : DONE : complete.
stream main : sent : createFile(None) --> myFileCreatorGroup
stream main : DONE : complete.
This shows the two execution streams the orchestrator runs. The first, initialization is internal to the Orchestrator and sets up group communication and loads the agents. The second, main, is the event stream specified in your AAL file.
If you do not see the “correct” output above, refer to the Troubleshooting section below. To confirm that the agent ran and executed the createFile method, do the following (from users on DETER):
ssh myNode.myExperiment.myGroup ls -l /tmp/newfile
Where “myNode.myExperiment.myGroup” is the domain name of a node on which you executed the Agent.
The sample code can be downloaded as a tar file here: FileCreator.tbz.
The sample Agent FileCreator always creates the same file, each time it is run. What if you wanted to create a different file? Or a series of files? It is possible to specify Agent configuration in an AAL file - configuration that can modify the internal variables of your agent at run time. See Magi Agent Configuration for details.
Error: No such file:
Run-time exception in agent <Daemon(daemon, started 140555403872000)> on node(s) control in method __init__, line 71, in file threadInterface.py. Error: [Errno 2] No such file or directory
Solution: You probably did not specify the correct “mainfile” in the IDL. It must not be pathed out and must match the name of the main agent implementation file, for example, “myAgent.py”.
Error: no attribute ‘getAgent’
Error: 'module' object has no attribute 'getAgent'
Solution: When an agent is run in threaded mode, the magi daemon expects the getAgent to exist in the agent module. So make sure your agent defines (and exports or makes available) a getAgent method and that returns an instance of your agent.