This page gives you a brief introduction on writing your own MAGI agent. It is designed to give you sample code, briefly explain it, then show you the pieces needed to run it. After reading this page you should be able to write and run a basic MAGI agent. Further details and more advanced agent information can be found in the Magi Agent Library document.
An agent runs in two modes: a threaded mode and a process mode. In the threaded mode, the MAGI daemon runs the agent as a thread in its own process space. Whereas, in the process mode, the MAGI daemon runs the agent in a process space separate from itself and communicates with the agent via a pipe or a socket.
In most cases you will want to use the Orchestrator and an Agent Activation Language file to run and coordinate your agent actions. In order to get the basic agent control (via remote procedure calls), you’ll need to derive your agent from the DispatchAgent base class.
The DispatchAgent implements a simple remote procedure call (RPC) mechanism used by the orchestrator (and available through the MAGI messaging API). This allows any method in a class derived from DispatchAgent to be invoked through an AAL/orchestrator (or directly using the messaging API). You almost always want to derive your agent from DispatchAgent. The DispatchAgent code simply loops, parsing incoming messages, looking for an event message. When it finds one, it attempts to call the method specified in the message with the arguments given in the message. You needn’t worry about message handling or parsing, you can simply write your agent implementation code and call the required methods.
To write and execute your agent you need three things: the agent code, an interface description file (IDL), and an AAL file. The agent code implements the agent. The IDL file describes the agent functions and specifies other configuration details, such as the location of the agent code and the execution mode, that help the MAGI daemon load and execute the agent code.
For this document, we will create a simple agent which creates a single file on a host. The agent is called FileCreator. For this example, the agent would run in threaded mode. The entire agent implementation of FileCreator is:
from magi.util.agent import DispatchAgent, agentmethod
from magi.util.processAgent import initializeProcessAgent
# The FileCreator agent implementation, derived from DispatchAgent.
class FileCreator(DispatchAgent):
def __init__(self):
DispatchAgent.__init__(self)
self.filename = '/tmp/newfile'
# A single method which creates the file named by self.filename.
# (The @agentmethod() decorator is not required, but is encouraged.
# it does nothing of substance now, but may in the future.)
@agentmethod()
def createFile(self, msg):
'''Create a file on the host.'''
# open and immediately close the file to create it.
open(self.filename, 'w').close()
# the getAgent() method must be defined somewhere for all agents.
# The Magi daemon invokes this mehod to get a reference to an
# agent. It uses this reference to run and interact with an agent
# instance.
def getAgent():
agent = FileCreator()
return agent
# In case the agent is run as a separate process, we need to
# create an instance of the agent, initialize the required
# parameters based on the received arguments, and then call the
# run method defined in DispatchAgent.
if __name__ == "__main__":
agent = FileCreator()
initializeProcessAgent(agent, sys.argv)
agent.run()
This is the complete agent code. It has a single method, createFile, which creates the file “/tmp/newfile” when called. Every agent, to run in threaded mode, needs to implement a method named getAgent which returns an instance of the agent to run. The MAGI daemon uses this method to get an instance of the agent to run as a local thread, and communicate with the agent instance. In order for an agent to run as a separate process, it should be runnable as a standalone process. When executed, the excutable should create an instance of the agent, initialize the instance based on the input parameters, and then run it.
The IDL file that goes along with the agent implementation is presented below. Note that the execution mode is “thread” and the inheritance is specified as “DispatchAgent”. The main agent implementation file that needs to be invoked is “FileCreator.py”, in the same directory as the IDL. The MAGI daemon reads the IDL file to get the agent execution information, and based on the same, executes the agent.
The IDL also lists other information about the agent, such as the variables and the methods that need to be exposed externally, and metadata about the information that the agent records. In this example, we expose the variable “filename” that represents the path of the file to be created. This information can be used by experimenters to understand the functioning of the agent, and also by tools such as MAGI Graph Creation Tool.
name: FileCreator
display: File Creator Agent
description: This agent creates a file on the test node.
execute: thread
mainfile: FileCreator.py
inherits:
- DispatchAgent
methods:
- name: createFile
help: Create the file
args: []
variables:
- name: filename
help: the full path of the file to create
type: string
Given the agent implementation and IDL above and the AAL fragment below, the method createFile will be called on all the experiment nodes associated with myAgentGroup in the complete AAL file.
The agent implementation has a default value set for the filename variable. First, we will use the default filename to create a file. Later we will show how to set this outside of the agent implementation.
eventstreams:
myStream:
- type: event
agent: myFileCreators
method: createFile
args: { }
MAGI agents are usually contained to a single directory. Create a local directory named “FileCreator” and copy the agent implementation code above to the file “FileCreator/FileCreator.py”. Now copy the IDL above to a file named “FileCreator/FileCreator.idl”. The file and directory can be named anything, but if you deviate from the naming scheme given, make sure the mainfile configuration in the IDL and the agent path in your AAL (below) matches your naming scheme.
Use the sample AAL file below, replacing the “PATH” string with the full path to your “FileCreator” directory. Note: the PATH you use must be a NFS mounted location on the experiment nodes. Also replace the the “NODES” string with the comma-separated list of nodes on your testbed on which you want to run the agent.
streamstarts: [main]
groups:
myFileCreatorGroup: [NODES]
agents:
myFileCreators:
group: myFileCreatorGroup
# (note: the "PATH" argument is the agent directory. The
# direcory must contain an IDL and agent implementation.)
path: PATH/FileCreator
execargs: []
eventstreams:
main:
- type: event
agent: myFileCreators
method: createFile
args: {}
Note
The AAL is a YAML formatted file. YAML files cannot have tabs, so if you are cutting and pasting this, make sure to remove possibly inserted tabs.
Once you have your AAL file, IDL file, and the agent code ready, you run the MAGI orchestrator to execute the event streams in your AAL file, and thus your agent code. Run the orchestrator passing it the AAL file location, and the project and experiment name to similar to the following.
/share/magi/current/magi_orchestrator.py --project myProject --experiment myExperiment --events myEvents.aal -o run.log -v
This command runs the orchestrator. The orchestrator connects to the experiment’s messaging layer, executes the events in the myEvents.aal file, and writes verbose output to run.log. On standard out, you should see orchestrator output like the following.
stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__
stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete.
stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup
stream initialization : done : trigger AgentLoadDone myFileCreators complete.
stream initialization : DONE : complete.
stream main : sent : createFile(None) --> myFileCreatorGroup
stream main : DONE : complete.
The output shows the two execution streams that the orchestrator runs. The first stream, initialization, is internal to the orchestrator and is run implicitly. It sets up group communication and loads the agents. The second, main, is the event stream specified in your AAL file.
If you do not see the “correct” output above, refer to the Troubleshooting section below. To confirm that the agent ran and executed the createFile method, do the following (from DETER Ops (users.isi.deterlab.net)):
ssh myNode.myExperiment.myProject ls -l /tmp/newfile
Where “myNode.myExperiment.myProject” is the domain name of a node on which you executed the agent.
The sample code can be downloaded as a tar file here: FileCreator.tbz.
The sample FileCreator agent always creates the same file, each time it is run. What if you wanted to create a different file? Or a series of files? It is possible to specify agnet configuration in an AAL file, configuration that can modify the internal variables of your agent at run time. See Magi Agent Configuration for details.
Error: No such file:
Run-time exception in agent <Daemon(daemon, started 140555403872000)> on node(s) control in method __init__, line 71, in file threadInterface.py. Error: [Errno 2] No such file or directory
Solution: You probably did not specify the correct “mainfile” in the IDL. It must not be pathed out and must match the name of the main agent implementation file, for example, “myAgent.py”.
Error: no attribute ‘getAgent’
Error: 'module' object has no attribute 'getAgent'
Solution: When an agent is run in threaded mode, the magi daemon expects the getAgent to exist in the agent module. So make sure your agent defines (and exports or makes available) a getAgent method and that returns an instance of your agent.