.. _specialize: Specialized User ================ This page gives you a brief introduction on writing your own MAGI agent. It is designed to give you sample code, briefly explain it, then show you the pieces needed to run it. After reading this page you should be able to write and run a basic MAGI agent. Further details and more advanced agent information can be found in the :doc:`agent-library` document. Basic Agent Information +++++++++++++++++++++++ An agent runs in two modes: a threaded mode and a process mode. In the threaded mode, the MAGI daemon runs the agent as a thread in its own process space. Whereas, in the process mode, the MAGI daemon runs the agent in a process space separate from itself and communicates with the agent via a pipe or a socket. DispatchAgent +++++++++++++ In most cases you will want to use the :ref:`orchestrator` and an :ref:`aal` file to run and coordinate your agent actions. In order to get the basic agent control (via remote procedure calls), you'll need to derive your agent from the *DispatchAgent* base class. The *DispatchAgent* implements a simple remote procedure call (RPC) mechanism used by the orchestrator (and available through the MAGI messaging API). This allows any method in a class derived from *DispatchAgent* to be invoked through an AAL/orchestrator (or directly using the messaging API). You almost always want to derive your agent from *DispatchAgent*. The *DispatchAgent* code simply loops, parsing incoming messages, looking for an event message. When it finds one, it attempts to call the method specified in the message with the arguments given in the message. You needn't worry about message handling or parsing, you can simply write your agent implementation code and call the required methods. To write and execute your agent you need three things: the agent code, an interface description file (IDL), and an AAL file. The agent code implements the agent. The IDL file describes the agent functions and specifies other configuration details, such as the location of the agent code and the execution mode, that help the MAGI daemon load and execute the agent code. For this document, we will create a simple agent which creates a single file on a host. The agent is called FileCreator. For this example, the agent would run in threaded mode. The entire agent implementation of FileCreator is: .. code-block:: python from magi.util.agent import DispatchAgent, agentmethod from magi.util.processAgent import initializeProcessAgent # The FileCreator agent implementation, derived from DispatchAgent. class FileCreator(DispatchAgent): def __init__(self): DispatchAgent.__init__(self) self.filename = '/tmp/newfile' # A single method which creates the file named by self.filename. # (The @agentmethod() decorator is not required, but is encouraged. # it does nothing of substance now, but may in the future.) @agentmethod() def createFile(self, msg): '''Create a file on the host.''' # open and immediately close the file to create it. open(self.filename, 'w').close() # the getAgent() method must be defined somewhere for all agents. # The Magi daemon invokes this mehod to get a reference to an # agent. It uses this reference to run and interact with an agent # instance. def getAgent(): agent = FileCreator() return agent # In case the agent is run as a separate process, we need to # create an instance of the agent, initialize the required # parameters based on the received arguments, and then call the # run method defined in DispatchAgent. if __name__ == "__main__": agent = FileCreator() initializeProcessAgent(agent, sys.argv) agent.run() This is the complete agent code. It has a single method, *createFile*, which creates the file "/tmp/newfile" when called. Every agent, to run in threaded mode, needs to implement a method named *getAgent* which returns an instance of the agent to run. The MAGI daemon uses this method to get an instance of the agent to run as a local thread, and communicate with the agent instance. In order for an agent to run as a separate process, it should be runnable as a standalone process. When executed, the excutable should create an instance of the agent, initialize the instance based on the input parameters, and then run it. The IDL file that goes along with the agent implementation is presented below. Note that the execution mode is "thread" and the inheritance is specified as "DispatchAgent". The main agent implementation file that needs to be invoked is "FileCreator.py", in the same directory as the IDL. The MAGI daemon reads the IDL file to get the agent execution information, and based on the same, executes the agent. The IDL also lists other information about the agent, such as the variables and the methods that need to be exposed externally, and metadata about the information that the agent records. In this example, we expose the variable "filename" that represents the path of the file to be created. This information can be used by experimenters to understand the functioning of the agent, and also by tools such as :ref:`magigraph`. .. code-block:: none name: FileCreator display: File Creator Agent description: This agent creates a file on the test node. execute: thread mainfile: FileCreator.py inherits: - DispatchAgent methods: - name: createFile help: Create the file args: [] variables: - name: filename help: the full path of the file to create type: string Given the agent implementation and IDL above and the AAL fragment below, the method *createFile* will be called on all the experiment nodes associated with *myAgentGroup* in the complete AAL file. The agent implementation has a default value set for the filename variable. First, we will use the default filename to create a file. Later we will show how to set this outside of the agent implementation. .. code-block:: yaml eventstreams: myStream: - type: event agent: myFileCreators method: createFile args: { } Deployment and Execution ++++++++++++++++++++++++ MAGI agents are usually contained to a single directory. Create a local directory named "FileCreator" and copy the agent implementation code above to the file "FileCreator/FileCreator.py". Now copy the IDL above to a file named "FileCreator/FileCreator.idl". The file and directory can be named anything, but if you deviate from the naming scheme given, make sure the *mainfile* configuration in the IDL and the agent *path* in your AAL (below) matches your naming scheme. Use the sample AAL file below, replacing the "PATH" string with the full path to your "FileCreator" directory. Note: the PATH you use must be a NFS mounted location on the experiment nodes. Also replace the the "NODES" string with the comma-separated list of nodes on your testbed on which you want to run the agent. .. code-block:: yaml streamstarts: [main] groups: myFileCreatorGroup: [NODES] agents: myFileCreators: group: myFileCreatorGroup # (note: the "PATH" argument is the agent directory. The # direcory must contain an IDL and agent implementation.) path: PATH/FileCreator execargs: [] eventstreams: main: - type: event agent: myFileCreators method: createFile args: {} .. note:: The AAL is a YAML formatted file. YAML files cannot have tabs, so if you are cutting and pasting this, make sure to remove possibly inserted tabs. Once you have your AAL file, IDL file, and the agent code ready, you run the MAGI orchestrator to execute the event streams in your AAL file, and thus your agent code. Run the orchestrator passing it the AAL file location, and the project and experiment name to similar to the following. .. code-block:: bash /share/magi/current/magi_orchestrator.py --project myProject --experiment myExperiment --events myEvents.aal -o run.log -v This command runs the orchestrator. The orchestrator connects to the experiment's messaging layer, executes the events in the *myEvents.aal* file, and writes verbose output to *run.log*. On standard out, you should see orchestrator output like the following. .. code-block:: none stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__ stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete. stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup stream initialization : done : trigger AgentLoadDone myFileCreators complete. stream initialization : DONE : complete. stream main : sent : createFile(None) --> myFileCreatorGroup stream main : DONE : complete. The output shows the two execution streams that the orchestrator runs. The first stream, *initialization*, is internal to the orchestrator and is run implicitly. It sets up group communication and loads the agents. The second, *main*, is the event stream specified in your AAL file. If you do not see the "correct" output above, refer to the Troubleshooting section below. To confirm that the agent ran and executed the *createFile* method, do the following (from DETER Ops (users.isi.deterlab.net)): .. code-block:: none ssh myNode.myExperiment.myProject ls -l /tmp/newfile Where "myNode.myExperiment.myProject" is the domain name of a node on which you executed the agent. The sample code can be downloaded as a tar file here: :download:`FileCreator.tbz <_static/FileCreator.tbz>`. Runtime Agent Configuration +++++++++++++++++++++++++++ The sample *FileCreator* agent always creates the same file, each time it is run. What if you wanted to create a different file? Or a series of files? It is possible to specify agnet configuration in an AAL file, configuration that can modify the internal variables of your agent at run time. See :doc:`agent-configuration` for details. Troubleshooting +++++++++++++++ General troubleshooting techniques: * Look at the Magi Daemon log file at **/var/log/magi/daemon.log** on your experiment nodes looking for errors. If there are syntax errors in Agent execution, they may show up here. * The Magi daemon will not load Agents more than once. To restart the Magi daemon, use the *agentTool* script to restart the daemons on all nodes. The script is available in the Montage repository and available on *users* on DETER. "agentTool :option:`--help`" will give a usage statement. **Error**: No such file: .. code-block:: bash Run-time exception in agent on node(s) control in method __init__, line 71, in file threadInterface.py. Error: [Errno 2] No such file or directory **Solution**: You probably did not specify the correct "mainfile" in the IDL. It must not be pathed out and must match the name of the main agent implementation file, for example, "myAgent.py". **Error**: no attribute 'getAgent' .. code-block:: bash Error: 'module' object has no attribute 'getAgent' **Solution**: When an agent is run in threaded mode, the magi daemon expects the *getAgent* to exist in the agent module. So make sure your agent defines (and exports or makes available) a *getAgent* method and that returns an instance of your agent.