Magi Agent Configuration ======================== In :doc:`specialized-user`, you saw how to create a basic agent. The sample agent created a single file on a test node. This document will explain how to use configuration in the AAL file to configure an agent at run time. Setting Agent Configuration +++++++++++++++++++++++++++ This document will expand the sample code of FileCreator. For reference, here is the agent code: .. code-block:: python from magi.util.agent import DispatchAgent, agentmethod from magi.util.processAgent import initializeProcessAgent # The FileCreator agent implementation, derived from DispatchAgent. class FileCreator(DispatchAgent): def __init__(self): DispatchAgent.__init__(self) self.filename = '/tmp/newfile' # A single method which creates the file named by self.filename. # (The @agentmethod() decorator is not required, but is encouraged. # it does nothing of substance now, but may in the future.) @agentmethod() def createFile(self, msg): '''Create a file on the host.''' # open and immediately close the file to create it. open(self.filename, 'w').close() # the getAgent() method must be defined somewhere for all agents. # The Magi daemon invokes this mehod to get a reference to an # agent. It uses this reference to run and interact with an agent # instance. def getAgent(): agent = FileCreator() return agent # In case the agent is run as a separate process, we need to # create an instance of the agent, initialize the required # parameters based on the received arguments, and then call the # run method defined in DispatchAgent. if __name__ == "__main__": agent = FileCreator() initializeProcessAgent(agent, sys.argv) agent.run() Note that if we reset the *self.filename* variable in the agent before invoking *createFile*, we can change the file that is created. The base class of *FileCreator*, *DispatchAgent*, itself is derived from a class that will let us do this. The *Agent* class implements two methods: *setConfiguration* and *confirmConfiguration*. The *setConfiguration* method will set the passed parameters as class instance variables. The *confirmConfiguration* method is meant to be re-implemented in your agent if you need confirm the variables set are valid for your agent. To set the *self.filename* variable in the FileCreator Agents, we modify the AAL to include a call to the *Agent* method *setConfiguration*, passing in a list of key-value pairs. (In this case it is a single key-value pair.) .. code-block:: yaml - type: event agent: myFileCreators method: setConfiguration args: filename: /tmp/myCreatedFile Note that you do not specify *self* when referencing an agent variable. We make sure to place this event in the AAL event stream prior to the *createFile* event. The complete AAL file is: .. code-block:: yaml streamstarts: [main] groups: myFileCreatorGroup: [NODES] agents: myFileCreators: group: myFileCreatorGroup # (note: the "PATH" argument is the agent directory. The # direcory must contain an IDL and agent implementation. It must # also contain a *__init__.py* file, which is required # for it to be considered as a valid python package.) path: PATH/FileCreator execargs: [] eventstreams: main: - type: event agent: myFileCreators method: setConfiguration args: filename: /tmp/myCreatedFile - type: event agent: myFileCreators method: createFile args: {} Now when we run the agent again (possibly using *agentTool* to restart the Magi daemons), we see the following events: .. code-block:: bash $ magi_orchestrator.py --project $project --experiment $experiment --events ./myEvents.aal -o run.log -v stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__ stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete. stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup stream initialization : done : trigger AgentLoadDone myFileCreators complete. stream initialization : DONE : complete. stream main : sent : setConfiguration(['/tmp/myCreatedFile ... ) --> myFileCreatorGroup stream main : sent : createFile(None) --> myFileCreatorGroup stream main : DONE : complete. $ ssh myNode.myExperiment.myGroup ls -l /tmp/myCreatedFile -rw-r--r-- 1 root root 0 Mar 5 13:55 /tmp/myCreatedFile $ And we see that our specified file, */tmp/myCreatedFile* was created. Confirming Valid Configuration ++++++++++++++++++++++++++++++ This works well, but the input to the agent is free-form. What if the user gives invalid input, like the wrong type or data that is not in a valid range? This is where the *Agent* *confirmConfiguration* method comes into play. *confirmConfiguration* should be written for any agent that wants to validate it's state. *confirmConfiguration* gets invoked after the user (via an AAL file) invokes *setConfiguration*. .. note:: The concept of an agent confirming user input will change in future releases of Magi. The Orchestrator (or other Magi/Montage components) will use the interface specification in the agent's IDL file to ensure the input to the agent is valid. Suppose our sample agent wanted to allow the user to create a file in only the /local directory on the host machine. The *confirmConfiguration* method that does this is: .. code-block:: python def confirmConfiguration(self): '''Make sure the user input is a string value and starts with "/local".''' if not isinstance(self.filename, (str, unicode)): return False if not self.filename.startswith('/local'): return False return True When we add this method to our sample agent, and run the experiment with the existing AAL file, which contains configuration that does *not* start with "/local", the Orchestrator gives us an error while executing the event stream: .. code-block:: bash $ magi_orchestrator.py --project $project --experiment $experiment --events ./myEvents.aal -o run.log -v stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__ stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete. stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup stream initialization : done : trigger AgentLoadDone myFileCreators complete. stream initialization : DONE : complete. stream main : sent : setConfiguration(['/tmp/myCreatedFile ... ) --> myFileCreatorGroup stream unknown : exit : method setConfiguration returned False on agent unknown in group unknown and on node(s): moat. $ Note that the Orchestrator exited with an error. If we now modify the AAL file to include a valid configuration, the Orchestrator succeeds. The updated AAL clause is: .. code-block:: yaml - type: event agent: myFileCreators method: setConfiguration args: filename: /local/myGreatFile When we run the Orchestrator with the modified AAL, it succeeds as the agent configuration is now valid: .. code-block:: bash $ magi_orchestrator.py --project $project --experiment $experiment --events ./myEvents.aal -o run.log -v stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__ stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete. stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup stream initialization : done : trigger AgentLoadDone myFileCreators complete. stream initialization : DONE : complete. stream main : sent : setConfiguration(['/local/myGreatFile ... ) --> myFileCreatorGroup stream main : sent : createFile(None) --> myFileCreatorGroup stream main : DONE : complete. $ And the "valid" file has been created on the machine: .. code-block:: bash $ ssh myNode.myExperiment.myGroup ls -l /local total 4 drwxrwxr-x 2 glawler Deter 4096 Mar 5 08:35 logs -rw-r--r-- 1 root root 0 Mar 5 14:33 myGreatFile $ Triggers and Event Stream Sequence Points +++++++++++++++++++++++++++++++++++++++++ If you run the AAL and Agent code above, you may see that it does not actually work. One small needed detail has been left out of the AAL file. Normally the Orchestrator will run through the events in the AAL as fast is it can. If we used the event streams in the AAL file as it now stands: .. code-block:: yaml eventstreams: main: - type: event agent: myFileCreators method: setConfiguration args: filename: /local/myGreatFile - type: event agent: myFileCreators method: createFile args: {} The Orchestrator will send two messages to the Agents in rapid succession: the *setConfiguration* and *createFile* event messages. If the *setConfiguration* call returns False, which it will given invalid input, the Orchestrator will not receive the message as it would have sent the messages and exited. So we need a way to tell the Orchestrator to wait for a response from *setConfiguration* before continuing. This is done by inserting a small pause, using a *trigger* which times out after 3 seconds: .. code-block:: yaml # Wait 3 seconds for a response to setConfiguration # timeout value is in milliseconds. - type: trigger triggers: [{timeout: 3000}] If we insert this trigger between *setConfiguration* and *createFile*, the orchestrator will receive the error message from the agent and exit on error. The full AAL file now is: .. code-block:: yaml streamstarts: [main] groups: myFileCreatorGroup: [witch, moat] agents: myFileCreators: group: myFileCreatorGroup path: PATH/FileCreator execargs: [] eventstreams: main: - type: event agent: myFileCreators method: setConfiguration args: filename: /local/myGreatFile - type: trigger triggers: [{timeout: 3000}] - type: event agent: myFileCreators method: createFile args: {} But how do we know that waiting for 3 seconds is a long enough time to wait? Wouldn't it be better if we could tell the orchestrator to wait for a response from the agent before continuing? We can do this using a *named trigger*. We add a *trigger* statement to the *setConfiguration* event clause and modify the trigger to wait for that event before continuing to process the event stream: .. code-block:: yaml - type: event agent: myFileCreators trigger: configDone method: setConfiguration args: filename: /local/myGreatFile # Wait for the event "configDone" from all fileCreator agents. - type: trigger triggers: [{event: configDone, agent: myFileCreators}] Now when *setConfiguration* is called on the agent, the daemon will send a trigger with the event *configDone* after the method has returned. With this modified trigger, the orchestrator will wait for the trigger event *configDone* before processing the next event in the event stream. Here is the orchestrator output now. Note that setConfiguration now "fires" a trigger (sends a trigger) and the orchestrator waits until the trigger is resolved before moving on. .. code-block:: bash $ magi_orchestrator.py --project $project --experiment $experiment --events ./myEvents.aal -o run.log -v stream initialization : sent : joinGroup myFileCreatorGroup --> __ALL__ stream initialization : done : trigger GroupBuildDone myFileCreatorGroup complete. stream initialization : sent : loadAgent myFileCreators --> myFileCreatorGroup stream initialization : done : trigger AgentLoadDone myFileCreators complete. stream initialization : DONE : complete. stream main : sent : setConfiguration(['/local/myGreatFile ... ) --> myFileCreatorGroup (fires trigger: configDone) stream main : done : trigger configDone complete. stream main : sent : createFile(None) --> myFileCreatorGroup stream main : DONE : complete. $ For reference, the new agent implementation, AAL file, and IDL, file can be downloaded as a tar file here: :download:`FileCreator-withconfig.tbz <_static/FileCreator-withconfig.tbz>`.