MAGI Tutorial

MAGI provides deterministic control over the various components in an experiment. It can be used to express and automate the procedural workflow for an experiment.

An experiment procedure can be expressed as streams of events that together form a workflow. An event is a mechanism to invoke a functional behavior on a node. A functional behavior description is known as an agent. An agent implementation is a piece of code that instruments a given functional behavior.

Agent Activation Language

Agent Activation Language (AAL) is a YAML based descriptive language that MAGI uses to describe an experiment’s procedural workflow. The entire experiment procedure needs to be expressed as part of an AAL file. An AAL representation has mainly 3 parts.

1. Event Streams

   An event stream is a list of events and wait states that is parsed and executed in linear order. An experimentation procedure can be expressed in an AAL by defining multiple event streams and connecting them according to their dependencies. Different event streams can execute concurrently and can be synchronized with each other using triggers.

   All the event streams that need to begin execution at the start are to be listed as part of a list called streamstarts.

   1.1 Events

   An event is a remote procedure call made to one or more experiment nodes. It executes a method of an agent implementation code. An event representation requires

   • Destination agent
   • Name of the method to be invoked
   • Arguments to be passed to the method

   1.2 Wait State and Triggers

   Wait state and triggers are used as a synchronization mechanism among different event streams.

   Wait states are holding points in an event stream. They pause the execution of an event stream until the completion of a specified trigger.

   Triggers are of two types

   • Event based triggers are received from an experiment on completion of an event.
   • Time based triggers wait for a specified amount time to elapse.

2. Agents

   An agent represents the relationship between a functional behavior and the corresponding implementation code. It is a mapping between a behavioral name and a set that contains the following

   1. One or more groups on which the agent needs to be executed.
   2. A pointer to the agent implementation code. As a prerequisite for the code to be executed, it needs to be available in the cache on all of the nodes and referred to by this name. If the code is not already cached it can be made available by one of the following ways, all of which would store a copy in the cache.

   • An accessible path to the code available either as a directory or a tarred file.
   • A base64 encoding of tarred code.

   3. A list of arguments to be passed while loading up the agent code.

3. Groups

   Groups provide a mapping from a functional name to a set of experiment nodes. It is a mechanism used to couple the procedure to the apparatus.

MAGI Orchestrator

The MAGI orchestrator is a tool that parses an AAL file and orchestrates an experiment based on the specified procedures.

1. Join Groups

   The orchestrator iterates over the list of groups and for each group sends a request to all the mapped nodes to join the group. A corresponding reply adds the replier to the group. Messages addressed to a group are sent to all the nodes that are part of the group.

2. Load Agents

   The orchestrator iterates over the list of agents and for each agent sends an agent load request to the mapped groups. An agent load message specifies to start an instance of the agent implementation and to put it in a listening mode, where the instance waits for further messages.

3. Execute Event Streams

   Next, the orchestrator concurrently executes all the event streams listed as part of streamstarts.

The MAGI orchestrator has a predefined event stream called exit. The purpose of this event stream is to unload all the agents and disjoin groups. All workflows should end up executing this stream for a clean exit.

Setting up a MAGI enabled experiment on DETER

MAGI is not tied to any particular testbed architecture. Following are basic instructions on how to setup an experiment with MAGI on DETER/Emulab.

If you aren’t already familiar with DETER, please go through the tutorial on how to setup an experiment.

Begin a new experiment and load the ns file that defines the network topology. Simple ns file modifications are required to support loading the MAGI backend on each node:

1. Create an additional unconnected node in the experiment called ‘control’. This is a special node required for the underlying messaging layer to work. It is the orchestrator’s gateway to the magi daemons running on the experiment nodes.
2. Run the python script /share/magi/current/magi_bootstrap.py as root on each node at startup. The bootstrap process installs the MAGI distribution along with the supporting tools.

tb-set-node-startcmd $node "sudo python /share/magi/current/magi_bootstrap.py"

Next create and swap in the experiment.

Once the experiment is swapped in, you can then run the MAGI orchestrator passing it the control node’s address and the AAL file to orchestrate.

/share/magi/current/magi_orchestrator.py -c control.exp.proj –f procedure.aal

Now you can try to work with the sample test cases.