Configure the MAGI Control Overlay

The MAGI bootstrap process establishes a networking overlay over which it communicates with the experiment nodes.

The control overlay provides a control plane across heterogeneous resources, such as containers, federation, and specialized hardware, on the DETERLab testbed.

The control overlay provides a network to reliably propagate discrete control events across the experiment. Hence it is critical to establish a robust and scalable overlay to ensure the experiments are orchestrated correctly.

MAGI provides multi-point to multi-point group communication. A “group” has a set of nodes as members. Each group has a logical name. Any member of the group can send a message to any other member of the group.

By default, one MAGI overlay is established for the experiment, and all the control messages are passed over it.

However, it may be necessary to establish two or more overlays based on the experiment topology embedding, control and data management requirements. The required overlay structure can be explicitly specified during the experiment bootstrap process.

MesDL

The overlay configuration, Messaging Description Language (mesdl.conf), is a yaml-based configuration of the required overlays and bridges in the experiment. The mesdl.conf defines all the overlays and bridges for the experiment.

The MAGI “overlay” is built on top of the control network on the DETER testbed. Nodes in the overlay can be thought of as being connected by virtual point to point links in magi. The overlay provides a way to communicate with all the magi-enabled nodes across testbed boundaries and even over the internet. “bridges” act as meeting places for all the members of the overlay it serves. ” For example:

# Establish two bridges on node3.
# Typically one is inward facing towards the experiment and one
# is outward facing towards the experimenter to use with magi tools
# magi tools can now connect to the experiment overlay through node3, port 18808
bridges:
   - { TCPServer: node3, port: 18808 }
   - { TCPServer: node3, port: 28808 }
# Establish two overlays for the experiment.
overlay:
   # Members of this overlay rendezvous at node3. All experiment nodes are part of
   # this overlay.
  - { type: TCPTransport, members: [ '__ALL__' ], server: node3, port: 28808 }
   # Members of this overlay rendezvous at node2.
   # node4 and node7 are members of this overlay.
  - { type: TCPTransport, members: [ 'node4', 'node7' ], server: node2, port: 48808 }

In the absence of the mesdl.conf magi creates a single overlay with all the experiment nodes as members. MAGI establishes the Control Node as a bridge.

For example, here is the mesdl.conf for the Client-Server tutorial

bridges:
    - {TCPServer: clientnode, port: 18808}
    - {TCPServer: clientnode, port: 28808}
overlay:
    - members: [__ALL__]
    port: 28808
    server: clientnode
    type: TCPTransport

Note

MAGI expects the Control Node to be MAGI-enabled. However, if that is not the case, the user must provide a custom mesdl.conf

Bootstrap Process

The magi_bootstrap.py is used to configure the overlay and start MAGI on the experiment nodes.

The magi_bootstrap.py tool it typically called when the node starts as follows:

tb-set-node-startcmd $NodeName "sudo python /share/magi/current/magi_bootstrap.py"

The magi bootstrap script can be used to install, configure, and start MAGI. The various command line options are as follows.

Usage: magi_bootstrap.py [options]

Options:
  -h, --help            show this help message and exit
  -n, --nokeys          Option to ignore creation and waiting for SSL
                        certificates
  -p RPATH, --path=RPATH
                        Location of the distribution
  -q, --quiet           Silence debugging information; default behavior
  -U, --noupdate        Do not update the system before installing Magi
  -N, --noinstall       Do not install magi and the supporting libraries
  -v, --verbose         Include debugging information
  -f, --force           Recreate magi.conf even if present. Cannot use along
                        with -c (see below)
  -m MESDL, --mesdl=MESDL
                        Path to the messaging overlay configuration file
  -c MAGICONF, --magiconf=MAGICONF
                        Path to the local node magi configuration file. Cannot
                        use along with -f (see above)
  -d DBCONF, --dbconf=DBCONF
                        Path to the data management configuration file
  -D, --nodataman       Do not install and setup data manager
  -o LOGFILE, --logfile=LOGFILE
                        Log file. Default: /tmp/magi_bootstrap.log