Have you ever struggled to map the experiment you envisioned to the testbed? Or perhaps to automate an experiment and scale it up? Our new set of tools should help.

We have just released a new DEW portal for Distributed Experiment Workflows at https://dew.isi.edu/. At the portal, you can use human-friendly text to design your experiment top-down, going from the vision in your head to a human-friendly representation of that vision. The portal will generate an NS file (topology) and a bash script for you. You can then use these to deploy your experiment on Deterlab. Conversely, you can perform bottom-up experiment design too. You can use our tools, integrated in the Ubuntu-DEW image, to capture user manual actions over time, on testbed nodes. Later you can use our tools to select those actions that worked, to be included in a runnable script. Very soon, we plan to introduce bash-to-DEW translators, which will help users load their scripts onto our portal and produce DEW representation of an experiment, thus closing the loop.

Top-down design

DEW helps users think about what their experiment should do, instead of how the topology looks like. DEW views an experiment as a collection of three components: scenario, constraints and bindings. In the scenario, users talk about actors (nodes in the experiment) and actions (human-friendly labels for commands to be executed on nodes). For example, users may say “server setup” to denote that some setup action will be performed on the server node. The real power of scenarios, however, shines when users introduce dependencies between actions. For example, the user may want to start the server only after the setup is complete. To enact this, the user could specify the following scenario:

server setup emit setup_done
when setup_done server start

The user can choose their own label for the event that denotes completion of the setup action, such as “setup_done”. Thus the scenario becomes a human-friendly summary of the actions in the experiment.

In the constraints section, users can specify topological constraints, such as which actors should be connected together, or which OS should be loaded on nodes.

In the bindings section, users specify the actual commands to be executed on nodes for each of the user-friendly labels from the scenario. For example, users may specify that “setup” actually means “sudo apt-get install apache2 -y” and that “start” means “cp apache.custom.conf /etc/apache2/apache2.conf && sudo service apache2 restart”. We provide built-in functions for some common experimental needs, for action and event bindings. One of those functions is “psuccess(A)” which denotes that we are waiting for action A to complete successfully. We could use this to specify binding for “setup_done” as “psuccess(setup)”.

Figure below shows the graphical view of event dependencies from DEW portal for our sample scenario.

Bottom-up design

Currently, users design their experiments bottom-up. We all start with a topology and then execute some commands manually on the nodes to find out what works. Did you ever wish you knew what you did 5 weeks ago when your experiment worked fine, and why it does not work today? We now have a tool for that!

If you swap in your experiment using our custom image Ubuntu-DEW, everything you type on your experiment nodes and the first 500 characters of your terminal output, along with the time, the current directory and the current user, will be logged in your project directory (in /proj/YourProject/logs/output_csv). But that’s not all! You can also have the testbed automatically save all changes you make to files on your nodes in a git repository. This can be very useful if e.g., you recall that some application you have spent days to configure worked fine last Tuesday, but you have since changed its configuration and you cannot recall what exactly you have changed.

Finally, once you are ready to take some time travel, you can use our flight_log tool to retrieve your experiment history and select the commands you want to include in your script. The tool will then produce a runnable script. A sample output of the flight_log tool is shown below:

This article is just a brief overview. For more details please visit our documentation at the DEW portal or subscribe to our DEW channel on YouTube. Please contacts us at [email protected] with any questions.