DeterLab: Resources, Capabilities, and Benefits

DeterLab users are cyber-security scientists, working in groups or projects to investigate, test, or experiment with cyber-security technology. The work of a project team typically focuses on a single experiment: an evolving set of resources used to create the object of study -- for example, a botnet -- and activities performed by the users to observe or modify its behavior. In creating and working with an experiment, users remotely access DeterLab via a Web-based portal that includes an integrated development environment, or workbench, that enables them to manage all aspects of experiment development and operation. The workbench provides a framework that encompasses several different types of experimentation, such observation of malware and defenses, assessment of defense effectiveness, or testing hypotheses about a defense mechanism.

DeterLab Experiment Lifecyle

More specifically, the term experiment is often used to describe the experimental apparatus or environment that users have constructed from network nodes and links, physical and virtual hosts, operating systems and other software, test fixtures, and measurement tools. In addition to the apparatus, an experiment also includes procedures and activity for interacting with the apparatus as it runs, and methods for reviewing the data resulting from its operation. In short, an experiment includes the entire experimental protocol of procedures for a particular research activity — and the experimental apparatus used to perform those procedures.

The Experiment Lifecycle

DeterLab users typically employ an iterative workflow, or experiment lifecycle, consisting of:

  • Initial design and construction of an experimental apparatus
  • Exploratory operation of the apparatus during iterative construction
  • Initial full test runs when operating the apparatus
  • Review and analysis of test run results, and
  • Modification of the apparatus and/or procedures based on test results.

By continually re-working the apparatus and/or protocol, researchers can develop an apparatus that consistently operates as expected, producing the log data, network traces and other output data that is the source for discovering experimental results.

DeterLab workbench: visual tracking of botnet experiment in progress

In this workflow, experimenters rely on the workbench to help them design and refine the various elements of their experiment, using a wide variety of technical resources including: local and remote networking, network extension via federation, a variety of high-capacity servers, flexible re-configuration of networking between servers and others nodes, a variety of host images for the nodes, and more -- a broad and rich set of network and computing resources to use as building blocks for experiments.
 
Of course, the experiment lifecycle also includes operation of an experimental apparatus, running the experiment using defined experimental procedures. DeterLab has several capabilities for running experiments -- starting, pausing, checkpointing, re-starting, stopping, storing, and re-loading -- as well for interacting with experiments, both via instrumentation and direct observation, as well as manual intervention to provide human activity, or trigger prepared events or inputs.

Workbench for Experiment Lifecycle Management

Throughout the experiment lifecycle, the role of the DeterLab workbench is to provide assistance and automation of experiment construction and operation tasks. The goal is to give the experimenter the detailed control of only the tasks where detail is important. For example, some parts of experiment design can re-use existing, detailed experiment components, rather than requiring construction of an experimental apparatus from a blank slate. Similarly, a great deal of experiment operational detail can automated, with the experimenter making detailed decisions about network links and node characteristics only for experimenter-selected parts of the experiment. Selectivity is important, because of the complexity of the large set of core computing and networking resources.

To help manage the complexity, the DeterLab workbench consists of a powerful and expanding set of tools and facilities that enable researchers to build, run, and manage experiments that are built on the underlying resource base. Part of the workbench is a repository of archived experiment fixtures, such as network definitions, real and virtual hosts, instrumentation software, software configurations, and other archived objects that can be used as a fixture or component in an experiment apparatus. The workbench also includes tools for creating new experiment fixtures, copying and modifying archived fixtures, and for combining them to define the infrastructure for a specific experiment.

DeterLab Workbench: Combining Elements of Two Prior Experiments

The shift from experiment design to execution is an important example of workbench use. After an experiment has been designed -- or revised based on running the experiment -- then before it can be run, it has to assembled from the real resource of DeterLab. Researchers can specify the design of an apparatus using design tools, and then rely on DeterLab’s capability to “realize” the experiment. Experiment realization is an automated process of creating the experiment's components by assembling a set the network and computing resources that meet the requirements of the experiment design. Experimenters gain several benefits from this aspect of the workbench:

DeterLab Workbench: Altering Node Attributes of Hosts in an Experiment's Network Definition

Broad: an experimental apparatus can draw on a wide range of computing systems and network infrastructure, including physical and virtual servers, client, routers, switches, network security devices, sensors, configurable by experimenters from high-level network design down to hardware functions in field-programmable gate arrays.
Extensible: experimenters can use a mix of local networking and computing, wide-area communication between DeterLab’s physical locations, and third-party network and computing resources – all integrated through DeterLab’s federated computing capabilities.
Flexible: experimenters may specify resource-use details or draw on existing “building blocks” and DeterLab tools that automate several details of experiment realization. 
Scalable: DeterLab provides over 400 compute nodes, with up to 10 network interfaces on each node, that can each support multiple apparatus elements by using virtualization techniques that support the experimenters’ goals. 
A compute node may be divided into 10's of conventional virtual machines or 100′s or 1000′s of simpler network forwarders or simulated network elements.  Experiments may be scaled out further by connect to external computing resources, via DeterLab federation techniques, and linking those resources to DeterLab’s hardware resource base.

Researcher and Community Benefits

A key benefit of the workbench is its ability to help experimenters harness that breadth and scalability, without having to become involved in every detail, but providing access to detail when needed by the experimenter.  The workbench  helps experimenters to structure their apparatus and activities in three major ways:

  • Scalable Design-Driven Experimentation using tools for construction of an experiment from models of real-world computing, and experimental goals, helping the experimenter realize these goals in a specific set of network and computing resources, for a realistic scale of systems and networks.
  • Re-usable Experiment Development using an archiving and browsing facility that enables users to draw on their own and other DeterLab users’ previous work. Researchers can save and re-use experimental fixtures such as network segments, complete virtual hosts, traffic generators and experiment instrumentation tools.  Many such fixtures can be reconfigured to tailor their use in a new experiment
  • Experiment Operation, Interaction and Recording using powerful and flexible tools for operating an experiment, controlling and interacting with a live running experiment, recording experimental data created by a running experiment, and analyzing the experimental data.

Taken together, these and other benefits are what make DeterLab a scientific instrument for researchers to perform the repeatable experimentation that is required for the science of cyber-security. Whether the scientific activity is experimentation, investigation, or testing, DeterLab provides the capabilities for running an experiment, re-running with same procedures and data, re-running with altered procedures or data, modifying the experiment based on observation and analysis, and iterating over the run-observe-analyze-modify-rerun cycle. Critically, an experiment can be repeated in various ways by the experiment’s creator, and also saved and later re-created as a copy for other scientists to use to repeat the experiment, validate experimental results, or build on the experiment for further research or investigation.

The emphasis on sharing, re-use and repeatability has become increasingly important, as DeterLab’s facilities and capabilities have continued to expand as a result of the DETER project team's efforts in its research program and in technology transfer into DeterLab. The DETER research program now has an central focus on the methods and technology to support the activity of cyber-security researchers in a scientific lab setting, with collaboration, re-use, and sharing between experimenters.

This focus on collaboration and sharing is part of the DETER project emphasis on the research community. DeterLab users have conducted hundreds of research projects and published more than 100 resulting scientific papers.  More than 1700 students, the crucial next cyber-security generation, have received hands-on training via DeterLab.

> More:  DeterLab Overview   |  Research Community  |  Get Started Using DeterLab