DETERLab provides various capabilities that allow cyber researchers to:

• Perform experimental research at scale and complexity representative of real world attacks
• Extract understanding through experimental research
• Collect, leverage, and share experimental artifacts and results
• Use methodologies and technologies for multi-party, ultra-scale and federated experiments

These capabilities allow the cybersecurity researcher to carry out experiments that are more useful, more reliable, and more complex, scalable, and realistic than anything possible in ordinary testbed environments.

DETERLab capabilities include:

• DETER Core
• Multi-resolution virtualization of experiment resources
• Predictive modeling of human behavior supporting definition of mental models, reactive goal-driven behavior, and combinations of deliberative/instinctive behaviors.
• Federation to connect heterogeneous resources from different owners with varying usage and access policies.
• Multi-party experiments technology that provides controlled but co-joined experiments, creating different views of the experiment environment for multiple experimenters or groups of experimenters within one unified experiment.
• Experiment orchestration, providing deterministic control over the various components in an experiment.
• Risky Experiment Management to allow controlled experiment interaction with the real Internet

DETER Core

The core hardware and software of the DETERLab testbed is based on Emulab with extensions for cybersecurity.

The experimenter’s interface is a web-based GUI that provides:

• remote access to DETERLab
• account management,
• experiment management,
• tools portal: MAGI, fedd, third party tools, etc
• My DETERLab - this dashboard includes access to your experiments, other projects currently running, information about available resources and general testbed news
• A command line interface where experimenters may SSH into a secure UNIX environment.

The software is open source and available to the cybersecurity research community in a stable GIT repository. New developments from the community may be ported back to the code.

DETER staff constantly adapts the testbed, improving scale and size of resources available -- experiments may scale up to 10’s of thousands of nodes.

DETER also provides support with fast turnaround times.

Multi-resolution Virtualization

It may seem like working on a live system is best - the real world - however, such a system is difficult to configure, unpredictable (per the nature of the Internet), difficult to repeat, and could even be dangerous, exposing its users accidentally or disrupting important networks. The key for real progress is to get as close to 'real world' scale as possible with as much scientific integrity as possible without jeopardizing people and systems.

This requires the modeling of large, complex systems at an appropriate level of fidelity. That level may be different for different parts of the modeled system. DETERLab allocates the computation power only where it is needed.

The DETERLab Containers system is an innovative approach to virtualizing experimental resources within larger experiments. It was designed and developed by DETERLab researchers to facilitate the modeling and emulation of highly scaled, complex, multi-resolution networking problems within the DETER testbed.

The term 'multi-resolution' refers to experiment resources of varying scale and fidelity. In other words, within an experiment there are certain resources that require whole computers available as high fidelity, high cost elements. However, other resources, do not require such high resolution and are just as effective when used as an abstraction, such as virtual machines and process elements. Containers allow researchers to use multiple levels of realism to scale an experiment exponentially without requiring a prohibitive amount of resources.

This subsystem has been used for a large scale experiment of 100,000 botnets. In this experiment, ~30 physical machines hosted 500 Containers configured as lightweight virtual machines in addition to a few high-fidelity physical OS nodes representing DDoS target servers. In turn, we modeled subnets of infected hosts, approximately 100 per each lightweight virtual machine, which amplified the scale to create an experiment with 50,000 infected or potentially infected end-hosts forming a bonnet and participating in creating the dynamics of the ensuing DDoS attack.

Predictive Modeling of Human Behavior

The Deter Agents Simulating Humans (DASH) toolkit is our agent platform for simulating human behavior as it relates to cybersecurity. In many situations, human activity affects the behavior of a networked system in a way that should be modeled for accurate assessment of its security properties. DASH provides both a library of agents and support for building new agents that model human behavior.

Agents built on the DASH platform can take advantage of a dual-process model, in which the agent may choose the next action either by 'instinctive' pattern-matching or by rational deliberation about goals and ways to achieve them. In both cases, a DASH agent's goals and next action are re-assessed on each decision cycle, allowing an appropriate response to unexpected changes in the environment. Since a major cause of non-optimal human behavior is an incorrect or incomplete model of the impact of security choices, DASH includes support for mental models that drive its rational behavior through the projection of possible actions and their effects. Using these components, agents built on DASH can model individuals that make suboptimal decisions in good faith based on inadequate information, or whose behavior changes under cognitive load, stress or fatigue.

DASH is in use by several different groups to model defenders, attackers and end users. It is used both in multi-agent simulations to explore the consequences of security policy or software, and to provide a realistic yet repeatable scenario for networked human subject experiments.

Federation

DETER has developed a mature federation technology. The DETER Federation system builds experiment environments from resources owned by different people where resources are still under control of their owners. Federation allows a project to connect heterogeneous resources from different owners with varying usage and access policies. This confluence of resources allows more efficient and faster collaboration that becomes greater than the sum of its parts.

Experimenters can access more resources, increasing the scale of their experimentation and/or include unique hardware or configuration properties that allow experimenters to embark on new kinds of experiments.

In addition to physical connections, Federation is highly valuable for creating collaborations and cross discipline experiments.  USC ISI has been using Federation as part of an ongoing consortium with the DOE Pacific National Labs, the University of Illinois and SRI to develop experimental capabilities for cyber physical systems in the power grid.

This capability includes the following features:

• On-demand creation of experimental scenarios spanning multiple, independently controlled facilities
• Increased scale, access to unique resources, accommodations of usage policy constraints, data and knowledge sharing and information hiding
• Ability to implement embedding, allocation, control and data management
• Decentralized policies that make it easier for researchers to gain access and for testbeds to make resources available securely.
• A collection of Attribute-Based Access Control (ABAC) tools that allow individual access control decisions for each experiment. These tools map principals to attributes and use the attribute to make an authorization decision. For example, if user1 has the login attribute, the related program will allow them to log in.

Federation also enables partner cluster deployments. See DETER Partners for more details.

Multi-party Experiments

Most interesting cybersecurity experiments involve more than one logical or physical party, due to the adversarial nature of the problem as well as the distributed, decentralized nature of the networked systems environment. DETERLab provides multi-party model experiment scenarios where the project may configure the flow of information to and level of visibility of participants. This capability is built on the DETER Federation architecture, only instead of creating federated testbeds, the system creates federated experiments.
Historically, experimentation has been focused on a single experimenter/researcher using the DETERLab facility to construct a scenario which often contains multiple parties acting as attacker/defender. However, in these cases the researcher constructing the experiment is fully cognitive of both sides of the scenario and all components of the experiment are equally visible. This approach affects scenario realism and may promote experimental bias (the designer knows the expected outcome e.g. DDOS packets will be blocked and thus may construct an experiment which defacto blocks DDOS packets).
In a multi-party experiment, the experimental apparatus is built from sub-components that are meshed to create the whole worldview. Each component has complete information only about their own sub-component, with only partial information about other sub-components. This form of information hiding is necessary in order to create realistic scenarios and eliminate bias.
This form of experiment may be used to model several different kinds of cyber-defense situations: adversarial situations (e.g., red-team/blue-team exercises); realistic forensic or defense scenarios (e.g., attack target with limited information about attacker); or partial collaboration situations in which separate organizations collaborate on defense without granting full visibility to collaborators.

Experiment Orchestration

The Montage AGent Infrastructure (MAGI) subsystem provides a scalable workflow management system for deterministic control over the various components in an experiment, such as networked elements and agents. It can be used to express and completely automate the sets of processes and procedures for the experiment, freeing up time and energy usually spent reproducing experiments from scratch every time they are run.

The experimentation workflow captures the sequence of concurrent steps or procedures that the experimenter wants to follow in order to orchestrate the experiment. It enables the experimenter to conduct an experiment through a deterministic sequence of steps expressed explicitly in AAL description. The descriptions provide a precise way to characterize and reproduce the experiment procedures and may be manipulated in several ways:

• Identical workflows can be run several times for statistical evaluation of results,
• Workflows may be parameterized to systematically explore the experimentation space and
• Alternative workflows may be easily created through variations and derivations of the original workflow.

In addition, MAGI provides constraint-based visualizations of an experiment-in-progress to help ensure experiment validity. This capability allows an experimenter to define constraints or parameters such as link utilization or CPU utilization. Frequently, if an experiment operates out of some expected range of behavior, the experimental results may not be valid. MAGI along with prior work in the Semantic Analyis Framework are tools to assist an experimenter to constrain her experiment to within valid ranges.

MAGI's infrastructure provides architecture for scalable control and instrumentation to handle the disparate elements in experiment scenarios that must be:

• easy to configure,
• deployed, initialized, configured,
• monitored and coordinated, and
• instrumented with real-time and post-mortem data collection.

Risky Experiment Management

Cybersecurity experiments by their fundamental nature may involve significant risk if not properly contained and controlled. At the same time, these experiments may well require some degree of interaction with the larger world to be useful.
DETERLab enables researchers to carry out experiments that interact with their larger environment while retaining control and safety. The DETER Risky Experiment Management Capability allows a properly vetted experimenter to set up gateway nodes within an experiment. These gateway nodes enable specific communication paths in and out of the testbed for specific traffic type and source/destination addresses.