DETERLab provides various capabilities that allow cyber researchers to:
- Perform experimental research at scale and complexity representative of real world attacks
- Extract understanding through experimental research
- Collect, leverage, and share experimental artifacts and results
- Use methodologies and technologies for multi-party, ultra-scale and federated experiments
These capabilities allow the cybersecurity researcher to carry out experiments that are more useful, more reliable, and more complex, scalable, and realistic than anything possible in ordinary testbed environments.
DETERLab capabilities include:
- DETER Core
- Multi-resolution virtualization of experiment resources
- Predictive modeling of human behavior supporting definition of mental models, reactive goal-driven behavior, and combinations of deliberative/instinctive behaviors.
- Federation to connect heterogeneous resources from different owners with varying usage and access policies.
- Multi-party experiments technology that provides controlled but co-joined experiments, creating different views of the experiment environment for multiple experimenters or groups of experimenters within one unified experiment.
- Experiment orchestration, providing deterministic control over the various components in an experiment.
- Risky Experiment Management to allow controlled experiment interaction with the real Internet
The core hardware and software of the DETERLab testbed is based on Emulab with extensions for cybersecurity.
The experimenter’s interface is a web-based GUI that provides:
- remote access to DETERLab
- account management,
- experiment management,
- tools portal: MAGI, fedd, third party tools, etc
- My DETERLab - this dashboard includes access to your experiments, other projects currently running, information about available resources and general testbed news
- A command line interface where experimenters may SSH into a secure UNIX environment.
The software is open source and available to the cybersecurity research community in a stable GIT repository. New developments from the community may be ported back to the code.
DETER staff constantly adapts the testbed, improving scale and size of resources available -- experiments may scale up to 10’s of thousands of nodes.
DETER also provides support with fast turnaround times.
This requires the modeling of large, complex systems at an appropriate level of fidelity. That level may be different for different parts of the modeled system. DETERLab allocates the computation power only where it is needed.
The DETERLab Containers system is an innovative approach to virtualizing experimental resources within larger experiments. It was designed and developed by DETERLab researchers to facilitate the modeling and emulation of highly scaled, complex, multi-resolution networking problems within the DETER testbed.
The term 'multi-resolution' refers to experiment resources of varying scale and fidelity. In other words, within an experiment there are certain resources that require whole computers available as high fidelity, high cost elements. However, other resources, do not require such high resolution and are just as effective when used as an abstraction, such as virtual machines and process elements. Containers allow researchers to use multiple levels of realism to scale an experiment exponentially without requiring a prohibitive amount of resources.
This subsystem has been used for a large scale experiment of 100,000 botnets. In this experiment, ~30 physical machines hosted 500 Containers configured as lightweight virtual machines in addition to a few high-fidelity physical OS nodes representing DDoS target servers. In turn, we modeled subnets of infected hosts, approximately 100 per each lightweight virtual machine, which amplified the scale to create an experiment with 50,000 infected or potentially infected end-hosts forming a bonnet and participating in creating the dynamics of the ensuing DDoS attack.
Agents built on the DASH platform can take advantage of a dual-process model, in which the agent may choose the next action either by 'instinctive' pattern-matching or by rational deliberation about goals and ways to achieve them. In both cases, a DASH agent's goals and next action are re-assessed on each decision cycle, allowing an appropriate response to unexpected changes in the environment. Since a major cause of non-optimal human behavior is an incorrect or incomplete model of the impact of security choices, DASH includes support for mental models that drive its rational behavior through the projection of possible actions and their effects. Using these components, agents built on DASH can model individuals that make suboptimal decisions in good faith based on inadequate information, or whose behavior changes under cognitive load, stress or fatigue.
DASH is in use by several different groups to model defenders, attackers and end users. It is used both in multi-agent simulations to explore the consequences of security policy or software, and to provide a realistic yet repeatable scenario for networked human subject experiments.
Experimenters can access more resources, increasing the scale of their experimentation and/or include unique hardware or configuration properties that allow experimenters to embark on new kinds of experiments.
In addition to physical connections, Federation is highly valuable for creating collaborations and cross discipline experiments. USC ISI has been using Federation as part of an ongoing consortium with the DOE Pacific National Labs, the University of Illinois and SRI to develop experimental capabilities for cyber physical systems in the power grid.
This capability includes the following features:
- On-demand creation of experimental scenarios spanning multiple, independently controlled facilities
- Increased scale, access to unique resources, accommodations of usage policy constraints, data and knowledge sharing and information hiding
- Ability to implement embedding, allocation, control and data management
- Decentralized policies that make it easier for researchers to gain access and for testbeds to make resources available securely.
- A collection of Attribute-Based Access Control (ABAC) tools that allow individual access control decisions for each experiment. These tools map principals to attributes and use the attribute to make an authorization decision. For example, if user1 has the login attribute, the related program will allow them to log in.
Historically, experimentation has been focused on a single experimenter/researcher using the DETERLab facility to construct a scenario which often contains multiple parties acting as attacker/defender. However, in these cases the researcher constructing the experiment is fully cognitive of both sides of the scenario and all components of the experiment are equally visible. This approach affects scenario realism and may promote experimental bias (the designer knows the expected outcome e.g. DDOS packets will be blocked and thus may construct an experiment which defacto blocks DDOS packets).
In a multi-party experiment, the experimental apparatus is built from sub-components that are meshed to create the whole worldview. Each component has complete information only about their own sub-component, with only partial information about other sub-components. This form of information hiding is necessary in order to create realistic scenarios and eliminate bias.
This form of experiment may be used to model several different kinds of cyber-defense situations: adversarial situations (e.g., red-team/blue-team exercises); realistic forensic or defense scenarios (e.g., attack target with limited information about attacker); or partial collaboration situations in which separate organizations collaborate on defense without granting full visibility to collaborators.
The Montage AGent Infrastructure (MAGI) subsystem provides a scalable workflow management system for deterministic control over the various components in an experiment, such as networked elements and agents. It can be used to express and completely automate the sets of processes and procedures for the experiment, freeing up time and energy usually spent reproducing experiments from scratch every time they are run.
The experimentation workflow captures the sequence of concurrent steps or procedures that the experimenter wants to follow in order to orchestrate the experiment. It enables the experimenter to conduct an experiment through a deterministic sequence of steps expressed explicitly in AAL description. The descriptions provide a precise way to characterize and reproduce the experiment procedures and may be manipulated in several ways:
- Identical workflows can be run several times for statistical evaluation of results,
- Workflows may be parameterized to systematically explore the experimentation space and
- Alternative workflows may be easily created through variations and derivations of the original workflow.
In addition, MAGI provides constraint-based visualizations of an experiment-in-progress to help ensure experiment validity. This capability allows an experimenter to define constraints or parameters such as link utilization or CPU utilization. Frequently, if an experiment operates out of some expected range of behavior, the experimental results may not be valid. MAGI along with prior work in the Semantic Analyis Framework are tools to assist an experimenter to constrain her experiment to within valid ranges.
MAGI's infrastructure provides architecture for scalable control and instrumentation to handle the disparate elements in experiment scenarios that must be:
- easy to configure,
- deployed, initialized, configured,
- monitored and coordinated, and
- instrumented with real-time and post-mortem data collection.
DETERLab enables researchers to carry out experiments that interact with their larger environment while retaining control and safety. The DETER Risky Experiment Management Capability allows a properly vetted experimenter to set up gateway nodes within an experiment. These gateway nodes enable specific communication paths in and out of the testbed for specific traffic type and source/destination addresses.