Joint Experimentation Enables Regional Cyber Protection

Commanders wrestling with control of cyberspace elements now have a new tool to help them secure their corner of cyberspace in an operational setting. The Adaptive Network Defense of Command and Control concept of operations enables joint force commander control of key terrain in cyberspace, based on assessments at an operational tempo. To achieve a joint force command objective, network operators concentrate cybersecurity and monitoring of command and control systems to maintain the initiative against adversarial attacks and provide enhanced situational awareness.

This approach was developed by the Joint Cyber Operations Joint Test (JCO JT), under the auspices of the director, Operational Test and Evaluation. It developed and evaluated a concept of operations and tactics, techniques and procedures (TTP) to secure command and control (C²) systems with commercially available technologies.

The JCO JT tested the effectiveness of the Adaptive Network Defense of Command and Control (AND-C²) TTP for the Virtual Secure Enclave (VSE) TTP. The VSE TTP provides methods to establish and employ a community of interest virtual private network, with anomaly detection, for protection and defense of joint task force (JTF) C² systems. The VSE TTP implements the AND-C² concept of operations by using a virtual secure enclave for C² protection.

The JCO JT employed a challenging test methodology for the VSE TTP. This methodology proved successful because of careful collaboration and deliberate planning. The JCO JT aligned testing with U.S. Pacific Command (PACOM) experimentation and a sister Joint Capability Technology Demonstration (JCTD) during a major theater exercise. Test planners closely coordinated with multiple red teams and created test conditions for quantitative analysis with statistical rigor.

The PACOM resources and assessment directorate pursued a three-step process of experimentation to evaluate the VSE, which was derived from a National Security Agency network security concept. First, PACOM performed several limited objective experiments using a design of experiments technique to collect information on protection effectiveness and a sensing methodology. Those experiments provided information on protection and sensing performance and compared the performance to traditional information assurance compliance. Additional factors for experimentation investigated areas of operational concern such as secure data sharing to and from external users, and resilience to various attack characteristics.

Second, PACOM performed a demonstration of a VSE-protected C² system as evidence that the operational system would maintain functionality before implementation on operational networks. Third, PACOM perused an operational demonstration through a joint test to fully validate the concept and integrate the capability into joint operations.

The JCO JT employed a test methodology in partnership with the Computer Adaptive Network Defense-in-Depth (CANDID) JCTD. Running a parallel joint test with a related JCTD effort allowed this JCTD to focus on the VSE materiel solution, while the joint test concentrated on the all-important TTP development. By leveraging a common configuration during a major exercise, the partnership allowed for a rapid spiral of VSE design, testing and TTP modification. The successful parallel efforts suggest that crucial materiel and related nonmateriel solutions can be delivered to operating forces under short timelines.

The VSE TTP assessment was framed around PACOM exercises Terminal Fury 2011 and 2012. Exercise planning played a critical role in refining exercise storylines to support JTF objectives for cyber defense of C² systems. The planning provided the integration of VSE TTP functions into the JTF staff and the requirement to establish concurrent test venues. As a result, the JTF operations and communications directors committed exercise resources to implement the VSE TTP into their decision-making processes and protected an exercise C² system with the VSE.

Shortly before exercise Terminal Fury 2011, real-world events—the earthquake and tsunami in Japan—interrupted the original exercise schedule. However, the planned concurrent test venue on the Joint Information Operations Range allowed for scheduled completion of the first planned test. For exercise Terminal Fury 2012, a significant focus was placed on a robust and live test over the secret Internet protocol router network. Exercise preparation with PACOM and its components contained VSE TTP training, and reservists received just-in-time training to execute the centralized management and monitoring of the VSE. A complete TTP evaluation was made possible by taking full advantage of blue and red team availability.

Typically, ethical hacking approaches involve free play to exercise a red/blue scenario, or extensive penetration-type testing to analyze equipment vulnerabilities. In contrast, the JCO JT employed an operational testing construct where prescribed test parameters outlined cyber red team activity. During exercise planning, the JCO JT detailed its support requirements with cyber red teams. The red teams developed a cyber playbook of 161 attacks, from which the white cell called attacks during the test. This construct is analogous to controlling flight test parameters with a flight plan and test cards. Realistic cyber attacks provided measurable and analyzable blue team responses to attack samples of reconnaissance, exfiltration/infiltration and denial of service.

These elicited blue team responses were the source of the VSE TTP evaluation. Extensive collaboration with cyber red teams served as a foundation for designing and executing the test.

The JT&E program required a TTP assessment based on analytical rigor to support informed decision making. Typically, tests rely heavily on qualitative surveys to verify performance. With a strong influence from PACOM J-8 and the Defense Department Office of the Director, Operational Test and Evaluation, the JCO JT performed statistical analysis at a high confidence level—90 percent—on quantitative measures such as protection and detection rates of cyber attacks and the speed of network anomaly detection.

The JCO JT white cell controlled sample counts to meet analytical goals using near-real-time attack information from custom network instrumentation. Additionally, the Joint Information Operations Range enabled testing with many more attacks than possible on operational networks. To assess VSE TTP improvement, test results were compared to warfighter-defined performance requirements and information assurance performance from fiscal year 2011 exercises. The analysis provided objective and decisive evidence that the VSE TTP effectively improves cyber defense of C² systems for joint operations.

The VSE TTP testing succeeded largely because of its test and rehearsal strategy. The empowered joint test took advantage of exercise Terminal Fury, the Joint Information Operations Range, national and service cyber red teams, and insights from historical cyber assessments. As demonstrated by the JCO JT and CANDID JCTD, operational testing drove a timely solution for joint operations within two years.

To institutionalize these tested concepts, PACOM sponsored the AND-C² concept of operations for Joint Staff validation as a joint concept of operations. The AND-C² concept of operations describes a functional methodology for providing the joint force commander with a tailorable capability to selectively protect and defend C² systems, using commercially available technologies. To synchronize network defense effectively for assured C² with joint operations, a capability such as the VSE must be implemented within joint operational planning and execution.


 

Maj. Jose Gonzalez, USAF, is the deputy test director, Joint Cyber Operations Joint Test.