Data Fabric May Tie Cloud Computing to Army's Future

Officials at one of the U.S. Army’s premier research and development centers are exploring the possibility of adding a so-called data fabric to the service’s original tactical cloud system. The concept could improve interoperability, aid the convergence of intelligence and operations information and allow service leaders to completely rethink future Army operations.

Data fabric is an architecture of services that integrates management of myriad cloud computing systems, making it easier and quicker for users to find the information they need regardless of which cloud system it resides within. Researchers at the Army Combat Capabilities Development Command's Command and Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance (C5ISR) Center plan to create a data layer for the service’s original tactical cloud system known as Rainmaker.

In 2010, Engility, which is now a part of the Science Applications International Corporation (SAIC), worked with the Army to build and deploy the cloud-based intelligence system. It is literally a cloud computing system in a box—in this case a CONEX or container express. The Army uses the large metal containers for shipping, storage and temporary shelters. Rainmaker “packs a punch of 1,800-plus processing cores and a petabyte of data storage,” according to the Engility website.

Each Rainmaker cloud can store half of the information housed in all of America’s academic libraries, the site adds. The system provides infrastructure, platform and software-as-a-service for Army operations, and it is being leveraged not only to consolidate current Army applications into the cloud but also to provide massive data processing and analytics to meet Army intelligence needs.

The original Rainmaker was designed strictly to increase processing power and data storage. “Back then, cloud computing for us in the intelligence community wasn’t about software-as-a-service or applications-as-a-service or containerizing software and deploying it. We weren’t really seeing that as our need back then,” recalls Alan Hansen, division chief, Intelligence Systems and Processing Division, Intelligence and Information Warfare Directorate, C5ISR Center. “Our niche was about processing. We were looking for computing environments that gave us a lot of processing capabilities along with lots of storage.”

That storage and processing capability was a game changer at the time. “The ability to marry up processing and storage was a big deal because now we could take lots of data and start exploiting data at scale. That was our early entry into the cloud world,” Hansen recalls.

The current Rainmaker project, which kicked off in October, adds a data layer, making information more accessible and interoperable. “In very simple terms, Rainmaker is a tactical data fabric that we are building that will unify these clouds and provide an agile data layer between all the clouds so that we can actually share information, and we can federate information at scale.”

Hansen says the service will deploy an initial capability next year for units in Europe.

The effort ultimately could impact the Army’s Capability Set 23, which is designed to improve network transport capacity and resiliency. With Capability Set 23, Army officials also expect to make converged mission command systems smarter by leveraging artificial intelligence and machine learning to bring in meaningful tactical cloud capabilities where applicable, given bandwidth restraints.

The implications, the researcher indicates, are extensive. For example, the Army currently relies on standard messaging—the Variable Message Format and the U.S. Message Text Format—to aid interoperability. “We did that because we had disparate systems, and this was the least common denominator to allow systems to share knowledge or share capabilities between one another,” Hansen explains.

But while the standard formats aid data sharing among different kinds of systems, such as intelligence, logistics, operations planning, medical and aviation, those standards are not perfect. “You’re limited by a standard that only allows certain information to flow between systems. Messaging is standard, and if that messaging doesn’t conform, it doesn’t get passed. It gets dropped,” Hansen asserts. “We have pigeonholed ourselves about how we perform operations because we’re stuck around this messaging.”

It is past time the service considers a new approach, Hansen indicates. “We’re moving into the 21st century, and we’re still using this 1980s approach to data interoperability.”

If not tied to those data standards, Army officials can rethink future operations. “This is really slick because now we can rethink the way we do interoperability, how we share information, how we do collaborative services. We can rethink the way we do information management on the battlefield, how we do battlefield planning in the future. Now I can come up with a whole different methodology and a whole new set of data,” Hansen says. “That’s a big deal. That’s a huge paradigm shift from where we are today, which is everything is centralized.”

Creating a data fabric could aid the Army’s convergence efforts. “The Army is looking at how to converge these information battlefield systems so they’re more cohesive, so they’re not disparate systems. They’re becoming more unified. And cloud computing has a lot to do with how they’re going to get there,” Hansen offers. “Now, in the last year, we’re looking at this convergence requirement, this operations-intelligence convergence, and you have the acquisition community looking at cloud computing as a means to bring this next-generation computing capability to the battlefield.”

The new Rainmaker project also could be a boon for machine learning. Effectively training machine learning algorithms requires large amounts of varied data. Machine learning deployed with today’s limited interoperability capabilities would produce inadequate results because of a lack of robust data, Hansen suggests.

To illustrate how machine learning could be enhanced, he cites an example of a radar system that provides more data than normally is used. “If you’re a radar system producing reports and locations, that’s nice, but what happens to all that parametric data we throw away? We don’t need them if we’re doing decision making, but if you’re doing machine learning, and you want to train a good tracker for radars, you want all that parametric data to feed the machine learning capability so that it enriches and enhances the machine learning experience.”

The Rainmaker effort may ultimately allow the service to use sensors more effectively. Often when intelligence personnel provide information on potential targets, he explains, that information goes unused because it doesn’t meet the needs of so-called targeteers. “We have machine learning programs that are looking at predictability—how an adversary behaves, their battlefield tactics, the terrain they’re operating in—and we are looking at developing analytics that proactively have the sensors on target as to where we think [enemy forces] are going to be as opposed to going out and finding them. In other words, we’re proactively posturing with sensors as opposed to reacting with sensors.”

Additionally, Rainmaker might affect data services approaches toward encryption. Rather than have various units share point-to-point encryption over radios, the data itself would be encrypted as it’s being stored, which is often referred to as data at rest encryption, and then it could potentially be transmitted over unsecure channels.

The service used a mission command information system and the Distributed Common Ground System-Army (DCGS-A) last spring to assess a data fabric capability during an experiment known as Enterprise Challenge. DCGS-A connects soldiers to the intelligence community, other services, multiple joint intelligence, surveillance and reconnaissance platforms and sensors, and Army mission command systems. It gives commanders the ability to view intelligence, surveillance and reconnaissance information in one place and integrates that information into tools that can support intelligence development. “We took a mission command information system and our DCGS-A system, and we showed how we were able to unify the systems collectively using a data fabric approach. That was our first step in this arena,” Hansen reports.

That resulted in a proposal to, and approval from, the Army’s Network Cross-Functional Team to move forward with the three-year Rainmaker project. But the goal is not to develop technology. The idea, instead, is to create specifications for acquisition officials. “We’re risk mitigating the concept of what this is and providing some specifications and standards to which the acquisition community can actually go out and buy an end-to-end solution across the Army,” Hansen says.

The Army used capabilities from companies working in the banking services sector, such as Hewlett Packard’s MapR and Cloudera’s Hortonworks. Hansen says there are similarities between the networking requirements of banks and warfighters, such as the need to use handheld devices and to complete a critical transmission—or a financial transaction—even if the network goes down. “Specifically in banking you have so-called data centers and then you have hand-held banking services. In banking, you have to make sure that service is maintained, and it’s protected, and if the network goes down you have to ensure that session continues because it’s a financial transaction. It’s very analogous to the military.”