Army Catches RIDE to Extended Reality Future
Researchers at the Institute for Creative Technologies built the Rapid Integration and Development Environment, a test bed for evaluating modeling and simulation technologies for internal use. But the system, which plays a role in creating the Army’s Tactical Computing Environment, is finding users across the Defense Department and in the defense industry.
The system, known as RIDE, has proved useful in developing a wide array of capabilities for technology-generated realities, including virtual, augmented and extended realities. RIDE combines features inherent to commercial game engines with many of the immersive technologies developed at the institute, which is commonly known as the ICT. The ICT is a university-affiliated research center associated with the University of Southern California and is directly supported with Defense Department funding.
An ICT website describes RIDE as a “state-of-the-art, modern simulation research and development framework that has proved invaluable in advancing the research of ICT, collaborators and stakeholders across multiple lines of effort.” RIDE combines and integrates the Army’s One World Terrain data and tools; machine learning; speech recognition; natural language processing; behaviors for artificial intelligence (AI) characters; and scenario event development. It is integrated with commercial game engines, allowing the reuse of visual art, 3D models and other simulation technologies in the common platform, which reduces the efforts required to create divergent simulation prototypes. While it was built using the Unity game engine, it also is designed to be agnostic, meaning it is usable with virtually any gaming engine.
The ICT team began building RIDE because they could not find the tools they needed, says Volkan Ustun, the ICT associate director of cognitive architectures. He runs a lot of experiments with AI and reinforcement learning that need fast simulation environments able to create data as quickly as possible for training AI models. The models created in the simulation environment can be shared across AI experiments, and the system can provide a common repository to reuse or repurpose technologies.
“When I was looking at things in the military domain two years ago, there was nothing that was capable of supporting the types of things that I need. RIDE provides me a realistic gaming engine-based simulation environment, which is capable of generating the data in a format that I can use for training these models,” Ustun says.
The system has come a long way in two years. “The impetus of RIDE was to help our internal research projects have more of a cohesive understanding and structure and then that very quickly turned into the RIDE of today, which is this platform we’re providing across the Department of Defense to jumpstart existing ideas and research and provide a foundation where multiple collaborators can work on integrated projects that might have disparate technologies,” explains Kyle McCullough, ICT director, modeling and simulation. “It’s definitely gotten bigger. It pretty quickly grew into a much greater use case than just what we were doing with it at ICT.”
The U.S. Army’s Simulation, Training and Technology Center (STTC) in Orlando is one of the major sponsors for the system. “They are using the RIDE platform as well—as have a number of our collaborators at STTC, other contractors working on science and technology research—all leveraging RIDE to begin to integrate our research, the same way we’ve done it within ICT, where we’re showing off everything we’ve done in this one cohesive environment,” McCullough reports.
The Office of Naval Research also has found RIDE useful. “A lot of collaborators and performers who work under the Human Performance Training and Education program under Dr. Peter Squire have built programs and software applications using the Unity game engine,” McCullough says.
RIDE also can work with existing or so-called “legacy” systems, such as the ICT’s Aerial Terrain Line of Sight Analysis System (ATLAS), which is hosted on MilGaming, the Army’s web portal for gaming systems used for training. The version of ATLAS available on MilGaming is roughly 18 months old, the researchers estimate. But now, if someone creates a new version with 3D data, RIDE can act as the core foundation to support that data. “It works across all of these other use cases—the prototyping, the modeling and simulation environments that we’ve built out, but it also can be pulled directly into the ATLAS software, which is now being built off of RIDE,” McCullough says.
He adds that the Office of Naval Research uses RIDE to connect disparate technologies. “One of the big pushes from that side has been to unify a lot of the software foundations, because those were all built in these stovepiped efforts across these different companies and collaborators. Some of the work that we’re doing, for instance with Lockheed Martin, is to tie all of that together into the RIDE platform, so you have this one foundational code base.”
ICT researchers tout that ability to connect disparate research from various disciplines as one of the primary benefits—and challenges—of RIDE. Arno Hartholt, ICT’s director of research and development integration, cites as an example the Army’s future Synthetic Training Environment, which will bring together live, virtual and constructive training environments to train dismounted and mobile troops, aerial platform crews and command post personnel wherever training is needed.
“How do you get researchers to work with developers, to work with industry, to work with Army subject matter experts and the Army as a whole? And what specific capabilities do you need to put together, and how do you make them work together that addresses a real need? RIDE is addressing those challenges and providing a blueprint on how it’s possible to charge a way forward,” Hartholt says.
The challenge, he indicates, is continuing to keep RIDE applicable to a wide variety of users while also growing its capabilities. “What we do moving forward is really listening to what the gaps are, to what the individual groups have, and trying to come up with common features while maintaining the extensibility and the principal design of RIDE. That is the main challenge: to make sure that any researcher and developer is able to leverage RIDE for their purposes while we grow it in terms of maturity and new capabilities.”
Future improvements will include using AI to enhance the behaviors of simulated characters, including enemy forces. Pattern-of-life behaviors also will likely be enhanced. For example, military simulations need to accurately depict behaviors of civilian populations.
Ustun says those behaviors include more realistic depictions of character behaviors or abilities in particular types of terrain. “On top of that, you have other constraints coming from military doctrine and guidelines and things like that where certain squads or infantry can do certain things in the environment, and you have to incorporate these things into these AI models. So, when you are training and creating behaviors, they have to follow certain guidelines while still trying to optimize their objectives within their environment for the task that they are trying to accomplish.”
A service-based architecture for RIDE also is likely. “One of the keys is RIDE right now supports a lot of these integrations that we’re doing for all of these various capabilities. What we’re learning as we build that out is where the requirements are to enhance, maybe, creating part of that capability as a service-based architecture that can be accessed,” McCullough states.
A great example of that, he says, is work the ICT team is performing for 3D data geo-specific terrain data modeling. “For instance, some of the work that we’re doing looks at being able to leverage that service-based architecture to support something like the Next-Generation Combat Vehicle, or any of the work that the Ground Vehicle Systems Center is doing. Especially when you think of using these types of things at the point of need, you’re not going to want every device running RIDE and having this full-bore simulation system,” McCullough adds.
The service-based architecture would allow users, for example, to note their current location in the simulation and where they want to get to. “RIDE is doing line-of-sight analysis on enemy locations. RIDE is using material ground classification, knows the specs of the vehicle that you’re in and is providing the best path at that moment in time,” McCullough elaborates.
Other potential improvements include extending RIDE to support more external machine learning frameworks; adding an ability for researchers and developers to more easily set up new scenarios, new contexts, and new ways to define scenarios and environments they need; enhanced supporting of data collection, creation and analysis; and the ability to run RIDE on different types of hardware to offer native support for augmented and virtual reality, which would advance the Army’s goal of providing technology-generated reality training whenever and wherever it is needed.
While RIDE’s future path could include becoming a commercial product, the goal is to provide the Army with a research and development platform, Hartholt suggests. “We’re a noncommercial entity, so we see it as a test bed. We will use RIDE to help the Army define its requirements and acquisition targets, and as such, we really help drive the agenda and identify what the opportunities and gaps are. I don’t see us turning RIDE into something that we start selling to the government. It’s meant to be an open platform in support of—as a start—Army research and development and then going beyond that.”
