Simulation Benefits Troops and Civilians

February 2008
By Rita Boland
E-mail About the Author

A computer-simulated forest fire burns toward a stand of tall Florida pines. University of Central Florida researchers created this program to discover whether virtual reality can help residents and policy makers see near- and long-term effects
of economic and political decisions on wildfire management.
From training to therapy, advances in science and reductions in cost are spurring new trends.

The creation of virtual worlds and robots is spiraling out from the military into a broad array of applications. While the defense community continues to advance its technology to offer better instruction and to improve the safety of troops, other fields of interest—ranging from the technical to the personal—are beginning to use similar tools to meet their needs.

At the Institute for Simulation and Training (IST) at the University of Central Florida, researchers are tackling various major and minor projects in areas from military to education. As the institute expands its fields of study, it also is adjusting its own approach to simulation. The IST is incorporating experts from fields such as psychology, sociology and the arts into projects to work along with engineers. Currently, 220 people from 12 technical disciplines work for the institute.

In addition to expanding the types of experts involved in creating the simulations, the university offers master’s and doctoral degrees in modeling and simulation because officials believe the skills are a fundamental part of most technical fields. By establishing the advanced degrees, the university encourages a few personnel in each discipline to have modeling and simulation foundational skills. The students then can specialize in aircraft, human factors or other focus areas relevant to their line of work. Dr. Randall Shumaker, director of the IST, explains that officials at the institute believe the study of modeling and simulation by professionals from other disciplines will become a major new trend.

He also believes that modeling and simulation is going to become a more pervasive capability. “Modeling and simulation needs to be thought of as a professional discipline in itself that has applications in other areas,” Shumaker says. The field is becoming less purely mathematical as social scientists try to understand how groups interact along with the implications of decisions.

This new take on development is evident even in the IST’s large programs. The biggest study at the institute is the human agent collaboration project from the U.S. Army Research Laboratory. The goals of the program are to understand how to integrate software agents (also called semi-autonomous agents) into teams by themselves and how to create mixed human and robot teams. Researchers also want to determine how to task robots at a high level. Shumaker states that the program—now in its second year—is making good progress.

Agents—key factors in modeling and simulation—are entities that perform actions for humans and come in more than one form. Tools such as Google are unembodied agents. Robots are embodied agents, meaning they move around in the physical world and can integrate with people in reality. Whether physical or virtual, the human interface on the agents is critical because people must believe they can trust what the agent tells them.

In the IST’s human agent training and simulation program, which studies team performance, experts are evaluating how agents and robot systems should look and act so humans are comfortable interacting with them. The researchers are exploring whether robots should act as robots or as people. Another factor in the project is dominance hierarchy and team structure—finding a structure that people are comfortable with and feel good working in.

One type of robot that humans, and especially military troops, encounter is the autonomous vehicle. In one experiment, IST scientists built a 1/35 scale model of an Iraqi city to test scale-model vehicles. In the next five years or so, Shumaker expects humans to stop thinking of the self-mobile vehicles as vessels that need human operation via a joystick or steering wheel and to begin turning over more autonomy to the vehicle. He says the robots will “interact at the level of a sheepdog as opposed to something you drive remotely.” People will tell the vehicle what it needs to accomplish, and they will have confidence that the robot will carry out the orders, Shumaker claims. The Army already is thinking about using the vehicles for scouting and equipment transport.

Other organizations have done some of the work already. The Defense Advanced Research Projects Agency’s recent Urban Challenge demonstrated the viability of these autonomous vehicles. The robots worked out the details of how to avoid people, navigate traffic and respond to stoplights and stop signs. While the early signs of success are there, Shumaker says questions remain about how the robots gain acceptance. For example, if the vehicles operate in an area where people are unfamiliar with them, how do the vehicles interact with the humans? Much work at the IST is aimed at determining how to make these interactions successful, both for the people managing the robots and the people the vehicle may encounter.

Two soldiers learn to work as a team while immersed in virtual reality space. Real-world teammates can be miles—or continents—apart, while their virtual reality avatars remain working together side-by-side.
In a military conflict location, the robots can afford to be slightly less user friendly, but, Shumaker states, “You still don’t want to run over people’s feet or run into the general’s car.” Gaining acceptance is the critical factor, according to the director. What should the unmanned vehicle look like; how should it act; and what would make people feel comfortable around it?

Shumaker illustrates the importance of comfortable human-robot interaction with the analogy of an intersection. If a manned vehicle, an unmanned vehicle and an oxcart pull up to a three-way stop, how do they determine who proceeds first? The manned vehicle driver easily could give a hand signal to the oxcart driver, but signals to and from the unmanned vehicle are more challenging. Researchers are working to determine how the unmanned vehicle would acknowledge the presence of the other vehicles. If the robot should wave someone on, does it need arms or another device? The development team has to search not only for the technical answers to the questions but cultural answers as well.

Personnel at the IST also are looking into applied cognition and training in immersive virtual environments, determining how to build usable and effective training systems. The research involves systems that measure physiological reactions to determine if a human is in distress and then consider whether an automated system could take over for a person, at least in the interim, if the human encountered too much stress.

An additional project at the IST, which researchers have been working on since last year, studies how to build high-performing teams when the participants have never met. The original design is focused on military operations with coalition partners, but it has applications to homeland security and other areas in which people have to form teams rapidly with members who have no experience working together personally.

Another more recent project called real-time high-performance computing is an Army initiative focused on how to create continuous multiplayer training simulations that have thousands of agents and humans interacting together. The program is addressing how to create large, real-time, real-world training, and in the early stages it is exploring whether high-performance computers can be used in real-time applications instead of in batch mode. The idea resembles Second Life integrated with real-world locations. Live actors can interact with agents or people playing roles from remote sites. The project could expand training opportunities by virtually creating necessary personnel for exercises. In a scenario requiring 100 people, but for which only five humans are available, 95 software agents could fill the remaining roles.

Other projects underway at the IST include spin-offs from military research to programs for the general public. “Now modeling and simulation applies to almost every endeavor you can think of,” Shumaker shares. In one project for the National Science Foundation, a million years of change in the Everglades is represented in a complex simulation and visualization. Users can explore different time periods and ecologies so they can learn what has happened in the Everglades and why it occurred. The simulation is designed for museum use.

Another civilian project creates a virtual reality simulation of forest fire management. Users make decisions about control of burns, harvesting and other factors and look at the profitability of the forest and managing it for long-term sustainability. Uncontrollable forces such as drought and lightning affect the results of the combination decision-making tool and game. Decision makers could use the virtual reality system to see the consequences of their choices played out over a long period of time. After many different runs of the simulation, users could work out statistics to determine the best courses of action.

Other programs at the institute involve helping the FederalLawEnforcementTrainingCenter prepare drivers and personnel who use heavy equipment, working with gaming technology and performing pilot studies for traumatic brain injuries. The IST has carried out pilot studies in speech and physical therapies. Shumaker explains that the programs are natural spin-offs of military training for more civil applications and are coming together now because of maturing technologies.

Civilian applications for military-intended simulation are only part of the change the simulation and training field is experiencing. Another major adjustment is the availability of less expensive computers and visualization devices in the simulation field. Other changes at the IST include its commitment to multidisciplinary research teams. “We use people from arts, theater, digital media to help us build very effective stories,” Shumaker says. An additional alteration he anticipates is integrating other subject matter experts into the development process and finding the most effective simulation answers whether or not they involve high-end computers and displays.

One such program selects teachers likely to be effective in urban schools. The IST built a prototype tool in which teachers stand in front of a screen and address a class with five student avatars. Each avatar represents a difficult-to-manage classroom personality type in an eighth grade English class. Teachers’ interactions with the simulations give observers clues as to how effective the teachers will be in an urban school. Shumaker explains that people buy into the avatars and are comfortable with them despite their cartoonish look because they have realistic facial expressions, body language and dialogue. In one experiment, a teacher asked to repeat the exercise because she felt she was making progress with one of the students. “That’s a really good sign that these things can be effective,” Shumaker states.

The skills learned in the simulation also could apply to other professionals such as counselors and sales people. The necessary technologies are reasonably available, and Shumaker believes that in a year or two people could use the programs on their home computers.

Shumaker shares that one of the biggest changes to the simulation and training field in recent years is an understanding of soft skills. In the past, computers judged how an aircraft would behave or how to launch a rocket. Now, simulations are more sophisticated and consider how clouds will act in certain circumstances or how people will respond to situations.

Simulations now not only deal with vehicles, but also with probabilistic interactions. They can help users understand how crowds will behave or how a change in the tax code might affect the economy. While simulations long have been excellent at building vehicles and training systems, they are advancing into how to model human behavior. Shumaker anticipates that human modeling will be an important trend in the field. The IST purchased a suite of instruments related to a cognitive model of the brain. The suite will help determine when someone is overloaded or bored, and respond appropriately.

Shumaker says that in the future, simulation will spin off more to medical applications, allowing patients to do rehabilitation therapy in their homes through immersive simulations. Another area he expects to accelerate is virtual games such as Second Life. These environments allow people with real-world disabilities to interact as though they were able-bodied. “I think that trend is really interesting and is going to continue,” he states. A world was created in Second Life for people with autism who have trouble with normal social cues. Shumaker shares that some autistic people who learned to use the game ventured out into the rest of the world to talk to other people.

The low cost of technology makes many of the current and future simulation trends possible. People can now buy what was considered a supercomputer less than two decades ago. For less than $400, they can purchase a terabyte of storage. These technologies are economical and high-performance, allowing gamers to wander around virtual worlds, such as Second Life, or actually navigate virtually around a real location, such as Rome.

Web Resources
Institute for Simulation and Training:
Defense Advanced Research Projects Agency Urban Challenge:
Army Research Laboratory:
Second Life: