Controlling Cybernetic Crowds

Human behavior modeling adds a new twist to simulations.

Future virtual training environments may provide soldiers with computer-generated opponents who realistically portray anger, fear and fatigue. Researchers are adding human behavioral and cultural data to software to accurately depict crowd and adversary reactions. By introducing these layers of authenticity, scientists hope to enrich the quality of the learning experience that simulation systems offer.

Computer-based military training has been in use several years, but advances in software and processing technology have allowed these systems to move from depicting only vehicles to showing individual soldiers and small units. The computer gaming industry has created a number of realistic combat environments now used as military training aids. However, while these systems can model the dynamics of a firefight, they still cannot accurately capture the complex interactions encountered in crowd control or humanitarian operations.

The University of Pennsylvania, Philadelphia, working with the U.S. Defense Department, has developed software that realistically models human behavior under a variety of circumstances by simulating the effects of stress and emotion on an individual’s judgment and decision making. This dynamic not only applies to simulated crowds and opponents, but also to friendly forces controlled by the training session participants. The goal is to create a plug-in program that will enhance the realism of a virtual simulation, explains Barry G. Silverman, a professor of electrical and systems engineering with the university’s School of Engineering and Applied Science.

Supported by a three-year $1.4 million grant from the Defense Modeling and Simulation Office, Alexandria, Virginia, the research was driven by the need to depict human behavior in military environments correctly, Silverman explains. In the past, software designers emphasized visual realism, such as accurate physical movement and the proper shape and flow of cloth and liquids. Because graphics, physics and kinesthetics have dominated game design, character behavior is usually manually pre-programmed to operate along a predictable script. This is a common practice in both commercial games and military simulations, he says.

The only independent action available to these semi-automated forces is a limited form of artificial intelligence that permits them to navigate without hitting obstacles such as trees and furniture. “Up to this point, there has not been a lot of research on why these characters do not behave more like humans. Why don’t they get scared or emotional? Why don’t they make mistakes or get tired?” he asks.

Designers also do not usually share their behavioral models with each other, often reinventing the same reactions without any guidance from other developers. “They make up what they think a tired person might look like or add some kind of stress and fear that intuitively looks good to them. Typically, this is done by a computer programmer who says, ‘This looks like a scared person,’” he explains.

The extensive work already invested in developing accurate motion, physics and navigation systems has aided the university’s work by freeing researchers’ time to concentrate on behavior, Silverman observes. He notes that an enormous amount of material on human behavior exists in the form of hundreds of physiological studies about factors such as how adrenaline, nutrition and malnutrition affect people. Stress studies provide additional data about the effect that time pressure, event stress and fatigue have on an individual’s judgment.

Data from the very best of these studies was compiled to make more accurate human behavior models called performance modifiers. Silverman explains that the crux of his work is bringing this information to life in a virtual character. “What modifies performance? If a football player has been playing for three and a half hours, what happens to his adrenaline in the third hour? We look for real models, not just things that are fun to shove into a video game,” he says.

Because of the varied nature of the data, multidisciplinary teams of specialists were brought together to help compile the performance modifiers. Drawn from across the University of Pennsylvania’s academic departments, team members include Asian studies specialists, political scientists, anthropologists and psychologists because, Silverman notes, many games and training aids do not accurately depict other cultures.

About 500 performance modifiers have been placed in an online archive. Silverman describes this as a treasure trove of reliable models based on the most up-to-date literature. In 2001, researchers converted about 100 of the performance modifiers into code for use in small-scale demonstrations of the technology.

Besides realistically portraying human behavior under stressful situations, the software also creates an accurate picture of a range of cultural reactions—from people in a village in Kosovo to the streets of Mogadishu. “The way we model emotions is heavily culture based—it’s essentially modeling a culture,” he says.

All of these behavioral and cultural aspects were united to model the first 12 hours of the 1993 battle in Mogadishu, Somalia, between the U.S. Army Rangers and Special Forces and the local militia. Emotions are an essential part of creating an accurate depiction of such an event, Silverman explains.

An important factor in events such as the battle in Mogadishu is the role of individuals in group dynamics. The virtual scenario involves clan members, militia, men, women and children—each with different motivations. Although they may be in the same country, he maintains that a blanket model cannot be applied. Each individual must be modeled separately to reflect his or her different roles and reactions to the situation.

Research determined that in any crowd behavior there are provocateurs and opportunists. Provocateurs come to these situations with a very specific mission, usually seeking to cause an incident. But studies indicate that opportunists are potentially more dangerous. These are young, unemployed and predominantly male individuals who are seeking some benefit from a chaotic situation. These people often cause the most injuries and property damage, Silverman observes.

A herd mentality is another factor in incidents involving crowds. Because people tend to bond with a group, being involved in a crowd elevates a person’s sense of fulfillment in the group’s mission. While this emotion is occurring, concerns for personal safety and standards of behavior shift. Inhibitions may suddenly drop and desires for certain events or things may shift as well. “You may have always had a preference to enrich yourself. But suddenly the crowd is running around and the police are occupied, so why not [start looting],” Silverman says.

Citing the Mogadishu scenario, Silverman notes that in the real engagement, the Somali crowd was ferocious and not afraid of getting hurt or being killed. Much of this behavior relates to how that nation’s culture was affected by its recent violent past.

Because the software models human behavior in these dynamic environments, when plugged into a virtual training system, it can help confirm military or law enforcement doctrine. In a military operation, following doctrine is a critical factor. A crowd will react differently to a column of tanks or soldiers in a fortified position than it does to a small unit trying to cross an open intersection. “The training value is in having the crowds reflect the way people would actually behave if you attempted to follow different tactics and doctrines,” he says.

Now in its third year of development, the software is almost ready for testing in large-scale military training environments where it will simulate crowds and other group dynamics. Silverman expects this to occur by early 2003.

The program is not memory intensive and can operate on any late-model personal computer. More than 100 individuals can be depicted in a scenario if the graphics are processed on a separate machine. This current stage of development involves identifying and solving compatibility issues such as software and interface standards, he says.

A number of small in-house demonstrations have been conducted. These scenarios were run with the laboratory’s homemade graphics and represent small unit actions such as ambushes at bridges, crowd scenes, terrorist missions and bombings. Silverman hopes to have the Mogadishu scenario and a famine relief mission ready for demonstration by mid-2003.

 Additional information on the University of Pennsylvania’s research on modeling is available on the World Wide Web at www.seas.upenn.edu.