Enable breadcrumbs token at /includes/pageheader.html.twig

Increased Autonomy Is Here

Transition success largely depends on trust.

Dr. Paul D. Nielsen, director and CEO, Software Engineering Institute, Carnegie Mellon University, speaks at the AFCEA/George Mason University Critical Issues in C4I Symposium.

Autonomous functionality is increasing. The evidence is everywhere from drones and self-driving cars to voice-controlled devices such as IBM's Watson and Amazon’s Echo. The key to successfully transitioning to these increasingly autonomous systems for the military and defense industry is trust, said Dr. Paul D. Nielsen, director and CEO, Software Engineering Institute, Carnegie Mellon University.

The morning keynote speaker at the AFCEA/George Mason University Critical Issues in C4I Symposium, Nielsen stressed, “We need to be able to trust that these systems will be safe and serve us.” The importance of trust doesn’t end with humans trusting machines. Eventually these systems need to trust themselves, they need to trust other systems and they need to trust the humans that created them.

Autonomous systems in use today are the result of decades of R&D. “Sensors cost pennies now and machines have more than ever,” Nielsen said. “Algorithms are also more adaptive. User interface is becoming more and more important. We are going to be talking to these systems not simply using a keyboard.”

For defense and the military, more autonomy can increase mission duration and safety. “But we need to keep the human in the loop. Humans will still be needed to maintain the endurance and persistence of these systems,” he said. “The challenge for the defense world is that the system may need to learn continuously. It will never be the same as when it went through the operational test.”

As we become more connected, machine to human, and machine to machine, we also open up the attack surface of these systems. “As we’ve learned from the Internet, a vulnerability somewhere is a vulnerability everywhere,” Nielsen said. “When we build this software we need to maintain it for life.”

The easiest way to build trust is through familiarity. “We don’t think twice now when we get on an airplane even though that airplane is flown mostly by an autopilot,”Nielsen related. “We naturally gain more autonomy as we age. We will have to do this with machines as well.”