Army Seeks Solutions for Evaluating New Technologies

While the U.S. Army sprints toward deploying the Next Generation Command and Control, or Next Gen C2 system, across the force beginning in fiscal year 2027, it faces a unique challenge: evaluating technologies it has never fielded before, such as data mesh and artificial intelligence.

Maj. Christopher Nelson, Capability Program Executive Command and Control Information Network, Army Test and Evaluation Command (ATEC), noted the unusual situation while serving on a C2 modernization panel at AFCEA International’s TechNet Augusta conference, Augusta, Georgia.

The panel, entitled Sprinting to Success: Continuous Evaluation for C2 and C2 Modernization, also included panel moderator Col. Don Sedivy, director, Transformation Integration, U.S. Army Cyber Center of Excellence; Lt. Col. Ruben Rangel, deputy chief of staff, 4th Infantry Division, which is one of two divisions undergoing experimentation with modernized C2 systems; and Lt. Col. Richard Ricketts, product manager for mounted command and control.

The panel discussed the Army’s overhaul of the system development and acquisition process, which relies on experimentation, soldier feedback and iterative development to deliver modernized capabilities to soldiers much more rapidly than traditional processes allowed. Sedivy described both the old and new approaches in his opening remarks.

“We've operated in sequential silos: develop, then test, then field. By the time a capability reaches the hands of the warfighter, the threat or the technology has moved on. We can no longer afford the luxury of that timeline,” he said. “We are bringing together three critical pillars of the modernization—the operational units, the capability developers and the independent evaluators—putting them together in the same room, like we've done today, in the same dirt at the same time. By iterating in real time, we're finding the critical function or friction points early. We're failing fast, fixing faster and dramatically reducing the fielding risk.”

In response to an audience member's question, Nelson acknowledged that Army leaders find themselves in uncharted territory with some new technologies. “On the evaluation side, I'll call a spade a spade. Evaluating data layers and AI is tough. How many data layers, true data layers, has the Army ever had? None. That's why they're asking for one,” he said. So, how many do you think ATEC has ever evaluated? None. And so, it is a challenge,” he said. “Evaluating AI is tough.”

He then invited feedback from others, explaining that asking a large language model how it arrived at an answer will simply validate the biases from users and from the model itself, which he declared unsustainable. “If industry has the answer to that, I am all ears because with emerging technology, evaluation strategies need to emerge, and I would love your feedback on how we can do that better. We are in uncharted territory with how we evaluate a lot of these things, and we have a lot of learning to do.”

The Army’s new acquisition process includes everyone concerned—such as the four panelists—collaborating together rather than having requirements personnel, commanders, program managers and testers working in isolation.

The major recalled having worked as a program manager before moving to ATEC and having a less-than-favorable view of test and evaluation personnel. “They were the people on the other side of the fence where we, as the [program manager], would throw a capability over to them, and they would be there with their red pens, their clipboards and their stopwatches, ready to call balls and strikes, write an exhaustive report on all of the issues and concerns with your system, and then send you on your way,” he offered. “And if that system failed, it would be years before you would ever get the capability or get any of that capability in the hands of the soldiers, and that's a problem.”

That old way of doing things no longer exists, he declared. “Let me be clear: that ATEC, that legacy way of doing things, is dead. And it's not sustainable in this kind of an environment. And so we've adopted what we call a continuous evaluation campaign, and the key word there is continuous.”

Although ATEC collaborates across organizations to more rapidly field technologies to soldiers, it still works independently, Nelson reassured the audience. “We've had 10 unique opportunities—over 10 in fact—unique opportunities to get in the dirt with the trail boss, with 4th ID, with 25th ID, who's not here on stage with me, and with our partners in the requirements community to put our hands on the system, evaluate the system and provide feedback. And let's be clear, we're calling balls and strikes the whole time.”

He noted that ATEC reports directly to the vice chief of staff of the Army and that its reports are not “all sunshine and roses” because if everything was perfect, the service would have stopped iterating, high-fived and called it a day.

He stressed, however, that ATEC will continue to act as an independent partner working closely with others. “I'm not going to sit up here and suggest that this is a way of doing things. It's the only way of doing things, and if ATEC wants to be able to be in the dirt with these fine gentlemen next to me over here, we're going to have to figure out how to evaluate at the speed of relevance.”

Nelson also explained that rapid technology development can still include rigorous evaluation and testing. “I think there's a common misconception out there that speed is somehow a competing or opposing force of rigor, and that we can't be rigorous and be fast at the same time. And I'll tell you right now, that is absolutely not the case. I would even offer that if you test early, integrate often and share your data across the enterprise, speed can actually be an enabler of rigor.”

TechNet Augusta is organized by AFCEA International with help from the U.S. Army Cyber Center of Excellence. SIGNAL Media is the official media of AFCEA International.