Digital Outcomes for Mission Success

Measuring investment performance in digital technologies to improve the Navy’s mission effectiveness has been gauged in many ways. 

Budget invested is the most cited, and it is among the inputs. But money spent will not inform decision makers if there was a digital capitalization of the service and how that may be driving value for sailors and marines. 

“It's not always quite as clear to every person working in an enterprise how we're judging outcomes within government. It is the number of features or the speed of code, those aren't outcomes, but we've talked about those, so we have boiled this down to mission outcomes,” said Justin Fanelli, technical director, Program Executive Office Digital, U.S. Navy. 

Service members need not only to perceive improvements, but they have to be measurable and comparable. 

"Not only are we looking at [return on investment] and the impact on a metric or cost-save or progression of our architecture, or improve performance to evaluate where we invest, but we're following up and closing the loop with those metrics to say, ‘Hey, we thought we were going to impact the user by improving their end-to-end trip response times in this environment by 20%. Well, look, we we did it by 30%, or we only did it by 10%’” said Mike Galbraith, chief digital innovation officer at the Department of the Navy.

Different organizations use various methods to measure how they are achieving their goals. Each criterion has its downsides and advantages. 

KPI, or key performance indicator, is almost a buzz word. It was originally thought to inform decision makers in terms of specific business objectives and is used to measure progress toward achieving those objectives, according to Gartner, a consultancy. But the Navy’s mission does not entirely align with goals like profitability. 

OKRs, objectives and key results, is a goal-setting framework that helps organizations set specific objectives and cascading strategies., according to Gartner. “OKRs are often written as statements that look like this, ‘The IT organization will achieve <objective> as measured by <key results>,’” said Troy Hiltbrand, vice president of information technology and strategy at Amare Global. 

This second category provides some guidance for the Navy’s service goals, but still relies on metrics that could lie closer to for-profit businesses. 

The Navy sought another metric that Gartner identified, the ODM, or outcome-driven metrics. This management framework focuses on outcomes or results, rather than inputs or activities. 

"The ODM framework is particularly interesting for an enterprise that is continuing to grow and get better,” Fanelli said. “Of course we have KPIs and OKRs and other things, but we had a number of different metrics efforts and it looked like it made sense to integrate and align those, and in doing so, we also thought it made sense to compare ourselves to other organizations inside and outside of the government who are trying to deliver the same capability.” 

To illustrate how this performance framework would operate, Galbraith offered an example: “There are instances where it takes 10 or 12 minutes for a customer to boot up their laptop.” Such a delay for a Department of the Navy user would be due to software and hardware issues, some related to the need to protect information and others linked to hardware that could be easily streamlined, according to Galbraith. 

All improvement demands a holistic view of how this hypothetical laptop complies with necessary steps and which processes can be streamlined or eliminated. Eventually, the outcome for the service member waiting for the machine to boot will be reduced waiting times if the ODM was applied correctly. 

About this specific hypothetical case, Fanelli commented, “customers are more satisfied and productive,” when they can use their tools as expected. 

Still, not all parts of a military service can use these indicators successfully. 

“We are [Program Executive Office] Digital, it’s more like industry and has a set of benchmarks; that's a little bit more like industry than someone who's building a tanker or a battleship,” Fanelli told SIGNAL Media in an interview.