The Risk and Opportunity of Open-Source Intelligence

Open-source data, which is information freely accessible to the public, could be leveraged for better domain awareness and decision-making, stated subject matter experts during a panel session at TechNet International 2024.

Taking the stage were Samantha Rix, C2 Innovation Cell technical director at U.S. European Command; Pragyansmita Nayak, chief data scientist at Hitachi Vantara Federal; Dominic Critchlow, chief scientist at Booz Allen Hamilton; and Richard Goodman, Europe, Middle East and Africa (EMEA) defense lead at Hexagon’s Safety, Infrastructure & Geospatial division for business development.

The panel spoke on the strategic importance of data interoperability for military organizations.

Rix, who moderated the discussion, began by providing context on the vast amount of data types, referring to it as a “Tower of Babel” problem. This is where data cannot be exchanged or translated between different systems, she stated. “With advancements in data science, database management, computing power, along with the introduction of [artificial intelligence], that has started to change the way that we approach this Tower of Babel.”

Rix therefore asked her fellow panelists how they believe the challenge of data ingestion can be overcome.

While Goodman said he believes the answer comes down to culture and procedures to help “loosen the reigns,” Critchlow offered further ideas on how the large amounts of data can be brought together to help answer the real questions.

“First, reliable API [application programming interface] services as they are the underlying foundation to get to our data,” Critchlow stated. “For doing data mesh, for doing data integration layers, industry needs to focus on having reliable API services. In today’s government systems, reliable APIs are not there, they are limited, just not accessible or not reliable at all.”

The government must put together strict requirements around APIs, he said. “Being very specific that every piece of data and every function needs to be accessible and controlled through the API, that’s going to allow us to move the needle forward in data modernization.”

Nayak echoed Critchlow’s statements, saying automation and orchestration would allow for different data sources to be catalogued and tagged, offering users the ability to search for data and see its quality and lineage.

The panel also spoke on open-source intelligence, discussing the opportunities it presents.

Open-source software development, as regularly used and developed by industry, could be an answer to the data ingestion problem, Critchlow said. “I believe that the defense and intelligence community is not doing enough there,” he stated.

Critchlow continued, saying that “open sourcing software is an amazing way to scale capabilities because, for example, the mechanic in the Marine Corps, or an IT [information technology] service provider from the Air Force can access the code repository, ask a question about it, run it and contribute to it. It allows us to build open standards and build common foundations reducing vendor lock-in.”

Following Rix’s follow-up question on open-source data, the speakers talked about the utility and hazards of nonmilitary sources.

“Data from outside military sources may be more informative,” Goodman answered. “It may capture things that the military just doesn’t look for.” Additionally, Goodman stated open-source data can help advance the trust level of military data.

Open-source data should go through the quality controls necessary for military sources, Nayak added. Additional processes will also be required, he said.

Critchlow added by speaking on Pokémon Go, a mobile game that was released in 2016. “Open-source data, especially something like Pokémon Go, is an opportunity for us to do more open-source intelligence and intelligence fusion staying at the unclassified level. This also allows us to share that information easily with partners, bringing more people to solve the problem that don't need clearances,” he said.

Still, there is risk involved in the use of data information, such as adversaries spiking certain data for personal advantage. According to Critchlow, however, that is a risk that can be worked through. “We should not shy away from using more open-source data because it allows us to be much more effective than we are today,” he said.

“Open-source data can be used as a starting point for us to then cue in to say, ‘Oh, there's a lot of reporting on Twitter that this thing is popping off, maybe I need to go verify that.’ The marking of that data and making sure we pay adherence to the provenance of that data and having it being marked as open source is very important,” Rix said.

The panel also spoke on classification of data, saying the rapidly growing amount of data creates a challenge for a human classification landscape. “We need to start implementing digital policies that are enforceable directly on those systems,” Critchlow offered.

Nayak, in response, added a call for a confidential query and trusted execution environment.

“Simple things like nomenclature need to be set and decided so users know what’s being referred to,” Goodman also stated.

Everything comes back to data, Rix concluded, emphasizing the vital importance of requirements to achieve interoperability in data.