Pandemic Turns Intelligence Data Upside Down
The U.S. Defense Intelligence Agency (DIA) has changed its approach to data services from an exclusively high-level activity to one at the lowest level. In moving from a centralized approach to a decentralized one, the agency has taken the same course as an army moving data from the command center down to the individual soldier in the foxhole.
This action was driven largely by the COVID-19 pandemic and the need for analysts to work from home, explains Doug Cossa, deputy chief information officer at the DIA. The agency had to establish an unclassified environment in which people could work at home, and it continues to move toward greater telework. “We don’t know what the percentage will be, but we are architecting this to be able to have an enduring teleworking presence.”
More than 80 percent of the agency’s workforce has been teleworking at one time or another, he notes. The agency expects this new model to continue after the virus passes into history. Business services such as email, human resources applications and financial systems have migrated to the low side, and this effort required rethinking the agency’s entire business data strategy. That included accommodating contracting, which constitutes a significant portion of the workforce.
Security played a major role in determining what migrated and how, he adds. Officials had to determine which data could not migrate and how to establish methods, such as a cross-domain service, that integrate the data. Not only was this a complex problem, but also it had to be solved in short period of time. For example, the agency’s email architecture was designed to support only a maximum of a couple-hundred users concurrently. Within days, the agency had well over 3,000 concurrent users integrated with normal business functions such as chat and calendar.
This work-at-home approach especially applies to open-source data, Cossa notes. Through security mechanisms and new data strategies, the agency has piloted doing much of its open-source data analysis at home.
And the proliferation of open source data is one of three areas in which data strategies are changing. Cossa says the agency is approaching open-source data as part of a community instead of as silos, particularly for collection disciplines. The DIA is standing up an open-source integration cell that uses various techniques to pull data from different disciplines including traditional collection methods and other data repositories from across the entire intelligence community.
The agency leverages partners outside of the intelligence community for some of its open-source data, Cossa allows. This builds on how the community has migrated away from making individual agency investments to community investments. For example, as agencies began to adopt common tools and services, data strategies also had to follow suit. No agency could share common tools to take advantage of a shared application without underlying shared data services, Cossa points out.
And the DIA relies on industry for much of its open-source data. “We buy data as a community, so we rely on industry,” he says. “We rely on academic partners, where we champion a number of joint projects with them.” These partnerships have led to a more shared strategy for managing and using data, he adds.
Another area of change is in the mechanisms' available preparation for analysis. Tools and techniques have provided a much easier approach to preparing data for analysis, which he describes as a big advantage across the intelligence community. Where in the past an expert might spend 80 percent of the time preparing data for importing into the right fields and only 20 percent on analysis, that ratio has flipped to the opposite values.
The third area of change is data governance. The agency has leveraged a number of new tools and techniques to help automate the analysis process so it can use the most appropriate data for a requirement.
But one of the biggest changes has been from a mindset of data ownership to one of data stewardship. Where people traditionally have been more protective of data, now they are sharing more. That does require revisiting policies and procedures, Cossa offers, but this represents more of a mindset change within the intelligence community.
The data environment has become so dynamic, with information changing every second, that tools and data strategies are changing to match its frequency. This has led to what Cossa says is data maturity. Part of that maturity is to be able to understand data and its source well enough to determine its validity, which is more of an issue now than in the past. “The big takeaway is showing and sharing your homework across the community,” he says.
Experts are looking to develop a community-wide strategy for ensuring analytic integrity as a regular function of data science. Adding new scientific capabilities, such as artificial intelligence, into the process will need to be taken into account when the policy standard is established. This is an ongoing process, Cossa adds.
Doug Cossa will be discussing intelligence community data issues in a panel at the AFCEA/INSA Intelligence and National Security Summit, being held online September 16-18.
