The Intelligence Community Challenges Industry

The Office of the Director of National Intelligence is daring industry to develop a new generation of intelligence technologies that would change the way analysts parse and process information. Its Intelligence Ventures in Exploratory Science and Technology effort, also known as In-VEST, aims to draw out the latest commercial technologies that could aid the community.

In-VEST is modeled after the Grand Challenge programs pioneered by the Defense Advanced Research Projects Agency (DARPA) that use prize competitions to promote innovation. The office also is working outside the traditional acquisition process in the hope of attracting new industry partners and cutting-edge technologies. 

The idea at the heart of this venture is to determine whether machines can generate finished intelligence, says David A. Honey, the office’s (ODNI’s) director of science and technology. Success in this realm would be extremely disruptive to the way intelligence organizations work today, he continues. If machines could do this work, analysts could devote more of their limited time to helping leaders understand the meaning of intelligence products, the work they prefer.

David M. Isaacson, program manager at the ODNI, explains that this effort evolved from the Intelligence Science and Technology Partnership, or In-STeP, which began about two years ago (SIGNAL Magazine, April 2015, “Intelligence Community Strives ...”). Through In-STeP, the community worked with industry to develop technical road maps in six areas. That led to the Intelligence Community Science and Technology Strategic Plan, which comes in both classified and unclassified versions. It helped form the basis for In-VEST. 

“We’re looking to catalyze disruptive research approaches to current intelligence community business models by encouraging commercial investment and informing intelligence acquisition communities of emerging technology opportunities,” Isaacson states.

The ODNI is taking a two-pronged approach to achieving this goal. One is a request for information (RFI) pathway in which the private sector responds with descriptions of its relevant activities. The second is the In-VEST Xploratory Challenge Series, a prize competition that kicks off this month. Some of the technologies to emerge from this series will be classified, but whenever possible, they will be unclassified and open, Isaacson says. 

The In-VEST challenge zeroes in on two new science areas identified in the six In-STeP technical road maps: data management and advanced analytics. These two activities are performed in the intelligence community every day, Isaacson observes, and they are strongly attached to intelligence operations.

Industry partners also identified seven challenge areas that would help shape the future intelligence production environment. The ODNI is using a portfolio method to tackle these challenges, Isaacson says, adding that this is the first time this has been tried in the intelligence community. With this structure, companies can work on an a la carte basis. They can skip a challenge and work on the RFI, or vice versa. The In-VEST program also is using a tiered approach for the challenge so that experts can view how different elements of a proposed solution perform. 

The challenge and RFI datasets will be diverse, but the questions to be asked of participants will be as consistent as possible, Isaacson states. The thrust is to make the challenge open and inclusive, but if one of the challenge areas does not generate a useful outcome, then it may be altered to use classified datasets.

In likening In-VEST to DARPA’s Grand Challenge for autonomous vehicles, Isaacson points out that the first race in that series produced no winner—none of the vehicles crossed the finish line, and all participants fell well short of the goal. But DARPA and the entrants learned from that effort, and it bore fruit the next year with several finishers. “What we want to do is get a snapshot of where the state of the art is today so we can start to work on these issues—identify the research challenges going forward,” Isaacson states.

In-VEST is taking this approach because many of the advanced technologies the community needs do not exist yet, so it would have difficulty establishing an acquisition program, Honey explains. Isaacson offers that the pairing of the challenge with the RFI process is evolutionary, but it allows for revolutionary developments. “It’s a methodical approach for producing revolutionary capabilities,” he imparts.

The first of the seven key areas in the challenge series is known as Xpress. It will explore whether algorithms can be used to generate finished intelligence that currently is human-based, says Isaacson, who also is an intelligence analyst. 

This would not turn over the entire generation of an intelligence product to automation. It would, however, pick up some of the slack in the intelligence production environment, Isaacson allows. “Not every area, not every issue, gets the same amount of resources,” he relates. “Can we compensate for that?”

The INT that stands to gain the most from Xpress is open source intelligence, or OSINT, because of the large amount of OSINT data that must be aggregated into a usable product, Isaacson offers. But other INTs could benefit as the In-VEST program progresses. Ultimately, Honey says, it will affect all the INTs because machines will tell particular intelligence collection and analysis systems what they need to do. The production cycle will become infinitely faster, he adds.

The Xpress challenge consists of two stages, with one question per stage. Xpress participants have two months to respond to the first challenge question. The top-scoring performers—probably about 50, Isaacson suggests—move to a second round of competition with a shorter response period. Up for grabs is about $1 million in prizes, the ODNI reports.

Xpress is being issued as a public prize challenge through a challenge management company, and it will be paired with a companion RFI. The other six Xploratory challenges will be similarly matched with RFIs. (See “The In-VEST Xploratory Challenge Series.”) Isaacson offers that the program seeks to issue the RFIs at the unclassified level while having the ability to receive classified responses. This opens the door to nontraditional contractors, he notes.

In the competition, any finished intelligence products generated by machines must meet the same analytical integrity standards that apply to human-generated products, Honey points out. The people judging the machine output in the program’s evaluation process will apply that quality standard. 

He continues that it is speculative as to whether this goal can be achieved, so it will not surprise ODNI experts if a challenge defines technological limits. However, this would help participants in the second round “cross the finish line” in the same manner that the DARPA Grand Challenge produced success in its second iteration.

Once Xpress concludes around mid-March 2017, the other six areas in the series will follow in a sequence to be determined. Isaacson notes that among the activities to be examined is hypothesis generation, an area where automation could aid analysts, he says. Other challenges will involve human-machine interfaces and collaboration between analysts and collectors. No two challenges will overlap in calendar terms, so the entire list of challenges should wrap up sometime in 2018. 

The potential changes if the In-VEST effort is successful are “gigantic,” both Honey and Isaacson declare. Analysts who need to generate a briefing on short notice will find tailored information already at their fingertips. “It’s not just that the machines can generate intelligence products on demand,” Honey describes. “With time, they’ll learn what kind of questions you ask, and they’ll anticipate. They will generate the products in advance.” 

Analysts will be freed up to perform the work they really want to do. This might entail focusing on strategic opportunities instead of generating updates for task forces, Isaacson suggests. Analysts currently must go through raw data feeds to produce needed material before they can perform the type of processing they would like.

They also will have more time to focus on driving collection instead of just producing pieces, Isaacson says. This will help eliminate collection gaps and improve efficiency. Similarly, officials more easily will be able to track which reports are read and which are not.

If successful, the results of In-VEST could generate “tremendous commercial interest,” Honey adds. The challenges could spark startups as people consider striking out on their own outside of their regular jobs. This will help drive the technology in an area from which the intelligence community will benefit, he points out.