Xpress Challenge Cooks Up a Baker's Dozen of Competitive Submissions

The Office of the Director of National Intelligence’s (ODNI’s) Xpress challenge, designed to spur industry and academia to develop machine-generated analysis systems, has produced its first results. The challenge launched April 6 and ended July 5.

David Isaacson, ODNI program manager, says it included 387 registrants across 42 countries. In the end, it featured 15 submissions, only two of which were nonresponsive. He defines a nonresponsive submission as one that was not computer-generated but instead was written manually by humans, not meeting the challenge’s requirement for machine-generated results.

The ODNI’s Office of Analytic Integrity and Standards (AIS) evaluated the remaining 13 entries based on the challenge’s established standards. With scores that defined the winners, the ODNI then requested the source code from each winning entrant to verify its effectiveness. Isaacson emphasizes that the source code samples were requested solely to confirm that the submissions were machine-generated. “We are not looking to hold source code in any way, shape or form,” he declares. “All we are asking to do is to have it long enough to run it and verify that it is a bona fide response.”

The judging criteria included how well the software described its sources and demonstrated national security relevance. After verification, officials expect to award $500,000 in prize money this month, he offers.

Reflecting on this first challenge, officials were pleased with participation. “We’re very happy with the number of respondents,” says David Honey, ODNI’s director of science and technology. At one point, he feared that the office would receive too many applicants to be able to analyze the entries effectively. Ultimately, the number was manageable, and the total submissions fell within the expected range, he adds.­­