AI, Data-Sharing and Avoiding the Next ‘Failure of Imagination’

The terrorist attacks upon the United States on Sept. 11, 2001, exposed what became known as a widely reported “failure of imagination.” Twenty-five years later, disruptive technologies, combined with advanced interoperability among federal agencies and partner nations, are creating a catch-22 scenario: empowering a more vigilant and prepared state while enabling adversarial nations.

One of the key challenges identified during Sept. 11 investigations was the inability to connect the dots. With the FBI and CIA covering different areas of responsibility, each agency owned separate pieces of one large and complex puzzle. The CIA had information about al-Qaida’s intentions, while the FBI knew of the terrorist organization’s activities within the United States, including pilot training.

In today’s age, an important question arises: how can tools such as artificial intelligence (AI) be used to make secure, data-driven decisions to avoid another tragic event?

A key issue to consider is the sharing of raw datasets across agencies to pull critical information into a common infrastructure and operational picture. AI-enabled tools could help connect the dots; however, protecting information through access controls is nonnegotiable, given the need to protect sensitive intelligence reporting, noted Amy McAuliffe, a former CIA executive who currently serves as a distinguished visiting professor of the practice at the University of Notre Dame.

The topic has long been discussed following the attacks on New York City and Washington, D.C. The 9/11 Commission Report extensively discussed all the missing pieces—every training session, every conversation and every journey taken by Osama bin Laden’s team to meticulously plan what is considered the most tragic day in U.S. history.

A 2004 FBI special report, which was made public in 2006, repeatedly emphasized the lack of processes and understanding of collaborative work with the CIA and other federal agencies. The report provides a list of recommendations, many of which call for more comprehensive and cohesive policies to receive, report and document classified information, whether in a formal or informal setting.

Last year, Lt. Col. Mark Bailey explored this topic in a written piece about the role AI could play in such a scenario. “AI has a unique role to play in overcoming such blind spots,” he wrote. “Machine learning systems excel at analyzing enormous data sets to surface anomalies or patterns too subtle for humans to notice.”

One challenge he noted, however, is the risk of false positives. A human-in-the-loop approach with a carefully balanced system could help solve this problem.

“Imagine if, in 2000, an AI system had correlated Mohammad Atta’s repeated U.S. entries with his connections to other suspicious individuals abroad,” he offered, mentioning the lead hijacker in the Sept. 11 attacks who piloted Flight 11 into the North Tower of the World Trade Center.

“The system might not have provided certainty, but it could have prioritized closer scrutiny,” Bailey wrote. Still, the use of modern technology should remain true to American values, he wrote.

For information sharing, AI could serve as a cross-agency layer for translation and integration, he noted. “AI offers a technical solution to the problem of stovepipes, though organization culture and trust remain equally important,” he said. “The commission’s call for ‘unity of effort’ could finally be realized through AI-enabled data sharing frameworks—if agencies are willing to use them.”

While much progress has been made, concerns exist around the use of AI-enabled tools.

“I don’t think we’re in a place yet where the intelligence community across the board is willing to go that far with the raw information,” McAuliffe said in an interview with SIGNAL Media. The key concern, she shared, is about protecting sources and methods.

“I think, on the FBI side, some of the information is more on the domestic enforcement side of the house, you have to make sure to not have the U.S. person’s information and other types of information in there that the CIA should not have access to under most circumstances,” McAuliffe explained.

The point was echoed during a July GOP House Intelligence Committee hearing on 25 years post-Sept. 11 by Seth Jones, president of the Defense and Security Department and Harold Brown Chair at the Center for Strategic and International Studies, or CSIS.

Previously, Jones served in the Office of the Secretary of Defense and U.S. Special Operations Command. He also served on a congressionally mandated panel to review the FBI’s implementation of counterterrorism recommendations in the 9/11 Commission Report.

“The unfortunate reality is that most of these organizations, none of them really are sufficiently equipped to deal with the threat from state-based actors,” he stated. “We are not sharing information. In fact, I would argue that our fusion centers and our JTFs and most of our state and local entities don’t even know what to look for, aren’t equipped to look for the activities.”

Therefore, he stated, federal agencies are ill-prepared for intelligence challenges within U.S. borders, nor do they have the appropriate infrastructure to take on the problem.

The challenge extends beyond U.S. agencies, with allies and partners playing an integral role in global security.

Famously, the August 6, 2001, Presidential Daily Brief contained information on al Qaeda’s potential plans. It was later revealed that the intelligence came from British sources, highlighting the significance of strategic collaboration.

“Today’s challenge is less about whether information can move—certainly it can move domestically; it can move internationally,” said David Cattler, former director of the Defense Counterintelligence and Security Agency. “It’s whether institutions can rapidly generate common understanding from an increasingly complex set of information,” he told SIGNAL Media.

Previously, Cattler served as NATO’s assistant secretary general for intelligence and security.

“You look at NATO, for example; there are 85 intelligence and security services across the allies, 32 nations. It’s a consensus-driven organization, which means in this context, you don’t have consensus unless all 32 allies agree,” Cattler noted. “Well, what if they each use AI? And they each have a different model, and that model has different weights, and it leads them to a fundamentally different conclusion, even based somewhat on the same datasets. How will that be reconciled when they come together to debate?”

The future challenge is not data integration but data confidence, Cattler suggested. A system of checks and balances is imperative, as a human in the loop is the best way to verify outcomes.

“For national security purposes, I would caution more safeguards in the system,” he said, noting that speed does not necessarily mean success.

McAuliffe also weighed in.

“I would say that’s much improved, but still imperfect,” she stated regarding information sharing with allies and partners. Longstanding partnerships, such as the Five Eyes—a post-World War II-established alliance comprising Australia, Canada, New Zealand, the United Kingdom and the United States—have worked together long enough to create a seamless set of policies for sharing raw intelligence.

“Sometimes, if you’re talking about a new partner or a partner we haven’t shared on a certain topic, that can become difficult,” McAuliffe said. “It depends, oftentimes [on] the level of classification of information you’re starting with, and then you’re getting it downgraded to a releasable level, or even if you’re getting it more to the unclassified level, that can make things difficult.

Actionable intelligence, however, is key to current-day security operations.

“If you look at the ability of the United States to tell our allies in Europe using some downgraded intelligence that Putin was going to invade Ukraine, it was key,” McAuliffe added.

The flip side of the coin is that sharing information could lead to a loss of the collection stream, which can sometimes take a decade to collect, she explained, which brings the conversation back to AI capabilities.

“There’s the AI angle, obviously, which can really help us sort through those massive amounts of information,” McAuliffe said. “A major issue is, of course, cyber. With the AI-enabled increases in cyber capabilities, so think Mythos, the ability of our adversaries to access any of this information is increased.”

Claude Mythos is Anthropic’s frontier AI model, which launched in June, only to be suspended days later by the U.S. government due to security concerns.

“In the short-term capability, the AI-enabled capabilities like Mythos ... help the person or nation conducting offensive cyber operations,” McAuliffe said, noting the difficulty of defending versus attacking networks.

“When you think about everything from Chinese penetration of U.S. critical infrastructure all the way up to is the IC going to be sharing information test beds, that is a concern,” she continued. “Over the longer term, if you talk to experts, they think that the systems like Mythos will actually enable the defense over the longer term.”

One such example is China’s use of state-sponsored cyber threat actors to target U.S. infrastructure.

Speaking on this at the July GOP House Intelligence Committee Hearing was Frank Cilluffo, director at the McCrary Institute for Cyber and Critical Infrastructure at Auburn University. Cilluffo served as a special assistant to President George W. Bush for Homeland Security immediately following the Sept. 11 attacks.

“Volt Typhoon should have been the wakeup call that everyone is paying attention to,” he stressed.

“Volt Typhoon was no longer computer network exploit or espionage; it was no longer intelligence preparation of the battlefield; it was operational preparation of the battlefield, i.e., there was no intent other than to compromise systems that could be, at the time of their choosing, exploited,” he stated. “When you start seeing the transition from exploit to attack, that’s a really bad day.”

Cilluffo therefore called for further operational collaboration, including private entities playing a vital role in today’s cyber defense frontlines.