US intelligence agencies' embrace of generative AI is at once wary and urgent

ARLINGTON, Virginia — Long before generative AI's boom, a Silicon Valley firm contracted to collect and analyze non-classified data on illicit Chinese fentanyl trafficking made a compelling case for its embrace by U.S. intelligence agencies.

The operation's results far exceeded human-only analysis, finding twice as many companies and 400% more people engaged in illegal or suspicious commerce in the deadly opioid.

Excited U.S. intelligence officials touted the results publicly — the AI made connections based mostly on internet and dark-web data — and shared them with Beijing authorities, urging a crackdown.

One important aspect of the 2019 operation, called Sable Spear, that has not previously been reported: The firm used generative AI to provide U.S. agencies — three years ahead of the release of OpenAI's groundbreaking ChatGPT product — with evidence summaries for potential criminal cases, saving countless work hours.

''You wouldn't be able to do that without artificial intelligence,'' said Brian Drake, the Defense Intelligence Agency's then-director of AI and the project coordinator.

The contractor, Rhombus Power, would later use generative AI to predict Russia's full-scale invasion of Ukraine with 80% certainty four months in advance, for a different U.S. government client. Rhombus says it also alerts government customers, who it declines to name, to imminent North Korean missile launches and Chinese space operations.

U.S. intelligence agencies are scrambling to embrace the AI revolution, believing they'll otherwise be smothered by exponential data growth as sensor-generated surveillance tech further blankets the planet.

Analysts require ''sophisticated artificial intelligence models that can digest mammoth amounts of open-source and clandestinely acquired information,'' CIA director William Burns r ecently wrote in Foreign Affairs. But that won't be simple.

The CIA's inaugural chief technology officer, Nand Mulchandani, thinks that because gen AI models ''hallucinate'' they are best treated as a ''crazy, drunk friend'' — capable of great insight and creativity but also bias-prone fibbers. There are also security and privacy issues: adversaries could steal and poison them, and they may contain sensitive personal data that officers aren't authorized to see.

That's not stopping the experimentation, though, which is mostly happening in secret.

An exception: Thousands of analysts across the 18 U.S. intelligence agencies now use a CIA-developed gen AI called Osiris. It runs on unclassified and publicly or commercially available data — what's known as open-source. It writes annotated summaries and its chatbot function lets analysts go deeper with queries.

Mulchandani said it employs multiple AI models from various commercial providers he would not name. Nor would he say whether the CIA is using gen AI for anything major on classified networks.

Mulchandani says gen AI is mostly good as a virtual assistant looking for "the needle in the needle stack.'' What it won't ever do, officials insist, is replace human analysts.

Linda Weissgold, who retired as deputy CIA director of analysis last year, thinks war-gaming will be a "killer app."

During her tenure, the agency was already using regular AI — algorithms and natural-language processing — for translation and tasks including alerting analysts during off hours to potentially important developments. The AI wouldn't be able to describe what happened — that would be classified — but could say ''here's something you need to come in and look at.''

Gen AI is expected to enhance such processes.

Its most potent intelligence use will be in predictive analysis, believes Rhombus Power's CEO, Anshu Roy. ''This is probably going to be one of the biggest paradigm shifts in the entire national security realm — the ability to predict what your adversaries are likely to do.''

AI bigshots vying for U.S. intelligence agency business include Microsoft, which announced on May 7 that it was offering OpenAI's GPT-4 for top-secret networks, though the product must still be accredited for work on classified networks.

A competitor, Primer AI, lists two unnamed intelligence agencies among its customers — which include military services, documents posted online for recent military AI workshops show. It offers AI-powered search in 100 languages to ''detect emerging signals of breaking events" of sources including Twitter, Telegram, Reddit and Discord and help identify ''key people, organizations, locations.'' Primer lists targeting among its technology's advertised uses. In a demo at an Army conference just days after the Oct. 7 Hamas attack on Israel, company executives described how their tech separates fact from fiction in the flood of online information from the Middle East.

Primer executives declined to be interviewed.

In the near term, how U.S. intelligence officials wield gen AI may be less important than counteracting how adversaries use it: To pierce U.S. defenses, spread disinformation and attempt to undermine Washington's ability to read their intent and capabilities.

And because Silicon Valley drives this technology, the White House is also concerned that any gen AI models adopted by U.S. agencies could be infiltrated and poisoned, something research indicates is very much a threat.

''If you speak to any researcher or developer that is training a large-language model, and ask them if it is possible to basically kind of delete one individual piece of information from an LLM and make it forget that -- and have a robust empirical guarantee of that forgetting -- that is not a thing that is possible,'' John Beieler, AI lead at the Office of the Director of National Intelligence, said in an interview.

It's one reason the intelligence community is not in "move-fast-and-break-things'' mode on gen AI adoption.

''We don't want to be in a world where we move quickly and deploy one of these things, and then two or three years from now realize that they have some information or some effect or some emergent behavior that we did not anticipate,'' Beieler said.

It's a concern, for instance, if government agencies decide to use AIs to explore bio- and cyber-weapons tech.

William Hartung, a senior researcher at the Quincy Institute for Responsible Statecraft, says intelligence agencies must carefully assess AIs for potential abuse lest they lead to unintended consequences such as unlawful surveillance or a rise in civilian casualties in conflicts.

Government officials insist they are sensitive to such concerns. Besides, they say, AI missions will vary widely depending on the agency involved. There's no one-size-fits-all.

Take the National Security Agency. It intercepts communications. Or the National Geospatial-Intelligence Agency (NGA). Its job includes seeing and understanding every inch of the planet. Then there is measurement and signature intel, which multiple agencies use to track threats using physical sensors.

Supercharging such missions with AI is a clear priority.

In December, the NGA issued a request for proposals for a completely new type of generative AI model. The aim is to use imagery it collects — from satellites and at ground level – to harvest precise geospatial intel with simple voice or text prompts. Gen AI models don't map roads and railways and "don't understand the basics of geography,'' the NGA's director of innovation, Mark Munsell, said in an interview.

Munsell said at an April conference in Arlington, Virginia that the U.S. government has currently only modeled and labeled about 3% of the planet.

But lots of vital intelligence work has nothing to do with data science, says Zachery Tyson Brown, a former defense intelligence officer. He believes intel agencies will invite disaster if they adopt gen AI too swiftly or completely. The models don't reason. They merely predict. And their designers can't entirely explain how they work.

Not the best tool, then, for matching wits with rival masters of deception.

''Intelligence analysis is usually more like the old trope about putting together a jigsaw puzzle, only with someone else constantly trying to steal your pieces while also placing pieces of an entirely different puzzle into the pile you're working with,'' Brown recently wrote in an in-house CIA journal. Analysts work with ''incomplete, ambiguous, often contradictory snippets of partial, unreliable information.''

They place considerable trust in instinct, colleagues and institutional memories.

''I don't see AI replacing analysts anytime soon,'' said Weissgold, the former CIA deputy director of analysis.