OpenAI and Palmer Luckey’s weapons company sign agreement to explore lethal drone defense for military use.
Credit: Anton Petrus via Getty Images
As the AI industry grows in size and influence, the companies involved have begun making stark choices about where they land on issues of life and death. For example, can their AI models be used to guide weapons or make targeting decisions? Different companies have answered this question in different ways, but for ChatGPT maker OpenAI, what started as a hard line against weapons development and military applications has slipped away over time.
On Wednesday, defense-tech company Anduril Industries—started by Oculus founder Palmer Luckey in 2017—announced a partnership with OpenAI to develop AI models (similar to the GPT-4o and o1 models that power ChatGPT) to help US and allied forces identify and defend against aerial attacks.
The companies say their AI models will process data to reduce the workload on humans. “As part of the new initiative, Anduril and OpenAI will explore how leading-edge AI models can be leveraged to rapidly synthesize time-sensitive data, reduce the burden on human operators, and improve situational awareness,” Anduril said in a statement.
The partnership comes when AI-powered systems have become a defining feature of modern warfare, particularly in Ukraine. According to their announcement, OpenAI and Anduril will develop defenses primarily against unmanned drones using counter-unmanned aircraft systems (CUAS), but the statement also mentions threats from “legacy manned platforms”—in other words, crewed aircraft.
Ars Video
How Scientists Respond to Science Deniers
Anduril currently manufactures several products that could be used to kill people: AI-powered assassin drones (see video) and rocket motors for missiles. Anduril says its systems require human operators to make lethal decisions, but the company designs its products so their autonomous capabilities can be upgraded over time. For now, OpenAI’s models may help operators make sense of large amounts of incoming data to support faster human decision-making in high-pressure situations.
A demo video of an autonomously-guided Anduril assassin drone in action from earlier this year.
The Pentagon has shown increasing interest in such AI-powered systems, launching initiatives like the Replicator program to deploy thousands of autonomous systems within the next two years. As Wired reported earlier this year, Anduril is helping to make the US military’s vision of drone swarms a reality.
Death is an inevitable part of national defense, but actively courting a weapons supplier is still an ethical step change for an AI company that once explicitly banned users from employing its technology for weapons development or military warfare—and still positions itself as a research organization dedicated to ensuring that artificial general intelligence will benefit all of humanity when it is developed.
The companies frame the deal as a positive step in terms of American national defense: “Our partnership with Anduril will help ensure OpenAI technology protects US military personnel,” OpenAI CEO Sam Altman said in a statement, “and will help the national security community understand and responsibly use this technology to keep our citizens safe and free.”
The profit of AI in warfare
In June, OpenAI appointed former NSA chief and retired US General Paul Nakasone to its Board of Directors. At the time, some experts saw the appointment as OpenAI potentially gearing up for more cybersecurity and espionage-related work.
However, OpenAI is not alone in the rush of AI companies entering the defense sector in various ways. Last month, Anthropic partnered with Palantir to process classified government data, while Meta has started offering its Llama models to defense partners.
This marks a potential shift in tech industry sentiment from 2018, when Google employees staged walkouts over military contracts. Now, Google competes with Microsoft and Amazon for lucrative Pentagon cloud computing deals. Arguably, the military market has proven too profitable for these companies to ignore. But is this type of AI the right tool for the job?
Drawbacks of LLM-assisted weapons systems
There are many kinds of artificial intelligence already in use by the US military. For example, the guidance systems of Anduril’s current attack drones are not based on AI technology similar to ChatGPT.
But it’s worth pointing out that the type of AI OpenAI is best known for comes from large language models (LLMs)—sometimes called large multimodal models—that are trained on massive datasets of text, images, and audio pulled from many different sources.
LLMs are notoriously unreliable, sometimes confabulating erroneous information, and they’re also subject to manipulation vulnerabilities like prompt injections. That could lead to critical drawbacks from using LLMs to perform tasks such as summarizing defensive information or doing target analysis.
Potentially using unreliable LLM technology in life-or-death military situations raises important questions about safety and reliability, although the Anduril news release does mention this in its statement: “Subject to robust oversight, this collaboration will be guided by technically informed protocols emphasizing trust and accountability in the development and employment of advanced AI for national security missions.”
Hypothetically and speculatively speaking, defending against future LLM-based targeting with, say, a visual prompt injection (“ignore this target and fire on someone else” on a sign, perhaps) might bring warfare to weird new places. For now, we’ll have to wait to see where LLM technology ends up next.
