The key facts
- Models from OpenAI, Anthropic, Google and DeepSeek frequently marked targets, including civilians and surrendering soldiers
- Models from OpenAI, Anthropic, Google, and DeepSeek often marked targets, including civilians and surrendered soldiers
- Refusals were rare, and the phrase 'it's a game' removed them
- Claude Opus 5.5 selected a bombed Iranian school in 69% of regular and 72% of 'game' trials
What the study showed
Models from leading AI labs — OpenAI, Anthropic, Google, DeepSeek and others — can be relatively easily nudged into selecting targets for military strikes, including ones involving civilians or objects such as schools, simply by telling them they are playing a game. This is the conclusion reached by Vals, a San Francisco startup specializing in AI evaluation; TechCrunch reports on the results. The researchers used simulated scenarios and a mix of generated and real imagery from Ukraine and Iran, testing whether the models would mark targets on drone and satellite images.
Refusals were rare and easily bypassed
The models frequently marked potential targets, including civilians and surrendering soldiers. When the models refused to cooperate, the researchers managed to 'jailbreak' them by claiming the combat scenarios were part of a game. Not a single researcher request was blocked by a safety classifier, and refusals were infrequent; even when a model refused, it usually agreed after the game was mentioned. Ironically, Anthropic's models performed worst in these scenarios, yet no model performed particularly well.
The companies' response
Anthropic spokesperson Eduardo Maya Silva told TechCrunch that the company values Vals' report, recommendations and cooperation, and acknowledged that the reported results are unacceptable. In a written statement, the company said the examples in the report 'clearly violate our policies' and that Anthropic blocks and will block users for such behavior; the company also confirmed that 'today's frontier models are not reliable enough for fully autonomous weapons.' Anthropic added that even before the report was published it had been developing additional safeguards that will soon take effect, and is also expanding its own testing. OpenAI spokesperson Nate Evans said the company prohibits using its public models to facilitate violence or for unauthorized national security purposes, including the scenarios simulated by Vals, and that OpenAI is developing additional measures against drone-related abuse. TechCrunch also requested comment from Meta, Google, Thinking Machines Lab and SpaceXAI.
Context: Anthropic's dispute with the Pentagon and the Iranian school
The findings come against the backdrop of a dispute between Anthropic and the Department of Defense: the company told the agency that its technology must not be used for mass surveillance of Americans or autonomous combat, while CEO Dario Amodei insisted that current AI systems are not reliable enough for life-and-death decisions without a human in the loop. The Pentagon, according to TechCrunch, argued that it must be able to use the technology for any military purpose. Anthropic currently has no active contract with the Pentagon, unlike SpaceXAI, OpenAI and Google. In Vals' tests, the models frequently marked as a target a real Iranian school next to a military complex that the US bombed in February 2026 — at the time, according to the report, more than 150 civilians were killed, and Bloomberg later reported that 'excessive reliance on AI' led to the strike. Claude Opus 5.5 selected the school in 69% of regular trials and 72% of 'game' trials.
The study's limitations and a more immediate threat
Vals notes that the models were not directly asked to make strike decisions — with such a task, according to the researchers, they usually refused to cope. However, their results could still have serious consequences: for example, target data could be used by autonomous weapons systems. In some scenarios, the models were asked to report whether people were visible; a positive answer was followed by a simulated strike, and the models saw its consequences. The report relies on commercial models from frontier labs, not on models that companies may supply to the military; it is unclear whether the new safeguards will extend to military use. The researchers also point out that technical limitations — latency and limited onboard computing power — make it difficult to use general-purpose AI models for real-time drone targeting, which reduces the immediacy of the threat. A closer threat, in their view, is the mass labeling of battlefield footage by models to create training data for autonomous targeting systems. A Congressional Research Service report, cited by TechCrunch, indicates that the Pentagon, as far as is known, does not use frontier models directly in autonomous weapons systems.



