The key facts

  • The draft bill allows blocking an AI model's release at a risk of 50+ deaths or over $1 billion in damage — in the event of a biothreat, a developer-control bypass, or a cyberattack
  • Independent auditors, registered with the US Department of Commerce, may be funded by AI labs provided that payment does not depend on results
  • A federal judge in Washington must be asked for a release ban; federal agencies are prohibited from halting models on their own

Anthropic's draft bill

As Politico reported, AI lab Anthropic circulated to Congress a draft AI safety bill. The document describes scenarios in which a model's release could be blocked: a 'foreseeable and imminent risk' of more than 50 deaths or over $1 billion in damage, provided the risk is tied to the creation or release of biological weapons, the model bypassing its developer's controls, or a cyberattack. The thresholds of 50 people and $1 billion match the definition of 'catastrophic risk' in the recently passed California law Senate Bill 53.

Assessment and blocking mechanism

Under the draft, assessment of a model's potential danger could be carried out by 'independent auditors' registered with the US Department of Commerce. They could potentially be paid by the AI labs themselves — provided that payment does not depend on the assessment results. Politico notes that it is unclear how such a system would guarantee that payment has no influence on the assessment. Earlier, DeepMind head Demis Hassabis proposed that an AI self-regulatory body resemble the financial regulator FINRA. If auditors establish an imminent catastrophic risk, they will have 72 hours to notify the Department of Commerce, after which the agency must ask a judge in Washington to halt the model's release. According to Politico, the bill explicitly prohibits any federal agency from independently stopping the distribution of a model.

Reaction and context

Anthropic did not respond to a TechCrunch request, but a company representative confirmed to Politico the existence of the draft: 'This bill reflects the proposals set out in Anthropic's Advanced AI Framework, which we have been promoting before federal and state legislators since June.' TechCrunch notes that the risks described in the draft differ from recent incidents troubling the industry: the main theme has been 'agentic' hacking — a series of cases in which AI programs, reportedly, disobeyed their operators and hacked websites on the open internet. Labs, despite warnings about existential threats, have generally resisted mandatory government oversight, preferring a voluntary system of 'neutral' built-in auditors within companies. Anthropic's proposal goes further than other industry initiatives — in particular, Nvidia CEO Jensen Huang recently argued that the AI industry does not need new laws at all.