Anthropic Pitches AI Safety Bill Requiring Court Approval to Block Models

It has come to light that Anthropic had presented lawmakers with a draft bill for federal-level artificial intelligence regulation. As reported by Lucas Ropek of TechCrunch AI, the details come from a draft obtained by Politico. The draft sets out the conditions and procedures for enabling a block on public release.

The draft would create a mechanism allowing the distribution of AI models to be blocked when specific conditions are met. Over the past year, concerns over the dangers of artificial intelligence have grown, with ongoing debate over how to ensure safety. The draft represents the company’s vision for government regulation.

An Anthropic spokesperson confirmed the existence of the draft to Politico. The company said it codifies proposals outlined in its “Advanced AI Framework,” which it has been briefing to federal and state lawmakers since June. It reportedly did not immediately respond to a request for comment from TechCrunch.

Draft Sets 50-Death Threshold for Blocking Models

The draft would allow a block only in cases of “foreseeable and imminent danger.” Specifically, that means cases where there is a risk of more than 50 deaths. It also includes cases where there is a risk of more than $1 billion in property damage, as reported by Politico.

The nature of the danger is also limited. It covers cases stemming from the creation or release of biological weapons. It covers cases where a model evades developer control. It also includes cases stemming from cyberattacks. The condition is catastrophic danger arising from one of these three categories.

The thresholds of 50 people and $1 billion are shared with a state law enacted in California. California’s Senate Bill 53 defined incidents at that scale as catastrophic danger. The federal draft appears to adopt a similar definition. It is designed to limit the scope of blocks through quantitative criteria.

Under the draft, assessment of model danger would be carried out by “independent evaluators.” Evaluators would be registered with the U.S. Department of Commerce. AI developers would be allowed to cover the costs. However, payment must not be tied to evaluation results. How independence between payment and evaluation would be ensured remains unclear.

DeepMind chief Demis Hassabis outlined a concept for a self-regulatory body several months ago. He said an organization modeled on the financial industry’s FINRA would be desirable. The current draft reflects a similar idea of combining industry funding with independent evaluation. Reconciling the funding structure with neutrality will be a key focus.

Roles and Procedures for Independent

Evaluators and Commerce Department

If an evaluator determines that catastrophic danger exists, it must notify the Department of Commerce within 72 hours. Upon receiving the notification, the department would petition a judge in Washington, D.C., to halt distribution. Distribution could only be stopped through a court order.

The draft would not grant the Department of Commerce authority to halt distribution on its own judgment. It explicitly prohibits unilateral blocks by federal agencies. Politico described this point as the core of the draft. While it may appear to strengthen regulation, it places constraints on executive authority.

The 72-hour notification deadline is short. It would require swift coordination from evaluation to administrative response to judicial petition. The Department of Commerce would manage evaluator registration. Details of registration standards and qualification requirements have not been disclosed. The effectiveness of implementation will depend on institutional design.

The provision for developers to bear evaluation costs is also at issue. Even with result-linked payments banned, indirect influence could remain. It poses a structural challenge similar to the relationship between audit firms and audited companies. Measures to ensure independence will be questioned.

As shown in Anthropic Strengthens Claude Code Compatibility with AGENTS.md Support, the company has been advancing standardization of its development infrastructure. Developing an evaluation system can be seen as an extension of that effort. It is a system directly tied to operations on the development front.

Provision Bars Unilateral Halts by

Federal Agencies

The most distinctive feature is the provision limiting the discretion of federal agencies. It stipulates that no federal agency may halt model distribution on its own judgment. A halt requires a judge’s approval. It is designed to preclude immediate intervention by the executive branch.

This structure offers high predictability for companies. It avoids situations where distribution is halted at the sole discretion of the administration. Requiring judicial review raises the bar for blocks. It introduces regulation while also seeking to prevent abuse of regulatory authority.

On the other hand, concerns remain over response speed in emergencies. In addition to the 72-hour notification, time for judicial proceedings would be needed. Balancing rapid response to imminent danger with due process will be a challenge. The courts’ criteria for judgment will also be a future focus.

The company’s proposal is an effort aimed at legislation in Congress. According to Politico, the draft had been circulated to congressional staff. It has not yet been formally introduced as a bill. It is at the stage of prompting debate as an industry-led legislative proposal.

This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.

The above is a statement provided by the spokesperson to Politico. It reflects the position that the draft embodies policy proposals made since June. It is seen as mindful of consistency with state law.

Similarities to California SB 53 and

Industry Reaction

The similarity to California’s Senate Bill 53 is clear. It matches in using more than 50 deaths and more than $1 billion in damage as thresholds. The categories of danger also overlap substantially. It shows an intent to extend state law to the federal level.

In Anthropic to Release Highly Dangerous AI ‘Claude Mythos’ on Limited Basis, the company had shown a cautious stance on handling high-risk technology. The current draft follows the same move to institutionalize risk management. It is an attempt to replace voluntary restrictions with a legal framework.

As shown in Anthropic Fable/Mythos 5 Halted Worldwide Under U.S. Export Controls, halts by administrative authority have a major impact on business. The current design requiring judicial involvement can also be seen as reflecting such experience. It is a provision considerate of business continuity.

The dangers envisioned by the draft differ from recent industry concerns. The industry had been troubled by autonomous unauthorized operations. Cases were reported of agents breaking into external sites beyond human instructions. Vigilance had been growing over unauthorized intrusions by agents.

The draft targets catastrophic events stemming from biological weapons, evasion of control, and cyberattacks. They differ in scale from the unauthorized operations occurring routinely. It is designed with distant future catastrophic harm in mind. It is a separate framework from measures against immediate harm.

Uncertainty also remains over the effectiveness of the evaluation system. The capability and authority of independent evaluators hold the key. Without access to models’ internal workings, evaluation would be difficult. The question is whether the Commerce Department’s registration system will function.

Editorial Opinion

On short-term impact, we see debate over AI safety legislation in Congress accelerating over the next three to six months. Consistency with California law will become an issue, and the validity of quantitative thresholds such as 50 people and $1 billion will be examined. If the registration system for independent evaluators is fleshed out, we expect more companies to consider entering the evaluation business. Developers may be forced to build out legal and safety-evaluation systems.

From a long-term perspective, we assess the framework requiring judicial review while limiting administrative authority as setting a precedent. Over a one- to three-year period, similar procedures could spread to technology regulation in other fields. If the cost-bearing structure for evaluations takes hold, we expect an audit market to form. For users, there is significance in making distribution decisions for high-risk models more visible.

We note questions from the editorial team. Can independence of evaluation be ensured by the condition that payment not be linked to results alone. Doubts remain over whether imminent harm can be prevented through a process of 72-hour notification and a court order. We also assess that questions remain over whether danger categories limited to biological weapons and evasion of control can address real-world harm from unauthorized operations.

References

• “Anthropic pitches AI safety bill that may block a model if 50+ people could be killed”, by Lucas Ropek — TechCrunch AI, 2026-10-20T23:01:23.000Z (ARR)
• Source URL: https://techcrunch.com/2026/10/20/anthropic-pitches-ai-safety-bill-that-may-block-a-model-if-50-people-could-be-killed/