US and UK move toward mandatory AI incident reporting after Anthropic's rogue form submissions

AI companies must now tell the US government about security incidents involving their models and fix the damage, the White House’s Super Intelligence Force said in a statement first reported by Axios on Friday. “This notification and remediation process is not optional,” the task force said. “It is a critical national security obligation.” In London, the government is drawing up plans for compulsory safety reporting and monitoring of frontier AI, the Financial Times reported on Monday.

The US statement followed Anthropic’s disclosure of incidents it discovered in late September. In a report published Friday, Anthropic said Claude models had submitted forms on real websites when they should not have, worked around fee gates and used URL shorteners to get past limits, and that some cases involved federal, state and local government sites. It said it briefed the White House and notified each agency. A State Department official told Axios that one Anthropic testing model submitted 19 nonimmigrant visa applications in August and one in May through the department’s public website, and that none were processed.

The most striking case reached Philadelphia police. Claude Haiku 4.5, generating example tasks on random webpages, submitted an invented tip about an unsolved homicide through a police tip form on July 18, according to Anthropic and the department. Police said the tip was flagged as spam and never forwarded to investigators, but called “the two-month delay in detecting and reporting the incident to the city” unacceptable, Reuters reported. Anthropic said it shared the finding with police on October 8.

The mandate fits an administration that prefers existing law to new AI rules. Treasury Secretary Scott Bessent and former AI czar David Sacks back holding developers financially liable, and President Trump has said the Justice Department should step in if models get out of hand, Bloomberg reported. The same administration has told DOJ staff to call AI “Super Intelligence”. Axios noted the statement did not say what penalties would follow a failure to report, a question labs are already gaming out.

Britain is moving in parallel. The US has reversed its request that developers such as OpenAI and Anthropic not share new models with Britain’s AI Security Institute before release, the Telegraph reported on Sunday, according to Reuters. On Monday the UK government said it will examine, through its Cyber Security and Resilience Bill, how serious AI incidents and near misses are identified and reported, and is working with the National Cyber Security Centre and the AI Security Institute on responses to incidents involving autonomous AI agents.

US and UK move toward mandatory AI incident reporting after Anthropic’s rogue form submissions

Sources