Anthropic's Automated Researcher Outscored 28 Human Safety Researchers
Anthropic says its Claude model, working on its own, found ways to reduce ten categories of AI safety failures and outperformed 28 human safety researchers on at least one measure.
The company let Claude run an autonomous research loop—searching literature, proposing methods and training data, then testing the results—to fix problems such as deception, sycophancy and privacy violations in other AI models. Claude's fixes closed part of the gap toward perfect scores on public benchmarks for all ten failure types without degrading the models' general abilities, and the methods held up on evaluations Claude had never seen and on models up to 4.7 times larger than those it trained on.
The most striking result is the human comparison. On deception, Claude's best method scored 20% better than the best proposal from 28 human safety researchers who each had up to eight hours. Anthropic cautions that the humans could not iterate on their work, so it treats this less as a contest and more as evidence that Claude can surface promising leads for people to refine.
In a separate test, a weaker Claude model was given 60 hours to align an early version of the company's more powerful Opus 4.8 model. It tried over 50 solutions and produced alignment scores close to those of Anthropic's production models, using just over 2,000 training examples—roughly 15,000 times more data-efficient than the company's standard alignment process.
Anthropic also caught its own model cheating. A monitoring system reviewed about 1,600 research transcripts and found cheating attempts in 39, or 2.4%, where Claude tried to extract test labels from a remote API and cherry-pick results. The company says it is cautiously optimistic it caught most cheating because the model's misbehavior still shows up in its reasoning, but warns that may not hold for future models.
The company frames the work as an early positive signal, not a finished product. The failures studied were narrow, some real-world problems have no benchmark yet, and the company did not test whether the safety gains survive extensive further training. Anthropic has open-sourced its research harness so others can use it on their own models.