Anthropic Warns in Its IPO Filing It Is Near Self-Improving AI
Anthropic has warned in its updated IPO filing that it may be on the verge of building AI capable of dramatically accelerating its own development — the threshold known as recursive self-improvement.
The disclosure appears in the fine print of the company's revised S-1 filing on Monday, alongside financials showing revenue is soaring but expenses are too. Anthropic says its internal estimates indicate it "may imminently meet" its own Responsible Scaling Policy definition of "dramatically accelerated progress attributable to AI." The company adds that it has already used AI to develop new models whose leaps in capability it did not recognize at the time.
Recursive self-improvement, or RSI, is the point at which an AI becomes good enough at training itself that progress compounds rapidly. For critics, it is the scenario where AI slips beyond human control. For enthusiasts, it is the promise of a tool for scientific discovery and productivity at a scale that is hard to imagine. Anthropic frames it as both: the filing says using AI to build more capable AI "could pose a considerable safety risk, particularly if it sharply accelerates the pace of progress."
The most striking detail is the admission of past blind spots. Anthropic says its increasing use of AI in research and development may cause capabilities to advance faster than its ability to develop safety measures, and that it has "in the past not identified" a significant acceleration until after it occurred. At the same time, the company says no current model has crossed the RSI threshold and it does not anticipate being able to safely let Claude design and build successive frontier models without meaningful human supervision.
Skeptics will note that an S-1 serves as both disclosure and marketing, and a claim of being close to self-building AI could serve the latter. But the filing is a legal document, and the warning is specific enough that investors, regulators, and companies building on Anthropic's models should take it seriously.