New METR Report Highlights Anthropic Using AI to Accelerate Research and Development

An 11-week investigation by the AI safety research organization METR has revealed that Anthropic is using artificial intelligence to dramatically accelerate its own research and development, with models improving at roughly twice the speed of a year ago [1]. The report, published on Thursday, offers a rare look inside one of the leading AI labs as it prepares to go public, though extensive redactions by the company have sparked concerns among researchers [1].

Acceleration of AI Research and Development

According to the METR report, Anthropic's AI models are improving roughly twice as quickly as they were a year ago [1]. Benchmark scores indicate that the company has made approximately two years of historical progress over the course of the last year [1]. METR researchers stated they are confident this acceleration was not simply caused by scaling up the computing power used during training, which has historically driven much of the progress in the field [1].

Instead, the report concludes that the majority of the acceleration as of October is driven by AI itself, with its contribution increasing over time [1]. Employee interviews and Slack polls supported these findings, with staff utilizing Anthropic's most capable internal model self-reporting roughly 12 times more productivity than they would have without AI assistance—about double the productivity reported a few months prior [1]. While human researchers are currently being assisted by AI rather than machines working entirely autonomously, METR warned that fully autonomous contributions may soon play a larger role [1].

Unreleased Models and Secrecy Concerns

Much of the acceleration and productivity gains documented by METR are attributed to unreleased internal models [1]. Over a six-month period, Anthropic completed training on three unreleased models codenamed Model 2, Model 3, and Model 4, which finished training in July, August-September, and September-October, respectively [1]. An older unreleased model from 2022, designated Unreleased 2022, was also identified [1].

While Anthropic provided METR with non-public data, confidential research, and staff interviews, the company exercised its contractual power to redact significant portions of the final report [1]. METR noted that Anthropic redacted the majority of the evidence the researchers found most informative, justifying the moves on the grounds of commercial sensitivity and to prevent handing rivals a developmental roadmap [1].

Anthropic cofounder Sam McCandlish acknowledged the heavy redactions in a post on X, stating that sharing the exact drivers of the speedup was deemed too risky [1]. He noted that the company plans to release its own report on AI progress speed and new safety steps in the coming weeks [1].

Implications for Recursive Self-Improvement

The findings have drawn significant backlash and alarm from researchers due to the apparent contradiction between Anthropic’s public calls to slow down AI development and its own rapid progress [1]. Marius Hobbhahn, CEO and founder of Apollo Research, stated that the findings suggest a pace of AI-enabled research even quicker than forecast in influential development scenarios [1]. The report brings renewed focus to the possibility of recursive self-improvement, where AI systems autonomously build successor systems, raising questions about whether safety measures can keep pace [1].

The investigation was conducted by METR following an 11-week probe inside Anthropic as part of an evaluation of its AI research and development [1]. The findings arrive while Anthropic prepares to go public, even as CEO Dario Amodei has publicly urged labs to slow down development [1].

The findings provide concrete evidence of AI accelerating its own development, offering a glimpse into how quickly frontier labs are progressing toward systems that can improve themselves with minimal human intervention [1].

The acceleration documented in the report may indicate that recursive self-improvement is closer than previously anticipated, potentially outpacing current safety frameworks [1].

Extensive redactions by Anthropic make it difficult for the public and independent researchers to fully assess the findings, uncertainties, and exact safety risks involved [1].