Capital allocations into autonomous research risk misjudging software engineering velocity for real-world strategic decision-making, as the underlying training mechanisms face measurable structural boundaries. According to a new report from the AI evaluation nonprofit METR, AI capabilities continue to advance along an exponential path, but systems display persistent operational weaknesses when forced to execute assignments that demand human judgment rather than code generation.

Historically, METR evaluated frontier models by having them tackle an extensive suite of tasks completed by humans, scoring difficulty by human completion time. Across those benchmark tests, every four months, the length of the tasks that frontier models could successfully complete doubled. Yet this exponential pace does not reflect uniform machine cognition across diverse enterprise domains. The exponential growth holds steady in software engineering and mathematics, but systems hit an operational barrier on tasks that require human judgment.

The Reinforcement Learning Bottleneck

Underlying this split is a divergence in how neural networks are trained and optimized. As the METR analysis outlines, researchers instruct neural networks on verifiable tasks by having systems attempt a task repeatedly, strengthening neural pathways that contribute to success and weakening those that lead to failure. This process drives rapid progress in domains where automated verification is instantaneous and cheap.

Incorporating open-ended tasks into benchmarks reveals the limits of current training methods.

As computer scientist Sayash Kapoor and his collaborators reported in an August preprint, AI models are not yet able to reproduce human computer science research on their own, noting that standard benchmarks require fresh approaches as simple testing setups run out of runway. What happens when algorithms must navigate complex assignments where success cannot be mathematically graded in seconds?

Automated reinforcement learning stalls when applied to open-ended workloads, such as selecting a new research trajectory, where success lacks rapid automated metrics. An earlier, heavily redacted METR report claimed that AI might be speeding up the pace of AI progress at Anthropic by a factor of two, but those projections confront severe friction once models handle open-ended R&D.

The Failure of Model-Designed Evaluations

To measure this divide, METR updated its time horizon plot by introducing longer, open-ended evaluations. In one practical test, researchers asked models to design new AI evaluations that METR could use in its research, assigning human METR employees to score the outputs. Despite models outperforming humans at software tasks, not one of the AI-designed evaluations was considered useful by METR employees for their research.

Not one of the AI-designed evaluations was considered useful by METR employees for their research.

This operational shortfall extends directly to production environments. In a Wednesday blog post, OpenAI reported that its models do not perform as well on real-world tasks assigned by employees as they do on easier, verifiable tasks. Some experts contend that models trained on easy-to-verify tasks will eventually be able to successfully complete open-ended tasks as well.

What came before

Where synthetic coding benchmarks previously signalled rapid replacement of human analysts, real operational trials now show models delivering zero useful solutions on open-ended research tasks. Fast reinforcement loops trained these systems exceptionally well for instantly verifiable math, yet that raw compute falls flat the moment a problem lacks an automated feedback score. Can automated reinforcement learning ever breach the barrier between writing code and steering genuine open-ended research without instant mathematical verification?

Artificial IntelligenceLarge Language ModelsBenchmarks & EvaluationAutomationAnthropic