The half-life of AI benchmarks has dropped from 38 to 19 months for releases following 2023. The industry faces a systemic methodology crisis: tests saturate faster than companies can recoup their model deployments.
To grasp the scale of the problem, examine how nine leading laboratories reported on tests from 2018 through 2026. A compilation of 224 benchmarks shows that the average lifespan of a measuring instrument is 23 months, but new models consume the test reserve twice as fast. The margin to the ceiling at launch remains identical, yet machines clear it twice as fast. This depreciation rate of measurement tools directly impacts corporate financial planning. The costs of calibrating internal test loops place a heavy burden on the budgets of specialized divisions: IT departments must regularly allocate funds to reconfigure validation environments, source independent auditors, and maintain proprietary testing infrastructure that burns through resources faster than it delivers predictable operational returns. In an environment where a company's investment cycle spans 3 to 5 years while the measurement tool expires in 19 months, chief financial officers lose the ability to accurately assess the ROI of AI deployments. Executives are forced to bake ongoing quality control system maintenance costs into CAPEX and OPEX, which inflates estimates and turns digital transformation budget forecasting into a high-risk venture.
The benchmark half-life is 23 months. For models released before 2023, it stood at 38 months; for those released later, it dropped to 19.
Businesses incur hidden costs for independent audits and internal testing because public metrics lose validity. A clear example is GPQA Diamond, released in November 2023, which died by criterion B within 24 months when Gemini 3 Pro scored 91.9%. The classic MMLU lasted longer but also exhausted itself after Gemini Ultra scored 90.04% in December 2023, 39 months post-release.
Methodology Problems and Hidden Expenses
Compounding the situation, tests break down not solely due to architectural progress. One in seven new benchmarks dies not from models, but from internal defects, a figure that rises to nearly half for agent environments. Developers poured resources into building measurement tools that disintegrated due to internal bugs before they could record the actual capabilities of the systems.
ARC-AGI-2 survived for 13 months and died after GPT-5.5 posted exactly 85.0% in April 2026 under verified conditions, hitting the target set by the ARC Prize.
Who will pay for the constant infrastructure recalibration and the development of internal test loops if standard benchmarks become obsolete before the investment planning cycle reaches completion?