In closed tests on legacy code migration, models previously showed a 5.4% success rate, whereas the new version handles 97.9% of tasks.
This performance leap of language models on low-resource code destroys the multi-billion-dollar manual refactoring market and strips mainframe vendors of their technical complexity moat. Exactly how many hundreds of millions of dollars in revenue will legacy system integrators forfeit now that artificial intelligence has learned to read COBOL in seconds?
Anatomy of a breakthrough and closed tests
The company published a new benchmark, MultiLangBench, detailing the results of a second, closed test on legacy code migration. Neural networks historically perform worst on such languages because there is far less open-source code and documentation available for COBOL compared to Python or JavaScript, while the vast majority of banking software has never left enterprise premises.
In a closed test of COBOL program migration, GPT-5.6 Sol successfully resolved 5.4% of assignments.
The new GPT-6.1 Sol managed 86.6% of tasks under identical settings, and 97.9% with extended reasoning time. MultiLangBench evaluates models on large-scale tasks rather than algorithmic contest exercises where a model must build a SQLite 3.40.1-compatible database engine from scratch across one of 20 languages.
Market shock for mainframes
Back in February, competing Anthropic announced that Claude Code takes over the most labor-intensive part of COBOL modernization: parsing and documenting legacy code. On that day, IBM stock plummeted by 13.2%, marking its steepest drop since October 2000.
On that day, shares of IBM—whose mainframes have powered this code for decades—slumped by 13.2%, their worst performance since October 2000.
In its blog post, OpenAI cites IBM estimates indicating that COBOL underpins over 40% of online banking systems, 80% of retail card transactions, and 95% of ATM operations. Averaged across 20 languages, GPT-6 Astra passed 87.4% of tests, compared to 26.8% for the predecessor GPT-5.6 Sol.
Model economics and hidden risks
The cost-efficient GPT-6 Luna model scored an average of 40.3% at approximately $0.77 per attempt, versus $3.48 for GPT-5.6 Sol. The critical caveat is that OpenAI both designed and executed both tests internally, with no vendor comparisons provided, given that MultiLangBench is published on GitHub under the Apache 2.0 license.
Meanwhile, COBOL fixed-point arithmetic truncates decimal places by default unless a programmer explicitly inserts rounding logic. A migrated program that introduces automated rounding will execute without a single runtime error while calculating entirely different financial figures; consequently, domain experts must audit the migration against the business logic and dependencies of the specific system prior to production deployment.
97,9%
Editorial analysis
We have written about this before: April 23 — OpenAI’s Clinical AI Outperforms Doctors; July 14 — OpenAI UX Crisis. Banking mainframes have relied on the impenetrable barrier of ancient code for decades, yet that defense vanished in seconds without a single hardware change from vendors. Automated migration previously stalled at a miserable 5.4% success rate, whereas OpenAI's new release resolves 97.9% of COBOL challenges outright. That impenetrable barrier of legacy mainframes simply evaporated the moment automated code translation hit 97.9%.
We covered this: April 23, July 14