AnswersPublished18 sources
OpenAI Reports Big COBOL Coding Gains—but Benchmarks Aren’t Proof of Safe Migration
OpenAI reports GPT 6 Astra averaged an 87.4% test pass rate across 20 languages, while GPT 6.1 Sol scored 86.6% on a COBOL migration test at low reasoning effort and 97.9% at higher effort.
Published byEdited with GPT-6 LunaImages generated with GPT Image 2
Research answer
Key takeaways
- OpenAI reports GPT 6 Astra averaged an 87.4% test pass rate across 20 languages, while GPT 6.1 Sol scored 86.6% on a COBOL migration test at low reasoning effort and 97.9% at higher effort.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: What did OpenAI announce on October 19, 2026, about its newest models’ improvements in writing and migrating code in low-resource languages. Article summary: On October 19, 2026, OpenAI announced stronger capabilities for writing and migrating software in low-resource languages, including COBOL, Fortran, Visual Basic and OCaml, supported by two new evaluations. The reported g. Topic tags: general, general web. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts with fake numbers, click
OpenAI’s October 19, 2026 announcement highlights progress in writing and migrating code in languages including COBOL, Fortran, Visual Basic and OCaml. The headline results are striking: GPT-6 Astra averaged an 87.4% pass rate on a 20-language coding benchmark, and GPT-6.1 Sol scored 97.9% on a separate COBOL migration evaluation with higher reasoning effort. Those figures measure performance on specific tests—not whether an AI-generated migration is ready to run in a business system. 2
18
19
Why a widely used language can still be “low-resource”
“Low-resource” describes the relative scarcity of material available for training and evaluating AI models; it does not mean a language has little real-world importance. Reporting on OpenAI’s announcement describes the languages as having limited training data. 25
COBOL illustrates the distinction. An estimate attributed to IBM puts it behind more than 40% of online banking systems and 95% of ATM transactions. That widespread use does not tell us how much suitable COBOL code is represented in AI training material. 24
What the two evaluations measure
MultiLangBench asks a model to build a SQLite-compatible database engine from scratch in each of 20 programming languages. The reported result is an average test pass rate across the languages. Coverage of the benchmark describes tests adapted from SQLite’s test suite. 2
26
The COBOL migration evaluation tests translations from COBOL into C++, Java, Kotlin, Python or Rust. The aim is to preserve the original program’s behavior, with hidden tests used to assess the translations. 19
21
These tests address two different tasks: creating software in languages that may be less familiar to a model, and translating existing software without changing its observable results. Their scores should not be compared as if they measured the same thing.
The reported scores, reasoning effort and costs
| Evaluation | Model and setting | Reported result |
|---|---|---|
| MultiLangBench, average across 20 languages | GPT-6 Astra, maximum reasoning | 87.4% pass rate, compared with 26.8% for GPT-5.6 Sol. |
| MultiLangBench | GPT-6 Luna | 40.3% pass rate; about $0.77 per sample, compared with $3.48 for GPT-5.6 Sol. |
| COBOL migration, low reasoning effort | GPT-6.1 Sol | 86.6% accuracy, compared with 5.4% for GPT-5.6 Sol; reported cost was 39% lower. |
| COBOL migration, higher reasoning effort | GPT-6.1 Sol | 97.9% accuracy. |
On MultiLangBench, GPT-6 Astra scored 72.1% on COBOL, 83.5% on Fortran and 89.4% on Visual Basic. The reported COBOL migration figures are from a separate evaluation, so the 97.9% result is not a measure of Astra’s COBOL coding score. 18
25
The reported cost comparison for GPT-6 Luna applies to its MultiLangBench result. The available figures do not establish an equivalent cost for Astra or for GPT-6.1 Sol’s higher-reasoning migration result. 18
What the results mean for code migration
The migration test is relevant to a central challenge in updating established software: a translation must preserve what the original program does, not just produce code that looks plausible. OpenAI’s reported score suggests substantial progress on its test set, especially compared with the earlier model’s low-reasoning result. 19
21
But passing a benchmark is not the same as verifying a real system. The results do not, on their own, establish that a model will preserve every business rule, handle every dependency or integration, or behave correctly on cases absent from the evaluation. Production use still calls for expert review and testing against the organization’s actual requirements and data.
What’s public—and what remains uncertain
Coverage of the announcement says OpenAI released MultiLangBench as an open benchmark. The available material does not establish that the separate COBOL migration evaluation’s hidden tests are also publicly available, so outsiders may not be able to reproduce that result in full. 19
28
The figures presented here are reported benchmark results; the sources provided do not document an independent replication. They are evidence of progress on defined tasks, not independent confirmation of production reliability.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "OpenAI Reports Big COBOL Coding Gains—but Benchmarks Aren’t Proof of Safe Migration"?
OpenAI reports GPT 6 Astra averaged an 87.4% test pass rate across 20 languages, while GPT 6.1 Sol scored 86.6% on a COBOL migration test at low reasoning effort and 97.9% at higher effort.