Databricks Adopts GLM 5.2 as Its Default Coding Engine: What the Internal Benchmark Revealed About AI Model Performance and Cost

Databricks, the data and AI company, published results from an internal coding benchmark in July 2026 that evaluated agentic AI models on real engineering tasks drawn from its own multi-million-line codebase (spanning Python, Go, TypeScript, Scala, and SQL). The benchmark's findings led the company to adopt the Chinese open-source model GLM 5.2 from Z.ai (formerly Zhipu AI) as its default coding engine. Here is what the benchmark revealed and why Databricks made the switch.

What the Benchmark Revealed

Databricks built its own test because it found public benchmarks like SWE-bench can be over-tuned for, and wanted to measure which agents could solve real tasks end to end against curated test suites . The evaluation yielded three major surprises.

Open-source models have reached the frontier. The Pareto frontier for coding tasks — meaning the best quality for a given cost — now includes models from OpenAI, Anthropic, and open-source providers. Databricks cofounder Matei Zaharia stated that "many models including open source ones are truly competitive now" . The company concluded that open models, and GLM 5.2 specifically, are now capable of handling the highest level of task difficulty they tested .

Token price is a misleading cost proxy. The benchmark found that a model's per-token price does not reliably predict actual total cost in agentic coding workflows. Larger models can be far more token-efficient, meaning a cheaper per-token model can end up costing more overall if it requires more tokens to complete the same task. This drove Databricks to evaluate models on real, end-to-end task completion cost rather than raw API rates .

Total cost of ownership favored GLM 5.2. Across Z.ai's API, GLM 5.2 is priced at roughly $1.40 per million input tokens and $4.40 per million output tokens . For a team processing 10 million tokens per month with a 50/50 input-output split, the total would be about $29 per month . Competitor models like Anthropic's Opus 4.8 at $5/$25 per million tokens can cost 3 to 6 times more for comparable or slightly better benchmark scores . On a per-task basis, one Databricks test showed GLM 5.2 using the Pi agent achieved an 87.5% pass rate at $1.25 per task, while Opus 4.8 high-effort using Claude Code achieved a comparable pass rate at $2.00 per task .

Why Databricks Adopted GLM 5.2

Performance matching frontier models at much lower cost. GLM 5.2 scored 62.1 on SWE-bench Pro, outperforming GPT-5.5 (58.6) and coming within a few points of Anthropic's Opus 4.8 . On FrontierSWE Dominance, it hit 74.4%, nearly tying Opus 4.8's 75.1% . Databricks' internal tests echoed these public benchmarks: the Chinese open-weight model matched or approached the capability of leading proprietary models on the same real-world engineering tasks .

Open-weight, MIT-licensed deployment flexibility. Because GLM 5.2 is MIT-licensed and fully open-weight, Databricks could deploy it in-house, fine-tune it, and tightly integrate it into its agentic coding workflow without per-seat licensing or vendor lock-in . This licensing model allows enterprises to run the model on their own infrastructure, avoiding recurring API costs for high-volume usage.

Fit for long-horizon, multi-step tasks. The benchmark focused on agentic coding edits that span many files and reasoning steps. GLM 5.2, with its 1-million-token context window and 744-billion-parameter mixture-of-experts architecture, was specifically optimized for this kind of repository-scale, long-horizon work rather than single-file autocomplete . On Terminal-Bench 2.1, which tests command-line and agentic task execution, it scored 81.0, making it the strongest open-source model and trailing only Claude Opus 4.8 (85.0) .