AnswersPublished19 sources
OpenAI’s Jalapeño Chip Claims Faster, More Efficient AI Inference Than Nvidia GB200 and GB300
OpenAI says Jalapeño delivered 1.5–1.9× more AI work per watt and 1.7–3.6× lower end to end latency than compared Nvidia GB200 and GB300 systems in InferenceX tests. For DeepSeek R1, OpenAI reported about 700 tokens per second per user on Jalapeño versus 169 on GB300 at minimum time between tokens.
Published byEdited with GPT-5.6 LunaImages generated with GPT Image 1.5
Research answer
Key takeaways
- OpenAI says Jalapeño delivered 1.5–1.9× more AI work per watt and 1.7–3.6× lower end to end latency than compared Nvidia GB200 and GB300 systems in InferenceX tests.
- For DeepSeek R1, OpenAI reported about 700 tokens per second per user on Jalapeño versus 169 on GB300 at minimum time between tokens.
- Jalapeño is designed for inference, not training, and OpenAI plans limited internal deployment by the end of 2026 rather than external sales.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: What did OpenAI’s custom Jalapeño inference chip, co-developed with Broadcom, demonstrate in its latest benchmarks against Nvidia’s GB200 an. Article summary: OpenAI says Jalapeño, its first Broadcom co-developed custom accelerator, beats the compared Nvidia GB200/GB300 rack systems on the tested inference workloads in both energy efficiency and response speed. These are early. Topic tags: general, documentation, general web, education, user generated. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text,
OpenAI’s first custom accelerator, Jalapeño, showed striking results in the company’s latest inference tests: more AI work per watt, lower end-to-end latency, and unusually high interactive token rates compared with the Nvidia GB200 and GB300 systems used as reference points. OpenAI says the gains appeared across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T. 3
4
The important qualification is that these are early, company-reported results from a defined InferenceX configuration—not an independent, universal comparison of every production AI workload or current and future Nvidia platform. 4
7
What Jalapeño reported in the benchmarks
Across the three tested models, OpenAI reported 1.5–1.9× more AI work per watt at peak throughput and 1.7–3.6× lower end-to-end latency than the comparison systems. For highly interactive workloads, OpenAI reported performance gains of 2.1–4.1×. 3
The clearest model-by-model results were:
GPT‑OSS 120B versus Nvidia GB200: Jalapeño reached a reported 85,448 peak mixed-throughput units per kilowatt, compared with 44,960 for GB200. OpenAI also reported 1.03 seconds of end-to-end latency, versus 1.80 seconds, and a minimum time between tokens of 0.69 milliseconds—equivalent to about 1,459 tokens per second per user, versus 535 on GB200.
4
DeepSeek R1 MXFP4 versus Nvidia GB300: Jalapeño delivered a reported 19,641 peak mixed-throughput units per kilowatt, versus 11,781 for GB300. End-to-end latency was 1.65 seconds compared with 5.99 seconds. At minimum time between tokens, OpenAI reported about 700 tokens per second per user, versus 169 on GB300.
6
Kimi K2.5 MXFP4 versus Nvidia GB300: OpenAI reported 1.5× higher peak throughput per kilowatt, 3.4× lower end-to-end latency, and about 694 tokens per second per user, compared with 182 on GB300.
“Tokens per second per user” is an interactivity measure: it describes how quickly an individual request receives generated tokens, rather than only measuring the total number of tokens a rack can produce under heavy parallel load. That distinction matters for chat, coding, and agent-style applications where perceived responsiveness is part of the product experience.
Why a custom inference chip can be faster
Jalapeño is an accelerator designed specifically for large language model inference, rather than a general-purpose GPU adapted to serve both training and inference workloads. OpenAI designed the architecture, Broadcom handled silicon implementation and networking, and Celestica contributed board, rack, and system integration. 17
32
The rack-scale design is built around a large connected network domain intended to keep a workload within one system. OpenAI says it places and retains model state—including the KV cache—locally where possible, reducing data movement between chips and the communication stalls that can slow generation. 4
6
The system also does not divide prefill and decode into separate fixed resource pools. Instead, OpenAI describes compute, memory, and networking resources as fungible across both phases of inference. Prefill processes the prompt and is relatively compute-intensive; decode generates the answer token by token and is often constrained by memory movement and bandwidth. 4
6
Public technical details remain incomplete. One technical description refers to a 128-chip system with roughly 1.7 exaFLOPS and 27 TB of HBM, but OpenAI’s benchmark announcement does not provide a complete rack bill of materials or enough detail to independently reproduce every result. 4
OpenAI says AI helped design the chip
OpenAI says Jalapeño moved from initial design to manufacturing tape-out in nine months, an unusually compressed timeline for an advanced custom accelerator. 17
21
The company also says its models assisted with design exploration, implementation, measurement, verification, arithmetic-circuit optimization, and programming. OpenAI reported that Codex with GPT‑Astra helped bring three previously unplanned open-weight models to high performance within two months. On selected GPT‑OSS attention and mixture-of-experts blocks, the company said AI-generated implementations were 1.5–1.8× faster than existing human-written implementations. Those figures apply to selected blocks, not complete model-serving results. 4
The benchmark’s limits
Jalapeño’s headline advantage should be read in context.
It is an inference accelerator, not a training GPU
Jalapeño is intended for serving trained models. It is not designed for model training, so OpenAI expects to continue using Nvidia, AMD, and other accelerators for training and for portions of its inference capacity. 4
6
That makes Jalapeño a specialized addition to OpenAI’s infrastructure rather than a complete replacement for the GPU systems used across the company’s workloads.
The test used a narrow serving profile
The comparison used SemiAnalysis’s public InferenceX benchmark with nominal single-turn requests containing an 8,000-token input and 1,000-token output. It measured full request serving, matched the systems for user experience, and normalized results using published chip power ratings. 4
7
SemiAnalysis said it witnessed runs in OpenAI’s lab but did not execute the complete benchmark suite itself. It also cautioned that the setup does not test longer-context or multi-turn agent workloads, routing, prefix caching, cache management, or offload behavior. Those software and serving choices can materially change production performance. 6
Power accounting affects the per-watt result
OpenAI lists Jalapeño at 700 watts of package power and says sustained measured power was at or below 550 watts on the tested workloads. The comparison used modeled figures of 1,200 watts for GB200 and 1,400 watts for GB300. As a result, the reported per-watt advantage depends partly on the workload, measurement point, and power-accounting method. 4
The Nvidia comparison is time-bound
The tests compared Jalapeño with the best available GB200 or GB300 results at the time, but newer Nvidia systems may be more relevant by the time Jalapeño is deployed at scale. Nvidia also reports GB300 DeepSeek R1 results using MLPerf’s different offline methodology. Those figures should not be directly compared with Jalapeño’s interactive tokens-per-user results. 1
2
4
Deployment and competitive strategy
OpenAI plans a limited deployment of Jalapeño in its own compute infrastructure by the end of 2026, with greater capacity expected in 2027. The company says a second generation is deep in development and a third generation is beginning to take shape. 4
6
For now, Jalapeño is an internal platform rather than a chip that outside companies can buy or rent. OpenAI hardware executive Richard Ho said internal demand is too high for external sales to be envisioned. 6
The strategy is therefore to supplement, not immediately displace, Nvidia and AMD. OpenAI still needs general-purpose accelerators for training, and its future infrastructure is expected to remain heterogeneous. Google already develops TPUs for training and inference, while Amazon and Microsoft have built in-house AI chips; Anthropic has also described an effort to develop custom silicon. 6
The practical takeaway
Jalapeño’s early results suggest that a workload-specific accelerator can outperform general-purpose systems on the metrics that matter most for interactive LLM serving: energy efficiency, end-to-end response time, and tokens delivered to each user.
But the evidence supports a narrower conclusion than “OpenAI has beaten Nvidia.” The reported lead applies to selected models, a specific InferenceX workload, and a comparison against GB200 and GB300 configurations. It remains to be seen how Jalapeño performs on broader production traffic, longer-context agents, different software stacks, training-adjacent workloads, and the Nvidia systems available when OpenAI’s deployment expands.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "OpenAI’s Jalapeño Chip Claims Faster, More Efficient AI Inference Than Nvidia GB200 and GB300"?
OpenAI says Jalapeño delivered 1.5–1.9× more AI work per watt and 1.7–3.6× lower end to end latency than compared Nvidia GB200 and GB300 systems in InferenceX tests.
What are the key points to validate first?
OpenAI says Jalapeño delivered 1.5–1.9× more AI work per watt and 1.7–3.6× lower end to end latency than compared Nvidia GB200 and GB300 systems in InferenceX tests. For DeepSeek R1, OpenAI reported about 700 tokens per second per user on Jalapeño versus 169 on GB300 at minimum time between tokens.
What should I do next in practice?
Jalapeño is designed for inference, not training, and OpenAI plans limited internal deployment by the end of 2026 rather than external sales.
Sources
openai.comJalapeño’s first results show industry-leading speed and efficiency in AI inference
techcrunch.comOpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show | TechCrunch