AgentX Benchmark Gives Nvidia a Major Lead in Long-Context AI Agents

SemiAnalysis’s AgentX 1.0 benchmark suggests that Nvidia’s CUDA-centered hardware and serving ecosystem retains a substantial advantage on realistic, long-context coding-agent workloads. In the tested SGLang configurations, the gap reached up to 5× better cost efficiency for Nvidia on GLM 5.3 and more than 20× higher performance on Qwen 3.5 at a stated target of 90 output tokens per second per user. 26

That is a significant result—but it is not a universal ranking of Nvidia over AMD. AgentX measures a particular combination of model, accelerator, runtime, workload, concurrency, and responsiveness target. AMD’s MI355X and ATOM serving engine delivered competitive results in selected configurations, while software updates changed the ordering of some systems. 14

What AgentX tested

AgentX 1.0 is part of SemiAnalysis’s InferenceX v3 benchmark suite. Unlike conventional inference tests built around fixed prompt and output lengths, it replays multi-turn coding-agent traffic with very long contexts, including scenarios approaching 1 million tokens. 13

The v1.0 dataset contains 393 opt-in, anonymized Claude Code sessions. Eligible sessions had at least 20 requests and were processed to remove duplicate requests, certain client-specific calls, and reconstructed inputs above 990,000 tokens. The median request contained 142,000 input tokens and 444 output tokens, and 44% of sessions included subagents. 8

This traffic pattern matters because coding agents repeatedly send evolving versions of a large conversation or codebase. The benchmark therefore emphasizes sustained interactive serving rather than a single long prompt followed by a long completion.

Nvidia’s reported advantage

The clearest Nvidia lead appeared in comparisons using broadly available SGLang serving configurations:

• On GLM 5.3, SemiAnalysis reported Nvidia hardware as up to approximately 5× more cost-efficient than AMD at the tested AgentX operating point, including a target of 150 output tokens per second per user. 2
• On Qwen 3.5, a cited SGLang comparison showed Nvidia delivering more than 20× higher performance at 90 output tokens per second per user. 6

These figures should be read as operating-point comparisons, not as a single score for an entire product line. InferenceX compares platforms at specified levels of interactivity and reports cost using the serving envelope observed for each platform. 79

The practical implication is that a fixed-length benchmark such as an 8k-input/1k-output test can miss bottlenecks that become decisive in multi-turn agent traffic. An accelerator may look competitive on isolated throughput while falling behind when it must keep many large, partially repeated contexts responsive at the same time.

Why long-context agents rewarded Nvidia’s stack

High prefix-cache reuse changes the workload

AgentX workloads reuse much of the context from one request to the next. SemiAnalysis describes prefix or KV-cache hit rates often above 95% in these agentic traces. 1

That changes the balance between prefill and decode. The system is not continually processing a completely new prompt; it is maintaining and extending large cached states while producing relatively short responses. Efficiently storing, locating, transferring, and reusing those states becomes central to both throughput and cost.

DRAM capacity affects how much state stays resident

Usable high-bandwidth memory capacity determines how much KV-cache state can remain on the accelerator. When the working set exceeds available device memory, serving systems may need to move data to host memory or otherwise manage cache state across a slower path. 1

Host-memory offload can expand the number of sessions a platform supports, but it introduces bandwidth and latency costs. A platform that keeps more of the active cache resident—or manages movement more efficiently—can maintain a higher level of interactivity at a given cost.

Tokenization overhead matters in short-output, many-turn traffic

SemiAnalysis also highlighted TensorRT-LLM’s boundary-aware incremental tokenization. Rather than repeatedly tokenizing an entire evolving prompt, the approach identifies stable boundaries and processes newly appended material. 1

That optimization is especially relevant to coding agents: requests can contain very large contexts but generate only a few hundred tokens. In such a workload, CPU-side preprocessing and repeated tokenization can become a meaningful part of serving overhead.

The broader lesson is that inference price-performance is a product of hardware × runtime × model × workload × latency target. FLOPS, memory bandwidth, or a single throughput number cannot describe the whole result.

AMD still produced important wins

AgentX did not show a blanket Nvidia victory. SemiAnalysis reported meaningful AMD wins or near-parity in selected model, throughput, and engine combinations, particularly on the MI355X paired with AMD’s ATOM serving engine. 1

The telemetry separates MI355X results for ATOM, SGLang, vLLM, and other configurations. That distinction is essential: an “AMD result” depends heavily on the serving engine, model implementation, and tuning used in the test. 4

ATOM’s specialized scheduling, cache handling, and AMD-focused kernels can perform well when the model and workload fit the MI355X’s memory configuration. In other words, AMD can be highly competitive when the operator is willing to use a runtime optimized specifically for the platform.

Why upstream vLLM and SGLang matter more to most buyers

A specialized engine can demonstrate what hardware is capable of under favorable conditions. But infrastructure buyers usually need more than a best-case kernel result. They also need model coverage, release cadence, tooling, deployment expertise, and an operational path that can be maintained over time.

SemiAnalysis says its recipes principally follow upstream vLLM and SGLang guidance to measure configurations customers can realistically deploy, rather than relying only on a benchmark-specific stack. 1

That makes upstream vLLM and SGLang the more decision-relevant comparison for many prospective AMD buyers. ATOM remains important evidence that AMD hardware can deliver strong performance in a suitable software environment, but its result should not be treated as interchangeable with the performance available through a common upstream serving layer.

This is an adoption and software-availability distinction—not a claim that ATOM is invalid or that specialized runtimes have no value.

Software updates can overturn hardware rankings

The benchmark also shows why inference comparisons age quickly. The telemetry includes an AMD MI355X MoRI/SGLang configuration dated August 21, alongside other AMD configurations measured on different dates. 4

An August 21 software update changed the ordering of at least one comparison, illustrating how scheduling improvements, kernels, cache movement, and model support can alter the apparent hardware hierarchy within days. 14

SemiAnalysis’s earlier MI355X work on Qwen 3.5 provides a similar warning: successive SGLang releases produced substantial performance gains over a 13-week period. 12

For buyers, the practical takeaway is to record the exact runtime version, configuration, model quantization, concurrency range, and target interactivity when comparing accelerators. A hardware verdict without that context is likely to become misleading as serving software improves.

Important limits of the result

AgentX is a valuable proxy for long-context coding agents, but it is not a representative census of every production workload. The dataset comes from an internal, opt-in trace corpus and contains 393 selected sessions rather than all enterprise-agent traffic. 8

Results may differ for:

• Short-context chat;
• Retrieval-augmented generation;
• Batch inference;
• Image or video workloads;
• Other agent frameworks;
• Different context-length distributions; or
• Different latency and throughput requirements.

Google TPUs were also absent from this initial comparative result set, so AgentX v1.0 cannot establish a three-way Nvidia-versus-AMD-versus-Google conclusion. 1

The comparison is also a moving target. SemiAnalysis identified Nvidia Rubin, AMD MI455X UALoE72, and newer TPU systems as forthcoming additions or future points of comparison. Their inclusion could change the competitive picture. 111

Why the open-source release matters

SemiAnalysis released the AgentX dataset, harness, configurations, telemetry, and related materials under the Apache 2.0 license. 1

The benchmark’s longer-term importance may be less about one Nvidia-versus-AMD headline and more about the engineering work it encourages. SemiAnalysis reported that AgentX had already become a target workload for more than 70 upstream pull requests spanning projects including vLLM, SGLang, TensorRT-LLM, ATOM, AITER, Dynamo, LMCache, and Mooncake. 1

That gives serving-framework developers a reproducible way to optimize for real agent behavior: high prefix reuse, large KV caches, many short turns, subagents, cache transfers, scheduling pressure, and tokenization overhead.

Bottom line

AgentX strengthens the case that Nvidia’s software and serving ecosystem remains a major advantage for long-context coding-agent inference. In the tested SGLang comparisons, Nvidia’s lead was reported at up to 5× on GLM 5.3 cost efficiency and above 20× on Qwen 3.5 performance at a specified interactivity target. 26

But the benchmark does not prove that Nvidia always wins. AMD’s MI355X and ATOM showed that competitive or better price-performance is possible in selected, purpose-built configurations, while rapid serving-software improvements can reverse rankings. The most useful conclusion for infrastructure teams is therefore not to pick a winner from one headline number, but to reproduce the comparison using their own model, runtime, context distribution, and latency target.