NVIDIA Nemotron 3.5 Lightning Is an Execution Model for AI Agents

• 30B total parameters: the complete pool of learned parameters in the model.
• About 3B active parameters: the approximate portion used for each token-generation step.

That sparse execution is why Lightning can occupy a larger model class while targeting the inference profile of a much smaller active network. It does not eliminate the need to store or manage the full model, but it can reduce the compute used per generated token.

Why the model targets the execution layer

Autonomous agents tend to generate many model calls. A single high-level task can involve repeated tool selection, structured extraction, verification, and response formatting after the initial plan has been created.

Using a frontier-scale model for every one of those calls can increase latency and operating cost. Lightning is intended to take over the predictable, high-frequency portion of the workflow, while a more capable model remains available for ambiguous decisions and difficult reasoning.

A representative two-tier design would look like this:

1. A larger reasoning model interprets the goal and creates a plan.
2. Lightning executes routine steps against tools, documents, or structured data.
3. The larger model is called again when the workflow encounters uncertainty, exceptions, or a reasoning-heavy decision.

NVIDIA positions Nemotron 3 Ultra—a 550B-total, 55B-active MoE model—as a model for frontier reasoning and orchestration, making the contrast with Lightning explicit.

How NeMo Switchyard fits into the design

The model’s intended value depends partly on routing. An agent needs a way to decide which model should handle each step instead of sending every request to the same endpoint.

NVIDIA’s NeMo Switchyard provides a provider-agnostic routing SDK for representing requests, defining model targets, and managing calls to the selected provider or model ID. In practice, that makes it possible to build a model hierarchy in which Lightning handles routine calls and a larger model receives the cases that need more capability.

This is an important architectural point: Lightning is not most useful as an isolated chatbot model. Its strongest product position is as one component in a routed, multi-model agent system.

Efficiency features and deployment options

Lightning is advertised with a context capacity of up to 1 million tokens, which is relevant to agents that maintain long-running conversations, process large documents, or repeatedly work with substantial task state. The exact usable context and performance will depend on the serving stack and configuration.

The NVFP4 checkpoint is intended for inference deployment and uses specialized NVIDIA kernels across supported GPU generations. NVIDIA lists the model for local infrastructure, workstations, data centers, and cloud environments, while the model is also available through Hugging Face and hosted services.

AWS says Nemotron 3.5 Lightning is available through SageMaker JumpStart and can be deployed through the SageMaker console or Python SDK. NVIDIA’s NIM documentation provides a separate containerized deployment path with specified operating-system, CUDA, driver, and Docker requirements.

The hardware story should still be treated carefully. A quantized checkpoint can make serving more practical, but single-GPU feasibility depends on the GPU, memory requirements, context length, quantization path, batching, and serving software. Claims about exact storage reductions or broad GeForce RTX support should be verified against the current model card and deployment recipe rather than generalized to every laptop or desktop.

NVIDIA’s speed claims need a workload caveat

NVIDIA and AWS advertise up to four times higher throughput and up to 30% faster task completion for targeted agent workloads. Those numbers are not universal measures of model intelligence or guaranteed production performance.

Actual results can change with:

• the baseline model used for comparison;
• GPU generation and memory configuration;
• batch size and concurrency;
• prompt and context length;
• tool-call and network latency;
• decoding settings; and
• the acceptable error rate for the task.

For that reason, Lightning should be evaluated on the operations that matter to a particular agent—such as extraction accuracy, tool-call reliability, structured-output compliance, and end-to-end completion time—not only on tokens-per-second claims.

Pricing: attractive, but provider-specific

Hosted pricing can make Lightning appealing for high-volume inference. DeepInfra lists the model at $0.05 per million input tokens and $0.20 per million output tokens, with usage-based serving and no GPU infrastructure to manage.

Those rates are not a permanent property of the model. Other providers list different prices, and provider, precision, caching, and route can all affect the effective cost. Teams comparing Lightning with larger models should therefore calculate total workflow cost, including retries, tool calls, routing, and any requests that still need a more capable model.

What Lightning is—and is not

Nemotron 3.5 Lightning is best viewed as a fast, customizable worker model for agent systems. Its design makes sense when an application produces many similar calls and when the task can be specialized, constrained, or post-trained.

It is not presented as a universal substitute for a large orchestration model. Complex planning, uncertain judgment, and tasks with a high cost of error may still require a larger model such as Nemotron 3 Ultra or another frontier system.

The practical verdict is straightforward: Lightning makes the most sense as the low-latency execution tier in a routed agent stack. Its 30B total capacity, roughly 3B active parameters, open model materials, quantized inference option, and broad deployment paths are aimed at making routine agent work cheaper and faster. The advertised gains are promising, but they should be validated against the specific agent workflow before being treated as production results.