AnswersPublished17 sources
Huawei OceanStor M900: Shared KV Cache at up to 64 PB for AI Inference SuperPoDs
Huawei introduced OceanStor M900 at HUAWEI CONNECT 2026 as a shared KV cache tier for hyperscale AI inference SuperPoDs, claiming up to 64 PB of capacity per cluster. Using the company’s Lingqu UnifiedBus interconnect, M900 pools and tiers KV Cache from accelerator memory and DRAM out to SSDs for long context and mu...
Published byEdited with GPT-5.6 TerraImages generated with GPT Image 2
Research answer
Key takeaways
- Huawei introduced OceanStor M900 at HUAWEI CONNECT 2026 as a shared KV cache tier for hyperscale AI inference SuperPoDs, claiming up to 64 PB of capacity per cluster.
- Using the company’s Lingqu UnifiedBus interconnect, M900 pools and tiers KV Cache from accelerator memory and DRAM out to SSDs for long context and multi turn agent workloads.
- Huawei claims roughly 60 μs access latency, 40 TB/s aggregate bandwidth and double the token throughput in an AI coding scenario, though detailed benchmark configurations have not been published.
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: What did Huawei announce at Huawei Connect 2026 about its OceanStor M900 AI memory storage for hyperscale inference SuperPoDs, including how. Article summary: Huawei announced OceanStor M900 Context Memory Storage: an SSD-backed, shared KV-cache tier for hyperscale AI inference SuperPoDs. It is intended to extend—not replace—accelerators’ HBM/DRAM or the SuperPoD interconnect:. Topic tags: general, general web. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts with fake numbers, clic
Huawei has introduced OceanStor M900 Context Memory Storage, a storage system intended for AI inference in hyperscale data centres. Rather than replacing the HBM or DRAM attached to AI accelerators, it adds a shared and scalable context-memory tier for SuperPoDs — large, tightly interconnected AI systems. Its focus is KV Cache, which can become a major constraint with very long prompts and multi-step AI-agent tasks. 2
23
Why KV Cache is becoming an infrastructure problem
During large language model inference, the KV Cache retains intermediate state for tokens the model has already processed. Reusing that state can avoid repeating work when a conversation continues or when an agent revisits related context over several steps.
Huawei says on-chip memory and DRAM capacity become limiting factors for ultra-long context windows and multi-turn inference. Its approach uses UnifiedBus, also called Lingqu, to create a global, multi-tier KV Cache with one-hop connections, extending the cache hierarchy from accelerator memory and DRAM to SSDs. 2
26
The practical goal is to keep much more context available for storage, sharing and reuse without requiring all of it to remain in the most expensive memory tiers.
Huawei’s capacity claims: up to 64 PB per cluster
Huawei says a single M900 cluster can provide up to 64 PB of KV Cache. It also says usable KV Cache per NPU can move from the gigabyte scale to the terabyte scale, potentially raising cache-hit rates for long-context requests. 2
26
This is shared memory at the SuperPoD level, not an increase in a chip’s local HBM. Its real-world value will therefore depend on how inference software places, retrieves and reuses KV Cache, as well as the actual mix of requests running in a deployment.
Native KV semantics and direct NPU-to-SSD access
According to Huawei, M900 combines a CPU, network controller and NAND controller in one architecture. The company says this provides native KV semantics and enables one-hop NPU-to-SSD access, avoiding protocol conversion and CPU forwarding. 20
26
Huawei’s published targets include:
- around 60 microseconds of access latency;
- up to 40 TB/s of aggregate bandwidth per cluster; and
- 1.5× the bandwidth of market reference solutions, according to the company.
26
These figures are vendor claims. Huawei’s announcement does not provide full details on the model used, concurrency level, cache-hit ratios, cluster configuration or comparison methodology. Independent benchmarks on representative production workloads will be needed to assess the claims properly.
Throughput and SSD-endurance claims need validation
For a typical AI coding inference scenario, Huawei reports double the token throughput and half the time to first token (TTFT). 14 Without a published baseline and workload configuration, those figures should not be treated as a universal result across models or inference stacks.
Huawei also describes KV-aware adaptive storage, in which placement decisions consider a cache item’s value and expected lifetime when selecting storage media. The company claims support for up to 24 DWPD — drive writes per day — and about a 16× improvement in SSD read/write lifespan through hybrid media and an optimised retention algorithm. 16
18
As with the performance figures, endurance claims will need to be tested under real write rates and cache-management policies.
Part of Huawei’s broader SuperPoD architecture
M900 is more than a stand-alone storage announcement. Huawei presents UnifiedBus as an interconnect for collaboration between clusters and SuperPoDs serving agentic AI, with M900 acting as the storage and context tier in that stack. 23
In Huawei’s design:
- Ascend accelerators provide compute;
- UnifiedBus/Lingqu provides interconnect and resource sharing; and
- OceanStor M900 adds persistent, shared KV Cache at petabyte scale.
That means M900 does not itself add model-compute capacity, nor does it replace accelerators or high-speed interconnects. Its promise is to ease the memory-pressure bottleneck that emerges when long context windows and repeated agent steps make KV Cache as important as compute capacity. 2
23
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "Huawei OceanStor M900: Shared KV Cache at up to 64 PB for AI Inference SuperPoDs"?
Huawei introduced OceanStor M900 at HUAWEI CONNECT 2026 as a shared KV cache tier for hyperscale AI inference SuperPoDs, claiming up to 64 PB of capacity per cluster.
What are the key points to validate first?
Huawei introduced OceanStor M900 at HUAWEI CONNECT 2026 as a shared KV cache tier for hyperscale AI inference SuperPoDs, claiming up to 64 PB of capacity per cluster. Using the company’s Lingqu UnifiedBus interconnect, M900 pools and tiers KV Cache from accelerator memory and DRAM out to SSDs for long context and multi turn agent workloads.
What should I do next in practice?
Huawei claims roughly 60 μs access latency, 40 TB/s aggregate bandwidth and double the token throughput in an AI coding scenario, though detailed benchmark configurations have not been published.
Sources
huawei.comHuawei Introduces OceanStor M900 Context Memory Storage to Accelerate AI Inference in Hyperscale Data Centers - Huawei