Supermicro, IBM and NVIDIA Launch Shared KV Cache Storage to Speed Up Long-Context AI

Key takeaway: Supermicro, IBM and NVIDIA announced a shared KV cache storage platform on November 5, 2026, that moves an AI model’s reusable working memory onto fast shared flash storage instead of recomputing it. In joint testing, it delivered first tokens up to 56 times faster on 130,000-token prompts — though the headline figure comes from vendor-run benchmarks, not independent tests.

Long prompts are getting expensive. As AI assistants remember more — whole codebases, long documents, hours of conversation — the working memory they build up grows too large for a graphics card to hold. On November 5, 2026, Supermicro, IBM and NVIDIA announced a joint answer: move that working memory into a shared storage layer built for AI inference, so it can be reused instead of rebuilt every time.

The product is a KV cache storage solution. It combines Supermicro Petascale all-flash storage servers, IBM Storage Scale software, NVIDIA’s Dynamo inference framework and NVIDIA Spectrum-X Ethernet networking. The three companies say it is aimed at multi-turn and agentic applications, coding assistants, multimodal AI and long-context retrieval-augmented generation (RAG) — the workloads where prompts keep getting longer.

What is a KV cache, and why does it matter?

When a language model reads a prompt, it converts the text into intermediate values called keys and values — the “KV cache.” If a later request reuses the same context, the model can read the stored values instead of redoing the math. That step, called prefill, is one of the slowest parts of serving a long prompt. The problem is size: large codebases, documents and conversation histories can create more key-value state than GPU and CPU memory can retain. Once the cache no longer fits, the model recomputes it from scratch. The new platform gives that state a persistent home on shared flash storage, so it survives across requests and across GPU nodes.

What the three companies actually built

Each partner supplies one layer of the stack:

• Supermicro Petascale all-flash storage servers, running AMD EPYC 9005 Series processors, provide the physical storage layer.
• IBM Storage Scale software manages the shared storage, including the “G4” storage tier in NVIDIA’s memory hierarchy.
• NVIDIA Dynamo adds cache-aware routing, separates prefill from token generation, and moves cached blocks through its KV Block Manager and NIXL transfer library.
• NVIDIA Spectrum-X Ethernet networking moves data between the components.

The platform is part of NVIDIA’s Inference Context Memory Storage Platform (ICMSP) initiative, introduced at CES 2026, in which the KV cache spreads across GPU memory, DRAM, SSDs and shared flash storage — with this platform providing the shared layer.

The benchmark numbers — and the fine print

The headline claim is up to 56 times faster time-to-first-token (TTFT) — the wait before a model starts answering — on 130,000-token prompts. The figures come from an IBM Redbook reference architecture, Context without Limits, published June 5, 2026 and updated August 20, 2026. At 130,000 tokens, it reports 32.14 seconds for GPU recomputation versus 0.570 seconds using the storage-backed cache.

A separate 200-request run, using 28 concurrent connections and 100 unique prompts representing 24 million tokens and 825 GB of KV cache data, reported throughput rising from 0.19 to 4.26 requests per second, with total processing time falling from 1,048.56 to 46.94 seconds. The cache began empty, so that average includes a cold-start period while reuse built up. Under a “noisy neighbour” test, with four other clients generating 200 GB/s of competing network traffic, the cached system sustained 3.6 requests per second and finished 200 requests in 55.56 seconds — an 18% throughput reduction from its clean-network cached result.

The fine print matters. All measurements came from one Supermicro GPU client node with four NVIDIA RTX PRO 6000 GPUs, connected to an eight-node Petascale storage cluster, running Dynamo v0.9.0+, vLLM v0.14.1, NVIDIA AIPerf and the openai/gpt-oss-120b model, with GPU-memory and system-DRAM cache tiers disabled to isolate the shared storage tier. The 56x figure is a vendor-run result on one specific model, prompt mix, concurrency level and network — not an independent customer deployment.

Why this matters beyond the benchmark

The launch lands as AI workloads shift from single questions to long-running agents. A coding assistant that holds an entire repository in context, or an agent working through a multi-step task over hours, reuses the same context again and again. Recomputing it each time wastes GPU time — the most expensive resource in an AI data center — and makes users wait. Shared storage also changes who can run long-context models: the Redbook’s sizing examples run from four storage nodes for small proof-of-concept deployments, to eight for the tested medium configuration, to 16 or more for large AI factories.

There is a real trade-off, though. The same Redbook reports client-facing bandwidth of up to 80 GB/s while the eight-node storage system reached up to 315 GB/s of read throughput — the client network, not the storage, was the bottleneck in the test configuration. Extra storage read headroom only helps if the network can deliver it.

Analysis: what is confirmed, what is not, and what to watch

What is confirmed: the three companies announced the joint platform on November 5, 2026; the stack and the IBM Redbook measurements are public; and the 56x figure traces to that Redbook, not to a new November study.

What is still unknown: pricing, availability dates, and performance on models other than gpt-oss-120b or on workloads with low context reuse. No independent customer results have been published.

What to watch: whether rival vendors announce comparable shared-cache products, and whether real deployments reproduce anything close to the lab numbers. The idea — treat an AI model’s working memory as a shared, persistent resource — is bigger than any one product.

Frequently asked questions

What is a KV cache in AI?

A KV cache is the set of key and value states a language model computes from a prompt. Storing them lets the model reuse earlier work instead of recomputing it when the same context appears again, which speeds up responses — especially for long prompts.

What did Supermicro, IBM and NVIDIA announce on November 5, 2026?

They announced a joint KV cache storage solution that moves reusable AI inference context onto shared flash storage. It combines Supermicro Petascale servers, IBM Storage Scale software, NVIDIA Dynamo and NVIDIA Spectrum-X networking.

How much faster is it, according to the companies?

In joint testing, the companies report up to 56 times faster time-to-first-token on 130,000-token prompts. The underlying IBM Redbook measured 32.14 seconds for GPU recomputation versus 0.570 seconds with the storage-backed cache.

Who is this platform for?

It is aimed at enterprises running multi-turn and agentic AI applications, coding assistants, multimodal AI and long-context retrieval-augmented generation — workloads where the same large context is reused across many requests.

Are the 56x results independently verified?

No. The figures come from joint testing by Supermicro, IBM and NVIDIA on a specific setup: one client node with four NVIDIA RTX PRO 6000 GPUs, an eight-node storage cluster, and the openai/gpt-oss-120b model. Independent customer results have not been published.

Sources: Supermicro press release via PR Newswire; Unite.AI; Crypto Briefing; IBM Redbook: Context without Limits. For related coverage, see NVIDIA’s $250M compute commitment for UK AI safety research and Google’s Astra edge AI chip.