KV Caching solution (key-value cache) integrates Supermicro Petascale storage servers, IBM Storage Scale software, NVIDIA Dynamo inference software, and NVIDIA Spectrum -X Ethernet switches.

Performance testing shows that, compared to systems without the KV cache, this KV caching solution can reduce the time required to generate the first token ( Time-to-First-Token, TTFT) under a long context window.

Customers can significantly improve the AI inference efficiency by leveraging a higher GPU utilization rate, greater throughput, and more scalable infrastructure.

San Jose, California, USA, November 5, 2026 / PRNewswire / -- Supermicro , Inc . (Nasdaq ticker: SMCI), a provider of IT solutions for AI, enterprises, storage, and the 5G/edge fields, offering DCBBS ® ( IT ) solutions, today announced the launch of a new KV cache storage solution jointly developed with IBM and NVIDIA. This joint solution integrates Supermicro Petascale all-flash storage servers, AMD EPYC 9005 series processors (“ Turin ”), IBM Storage Scale software, NVIDIA Dynamo software for inference management, and Spectrum-X Ethernet networking technology.

Supermicro indicates that when these technologies are combined, they can significantly reduce the time required for the AI model to start generating responses. This advantage becomes particularly important as more and more enterprises adopt agent AI workflows and long-context AI workflows. By expanding the KV cache capacity beyond local GPU memory, this solution can improve GPU utilization and inference throughput, while also supporting high availability of the KV cached content, and allowing for the expansion of the KV cache capacity without interrupting services. As a result, a more scalable and robust inference architecture is formed, providing a faster and more responsive AI experience.

Supermicro, President and CEO, Charles Liang stated: "As the workload of AI expands from training to inference, enterprises that deploy high-throughput inference workloads and neocloud providers, especially those dealing with intelligent agents AI as well as generative AI workloads that require long contexts and multiple rounds of interaction, must eliminate bottlenecks to improve GPU efficiency. Our combined solution of IBM and NVIDIA clearly demonstrates these performance advantages."

Click here to view a video summary of this press release.

Inference workloads are divided into two phases: prefill and decode. During the prefill phase, the input text is converted into token and processed by a large language model (LLM), generating the intermediate key-value state required for inference (KV state). After storing these previously computed KV states in a persistent KV cache, the system can reuse them in subsequent requests without having to recalculate them for the same context. Agent workflows, programming assistants, long-context retrieval enhanced generation (RAG), multimodal AI, and multi-round interaction applications may generate a large amount of KV state data when processing large codebases, documents, conversations, images, and other contexts. As a result, the memory capacity of GPU and CPU can quickly be exhausted, which makes it necessary to place the KV cache in a high-capacity shared network storage.

The KV caching solutions for Supermicro, IBM, and NVIDIA provide this persistent storage layer, thereby helping to improve inference efficiency, throughput, and scalability, as well as reducing unnecessary calculations.

The main advantages of this KV caching solution for customers include:

• Joint verification scheme – Supermicro, IBM, and NVIDIA have been extensively tested at the cluster level to measure performance, including scenarios with high concurrent requests, large network traffic, and other workloads that may affect performance (“noisy neighbors”).
• Significantly faster and more consistent inference performance – In a joint test conducted by Supermicro and NVIDIA on a large language model with a parameter scale of 120 billion, persisting the KV cache reduced the TTFT time for 130,000 token prompts by up to 56 times.[1] When the input length increased from 30,000 token to 130,000 token, enabling the KV cache only increased the time by 0.377 seconds, whereas not enabling the KV cache increased it by 29.55 seconds. This makes the performance of long-context workloads more consistent.
• Higher GPU utilization and throughput – by keeping frequently used contexts outside of GPU memory, this approach helps to reduce GPU idle time, increase inference throughput, and handle more concurrent requests without the need to proportionally expand GPU capacity.
• Enterprise-oriented robustness and reliability: AI operations – High-availability persistent KV caching helps to protect the stored inference state, while providing shared access for multiple GPU nodes and clusters. This supports more robust, large-scale deployable AI services that make more efficient use of storage capacity.
• Verified and ready for direct deployment, the AI infrastructure – consisting of Supermicro, IBM, and NVIDIA – has been jointly tested in high-concurrency request scenarios as well as in the “noisy neighbors” use cases. This integrated architecture combines NVIDIA Dynamo, IBM Storage Scale software, Supermicro Petascale servers, and NVIDIA Spectrum-X network technology.
• Accelerated deployment services – With the factory rack integration and on-site deployment services provided by Supermicro, the deployment time for this solution can be reduced from several weeks to just a few days.

IBM Storage General Manager Sam Werner stated: "Enterprises need a fundamentally better way to expand AI inference. IBM, Supermicro, and NVIDIA jointly provide a powerful and robust KV caching solution that enables customers to maximize their GPU investments, expand storage capacity without relying on computational power expansion, and significantly improve infrastructure efficiency. This collaboration is an important step in making enterprise AI inference more scalable, reliable, and cost-effective."

NVIDIA AI Inference Product Lead Shruti Koparkar indicates that the agent AI and long-context applications rely on efficient reuse of context across multiple steps. NVIDIA Dynamo helps to coordinate distributed reasoning, enabling enterprises to obtain more available throughput from each GPU. By combining NVIDIA Dynamo and Spectrum - X Ethernet with Supermicro Petascale storage servers and IBM Storage Scale, this KV caching solution constructs a powerful shared storage layer that can reduce unnecessary computations, shorten the time required for the first token generation, and assist customers in expanding robust AI reasoning services.

Click here to learn more about the solutions for Supermicro and IBM. Click here to read IBM Redbook " Context without Limits : A High - Performance KV Cache Platform for Large - Scale AI Inference ".

Regarding Super Micro Computer, Inc, and Supermicro (Nasdaq ticker: SMCI), they are global leaders in providing application-optimized, comprehensive IT solutions. Supermicro was founded and is headquartered in San Jose, California, with a mission to be at the forefront of bringing innovations in enterprise, cloud, AI, and 5G telecommunications/edge IT infrastructure to the market. The company offers complete IT solutions that include servers, AI, storage, IoT and switching systems, software, and support services. Supermicro's expertise in motherboard, power supply, and chassis design supports its research and development and manufacturing efforts, and drives next-generation innovation from cloud to edge, serving customers worldwide. The company's products are independently designed and manufactured in the United States, Taiwan, and the Netherlands, and leverage global operations to achieve scalability and efficiency improvements. In addition, these products are optimized to enhance TCO and reduce environmental impact (green computing). With its award-winning Server Building Block Solutions ® product portfolio, customers can choose from a range of systems composed of flexible, reusable modules and configure them precisely to meet their specific workloads and application requirements. These systems support various form factors, processors, memory, GPU, storage, networking, power supply, and cooling solutions (air conditioning, natural air cooling, or liquid cooling).

Supermicro, Server Building Block Solutions and We Keep IT Green are trademarks and/or registered trademarks of Super Micro Computer, Inc.

All other brands, names, and trademarks are the property of their respective owners.

1 For details of the results, see IBM Redbook : https :// www.redbooks.ibm.com / docs / MD260021 / MD260021.html.