# NVIDIA's ReplaySSM cuts state memory traffic in SSM decoding for hybrid language models

Canonical page: https://technooptimist.io/stories/nvidia-s-replayssm-cuts-state-memory-traffic-in-ssm-decoding-for-hybrid-ce8b2cba
Date: 12 Oct 2026
About: [NVIDIA](https://technooptimist.io/companies/nvidia.md)
Original source: https://developer.nvidia.com/blog/replayssm-rethinking-state-storage-for-faster-hybrid-model-decoding/

NVIDIA describes ReplaySSM, a way to store the state of state-space model layers as a periodic checkpoint plus a buffer of recent inputs, instead of rewriting the full state at every decoding step. This roughly halves the memory traffic that dominates decoding in these hybrid models, and the method is integrated into TensorRT-LLM, vLLM, SGLang and FlashInfer.

## Key facts

- At batch size 256 the SSM kernel runs 1.46x to 1.80x faster than tuned vLLM kernels, and the full decode step 1.20x to 1.47x faster
- With four draft tokens on two 120B-class models, throughput at batch size 512 is 1.85x and 2.10x vLLM's speculative decoding
- On GPQA-Diamond a shared NVFP4 checkpoint loses 3 to 31 points versus FP32; a split-precision setup brings three of four models within about 2 points
- No speedup at batch size 1, and end-to-end gains are smaller than kernel-level gains; the blog says it runs in production in GLM-5.3-Flash and Kimi-K3

---

The Techno Optimist tracks every frontier of technology, program by program: https://technooptimist.io/ (a Markdown version of any page: add .md to its URL; site map for agents: https://technooptimist.io/llms.txt).
Data API (JSON, realtime stream): https://technooptimist.io/developers. People read 5 pages free, then Reader $10/month; these Markdown pages are open to agents.