by NVIDIAUS
First product of NVIDIA's $20B non-exclusive licence from Groq (announced Dec 24, 2025). 256 LPUs in a liquid-cooled rack optimized for ultra-low-latency token decoding. Complements GPUs for the full inference pipeline.
256 LPUs per liquid-cooled rack, 640 Tb/s interconnect; H2 2026
Updated 24 Dec 2025Checked 10 Oct0 updates this week
Language Processing Unit optimized for decoding: generating one token at a time. Trades dense matrix math (prefill) for minimal latency and bandwidth.
Liquid cooling allows higher chip density and lower power overhead per unit compute, enabling 256 LPUs in one rack.
GPUs handle prefill: ingesting many tokens at once. LPUs handle decode: one token per cycle. Together they optimize end-to-end inference latency and throughput.
| Spec | Groq 3 LPX |
|---|---|
| LPUs per rack | 256R (reported) |
| Rack interconnect bandwidth | 640 Tb/sR (reported) |
| On-chip SRAM per LPU | 128 GBR (reported) |
R reported by the company
NVIDIA
AI GPUs, rack systems, CUDA, networking
NVIDIA designs the GPUs, networking and software behind most AI training and inference. Its CUDA software made GPUs the default for AI. In October 2025 it became the first company worth $5 trillion.
