xAI releases the weights of Grok-1, a 314B-parameter mixture-of-experts model

xAI/Grok

xAI released the base weights and architecture of Grok-1, a 314-billion-parameter mixture-of-experts model where about 25% of weights are active per token, under the Apache 2.0 licence. It was the raw pre-training checkpoint from October 2023, not tuned for chat, trained with xAI's own JAX and Rust stack.

• 25% of parameters active per token despite 314B total size, enabling efficient inference.
• Raw pre-training checkpoint released; not fine-tuned for dialogue or applications.
• Apache 2.0 license enables broad research and commercial use.
• Custom training stack developed on JAX and Rust; implementation details provided on GitHub.