2026On-site power generation with gas turbines and batteries (next)
Q4 2026Colossus 3 construction and expansion to 2.5-3 GW
Current obstacles
Power supply and grid constraintsGrids cannot deliver 2 GW continuous power needed. On-site generation solves this but requires permits, fuel supply and approvals.
Cooling and thermal managementManaging heat from 555k GPUs at full capacity requires sophisticated cooling. Must dissipate gigawatts of thermal energy.
Physics limits
GPU communication bandwidthAs GPU counts scale to 500,000+, the total network bandwidth needed to synchronize training across devices grows exponentially. Interconnect bandwidth becomes the bottleneck limiting larger clusters.
Power delivery physicsDelivering and distributing 2 GW of power to a single facility, and then to 555,000 individual GPUs, encounters resistance and loss limits. At some point, transmission losses dominate.
How it works
3 parts
Distributed training
Massive parallel computing
555k GPUs enable training enormous AI models in parallel. This frontier approach trains models with trillions of parameters.
Power infrastructure
On-site generation bypasses grid
On-site gas turbines and Tesla Megapacks eliminate grid dependency. Public grid couldn't deliver 2 GW continuous, enabling unprecedented scale.
GPU architecture
NVIDIA's latest generations
NVIDIA GB200 and GB300 with high-bandwidth interconnects enabling efficient coordination across 550k+ devices.
xAI, founded by Elon Musk in 2023, makes the Grok models used on X, which it absorbed in 2025. It built its Colossus supercomputer in Memphis unusually fast. SpaceX bought xAI in February 2026, valuing it at $250B.
Bought by SpaceX on 2 February 2026, in a deal valuing SpaceX at $1T and xAI at $250B.
Colossus went from empty building to 100,000 H100 GPUs in about four months in 2024.
Grok 4 (July 2025) and Grok 4.1 (November 2025) ranked near the top on reasoning tests.