gpt-oss

Physics limits

• Shrinking numbers loses informationRunning on smaller hardware means storing weights in 4 to 8 bits instead of 16. Below about 4 bits accuracy drops quickly, so model size sets a floor on memory needed.
• Small models hold fewer factsMeasured knowledge capacity is about 2 bits of facts per parameter, so a 20B-parameter model simply cannot store what a trillion-parameter model can.
• Trained for likely text, not true textThe model learns to produce plausible words. Facts seen only once or twice in training can't be recalled reliably, so some rate of confident errors is built into the method.