Context window

The maximum amount of text, measured in tokens, that a language model can take into account at once.

• A token is about three-quarters of an English word, so 1 million tokens is roughly 750,000 words: several long novels or a mid-sized codebase.
• Everything the model knows about the current task must fit inside it; anything outside is invisible unless supplied again.
• Windows grew from about 2,000 tokens in 2020 to 1 million or more in some models by 2024–26.
• Long windows are costly, since the memory the model keeps per token grows with length, and accuracy often drops on details buried mid-document.