In less than two years, the context capacity of mainstream large language models has jumped from roughly 8,000 tokens to 1 million. Yet across the last two rounds of flagship releases, most have stopped at 260,000 to 400,000 tokens. The numbers are still rising, but the focus of improvement has shifted from “how much can fit” to “can the model complete the task accurately.”
What this is
Context length refers to the total amount of text a model can read in one pass. A token is the basic unit into which a model divides text, and context windows are usually measured in tokens. The longer the context, the more material a model can theoretically process at once, including long contracts, codebases, and multi-session records. The tradeoff is that more input also means longer inference times and higher costs.
This is not an argument that the industry has stopped expanding context windows. It points to a subtler change: as models become more capable, simply enlarging the window cannot solve information omission, mislocated details, or inconsistencies across a passage. The scarce capability is now finding the relevant information within a massive input and using it correctly.
Industry view
One view holds that million-token contexts already cover most knowledge work, making further expansion increasingly marginal. Investment would be better directed toward retrieval-augmented generation—which has a model search an external repository for relevant passages before generating an answer—and more reliable memory mechanisms. OpenAI and Google continue adjusting the context limits of their flagship models, further indicating that window size is no longer a standalone selling point.
But plateauing at 260,000 to 400,000 tokens may also mean vendors have not found a solution that is both cost-effective and fast enough. Another risk is that, as windows expand, users are more likely to mistake “the model has read it” for “the model understands and remembers it.” Long context may improve task completion rates, but it may also amplify the risk that critical information gets overlooked.
Impact on regular people
For enterprise IT: When evaluating models, teams should look beyond the advertised window size. They should test information-locating accuracy across long documents, the cost of processing each request, and response times. Retrieval, access controls, and auditing remain essential when models are integrated into existing systems.
For individual professionals: Uploading an entire report or chat log does not guarantee a reliable conclusion. Important evidence should still be highlighted separately. Long-context capabilities are more likely to become one component of a workflow than a replacement for process judgment.
For the consumer market: Features advertised as supporting “1 million tokens” will become increasingly common, but product differentiation will hinge more on the reliability of retrieval, memory, and citations. If vendors continue promoting window size alone, marketing will keep running ahead of the actual user experience.