Back to home

Compare

Comparing: Context Windows Plateau at the Million-Token Scale & 大模型上下文停在百万级 — 长文本扩张正在让位给可靠性

AEN
LocalLLaMAOpenAIGoogle Gemini·

Context Windows Plateau at the Million-Token Scale

In less than two years, the context capacity of mainstream large language models has jumped from roughly 8,000 tokens to 1 million. Yet across the last two rounds of flagship releases, most have stopped at 260,000 to 400,000 tokens. The numbers are still rising, but the focus of improvement has shifted from “how much can fit” to “can the model complete the task accurately.”

What this is

Context length refers to the total amount of text a model can read in one pass. A token is the basic unit into which a model divides text, and context windows are usually measured in tokens. The longer the context, the more material a model can theoretically process at once, including long contracts, codebases, and multi-session records. The tradeoff is that more input also means longer inference times and higher costs.

This is not an argument that the industry has stopped expanding context windows. It points to a subtler change: as models become more capable, simply enlarging the window cannot solve information omission, mislocated details, or inconsistencies across a passage. The scarce capability is now finding the relevant information within a massive input and using it correctly.

Industry view

One view holds that million-token contexts already cover most knowledge work, making further expansion increasingly marginal. Investment would be better directed toward retrieval-augmented generation—which has a model search an external repository for relevant passages before generating an answer—and more reliable memory mechanisms. OpenAI and Google continue adjusting the context limits of their flagship models, further indicating that window size is no longer a standalone selling point.

But plateauing at 260,000 to 400,000 tokens may also mean vendors have not found a solution that is both cost-effective and fast enough. Another risk is that, as windows expand, users are more likely to mistake “the model has read it” for “the model understands and remembers it.” Long context may improve task completion rates, but it may also amplify the risk that critical information gets overlooked.

Impact on regular people

For enterprise IT: When evaluating models, teams should look beyond the advertised window size. They should test information-locating accuracy across long documents, the cost of processing each request, and response times. Retrieval, access controls, and auditing remain essential when models are integrated into existing systems.

For individual professionals: Uploading an entire report or chat log does not guarantee a reliable conclusion. Important evidence should still be highlighted separately. Long-context capabilities are more likely to become one component of a workflow than a replacement for process judgment.

For the consumer market: Features advertised as supporting “1 million tokens” will become increasingly common, but product differentiation will hinge more on the reliability of retrieval, memory, and citations. If vendors continue promoting window size alone, marketing will keep running ahead of the actual user experience.

BZH
LocalLLaMAOpenAIGoogle Gemini·

大模型上下文停在百万级 — 长文本扩张正在让位给可靠性

不到两年,主流大模型的可处理上下文从约 8000 个 token 增至 100 万个,但近两轮旗舰模型多停留在 26 万至 40 万级。数字仍在扩张,增长重点却已从“装下多少”转向“能否准确完成任务”。

这是什么

上下文长度,指模型一次能够读取的文字总量;token 是模型拆分文本的基本单位,上下文窗口通常以 token 计量。上下文越长,模型理论上越能同时处理长合同、代码库和多轮记录,但输入越多,推理时间与成本也越高。

这次讨论并非断言行业已经停止扩容,而是指出一个更微妙的变化:模型能力提高后,单纯增加窗口无法解决信息遗漏、定位错误和前后不一致。真正稀缺的能力,正变成在海量输入中找回关键信息并正确使用。

行业怎么看

一种看法认为,百万级上下文已经覆盖多数知识工作,继续扩大窗口的边际收益有限;相比之下,检索增强生成(让模型先从外部资料库查找相关片段,再生成答案)和更可靠的记忆机制更值得投入。OpenAI、Google 等公司持续调整旗舰模型的上下文上限,也说明窗口大小已不再是独立卖点。

但把上限停在 26 万至 40 万级,也可能意味着厂商尚未找到成本可控、速度可接受的方案。另一个风险是:窗口越大,用户越容易把“模型读过”误认为“模型理解并记住”。长上下文可能提高完成率,也可能放大关键信息被忽略的概率。

对普通人的影响

对企业 IT:评估模型时不应只看标称窗口,还要测试长文档中的信息定位准确率、单次处理成本和响应时间。模型接入现有系统时,检索、权限与审计仍不能省。

对个人职场:上传整份报告或聊天记录并不等于获得可靠结论,重要依据仍应单独标出。长文本能力更可能成为工作流的一部分,而不是替代流程判断。

对消费市场:标称“支持百万 token”的功能会越来越多,但产品差异将更多体现在检索、记忆和引用是否可靠。厂商若只宣传窗口大小,营销会继续快于实际体验。