30 hours. That's how long Qwen3.8 (the latest open-source release from Alibaba's Tongyi Qianwen) needed to finish all questions in a Reddit benchmark — and it still failed on 16 cases due to a 32K output token limit. On the same task, the underappreciated open-source model Muse Glimmer took only 3–4 hours and actually scored higher than Qwen3.8 on "implicit knowledge."
This comparison is worth noting: the intuition that "bigger models are stronger" is being broken by architectural design choices.
What this is
The test was posted to the community by r/LocalLLaMA user Ok-Inevitable8391, covering Qwen3.8 (in both high- and medium-effort modes), Muse Glimmer, plus the Claude series (data drawn from citations in the public repository embedeval).
Qwen3.8's high-effort mode took nearly 30 hours total and ultimately failed 16 questions due to the token cap; medium-effort mode and Muse Glimmer each finished in just 3–4 hours, and Muse outscored Qwen3.8 on implicit knowledge tests — the kind of facts the model has to "figure out from training data on its own."
The author flags Muse's core advantage: its sliding window attention mechanism (each layer attends only to the most recent tokens, not the full history), which delivers KV cache efficiency gains (less VRAM consumed by caching) and lets the model pack in longer context windows.
Industry view
One counterargument is blunt: implicit knowledge inherently favors smaller models — it tests what the model "remembers," not what it can "retrieve." Pair Qwen3.8 with RAG (retrieval-augmented generation — let the model look things up before answering) to fill the context, and the result would likely swing back the other way.
But that's exactly why this is worth thinking through: when a task can be solved with RAG or longer context, "packing all knowledge into the model's head" isn't a cost-effective design — VRAM, inference cost, and deployment thresholds all have to be paid for by big parameters. We believe architecturally smarter models are likelier to reach production than those that simply stack parameters.
Another piece of context that's easy to miss: Muse Glimmer is a severely underappreciated open-source option, and Chinese teams working on model compression and efficient architectures aren't a one-shop story — they're quietly rewriting what counts as SOTA.
Impact on regular people
For enterprise IT: Local deployment is becoming more feasible. One machine running a model that holds its own against mainstream options means data can stay inside the company, off the cloud, and off token-based billing — a real option for compliance-constrained industries.
For individual professionals: ChatGPT won't be replaced overnight. But if you need long-document analysis or long codebase comprehension, the "longer context" that sliding window architecture unlocks will turn into the new baseline — worth tracking when picking tools.
For consumer markets: Falling inference costs will eventually flow through to AI product pricing — the growing wave of "unlimited, no price hike" features is being powered by this architectural dividend, not by big-model companies being generous.