What This Is
An overseas user ran three models from Alibaba's Tongyi Qianwen (Qwen) 3.8 series—the Flash, Swift, and a quantized 27B variant—locally on an RTX 5060 Ti 16GB GPU. The test set: 14 programming problems pulled from AtCoder over the past 30 days, verified free of training-data contamination. Four thinking-intensity tiers were compared: off / low / medium / ultra-high. Total runtime: ~11 hours; per-problem thinking budget capped at 32K tokens.
Industry View
The findings are blunt: across the low, medium, and high thinking tiers, the three models differed by no more than ±2 problems solved. Pushing thinking intensity to its maximum cost 33% to 57% more time—the extra tokens were spent almost entirely on problems the model was going to fail anyway, i.e., hitting the budget ceiling and still producing no code. The only tier that consistently underperformed was "thinking off": with reasoning disabled, the models either refused to write code (DNF) or produced incorrect output (ERR).
The pushback is fair: 14 problems is a small sample, and statistical noise is real; consumer-grade hardware may diverge from enterprise GPU clusters running longer thinking budgets. But the finding aligns with internal-benchmark observations from Anthropic and Google over recent months—reasoning models show diminishing marginal returns, and "thinking" behaves more like a switch than a stepless throttle.
Impact on Regular People
For enterprise IT: When we procure APIs carrying "deep thinking / reasoning" modes, we should require vendors to run head-to-head comparisons on our own business scenarios rather than defaulting to the highest tier. The extra token spend does not necessarily translate to proportional accuracy gains.
For individual professionals: When we use ChatGPT, Tongyi, Wenxin, or similar tools to solve problems or draft plans, we shouldn't fetishize the "deep thinking" button. When time is tight, turning it off is often better than waiting—the latency stretches significantly while the output rarely improves.
For the consumer market: The "premium reasoning" bundled into subscription tiers is being challenged by tests like this one. We expect consumer subscription tiers to diverge more sharply over the next year—base plans with reasoning off, Pro plans keeping it on—with pricing becoming more transparent than today's opaque bundles.