In 2022, Google researchers discovered a counterintuitive phenomenon: appending "Let's think step by step" to a question caused large language models (LLMs) to take a significant leap in performance on math and logic problems. We've noted that this technique, named Chain-of-Thought (CoT), is the shared underlying capability that lets today's mainstream LLMs — ChatGPT, Claude, ERNIE Bot, and others — handle complex problems. In other words, the AI you see "thinking" today runs on this foundation.

What This Is

The core idea behind CoT is plain: instead of letting AI jump straight to an answer, force it to write out its reasoning step by step before reaching a conclusion — essentially "drafting" before answering. Before CoT, LLMs tackling multi-step reasoning often produced answers that looked plausible but had broken logic. CoT makes intermediate steps explicit, avoiding the "hallucinated" logical leaps. Today it's a standard capability across all major LLMs; only the implementation details vary by vendor.

Industry View

Supporters frame CoT as the pivotal shift that took LLMs from "parroting" to genuinely thinking — and as the infrastructure underpinning today's hot directions like AI Agents (autonomous assistants that can execute task sequences) and complex task planning.

But the skepticism is equally pointed. One critique — worth paying attention to — goes straight to the substance: CoT is just "longer pattern matching," not real reasoning. The model is stitching together more plausible-looking intermediate steps, and it still falls apart on problems outside its training data. A second, more practical risk: CoT significantly increases token (the smallest unit of text a model processes or generates) consumption, potentially multiplying inference costs by several times. Other research notes the technique requires model scale — LLMs with fewer than 1 billion parameters can barely use it.

Impact on Regular People

  • For enterprise IT: Calling an AI API with "reasoning enhancement" can cost 2–5x per call compared to the non-enhanced version, but accuracy gains are significant. Whether to enable it depends on task complexity.
  • For professionals: When using AI for data analysis or proposal writing, explicitly ask it to "walk through the reasoning step by step." Output quality is usually higher than asking directly for conclusions — but always manually verify the key numbers.
  • For consumer markets: AI assistant products still stuck at "just give me the answer" are almost certainly running older tech stacks, and the experience gap will only widen.