Chain-of-Thought Prompting Explained

The mechanism

Language models generate output token by token, and each new token can attend to everything written before it — including the model's own prior reasoning. Asking for step-by-step reasoning gives the model intermediate 'scratch space' to work through a problem instead of jumping straight to an answer.

Where it helps most

Multi-step arithmetic, logic puzzles, and multi-constraint planning tasks show the largest gains from chain-of-thought prompting, because these are exactly the tasks where skipping intermediate steps causes errors to compound.

The tradeoff

Longer reasoning traces cost more tokens and latency, so production systems often ask for reasoning internally while returning only a clean final answer to the user.