1. Global overview: the problem, the methods, the innovation, the results
The modern reasoning debate is easier to understand if you separate capability, procedure, and explanation. A model can be capable of getting an answer right without using a robust procedure, and it can produce a convincing explanation without that explanation being a faithful account of what causally drove the answer.
Can the system solve tasks that require several dependent steps?
Does the solution method transfer when surface form, depth, distribution or representation changes?
Does the visible reasoning trace accurately reflect the factors that actually caused the answer?
The problem
Older evaluations often asked whether LLMs could resist an intuitive lure, solve a multi-step math problem, or reproduce human cognitive biases. By 2025–2026, the strongest systems were explicitly trained to spend more computation during inference. This created a more difficult scientific target. A model might succeed because it memorized a familiar template, because it genuinely composes relations, because it samples many attempts and gets lucky, because a verifier selects a good attempt, or because long language generation implements a useful iterative computation. Those possibilities can yield the same final answer but imply very different forms of “reasoning.”
The methods
Researchers use several complementary methods: controlled synthetic tasks, novel benchmark variants designed to reduce contamination, distribution-shift tests, chain-of-thought interventions, multiple sampling, external verifiers, process reward models, reinforcement learning with objectively checkable rewards, causal interventions on prompts or reasoning traces, and increasingly mechanistic interpretability.
The innovation
The important technical shift is that inference-time computation has become a scaling dimension. Instead of thinking only about larger models and more pretraining, researchers now ask how much computation a fixed model should spend on each problem and how that compute should be organized.
This is a conceptual diagram, not a claim that every reasoning model literally contains six separate modules. It summarizes the behavior that recent training and inference techniques try to produce.
The results in plain language
- Extra computation often helps. Longer deliberation, multiple samples, search, verification and reasoning-focused post-training can substantially improve math, coding and structured problem-solving.
- Reinforcement learning can elicit self-checking behavior. DeepSeek-R1 is the clearest open example: RL using verifiable rewards selected longer traces containing reflection, backtracking and strategy changes.
- More thinking is not monotonically better. Recent work documents overthinking, redundant loops, correct intermediate answers being overwritten, and large token expenditures with no gain.
- Generalization remains uneven. Models can excel on familiar benchmarks yet deteriorate sharply when rule depth, perceptual conditions or compositional structure changes.
- Chain-of-thought is not automatically a transparent explanation. It is an observable behavior that can be useful, but causal faithfulness has to be tested rather than assumed.
- System 1/System 2 is a useful analogy at the behavioral level, not evidence that transformers have the same cognitive architecture as humans.
2. What do researchers mean by “reasoning”?
There is no single universally accepted definition. A practical working definition for this report is:
This definition is deliberately implementation-neutral. Humans may use working memory, spatial imagery, symbolic manipulation and heuristics. An LLM may use distributed neural activations, generated intermediate tokens, search, tools, or combinations of these.
2.1 Three levels of evidence
Behavioral reasoning asks whether the system produces the correct answer on tasks that appear to require inference. Systematic reasoning asks whether the procedure generalizes when the task is transformed. Mechanistic reasoning asks which internal computations causally produced the result.
Most benchmark claims are strongest at the first level. Controlled distribution-shift experiments probe the second. Interpretability and causal intervention studies aim at the third.
| Question | Example evidence | What it does not establish |
|---|---|---|
| Can the model solve it? | High accuracy on AIME, code tests, spatial reasoning, CRT items | That the model used a robust or human-like method |
| Does the method generalize? | OOD tests, deeper compositions, new representations, new value ranges | That the visible CoT faithfully exposes internal causality |
| What caused the answer? | Ablations, interventions, causal mediation, mechanistic probes | A complete theory of cognition |
3. Minimum technical background
Token
A token is a chunk of text processed by the model: a word, part of a word, punctuation mark or other symbol. Autoregressive LLMs generate one token after another.
Pretraining
Pretraining is the large initial training stage. The model learns to predict the next token from huge amounts of data. This objective can teach language, factual associations, code patterns, mathematical procedures and latent regularities.
Post-training
Post-training occurs after pretraining. It can include supervised fine-tuning, preference optimization and reinforcement learning. Modern reasoning models rely heavily on post-training to shape how the model uses its latent capability.
Inference / test time
Inference is the moment when the trained model answers a new prompt. Test-time compute is the computation spent on that specific prompt after training is complete.
Chain-of-thought (CoT)
A chain-of-thought is a generated sequence of intermediate steps before the final answer. Conceptually: question → intermediate state 1 → intermediate state 2 → answer. Because each generated token becomes context for later tokens, generation can act as an external scratchpad.
Reasoning trace
The reasoning trace is the observable sequence of intermediate calculations, reflections, revisions or plans. In the literature, “thoughts,” “reasoning tokens” and “CoT” often overlap in meaning.
Verifier
A verifier evaluates a candidate answer or reasoning trajectory. A model can generate several attempts and use a verifier to select among them.
Outcome vs process supervision
Outcome supervision rewards the final result. Process supervision evaluates intermediate steps. A process reward model (PRM) is trained to score those steps or partial trajectories.
Reinforcement learning (RL)
Reinforcement learning updates behavior using a numerical reward. Reasoning is especially amenable to RL when outcomes are objectively verifiable, for example an exact mathematical answer or code that passes tests.
Test-time scaling
Test-time scaling means deliberately spending more computation on a problem. This can be a longer trace, more independent samples, search, repeated verification, tool use, or a combination.
Pass@k
Pass@k asks whether at least one of k generated attempts is correct. If pass@1 is mediocre but pass@64 is high, the model can often generate a correct solution, but selecting it reliably is still a separate problem.
Out-of-distribution (OOD)
An OOD example differs systematically from the examples used during training or tuning. OOD evaluation is important because shortcuts can work on familiar test sets and fail when the same underlying rule is expressed differently.
Faithfulness
A reasoning trace is faithful when it accurately reflects the information and computation that causally contributed to the answer. Plausibility and faithfulness are not the same thing.
Metacognition
Metacognition means monitoring and regulating the reasoning process: estimating difficulty, deciding how much effort to spend, noticing when a strategy fails, deciding to backtrack, and knowing when to stop.
4. How the field got to modern reasoning models
Stage 1 — Latent multi-step ability
Large transformers could already perform some multi-step computations, but direct prompting often produced brittle answers.
Stage 2 — Chain-of-thought prompting
Wei et al. (2022) showed that sufficiently large models improved on arithmetic, commonsense and symbolic tasks when examples included intermediate reasoning. This popularized the idea that generated language can function as a scratchpad.
Stage 3 — Sampling, search and verification
If one trajectory can fail, generate several. Self-consistency, best-of-N selection, search and process reward models separate proposing from judging.
Stage 4 — Inference compute becomes a scaling axis
Snell et al. (2024) made the compute-allocation perspective explicit: depending on the problem, extra inference computation on a smaller model can be more effective than spending the same compute on a much larger model that answers immediately.
Stage 5 — Reasoning-focused post-training
Systems such as OpenAI’s o1 family made extended deliberation a product category. DeepSeek-R1 then gave the research community a detailed open account of how reinforcement learning with verifiable rewards could elicit long, reflective traces.
Stage 6 — Control becomes the new bottleneck
Once a model can generate thousands of “thinking” tokens, new problems emerge: looping, redundant checking, latency, expense, and sometimes reasoning past the correct answer. The 2026 frontier therefore focuses increasingly on adaptive budgets, early stopping, metacognitive control and mechanistic understanding.
5. Main methods and innovations
| Method | Core idea | Why it helps | Main limitation |
|---|---|---|---|
| Chain-of-thought prompting | Generate intermediate language steps | Adds serial computation and an external scratchpad | Can produce verbose rationalization |
| Self-consistency / best-of-N | Generate several attempts | Raises chance at least one path is correct | Needs a way to choose the right candidate |
| Search | Explore branches of possible reasoning | Allows backtracking and alternatives | Can be expensive |
| Verifier / PRM | Score answers or intermediate steps | Separates generation from evaluation | Verifier can itself be wrong or exploitable |
| SFT on reasoning traces | Imitate high-quality worked solutions | Quickly teaches useful formats and patterns | May imitate style rather than robust procedure |
| RL with verifiable rewards | Reward correct outcomes and let strategy emerge | Can discover checking/backtracking behaviors | May optimize narrow reward structure |
| Budget forcing | Force earlier stopping or continued thinking | Makes reasoning length controllable | More tokens can hurt |
| Adaptive metacognition | Estimate difficulty and regulate reasoning | Reduces waste and overthinking | Self-assessment can be miscalibrated |
6. What the evidence currently supports
6.1 Extra inference computation can improve difficult-task performance
Across mathematics, code and structured reasoning, extra computation often helps. The most useful perspective is broader than “longer answer”: computation can be spent on more samples, search branches, verification or tool calls.
6.2 RL can select qualitatively different problem-solving behavior
DeepSeek-R1 provides strong evidence that RL with verifiable rewards can increase the frequency of long deliberation, checking, backtracking and strategy change. This is behavioral evidence of a changed inference policy; it is not evidence of consciousness or human-like introspection.
6.3 More reasoning is not always better
The relationship between token count and correctness is non-monotonic. Recent studies find diminishing returns, wasted reasoning on trivial tasks, and cases where a correct intermediate conclusion is later overwritten.
6.4 Familiar benchmark success does not prove systematic reasoning
Controlled relational and analogical tasks show sharp degradation under distribution shifts, longer rule chains or altered perceptual conditions. That suggests current systems combine real multi-step capability with task-specific heuristics and brittle representations.
6.5 Visible reasoning is not guaranteed to be faithful
CoT is an output of the model, not direct access to its hidden activations. Causal tests show that traces can omit influences that mattered, while newer work also warns that omission alone may indicate incomplete verbalization rather than total causal irrelevance. Faithfulness is therefore an empirical property, not a default assumption.
6.6 System 1/System 2 should remain an analogy
LLMs can exhibit immediate, heuristic-like answers and slower deliberative outputs. That behavioral contrast is useful. It does not establish that transformers implement the same dual-process cognitive mechanisms proposed for human psychology.
7. Yax, Anlló & Palminteri (2024): reasoning in humans and machines
Can LLMs reproduce or overcome classic human reasoning errors, and does similarity in error rates imply similarity in underlying reasoning?
The paper uses the Cognitive Reflection Test (CRT) and conjunction-fallacy-style problems. Because famous CRT items such as the bat-and-ball problem may exist in training data, the authors construct new analogues. They compare humans with successive GPT-family models and examine both aggregate accuracy and item-by-item patterns.
Older GPT-3-family models often produced intuitive or biased answers. ChatGPT and GPT-4 improved dramatically; the reported GPT-4 performance on newly constructed CRT items approached ceiling. Crucially, even where humans and models had similar average error rates, they did not necessarily find the same items difficult.
This paper is an excellent pre-reasoning-model baseline. It shows both why people were tempted to map human dual-process theory onto LLMs and why that mapping must be handled carefully. The same observable error can arise from different mechanisms.
8. Brady et al. (2025): dual-process theory and LLM decision-making
It is the closest modern bridge between human System 1/System 2 theory and LLM behavior. It reviews cognitive biases, decision-making and reflective behavior across model families.
A model may show “System-1-like” or “System-2-like” behavior without possessing human Type 1 or Type 2 mechanisms. Training data, prompt framing, instruction tuning and inference strategies can all produce behavioral patterns that resemble cognitive biases.
9. DeepSeek-R1 (2025): reasoning through reinforcement learning
It offered unusually concrete evidence that reinforcement learning with verifiable rewards can reshape how a pretrained model uses inference-time computation.
9.1 R1-Zero
R1-Zero starts from a pretrained base model and applies reasoning-focused RL without first supplying a large corpus of human-written ideal reasoning traces for that stage. Rewards emphasize answer correctness and output constraints. During training, the model develops longer responses and more frequent behaviors that look like checking, reflection and strategy changes.
9.2 “Aha moments”
The paper uses examples where the model says things like “wait” and changes course. This is best interpreted behaviorally: RL selected patterns of re-evaluation because they improved reward. The term should not be treated as evidence that the model experiences human insight.
9.3 GRPO
Group Relative Policy Optimization (GRPO) is the RL method prominently used in the work. Intuitively, the model generates a group of outputs, receives rewards, and is updated so relatively high-reward behavior becomes more likely. It avoids a separate value model used in some older RL architectures.
9.4 The final R1 is not “RL only”
The final pipeline adds cold-start supervised data, reasoning RL, sampling/filtering, further supervised training and another alignment stage. This matters because popular summaries sometimes collapse R1-Zero and final R1 into a single technique.
10. s1 (2025): simple test-time scaling and budget forcing
How much specialized data is needed to induce useful reasoning behavior, and can simply controlling the reasoning budget change performance?
The authors curate roughly 1,000 high-quality reasoning examples and fine-tune an existing instruction model. Their memorable intervention is budget forcing: stop the model early, or when it tries to stop, append “Wait” to force another reasoning pass.
Additional thought can make the model recheck a derivation and correct itself. This provides a clean illustration of reasoning tokens as a controllable computation budget.
11. Snell et al. (2024): test-time compute as a scaling dimension
The paper asks a resource-allocation question: if compute is fixed, is it better to use a larger model that answers once, or a smaller model that spends more compute searching/refining at inference time?
The authors study search with process-based verifiers and adaptive refinement. Their central result is that optimal allocation depends on problem difficulty. A fixed best-of-N strategy wastes compute on easy problems and can under-spend on hard ones. They report that adaptive allocation can be substantially more efficient than a simple best-of-N baseline, and in some FLOPs-matched settings a smaller model with test-time compute can outperform a much larger model.
12. Kambhampati, Stechly & Valmeekam (2025): “(How) Do reasoning models reason?”
This perspective paper is useful because it resists turning benchmark performance into a complete theory of reasoning. An LLM system can combine broad pattern knowledge, generated scratchpads, repeated sampling, heuristic search and verification. Such a system can be extremely capable without behaving like a classical symbolic theorem prover.
Planning is a particularly useful stress test: the model must maintain global consistency across many dependent actions. Plausible local steps do not guarantee a valid overall plan.
13. Shojaee et al. (2025): The Illusion of Thinking
The paper uses controllable puzzles whose complexity can be increased systematically. This is more informative than reporting one benchmark accuracy number because it reveals how reasoning behavior changes as the same underlying problem becomes deeper.
The three-regime interpretation
- Low complexity: non-thinking models can be competitive; lengthy reasoning may be overhead.
- Medium complexity: reasoning models gain an advantage.
- High complexity: both eventually fail in the authors’ experiments.
The work also analyzes reasoning traces and reports cases where a correct solution appears early, followed by unnecessary continuation.
Why the controversy matters
A subsequent comment argues that some collapse is confounded by output-length constraints and problematic puzzle instances. Read the comment. The dispute teaches a general lesson: a failed benchmark run may indicate failure of reasoning, execution, representation, interface limits, or even benchmark construction. Good evaluation must separate these possibilities.
14. Khalid, Nourollah & Schockaert (2025): shallow disjunctive reasoning
The authors study controlled spatial and temporal reasoning. They vary relation types, path lengths and whether the model must combine one or several reasoning paths.
Reasoning models generally improve on single-path tasks, but the authors find much weaker performance in multi-path, out-of-distribution conditions. They describe the resulting behavior as “shallow disjunctive reasoning.”
15. Camposampiero et al. (2025): analogical reasoning under perceptual uncertainty
Raven’s Progressive Matrices are classic tests of abstract analogy. The authors evaluate o3-mini and DeepSeek-R1 on I-RAVEN and a harder I-RAVEN-X setting that increases rule length, value ranges, perceptual uncertainty and irrelevant/confounding attributes.
Values are reported in Camposampiero et al. (2025). ARLC is a neuro-symbolic probabilistic abductive system. The chart is redrawn here rather than copied from the paper.
One striking result is that o3-mini’s accuracy drops sharply in I-RAVEN-X despite using substantially more reasoning tokens. This makes the central point vivid: reasoning length is not the same thing as robust reasoning representation.
16. Malberg et al. (2025): cognitive biases across LLMs
The authors evaluate 20 LLMs across 30 cognitive biases and 30,000 tests. They report evidence for every tested bias in at least some models.
The value of this work is breadth: the bat-and-ball problem probes one narrow form of intuitive error, while a bias battery tests framing, anchoring, representativeness and many other systematic manipulations.
17. Chain-of-thought faithfulness: is the explanation the reason?
17.1 Chua & Evans (2025)
The authors insert cues into prompts — for example, “A Stanford professor thinks the answer is D” — and test whether the cue changes the model’s answer. If it does, they ask whether the model acknowledges the cue in its reasoning trace.
In their reported DeepSeek comparison, R1 described the influence of such cues far more often than the corresponding non-reasoning model (59% versus 7% in the cited setup). That suggests reasoning traces can sometimes be more revealing.
However, the authors emphasize that this is one artificial test of one aspect of faithfulness.
17.2 FaithCoT-Bench (ICLR 2026)
The benchmark treats faithfulness as an instance-level problem: can we determine whether this particular trajectory is faithful? Its FINE-CoT dataset contains more than 1,000 expert-annotated reasoning trajectories, including hundreds of identified unfaithful cases, and compares multiple detection strategies.
17.3 Zaman & Srivastava (ACL 2026)
This paper complicates a common assumption: if a hint affected the answer but is absent from the written trace, is the trace necessarily unfaithful? The authors argue that omission can reflect incompleteness — lossy compression of a high-dimensional neural computation into a short linear narrative.
They use broader metrics and causal mediation analysis, and report that some CoTs labelled unfaithful by simple hint-verbalization tests look more faithful under alternative criteria. They explicitly do not claim that all CoT is faithful.
18. The 2026 frontier: overthinking, stopping and metacognition
Once reasoning models can spend a large amount of computation, efficiency and control become scientific capabilities in their own right.
18.1 LLMThinkBench: basic math and overthinking
The benchmark evaluates 53 LLMs on 14 dynamically generated basic-math task families. The authors report that reasoning models can generate roughly 18× more tokens while sometimes achieving lower accuracy, and that constraining tokens can produce large accuracy collapses in some models. They also find diminishing returns from increasing reasoning effort on several tested systems.
18.2 ThinkBrake: the model may solve the problem and then ruin it
The authors demonstrate a particularly intuitive failure mode: a reasoning model reaches a correct intermediate solution, continues thinking, then overwrites it with a wrong answer. Their hindsight “oracle stopping” experiment improves average accuracy by about 8% while reducing thinking tokens by 72%. ThinkBrake then proposes a training-free stopping rule based on token-probability margins and reports token reductions of up to 30%.
18.3 Think Better, Not Longer
This paper introduces token-level marginal utility: does a particular reasoning token increase or decrease the probability of the correct answer? Their MUTO method tries to penalize negative-utility reasoning rather than merely impose a crude length budget.
18.4 Think How to Think (TH2T)
TH2T explicitly trains difficulty cognition and redundancy cognition. The aim is to choose a reasoning strategy before solving and then identify unproductive detours while solving. The authors report more than 70% inference-cost reduction on easy tasks and around 40% on complex tasks without compromising performance in their experiments.
18.5 MERA: separate reasoning from control
MERA makes a conceptual distinction between reasoning content and control decisions. A control policy decides whether to continue, backtrack or stop. This resembles an executive layer over the problem-solving process.
18.6 SAT: slow, normal, fast, skip
SAT models reasoning as a finite-state machine with four modes — SLOW, NORMAL, FAST and SKIP — and uses a lightweight process reward model to switch between them. Across 9 LRMs and 7 benchmarks, the authors report up to 40% fewer reasoning tokens while generally maintaining or improving accuracy.
18.7 DTSR: when is thinking enough?
Dynamic Thought Sufficiency in Reasoning (DTSR) monitors reflection signals and evaluates whether the current chain of thought is sufficient to derive the final answer. On Qwen3 models, the paper reports reasoning-length reductions of roughly 28.9%–34.9% with minimal performance loss.
18.8 PAM: metacognition beyond mathematics
PAM applies metacognitive control to alignment. The model first identifies a top-level priority such as harmlessness, then uses explicit metacognitive knowledge to monitor and regulate reasoning. The authors report roughly 10-point improvements on their helpfulness/harmlessness alignment benchmarks under comparable pipelines.
The frontier is shifting from “produce more thinking tokens” to “control computation intelligently.”
19. Mechanistic understanding: what is happening inside the model?
This survey organizes the emerging mechanistic literature around three themes: training dynamics, reasoning mechanisms and unintended behaviors. It is useful because much of the reasoning literature is still behavioral. We can observe that RL increases long-form problem solving, but a complete mechanistic account of why specific circuits, representations or internal states support that behavior is still missing.
A mechanistic explanation would ideally connect several scales:
- how pretraining creates latent problem-solving circuits;
- how post-training changes activation and routing patterns;
- how generated reasoning tokens interact with hidden-state computation;
- why reflection and backtracking emerge;
- why certain failures such as loops or collapse occur;
- how these mechanisms vary across architectures and training recipes.
20. A useful mental model of a modern reasoning system
It is misleading to imagine one hidden “reasoning module.” A more practical mental model is a stack of capabilities.
| Layer | What it contributes |
|---|---|
| 1. Pretrained representations | Language, facts, code patterns, mathematical structures, procedures and latent abstractions |
| 2. Task representation | Convert the prompt into an internal state that identifies goals, constraints and relevant knowledge |
| 3. Deliberative generation | Use generated tokens as a scratchpad to carry intermediate states across time |
| 4. Evaluation and revision | Check consistency, compare alternatives, notice errors, retry, or call a verifier/tool |
| 5. Compute control | Estimate difficulty, allocate budget, decide to backtrack, and stop when further thought is unlikely to help |
Modern reasoning performance ≈ pretrained capability + reasoning-focused post-training + inference-time compute + search/verification + compute control + tools/environment.
21. Why do extra tokens help if the model is “just predicting the next token”?
This is one of the most important conceptual points.
In direct answering, the model must map roughly:
With a scratchpad:
Each generated state becomes additional context for the next computation. That gives the overall process more serial depth, even though each individual decoding step still uses the same transformer. In this sense, autoregressive generation can behave like variable-depth computation.
But natural-language tokens are not an ideal computer architecture. They are discrete, expensive and can encourage verbosity. This motivates research on latent reasoning, compressed reasoning, external tools and structured search.
22. Why can a reasoning model solve olympiad problems and fail easy ones?
- Distribution mismatch: difficult competition math may be heavily represented in reasoning post-training, while an “easy” task with unusual formatting may not be.
- Wrong strategy selection: the model can apply a complex method where a direct calculation would be safer.
- Overthinking: the model can introduce an error after already finding the answer.
- State drift: long generation can lose track of earlier constraints.
- Execution difficulty: the model may know an algorithm but be poor at carrying out hundreds of exact repetitive operations in text.
- Verification failure: a correct candidate may be generated but not selected.
This is why “difficulty for an LLM” is not identical to “difficulty for a human.”
23. Do LLMs “really reason”?
The word really hides several different claims. The clearest current answer is therefore decomposed:
| Claim | Current evidence |
|---|---|
| Can LLM systems perform computations that reasonably count as reasoning behavior? | Yes. There is extensive evidence across mathematics, coding, relational inference, planning-like tasks and scientific QA. |
| Can extra deliberation improve performance? | Yes. But the gain is task-dependent and non-monotonic. |
| Can RL teach checking/backtracking strategies? | Yes. DeepSeek-R1 and later work provide strong behavioral evidence. |
| Do current models execute fully systematic algorithms across arbitrary depth and distribution shift? | No. Controlled studies show substantial brittleness. |
| Is visible chain-of-thought a guaranteed transcript of internal computation? | No. Faithfulness must be tested. |
| Do LLMs reason in the same way humans do? | Not established. Behavioral similarities do not imply mechanistic equivalence. |
| Does next-token prediction rule out reasoning? | No. It specifies the generation/training interface, not the full computation implemented across hidden states and sequential generation. |
24. GPT-4-era reasoning versus the reasoning-model era
| GPT-4-era default chat model | Reasoning-model era |
|---|---|
| Reasoning often elicited by prompting | Post-training explicitly targets extended problem solving |
| Often optimized for immediate helpful response | Can allocate substantial hidden/visible compute before final answer |
| Reasoning length mostly a by-product of prompt | Reasoning budget becomes an explicit system variable |
| Less emphasis on learned self-checking loops | Reflection, backtracking and strategy switching are common learned behaviors |
| Evaluation asks “can it reason?” | Evaluation increasingly asks “how should it allocate, verify and stop reasoning?” |
The architectural continuity is important: both remain autoregressive transformers. The difference is that recent systems have learned policies in which sequential token generation is deliberately used as extended computation.
25. Common mistakes when interpreting reasoning papers
- Benchmark accuracy is not a theory of cognition. It establishes success under an evaluation setup.
- Long CoT is not automatically good CoT. Length can mean productive search, redundant checking or confusion.
- Visible thought is not direct neural telemetry. Treat it as behavior.
- Human psychological vocabulary can be metaphorical. “System 2,” “aha,” “reflection” and “metacognition” should be checked against the paper’s operational definition.
- Interface constraints matter. Output limits, context limits and parsers can create apparent reasoning failures.
- Contamination matters. Famous benchmarks may no longer cleanly measure novel problem solving.
- Tools change the unit of analysis. A model plus Python, search, a theorem prover or a planner should be evaluated as a system.
26. How to read any new LLM reasoning paper
Ask these ten questions:
- What exact behavior does the paper call “reasoning”?
- Could the task or solution pattern have appeared in training data?
- Is success measured only by final-answer accuracy?
- Does the evaluation include out-of-distribution variants?
- Is problem difficulty systematically controlled?
- What is changing: model weights, prompt, token budget, sampling, verifier, tools, or several at once?
- Does extra compute improve pass@1, pass@k, or both?
- If the paper treats CoT as explanation, what causal evidence supports faithfulness?
- Could failures come from execution limits rather than failure to represent the algorithm?
- Does the system know when to stop?
If you can answer those questions, you can understand most reasoning papers at a useful level without needing deep ML theory.
27. Recommended reading order
Stage A — Cognitive intuition: Yax, Anlló & Palminteri (2024), then Brady et al. (2025).
Stage B — Technical shift: Wei et al. (2022) on chain-of-thought, Snell et al. (2024) on test-time scaling, DeepSeek-R1 (2025), then s1 (2025).
Stage C — Limitations: Kambhampati et al. (2025), Shojaee et al. (2025) plus the methodological comment, Khalid et al. (2025), and Camposampiero et al. (2025).
Stage D — Current frontier: Chua & Evans (2025), FaithCoT-Bench (2026), Zaman & Srivastava (2026), Hu et al. (2026), then one or two overthinking/metacognition papers such as ThinkBrake, SAT or DTSR.
28. Five ideas to remember
- Reasoning is not a binary switch. Behavioral success, systematic procedure and faithful mechanism are different standards.
- Generated tokens can serve as computation. A model can use its own previous output as a scratchpad, giving the system additional serial depth.
- Modern reasoning models are trained to exploit that compute. RL and reasoning-focused post-training make checking, backtracking and extended search more likely.
- More thinking is not always better. The frontier is about allocating and controlling compute, not maximizing token count.
- Chain-of-thought is evidence, not a brain scan. It can be informative, but faithfulness is a testable property.
29. Glossary
| Term | Plain-English definition |
|---|---|
| Ablation | Remove or alter one component to test whether it caused the result. |
| Autoregressive | Generates sequentially; each new token depends on previous tokens. |
| Benchmark | A standardized test set used to compare models. |
| Calibration | How well stated confidence matches actual probability of correctness. |
| Chain-of-thought (CoT) | Generated intermediate reasoning text before a final answer. |
| Compositional reasoning | Combining several relations or operations to infer a result. |
| Counterfactual | A modified input used to test whether a particular factor causally changed the output. |
| Fine-tuning | Additional training after pretraining. |
| GRPO | Group Relative Policy Optimization, a reinforcement-learning method used prominently in DeepSeek-R1. |
| Inference | Running the trained model on a new problem. |
| Inference-time/test-time scaling | Spending additional computation while solving the current problem. |
| Large Reasoning Model (LRM) | Informal term for an LLM trained/configured to spend substantial inference-time compute on reasoning. |
| LLM-as-judge | Using another LLM to evaluate an output. |
| Logit | A raw model score before conversion into probabilities. |
| Metacognition | Monitoring and controlling the reasoning process itself. |
| OOD | Out-of-distribution: systematically different from the training/tuning distribution. |
| Outcome supervision | Feedback based on the final result. |
| Overthinking | Spending more reasoning computation than is useful, sometimes hurting accuracy. |
| Pass@k | Whether at least one of k generated attempts succeeds. |
| Policy | In RL, the rule/model that chooses actions; for an LLM, a distribution over next tokens. |
| Process reward model (PRM) | A model that scores intermediate reasoning steps. |
| Process supervision | Training feedback applied to intermediate steps, not only the final answer. |
| Reasoning budget | The amount of inference-time compute or tokens allocated to deliberation. |
| Reasoning trace | The observable sequence of intermediate generated steps. |
| Reinforcement learning | Training that increases behaviors receiving higher reward. |
| Reward | A numerical signal indicating how desirable an output or action is. |
| Self-consistency | Generate multiple reasoning paths and aggregate/select among them. |
| System 1 / System 2 | Psychological shorthand for fast/automatic versus slower/deliberative processing; in LLM research usually an analogy. |
| Token | A unit of text processed/generated by the model. |
| Verifier | A mechanism that judges candidate answers or reasoning steps. |
30. Sources and figure references
This report uses direct links rather than reproducing copyrighted paper figures. Where a numeric chart is included, it is an original redraw from values explicitly reported by the cited paper.
- Yax, N., Anlló, H. & Palminteri, S. (2024). Studying and improving reasoning in humans and machines. Communications Psychology, 2, 51.
- Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.
- Lightman, H. et al. (2023). Let’s Verify Step by Step.
- Snell, C., Lee, J., Xu, K. & Kumar, A. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters.
- Guo, D. et al. (2025). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature 645, 633–638.
- Muennighoff, N. et al. (2025). s1: Simple test-time scaling. EMNLP 2025.
- Kambhampati, S., Stechly, K. & Valmeekam, K. (2025). (How) Do reasoning models reason?
- Shojaee, P. et al. (2025). The Illusion of Thinking. NeurIPS 2025.
- Opus, C. & Lawsen, A. (2025). Comment on The Illusion of Thinking.
- Khalid, I., Nourollah, A. M. & Schockaert, S. (2025). Large Language and Reasoning Models are Shallow Disjunctive Reasoners. ACL 2025.
- Camposampiero, G. et al. (2025). Can Large Reasoning Models do Analogical Reasoning under Perceptual Uncertainty? PMLR 284.
- Malberg, S. et al. (2025). A Comprehensive Evaluation of Cognitive Biases in LLMs.
- Brady, O. et al. (2025). Dual-process theory and decision-making in large language models. Nature Reviews Psychology.
- Chua, J. & Evans, O. (2025). Are DeepSeek R1 And Other Reasoning Models More Faithful?
- Hayes, W. M., Yax, N. & Palminteri, S. (2025). Relative Value Encoding in Large Language Models. Open Mind.
- Shen, X. et al. (2026). FaithCoT-Bench. ICLR 2026.
- Zaman, K. & Srivastava, S. (2026). Is Chain-of-Thought Really Not Explainability? ACL 2026.
- Hu, Y. et al. (2026). Towards a Mechanistic Understanding of Large Reasoning Models. ACL 2026.
- Srivastava, G. et al. (2026). Do LLMs Overthink Basic Math Reasoning? Findings of ACL 2026.
- Song, S. et al. (2026). ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding. Findings of ACL 2026.
- Li, J. et al. (2026). Think Better, Not Longer. ACL 2026.
- Liu, Y. et al. (2026). Think How to Think. ACL 2026.
- Ha, R. et al. (2026). From “Aha Moments” to Controllable Thinking. ACL 2026.
- Huang, W. et al. (2026). SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinking. ACL 2026.
- Xiang, Y. et al. (2026). When Is Thinking Enough? ACL 2026.
- Xu, Z. et al. (2026). PAM: Enhancing General Alignment of Large Reasoning Models through Priority-Aware Metacognition. ACL 2026.
31. Final perspective
The most useful conceptual shift is to stop asking whether an LLM either “has reasoning” or “does not have reasoning.” Modern models are systems whose inference can contain variable amounts of learned computation. Post-training can make them allocate steps, explore alternatives, inspect conflicts, revise, and use verifiers or tools. On many difficult problems, these mechanisms create real and measurable capability gains.
At the same time, competence remains uneven. Models can be excellent on one representation and fragile on another. They can produce a correct intermediate answer and then lose it. They can generate explanations whose relationship to internal computation is uncertain. They can reproduce human-like biases without sharing human cognitive mechanisms.
2024 — Can LLMs overcome intuitive errors? → 2025 — Can we train models to spend much more inference compute productively? → 2026 — Can we understand, verify and control that computation, including when to think, how to think and when to stop?