When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
The paper explores the dynamics of reasoning in large language models (LLMs) through the lens of entropy phase transitions. It reveals that reasoning is not a fixed attribute but a dynamic state that can be influenced by early-stage entropy dynamics. The authors propose a new framework, EDRM, which adapts inference strategies based on entropy, leading to improved efficiency and accuracy in LLM performance.
- ▪Chain-of-thought reasoning often yields marginal or negative gains on tasks while increasing token consumption.
- ▪The study introduces EDRM, a framework that uses early decoding entropy to select inference strategies adaptively.
- ▪EDRM demonstrates significant improvements in accuracy and token savings across multiple benchmarks and LLMs.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.22873 (cs) [Submitted on 20 May 2026] Title:When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions Authors:Wei Xia, Haoqing Wang, Zhi-Hong Deng, Yehui Tang View a PDF of the paper titled When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions, by Wei Xia and 3 other authors View PDF HTML (experimental) Abstract:Chain-of-thought (CoT) reasoning has become the default strategy for enhancing LLM capabilities, yet its application raises a fundamental question: when is explicit reasoning actually beneficial? Empirical evidence reveals a striking paradox: CoT often provides marginal or even negative gains on factual and open-ended tasks while multiplying token consumption.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
