The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models
A recent study explores the effectiveness of chain-of-thought (CoT) prompting in small language models for arithmetic tasks. The research identifies a positional shortcut where models tend to copy the last number before the answer delimiter, significantly impacting accuracy. This finding raises questions about the genuine computational capabilities of these models when using CoT prompting.
- ▪The study focuses on small language models with 1-3 billion parameters.
- ▪It finds that models often copy the last number before the answer delimiter, affecting their performance.
- ▪Accuracy can drop to near-zero if the trailing number is incorrect, despite correct intermediate reasoning.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.22870 (cs) [Submitted on 20 May 2026] Title:The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models Authors:Ming Liu View a PDF of the paper titled The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models, by Ming Liu View PDF HTML (experimental) Abstract:Chain-of-thought (CoT) prompting is necessary for arithmetic in small language models, yet shuffling its steps preserves most performance.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
