What Sudoku Reveals About AI Reasoning Architectures and the Future
A recent comparison of AI models in solving Sudoku puzzles highlights significant differences in reasoning capabilities. Kona, a new model, achieved a 96.2% success rate in solving puzzles quickly, while popular LLMs struggled with a mere 2% success rate. The findings suggest that the architecture of LLMs is not well-suited for spatial reasoning tasks like Sudoku due to their sequential token generation process.
- ▪Kona solved 96.2% of Sudoku puzzles at an average time of 313 milliseconds.
- ▪Frontier LLMs, including GPT-5.2 and Claude Opus 4.5, had a solve rate of only 2%, taking up to 90 seconds.
- ▪The LLMs' architecture limits their ability to handle constraint satisfaction tasks effectively.
Opening excerpt (first ~120 words) tap to expand
Our Sudoku demo is a simple way to see the difference yourself.We have Kona solve hard sudoku puzzles in real-time alongside a group of popular LLMs, including GPT-5.2, Claude Opus 4.5 and Sonnet 4.5, Gemini 3 Pro, and DeepSeek V3.2. After about a week of public access, Kona solved 96.2% of puzzles at an average of 313 milliseconds. Frontier LLMs together had a solve rate of only 2%, taking up to 90 seconds before an incorrect answer or time out.The reason LLMs struggle with sudoku is not that the rules are especially complicated; A sudoku cell is 81 cells arranged in a 9x9 grid, where each row, column, and 3x3 box must contain the digits 1 through 9 exactly once.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Logicalintelligence.
Discussion
0 commentsMore from Logicalintelligence
