A mathematical theory of balancing relational generalization and memorization
A new mathematical theory explores how learning systems balance relational generalization and memorization of exceptions. The authors introduce a task called transitive inference with exceptions to investigate this balance. Their findings suggest that while models can achieve this balance, successful generalization is sensitive to the representational geometry used.
- ▪The study addresses the challenge of balancing general rules and exceptions in learning systems.
- ▪A novel task, transitive inference with exceptions, is introduced to test relational generalization.
- ▪The research finds that neural network models can balance generalization and memorization, but with specific representational constraints.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.22972 (cs) [Submitted on 21 May 2026] Title:A mathematical theory of balancing relational generalization and memorization Authors:Luke Cheng, Samuel Lippl View a PDF of the paper titled A mathematical theory of balancing relational generalization and memorization, by Luke Cheng and 1 other authors View PDF HTML (experimental) Abstract:Humans, animals, and modern machine learning models exhibit impressive abilities to learn complex behaviors and generalize these behaviors to unseen situations. This ability requires us to learn rules and regularities that allow for such generalizations. At the same time, in most complex environments, any rule will have its exceptions.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
