Imperfect World Models are Exploitable
A new paper discusses the concept of model exploitation in reinforcement learning. The authors propose a definition that highlights the discrepancies between preferred policies in world models and actual environmental transitions. Their findings establish a connection between reward hacking and model exploitation, emphasizing the challenges of safe planning.
- ▪The paper introduces a novel definition of model exploitation in reinforcement learning.
- ▪It shows that exploitation is essentially unavoidable on large policy sets.
- ▪The authors develop a general theory that connects reward hacking and model exploitation.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.15960 (cs) [Submitted on 15 May 2026] Title:Imperfect World Models are Exploitable Authors:Logan Mondal Bhamidipaty (University of Edinburgh), Esmeralda S. Whitammer (University of Edinburgh), David Abel (University of Edinburgh), Mykel J. Kochenderfer (Stanford University), Subramanian Ramamoorthy (University of Edinburgh) View a PDF of the paper titled Imperfect World Models are Exploitable, by Logan Mondal Bhamidipaty (University of Edinburgh) and 4 other authors View PDF HTML (experimental) Abstract:We propose a novel definition of model exploitation in reinforcement learning.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
