Process Rewards with Learned Reliability
The paper introduces BetaPRM, a distributional Process Reward Model that predicts both success probability and reliability of predictions. This model allows downstream applications to differentiate between reliable and uncertain rewards, enhancing decision-making processes. Additionally, the Adaptive Computation Allocation method utilizes this reliability signal to optimize computation resources effectively.
- ▪BetaPRM predicts both a step-level success probability and the reliability of that prediction.
- ▪The model improves PRM-guided Best-of-N selection while maintaining standard step-level error detection.
- ▪Adaptive Computation Allocation uses the learned reliability signal to optimize computation usage.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Computation and Language arXiv:2605.15529 (cs) [Submitted on 15 May 2026] Title:Process Rewards with Learned Reliability Authors:Jinyuan Li, Langlin Huang, Chengsong Huang, Shaoyang Xu, Donghong Cai, Yuyi Yang, Wenxuan Zhang, Jiaxin Huang View a PDF of the paper titled Process Rewards with Learned Reliability, by Jinyuan Li and 7 other authors View PDF HTML (experimental) Abstract:Process Reward Models (PRMs) provide step-level feedback for reasoning, but current PRMs usually output only a single reward score for each step. Downstream methods must therefore treat imperfect step-level reward predictions as reliable decision signals, with no indication of when these predictions should be trusted.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
