Score-Based One-step MeanFlow Policy Optimization
The paper introduces Score-Based One-step MeanFlow Policy Optimization (SOM), an innovative actor-critic algorithm designed for online reinforcement learning. SOM addresses the computational challenges of existing methods by constructing a target velocity field directly from the Q-function, allowing for efficient policy optimization. The results demonstrate that SOM achieves state-of-the-art performance on locomotion tasks while significantly reducing training and inference times.
- ▪SOM is an actor-critic algorithm that optimizes policy in online reinforcement learning.
- ▪It constructs the target velocity field from the Q-function using score estimation and a probability flow ODE.
- ▪SOM outperforms previous methods in locomotion tasks with a single generation step.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2605.23365 (cs) [Submitted on 22 May 2026] Title:Score-Based One-step MeanFlow Policy Optimization Authors:Kyungyoon Kim, Donghyeon Ki, Hee-Jun Ahn, Byung-Jun Lee View a PDF of the paper titled Score-Based One-step MeanFlow Policy Optimization, by Kyungyoon Kim and 3 other authors View PDF HTML (experimental) Abstract:Diffusion and flow matching have emerged as expressive policy classes in reinforcement learning, but their reliance on multi-step denoising imposes substantial computational overhead at inference time, which is particularly problematic in online RL. MeanFlow offers a promising alternative by learning an average velocity field that maps noise to data in a single network evaluation.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
