Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models
The paper presents a novel approach to optimizing Diffusion Multi-Modal Large Language Models (dMLLMs) using Hierarchical Reinforcement Learning. It addresses challenges in reward assignment during the image generation process by introducing a Sketch-Then-Paint training scheme. Experimental results indicate significant improvements in image quality and user preference metrics.
- ▪The proposed method, Hierarchical Token GRPO (HT-GRPO), integrates hierarchical generation into the policy optimization process.
- ▪HT-GRPO organizes updates into three stages: global, structure, and refinement.
- ▪Experiments show substantial gains on the GenEval and DPG benchmarks using two dMLLM backbones.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.16842 (cs) [Submitted on 16 May 2026] Title:Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models Authors:Siqi Luo, Jianghan Shen, Yi Xin, Huayu Zheng, Haoxing Chen, Yan Tai, Yue Li, Junjun He, Yihao Liu, Guangtao Zhai, Yuewen Cao, Xiaohong Liu View a PDF of the paper titled Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models, by Siqi Luo and 11 other authors View PDF HTML (experimental) Abstract:Diffusion Multi-Modal Large Language Models (dMLLMs) are powerful for image generation, but optimizing them through reinforcement learning (RL) remains a major challenge.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
