When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs
The paper explores the impact of multi-agent reinforcement learning (RL) on large language model (LLM) workflows. It compares Shared-Policy and Isolated-Policy training methods, revealing that improvements depend on various factors including workflow and task. The findings indicate that while multi-agent RL can enhance performance, it introduces distinct patterns of failure based on the chosen policy-sharing approach.
- ▪Multi-agent LLM workflows aim to improve end-task accuracy through specialized roles.
- ▪The study compares Shared-Policy training with Isolated-Policy training across different workflows and model scales.
- ▪Isolated-Policy training often achieves higher peak accuracy but can lead to significant performance drops.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.24202 (cs) [Submitted on 22 May 2026] Title:When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs Authors:Yifan Zeng, Yiran Wu, Yaolun Zhang, Wentian Zhao, Kun Wan, Qingyun Wu, Huazheng Wang View a PDF of the paper titled When Does Multi-Agent RL Improve LLM Workflows? Workflow, Scale, and Policy-Sharing Tradeoffs, by Yifan Zeng and 6 other authors View PDF HTML (experimental) Abstract:Multi-agent LLM workflows route inference through specialized roles to lift end-task accuracy, but jointly training those roles with reinforcement learning is unstable in ways that are poorly understood.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
