Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism
The paper presents a new method called PAT for improving the efficiency of Reinforcement Learning from Human Feedback (RLHF) training. PAT employs adaptive tensor parallelism to dynamically adjust configurations during the generation stage, addressing issues of underutilization of GPUs. Evaluations show that this method significantly reduces generation latency and overall training iteration time compared to existing frameworks.
- ▪Reinforcement Learning from Human Feedback (RLHF) is a key post-training paradigm for enhancing model quality.
- ▪The proposed PAT method dynamically reconfigures tensor parallelism during RLHF training to improve efficiency.
- ▪Evaluations indicate that PAT can reduce generation latency by up to 34.6% and end-to-end training iteration latency by up to 27.2%.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.23945 (cs) [Submitted on 3 May 2026] Title:Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism Authors:Long Zhao, Qinghe Wang, Jiaan Zhu, Youhui Bai, Zewen Jin, Chaoyi Ruan, Shengnan Wang, Cheng Li View a PDF of the paper titled Accelerating Long-Tail Generation in Synchronous RLHF Training via Adaptive Tensor Parallelism, by Long Zhao and 7 other authors View PDF HTML (experimental) Abstract:Reinforcement Learning from Human Feedback (RLHF) has become a key post-training paradigm for improving model quality.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
