Prism: Demystifying Retention and Interaction in Mid-Training
PRISM is an empirical study examining mid-training design choices in large language models, showing that mid-training on high-quality data significantly improves performance on reasoning benchmarks. The research finds that mid-training restructures most model weights and enables more effective subsequent reinforcement learning, while direct reinforcement learning without mid-training is far less effective. Data composition during mid-training, especially inclusion of science data, has a major impact on downstream performance, more so than adjustments during reinforcement learning.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Machine Learning arXiv:2603.17074 (cs) [Submitted on 17 Mar 2026 (v1), last revised 24 Mar 2026 (this version, v3)] Title:PRISM: Demystifying Retention and Interaction in Mid-Training Authors:Bharat Runwal, Ashish Agrawal, Anurag Roy, Rameswar Panda View a PDF of the paper titled PRISM: Demystifying Retention and Interaction in Mid-Training, by Bharat Runwal and 3 other authors View PDF Abstract:We present PRISM, a comprehensive empirical study of mid-training design choices for large language models.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv.org.
Discussion
0 commentsMore from arXiv.org