EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation
The paper presents EDGE-OPD, a method for improving On-Policy Distillation (OPD) in machine learning. It addresses challenges related to the use of privileged context during training, which can inadvertently alter model behavior. The authors propose a solution that incorporates guided rollouts and an evidence mask to enhance learning outcomes.
- ▪On-Policy Distillation (OPD) is an effective post-training paradigm for large language models.
- ▪The proposed EDGE-OPD method uses guided rollouts to inject privileged-context behavior during training.
- ▪Empirical results show that traditional OPSD fails to learn target identities, while EDGE-OPD succeeds.
Opening excerpt (first ~120 words) tap to expand
Computer Science > Artificial Intelligence arXiv:2605.23493 (cs) [Submitted on 22 May 2026] Title:EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation Authors:Aristotelis Lazaridis, Dylan Bates, Aman Sharma, Brian King, Vincent Lu, Jack FitzGerald View a PDF of the paper titled EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation, by Aristotelis Lazaridis and 5 other authors View PDF HTML (experimental) Abstract:On-Policy Distillation (OPD) has gained wide attraction as an LLM post-training paradigm due to its effectiveness in improving capabilities without introducing model distribution drift, and consequently, regression in general tasks.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at arXiv cs.AI.
Discussion
0 commentsMore from arXiv cs.AI
