Mismatch Matters: On-Policy Distillation Beyond Token Agreement
On-policy distillation (OPD) has emerged as a core component of modern LLM post-training pipelines, yet we reveal a failure mode: degenerate agreement, where students exploit repetitive loops to achieve near-perfect token agreement with the teacher despite globally flawed responses. We therefore shift our focus from agreement to teacher-student mismatch, and find that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens. Student-excess tokens
Record details
Published: 10 August 2026
Source: arXiv cs.AI
Category: Research
Topics: Regulation · Children & education
Retrieved: 11 August 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation
arXiv cs.AI · 10 August 2026
SR-OPSD: Self-Referenced On-Policy Self-Distillation
arXiv cs.AI · 10 August 2026
Social Media as a Driver of Obesity in Children and Adolescents (Aged 6-18 Years): It Is Time for Regulatory Action
JMIR (Journal of Medical Internet Research) · 10 August 2026
WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training
arXiv · 10 August 2026
Where Does AI Innovation Go? Measuring Research Attention Imbalance in AI Music
arXiv cs.CY · 10 August 2026
Implementation of Split Deadlines in a Large CS1 Course
arXiv cs.CY · 10 August 2026
How to cite this record
ethics.ai (10 August 2026), “Mismatch Matters: On-Policy Distillation Beyond Token Agreement,” evidence record 18254, https://ethics.ai/record/18254 (originally published by arXiv cs.AI).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.