Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation
Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization-aware distillation (QAD) helps recover accuracy lost under low bit quantization by training a quantized student to match the output distribution of a frozen higher precision teacher via a KL-divergence loss. In this work, we first provide a representation level diagnosis of QAD: output matching alo
Record details
Published: 4 June 2026
Source: arXiv
Category: Research
Topics: Healthcare · Children & education · Environment
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
arXiv · 7 August 2026
Social Media as a Driver of Obesity in Children and Adolescents (Aged 6-18 Years): It Is Time for Regulatory Action
JMIR (Journal of Medical Internet Research) · 10 August 2026
A future for the world's children? A WHO–UNICEF–Lancet Commission
OpenAlex · 1 February 2020
Long Live Fine-Tuning: Task-Specific Transformers Outperform Zero-Shot LLMs for Misinformation Response Classification on Reddit
arXiv · 2 June 2026
PySynthea: A Python-Native Framework for Scalable Synthetic Healthcare Data Generation
arXiv · 2 June 2026
TriEval: A Resource-Efficient Pipeline for LLM Bias, Toxicity, and Truthfulness Assessment
arXiv · 2 June 2026
How to cite this record
ethics.ai (4 June 2026), “Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation,” evidence record 1457, https://ethics.ai/record/1457 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.