Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
We convert 21 of 28 full-attention layers of Qwen3-0.6B-Base into KDA (Kimi Delta Attention) linear-attention layers on a single consumer-grade GPU budget, and ask a simple question: what exactly does the conversion break? After surgery, hidden-state alignment and end-to-end KL distillation drive the student close to its teacher in perplexity, yet multiple-choice accuracy stays near random chance (25-29% vs. the teacher's 50.6% on C-Eval). Using a four-permutation diagnostic that rotates answer
Record details
Published: 3 August 2026
Source: arXiv cs.LG
Category: Research
Topics: Safety & alignment · Healthcare · Children & education
Retrieved: 5 August 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
The Agency Gap in AI-Supported Writing: How Reactive and Proactive Agent Designs Shape Multimodal Reasoning
arXiv cs.CY · 30 July 2026
When AI Takes the Couch: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models
arXiv cs.CY · 21 July 2026
Fine-Tuning, Retrieval-Augmented Generation, and Hybrid Large Language Models for Postoperative Decision Support: Comparative Analysis
JMIR (Journal of Medical Internet Research) · 14 July 2026
Phoneme-Level Mispronunciation Screening in Polish-Speaking Children with an Explainable Assistant
arXiv · 23 June 2026
LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
arXiv · 24 May 2026
Robotics-Inspired Guardrails for Foundation Models in Socially Sensitive Domains
arXiv · 19 May 2026
How to cite this record
ethics.ai (3 August 2026), “Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model,” evidence record 16581, https://ethics.ai/record/16581 (originally published by arXiv cs.LG).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.