A Layer-wise Analysis of Supervised Fine-Tuning
While critical for alignment, Supervised Fine-Tuning (SFT) incurs the risk of catastrophic forgetting, yet the layer-wise emergence of instruction-following capabilities remains elusive. We investigate this mechanism via a comprehensive analysis utilizing information-theoretic, geometric, and optimization metrics across model scales (1B-32B). Our experiments reveal a distinct depth-dependent pattern: middle layers (20\%-80\%) are stable, whereas final layers exhibit high sensitivity. Leveraging
Record details
Published: 12 April 2026
Source: arXiv
Category: Research
Topics: Safety & alignment · Finance, VC & PE
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Identifying and Mitigating Gender Cues in Academic Recommendation Letters: An Interpretability Case Study
arXiv · 14 April 2026
Conflicts Make Large Reasoning Models Vulnerable to Attacks
arXiv · 10 April 2026
Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment
arXiv · 14 April 2026
CLIP Architecture for Abdominal CT Image-Text Alignment and Zero-Shot Learning: Investigating Batch Composition and Data Scaling
arXiv · 15 April 2026
What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal
arXiv · 9 April 2026
From Safety Risk to Design Principle: Peer-Preservation in Multi-Agent LLM Systems and Its Implications for Orchestrated Democratic Discourse Analysis
arXiv · 9 April 2026
How to cite this record
ethics.ai (12 April 2026), “A Layer-wise Analysis of Supervised Fine-Tuning,” evidence record 6001, https://ethics.ai/record/6001 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.