The Geometric Inductive Bias of Grokking: Bypassing Phase Transitions via Architectural Topology
Mechanistic interpretability typically relies on post-hoc analysis of trained networks. We instead adopt an interventional approach: testing hypotheses a priori by modifying architectural topology to observe training dynamics. We study grokking - delayed generalization in Transformers trained on cyclic modular addition (Zp) - investigating if specific architectural degrees of freedom prolong the memorization phase. We identify two independent structural factors in standard Transformers: unbounde
Record details
Published: 5 March 2026
Source: arXiv
Category: Research
Topics: Bias & fairness · Safety & alignment · Finance, VC & PE
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment
arXiv · 14 April 2026
Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation
arXiv · 8 May 2026
Representation Alignment Rests on Linear Structure
arXiv · 22 May 2026
Algorithmic Fragility and Persona Bias in LLM-Generated Autistic Communication
arXiv · 26 May 2026
The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs
arXiv · 9 March 2026
Evaluating LLM-Based Grant Proposal Review via Structured Perturbations
arXiv · 9 March 2026
How to cite this record
ethics.ai (5 March 2026), “The Geometric Inductive Bias of Grokking: Bypassing Phase Transitions via Architectural Topology,” evidence record 7651, https://ethics.ai/record/7651 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.