A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
We investigate the geometry of predictive information across the layers of large language models (LLMs). We repurpose representation lenses-learned affine maps trained to predict the next token from intermediate residual streams-as geometric diagnostic tools. Rather than asking what the model predicts at each layer, we ask where predictive information resides and how it evolves across depth. We define at each layer a predictive readout subspace as the dominant k-dimensional singular subspace of
Record details
Published: 9 May 2026
Source: arXiv
Category: Research
Topics: Healthcare · Finance, VC & PE
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
TokaMind for Power Grid: Cross-Domain Transfer from Fusion Plasma
arXiv · 10 May 2026
CORTEG: Foundation Models Enable Cross-Modality Representation Transfer from Scalp to Intracranial Brain Recordings
arXiv · 11 May 2026
When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
arXiv · 7 May 2026
Anatomy-Slot: Unsupervised Anatomical Factorization for Homologous Bilateral Reasoning in Retinal Diagnosis
arXiv · 13 May 2026
From Awareness to Action: Understanding and Overcoming the Research-Practice Gap in Algorithmic Fairness for Public Health
arXiv · 2 May 2026
Mapping how LLMs debate societal issues when shadowing human personality traits, sociodemographics and social media behavior
arXiv · 30 April 2026
How to cite this record
ethics.ai (9 May 2026), “A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases,” evidence record 4661, https://ethics.ai/record/4661 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.