When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models
Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input. We investigate the root causes of these failure modes with a mechanistic analysis focusing on the decoder-based VLMs. We trace these failure modes to a geometric over-alignment: to bridge the modality gap required by attention mechanisms, decoder-based VLMs over-align visual embeddings with
Record details
Published: 7 May 2026
Source: arXiv
Category: Research
Topics: Safety & alignment · Healthcare · Finance, VC & PE
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Retrieval-Guided Generation for Safer Histopathology Image Captioning
arXiv · 27 April 2026
CLIP Architecture for Abdominal CT Image-Text Alignment and Zero-Shot Learning: Investigating Batch Composition and Data Scaling
arXiv · 15 April 2026
Identifying and Mitigating Gender Cues in Academic Recommendation Letters: An Interpretability Case Study
arXiv · 14 April 2026
Physiological and Semantic Patterns in Medical Teams Using an Intelligent Tutoring System
arXiv · 31 March 2026
Autoregressive EHR Foundation Models with Multimodal Inputs
arXiv cs.LG · 24 July 2026
DOD selects Accenture to investigate ‘existential supply chain vulnerability’ threatening military medicine
DefenseScoop · 17 July 2026
How to cite this record
ethics.ai (7 May 2026), “When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models,” evidence record 4848, https://ethics.ai/record/4848 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.