To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs
When VLMs answer correctly, do they genuinely rely on visual information? We introduce a Tri-Layer Diagnostic Framework with three per-sample metrics: Latent Anomaly Detection, Visual Necessity Score, and Competition Score, which disentangle perception, dependency, and alignment failures. Across 9 VLMs and 9,000 model-sample pairs under counterfactual blind, noise, and conflict interventions, 72.9% of samples exhibit Visual Sycophancy, a Split Beliefs pattern in which internal evidence is preser
Record details
Published: 19 March 2026
Source: arXiv
Category: Research
Topics: Safety & alignment · Healthcare
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Detection Is Cheap, Routing Is Learned: Why Refusal-Based Alignment Evaluation Fails
arXiv · 18 March 2026
Interpretable Cross-Domain Few-Shot Learning with Rectified Target-Domain Local Alignment
arXiv · 18 March 2026
Adoption and Effectiveness of AI-Based Anomaly Detection for Cross Provider Health Data Exchange
arXiv · 19 March 2026
Do Understanding and Generation Fight? A Diagnostic Study of DPO for Unified Multimodal Models
arXiv · 17 March 2026
HiPath: Hierarchical Vision-Language Alignment for Structured Pathology Report Prediction
arXiv · 20 March 2026
DMMRL: Disentangled Multi-Modal Representation Learning via Variational Autoencoders for Molecular Property Prediction
arXiv · 22 March 2026
How to cite this record
ethics.ai (19 March 2026), “To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs,” evidence record 7040, https://ethics.ai/record/7040 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.