AI ethics evidence index Search
Source-linked daily record
Evidence record 17068 · automatically gathered

What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)

Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a single access modality (model APIs), perform a single run per prompt, and report accuracy as the primary outcome metric, without accounting for conditions such as web search that may have effects on model behavior in deployment. We audit these assumptions for one of the most widely-used LLMs, comparing two modalities, Chat

Record details

Published: 6 August 2026
Source: arXiv
Category: Research
Topics: Transparency
Retrieved: 7 August 2026

source-onlyevidence status

These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.

Shared topic: Transparency

The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images

arXiv cs.AI · 6 August 2026

Shared topic: Transparency

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

arXiv · 6 August 2026

Shared topic: Transparency

Signal or Spurious Cue? A Randomized Audit of Survey-Country Metadata in LLM Social Inference

arXiv cs.AI · 6 August 2026

Shared topic: Transparency

Cleo: A Transparent and Controllable Chatbot for Conversational Commerce

arXiv cs.HC · 6 August 2026

Shared topic: Transparency

Challenges in Evaluating Explanation Methods for Static and Evolving Data

arXiv cs.AI · 6 August 2026

Shared topic: Transparency

The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping

arXiv cs.AI · 6 August 2026

How to cite this record

ethics.ai (6 August 2026), “What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations),” evidence record 17068, https://ethics.ai/record/17068 (originally published by arXiv).

JSON

Use and limitations

This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.