AI ethics evidence index Search
Source-linked daily record
Evidence record 5701 · automatically gathered

Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering

Large Language Models are increasingly used as judges to evaluate code artifacts when exhaustive human review or executable test coverage is unavailable. LLM-judge is increasingly relevant in agentic software engineering workflows, where it can help rank candidate solutions and guide patch selection. While attractive for scale, current practice lacks a principled account of reliability and bias: repeated evaluations of the same case can disagree; small prompt edits can swing outcomes; and seemin

Record details

Published: 18 April 2026
Source: arXiv
Category: Research
Topics: Bias & fairness · Agents & autonomy · Transparency
Retrieved: 14 July 2026

source-onlyevidence status

These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment

arXiv · 21 April 2026

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks

arXiv · 9 April 2026

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

TRUST: A Framework for Decentralized AI Service v.0.1

arXiv · 29 April 2026

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

VulnAgent-R2: Evidence-Calibrated Multi-Agent Auditing for Repository-Level Vulnerability Detection

arXiv · 11 March 2026

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

Ethics of AI in Education: Towards a Community-Wide Framework

OpenAlex · 9 April 2021

Shared topic: Bias & fairness · Shared topic: Agents & autonomy

Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI

OpenAlex · 18 March 2021

How to cite this record

ethics.ai (18 April 2026), “Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering,” evidence record 5701, https://ethics.ai/record/5701 (originally published by arXiv).

JSON

Use and limitations

This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.