AI ethics evidence index Search
Source-linked daily record
Evidence record 5773 · automatically gathered

When Fairness Metrics Disagree: Evaluating the Reliability of Demographic Fairness Assessment in Machine Learning

The evaluation of fairness in machine learning systems has become a central concern in high-stakes applications, including biometric recognition, healthcare decision-making, and automated risk assessment. Existing approaches typically rely on a small number of fairness metrics to assess model behaviour across group partitions, implicitly assuming that these metrics provide consistent and reliable conclusions. However, different fairness metrics capture distinct statistical properties of model pe

Record details

Published: 16 April 2026
Source: arXiv
Category: Research
Topics: Bias & fairness · Privacy · Healthcare
Retrieved: 14 July 2026

source-onlyevidence status

These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.

Same publisher byline · Shared topic: Bias & fairness

Why Aggregate Accuracy is Inadequate for Evaluating Fairness in Law Enforcement Facial Recognition Systems

arXiv · 30 March 2026

Shared topic: Bias & fairness · Shared topic: Privacy

FairHealth: An Open-Source Python Library for Trustworthy Healthcare AI in Low-Resource Settings

arXiv · 5 May 2026

Shared topic: Bias & fairness · Shared topic: Privacy

Ethical Fairness in Ubiquitous Health Sensing without Known Attributes

arXiv · 10 March 2026

Same publisher byline · Shared topic: Bias & fairness

Operational AI Deployment Assurance: Governance-State Orchestration Under Threshold-Sensitive Deployment Conditions -- A Governance Framework for High-Stakes AI Systems

arXiv · 27 May 2026

Shared topic: Bias & fairness · Shared topic: Privacy

Reimagining psychiatric care with agentic AI: promise, challenges, and a roadmap forward

OpenAlex · 16 February 2026

Shared topic: Bias & fairness · Shared topic: Privacy

Governing Healthcare AI in the Real World: How Fairness, Transparency, and Human Oversight Can Coexist: A Narrative Review

OpenAlex · 6 February 2026

How to cite this record

ethics.ai (16 April 2026), “When Fairness Metrics Disagree: Evaluating the Reliability of Demographic Fairness Assessment in Machine Learning,” evidence record 5773, https://ethics.ai/record/5773 (originally published by arXiv).

JSON

Use and limitations

This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.