AI ethics evidence index Search
Source-linked daily record
Evidence record 4490 · automatically gathered

Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Science

LLM annotators are increasingly used in computational social science (CSS), but it is unclear whether their alignment-shaped errors preserve the empirical conclusions a researcher would report. We audit three open-source 7B instruction-tuned models (Zephyr, Mistral-Instruct, Qwen2.5-Instruct) across six TweetEval tasks under four prompt conditions (72 cells) and find that social-desirability failures do not run in a single direction. Zephyr exhibits leniency bias, systematically under-applying h

Record details

Published: 12 May 2026
Source: arXiv
Category: Research
Topics: Bias & fairness · Safety & alignment · Transparency
Retrieved: 14 July 2026

source-onlyevidence status

These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.

Also mentions Mistral · Shared topic: Bias & fairness

EQUITRIAGE: A Fairness Audit of Gender Bias in LLM-Based Emergency Department Triage

arXiv · 5 May 2026

Also mentions Mistral · Shared topic: Safety & alignment

Mind the Gap: Pitfalls of LLM Alignment with Asian Public Opinion

arXiv · 6 March 2026

Also mentions Mistral · Shared topic: Safety & alignment

Phoenix-VL 1.5 Medium Technical Report

arXiv · 11 May 2026

Also mentions Mistral · Shared topic: Bias & fairness

Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AI

arXiv · 2 May 2026

Also mentions Mistral · Shared topic: Safety & alignment

Breaking Safety at the Token Boundary: How BPE Tokenization Creates Exploitable Gaps in LLM Alignment

arXiv · 1 May 2026

Also mentions Mistral · Shared topic: Safety & alignment

Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation

arXiv · 1 June 2026

How to cite this record

ethics.ai (12 May 2026), “Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Science,” evidence record 4490, https://ethics.ai/record/4490 (originally published by arXiv).

JSON

Use and limitations

This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.