ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
The alignment of Large Language Models (LLMs) utilizes Reinforcement Learning from AI Feedback (RLAIF) for non-verifiable domains such as long-form question answering and open-ended instruction following. These domains often rely on LLM based auto-raters to provide granular, multi-tier discrete rewards (e.g., 1-10 rubrics) that are inherently stochastic due to prompt sensitivity and sampling randomness. We empirically verify the stochasticity of auto-raters that can propagate and corrupt standar
Record details
Published: 12 May 2026
Source: arXiv
Category: Research
Topics: Regulation · Safety & alignment
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
BSO: Safety Alignment Is Density Ratio Matching
arXiv · 12 May 2026
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
arXiv · 13 May 2026
On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment
arXiv · 12 May 2026
Positive Alignment: Artificial Intelligence for Human Flourishing
arXiv · 11 May 2026
TRACE: Distilling Where It Matters via Token-Routed Self On-Policy Alignment
arXiv · 11 May 2026
Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
arXiv · 11 May 2026
How to cite this record
ethics.ai (12 May 2026), “ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization,” evidence record 4440, https://ethics.ai/record/4440 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.