Fidelity Probes for Specification--Code Alignment
We introduce fidelity probes: natural-language questions generated from a reference artifact with code-derived ground-truth answers, answered from a candidate specification. The fraction of agreeing probes, which we call the fidelity, decomposes into contradiction and coverage-gap rates that drive targeted spec edits to convergence. On a 15-program, roughly 12k-line COBOL benchmark (AWS CardDemo), we raise frozen-test specification fidelity from 0.63 to 0.94 over eight iterations, with the plate
Record details
Published: 17 May 2026
Source: arXiv
Category: Research
Topics: Safety & alignment
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Presentation: Leveraging Adversary Emulation for GenAI Red Teaming
InfoQ AI/ML · 10 August 2026
Algorithmic Constitutionalism
arXiv · 16 May 2026
Glass Box at Orbit: A Constitutional AI Verification Framework for Trustworthy Autonomous CubeSat Intelligence
arXiv · 2 June 2026
Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation
arXiv · 4 June 2026
From Rights to Rites: Expectations Management in Smart-Home AI
arXiv · 26 April 2026
AI-Driven Modular Services for Accessible Multilingual Education in Immersive Extended Reality Settings: Integrating Speech Processing, Translation, and Sign Language Rendering
arXiv · 7 April 2026
How to cite this record
ethics.ai (17 May 2026), “Fidelity Probes for Specification--Code Alignment,” evidence record 4204, https://ethics.ai/record/4204 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.