2025 · Preprint · Preprints.org
The Coverage-Deferral Trade-Off: Fairness Implications of Conformal Prediction in Human-in-the-Loop Decision Systems
A study of how conformal prediction changes both uncertainty coverage and who is referred for human review.
Research question
Whose case gets a second look?
Evidence and scope
A prediction set can contain both possible answers. In this study, those cases go to a human reviewer. I compared a single calibration threshold with thresholds fitted separately for each group. The choice of threshold changes how often the right answer is in the set and how often a person has to wait for review.
On the Adult income dataset, groupwise calibration narrowed the gap in coverage from 5.0 to 1.2 percentage points. The gap in referral to a human grew from 5.1 to 12.9 points. Table 4 reports these results over 100 random seeds, with 90% target coverage and simulated review.
- Contribution
- Davut Emre Taşar · Sole author
- Source attribution
- Davut Emre Taşar
- Publication status
- Preprint · Preprints.org
- Publication or platform
- Preprints.org
- Identifier
- 10.20944/preprints202512.2631.v1
Sources
Research areas · Fairness · Uncertainty