AI Insight
This paper addresses a critical problem in evaluating fairness in multi-task learning systems: different algorithms are inadvertently compared under different semantic thresholds when fairness metrics are derived from each model's own internal representations. The authors propose ReLiF, a framework that uses fixed reference tolerances for consistent fairness auditing across models, combined with a feedback controller to maintain effective fairness regularization during training. Experiments on clinical time-series data and computer vision tasks demonstrate that this fixed-threshold approach reveals genuine utility-fairness trade-offs that variable thresholds can obscure.
Why it matters
Fair machine learning systems are increasingly deployed in high-stakes domains like healthcare and autonomous systems. This work provides a more reliable methodology for comparing fairness across different models, which is essential for practitioners and policymakers to make informed decisions about which algorithms to deploy when fairness and performance must be balanced.
arXiv:2606.10632v1 Announce Type: cross
Abstract: Lipschitz-style individual fairness formalizes the idea that semantically similar examples should receive similar predictions, but its evaluation in multi-task learning (MTL) can be confounded by method-induced representation scales. This paper identifies threshold confounding: when the auditing tolerance is derived from each model’s own representation distances, different algorithms are compared under different semantic thresholds. A threshold-drift analysis further shows how Bias rankings can change and identifies sufficient conditions for ranking preservation.
We propose textbf{ReLiF}, a reliability-aware framework that separates evaluation-time fixed-$delta$ auditing from training-time controlled regularization. ReLiF uses a shared reference tolerance for comparable auditing and a violation-rate feedback controller to keep the Lipschitz surrogate active without letting it dominate stochastic training. This work also develops supporting analysis for threshold drift, reference-tolerance selection, and the relationship between the huberized training surrogate and its unsmoothed positive-margin counterpart.
Experiments on clinical time-series benchmarks and NYUv2 (NYU Depth V2) dense prediction show that fixed-$delta$ auditing exposes utility–fairness trade-offs that method-dependent thresholds can obscure. On NYUv2 with a ResNet50 backbone, ReLiF achieves competitive utility while substantially reducing aligned bias under shared fixed thresholds. On clinical benchmarks, ReLiF yields controlled fairness-regularized trade-offs, while fixed-$delta$ auditing reveals that task-balancing baselines can sometimes achieve lower bias and that genuine utility–fairness trade-offs persist. These results support fixed-$delta$ auditing as a semantically consistent protocol for evaluating Lipschitz fairness in MTL.