University of Twente Student Theses


Leveraged calibrated loss for learning to defer

Steege, J.M. ter (2023) Leveraged calibrated loss for learning to defer.

[img] PDF
Abstract:The Learning to Defer (L2D) framework is designed to enhance the safety of AI systems by incorporating human intervention in decision-making when it is likely to lead to more accurate results than the model alone. In this paper, we propose a new family of surrogate losses, called the Leveraged One-vs-All (LOvA) loss, which for the first time introduces a leverage parameter to consider the trade-off between expert correctness and incorrectness. Our theoretical analysis derives a generalized result for Bayes risk consistency of the LOvA loss in the L2D system, providing guidance for selecting the leverage parameter. Additionally, we establish that the decision margin increases, which lowers the misclassification rate, resulting in a more robust and deterministic classification by our system. In our experiments, we validate the guidance offered by our theoretical analysis and demonstrate that our proposed LOvA loss performs significantly better than other state-of-the-art L2D systems on real-world datasets.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:31 mathematics
Programme:Applied Mathematics MSc (60348)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page