University of Twente Student Theses
Leveraged calibrated loss for learning to defer
Steege, J.M. ter (2023) Leveraged calibrated loss for learning to defer.
|
PDF
1MB |
| Abstract: | The Learning to Defer (L2D) framework is designed to enhance the safety of AI systems by incorporating human intervention in decision-making when it is likely to lead to more accurate results than the model alone. In this paper, we propose a new family of surrogate losses, called the Leveraged One-vs-All (LOvA) loss, which for the first time introduces a leverage parameter to consider the trade-off between expert correctness and incorrectness. Our theoretical analysis derives a generalized result for Bayes risk consistency of the LOvA loss in the L2D system, providing guidance for selecting the leverage parameter. Additionally, we establish that the decision margin increases, which lowers the misclassification rate, resulting in a more robust and deterministic classification by our system. In our experiments, we validate the guidance offered by our theoretical analysis and demonstrate that our proposed LOvA loss performs significantly better than other state-of-the-art L2D systems on real-world datasets. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 31 mathematics |
| Programme: | Applied Mathematics MSc (60348) |
| Link to this item: | https://purl.utwente.nl/essays/94726 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page