← Publications

ISVC 2022 · 2022

Learning When to Say "I Don't Know"

N. Kashani Motlagh, J. Davis, T. Anderson, J. Gwinnup

Paper arXiv Code

We introduce a reject-option classification framework that learns per-class softmax thresholds from validation data, maximizing retained accuracy while delivering calibrated abstentions. The method underpins production guardrails across image, text, and 2-D benchmarks and received the Best Paper Award at ISVC 2022.

Highlights

  • Won Best Paper at ISVC 2022; seeded follow-on work for the MVA 2025 journal extension.
  • Delivered +0.4% selective accuracy and +1.3% coverage gains on ImageNet over global thresholding.

Artifacts & reproduction

learning-idk

GitHub

PyTorch toolkit for per-class reject-option training with binomial threshold search, dashboards, and CLI.

  • Tune per-class thresholds on ImageNet, iNat, or custom datasets with one command.
  • Export coverage/accuracy curves and selective accuracy plots for reports.
  • Integrate abstention policies into existing Torch models via lightweight hooks.
  • Production-ready toolkit that underpins selective prediction guardrails across multiple domains (vision, text, and 2-D data).