I train AI systems to know when they don't know

I am a PhD candidate at Ohio State, finishing in May 2026, advised by Jim Davis. My research is about making machine learning systems that know when to stop guessing. The question first came up in 2021, when Jim and I were staring at t-SNE plots of image classifiers. The models were making confused, unreliable predictions in regions where class clusters overlapped — but scattered among the noise were pockets of clean, well-separated examples where the model was consistently right. That pattern stuck with me. I wanted to know whether we could learn which parts of the decision space are actually trustworthy, and build systems that act accordingly.

The simplest way I can explain my work is this: I teach AI to stop making things up. Everyone who has used ChatGPT has seen it confidently produce something false. In a casual conversation that is annoying. In production — medical imaging, defense systems, content moderation — a model that guesses wrong with high confidence is worse than one that gives no answer at all. The core problem is that most ML systems are trained to always produce output, with no mechanism to say “I am not sure about this.” My research gives them that mechanism.

The technical framing is selective prediction and abstention. I study how models can recognize when they have landed in a dirty region of the decision space — where the data is ambiguous, overlapping, or out of distribution — and either find the action that gets them to a clean state or back out and abstain as fast as possible. When a model abstains, the question gets routed to a human or a more capable system. A model that says “I don’t know” and defers is more reliable than one that forces an answer it cannot support.

The problem becomes more interesting in large language models. In image classification, a dirty region is relatively static — class boundaries overlap and that is that. In language, what counts as dirty depends on context, phrasing, and the specific knowledge required. A question that is unanswerable given one prompt can become straightforward with a small amount of additional reasoning or retrieval. My current work trains models to distinguish between these cases: questions they could resolve with more computation, questions that need external context, and questions that are genuinely beyond reach. The goal is structured uncertainty — not a single “I don’t know” reflex, but a diagnosis of why the model is uncertain and a routing decision for what should happen next.

On a given day I write Python and PyTorch, run experiments on HPC clusters, and build the tooling that holds research together. I have recently been spending a lot of time on agentic AI workflows, which turn out to be a natural fit for the routing and abstention problems I already think about. I am looking for research scientist or applied ML engineering roles after graduation. I am a U.S. citizen and comfortable working with CUI and DoD requirements.