Nick Kashani Motlagh

A classifier names a class for every input, including the ones it is plainly confused by. The standard fix — threshold the softmax — needs a rejection cost or coverage target that real deployments rarely have. On binary problems it rejects nothing at all, since one of two softmax scores is always at least 0.5.

B-CDF learns per-class abstention thresholds from a validation set, post-hoc, with no retraining. The constraint is inverted: rather than targeting accuracy on what the model keeps, it requires the rejected set to look like a coin flip — so a model cannot inflate its score by quietly discarding answers it had right.

Multimodal translation systems get an image to resolve an ambiguous sentence — “that's lots of bucks” is about deer or about dollars. But a model that always guesses “dollars” still scores about 50%, so the benchmarks could not separate genuine evidence use from a lucky prior. The field largely concluded that images do not help.

ImageCoMMuTE holds the translation fixed and swaps the image, then asks whether the correct image lowers the model's perplexity for the correct translation. That is an intervention on the evidence channel rather than an inference from aggregate scores. Group variants give credit only when a model resolves both readings of the same ambiguity.

RAG systems revise their own draft answers and average accuracy goes up, so refinement looks free. The average hides the trade: the same retrieval step that repairs one answer silently overwrites another that was already correct.

Score both branches for every question and label each outcome preserved, repaired, harmed, or unrecovered — the value of refining is then exactly repair minus harm. A two-head policy routes each question to answer, refine, or abstain.