Assessing the Role of Imagery in Multimodal Machine Translation

Published at WMT 2024, this work designs imagery-sensitive contrastive metrics for multimodal machine translation and evaluates three English-to-French model families. The experiments hold a candidate translation fixed while varying matched and mismatched visual context, testing whether the image moves model uncertainty in the intended direction. These score-based diagnostics do not by themselves establish generated-translation quality or causal evidence use.