GainMasque: Auditing Masked Retrieval Gains and Repairing Evidence Integration

Incorrect information can undermine content quality, veracity, and integrity, and these challenges are amplified when such information is shared within multimodal information streams. Retrieval-augmented detection methods usually incorporate external evidence to improve predictions, but retrieval gains can mask whether model predictions depend on the retrieved evidence. To address this issue, we propose GainMasque, an auditing and repair framework that disentangles retrieval gain from genuine evidence gain. First, GainMasque audits retrieval by comparing predictions with original and controlled evidence swaps, uncovering when retrieval gains remain without claim-conditioned evidence use. Then, we delineate systematic sources of masked retrieval gain, including retrieval and fusion mechanisms that rely on dataset, source, or label-correlated signals rather than evidence. To address these failures, we further introduce Decoupled Residual Evidence Fusion (DREF), which freezes the content model, corrects test-time shifts in the evidence score through bias alignment, and controls the evidence contribution using relevance- and agreement-based trust. GainMasque is backbone-agnostic and can be applied across multiple multimodal encoders and retrieval settings. Experimental results across diverse datasets and vision-language models show that retrieval gain often does not imply evidence gain, while DREF yields evidence gain with claim-conditioned scorers and bounds losses under noisy or corrupted retrieval. Together, GainMasque provides a systematic framework for auditing what drives retrieval gains and repairing how retrieval evidence shapes predictions.