True Evidence, Wrong State: Repair or Rollback in Visual Counterfactuals

Counterfactual visual reasoning asks about an edited scene, while source images remain truthful about the scene before the edit. Rechecking this evidence can repair a wrong answer or roll a correct edited-state answer back to an obsolete one. In a local source-directed response, correction depends on how evidence offsets or repeats candidate error, captured by a cross-moment that permits opposite effects under identical marginal error distributions. Source strength controls whether and how long state risk improves, whereas query margins govern changes in the answer. Obsolete source displacement motivates state-compatible construction, while the interaction of construction quality and edit conflict motivates a separate source-retention decision. On MindEdit-L4, source evidence changes accuracy by −16.0/+15.0 percentage points for correct/unfinished post-state descriptions; fixed-bank exchanges isolate retention reversals, and separately trained replacement systems improve over append by 9.7–11.8 points across two families and two tasks. Evidence value depends on the queried state and candidate–source relationship, rather than on source truthfulness alone.