Preserve Content, Disrupt Structure: Patch-Shuffle Counterfactual Distillation for Visual Reasoning

Multimodal on-policy distillation (OPD) provides dense, token-level teacher guidance on student-generated reasoning responses. To identify corrections that depend on visual evidence, previous methods contrast teacher predictions on original inputs and negative views created by masking or downsampling. However, these negatives mainly probe evidence availability and can miss signals for spatial relations and object–attribute bindings. We introduce Patch-Shuffle Counterfactual Distillation (PSCD), which constructs a negative visual view by shuffling patches within the teacher's evidence crop. This preserves pixels within each patch while disrupting spatial and compositional organization. At each student-generated prefix, PSCD compares the frozen teacher's predictions on the intact and shuffled crops, retaining only changes that support the intact teacher's correction of the student to construct a student-anchored distillation target. Experiments on two Qwen3.5 model scales show gains in spatial and compositional reasoning alongside improved fine-grained visual understanding. Among the evaluated Qwen3.5-4B-based methods, PSCD leads on all six spatial benchmarks and outperforms the strongest baseline on all three compositional understanding benchmarks.