- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates a critical failure mode in MLLMs: the Symbolic Visual Gap. The authors observe that replacing visual inputs with their underlying symbolic representations (e.g., generating code or descriptive captions) dramatically improves downstream reasoning accuracy by up to ~23% under identical model weights, prompts, and decoding configurations. Through fine-grained reasoning-trace inspections and answer-to-evidence attention analyses, the authors reveal that while models frequently verbalize the correct visual facts during perception, they fail to select them as premises during deduction, preferentially attending to distracting evidence instead. To bridge this bottleneck between perception and reasoning, the authors propose MM-OPD, an on-policy self-distillation framework. MM-OPD leverages privileged symbolic views in a frozen self-teacher to construct centered residual targets, effectively steering the student’s on-policy image-conditioned rollouts toward correct visual evidence without test-time overhead.
Critical strengths
Method wise and motivation wise this paper has merits in both.
Validated on both 4B and 9B backbones across 18 diverse benchmarks (general VQA, math, charts, OCR), MM-OPD consistently outperforms standard SFT, on-policy RL baselines, and vanilla OPSD.
The framework only requires symbolic representations during offline post-training. At test time, the model executes standard single-image question answering without requiring costs.
Critical weaknesses
Structured visuals (e.g., charts, geometric figures, Venn diagrams) map cleanly to executable code. However, for unconstrained in-the-wild natural scenes, dense captions often fail to capture complex 3D relationships, subtle textures, or high-density background interactions. The paper does not currently thoroughly discuss the upper limit of descriptive captions in representing open-domain images.
Calculating on-policy rollouts and evaluating four separate forward passes for each token across length budgets up to 8,192 tokens represents non-trivial training FLOPs compared to standard SFT or single-teacher distillation. Any comparisons?
Questions for authors
Any results other than qwen-family, like gemma or muse Glimmer?