- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper identifies the “visual evidence–credit assignment gap” in latent visual reasoning (LVR), where models may produce correct answers without encoding or using decisive visual evidence in their latent tokens. To address this issue, the authors propose ReaLVR, which selects latent tokens based on attention differences between correct and incorrect answers and uses contrastive learning with relevant and mismatched visual evidence to supervise them, without modifying the model architecture or inference procedure. Experiments across Qwen, InternVL, and Gemma models—including a 235B-parameter model—combine benchmark comparisons, component ablations, and mechanistic analyses to evaluate whether ReaLVR improves the visual sensitivity and causal relevance of latent representations.
Critical strengths
Counterfactual image edits, latent-token replacement, and component ablations provide useful evidence that the tested LVR baselines are insufficiently sensitive to key visual changes, while ReaLVR increases the visual responsiveness of latent states and the answer’s dependence on them.
Critical weaknesses
-
Limited and non-uniform advantage over the strongest alternative. On Qwen2.5-VL-7B, ReaLVR outperforms ILVR on MMVP, HRBench-4K, and MME-RealWorld, but underperforms it on BLINK and HRBench-8K. Its five-task average improves by only 0.8 percentage points. Thus, ReaLVR provides genuine gains, but does not demonstrate consistent superiority over the strongest alternative. Without an equal-compute comparison, the magnitude and practical significance of this advantage remain unclear.
-
The 235B evaluation does not isolate the contribution of ReaLVR. For Qwen3-VL-235B, Table 2 compares ReaLVR only against Direct decoding and LVR-SFT, and evaluates only three benchmarks. A matched LVR-RL baseline is missing. Consequently, the reported gains cannot be attributed specifically to the proposed evidence-credit objective rather than to Stage-2 RL more generally. The experiment establishes feasibility at this scale, but not a controlled advantage of ReaLVR.
Questions for authors
-
Was ILVR reproduced from the same Stage-1 checkpoint with matched update steps, rollout budget, image-token limit, data order, and checkpoint-selection procedure? Could the authors report paired-seed differences?
-
What are ReaLVR’s additional GPU-hours, wall-clock time, throughput, and peak-memory requirements relative to ILVR or LVR-RL?
-
Could the authors provide a matched LVR-RL baseline for Qwen3-VL-235B? Without it, how can the reported gain be separated from generic Stage-2 RL?
2 comments
Author response to Reviewer WSJb (Part 2/2)
W2/Q3. Isolating the evidence objective at 235B.
We have completed the matched 235B LVR-RL control: it removes the evidence objective while retaining the other training conditions. The public LVR implementation did not support our distributed setup at submission, and the necessary adaptation is now complete.
The new control uses Qwen3-VL-235B-A22B on 200 nodes (1,600 GCDs). LVR-RL and ReaLVR share the Stage-1 checkpoint, training data and order, update and rollout budgets, image-token limit, and checkpoint-selection rule. The other task scores come from the original manuscript's Table 2; three-task averages are computed from the displayed task scores.
| Method | Source | MMVP (%) ↑ | BLINK (%) ↑ | MME-RealWorld (%) ↑ | Three-task average (%) ↑ |
|---|---|---|---|---|---|
| Direct decoding | Original Table 2 | 80.0 | 73.5 | 69.8 | 74.4 |
| LVR-SFT | Original Table 2 | 80.9 | 74.2 | 70.4 | 75.2 |
| LVR-RL | New matched control () | 81.3 | 74.7 | 70.6 | 75.5 |
| ReaLVR | Original Table 2 | 81.9 | 75.4 | 71.0 | 76.1 |
| ReaLVR − matched LVR-RL | Observed additional gain | +0.6 pp | +0.7 pp | +0.4 pp | +0.57 pp |
Across these three benchmarks, outcome-only RL improves over LVR-SFT by +0.37 pp on average, and evidence supervision provides a further +0.57 pp over the matched LVR-RL control. We will include this control in Table 2 of the revised manuscript.
Thank you again for your time and valuable feedback. We hope the above content can solve your doubts. We have updated all the newly added experiments. Please let us know if there are further questions.
Author response to Reviewer WSJb (Part 1/2)
We sincerely appreciate your careful review and your recognition of the counterfactual, latent-replacement, and ablation evidence. Thank you for identifying the comparisons needed to clarify the practical benefit and the 235B result. We address each point below.
W1/Q1. Comparison with ILVR and the size of the improvement.
ReaLVR improves the five-task average over ILVR-Stage2, the interleaved latent-reasoning baseline, under matched training conditions. The following results are reproduced from the original manuscript's Table 1 for Qwen2.5-VL-7B:
The benefit is in average performance rather than every benchmark. The original manuscript's Table 2 also reports average gains over LVR-RL of +3.1 pp on InternVL3-8B and +2.6 pp on Gemma-3-12B.
We verified the following shared settings for the ILVR-Stage2 and ReaLVR comparison:
The within-seed differences below are ReaLVR minus ILVR-Stage2. Positive values favor ReaLVR:
The five-task average improvement is positive in all three paired runs.
W1/Q2. Equal-compute comparison and training overhead.
At the same update count, ReaLVR costs less than ILVR-Stage2 and more than LVR-RL; under the same compute budget, it retains the highest five-task average. We profiled 100 Stage-2 updates on 64 nodes (512 AMD MI250X GCDs, global batch 512). A GCD-hour counts one ROCm-visible compute die for one hour.
For the equal-compute comparison, all methods receive a Stage-2 budget of B = 1,234 GCD-hours per seed. Results are from three training seeds. We select the best validation checkpoint reached within each method's budget using the same selection rule:
The average-accuracy advantage therefore persists when the methods receive the same training-compute budget.