- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes CILVR, a latent visual reasoning framework based on counterfactual image pairs. CILVR encourages the model to actually use its intermediate latent states. The authors swap the latent representations between paired images and train the model to predict the answer associated with the transferred latent. The paper further introduces CA-GRPO for latent-space policy optimization and constructs the CFVR dataset with 304K verified counterfactual image pairs.
Critical strengths
-
The paper targets the latent-bypass problem: the possibility that a model produces meaningful-looking latent states while the decoder largely ignores them.
-
The latent swap objective is a well-motivated formulation, as it directly links changes in latent representations to changes in model predictions, rather than relying only on representation alignment.
Critical weaknesses
-
The current experiments mainly show that replacing or perturbing latent states affects model outputs (sec 4.4), but provide limited analysis of what information is actually encoded in these latent representations and how consistently this information supports downstream reasoning.
-
A substantial part of the comparison still relies on models such as GPT-4o, which makes it difficult to assess the competitiveness of CILVR against the current SOTA in multimodal reasoning. Although a few newer proprietary models are included, the comparison would benefit from a more systematic set of recent baselines such as GPT-5.6sol / Qwen3.5/ Claude Fable/ opus.
-
The technical notation is overly dense. The method introduces a large number of notations, objectives, training stages, intervention terms, and RL-specific quantities with insufficient motivation or intuitive explanation. The transition from counterfactual latent swapping to the direction-conditioned Gaussian policy is particularly abrupt, making the method substantially harder to understand than necessary.
-
Basic presentation and formatting issues remain throughout the paper; for example, multiple table captions are placed below the tables rather than above them. I think the author should be more careful about the manuscript formatting.
Questions for authors
Alongside the weaknesses, I have the following questions:
-
How sensitive is CILVR to the quality of the counterfactual edits and verification pipeline?
-
How does performance change when using fewer counterfactual pairs or substantially less RL compute?
-
Can the authors clarify whether the gains are consistent across different edit types in CFVR?
-
Why does GPT -5 achieve better performance yet not bold in Table 1?
I hope the authors will successfully address my concerns and I may raise my scores. I hope the authors could put more attention on the manuscript; I don't think the formatting problem should have shown up on submission.