- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CrystaL, a single-stage dual-path framework for learning useful visual latent states in MLLMs without external visual teachers or intermediate visual annotations. The intact path processes the original image and generates latent states, while a parameter-shared corrupted path receives degraded visual input and uses the intact-path latent states through layer-wise latent replacement to predict the same answer. The model is trained with answer supervision together with predictive and attention consistency losses, while only the intact path is retained at inference.
Critical strengths
- Well-motivated information-routing formulation. CrystaL addresses an important weakness of latent visual reasoning: answer supervision alone does not guarantee that intermediate latent states become functionally useful. The dual-path design provides a simple way to increase dependence on latent information without prescribing what the latent states must encode or requiring a staged visual bottleneck. The layer-wise transfer of intact-conditioned latent states to the corrupted predictor is particularly well aligned with this objective.
- Strong functional evidence for latent utility. The controlled intervention experiments are a notable strength. Supplying intact-conditioned latent states consistently improves performance under clean, blurred, and shuffled-option conditions, providing stronger evidence than simply observing higher final benchmark accuracy. The broad improvements across high-resolution perception, spatial reasoning, and visual search, together with transfer to a second backbone, further support the robustness of the approach.
Critical weaknesses
The contribution of the CrystaL-specific routing mechanism is not fully disentangled from latent fine-tuning itself. The paper compares CrystaL with Direct-FT, but this control does not appear to provide the same latent computation slots. More importantly, the corruption ablation shows that training without corruption already reaches an average score of 70.9, compared with 72.0 for the default moderate blur setting. Likewise, corrupted-path CE alone reaches 71.0, while the full combination of KL and attention consistency reaches 72.0. Thus, a substantial portion of the total improvement may arise from introducing latent computation and task-specific fine-tuning rather than from the proposed corruption-based information-routing mechanism itself. This attribution issue is important because the latter is the central methodological claim.
Questions for authors
- Can the authors provide an intact-only latent fine-tuning baseline using the same number of latent tokens, training data, optimization budget, and autoregressive latent generation, but without the corrupted path, latent transfer, KL loss, or attention consistency? This comparison would directly clarify how much of the gain comes from latent computation itself versus the proposed dual-path routing mechanism.
- In the no-corruption setting, where the two paths receive the same visual input, which parts of the objective remain non-trivial in practice? In particular, how large are the KL and attention-consistency losses in this setting, and what explains the substantial improvement over the base model?
3 comments
Thank you for your response.
I will maintain my rating of 3. Unfortunately, I am not very familiar with this research area, so my confidence score is 2.
I am also somewhat puzzled as to why several papers assigned to me for ICLR this year fall outside my areas of expertise. I would appreciate it if paper assignments could be more closely aligned with my broader research area, as they have been in previous years.
Best wishes !
Dear Reviewer,
Thank you for your time and thoughtful feedback. We sincerely appreciate your consideration of our work and your transparency regarding your familiarity with this research area.
We also understand your concerns about the reviewer assignment process and hope that future assignments will be better aligned with your research expertise.
Thank you again for your valuable time and effort throughout the review process.
(1) Disentangling CrystaL's routing mechanism from latent fine-tuning.
We clarify that
Direct-FTin the paper actually corresponds to Intact-Only Latent Fine-Tuning under the 8-diverse-token setting, trained with the exact same data mixture and optimization recipe as CrystaL (4,000 optimizer steps, per-device batch size 4 across 4$\times$A40 GPUs, cosine schedule with 2×10−4 peak LR and 0.05 warmup ratio). As shown in Figure 3, CrystaL consistently outperforms Intact-Only Latent Fine-Tuning across all 8 benchmarks.Furthermore, inspired by the reviewer's comment, we added an Answer-Only Direct Fine-Tuning (
Answer-Only FT) baseline (directly supervising final answers without latent tokens) to cleanly separate the benefit of latent tokens from our dual-path routing mechanism:As shown above,
Answer-Only FTyields virtually no gain over the baseQwen2.5-VL-7Bmodel (67.62% vs. 67.68% AVG), indicating that naively fine-tuning on final answers does not meaningfully improve multimodal reasoning. In contrast, CrystaL boosts the average score to 72.00% (+4.38 points overAnswer-Only FT), confirming the critical role of our latent reasoning paradigm.(Q2) Objective behavior, loss magnitudes, and gains in the
no-corruptionsetting.In the
no-corruptionsetting, because both pathways receive identical clean visual inputs and share weights, they produce identical latent representations and output distributions. Consequently, both the KL divergence and attention-consistency losses collapse to zero and become trivially inactive. Under this degenerate condition, the dual-path objective mathematically reduces to the Intact-Only Latent Fine-Tuning baseline discussed in (1) (effectively scaling the cross-entropy gradient by 2×). Thus, the initial gain ofno-corruptionover the base model stems entirely from supervised fine-tuning with extra latent computation steps on clean images.We also apologize for an imprecise wording regarding the
"CE alone (71.0)"ablation. As defined in Eq. (3), the CE objective actually comprises the cross-entropy losses from both the intact and corrupted pathways (LCE=LCEintact+LCEcorrupt), rather than the corrupted path alone. Because the intact-path term directly optimizes generation under the full visual context, the performance under"CE alone"naturally aligns closely with that of Intact-Only Latent Fine-Tuning. We will revise the text to make this formulation explicit.Crucially, introducing visual corruption to activate our dual-path alignment (CrystaL) yields substantial further gains beyond this
no-corruptionbaseline. We additionally evaluatedno-corruptionon V*Bench and POPE, alongside the overall average across all 8 benchmarks in Table 1:no-corruption(Intact-Only Latent FT)While
no-corruptionimproves the 8-benchmark average from 71.1 to 73.7 via standard latent fine-tuning, activating our dual-path corruption and alignment mechanism pushes performance to 75.4 overall—including a notable +5.3 point surge on V*Bench (77.4 → 82.7) and +1.7 points on POPE (87.0 → 88.7). This confirms that the improvements overno-corruptionare far from marginal, directly validating our dual-path alignment mechanism over naive latent fine-tuning.