- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ViPLE, a parallel latent interface for frozen multimodal models. It extends LRT with a final cross-attention read of fused image-question states, producing 256 latent slots through an encoder trained with supervised alignment, on-policy distillation, and reinforcement learning. Experiments across five backbones and six benchmarks show improved accuracy under constrained sequential decoding budgets.
Critical strengths
The paper addresses a practical efficiency problem and provides broad evaluation across model scales and families. The latent-replacement controls and retrained cross-attention ablation provide useful evidence that both the learned representations and the final prompt read contribute to performance.
Critical weaknesses
- The technical novelty appears limited. Parallel latent construction, a frozen consuming model, and the encoder backbone closely follow LRT (Jiang et al., 2026). The main architectural addition is a final cross-attention read of fused prompt states. Although Table 2 supports its empirical value, comparisons with other trained latent interfaces are needed to establish the contribution more convincingly.
- The motivation for the added prompt read needs further development. The upstream encoder already consumes the same fused image-question states. Comparisons with parameter-matched alternatives or different read placements would help establish why this particular design is preferable to simply adding capacity or another route to the prompt.
- The overall computational cost is insufficiently characterized. The token metric excludes the encoder and the 256-slot backbone prefill, so fewer decoded tokens do not establish lower end-to-end latency or total computation. Please report latency, peak memory, and FLOPs, together with per-stage training GPU-hours. Freezing the backbone still requires backpropagation through it to train the encoder.
- Figure 6 raises questions about the necessity and stability of the training pipeline. WeMath accuracy falls from 45.6% after SFT to 34.6% after OPD, then reaches 65.4% after ten RL steps. The paper documents the accompanying response-length changes and the ten-word reward gate, but does not isolate their effects. Comparing SFT-to-RL with the full pipeline, and correctness-only RL with the gated reward, across training seeds would clarify OPD's benefit and the source of the recovery.
- The decoding comparison contains a protocol difference: Full-CoT uses a continuation fallback, whereas ViPLE uses direct force-close termination. Matched-termination evaluation and a baseline trained to produce concise textual answers would help separate the benefits of latent conditioning from termination behavior and additional task-specific training.
Reference:
[1] Cong Jiang, Xiaofeng Zhang, Fangzhi Zhu, XiaoWei Chen, Junxiong Zhu, & Zheng Zhang (2026). Rethinking LLM Reasoning: From Explicit Trajectories to Latent Representations. In The Fourteenth International Conference on Learning Representations. https://openreview.net/forum?id=CbK7lYbmv8
Questions for authors
See weakness.
3 comments
Fianlly, we respectfully believe that a Clear Rejection (1) is disproportionately harsh, particularly given your acknowledgment of our empirical contributions and the concerns resolved through our additional experiments.
We sincerely hope you will reconsider your assessment and raise your rating accordingly.
[Urgent] Follow-up
Fianlly, we respectfully believe that a Clear Rejection (1) is disproportionately harsh, particularly given your acknowledgment of our empirical contributions and the concerns resolved through our additional experiments.
We sincerely hope you will reconsider your assessment and raise your rating accordingly.
Response to rebuttal
I thank the authors for the thorough response and additional experiments. These results resolved some of my concerns regarding training cost and training pipeline.
However, my concern about the technical novelty of ViPLE remains. The response primarily emphasizes differences in the architecture design and research domain relative to LRT. These differences alone do not establish the technical contribution ViPLE provides beyond adapting an existing latent-reasoning framework to multimodal inputs. While the final prompt-read ablation supports empirical usefulness of the component, it does not sufficiently explain why this particular design is necessary.
Therefore, I'm inclined to maintain my score.