- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how to shorten explicit reasoning in vision-language models while retaining problem-solving ability. LatentWeave jointly trains two components. First, a rubric evaluates complete reasoning trajectories for necessary stages, visible reasoning structure, compression safety, answer grounding, and related criteria; this feedback is combined with correctness and length signals for reinforcement learning. Second, the model selectively adds continuous residual updates to the input embeddings of subsequent ordinary tokens. These updates can access an additional visual feature channel, are activated by a routing module, and introduce no extra output tokens or KV sequence positions.
The evaluation covers six visual reasoning benchmarks. Relative to Qwen3-VL-8B-Thinking, two principal checkpoints reduce mean visible output length from 3,649 tokens to 1,252 and 660 tokens, corresponding to 2.92× and 5.53× compression. Macro-average accuracy decreases from 74.69% to 73.30% and 71.82%, respectively, or by 1.39 and 2.87 percentage points. The paper also reports same-backbone training trajectories, a text-only rubric control, repeated-sampling evaluation on MathV-G100, inference-time component interventions, and reasoning-trace case studies. The compression ratios concern visible output length, rather than measured end-to-end speedups.
Critical strengths
1. The paper provides an empirically supported approach to visual reasoning compression, with evidence beyond individual examples. Table 1 demonstrates substantial output-length reductions across six benchmarks and explicitly reports their accuracy costs: 1.39 percentage points in macro-average accuracy at 2.92× compression, and 2.87 points at 5.53× compression. The paper also exposes differences across tasks rather than reporting only an aggregate. These evaluated operating points are useful for applications with constrained visual reasoning budgets and are the main reason for my positive recommendation.
2. The joint design is technically meaningful and is evaluated from complementary perspectives. Applying continuous residuals at existing token positions allows explicit reasoning selection, latent state updates, and visual information access to be learned within the same generation process. Appendix Figure 5 includes a text-only comparison across all six benchmarks; Table 2 examines both single-sample accuracy and repeated-sampling solution coverage; and Table 3 tests the trained model’s dependence on the visual channel, latent updates, and routing. These experiments do not completely isolate the causal contribution of each component, but together they provide useful evidence for the joint system beyond compression ratios or selected successful traces.
Critical weaknesses
1. The methodological novelty appears incremental, with the contribution primarily in a targeted integration of established ideas. This is my main reservation. RL-based length control, rubric-based rewards, integration of continuous representations into textual generation, and routing between discrete and latent computation have precedents in L1, RaR, HRPO, and ThinkRouter, respectively. LatentWeave has concrete technical distinctions: bounded additive residuals derived from hidden states, visual evidence access, and joint training of semantic routing and frequency regulation with a compression-oriented rubric. These distinctions deserve credit. Nevertheless, the contribution currently appears strongest as a system integration and empirical advance for visual reasoning compression, with a more limited standalone methodological advance. The paper should more clearly establish which distinctions address limitations of existing approaches and why they matter for the reported gains. 2. The incremental benefit of the added design over simpler alternatives is not yet sufficiently established. When the contribution rests primarily on a combination of methods, the benefit of that combination is especially important. Appendix Figure 5 shows that Pseudo action and LatentWeave both reach 5.53× compression, with relative macro-accuracy losses of 4.67% and 3.84%, corresponding to an advantage of approximately 0.62 absolute percentage points for LatentWeave. This is positive evidence, but the large compression ratio is already attainable with this text-only approach. Moreover, the curves represent different training lineages, limiting assessment of the stable incremental benefit. The two comparisons in Table 2 do not simultaneously match training budget and output length, and a Standard RL plateau under one recipe does not establish a well-optimized text-only accuracy–length frontier. The results therefore support the effectiveness of the complete system more strongly than they establish the value of its most distinctive design choices. This is a concern about relative contribution, not a claim that the 0.62-point gain is worthless or statistically insignificant.
Questions for authors
Additional experimental comparison with the closest prior methods. Section 2 discusses HRPO and ThinkRouter, but neither has results in Table 1 or the appendices. Could the authors prioritize adapting at least one of these methods to the same Qwen3-VL-8B-Thinking backbone and compare its accuracy–inference-cost tradeoff with LatentWeave on the same visual benchmarks and evaluation protocol? Please report accuracy, visible output tokens, and additional latent computation at comparable budgets; for methods requiring training, use the same training data and comparable training budgets. These results would directly test the practical gain over prior hybrid latent reasoning methods. This is a request for an additional baseline experiment.
We thank Reviewer 32G1 for recognizing the empirical value of LatentWeave. We clarify the contribution and provide revised controls and a same-backbone ThinkRouter comparison.
Relation to prior work and novelty
LatentWeave learns to interleave concise explicit reasoning with visually informed latent updates. Each Action both updates the model state and marks a boundary in the visible reasoning. Rubric feedback assesses the reasoning and its continuity across these boundaries, jointly training text generation and the boundary policy (Submission §2--3; Revised §2--3).
Correctness and length rewards evaluate the answer and its cost, but do not explicitly assess whether the reasoning provides sufficient premises, valid deductions, and necessary checks. A rubric supplies that assessment, yet alone remains text-only compression. In LatentWeave, the learned boundary links the reasoning being assessed to the state update being executed: explicit reasoning and latent updates develop together during training, rather than replacing selected spans of a pre-existing CoT. The feedback remains trajectory-level, not a local causal score for each Action.
ActionNet makes this boundary computational rather than merely annotational: it injects a bounded, visually informed residual at an existing token position, without extra tokens or KV positions. The key empirical question is therefore whether the benefit comes only from showing boundaries to the judge, or also from learning where to place them and executing latent updates there. Revised Table 2 and Figure 5 address this through the controls below.
A complementary contribution is compression-aware frequency regulation. As reasoning traces shorten, fewer positions remain available for latent actions. We adapt the distance-prior scale when mean Action counts fall below the target, preserving execution opportunities while leaving context-dependent placement to the learned semantic router. The fixed-versus-adaptive comparison in Revised Table 2 and Figure 5 isolates this design: adaptive scaling produces shorter outputs at both reported checkpoints, with an accuracy trade-off at step 300 and similar accuracy at step 750.
Incremental benefit over simpler alternatives
Revised Table 2 and Figure 5 test random boundaries, learned boundaries, and latent execution under the same revised training and evaluation settings. The following comparison uses step 750 throughout; Random gate corresponds to the submission's Pseudo action.
Learned boundaries improve accuracy over random markers but use more tokens. Executing visual latent updates improves accuracy further while shortening the trace: +0.30 pp and 683 → 612 tokens relative to Learned gate. Relative to Random gate, the full model gains 0.75 pp at nearly the same length. Thus, judge-visible segmentation alone does not account for the full model's gain.
Two additional controls support this interpretation (Revised Table 2): at step 300, LatentWeave exceeds boundary-free Rubric RL by 0.53 pp at nearly matched length (1,137 vs. 1,148 tokens); at step 750, adding Visual KV improves accuracy by 0.16 pp and reduces length from 731 to 612 tokens. Figure 5 and Appendix E.1 report the full trajectories.
Requested baseline and computation comparison
We add ThinkRouter on the same Qwen3-VL-8B-Thinking backbone and six benchmarks (Revised Table 1), using temperature 0.6, default threshold 0.7, and top-10 embedding mixing without per-benchmark tuning.
For computation accounting, ThinkRouter uses soft routing on 90.61% of thinking steps (step-weighted; MathVision: 90.90%). This measured frequency weights the embedding-mixing cost in its amortized overhead estimate.
Overhead follows Revised Appendix C.3's per-output-token accounting. ThinkRouter's bound is 0.9090×0.00008192/15.136×100%; answer tokens only lower it. LatentWeave assumes 1,024 visual tokens. These are arithmetic estimates, not latency measurements. ThinkRouter is training-free; its default operating point is not token-budget matched.