acceptodds
Under review as a conference paper at ICLR 2027

DepLatent: Visual Dependency-Guided Latent Multimodal Reasoning

Abstract

Explicit chain-of-thought (CoT) improves multimodal reasoning but incurs substantial inference cost with long and verbose traces. Continuous latent reasoning reduces the number of generated tokens, yet it often underperforms explicit CoT on visually grounded tasks involved with multi-step planning. We first investigate this performance gap through fine-grained error analysis and image-masking interventions, suggesting that latent states may be weakly tied to task-relevant visual evidence and exhibit limited step-wise variation. In this paper, we introduce DepLatent, a dependency-guided latent supervision framework that uses region-wise image masking during explicit CoT reasoning to estimate dependencies between reasoning tokens and visual regions. These estimates provide two complementary training signals: dependency-weighted alignment between latent states and relevant visual features, and step-level reconstruction of CoT segments identified by changes in visual dependency. At inference, the multi-modal model reasons using compact continuous latent tokens, avoiding lengthy CoT generation. Across six multimodal reasoning benchmarks and three vision-language backbones with varied sizes, DepLatent consistently improves reasoning accuracy over explicit CoT and latent reasoning baselines while significantly reducing inference latency. Notably, it achieves an average 6.2% improvement over the original Qwen2.5-VL-7B model and speedups over explicit CoT of 2.47 on the BLINK benchmark.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
CoT, Latent Reasoning, VLM

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_KavdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors study the gap between explicit chain-of-thought reasoning and latent multimodal reasoning. In this paper, the authors propose DepLatent, which uses region-wise masking to estimate visual dependencies. These signals supervise visual alignment and step-level reconstruction in the latent space. Experiments on three VLM backbones and six benchmarks show consistent gains over latent reasoning baselines, with lower inference latency than explicit CoT.

Critical strengths
  1. I think the problem is important. Improving latent multimodal reasoning while preserving inference efficiency is a meaningful direction.

  2. The empirical gains are quite strong across multiple backbones and benchmarks. DepLatent consistently improves over Heima and other latent reasoning baselines.

  3. The method achieves substantial speedups over explicit CoT while maintaining or improving accuracy.

Critical weaknesses
  1. The validation of the visual dependency signal is not sufficiently independent of its construction. Eq. 3 defines dependency through masking-induced likelihood changes, while Fig. 3(b) evaluates it using masking-induced performance drops. Although this provides some downstream evidence, it does not establish whether the identified regions correspond to semantically meaningful reasoning evidence.

  2. I am not fully convinced by the claim that weak visual grounding is the main reason latent reasoning underperforms explicit CoT. The evidence mainly relies on hidden-state sensitivity under image masking, but the connection between this metric and reasoning quality is not sufficiently justified.

  3. There seem to be inconsistencies in the objective formulation. Eq. 6 uses positive cosine similarity inside a loss that is minimized, and the term S[m,n]S[m,n] contains an unbound index mm. Eq. 7 also reconstructs LvL_v segments but sums over LkL_k. These issues make the actual training objective difficult to verify. I don't know whether the authors have put enough attention on the manuscript.

  4. The method introduces substantial training complexity. DepLatent requires region-wise masking, dependency-map computation, an additional visual alignment module, and a reconstruction decoder. For Llama-3.2-11B, training time increases from 48h to 64h compared with Heima.

Questions for authors

Alongside the weakness, I have the following questions:

  1. Could the authors provide an independent evaluation showing that high-dependency regions correspond to semantically meaningful visual evidence, rather than simply regions that cause large likelihood changes when masked?

  2. Could the authors clarify how the hidden-state distributions in the JSD-based visual sensitivity analysis are constructed, and why this metric is a reliable proxy for visual grounding?

  3. Is the sign in Eq. 6 correct? If the objective is minimized, why does the visual alignment term use positive cosine similarity? Moreover, where the mm in Eq6 comes from?

  4. In Eq. 7, should the summation index run over LvL_v instead of LkL_k?

  5. What is the additional computational overhead of dependency-map construction, especially for larger backbones?

3 comments

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate the recognition of the significance of our research problem (wLNR, Kavd), the value of our empirical analysis of latent visual reasoning (UZ3v, wLNR), the motivation behind our dual-factorized supervision framework (UZ3v, wLNR), and the substantial performance gains and strong inference efficiency achieved across multiple backbones (UZ3v, wLNR, Kavd).

Below we address your questions:

W1&Q1. Validation of Visual Dependency Signal

We thank the reviewer for raising this concern. As suggested, we conducted a new independent evaluation by comparing model-selected high-dependency (threshold = 0.7) regions with human-annotated reasoning-relevant regions. Using human annotations as the reference, the comparison yields an overall precision of 92.4%, recall of 87.2%, and F1 score of 89.7%, indicating that high-dependency regions capture human-recognized reasoning evidence. We also provide qualitative visualizations in Appendix B.

Table R1. Validation of visual dependency signal with human review on 100 randomly sampled data. H: human-selected regions; M: model-selected regions; C: matched regions. Precision =C/M= C/M; Recall =C/H= C/H; F1 =2C/(H+M)= 2C/(H+M).

DatasetExamplesHuman (H)Model (M)Matched (C)Precision (%)Recall (%)F1 (%)
EMMA5016815814591.886.389.0
MathVista5015314513593.188.290.6
Overall10032130328092.487.289.7

W2&Q2. Weak Visual Grounding and JSD

We understand the reviewer’s concern, and our results suggest that weak visual grounding is an important factor in the performance degradation of latent reasoning in the following aspects:

  • Error patterns. We compare the error patterns of latent and explicit reasoning based on 200 samples with the same model backbone and training data. Perception errors account for a substantial proportion of failures in latent reasoning.
  • Performance with visual alignment: Fig. 4(a) shows that dependency-weighted visual alignment outperforms random and uniform alignment, supporting the importance of enhancing reasoning-relevant visual evidence for latent reasoning.
  • Hidden-state sensitivity based on JSD. We apologize for the confusion caused by “hidden-state distributions”. To evaluate the sensitivity on the visual region, we first obtain the last-layer reasoning states under the original and masked visual inputs and apply softmax along the hidden-feature dimension of each state. We then compute JSD between the resulting distributions at corresponding reasoning positions and average over these positions. For explicit reasoning, we use the same teacher-forced reasoning sequence under both inputs to align reasoning positions; for latent reasoning, we compare states at the same latent positions.

1. From an information-theoretic perspective, this metric quantifies the change in normalized hidden features induced by removing visual information. A higher value suggests that the visual input contributes substantial information to the reasoning trace. 2. The visualizations in Appendix B show the qualitative interpretation. 3. We evaluate this metric on Qwen2.5 and Qwen3 across diverse domains. As shown in Figure 2(b), latent reasoning models consistently show substantially lower visual sensitivity, supporting the consistency of this finding across models and tasks.

We will carefully revise it according to your valuable comments.

W3&Q3&Q4. Training Formulas

We apologize for the notation errors. We use the cosine distance 1−cos⁡1-\cos, averaged over reasoning tokens and visual regions. Equation (7) sums to LvL_v. These notation corrections do not change the actual implementation.

W4&Q5. Training costs

Thank you for raising the concern regarding the training cost. We agree that DepLatent introduces both implementation complexity and computational overhead. As shown in Table 1, DepLatent outperforms Heima by 7.0% on Llama-3.2-11B-Vision-Instruct, 7.4% on Qwen3-VL-8B-Instruct, and 6.1% on Qwen2.5-VL-7B-Instruct. Therefore, we view this increase in training GPU-hours as a reasonable trade-off in exchange for the performance gains during inference.

New results on larger models. Due to resource limits, we have tried our best to compare our model with Heima using Qwen2.5-VL-32B-Instruct. Both of them are trained with the same training settings: using Qwen2.5-32B-Instruct as the decoder, training steps=1K. As shown in Table R2, our DepLatent can improve over Heima by 5.4%, effectively extending into larger model backbones using a modest increase in training hours.

Table R2: Training cost (GPU-hours) and performance comparison across model backbones. GPU-hours denote cumulative GPU usage across all GPUs; total GPU-hours include dependency-map construction and latent training.

BackboneMethodMap construction (GPU-hours)Map storageLatent training (GPU-hours)Total (GPU-hours)EMMA accuracy
Qwen2.5-VL-32B-InstructHeima———~5832.3
Qwen2.5-VL-32B-InstructDepLatent~180.90GB~50~6837.7
Llama-3.2-11B-VisionHeima———~4820.4
Llama-3.2-11B-VisionDepLatent~140.91GB~50~6430.7
Qwen2.5-VL-7B-InstructHeima———~2422.6
Qwen2.5-VL-7B-InstructDepLatent~60.63GB~20~2634.3
Qwen3-VL-8B-InstructHeima———~2724.1
Qwen3-VL-8B-InstructDepLatent~80.86GB~23~3134.8

Thanks for the time, I think the answers have addressed my concerns, no more questions. I'll raise my score to 3

Reviewer_wLNROpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper first identifies that continuous latent multimodal reasoning is less sensitive to task-relevant visual evidence than explicit chain-of-thought reasoning, as suggested by its error analysis and image-masking interventions. Based on this observation, it proposes DepLatent, a dependency-guided supervision framework for compact latent reasoning. In teacher mode, the method estimates token-region dependencies by masking clustered visual regions and measuring the resulting drop in reasoning-token log-probability. In student mode, saliency-magnitude latent tokens are aligned with dependency-weighted visual features, while shift-direction latent tokens reconstruct CoT segments defined by changes in visual dependency. The auxiliary projector and decoder are removed at inference. Experiments cover three vision-language backbones and six multimodal reasoning benchmark families using LLaVA-CoT-100K.

Critical strengths
  1. Useful analysis of latent visual grounding: this paper analyzes error types on 200 examples, measures how explicit and latent reasoning states respond to image masking, and contrasts the effect of masking high- and low-dependency regions. These analyses lead to a coherent hypothesis: compact latent states underuse task-relevant visual evidence.

  2. The proposed method is technically sound. Region masking converts visual dependency into a training signal. Separating saliency-magnitude latents from shift-direction latents is also a sensible attempt to model visual evidence and reasoning-step evolution separately.

  3. The empirical study is broad and well ablated. Experiments cover three VLM backbones and multi-discipline, mathematical, and vision-centric tasks.

Critical weaknesses
  1. The visual-dependency diagnostic is not defined precisely. Figure 2 reports Jensen-Shannon divergence between hidden-state "distributions" without explaining how hidden vectors are converted into probability distributions. This metric supports the main motivation and should be reproducible.

  2. Training and efficiency reporting are incomplete. Key implementation details, including the optimizer, learning rate, batch size, training steps, loss weights, and clustering threshold are missing. Table 7 does not clarify whether the 16-48 masked evaluations used to build dependency maps are included, while Table 2 omits the inference device, precision, decoding settings, output lengths, repetitions, and variance.

  3. The core mathematical specification is inconsistent. In the current form, Equation 6 uses positive cosine similarity inside a minimized loss, contains a free index m, and is inconsistent with Equation 8. Equation 7 sums to L_k although the text defines L_v reasoning segments, while the shape of Q is unclear between Equations 4 and 5.

Questions for authors

Q1. Please provide the exact implemented forms of Equations 4-7, including the shape of Q. Should Equation 6 use negative cosine or 1-cosine and sum S[m,n] over m? Should Equation 7 sum to L_v?

Q2. How are last-layer hidden states converted into probability distributions for the Jensen-Shannon divergence in Figure 2?

Q3. Are dependency maps precomputed? Please report their full GPU/wall-clock and storage cost and clarify whether Table 7 includes this stage.

Q4. More impmentation details should be inclucded as well.

3 comments

We sincerely thank all reviewers for their thoughtful and constructive feedback. We appreciate the recognition of the significance of our research problem (wLNR, Kavd), the value of our empirical analysis of latent visual reasoning (UZ3v, wLNR), the motivation behind our dual-factorized supervision framework (UZ3v, wLNR), and the substantial performance gains and strong inference efficiency achieved across multiple backbones (UZ3v, wLNR, Kavd).

Below we address your questions:

W1&Q2. Jensen-Shannon divergence

We apologize for the confusion caused by “hidden-state distributions.” To evaluate the sensitivity to the visual region, we first obtain the last-layer reasoning states under the original and masked visual inputs and apply softmax along the hidden-feature dimension of each state. We then compute JSD between the resulting distributions at corresponding reasoning positions and average over these positions. For explicit reasoning, we use the same teacher-forced reasoning sequence under both inputs to align reasoning positions; for latent reasoning, we compare states at the same latent positions.

Task and model generalizability. We evaluate this metric on Qwen2.5 and Qwen3 across diverse domains: mathematics, physics, chemistry, and coding. As shown in Figure 2(b), latent reasoning models consistently show substantially lower visual sensitivity, supporting the consistency of this finding across models and tasks.

We will carefully revise it according to your valuable comments.

W2&Q4. Training GPU Hours and Inference Acceleration

We thank the reviewer for these helpful suggestions. For a fair comparison, vanilla SFT, Heima, and DepLatent using the same backbone are compared under identical settings. The training and inference settings of our method are shown in Table R1 and Table R2.

Table R1. Training settings.

SettingLlama-3.2-11B-VisionQwen3-VL-8BQwen2.5-VL-7B
OptimizerAdamWAdamWAdamW
Learning rate10−410^{-4}10−510^{-5}10−510^{-5}
Distributed trainingFSDPDeepSpeed ZeRO-3DeepSpeed ZeRO-3
Weight decay0.010.10.1
Warm-up steps100500500
Batch size888
Training duration1 epoch4,000 steps4,000 steps
λans\lambda_{\mathrm{ans}}1.01.01.0
λlogic\lambda_{\mathrm{logic}}0.80.80.8
λvisual\lambda_{\mathrm{visual}}0.80.80.8
Clustering threshold0.340.340.34

Table R2. Inference settings.

SettingValue
Inference deviceNVIDIA RTX PRO 6000 GPU
PrecisionBF16
Inference batch size1
Decoding methodSampling
Top-k1
Number of beams1
Temperature0.01
Maximum new tokens2,048
  • Loss weights analysis. Tables R3 and R4 show the hyperparameter analysis of weights λlogic\lambda_{\text{logic}} and λvisual\lambda_{\text{visual}}, demonstrating the effectiveness of our dual supervision design. We set λans\lambda_{\text{ans}}, λlogic\lambda_{\text{logic}} and λvisual\lambda_{\text{visual}} as 1.0, 0.8, and 0.8.

Table R3. Parameter analysis of λvisual\lambda_{\text{visual}} (λlogic=0.8\lambda_{\text{logic}} = 0.8) with the Qwen2.5-VL-7B-Instruct model.

λvisual\lambda_{\mathrm{visual}}00.20.40.60.81.0
EMMA30.628.533.533.134.334.0
Rel. Depth78.282.983.284.184.183.6

Table R4. Parameter analysis of λlogic\lambda_{\text{logic}} (λvisual=0.8\lambda_{\text{visual}} = 0.8) with the Qwen2.5-VL-7B-Instruct model.

λlogic\lambda_{\mathrm{logic}}00.20.40.60.81.0
EMMA32.830.534.232.534.333.8
Rel. Depth82.083.380.282.284.183.9
  • Clustering threshold analysis. We use agglomerative clustering and cosine distance, defined as 1-cosine similarity. As shown in Table R5, a smaller distance threshold yields more fine-grained regions, whereas a larger threshold allows more merges and produces fewer, coarser regions. We set the distance threshold to 0.34 in our work.

Table R5. Effect of the clustering threshold on visual region counts and downstream accuracy with Qwen-2.5-7B-Instruct. Min, Median, and Max denote the minimum, median, and maximum numbers of visual regions per example across each benchmark.

ThresholdEMMA: MinEMMA: MedianEMMA: MaxEMMA: Acc.MathVista: MinMathVista: MedianMathVista: MaxMathVista: Acc.MMStar: MinMMStar: MedianMMStar: MaxMMStar: Acc.
0.3024384832.622364870.826404863.1
0.3422344634.320324572.724364767.5
0.3820304333.518294271.421324466.2
0.4218273931.717253869.619294164.8
0.4616233529.016223468.216253765.9
  • Inference acceleration. We additionally report the number of generated tokens and the breakdown of inference latency under three repetitions in Table R6: we can observe that our model can improve performance by 3.2% with the Qwen2.5-VL-7B-instruct model compared with the explicit SFT-based reasoning method while retaining the 2.5x end-to-end speedups on the Rel. Depth task.

Table R6. Latency and generated tokens under three repetitions. Latency is reported as mean ± standard deviation.

MethodRel. Depth: Output tokensRel. Depth: Visual prefill (s)Rel. Depth: Total latency (s)MathVerse: Output tokensMathVerse: Visual prefill (s)MathVerse: Total latency (s)
Llama-3.2-11B- vision180.2≈0.06\approx 0.065.900±0.0595.900 \pm 0.059400.1≈0.05\approx 0.0513.050±0.13013.050 \pm 0.130
LLaVA-CoT380.3≈0.06\approx 0.0612.590±0.11512.590 \pm 0.115632.0≈0.05\approx 0.0516.810±0.15716.810 \pm 0.157
Ours64.8≈0.06\approx 0.062.170±0.0232.170 \pm 0.02329.2≈0.05\approx 0.050.930±0.0130.930 \pm 0.013
Qwen2.5-VL-7B50.3≈0.22\approx 0.221.430±0.0141.430 \pm 0.014686.9≈0.17\approx 0.1710.900±0.10910.900 \pm 0.109
Qwen2.5-VL-7B + vanilla SFT65.6≈0.22\approx 0.221.800±0.0201.800 \pm 0.020963.4≈0.17\approx 0.1715.510±0.15415.510 \pm 0.154
Ours23.9≈0.22\approx 0.220.730±0.0030.730 \pm 0.00313.7≈0.17\approx 0.170.410±0.0060.410 \pm 0.006

W3&Q1. Training Formulas

Thanks for pointing this out. We use the cosine distance 1−cos⁡1-\cos, averaged over reasoning tokens and visual regions. Equation (7) sums to LvL_v. The shape of QQ is the same as Xvis\mathbf{X}_\text{vis}. These notation corrections do not change the actual implementation.

Q3. Training costs

We sincerely thank the reviewer for highlighting this point. We report the breakdown of training costs in GPU-hours and the storage cost in Table R7. Compared with Heima in Table 1, DepLatent improves average accuracy by 7.0% on Llama-3.2-11B-Vision-Instruct, 7.4% on Qwen3-VL-8B-Instruct, and 6.1% on Qwen2.5-VL-7B-Instruct. Therefore, we view this increase in training GPU-hours as a reasonable trade-off in exchange for the performance gains and substantial speedup during inference.

Table R7: Training cost analysis in GPU-hours on the same hardware (NVIDIA RTX PRO 6000).

MethodBackboneMap construction (GPU-hours)Map storageLatent training (GPU-hours)Total (GPU-hours)
HeimaLlama-3.2-11B-Vision———~48
DepLatentLlama-3.2-11B-Vision~140.91GB~50~64
HeimaQwen2.5-VL-7B-Instruct———~24
DepLatentQwen2.5-VL-7B-Instruct~60.63GB~20~26
HeimaQwen3-VL-8B-Instruct———~27
DepLatentQwen3-VL-8B-Instruct~80.86GB~23~31
Reviewer_UZ3vOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DepLatent transfers supervision from explicit multimodal chain-of-thought into two groups of continuous latent tokens. Region-wise attention masking estimates token-to-region dependencies. These weight visual alignment for one group and determine CoT segments that the second group learns to reconstruct as text. The reconstruction decoder is removed at inference. Experiments cover three VLM backbones and six benchmark families, comparing explicit SFT and latent reasoning baselines, ablating components, and measuring inference and training cost.

Critical strengths
  1. DepLatent uses task-dependent visual evidence to guide both what latent states preserve and how step-level reconstruction is organized. This gives the two supervision signals a common rationale grounded in the reasoning trace. The component ablations in Table 3 report higher scores for the combined model than for either latent group alone. I view this supervision design and its empirical support as the main methodological contribution.

  2. Table 1 reports gains across three backbones, with a common 100K-example training source for the main SFT/Heima comparisons. On Qwen2.5-VL-7B, DepLatent reaches 59.5 overall, compared with 53.3 for the base model, 55.4 for vanilla SFT, and 53.6 for Heima; EMMA improves from 21.5 to 34.3 relative to the base model. Table 2 also reports lower inference latency than explicit CoT. These accuracy and efficiency results are the main basis for my weak-accept recommendation, with the reporting issues below still requiring clarification.

Critical weaknesses
  1. The loss equations need clarification. Equation (6) defines a positive weighted cosine similarity, and Equation (8) adds it to the losses being minimized, which would discourage alignment if the visual weight is positive. The token index m is not explicitly aggregated, and Equation (7) sums to L_k despite reconstructing L_v segments. These may be notation errors. Expressions matching the implementation, including the aggregation and signed loss weights, are needed to distinguish a presentation problem from an error in the training objective.

  2. Figure 4 and the tables need to be reconciled before the finer-grained ablation claims can be assessed confidently. The orange bars for EMMA, MathVista, Rel. Refl., and Rel. Depth. appear roughly ten percentage points above the Qwen2.5-VL-7B values in Tables 1, 3, and 5, with no different setting identified. The scores 34.3, 72.7, 45.0, and 84.1 average to about 59.0, as in Table 4, whereas Table 3 gives 58.5. My positive recommendation gives more weight to the main cross-backbone comparisons, but the current discrepancies leave the advantage of dependency weighting and segmentation over simpler alternatives uncertain.

Questions for authors
  1. Please provide the visual-alignment and reconstruction losses as implemented. Is the visual term negative cosine similarity or cosine distance, how is m aggregated, and what are the weights and their signs? Should Equation (7) sum to L_v? Please clarify whether the corrections affect only the notation or also the reported experiments.

  2. Could you provide the numerical data and setting for Figure 4, reconcile its proposed-method bars with Tables 1/3/5, and correct or explain the Table 3 average? Do the gains over uniform/random alignment and fixed-interval segmentation hold after this reconciliation? This clarification matters for the claimed benefit of the specific supervision choices.

  3. Does Table 7 include constructing all 16–48 masked-region dependency maps per example, and are the maps cached or recomputed during training? Please also specify the batch size, decoding/output-length settings, and inclusion of visual prefill for Table 2, so the training overhead and inference savings can be compared fairly.

3 comments

We sincerely thank all reviewers for your thoughtful and constructive feedback. We appreciate the recognition of the significance of our research problem (wLNR, Kavd), the value of our empirical analysis of latent visual reasoning (UZ3v, wLNR), the motivation behind our dual-factorized supervision framework (UZ3v, wLNR), and the substantial performance gains and strong inference efficiency achieved across multiple backbones (UZ3v, wLNR, Kavd).

Below we address your questions:

W1&Q1. Training Formulas

Thanks for pointing this out. We use the cosine distance 1−cos⁡1-\cos, averaged over reasoning tokens and visual regions, with positive loss weights λans=1.0\lambda_{\mathrm{ans}}=1.0 and λlogic=λvisual=0.8\lambda_{\mathrm{logic}}=\lambda_{\mathrm{visual}}=0.8. Equation (7) sums to LvL_v. These notation corrections do not change the actual implementation and experimental results.

Tables R1 (a) and (b) show the hyperparameter analysis of weights λvisual\lambda_{\text{visual}} and λlogic\lambda_{\text{logic}}, demonstrating the effectiveness of our dual supervision design.

Table R1 (a). Parameter analysis of λvisual\lambda_{\text{visual}} (λlogic=0.8\lambda_{\text{logic}} = 0.8) with the Qwen2.5-VL-7B-Instruct model.

λvisual\lambda_{\mathrm{visual}}00.20.40.60.81.0
EMMA30.628.533.533.134.334.0
Rel. Depth78.282.983.284.184.183.6

Table R1 (b). Parameter analysis of λlogic\lambda_{\text{logic}} (λvisual=0.8\lambda_{\text{visual}} = 0.8) with the Qwen2.5-VL-7B-Instruct model.

λlogic\lambda_{\mathrm{logic}}00.20.40.60.81.0
EMMA32.830.534.232.534.333.8
Rel. Depth82.083.380.282.284.183.9

W2&Q2. Alignment and Segmentation

We apologize for the confusion caused by the label error. We revised the matplotlib labels in Figure 4, and the data for Figure 4 is shown in Table R2, showing the effectiveness of our alignment and segmentation strategy. For a fair comparison, variants using the same backbone are compared under identical training (Tables R3) and inference (Table R4) settings.

Table R2 (a). Ablation study of alignment strategy with Qwen2.5-VL-7B-Instruct model.

MethodEMMAMathVistaRel. Ref.Rel. Depth.
Random25.965.636.575.9
Uniform27.864.242.381.7
Ours34.372.745.084.1

Table R2 (b). Ablation study of segmentation strategy with Qwen2.5-VL-7B-Instruct model.

MethodEMMAMathVistaRel. Ref.Rel. Depth.
w/o Seg.31.270.541.579.6
Fixed-interval30.168.736.278.9
Ours34.372.745.084.1

Table R3. Training settings.

SettingLlama-3.2-11B-VisionQwen3-VL-8BQwen2.5-VL-7B
OptimizerAdamWAdamWAdamW
Learning rate10−410^{-4}10−510^{-5}10−510^{-5}
Distributed trainingFSDPDeepSpeed ZeRO-3DeepSpeed ZeRO-3
Weight decay0.010.10.1
Warm-up steps100500500
Batch size888
Training duration1 epoch4,000 steps4,000 steps
λans\lambda_{\mathrm{ans}}1.01.01.0
λlogic\lambda_{\mathrm{logic}}0.80.80.8
λvisual\lambda_{\mathrm{visual}}0.80.80.8
Clustering threshold0.340.340.34

Table R4. Inference settings.

SettingValue
Inference deviceNVIDIA RTX PRO 6000 GPU
PrecisionBF16
Inference batch size1
Decoding methodSampling
Top-k1
Number of beams1
Temperature0.01
Maximum new tokens2,048

Q3. Training GPU-hours and Inference Acceleration

We sincerely thank the reviewer for highlighting this point. We additionally report the number of generated tokens and the breakdown of inference latency under three repetitions in Table R5, showing that our model can improve performance by 3.2% with the Qwen2.5-VL-7B-instruct model compared with the explicit SFT-based reasoning method while retaining the 2.5x end-to-end speedups on Rel. Depth tasks. The training costs in Table 7 include the construction of dependency maps, which are precomputed and cached to reduce the training costs.

Therefore, we view this increase in training GPU-hours as a reasonable trade-off in exchange for the performance gains and substantial speedup during inference.

Table R5. Latency and generated tokens under three repetitions. Latency is reported as mean ± standard deviation.

MethodRel. Depth: Output tokensRel. Depth: Visual prefill (s)Rel. Depth: Total latency (s)MathVerse: Output tokensMathVerse: Visual prefill (s)MathVerse: Total latency (s)
Llama-3.2-11B- vision180.2≈0.06\approx 0.065.900±0.0595.900 \pm 0.059400.1≈0.05\approx 0.0513.050±0.13013.050 \pm 0.130
LLaVA-CoT380.3≈0.06\approx 0.0612.590±0.11512.590 \pm 0.115632.0≈0.05\approx 0.0516.810±0.15716.810 \pm 0.157
Ours64.8≈0.06\approx 0.062.170±0.0232.170 \pm 0.02329.2≈0.05\approx 0.050.930±0.0130.930 \pm 0.013
Qwen2.5-VL-7B50.3≈0.22\approx 0.221.430±0.0141.430 \pm 0.014686.9≈0.17\approx 0.1710.900±0.10910.900 \pm 0.109
Qwen2.5-VL-7B + vanilla SFT65.6≈0.22\approx 0.221.800±0.0201.800 \pm 0.020963.4≈0.17\approx 0.1715.510±0.15415.510 \pm 0.154
Ours23.9≈0.22\approx 0.220.730±0.0030.730 \pm 0.00313.7≈0.17\approx 0.170.410±0.0060.410 \pm 0.006

Thank you for taking the time to address my questions. Your answers have resolved my concerns, so I will increase my overall score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.