acceptodds
Under review as a conference paper at ICLR 2027

Confident Thinking: Multi-Layer Entropy-Guided Latent Reasoning for Multimodal Long-Context RAG

Abstract

Multimodal long-context retrieval-augmented generation (MLRAG) is increasingly important for knowledge-intensive question answering, as it enables models to retrieve and reason over extensive textual and visual evidence. However, during answer generation, multimodal large language models remain prone to hallucination when synthesizing retrieved cross-modal evidence, particularly when the evidence is heterogeneous, redundant, or conflicting. Existing methods commonly employ explicit chain-of-thought (CoT) to facilitate evidence reasoning, but its token-by-token generation forces the model to commit to discrete intermediate states, allowing unreliable predictions to propagate through subsequent steps and ultimately yield hallucinated answers. To address this problem, we propose Confident Thinking, a confidence-guided framework that switches from explicit to latent reasoning upon detecting genuine reasoning uncertainty, preserving candidate semantics in continuous space while incorporating retrieved evidence to guide subsequent updates. First, we introduce a multi-layer entropy consensus mechanism that evaluates predictive uncertainty across the last few Transformer layers and triggers latent reasoning only when all these layers exhibit high entropy, reducing unnecessary switches caused by final-layer-specific ambiguity. Second, we develop an evidence-calibrated soft-thinking mechanism that dynamically selects question- and state-relevant multimodal evidence and injects it into soft tokens through uncertainty-aware gating, constraining recursive latent updates to mitigate reasoning drift and error accumulation. Extensive experiments on multiple multimodal long-context question-answering benchmarks demonstrate that Confident Thinking consistently improves answer accuracy and overall performance across different settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Long-Context RAG, Latent Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KmxuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Confident Thinking, a training-free decoding framework for multimodal long-context RAG that switches from explicit chain-of-thought reasoning to latent soft reasoning when genuine uncertainty is detected. Its two core components are MEC, which activates latent reasoning only when uncertainty persists across multiple late Transformer layers rather than being confined to the final layer, and ECST, which grounds latent updates in dynamically selected multimodal evidence. Experiments show consistent accuracy improvements, while ablations demonstrate that soft reasoning without MEC actually degrades performance.

Critical strengths
  1. The distinction between final-layer-specific pseudo-confusion and persistent cross-layer uncertainty is conceptually compelling. The empirical finding that naïve final-layer entropy triggering hurts performance (48.49 vs. 50.01) while MEC recovers and improves upon it (51.65) provides strong evidence that the multi-layer consensus design is not merely additive but corrective.

  2. The framework requires no additional training and is applied purely at inference time, using the same hyperparameters across three distinct MLLM backbones. This practicality is a genuine advantage for researchers and practitioners who wish to improve reasoning without costly fine-tuning.

  3. Table 3 isolates the contribution of each component step by step. This progressive decomposition convincingly shows that the components are complementary and that MEC is the critical enabler.

  4. The qualitative examples in Figure 4 effectively demonstrate how the method mitigates evidence drift and avoids premature commitment under semantically mismatched or out-of-scope retrieved content, grounding the quantitative gains in concrete failure modes.

Critical weaknesses
  1. While improvements on MMLongBench are substantial (+2.95 to +5.32 points), the gains on DocBench are highly backbone-dependent: Qwen3.6-27B and GLM-4.6V improve by +3.20 each, but Gemma-4-31B-it gains only +0.21 points. This suggests the method’s value diminishes with stronger base models, limiting its universal appeal.

  2. Although Coconut, Soft Thinking, DMLR, and SoftCoT are discussed in related work, none are directly evaluated on the same benchmarks. Without these head-to-head comparisons, it is difficult to assess whether the gains come from latent reasoning per se or specifically from the proposed MEC and evidence calibration mechanisms.

  3. The method requires access to intermediate Transformer hidden states and the language-model head, which prevents application to popular black-box API models such as GPT-4o, Gemini, and Claude. Given that many production multimodal RAG pipelines rely on proprietary APIs, this substantially narrows the method’s practical applicability.

Questions for authors

see weakness

5 comments

[W1]: While improvements on MMLongBench are substantial (+2.95 to +5.32 points), the gains on DocBench are highly backbone-dependent: Qwen3.6-27B and GLM-4.6V improve by +3.20 each, but Gemma-4-31B-it gains only +0.21 points. This suggests the method’s value diminishes with stronger base models, limiting its universal appeal.

We thank the reviewer for highlighting the variation across backbones. We agree that the improvement for Gemma-4-31B-it on DocBench is modest, and that the magnitude of the benefit is not uniform across the evaluated settings.

However, we respectfully suggest that the results do not establish a general trend of diminishing benefits with stronger backbones. On MMLongBench-Doc, Gemma-4-31B-it has the highest baseline accuracy among the three evaluated backbones, yet also achieves the largest improvement: from 60.67 to 65.99 (+5.32 points), compared with +4.76 for Qwen3.6-27B and +2.95 for GLM-4.6V-Flash. Thus, the same strong backbone benefits substantially on one benchmark and only marginally on the other. The DocBench results further suggest that baseline accuracy alone does not explain this difference. Gemma and Qwen start from similar accuracies of 83.56 and 82.73, respectively, but obtain different gains. For Gemma, the category-level results are mixed: Text and Metadata improve by 0.80 and 0.46 points, respectively, while Multimodal decreases by 0.77 points. Our evidence therefore supports applicability across the evaluated backbones with varying performance gains, rather than uniformly large improvements in every setting. We appreciate the reviewer's concern about generality.

[W2] Although Coconut, Soft Thinking, DMLR, and SoftCoT are discussed in related work, none are directly evaluated on the same benchmarks. Without these head-to-head comparisons, it is difficult to assess whether the gains come from latent reasoning per se or specifically from the proposed MEC and evidence calibration mechanisms.

We thank the reviewer for this suggestion. We have completed additional comparisons with Soft Thinking [2], DMLR [3], and SoftCoT [4] on MMLongBench-Doc. Regarding Coconut, its original method requires CoT-supervised curriculum training. Since MMLongBench-Doc does not provide a separate CoT-annotated training set, adapting Coconut would require additional training-data preparation and model training, which we have not undertaken. For these comparisons, we retain the same RAG retrieval pipeline and replace the generation-stage reasoning method with each baseline. To accommodate DMLR’s image-conditioned reasoning mechanism, we convert the retrieved evidence into images and provide them together with the question. These additional experiments are now included in Section 4.4 of the revised manuscript.

MethodBackboneAcademicBrochureFinancialGuidebookIndustryReportTutorialACC ↑F1 ↑
Soft ThinkingQwen3.6-27B51.2356.6348.6870.6553.7348.9749.7853.6352.34
DMLRQwen3.6-27B41.7136.0061.5440.6561.7347.9534.0645.3840.97
Confident ThinkingQwen3.6-27B51.9162.7549.7172.2254.9750.5151.8554.7755.48
Soft ThinkingGLM-4.6V-Flash38.6950.0039.9952.8145.6238.3642.0342.7538.44
DMLRGLM-4.6V-Flash37.6934.0041.8838.7159.2643.1530.4340.1134.34
Confident ThinkingGLM-4.6V-Flash41.0757.0041.1755.5646.8640.7542.5243.7239.49
Soft ThinkingGemma-4-31B-it60.4860.2062.6567.2454.6067.5967.2663.9960.80
DMLRGemma-4-31B-it41.3339.8058.1240.7961.7347.5932.3545.0540.02
Confident ThinkingGemma-4-31B-it61.8965.3163.1874.5858.7169.0767.6965.9961.15

As shown in Table 1, Confident Thinking achieves higher overall ACC and F1 than Soft Thinking and the adapted DMLR baseline across all three backbones. Compared with Soft Thinking, ACC improves by 1.14 points on Qwen3.6-27B, 0.97 points on GLM-4.6V-Flash, and 2.00 points on Gemma-4-31B-it. Together with the component ablations in Table 3 of the paper, these results support the benefits of MEC and evidence calibration beyond latent reasoning alone.

For SoftCoT, we use Qwen2.5-7B-Instruct, following the Qwen backbone configuration evaluated in its original paper. Qwen2.5-7B-Instruct itself has a limited context window, and some samples exceed this limit when accounting for the complete input context and the reserved generation budget. We therefore evaluate SoftCoT, DocAgent, and Confident Thinking on the same subset of MMLongBench-Doc that fits within this limit, preserving the retrieved evidence without truncation.

MethodBackboneAcademicBrochureFinancialGuidebookIndustryReportTutorialACC ↑F1 ↑
DocAgentQwen2.5-7B59.4041.1054.8441.3066.6750.6444.1450.8748.40
SoftCoTQwen2.5-7B57.8938.3658.0644.5766.6751.0645.9551.2749.87
Confident ThinkingQwen2.5-7B53.3842.4754.8440.2268.0656.6045.0551.9450.05

Table R2 compares the methods using the same Qwen2.5-7B backbone. Confident Thinking outperforms both DocAgent and SoftCoT in overall ACC and F1. Compared with SoftCoT, ACC increases from 51.27 to 51.94 and F1 from 49.87 to 50.05. These gains are achieved without additional training, whereas SoftCoT relies on a trained projection module.

[W3]: The method requires access to intermediate Transformer hidden states and the language-model head, which prevents application to popular black-box API models such as GPT-4o, Gemini, and Claude. Given that many production multimodal RAG pipelines rely on proprietary APIs, this substantially narrows the method's practical applicability.

We thank the reviewer for this comment. As acknowledged in Appendix A.4 (Limitations), Confident Thinking requires access to model internals and therefore cannot be applied through standard black-box APIs. Our method targets open-weight models whose inference process can be directly controlled.

We recognize the importance of API-based pipelines. However, some multimodal RAG applications use locally deployed open-weight models to meet data privacy requirements. For these applications, Confident Thinking provides a training-free approach to improving generation reliability, supported by our evaluations on three open-weight multimodal backbones. Beyond this deployment setting, our findings on cross-layer uncertainty and evidence-calibrated latent reasoning may also inform reasoning mechanism design by developers with access to model internals.

Reviewer_foy5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Confident Thinking, a training-free reasoning framework for multimodal long-context RAG. The method identifies reasoning uncertainty through a multi-layer entropy consensus (MEC) and dynamically integrates retrieved evidence using evidence-calibrated soft reasoning (ECST) to reduce error propagation and hallucinations. Experiments were conducted on MMLongBench-Doc and DocBench, showing performance improvements across different models, and ablation studies confirmed the effectiveness of the main modules.

Critical strengths
  • The method design is quite targeted. It combines multi-level entropy judgment with evidence-calibrated soft reasoning to separately handle reasoning uncertainty identification and evidence constraint issues, and the overall design is reasonable.

  • The cross-model experiments and ablation tests are fairly thorough. Performance improvements were verified on three different models and two benchmarks, and the ablation experiments also show the independent contributions of MEC and ECST.

Critical weaknesses
  • Key method details are insufficient. ECST doesn't clearly explain how the correlation of multimodal evidence is calculated, how the evidence representation is built, or how the calibration weights are determined. These details directly affect the reproducibility of the method.

  • There's a lack of fair comparison with directly related methods. The main experiments use DocAgent as the baseline, but there's no comparison with related reasoning methods like Soft Thinking and DMLR under the same settings, nor is the actual reasoning delay quantified, making it hard to fully judge the method's relative advantages.

Questions for authors
  • Can you clarify the specific implementation of multimodal evidence selection, representation construction, and weight calibration in ECST?

  • Could you also add a comparison with the unified setups of Soft Thinking and DMLR, and report the actual inference delay?

4 comments

[W1&Q1] Can you clarify the specific implementation of multimodal evidence selection, representation construction, and weight calibration in ECST?

We thank the reviewer for this helpful comment. We agree that several implementation details of ECST were not sufficiently explained in the original manuscript. In ECST, we construct evidence representations using the backbone's existing multimodal encoding pipeline. Textual evidence is represented using token embeddings, while visual evidence is processed by the vision encoder and mapped into the language model's embedding space through the multimodal projector. Mean pooling is then applied to obtain evidence vectors compatible with the soft tokens. At each soft-thinking step, we compute cosine similarity between the retrieved evidence representations and the query representation incorporating the input question and current reasoning state, selecting the top-kk most relevant items (k=3k=3). The selection is updated as reasoning progresses to reflect changes in evidence relevance.

As defined in Eq. (5), the selected representations are averaged to form the evidence anchor, which is combined with the original soft token as

\widetilde{s}_t = s_t + \lambda(a_t-s_t).$$ Here, $\lambda=0.3$ is fixed, assigning weights of $0.3$ and $0.7$ to the evidence anchor and original soft token, respectively. We have revised Sections 3.2 and 4.1 to clarify these implementation details. >[W2&Q2] Could you also add a comparison with the unified setups of Soft Thinking and DMLR, and report the actual inference delay? We thank the reviewer for highlighting the need for direct comparisons with closely related reasoning methods. We have evaluated Soft Thinking and DMLR on MMLongBench-Doc using the same RAG retrieval pipeline and backbones as Confident Thinking, replacing the generation-stage reasoning method with each baseline. DMLR was originally evaluated on multimodal reasoning tasks with given images and questions, while our task requires answering questions using retrieved document evidence. We therefore convert the retrieved evidence into images and provide them together with the question. This enables DMLR's visual injection mechanism to use the retrieved information in our RAG setting. These additional experiments are now included in Section 4.4 of the revised manuscript. The results are summarized below. | Method | Backbone | Academic | Brochure | Financial | Guidebook | Industry | Report | Tutorial | ACC ↑ | F1 ↑ | |---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | Soft Thinking | Qwen3.6-27B | 51.23 | 56.63 | 48.68 | 70.65 | 53.73 | 48.97 | 49.78 | 53.63 | 52.34 | | DMLR | Qwen3.6-27B | 41.71 | 36.00 | **61.54** | 40.65 | **61.73** | 47.95 | 34.06 | 45.38 | 40.97 | | **Confident Thinking** | Qwen3.6-27B | **51.91** | **62.75** | 49.71 | **72.22** | 54.97 | **50.51** | **51.85** | **54.77** | **55.48** | | Soft Thinking | GLM-4.6V-Flash | 38.69 | 50.00 | 39.99 | 52.81 | 45.62 | 38.36 | 42.03 | 42.75 | 38.44 | | DMLR | GLM-4.6V-Flash | 37.69 | 34.00 | **41.88** | 38.71 | **59.26** | **43.15** | 30.43 | 40.11 | 34.34 | | **Confident Thinking** | GLM-4.6V-Flash | **41.07** | **57.00** | 41.17 | **55.56** | 46.86 | 40.75 | **42.52** | **43.72** | **39.49** | | Soft Thinking | Gemma-4-31B-it | 60.48 | 60.20 | 62.65 | 67.24 | 54.60 | 67.59 | 67.26 | 63.99 | 60.80 | | DMLR | Gemma-4-31B-it | 41.33 | 39.80 | 58.12 | 40.79 | **61.73** | 47.59 | 32.35 | 45.05 | 40.02 | | **Confident Thinking** | Gemma-4-31B-it | **61.89** | **65.31** | **63.18** | **74.58** | 58.71 | **69.07** | **67.69** | **65.99** | **61.15** | Confident Thinking achieves higher overall ACC and F1 than both baselines across all three backbones. Compared with Soft Thinking, ACC improves by 1.14 points on Qwen3.6-27B, 0.97 points on GLM-4.6V-Flash, and 2.00 points on Gemma-4-31B-it. The corresponding ACC gains over the adapted DMLR baseline are 9.39, 3.61, and 20.94 points, respectively. These results support the effectiveness of Confident Thinking relative to the evaluated latent reasoning baselines in multimodal long-context RAG.

Thank you for your reply. After considering the review of other reviewers, I believe the current results of the entropy-based research are reasonable. However, the current method I understand is more like a combination of variations of existing methods, such as multiple voting to measure uncertainty, or using the logits probabilities of the model to measure uncertainty, etc. The deterministic models with low entropy have already received a lot of research, and RAG is also a very mature research project. The current content may lack new insights to me. Therefore, I maintain the original rating.

Thank you for the follow-up and for acknowledging the empirical results. We appreciate the opportunity to clarify the remaining concern about novelty. We respectfully disagree that our contribution is adequately characterized as multiple voting or the routine use of predictive entropy.

1. Our task is multimodal long-context RAG over long documents. As DocAgent [1] highlights, answering a single question can require retrieving and connecting evidence scattered across sections and presented in different forms, such as text, tables, and figures. Even when the relevant pages have been retrieved, the generator must still determine which pieces of evidence support the same claim. Reliable generation from such heterogeneous evidence remains an important research problem within multimodal long-context RAG.

2. Our motivation is to reduce hallucinations caused by premature commitments. In Figure 4, both relevant charts are available, but the global figure of 8% is an aggregate for several earlier iOS versions. Once an explicit CoT step treats it as the rate for iOS 7 alone, subsequent steps inherit a false premise and produce a comparison the charts do not support. We introduce latent reasoning so that, when an interpretation is uncertain, the model can retain multiple plausible continuations and use the retrieved evidence before committing to one.

3. Cross-layer entropy determines when to switch to latent reasoning; retrieved evidence guides each latent step. Our contribution is to distinguish two cases that both exhibit high final-layer entropy but call for different actions. Even if three of the last four layers have high entropy, MEC can continue discrete decoding from the remaining intermediate layer if it meets the low-entropy and entropy-gap criteria. We call this pseudo-confusion. Only when high entropy persists across all four layers over consecutive steps does MEC treat it as genuine confusion and switch from explicit CoT to latent reasoning. This decision does not depend on agreement among predicted tokens or sampled answers. Once in latent space, ECST selects retrieved evidence based on the question and current reasoning state, then mixes the resulting evidence anchor into the soft token used for the next step.

Table 3 illustrates why this distinction matters. On MMLongBench-Doc with Qwen3.6-27B, triggering soft thinking from final-layer entropy alone yields 48.49 ACC, below standard CoT (50.01). MEC with selective soft thinking reaches 53.92 before evidence calibration; adding the evidence anchor raises ACC to 54.77.

[1] Sun et al. (2025). DocAgent: An Agentic Framework for Multi-Modal Long-Context Document Understanding. EMNLP.

We wanted to follow up on our latest reply: did our explanation of when Confident Thinking switches to latent reasoning and how retrieved evidence guides each latent step address your concern about novelty? If any questions remain, we’d be happy to discuss them. If you feel our response has addressed your concerns, would you be open to revisiting your assessment? We’d welcome your thoughts.

Reviewer_QLryOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary
  1. This paper studies multimodal long-document question answering, focusing on when a model needs to continue reasoning and how retrieved evidence can reduce reasoning drift.
  2. The method uses predictive entropy across multiple model layers to decide whether to enter "soft thinking". Soft thinking continues reasoning with weighted representations of candidate tokens and uses retrieved evidence to refine these representations. The entire process requires no additional training.
  3. Overall, the method outperforms the main baseline, DocAgent, on MMLongBench-Doc and DocBench.
Critical strengths
  1. The method enters soft thinking only under specific conditions and limits the number of consecutive soft thinking steps. This design helps control the extent of its intervention.
  2. The experiments cover multiple models and two document question answering benchmarks, with stepwise ablations to examine the contribution of each module.
  3. The paper is clearly structured and presents the motivation and main workflow in an accessible way.
Critical weaknesses
  1. Low entropy indicates higher model confidence, but does not guarantee correctness. The paper does not adequately verify whether intermediate layers with low entropy are more reliable or assess whether their confidence is well calibrated, leaving the core mechanism insufficiently supported.
  2. Key details of evidence calibration remain unclear, including how the evidence vector is constructed, how it is mapped into the word embedding space, and how the mixing weights are determined. The "uncertainty-aware gating" mentioned in the abstract also lacks an explicit computational rule.
  3. The paper does not directly compare with methods that have closely related mechanisms, such as CLD [1] and DyLaR [2]. It is therefore difficult to determine whether the gains come mainly from selection across layers, dynamic switching, or evidence calibration.

[1] CLD: https://arxiv.org/abs/2606.21906 [2] DyLaR: https://ojs.aaai.org/index.php/AAAI/article/view/40513

Questions for authors
  1. Could the authors verify whether the selected low-entropy intermediate layers are more accurate and analyze cases in which the model is confident but produces incorrect answers?
  2. Could the authors specify how the evidence vector is constructed and how the mixing weights are computed, and provide the missing hyperparameters?
  3. With the same retrieval results and computational budget, does the proposed method outperform adapted versions of related methods such as CLD and DyLaR?
  4. Could the authors report statistical significance and inference time?
5 comments

[W1&Q1]:Could the authors verify whether the selected low-entropy intermediate layers are more accurate and analyze cases in which the model is confident but produces incorrect answers?

We thank the reviewer for this thoughtful comment. We agree that low entropy indicates higher model confidence but does not necessarily imply correctness. In contrast, high entropy reflects greater predictive uncertainty and has been associated with factual errors or unfaithful generation in prior studies (Huang et al. [1]; Das et al. [2]). This distinction motivates our work: rather than treating low entropy as evidence of correctness, we focus on addressing persistent high entropy during reasoning, helping the model recover more confident predictions while maintaining grounding in retrieved evidence and improving answer accuracy.

To this end, MEC examines predictive entropy across multiple late Transformer layers. When an intermediate layer satisfies Eq. (2),

H_t^{(L)} \geq \tau_H, \qquad H_t^{(\ell_t^\star)} < \min\left\\{\tau_L, H_t^{(L)} - \delta\right\\},

its prediction is adopted to avoid unnecessary latent reasoning. When persistent high entropy is detected across all candidate layers according to Eq. (3),

A_t = \left\\{\ell \in \mathcal{C} \mid H_i^{(\ell)} \geq \tau_H,\\; \forall i \in \\{t-K+1,\ldots,t\\}\right\\},

with At=CA_t=\mathcal{C}, ECST is activated to defer uncertain discrete commitments and incorporate retrieved evidence into soft-token updates, as defined in Eq. (5):

\widetilde{s}_t = s_t + \lambda(a_t-s_t).$$ The MEC-only ablation in Table 3 supports this strategy, improving accuracy from 50.01 to 51.65 on MMLongBench-Doc with Qwen3.6-27B, even without soft thinking or evidence calibration. Meanwhile, Case 1030 illustrates that low entropy does not guarantee correctness: Confident Thinking incorrectly selected 30.0% despite having calculated 30.3%, with the incorrect digit generated at an entropy of approximately $6.94\times10^{-6}$. We acknowledge that these results support the effectiveness of our strategy at the answer level but do not directly establish layer-wise prediction accuracy or probability calibration. Our goal is therefore to reduce error propagation under persistent uncertainty, rather than assume that lower-entropy predictions are necessarily correct. [1] Yanwen Huang, Yong Zhang, Ning Cheng, Zhitao Li, Shaojun Wang, and Jing Xiao. 2025. Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5174–5193, Vienna, Austria. Association for Computational Linguistics. [2] Souvik Das, Lifeng Jin, Linfeng Song, Haitao Mi, Baolin Peng, and Dong Yu. 2025. Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6589–6600, Abu Dhabi, UAE. Association for Computational Linguistics.
AuthorsOpenReview ↗

[W2&Q2] and [W3&Q3]

[W2&Q2]: Could the authors specify how the evidence vector is constructed and how the mixing weights are computed, and provide the missing hyperparameters?

We thank the reviewer for this helpful comment. We agree that several implementation details of evidence calibration were not sufficiently explained in the original manuscript. In ECST, we construct evidence representations using the backbone's existing multimodal encoding pipeline. Textual evidence is represented using token embeddings, while visual evidence is processed by the vision encoder and mapped into the language model's embedding space through the multimodal projector. Mean pooling is then applied to obtain evidence vectors compatible with the soft tokens. At each soft-thinking step, we select the top-kk relevant evidence items using cosine similarity (k=3k=3). As defined in Eq. (5), the selected representations are averaged to form the evidence anchor, which is combined with the original soft token as

\widetilde{s}_t = s_t + \lambda(a_t-s_t).$$ Here, $\lambda=0.3$ is fixed, assigning weights of $0.3$ and $0.7$ to the evidence anchor and original soft token, respectively. We also acknowledge that the term "uncertainty-aware gating" in the abstract was inaccurate. It refers to the use of high- and low-entropy thresholds in MEC, rather than an additional gating operation within ECST. Specifically, MEC identifies final-layer-specific uncertainty according to Eq. (2): $$H_t^{(L)} \geq \tau_H, \qquad H_t^{(\ell_t^\star)} < \min\left\\{\tau_L, H_t^{(L)} - \delta\right\\}.$$ ECST is activated when $A_t=\mathcal{C}$ in Eq. (3), where $$A_t = \left\\{\ell \in \mathcal{C} \mid H_i^{(\ell)} \geq \tau_H,\\; \forall i \in \\{t-K+1,\ldots,t\\}\right\\}.$$ ECST terminates when a sufficiently low-entropy prediction re-emerges or the maximum soft-thinking budget is reached. We have removed the expression from the abstract and Introduction. For completeness, the hyperparameters are set to $k=3$, $\lambda=0.3$, $w=4$, $\tau_H=1.0$, $\tau_L=0.2$, $\delta=0.08$, $K=2$, and $M=3$. We have revised Sections 3.2 and 4.1 accordingly. >[W3&Q3]: With the same retrieval results and computational budget, does the proposed method outperform adapted versions of related methods such as CLD and DyLaR? We thank the reviewer for highlighting these closely related methods. Regarding DyLaR, we were unable to locate a publicly available official implementation when preparing this response, so a direct experimental comparison is not included. To broaden the comparison, we have evaluated CLD and Soft Thinking, a related latent reasoning baseline, on MMLongBench-Doc across three backbones. For both baselines, we retain the same RAG pipeline and replace only the generation-stage reasoning method. The results are summarized below. | Method | Backbone | Academic | Brochure | Financial | Guidebook | Industry | Report | Tutorial | ACC ↑ | F1 ↑ | |---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | CLD | Qwen3.6-27B | 50.72 | 55.00 | 49.54 | 71.29 | 53.49 | 48.29 | 49.06 | 53.27 | 51.84 | | Soft Thinking | Qwen3.6-27B | 51.23 | 56.63 | 48.68 | 70.65 | 53.73 | 48.97 | 49.78 | 53.63 | 52.34 | | **Confident Thinking** | Qwen3.6-27B | **51.91** | **62.75** | **49.71** | **72.22** | **54.97** | **50.51** | **51.85** | **54.77** | **55.48** | | CLD | GLM-4.6V-Flash | 40.71 | 37.00 | 35.85 | 55.45 | **55.56** | 38.70 | 38.41 | 42.23 | 38.75 | | Soft Thinking | GLM-4.6V-Flash | 38.69 | 50.00 | 39.99 | 52.81 | 45.62 | 38.36 | 42.03 | 42.75 | 38.44 | | **Confident Thinking** | GLM-4.6V-Flash | **41.07** | **57.00** | **41.17** | **55.56** | 46.86 | **40.75** | **42.52** | **43.72** | **39.49** | | CLD | Gemma-4-31B-it | 60.33 | 60.20 | 61.94 | 69.50 | 54.60 | 68.97 | 67.26 | 64.59 | 60.32 | | Soft Thinking | Gemma-4-31B-it | 60.48 | 60.20 | 62.65 | 67.24 | 54.60 | 67.59 | 67.26 | 63.99 | 60.80 | | **Confident Thinking** | Gemma-4-31B-it | **61.89** | **65.31** | **63.18** | **74.58** | **58.71** | **69.07** | **67.69** | **65.99** | **61.15** | Confident Thinking achieves higher overall ACC and F1 than both baselines across all three backbones. On Qwen3.6-27B, GLM-4.6V-Flash, and Gemma-4-31B-it, the respective ACC gains are 1.50, 1.49, and 1.40 points over CLD, and 1.14, 0.97, and 2.00 points over Soft Thinking. The component ablations in Table 3 of the paper further show that, on Qwen3.6-27B, accuracy increases from 50.01 with standard CoT to 51.65 with MEC-only, 53.92 with MEC plus soft thinking, and 54.77 after adding evidence calibration. These results support incremental benefits from selective latent reasoning and evidence calibration beyond intermediate-layer selection alone.

[Q4]: Could the authors report statistical significance and inference time?

Thank you for this suggestion. We additionally evaluate inference efficiency on MMLongBench-Doc across three backbone models. All experiments are conducted using Qwen3.6-27B, GLM-4.6V-Flash, and Gemma-4-31B-it, with two NVIDIA A800 80GB GPUs. We report ACC, F1, and average inference time per question, as summarized below.

MethodBackboneACC (%) ↑F1 (%) ↑Avg. Time (s/question) ↓
CLDQwen3.6-27B53.2751.8475.32
Soft ThinkingQwen3.6-27B53.6352.3454.75
Confident ThinkingQwen3.6-27B54.7755.4862.69
CLDGLM-4.6V-Flash42.2338.7562.54
Soft ThinkingGLM-4.6V-Flash42.7538.4455.07
Confident ThinkingGLM-4.6V-Flash43.7239.4958.44
CLDGemma-4-31B-it64.5960.32125.36
Soft ThinkingGemma-4-31B-it63.9960.80103.81
Confident ThinkingGemma-4-31B-it65.9961.15118.53

CLD exhibits the highest latency, consistent with the additional computation required for candidate-layer logit evaluation and entropy-based layer selection at each decoding step. Soft Thinking incurs lower latency by directly constructing probability-weighted token embeddings without additional cross-layer assessment or evidence calibration. Our method introduces moderate overhead from multi-layer uncertainty evaluation and dynamic evidence calibration, while selective activation and bounded soft-thinking phases limit unnecessary latent updates. Consequently, Confident Thinking achieves intermediate latency, reducing inference time relative to CLD by 16.8%, 6.6%, and 5.4% across the three backbones, respectively, while consistently achieving higher ACC and F1 than both baselines.

Comments on the paper

1 comment

We sincerely thank the reviewers for their careful evaluation and constructive feedback. In response to their comments, we have revised the original manuscript to strengthen its experimental validation and clarify the methodology. Below, we summarize the recognized strengths and the main additions made during the rebuttal.

The reviewers acknowledged the targeted combination of uncertainty identification and evidence-calibrated reasoning (foy5), the compelling distinction between final-layer-specific uncertainty and persistent cross-layer uncertainty (Kmxu), and the selective activation and bounded duration of latent reasoning (QLry). They also appreciated the evaluations and component ablations (QLry, foy5, Kmxu), as well as the practicality of a training-free framework with shared hyperparameters across backbones (Kmxu).

1. Additional comparisons with CLD, Soft Thinking, DMLR, and SoftCoT. We expanded the experimental evaluation to directly compare Confident Thinking with these four closely related methods. For CLD, Soft Thinking, and adapted DMLR, we retain the same RAG retrieval pipeline and evaluate the generation-stage reasoning methods across three backbones. Confident Thinking achieves higher overall ACC and F1 than all three baselines. On Qwen3.6-27B, GLM-4.6V-Flash, and Gemma-4-31B-it, respectively, the ACC gains are 1.50, 1.49, and 1.40 points over CLD, and 1.14, 0.97, and 2.00 points over Soft Thinking. For SoftCoT, we use Qwen2.5-7B-Instruct and evaluate all compared methods on the same subset that fits within the model’s context limit. Confident Thinking improves ACC from 51.27 to 51.94 and F1 from 49.87 to 50.05 over SoftCoT, without additional training. Together with the existing component ablations, these comparisons support the benefits of MEC and evidence calibration beyond latent reasoning alone. The comparisons with Soft Thinking and DMLR have been incorporated into Section 4.4, while the additional CLD and SoftCoT results are provided in our rebuttal responses.

2. Revised ECST implementation details and hyperparameter specifications. To address the reproducibility concerns raised by QLry and foy5, we revised Sections 3.2 and 4.1 to explain evidence representation construction, its compatibility with the model’s embedding space, and evidence selection conditioned on the question and current reasoning state. We also specified the complete hyperparameter settings, including top-three evidence selection and a fixed evidence mixing weight of 0.3. We corrected the wording in the abstract and Introduction to clarify that uncertainty controls ECST activation through MEC, while evidence mixing uses a fixed weight. These revisions make the implementation and the responsibilities of the two components more explicit.

3. Additional inference-time evaluation. To address the efficiency concerns, we additionally evaluated average inference time per question on MMLongBench-Doc across three backbones using two NVIDIA A800 80GB GPUs. Compared with CLD, Confident Thinking reduces inference time from 75.32 to 62.69 seconds on Qwen3.6-27B, 62.54 to 58.44 seconds on GLM-4.6V-Flash, and 125.36 to 118.53 seconds on Gemma-4-31B-it, while achieving higher overall ACC and F1 on each backbone. We also report the additional inference cost relative to Soft Thinking alongside the accuracy improvements, making the accuracy–latency trade-off explicit. These measurements are provided in our response to QLry.

We believe these additional comparisons, timing measurements, and methodological clarifications strengthen the empirical support for Confident Thinking and provide a clearer account of its effectiveness and computational cost. We sincerely thank the Area Chair and reviewers for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.