- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Confident Thinking, a training-free decoding framework for multimodal long-context RAG that switches from explicit chain-of-thought reasoning to latent soft reasoning when genuine uncertainty is detected. Its two core components are MEC, which activates latent reasoning only when uncertainty persists across multiple late Transformer layers rather than being confined to the final layer, and ECST, which grounds latent updates in dynamically selected multimodal evidence. Experiments show consistent accuracy improvements, while ablations demonstrate that soft reasoning without MEC actually degrades performance.
Critical strengths
-
The distinction between final-layer-specific pseudo-confusion and persistent cross-layer uncertainty is conceptually compelling. The empirical finding that naïve final-layer entropy triggering hurts performance (48.49 vs. 50.01) while MEC recovers and improves upon it (51.65) provides strong evidence that the multi-layer consensus design is not merely additive but corrective.
-
The framework requires no additional training and is applied purely at inference time, using the same hyperparameters across three distinct MLLM backbones. This practicality is a genuine advantage for researchers and practitioners who wish to improve reasoning without costly fine-tuning.
-
Table 3 isolates the contribution of each component step by step. This progressive decomposition convincingly shows that the components are complementary and that MEC is the critical enabler.
-
The qualitative examples in Figure 4 effectively demonstrate how the method mitigates evidence drift and avoids premature commitment under semantically mismatched or out-of-scope retrieved content, grounding the quantitative gains in concrete failure modes.
Critical weaknesses
-
While improvements on MMLongBench are substantial (+2.95 to +5.32 points), the gains on DocBench are highly backbone-dependent: Qwen3.6-27B and GLM-4.6V improve by +3.20 each, but Gemma-4-31B-it gains only +0.21 points. This suggests the method’s value diminishes with stronger base models, limiting its universal appeal.
-
Although Coconut, Soft Thinking, DMLR, and SoftCoT are discussed in related work, none are directly evaluated on the same benchmarks. Without these head-to-head comparisons, it is difficult to assess whether the gains come from latent reasoning per se or specifically from the proposed MEC and evidence calibration mechanisms.
-
The method requires access to intermediate Transformer hidden states and the language-model head, which prevents application to popular black-box API models such as GPT-4o, Gemini, and Claude. Given that many production multimodal RAG pipelines rely on proprietary APIs, this substantially narrows the method’s practical applicability.
Questions for authors
see weakness
5 comments
[W2]
[W2] Although Coconut, Soft Thinking, DMLR, and SoftCoT are discussed in related work, none are directly evaluated on the same benchmarks. Without these head-to-head comparisons, it is difficult to assess whether the gains come from latent reasoning per se or specifically from the proposed MEC and evidence calibration mechanisms.
We thank the reviewer for this suggestion. We have completed additional comparisons with Soft Thinking [2], DMLR [3], and SoftCoT [4] on MMLongBench-Doc. Regarding Coconut, its original method requires CoT-supervised curriculum training. Since MMLongBench-Doc does not provide a separate CoT-annotated training set, adapting Coconut would require additional training-data preparation and model training, which we have not undertaken. For these comparisons, we retain the same RAG retrieval pipeline and replace the generation-stage reasoning method with each baseline. To accommodate DMLR’s image-conditioned reasoning mechanism, we convert the retrieved evidence into images and provide them together with the question. These additional experiments are now included in Section 4.4 of the revised manuscript.
| Method | Backbone | Academic | Brochure | Financial | Guidebook | Industry | Report | Tutorial | ACC ↑ | F1 ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| Soft Thinking | Qwen3.6-27B | 51.23 | 56.63 | 48.68 | 70.65 | 53.73 | 48.97 | 49.78 | 53.63 | 52.34 |
| DMLR | Qwen3.6-27B | 41.71 | 36.00 | 61.54 | 40.65 | 61.73 | 47.95 | 34.06 | 45.38 | 40.97 |
| Confident Thinking | Qwen3.6-27B | 51.91 | 62.75 | 49.71 | 72.22 | 54.97 | 50.51 | 51.85 | 54.77 | 55.48 |
| Soft Thinking | GLM-4.6V-Flash | 38.69 | 50.00 | 39.99 | 52.81 | 45.62 | 38.36 | 42.03 | 42.75 | 38.44 |
| DMLR | GLM-4.6V-Flash | 37.69 | 34.00 | 41.88 | 38.71 | 59.26 | 43.15 | 30.43 | 40.11 | 34.34 |
| Confident Thinking | GLM-4.6V-Flash | 41.07 | 57.00 | 41.17 | 55.56 | 46.86 | 40.75 | 42.52 | 43.72 | 39.49 |
| Soft Thinking | Gemma-4-31B-it | 60.48 | 60.20 | 62.65 | 67.24 | 54.60 | 67.59 | 67.26 | 63.99 | 60.80 |
| DMLR | Gemma-4-31B-it | 41.33 | 39.80 | 58.12 | 40.79 | 61.73 | 47.59 | 32.35 | 45.05 | 40.02 |
| Confident Thinking | Gemma-4-31B-it | 61.89 | 65.31 | 63.18 | 74.58 | 58.71 | 69.07 | 67.69 | 65.99 | 61.15 |
As shown in Table 1, Confident Thinking achieves higher overall ACC and F1 than Soft Thinking and the adapted DMLR baseline across all three backbones. Compared with Soft Thinking, ACC improves by 1.14 points on Qwen3.6-27B, 0.97 points on GLM-4.6V-Flash, and 2.00 points on Gemma-4-31B-it. Together with the component ablations in Table 3 of the paper, these results support the benefits of MEC and evidence calibration beyond latent reasoning alone.
For SoftCoT, we use Qwen2.5-7B-Instruct, following the Qwen backbone configuration evaluated in its original paper. Qwen2.5-7B-Instruct itself has a limited context window, and some samples exceed this limit when accounting for the complete input context and the reserved generation budget. We therefore evaluate SoftCoT, DocAgent, and Confident Thinking on the same subset of MMLongBench-Doc that fits within this limit, preserving the retrieved evidence without truncation.
| Method | Backbone | Academic | Brochure | Financial | Guidebook | Industry | Report | Tutorial | ACC ↑ | F1 ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| DocAgent | Qwen2.5-7B | 59.40 | 41.10 | 54.84 | 41.30 | 66.67 | 50.64 | 44.14 | 50.87 | 48.40 |
| SoftCoT | Qwen2.5-7B | 57.89 | 38.36 | 58.06 | 44.57 | 66.67 | 51.06 | 45.95 | 51.27 | 49.87 |
| Confident Thinking | Qwen2.5-7B | 53.38 | 42.47 | 54.84 | 40.22 | 68.06 | 56.60 | 45.05 | 51.94 | 50.05 |
Table R2 compares the methods using the same Qwen2.5-7B backbone. Confident Thinking outperforms both DocAgent and SoftCoT in overall ACC and F1. Compared with SoftCoT, ACC increases from 51.27 to 51.94 and F1 from 49.87 to 50.05. These gains are achieved without additional training, whereas SoftCoT relies on a trained projection module.
[W3]
[W3]: The method requires access to intermediate Transformer hidden states and the language-model head, which prevents application to popular black-box API models such as GPT-4o, Gemini, and Claude. Given that many production multimodal RAG pipelines rely on proprietary APIs, this substantially narrows the method's practical applicability.
We thank the reviewer for this comment. As acknowledged in Appendix A.4 (Limitations), Confident Thinking requires access to model internals and therefore cannot be applied through standard black-box APIs. Our method targets open-weight models whose inference process can be directly controlled.
We recognize the importance of API-based pipelines. However, some multimodal RAG applications use locally deployed open-weight models to meet data privacy requirements. For these applications, Confident Thinking provides a training-free approach to improving generation reliability, supported by our evaluations on three open-weight multimodal backbones. Beyond this deployment setting, our findings on cross-layer uncertainty and evidence-calibrated latent reasoning may also inform reasoning mechanism design by developers with access to model internals.
[W1]
We thank the reviewer for highlighting the variation across backbones. We agree that the improvement for Gemma-4-31B-it on DocBench is modest, and that the magnitude of the benefit is not uniform across the evaluated settings.
However, we respectfully suggest that the results do not establish a general trend of diminishing benefits with stronger backbones. On MMLongBench-Doc, Gemma-4-31B-it has the highest baseline accuracy among the three evaluated backbones, yet also achieves the largest improvement: from 60.67 to 65.99 (+5.32 points), compared with +4.76 for Qwen3.6-27B and +2.95 for GLM-4.6V-Flash. Thus, the same strong backbone benefits substantially on one benchmark and only marginally on the other. The DocBench results further suggest that baseline accuracy alone does not explain this difference. Gemma and Qwen start from similar accuracies of 83.56 and 82.73, respectively, but obtain different gains. For Gemma, the category-level results are mixed: Text and Metadata improve by 0.80 and 0.46 points, respectively, while Multimodal decreases by 0.77 points. Our evidence therefore supports applicability across the evaluated backbones with varying performance gains, rather than uniformly large improvements in every setting. We appreciate the reviewer's concern about generality.