acceptodds
Under review as a conference paper at ICLR 2027

DRT: Dense Reasoning Trace for Efficient and Grounded Multimodal Reasoning

Abstract

Despite the remarkable progress in Multimodal Large Language Models (MLLMs), prevailing Chain-of-Thought (CoT) paradigms remain largely confined to the space of verbose, free-form natural-language reasoning. Consequently, they inherently incur excessive linguistic overhead, leading to information dilution and weak visual grounding. To address this challenge, we propose **Dense Reasoning Trace (DRT)**, a paradigm that departs from verbose, free-form CoT by expressing reasoning as compact structured traces, which include concise intermediate states with symbolic connectors and disentangle visual observations from logical deductions. First, we introduce the **Dense Trace Initialization** to internalize the DRT reasoning mode into the model, substantially improving token efficiency while preserving visual evidence. To further enable the model to faithfully capture the logical relations within traces, we propose the **Trace-Grounded Reinforcement Learning** framework, which builds reference traces through a tri-perspective verification pipeline and employs Trace-Grounded GRPO with structured rewards, encouraging the model to generate concise DRT-style traces with reduced hallucination and stronger logical grounding. Extensive experiments on challenging reasoning benchmarks show that DRT achieves 5.5 token efficiency improvement while improving 1.3 accuracy points over the Qwen3-VL baseline. These findings suggest that complex multimodal reasoning may not require verbose natural-language traces, opening a more efficient path for next-generation MLLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Efficient Reasoning, Structured Chain-of-Thought, Reinforcement Learning, Process Supervision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tKPiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper replaces verbose free-form multimodal CoT with Dense Reasoning Trace (DRT): compact, structured traces that compress wording via symbolic connectors, place visual observations in <visual> and deduction in <think> with the two explicitly disentangled, and close with a short answer. Training proceeds in two stages. DenseTrace Initialization has GPT-5.1 rewrite 200K multimodal samples into the DRT format for SFT; the Trace-Grounded RL stage first builds reference traces through a tri-perspective verification pipeline, then optimizes Trace-Grounded GRPO with step-level partial credit, a hallucination penalty, and exploration bonuses. On five reasoning benchmarks, DRT delivers a 5.5× token-efficiency gain while adding 1.3 accuracy points over the Qwen3-VL baseline.

Critical strengths
  1. Clear practical value. A 5.5× cut in output tokens with slightly better accuracy is a strong operating point, and the accuracy–efficiency trade-off dominates the compared efficient-reasoning methods, which matters directly for lowering multimodal inference cost.
  2. Complete and careful training design. Reference traces pass tri-perspective verification; the reward combines process-level partial credit, a multiplicative hallucination correction, and difficulty-aware exploration bonuses — the pieces interlock, and ablations support the necessity of each component.
Critical weaknesses
  1. Compression still has an accuracy ceiling. In Table 1, DRT-RL averages 67.2, well behind Qwen-Thinking at 72.2, so the "accuracy preserved" framing is overstated — the honest takeaway is a better trade-off, not free efficiency.
  2. Heavy reliance on strong external models. GPT-5.1 performs both the SFT rewriting and the RL reference verification, and reward judging depends on Qwen3-235B; much of the gain is therefore distillation plus verification, with the method's own novelty concentrated in the format. Sensitivity to the reward hyperparameters (the λ's, N_cap) is not studied.
Questions for authors
  1. Under a matched token budget, how does DRT-RL stack up against Qwen-Thinking?
  2. Does the DRT format transfer to backbones other than Qwen3-VL?
  3. What fraction of candidate trajectories survives tri-perspective verification, and how is verifier-induced bias assessed?
2 comments

[Question 1] Under a matched token budget, how does DRT-RL stack up against Qwen-Thinking?

We have not yet completed a controlled matched-budget comparison with Qwen-Thinking. DRT-RL is trained through SFT and RL to produce compact traces, so increasing its generation limit alone does not ensure that it uses the additional token budget.

Our accuracy-preserving claim refers specifically to Qwen3-VL-8B-Instruct: DRT-RL maintains the Instruct baseline's average reasoning performance while substantially reducing output tokens, but does not match Qwen-Thinking's higher accuracy. We will clarify this scope and acknowledge controllable reasoning-budget scaling as a limitation.

[Question 2] Does the DRT format transfer to backbones other than Qwen3-VL?

We applied the complete DRT SFT+RL pipeline to Qwen3-VL-2B-Instruct and InternVL3.5-8B and evaluated both on all five benchmarks. Each cell reports accuracy (%) / mean full-output tokens. AVG is the arithmetic mean across the five benchmarks.

ModelSourceMathVistaMathVerseLogicVistaGSM8KVideo-HolmesAVG
Qwen3-VL-2B-InstructOfficial62.30 / 1,899.949.01 / 6,137.640.27 / 7,882.585.75 / 904.830.65 / 2,528.353.60 / 3,870.6
Qwen3-VL-2B-InstructSFT+RL62.90 / 174.550.13 / 251.341.61 / 179.380.97 / 181.734.73 / 141.254.07 / 185.6
InternVL3.5-8BOfficial78.50 / 4,057.461.93 / 7,304.358.84 / 9,925.294.31 / 2,586.441.54 / 5,048.667.02 / 5,784.4
InternVL3.5-8BSFT+RL74.50 / 179.454.02 / 254.555.03 / 192.393.03 / 175.939.36 / 120.763.19 / 184.6

On Qwen3-VL-2B, DRT-RL improves accuracy on all four multimodal benchmarks while substantially reducing output length, although GSM8K accuracy decreases. On InternVL3.5-8B, DRT-RL likewise produces much shorter traces, but with lower accuracy across all five benchmarks. These results show that compact DRT reasoning transfers to both a smaller backbone and a different model family, while accuracy preservation remains backbone- and task-dependent.

[Question 3] What fraction of candidate trajectories survives tri-perspective verification, and how is verifier-induced bias assessed?

Among 28,479 valid candidate trajectories, 95.68% pass visual fidelity, 88.73% pass reasoning validity, and 61.20% pass answer correctness. Overall, 14,724 trajectories (51.70%) pass all three criteria and are retained for training.

Motivated by concerns about self-preference in LLM judgments [1], we assess dependence on the original verifier through a cross-family audit of all 14,724 retained trajectories. We use two models from different families: Gemini-3.8-Flash and Seed-2.1-Pro-0915. GPT-5.1, which generated and originally verified the trajectories, is excluded from the audit panel. Both auditors are blinded to the generator identity, the original verification decision, and each other's judgments. Each independently solves the original problem before evaluating the trajectory's visual fidelity, reasoning validity, and answer correctness.

Cross-family audit outcomeTrajectoriesFraction
Both auditors pass11,00574.74%
Both auditors flag an error1,79612.20%
Disagreement or uncertainty1,92313.06%

Within the 1,796 jointly flagged trajectories, the auditors jointly localize one erroneous step in 664, two in 506, three in 311, and four or more in 315.

These results indicate that tri-perspective verification filters the candidate pool but does not guarantee error-free reasoning. We will report verification pass rates, cross-family agreement and disagreement, and error severity rather than treat any single verifier as ground truth. The downstream RL results support the practical utility of the retained data, while the audit highlights residual quality concerns and verifier dependence.

[1] Chen, Zhi-Yuan, et al. "Beyond the surface: Measuring self-preference in llm judgments." Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

Thank you for the detailed response and additional experiments, which address most of my concerns. Therefore, I will keep my positive rating.

Reviewer_cqhvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DRT replaces free-form chain of thought in an MLLM with a compact, telegraphic trace that keeps visual observations (<visual>) separate from symbolic deduction (<think>).

Qwen3-VL-8B-Instruct is first fine-tuned on ~200K Vision-R1-cold samples that GPT-5.1 rewrote into DRT format. It is then trained with "Trace-Grounded GRPO." The reward combines answer correctness, partial step-match credit, a hallucination penalty, and step/exploration bonuses, with a Qwen3-235B judge.

Across five benchmarks, DRT-RL averages 67.2 accuracy at 206.9 tokens, vs. 65.9 at 1,139.8 tokens for Qwen-Standard, with higher QPS and lower latency.

Critical strengths

A simple, inspectable compression format. The reasoning stays explicit and readable instead of moving into latent states. Output length drops about 5.5× with no loss in average accuracy, and the reward is reasonably robust to the choice of judge (Tab. 5–7). The full prompts and training cost are reported too.

Good controls against the obvious alternatives.

  • Concise-CoT (App. H): after RL, DRT-RL beats the token-matched and content-matched controls on all four multimodal benchmarks, by 2.7 and 2.3 points on average, with 18.4% and 15.3% fewer tokens (Table 8).
  • Length-penalty GRPO (rebuttal): with the same backbone and RL data, DRT-RL reaches 67.2 vs. 62.7 average accuracy at 206.9 vs. 145.7 tokens.

Together, these are good evidence that the format itself matters, not only shorter outputs.

Critical weaknesses

The rebuttal addressed several of my concerns:

  • Efficiency: the steady-state re-measurement with continuous batching gives 17.35 vs. 8.29 QPS and 4.27 vs. 11.44 s for DRT-RL vs. Qwen-Standard (on H800 rather than A100), and the Qwen-DA 128-concurrency setting is now labelled.
  • Variance: five seeds give 66.93 ± 0.35 vs. 65.9, with sample counts and decoding settings reported.
  • Format reward: all 7,224 multimodal outputs contain exactly one non-empty <visual> block.

1. The grounding claims are less clearly supported. On the standard benchmarks, DRT-RL does not improve over Qwen-Standard:

  • HallusionBench 58.80 vs. 60.25
  • POPE F1 88.87 vs. 89.16
  • MathVerse vision-only 62.42 vs. 64.34

The positive evidence (Fig. 3(c), SPD-Faith) is reasoning-specific, and much of it relies on LLM judges. GPT-5.1 also generates and verifies the reference traces. There is still no ablation of the visual-block separation. I understand the authors' point that separation and compact deduction are coupled, but without such an ablation the grounding role of <visual> is hard to assess.

2. The accuracy gain is modest and uneven. The +1.0–1.3 average gain is concentrated on Video-Holmes and MathVerse, and DRT-RL is slightly below Qwen-Standard on MathVista and GSM8K. Qwen-Thinking (72.2) remains about five points higher, and DRT could not yet be applied stably to the Thinking model. I think this is fine for a compression method, but the abstract would read better as "comparable accuracy at much lower cost."

Questions for authors
  1. Would you consider toning down the grounding / hallucination claims in the abstract and introduction to match the HallusionBench, POPE and MathVerse vision-only results? I would be happy to raise my score if they are revised accordingly.
  2. Could the App. A cost projection be updated with the steady-state numbers, noting the H800 vs. A100 change?
  3. Could the new results (five-seed table, length-penalty baseline, hallucination benchmarks) go into the main paper or a clearly referenced appendix?
  4. If an ablation of the visual-block separation is feasible, even at a smaller scale, it would help clarify what the separation contributes.
4 comments

[Question 1] Why do you describe the configurations as identical when Qwen-DA runs at 128-way concurrency and the other methods at 64? Was QPS measured with continuous batching or batch-synchronously? Could you re-run Tab. 4 with uniform concurrency and continuous batching, and update the App. A cost projection?

Thank you for highlighting the ambiguity in our efficiency description. Qwen-DA produces extremely short responses and does not saturate the GPU at concurrency 64, so this setting underestimates its achievable throughput. We therefore retain Qwen-DA’s saturated result at concurrency 128 and explicitly label this setting.

Our evaluation uses vLLM continuous batching with continuous client-side request refill, not batch-synchronous execution. The original full-run QPS, used in Appendix A's cost estimate, includes the final drain period. Once the request queue is exhausted, concurrency falls below the configured level, and a few long responses can substantially lower full-run QPS. This tail effect explains why QPS × mean latency can be substantially below the configured concurrency for long-tailed methods. The new measurement isolates the steady-state window. For concurrency C, the timing window ends when the C-th request from the end completes, excluding the subsequent underfilled drain period. Requests still in flight at the cutoff contribute to concurrency occupancy but not to the mean latency of completed requests. Thus, QPS × mean latency need not equal C, particularly for long-tailed responses.

Due to GPU availability, the new measurements use a single H800 per method–benchmark pair rather than the original single A100. All other settings remain unchanged. Entries below report QPS / mean latency in seconds, with arithmetic averages across benchmarks.

MethodConcurrencyMathVistaMathVerseLogicVistaGSM8KVideo-HolmesAVG QPS ↑\uparrowAVG Latency ↓\downarrow
Qwen-DA12839.29 / 3.0246.97 / 2.6982.57 / 1.32332.89 / 0.349.20 / 13.38102.184.15
Qwen-DRT6413.00 / 3.293.81 / 16.088.18 / 4.0162.87 / 0.958.88 / 6.7019.356.21
Qwen-Standard645.57 / 7.743.63 / 16.551.88 / 23.8821.60 / 2.768.80 / 6.298.2911.44
Qwen-Thinking641.86 / 30.001.08 / 58.100.63 / 85.364.34 / 14.091.15 / 51.581.8147.83
CoD6425.71 / 1.956.00 / 9.4830.93 / 1.6492.87 / 0.677.33 / 8.4932.574.45
ThinkLess6412.81 / 4.8215.83 / 4.0014.83 / 4.0228.25 / 2.204.57 / 13.7115.265.75
VisionThink649.13 / 6.459.18 / 6.8510.70 / 5.1431.43 / 1.962.54 / 21.1512.608.31
DRT-SFT6416.67 / 3.6116.11 / 3.9318.17 / 3.1342.82 / 1.467.21 / 8.7520.204.18
DRT-RL6414.65 / 4.1513.77 / 4.5814.48 / 3.8634.80 / 1.799.02 / 6.9717.354.27

DRT-RL achieves higher average throughput and lower average latency than Qwen-Standard: 17.35 vs. 8.29 QPS and 4.27 vs. 11.44 seconds, respectively. The discrepancies highlighted by the reviewer, particularly CoD on LogicVista and Qwen-Standard on GSM8K, reflect the impact of a few exceptionally long responses. Although most requests finish quickly, these outliers prolong the final drain and substantially lower full-run QPS, even under continuous batching. DRT's more controlled response lengths reduce its sensitivity to this drain effect.

[Question 2] Most of the +1.3 average gain comes from Video-Holmes. Could you report mean and variance over multiple seeds, per-benchmark confidence intervals, and the evaluation splits, sample counts and decoding parameters?

We repeated inference with the fixed DRT-RL checkpoint using five seeds (42–46). The table reports mean ± SD and 95% confidence intervals for the across-seed mean. Qwen-Standard values are the original main-table results.

BenchmarkSamplesQwen-StandardDRT-RL (mean ± SD, 5 seeds)DRT-RL 95% CI
MathVista1,00077.275.82 ± 0.88[74.73, 76.91]
MathVerse3,94062.164.82 ± 0.42[64.30, 65.34]
LogicVista44755.356.29 ± 1.43[54.51, 58.06]
GSM8K1,31995.294.12 ± 0.57[93.41, 94.82]
Video-Holmes1,83739.643.59 ± 0.42[43.07, 44.11]
AVG-65.966.93 ± 0.35[66.49, 67.36]

The fixed evaluation sets are MathVista_MINI, MathVerse_MINI, LogicVista, GSM8K test, and Video-Holmes. MathVerse comprises five variants of 788 base questions, totaling 3,940 evaluated instances. Decoding uses temperature 0.7, top-p 0.8, top-k 20, a maximum of 16,384 generated tokens, repetition penalty 1.0, presence penalty 1.5, and one response per request.

The five-seed results closely reproduce the main-table scores. Video-Holmes contributes the largest gain, and its DRT-RL score remains consistent across inference seeds.

[Question 3] Beyond LLM judges (one of which also generated the references), can you show reduced hallucination on a standard benchmark such as HallusionBench, POPE or MathVerse vision-only, or with human annotation? Is there an ablation without the visual-block separation, and why doesn't the RL format reward check that block?

We evaluated the same checkpoints on the three requested benchmarks using the same generation settings.

BenchmarkMetricQwen-StandardDRT-RL
HallusionBenchAverage60.2558.80
POPEF189.1688.87
MathVerse Vision-onlyAccuracy64.3462.42

These results do not show gains on the three requested benchmarks. HallusionBench and POPE primarily measure perceptual hallucination in final answers, whereas DRT's <visual> / <think> separation is designed to distinguish visual observations from logical deductions. Our positive evidence concerns reasoning-specific evaluations. Figure 3(c), covering 4,671 samples, shows fewer reasoning steps based on unsupported assumptions or fabricated visual evidence. On the external SPD-Faith benchmark, DRT-RL reduces the contradiction rate from 27.64% to 19.08% on single-difference cases and increases DRF from 21.97% to 29.37% on multi-difference cases. Visual-block ablation. Visual–reasoning separation and compact deduction are coupled in the DRT format used throughout training. Removing the separation would also change the representation underlying compact deduction, so we evaluate the integrated format rather than isolate separation through a standalone ablation. Format reward. Our RL data include both multimodal and text-only samples, so the format reward does not universally require a <visual> block. The structure learned during SFT is retained after RL: all 7,224 multimodal evaluation outputs contain exactly one non-empty <visual> block.

[Question 4] Could you add a same-backbone, same-data RL baseline with a length penalty (e.g., GRPO with a length penalty) to the main table, and report DRT applied to the Thinking model? Qwen-Thinking (72.2) is currently five points above DRT-RL.

We added an ordinary-CoT GRPO baseline initialized directly from Qwen3-VL-8B-Instruct and trained on the same RL data for 170 optimization steps. Its reward subtracts a length penalty from answer correctness: the penalty is zero up to 300 output tokens, increases linearly to 1 at 1,000 tokens, and is capped thereafter. All other optimization settings match DRT-RL.

ModelMathVistaMathVerseLogicVistaGSM8KVideo-HolmesAVG Acc.AVG Tokens
Qwen-Standard77.262.155.395.239.665.91,139.8
Qwen-Thinking79.974.268.296.142.572.22,427.2
Length Penalty72.546.557.394.243.062.7145.7
DRT-RL76.864.356.894.343.767.2206.9

Compared with this direct length-penalty baseline, DRT-RL achieves 4.5 points higher average accuracy at 206.9 vs. 145.7 output tokens, with the largest gap on MathVerse (64.3 vs. 46.5).

Thinking initialization. Our preliminary attempt to apply DRT SFT to Qwen3-VL-8B-Thinking did not yield a stable checkpoint, so we report results with Instruct initialization. Qwen-Thinking achieves 5.0 points higher average accuracy, while DRT-RL uses 91.5% fewer output tokens.

Reviewer_cqhvOpenReview ↗

Revised comment after rebuttal

I thank the authors for the additional experiments.

Efficiency and variance are now largely addressed:

  • Steady-state re-measurement under continuous batching: DRT-RL 17.35 QPS / 4.27 s vs. 8.29 QPS / 11.44 s for Qwen-Standard (though on H800 rather than A100).
  • Five seeds: 66.93 ± 0.35 vs. 65.9, with the gain still concentrated on Video-Holmes and MathVerse.
  • The new GRPO length-penalty baseline is useful; DRT-RL is clearly more accurate at a similar budget (67.2 vs. 62.7).

The grounding claims are less clearly supported. On the requested benchmarks DRT-RL does not improve over Qwen-Standard:

  • HallusionBench 58.80 vs. 60.25
  • POPE F1 88.87 vs. 89.16
  • MathVerse vision-only 62.42 vs. 64.34

There is also still no ablation of the visual-block separation, so the grounding and faithfulness claims rest mainly on LLM judges.

I keep my score of 2, and I would be happy to raise my score if the grounding claims are toned down accordingly.

Thank you for clarifying this concern. Our intended design is to record model-reported visual observations separately from compact logical deductions, making the distinction between observations, assumptions, and derived conclusions explicit. This separation aims to reduce the risk that assumptions introduced during reasoning are later treated as if they had been directly observed in the image and reused as visual premises in subsequent deductions.

We will clarify two possible sources of unsupported visual claims: errors in the model’s perception of the input, and assumptions introduced during reasoning that are subsequently presented as visual observations. DRT’s separation is designed to address the latter. Our original use of “grounding” did not make this distinction sufficiently clear. The results on HallusionBench, POPE, and MathVerse Vision-only do not show an improvement over Qwen-Standard, and we will report them explicitly without claiming that DRT reduces errors originating in visual perception.

Specifically, we will replace broad descriptions of visual-block separation as improving visual grounding or faithfulness with a more precise statement of its intended role: reducing reasoning-induced visual hallucinations. The main revisions are as follows:

  • Introduction, L79: “In this way, DRT directly addresses the limitation of CoT by reorganizing explicit reasoning, mitigating informational dilution while aiming to reduce reasoning-induced visual hallucinations.”

  • Introduction, L103: “By reformulating the surface organization of explicit reasoning, DRT mitigates informational dilution and aims to reduce reasoning-induced visual hallucinations.”

  • Method, L179: “To address these limitations, we propose Dense Reasoning Trace (DRT), a reasoning paradigm with a structured representation designed to improve reasoning efficiency and reduce reasoning-induced visual hallucinations.”

  • Method, L183: “To avoid treating inferred assumptions as visual observations, DRT further disentangles visual perception from logical deduction, so that model-reported observations and inferred conclusions are represented in distinct stages.”

  • Method, L188: “Visual Observations (<visual>)”

  • Method, L189: “By recording model-reported visual observations separately from downstream inference, this component provides an explicit reference for compact deductions, with the aim of reducing reasoning-induced visual hallucinations.”

  • Method, L203: “This representation improves reasoning efficiency by reducing linguistic redundancy and aims to reduce reasoning-induced visual hallucinations by keeping reported visual observations distinct from inferred assumptions.”

  • Evaluation Step-level analysis: We will explain the intended role of separating visual observations from deductions and interpret Figure 3(c) within the DRT SFT+RL framework. The figure shows lower judge-estimated hallucination ratios among effective reasoning steps for process-supervised DRT variants than for DRT-SFT and answer-only RL.

  • Evaluation additional benchmarks: We will add the completed HallusionBench, POPE, and MathVerse Vision-only evaluations to the appendix. These results show no improvement over Qwen-Standard. We will distinguish these findings from the design objective of limiting reasoning-induced visual hallucinations, without claiming that DRT improves the model’s initial visual perception.

  • Conclusion, L477: “We introduce DRT, a compact representation for efficient multimodal reasoning. By separating model-reported visual observations from logical deductions and compressing intermediate reasoning into compact traces, DRT mitigates informational dilution while aiming to reduce reasoning-induced visual hallucinations.”

Thank you again for your careful reading and constructive feedback, which have helped us clarify the intended scope of our work and improve its presentation. We hope the proposed revisions address your concern and would be happy to discuss any remaining questions or provide further clarification.

Reviewer_z3UcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Dense Reasoning Trace (DRT), a framework for efficient and grounded multimodal reasoning. DRT replaces verbose CoT reasoning with compact, structured traces that separate visual observations, logical deductions, and final answers. The framework combines SFT on compressed reasoning traces with reinforcement learning using step-level rewards to encourage correct and grounded reasoning. Experiments across five benchmarks report a 5.5× improvement in token efficiency and a 1.3 percentage-point accuracy gain over the Qwen3-VL baseline.

Critical strengths
  1. The paper is well-organized and presented.

  2. The proposed approach demonstrates promising accuracy vs. efficiency trade-offs.

Critical weaknesses

1. Novelty and Missing Related Work

The novelty of the proposed DRT representation is not sufficiently established. Several closely related works, particularly Vision-SR1 (ICLR 2026) and Visionary-R1 (2025), have already explored separating visual observations, reasoning, and final answers. These approaches share substantial conceptual and representational similarities with DRT, yet neither is discussed or cited.

I acknowledge that DRT introduces different training methods, along with additional elements such as the [Priors] anchor and compressed, arrow-connected reasoning steps. These are meaningful implementation differences, but the underlying decomposition is already present in prior work. The authors should clarify the novelty of DRT and provide a more direct comparison.

Relevant missing references include:

  • Vision-SR1: Self-Rewarding Vision-Language Model via Reasoning Decomposition and Multi-Reward Policy Optimization (ICLR 2026).
  • Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning (2025).

2. Insufficient Evaluation of Visual Grounding and Reasoning Faithfulness

Improving visual grounding and reasoning faithfulness is the motivation, but the experiments mainly evaluate final accuracy and LLM-judged reasoning quality. These measurements do not necessarily establish that the generated reasoning is grounded in visual evidence or faithfully reflects the model's decision-making process.

The authors define reasoning faithfulness in terms of whether deductive steps are justified by observations, priors, or established knowledge. However, this primarily captures logical validity and evidential consistency, rather than faithfulness. This distinction has been extensively discussed in the CoT faithfulness literature, which is also largely missing from the paper. The authors should better position their definition (faithfulness, perceptual faithfulness, etc.) within this literature and provide more direct evidence supporting the grounding and faithfulness claims.

Some missing references:

  • Measuring Faithfulness in Chain-of-Thought Reasoning.
  • Faithful-First Reasoning, Planning, and Acting for Multimodal LLMs.
  • SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models.

3. Limited Model Generalizability

The main SFT and RL experiments are conducted only on Qwen3-VL-8B-Instruct. Although the appendix includes zero-shot DRT prompting experiments on other models, these do not establish whether the proposed training framework generalizes across model architectures and scales. Additional experiments involving different model families or model sizes would provide stronger evidence for the generalizability of the proposed approach.

Questions for authors

See above.

4 comments

[Question 1] Novelty and Missing Related Work

Thank you for pointing out these closely related works. Visionary-R1 uses a caption, reasoning, and answer format with a caption reward to improve visual grounding and mitigate visual shortcuts. Vision-SR1 develops a self-rewarding framework that separately optimizes visual descriptions and language reasoning through multi-reward policy optimization. We agree that this three-part decomposition is not unique to DRT.

DRT differs in its compact reasoning representation and the training objectives designed for it. It expresses reasoning as compact, semi-formal state transitions with explicit priors, concise intermediate states, and symbolic connectors. Dense Trace Initialization first teaches this representation through SFT. Trace-Grounded RL then optimizes correspondence with reference steps, effective deductions, and reasoning density while penalizing hallucination-related errors. The objective is to reduce narrative and decoding overhead while retaining explicit intermediate deductions.

We will cite both works and revise the Related Work section to position DRT's contribution in this compact representation and its corresponding training framework, rather than in the three-part decomposition itself.

[Question 2] Insufficient Evaluation of Visual Grounding and Reasoning Faithfulness

Thank you for highlighting this distinction. Following Faithful-First [1], we distinguish perceptual faithfulness, which concerns whether visual claims are supported by the input, from behavioral faithfulness, which concerns whether the visible trace reflects the process producing the final answer. Our original analysis primarily evaluates evidential grounding and logical validity, rather than directly testing the answer’s dependence on the trace.

We therefore add SPD-Faith [2] to evaluate evidence-grounded reasoning and Lanham-style interventions [3] to examine behavioral dependence. For the interventions, we report answer-flip rates under localized reasoning errors and meaning-preserving rewrites. Content-specific sensitivity is the difference between the answer-flip rates under reasoning errors and meaning-preserving rewrites.

EvaluationMetricQwen-StandardDRT-RL
SPD-FaithDQR ↑\uparrow15.8212.61
DS ↑\uparrow48.5549.40
DRF ↑\uparrow21.9729.37
Content interventionMistake flip ↑\uparrow8.8113.21
Equivalent flip ↓\downarrow0.941.57
Content-specific sensitivity ↑\uparrow7.8611.64

On SPD-Faith, DRT-RL improves DRF, indicating a higher proportion of visually supported reasoning claims. DS remains similar, while DQR decreases. The improvement therefore concerns evidence-grounded reasoning rather than uniformly better perception. Under content interventions, DRT-RL is more responsive to injected reasoning errors, with an answer-flip rate of 13.21% versus 8.81%. Flip rates under meaning-preserving rewrites remain low for both models, at 1.57% and 0.94%, respectively. The higher content-specific sensitivity provides evidence that DRT-RL’s final answers depend more strongly on trace semantics under these interventions. We will revise the definitions and add these results, distinguishing behavioral intervention evidence from visual grounding and logical-validity assessments rather than inferring faithfulness solely from final accuracy or LLM-judged reasoning quality.

[1] Li, Junxian, et al. "Faithful-First Reasoning, Planning, and Acting for Multimodal LLMs." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

[2] Lv, Weijiang, et al. "SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models." Findings of the Association for Computational Linguistics: ACL 2026. 2026.

[3] Lanham, Tamera, et al. "Measuring faithfulness in chain-of-thought reasoning." arXiv preprint arXiv:2307.13702 (2023).

[Question 3] Limited Model Generalizability

Thank you for this suggestion. We applied the complete DRT SFT+RL pipeline to Qwen3-VL-2B-Instruct and InternVL3.5-8B and evaluated both on all five benchmarks. Each cell reports accuracy (%) / mean full-output tokens. AVG is the arithmetic mean across the five benchmarks.

ModelSourceMathVistaMathVerseLogicVistaGSM8KVideo-HolmesAVG
Qwen3-VL-2B-InstructOfficial62.30 / 1,899.949.01 / 6,137.640.27 / 7,882.585.75 / 904.830.65 / 2,528.353.60 / 3,870.6
Qwen3-VL-2B-InstructSFT+RL62.90 / 174.550.13 / 251.341.61 / 179.380.97 / 181.734.73 / 141.254.07 / 185.6
InternVL3.5-8BOfficial78.50 / 4,057.461.93 / 7,304.358.84 / 9,925.294.31 / 2,586.441.54 / 5,048.667.02 / 5,784.4
InternVL3.5-8BSFT+RL74.50 / 179.454.02 / 254.555.03 / 192.393.03 / 175.939.36 / 120.763.19 / 184.6

On Qwen3-VL-2B, DRT-RL improves accuracy on all four multimodal benchmarks while substantially reducing output length, although GSM8K accuracy decreases. On InternVL3.5-8B, DRT-RL likewise produces much shorter traces, but with lower accuracy across all five benchmarks. These results show that compact DRT reasoning transfers to both a smaller backbone and a different model family, while accuracy preservation remains backbone- and task-dependent.

Thank you to the authors for the response. It addresses most of my concerns, but I believe the novelty and the missing related work still warrant further discussion, particularly why these closely related studies were not included in the first place and why comparisons are still missing. While these concerns are not fully resolved, I have raised my score in recognition of the authors’ response and would encourage them to address these issues in the revision or a future resubmission.

Thank you for your careful follow-up and for recognizing our additional analyses and experiments.

Our initial literature review and baseline selection focused on efficient reasoning, particularly post-training methods for reducing latency and inference cost. We developed DRT’s structured representation toward this goal, but did not adequately cover closely related work on multimodal reasoning decomposition. This led to the omission of these studies from both our discussion and comparisons. We apologize for this oversight.

Our earlier response outlined DRT’s intended contribution but did not provide a sufficiently explicit comparison. In the revision, we will expand the “Reasoning Models and Test-Time Scaling” paragraph to discuss structured multimodal reasoning and compare these methods with DRT in terms of representation and training objectives. The proposed revision is as follows:

Related Work, L124: Prior work has also explored organizing multimodal reasoning into distinct stages. Visionary-R1 [1] uses a caption, reasoning, and answer format, with a caption reward that evaluates whether the generated description supports correct answers without access to the image. Vision-SR1 [2] evaluates visual descriptions through an additional rollout without the image and separately computes advantages for visual and answer rewards. LLaVA-CoT [3] teaches a four-stage format consisting of summary, caption, reasoning, and conclusion through supervised fine-tuning. Although deeper reasoning and carefully designed structures have improved multimodal reasoning performance, longer traces also increase inference costs and may allow unsupported assumptions to propagate through subsequent deductions. This overhead is particularly limiting in high-throughput or latency-sensitive applications.

This added discussion will make our methodological comparison with Visionary-R1, Vision-SR1, and related methods explicit. It will highlight DRT’s focus on reducing latency and inference cost through compact intermediate deductions and their associated training objectives.

Thank you again for your careful reading and constructive feedback, which have helped us better position DRT in relation to prior work and clarify its contribution. We hope the proposed revisions address your remaining concerns and would be happy to discuss any further questions or provide additional clarification.

[1] Xia, Jiaer, et al. "Visionary-r1: Mitigating shortcuts in visual reasoning with reinforcement learning." arXiv preprint arXiv:2505.14677 (2025).

[2] Li, Zongxia, et al. "Vision-SR1: Self-rewarding vision-language model via reasoning decomposition and multi-reward policy optimization." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Xu, Guowei, et al. "Llava-cot: Let vision language models reason step-by-step." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.