acceptodds
Withdrawn from ICLR 2027

Think, Compress, Answer: Reinforcement Learning for Multimodal Grounding via Reasoning Compression

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Extended reasoning has substantially improved multimodal large language models (MLLMs) on tasks requiring composi- tional and relational inference. However, multimodal ground- ing imposes a distinct requirement: reasoning must ultimately converge on a target-specific compact representation that sup- ports precise localization. As reasoning trajectories grow longer, they often expand into broad scene narratives and peripheral relational details, which can dilute grounded per- ceptual evidence and lead to ambiguous or incorrect predic- tions. Motivated by the information bottleneck principle, we propose Think-Compress-Answer (TCA), a reinforcement learning framework that decomposes multimodal grounding into three phases: free-form reasoning, target-aware context compression, and answer generation conditioned on the com- pressed context. TCA is optimized through a multi-stage Group Relative Policy Optimization (GRPO) with progressive rewards that establish the structured output protocol, mini- mize reasoning-specific information while preserving target- relevant content, and directly improve localization accuracy. This staged design enables MLLMs to retain the benefits of extended reasoning while distilling it into the minimal con- text required for grounding. Experiments across six multi- modal grounding benchmarks show that TCA attains substan- tial zero-shot performance gains, demonstrating the effective- ness of reasoning compression

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Learning, Information Bottleneck, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xKFTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Think-Compress-Answer (TCA), a reinforcement learning framework for multimodal grounding that inserts an explicit compressed context between reasoning and answer generation. A four-stage GRPO curriculum optimizes output format, compression rate, compression fidelity, and localization accuracy.

Critical strengths
  1. The method addresses a clear task-specific concern: retaining useful reasoning while focusing the final prediction on target-relevant evidence.

  2. Experiments cover both image and video segmentation and show consistent improvements over the base model across six benchmarks.

Critical weaknesses
  1. The compression-rate reward is theoretically reasonable, but the paper does not show that it produces target-relevant and semantically sufficient compression. It also lacks comparisons with simpler alternatives, such as length penalties, summarization rewards, and target-description rewards.

  2. The training comparisons are incomplete. The paper does not compare the proposed method with SFT, vanilla GRPO, or an RL baseline that directly predicts localization information without reasoning traces. Therefore, it is unclear whether the performance gains come from the proposed method or the training data.

  3. No appendix or supplementary materials are provided in current PDF.

Questions for authors
  1. The ablations specify an “equal optimization budget,” but the allocation of training data and computational resources from omitted stages remains unclear. Please clarify how these resources are redistributed. Also, including the S1+S3+S4 variant and a single-stage joint-training baseline would help establish the necessity of staged optimization and its ordering.

  2. Although task performance improves, the extent to which hallucination and drift in the reasoning outputs are reduced remains unclear. The evaluation does not discuss whether training makes the original reasoning (T) more target-focused, the compressed text (c) more accurate, or the final answer less susceptible to erroneous or irrelevant reasoning.

  3. The added compression field may not constitute an effective information bottleneck. Under standard autoregressive generation, answer tokens can still attend to the full reasoning trace, allowing hallucination and drift to influence the answer. A comparison experiment (e.g. masking or removing (T) during answer generation) can better demonstrate independent contribution of the compressed context.

If the weaknesses and questions are well responded, I will raise my ratings accordingly.

Reviewer_3a7XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Think-Compress-Answer (TCA), a reinforcement learning framework for multimodal grounding that decomposes inference into reasoning, target-aware context compression, and answer generation. TCA is trained with a four-stage GRPO curriculum that progressively enforces output formatting, encourages information compression, improves compression fidelity through an MLLM judge, and optimizes grounding accuracy. Using Qwen2.5-Omni-7B with SAM2, the method is trained on RefAVS and ReVOS data and evaluated in a zero-shot setting on six benchmarks covering referring expression and referring video object segmentation. The experiments, stage-wise ablations, backbone and decoder comparisons, and qualitative analyses show that TCA generally improves over reasoning-guided baselines and achieves competitive performance with supervised methods.

Critical strengths

The paper introduces an explicit intermediate context between multimodal reasoning and grounding, and organizes training around several interpretable reward components. It evaluates the framework across multiple grounding benchmarks and includes ablations over training stages, backbones, decoders, and judge models, providing a useful empirical basis for further studying reasoning compression.

Critical weaknesses

The central weakness is that the paper does not sufficiently validate its core claim that reasoning compression is necessary for precise localization or demonstrate that the proposed output structure forms a genuine information bottleneck.

Questions for authors

1.The core motivation frames compression as a necessary condition for precise localization, but this claim lacks direct evidence. For tasks involving relations among multiple objects, occlusion, or temporal changes, accurate localization may depend on relatively rich contextual information. It is vital to provide analyses or experiments demonstrating that compression is the key factor driving the observed improvements.

2.It is unclear whether the proposed method genuinely establishes an information bottleneck, since the answer may still bypass the compressed text. The authors should clarify the conditional distribution used for answer generation and compare settings that use only the compressed context, retain the complete reasoning, or delete, shuffle, or replace the compressed text, in order to verify the sufficiency and actual role of the compressed context.

3.The connection between the practical compression reward and the mutual-information upper bound is unclear. Eqs. (8-10) apply a sigmoid to a KL-based information-rate estimate, which is not equivalent to directly minimizing the upper bound and may weaken its information-bottleneck interpretation. Please compare this reward with raw information-rate and length-regularization objectives, and report the relationships among compression length, information rate, target relevance, and localization accuracy.

4.The third-stage reward mixes compression-quality evaluation with ground-truth target supervision, so the source of the observed improvement is not sufficiently disentangled. It will be helpful to compare target-only, compression-only without the highlighted target, and no-Stage-3 reward settings. It will also be useful to analyze samples with correct and incorrect reasoning separately, to clarify whether Stage 3 primarily improves compression or helps correct reasoning errors.

5.The experimental report contains numerical and setup inconsistencies that affect the interpretation of the gains and reproducibility. Tables B1/B3 report a ReVOS baseline of 38.4, whereas Table B7 reports 48.8 for the unoptimized model, despite identical TCA results of 60.5. In addition, the paper lists 1,258 ReVOS videos, while the supplement lists 512 videos and 1,258 expressions; statistics should be clarified and unified.

Reviewer_kdNYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to use extended reasoning for multimodal grounding without retaining too much irrelevant scene information. Think-Compress-Answer adds a compressed text field between the reasoning trace and the grounding answer. It uses four GRPO stages for output format, a compression-rate surrogate, compression fidelity, and grounding accuracy. The paper reports clear improvements over the base model and includes stage ablations under an equal optimization budget.

Critical strengths
  • The problem is useful. Grounding requires selecting the correct target, but a long reasoning trace may also describe many irrelevant objects and relations. This issue is not specific to one dataset, and related work has studied the tension between reasoning length and visual grounding. TCA evaluates the same training idea on both image and video referring segmentation. These are substantial gains over the unoptimized backbone.

  • The experiments provide useful evidence for the training curriculum. Table 3 explicitly uses an equal optimization budget and includes S1+S4 and S1+S2+S4, rather than only comparing the full method with the base model. These comparisons support the value of the intermediate stages.

Critical weaknesses
  • My main concern is that the conceptual difference from DR2Seg is smaller than the paper's discussion suggests. DR2Seg already generates a reasoning trace, a target description, and an answer. During training, a second rollout uses the image and description to check whether the description supports a correct answer. It also discusses the description as an information bottleneck and introduces a reasoning-length reward. Omni-R1, ReVSeg, and CoT-RVS also use target-specific descriptions between high-level reasoning and localization, although their information flow and training differ.

TCA's more specific contribution is the conditional likelihood-ratio reward, the GT-aware fidelity judge, and their four-stage training schedule. This combination may be useful, but the current experiments do not show why it is preferable to description self-verification or a simpler summary/length objective.

  • The authors do discuss degenerate compression, including empty, generic, or query-copying outputs, and they refer to failure cases in the supplement. This should be acknowledged. However, the supplied main paper gives little evidence about when compression removes information needed to distinguish targets. Figure 4 reduces a query involving a black guitar and a left-side location to “guitarist”. This can work if the target is already unambiguous, but it leaves open what happens with several similar people or with temporal relations.
Questions for authors
  • Under the same backbone, data, optimization budget, and comparable feedback cost, does the proposed reward improve over a DR2Seg-style description reward or a simpler length-matched summary objective? Please provide any existing comparison and explain which part supports the specific benefit of the rate reward.
  • Which checkpoint and training data produced the Table 2 results? Was MeViS excluded when training that baseline? Please distinguish reproduced results from published numbers and specify the System 2 and inference settings.
  • Please provide the judge model/version and prompt, any fidelity-reward validation, and available statistics for T/c lengths, the rate surrogate, and training/inference cost. How should the optimization claim for Equation (10) be interpreted relative to Equation (8)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.