acceptodds
Withdrawn from ICLR 2027

From Stages to States: Unified Iterative Reasoning for MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Chain-of-Thought (MCoT) substantially enhances the complex reasoning capabilities of Multimodal Large Language Models (MLLMs) by explicitly modeling intermediate reasoning processes. However, most learning-based MCoT methods follow a stage-centric paradigm, where different reasoning behaviors are assigned to predefined stages and optimized separately. This results in fragmented reasoning and a training-inference reasoning-state gap. To address these limitations, we propose ReState, a unified iterative reasoning framework that reformulates multimodal reasoning from a stage-driven pipeline into a state-driven dynamic process. It comprises two complementary components: Unified Reasoning State Modeling (URSM) and Closed-Loop Reasoning-State Learning (CRSL). URSM unifies reasoning construction, correct reasoning preservation, and erroneous reasoning correction as reasoning-state transitions within a shared state space, enabling these heterogeneous reasoning behaviors to be learned jointly. CRSL further incorporates self-generated historical reasoning states during training and recursively updates the current state during inference, thereby mitigating the reasoning-state gap and enabling iterative self-refinement without requiring an additional reflection module, critic, or independent correction stage. Furthermore, we construct VersaCoT-10K, a high-quality CoT dataset spanning eight visual reasoning domains, which provides diverse supervision for multi-domain reasoning and reasoning-state refinement. Extensive experiments demonstrate that ReState exhibits strong adaptability across different MLLM backbones and achieves strong performance on multiple public reasoning benchmarks, supporting the shift from stage-centric reasoning toward state-centric reasoning. Our data and code will be open source.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Unified Iterative Reasoning, Multimodal Large Language Models, Restate Framework, From Stages to States

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ggN3OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ReState, a unified iterative reasoning framework for MLLMs that treats intermediate reasoning chains as updatable states. Unified Reasoning State Modeling (URSM) jointly trains a single model to construct reasoning from empty states, preserve correct reasoning, and correct erroneous reasoning. Closed-Loop Reasoning-State Learning (CRSL) incorporates self-generated historical states during training and recursively refines reasoning during inference to mitigate the training–inference reasoning-state gap. The authors also introduce VersaCoT-10K, a dataset of 10,000 single-image multiple-choice samples spanning eight visual reasoning domains.

Critical strengths
  1. The paper addresses a practically relevant problem: the mismatch between pre-constructed reasoning chains used during training and model-generated reasoning encountered at inference. Incorporating self-generated historical states is a well-motivated approach to mitigating this mismatch.
  2. ReState offers a simple, unified framework for reasoning construction, preservation, and correction within a single model. Reusing the same model for iterative refinement avoids the need to deploy a separate critic or correction model.
Critical weaknesses
  1. The paper uses CRSL to iteratively refine reasoning at inference time to improve answer accuracy, but does not provide an explicit adaptive stopping criterion. According to Appendix C, the refinement trajectory is fixed at nine rounds, and the peak accuracy is reported. However, it is unclear whether the best-performing round is selected using a validation set. In practical deployment, GT answers are unavailable to identify the best round, and the paper does not specify a mechanism for determining when to stop or which intermediate output to retain. This leaves a gap between the reported evaluation protocol and a deployable inference procedure.
  2. The incremental contributions and performance advantages over MIND require stronger justification. The construction of positive and negative reasoning states follows MIND’s RAD paradigm, and the ScienceQA and M³CoT experiments use the corresponding RAD versions. The main additions thus consist of unified state updates within a single model and feedback from self-generated historical states. Yet ReState outperforms MIND by only 0.18–0.91 percentage points on M³CoT and 0.90–1.70 percentage points on VersaCoT-10K. Without standard deviations across independent runs or statistical significance tests, and given that the reported results represent peak accuracy along the refinement trajectory, the robustness of these gains remains unclear. Moreover, in the non-iterative setting, although inference cost is comparable to MIND, the gains on VersaCoT-10K are narrow to 0.15–0.90 percentage points.
Questions for authors
  1. To clarify whether the gains reflect consistent error correction rather than answer fluctuations across rounds, could the authors report the proportions of samples transitioning from incorrect to correct and from correct to incorrect at each refinement round?
  2. The dataset’s reasoning annotations and quality assessment both rely on Gemini. Did the authors conduct human verification? Could you provide a user study to substantiate the dataset’s quality and reliability?
  3. The construction of VersaCoT-10K is described only briefly in Section 5, with most details deferred to Appendix A. Could the authors provide more specific construction details in Section 5? Otherwise, readers may find it difficult to understand how the dataset is constructed.
  4. All experiments are conducted on Qwen series. Could the authors evaluate ReState on MLLMs from other model families to verify its generality?
Reviewer_9JzSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ReState, a unified iterative reasoning framework for multimodal large language models, aiming to address fragmented reasoning in stage-based MCoT methods and the mismatch between training and inference reasoning states. ReState introduces two components: Unified Reasoning State Modeling (URSM), which jointly learns reasoning construction, preservation, and correction, and Closed-Loop Reasoning-State Learning (CRSL), which incorporates self-generated historical states during training and enables iterative refinement during inference. The authors also construct VersaCoT-10K, a dataset covering eight visual reasoning domains. Experiments on VersaCoT-10K, ScienceQA, and M3CoT across three Qwen backbones show improvements over existing baselines. Additional ablation studies and iterative refinement analyses are conducted to evaluate the contributions of the proposed components.

Critical strengths

1. Robust empirical improvements across multiple benchmarks and model backbone architectures. On three multimodal reasoning benchmarks and three different Qwen model backbone architectures, ReState consistently achieves higher accuracy compared to Direct SFT, Multimodal-CoT, and MIND. These results demonstrate that the framework is effective across diverse reasoning tasks and model architectures, rather than being tailored to a single experimental setting.

2. A unified and practical framework for iterative reasoning optimization. ReState integrates the construction, retention, and refinement of reasoning steps into a single model, eliminating the need for separate external critics or correction modules. Ablation studies on individual components indicate that unified state modeling, the learning of historical states, and iterative optimization all contribute to performance gains. This provides strong empirical support for the overall framework, although the underlying mechanisms warrant further investigation.

Critical weaknesses

1. Limited novelty and unclear distinction from existing self-correction methods. The key ideas behind unified reasoning state modeling and training with self-generated historical states substantially overlap with prior training-based self-correction approaches, including RISE, SCoRe, and Sherlock. However, the paper does not adequately clarify the methodological differences between ReState and these closely related works, and it lacks direct comparisons with representative self-correction baselines. As a result, it remains unclear whether ReState introduces meaningful methodological advances beyond existing iterative refinement frameworks.

2. Insufficient evidence supporting the core motivation and concerns about experimental fairness. Although the paper aims to reduce the mismatch between reasoning states encountered during training and inference, the experimental results only show that incorporating historical states improves prediction accuracy. They do not directly establish whether the underlying distribution mismatch has actually been reduced. In addition, ReState reports the highest accuracy obtained across nine inference rounds, while the competing methods generally involve considerably fewer reasoning steps. Without direct evidence of improved distribution alignment or evaluations under comparable inference budgets and consistent protocols, it is difficult to determine whether the reported gains genuinely result from the proposed mechanisms.

  1. RISE: Recursive Introspection: Teaching Language Model Agents How to Self-Improve. NeurIPS 2024.

  2. SCoRe: Training Language Models to Self-Correct via Reinforcement Learning. ICLR 2025.

  3. Sherlock: Sherlock: Self-Correcting Reasoning in Vision-Language Models.

Questions for authors

1. Novelty compared with existing self-correction methods. How does ReState fundamentally differ from prior methods such as RISE, SCoRe, and Sherlock? What new capabilities do URSM and CRSL provide beyond existing training-based self-correction approaches?

2. Validation of the training and inference mismatch. The paper claims that CRSL reduces the mismatch between reasoning states during training and inference. Can the authors provide direct evidence that this mismatch is actually reduced? How can they distinguish this effect from the benefits of additional training data or reasoning supervision?

3. Necessity of unified reasoning-state modeling. Can the authors compare URSM with a stage-based approach trained using the same positive, negative, and historical reasoning states under equal training budgets? This would help determine whether the improvements come from unified modeling itself or simply from additional training supervision.

4. Fairness of inference comparisons. ReState reports the highest accuracy across nine refinement rounds, while competing methods generally use fewer inference steps. Why is the peak accuracy reported instead of the performance at a fixed round? Can the authors provide comparisons using the same inference budget, with the number of rounds selected on validation data?

Reviewer_65biOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that learning-based multimodal CoT methods are "stage-centric": separate models or objectives handle CoT generation and answer inference, which fragments reasoning and causes a train–inference mismatch, because the answer stage is trained on gold CoTs but sees generated ones at test time. ReState trains a single MLLM as a state-update operator, (S_{t+1}, A_{t+1}) = M(X, S_t). The current state S_t is empty, a correct CoT, or a corrupted CoT (URSM), and during training it is also drawn from the model's own earlier outputs kept in a small per-sample history pool (CRSL). At inference the model is applied recursively for up to 9 rounds. The authors also build VersaCoT-10K, 10K multiple-choice VQA items from 8 domains with Gemini-generated and Gemini-filtered CoTs. On VersaCoT-10K, ScienceQA and M³CoT, with three Qwen backbones, ReState beats direct SFT, Multimodal-CoT and MIND, by roughly 0.2–1.8 points over MIND.

Critical strengths
  1. A single deployed model, with consistent though small gains across three backbones and three datasets. Merging generation and correction into one model removes MIND's two-model deployment, and training is reported to be about 31% faster. The improvement direction is consistent in every backbone–dataset pair, which makes the approach a practical simplification even if the gains are modest.
Critical weaknesses
  1. Peak selection on a saturated, noisy trajectory. Appendix C states that ReState runs a fixed 9-round trajectory and reports “the peak dataset-level accuracy along this trajectory,” i.e., the best round is chosen on the test set. In Fig. 4, almost all of the gain comes from round 0 to 1. VersaCoT-10K has a 1,000-item validation split that could have been used to fix the number of rounds in advance.
  2. The cost of iteration is not accounted for. Nine refinement rounds cost roughly 10× the inference of a single pass, but the paper claims cost parity with MIND only for the non-iterative setting, where the reported gain over MIND is just 0.15–0.90 points.
  3. What's the difference with rior work on self-correction training? ReState’s core recipe has three parts: the model conditions on a previous (possibly wrong) attempt, it is trained on its own generated attempts, and it refines iteratively at inference. This is essentially the setting of RISE (Qu et al., NeurIPS 2024) and SCoRe (Kumar et al., ICLR 2025). SCoRe in particular already identifies the mismatch between offline correction data and the model’s own error distribution as the central problem, which is the same motivation as CRSL’s “reasoning-state gap.” In the multimodal setting, R3V (Cheng et al., NAACL 2025) also trains VLMs to self-reflect using their own correct and incorrect rationales. None of these works is discussed or compared, and the only learned baselines are Multimodal-CoT and MIND. It is therefore unclear what the “state-centric” formulation adds beyond existing self-correction training.
Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.