- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SC-CoIT studies error recovery in interleaved text-image reasoning. Starting from ground-truth Zebra-CoT trajectories, the method samples model-generated errors and constructs recovery examples for two failure types: valid textual actions that are incorrectly rendered, and invalid textual actions that require correction. Textual reflections guide image recovery, while a visual rejection signal prompts textual recovery; the corrected nodes reconnect to the original reference trajectory. Iterative error replay refreshes the training examples using successive checkpoints. Built on BAGEL-Zebra-CoT, the method reports improved performance on VisWorld-Eval and analyzes error detection and recovery behavior.
Critical strengths
- Learning to recover from intermediate errors is an interesting and practically relevant direction for interleaved multimodal reasoning.
- The two complementary recovery branches provide a meaningful approach to correcting both textual and visual mistakes.
- Substantial gains on VisWorld-Eval, branch-specific ablations, and detection-and-repair measurements provide encouraging empirical evidence.
Critical weaknesses
- The coarse error taxonomy and lack of analysis of scenario-specific failures, compound errors, and recovery rates by subtype leave the transferability of the learned correction mechanism unclear.
- How are complete recovery trajectories validated to ensure that VLM-generated reflections agree with ground-truth recovery nodes and remain consistent with subsequent reasoning, and how are inconsistent samples handled?
- Please count how many tasks were actually done correctly because of reflection. Did the performance improve because of reflection or because more rounds of continued training were done?
Questions for authors
- What evidence isolates the contribution of explicit reflection from additional training and error-recovery supervision without reflection?
- What validation evidence shows that VLM-generated reflections agree with the ground-truth recovery nodes and remain consistent with the subsequent trajectories?
4 comments
Response to Questions2
Appendix A.1 provides the inputs and prompt used by the VLM to generate reflection. As shown there, the VLM sees the current instruction, the previous correct state, the incorrect image, and the GT image for the current step at the same time. The prompt asks the VLM to keep the language and style of the original reasoning trajectory, point out the specific error, and state the corrections needed to recover the GT state. Since the VLM uses the GT trajectory as a reference, its reflection is consistent with the later trajectory in most cases. We also added annotated examples in Appendix A.1 (Figures 5 and 6) to further show this consistency.
a gentle follow-up on our response above
We wanted to follow up on the comparisons and examples in our reply. Have the extra-training and binary-feedback comparisons helped clarify what explicit reflection adds? We also included examples in Appendix A.1 to show how reflections match the recovery steps. We would appreciate hearing whether these are helpful and what still needs clarification.
The authors added extra-training and no-reflection comparisons and qualitative examples, but my concerns remain unresolved because the response lacks systematic validation of recovery trajectories, clarification of how inconsistent samples are handled, and error-subtype analyses; I therefore maintain my original score.
Response to Questions1
To separate the effects of explicit reflection, error recovery supervision without reflection, and extra training, we added two experiments and report the comparison below.
First, we trained a model without explicit reflection. For Draw Errors, we replaced the reflection text with a label that only indicates whether the image is correct. For Policy Errors, we replaced the red-x generation task with simple binary classification using the Reflect Head described in the paper. Without explicit error diagnosis, the model improves only slightly over the single-branch Reflect Head. This shows that adding Draw feedback that only judges image correctness brings a small gain on top of binary Policy gating. Binary feedback can trigger a retry, but it does not say what needs to be repaired. Redrawing is therefore closer to resampling under the original conditions. Without specific correction conditions, extra sampling in the continuous space has more difficulty recovering the visual evidence needed for later reasoning than sampling in the discrete text space.
We also ran an extra-training experiment. We used only correct training trajectories and trained for the same number of extra steps. The model shows overfitting. This indicates that extra training alone does not bring gains.
Table 1: Comparison of extra training, negative-sample supervision, and the full model