acceptodds
Under review as a conference paper at ICLR 2027

Sperry: Multimodal Reasoning with Unified Model

Abstract

Current test-time inference in multimodal models largely relies on textual thinking traces, which are insufficient for reasoning processes that are hard to verbalize, such as spatial reasoning and visual geometry. Existing alternatives use external tools or latent-space reasoning to obtain visual thoughts, but they either lack flexible pixel-level generation or fail to expose interpretable intermediate reasoning. Unified multimodal models that generate both text and images offer a promising foundation, yet their post-training for rigorous multimodal reasoning remains underexplored due to the lack of high-quality interleaved text-visual reasoning data and effective training recipes. We introduce Sperry-43K, a dataset of interleaved text-visual reasoning traces across diverse visual reasoning tasks, together with SperryBench, a stratified evaluation benchmark. We further propose a complete post-training pipeline combining two-stage supervised fine-tuning with reinforcement learning. To reduce training-inference mismatch during RL, we introduce Generative Self-Prompted Policy Optimization (GSPPO). Our 14B model improves the base unified multimodal model from 19.1% to 51.9% on SperryBench, a held-out split of its own task generators, outperforming much larger public models such as Qwen3.5-397B and Gemma-4-31B evaluated zero-shot, as well as a latent-reasoning baseline trained on the same data, and providing a foundation for generative multimodal reasoning. Code and data will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unified Model, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zMUUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work teaches a unified multimodal model to "think in images" by emitting intermediate visual states during reasoning. The authors assemble Sperry-43K — 42,682 interleaved text-image traces across six tasks spanning mental simulation/imagination and 3D spatial intelligence — together with the 2,249-sample SperryBench, and pair them with a full post-training recipe: two-stage SFT that trains the understanding and generation branches in sequence, followed by GSPPO, an RL stage in which the generation branch is optimized on prompts sampled from the model's own text policy to close the train–inference mismatch. Built on Bagel, the 14B model moves from 19.1% to 51.9% on SperryBench, surpassing far larger open models such as Qwen3.5-397B and Gemma-4-31B, as well as a latent-reasoning baseline trained on identical data.

Critical strengths
  1. A complete and well-motivated training recipe. Simulator-rendered data makes intermediate states correct by construction; sequential two-branch SFT resolves the mismatched optimization horizons of text and image modeling; GSPPO directly targets the prompt-distribution gap faced by the generation branch. The pipeline is a useful reference for post-training UMMs for reasoning.
  2. Well-chosen task scope. Mental simulation and spatial intelligence are exactly where text-only chains fall short, and carrying state updates through generated intermediate images is a sensible and promising paradigm.
Critical weaknesses
  1. The headline evaluation is confined to the authors' own benchmark, whose six task families mirror the training distribution; there is little evidence of generalization to independent benchmarks, and 51.9% absolute accuracy leaves the method's real competitiveness in open settings unclear.
  2. Gaps in comparison and supervision. The latent-reasoning baseline comparison does not establish matched capacity or training budget, and the RL reward verifies only the generated images programmatically — the textual reasoning receives no process-level supervision.
Questions for authors
  1. Are there generalization results on established spatial-reasoning or geometry benchmarks outside SperryBench?
  2. How much of the improvement comes from GSPPO versus DiffusionNFT with dataset prompts?
  3. What are the per-image generation latency and the overall inference cost?
6 comments
AuthorsOpenReview ↗

Rebuttal for reviewer zMUU (1/2)

Thank you for the detailed review and for recognizing both the recipe and the task scope. We answer your points in order.

1. Generalization to independent benchmarks (Weakness 1, Question 1). We agree that generalization has to be tested beyond SperryBench, since its six task families mirror the training data. Our experiments test it on independent benchmarks that probe the spatial-reasoning and visual-geometry abilities you ask about, and test retention on general benchmarks.

BabyVision subtype (%)Qwen3-VL-235BBagelSperry
2D pattern completion25.060.075.0
3D pattern completion29.655.677.8
3D visualization29.644.444.4
Maze11.720.030.0
BenchmarkReferenceSperry
VisWorld-Eval Maze-I, complete correct route (%)37.1 (after SFT)52.7
MMBench v1.182.4 (Bagel)83.5
GenEval0.82 (Bagel)0.78

Sperry scores higher than the much larger Qwen3-VL-235B on four of the five BabyVision subtypes, and the VisWorld-Eval mazes were generated and rendered independently of ours. In addition, we also evaluated SpatialViz-Bench (ICLR 2026), whose mental-rotation tasks use transformations that none of our training tasks contains.

SpatialViz-Bench (%)BagelSperrySperry - Bagel
Mental rotation25.233.3+8.2
of which 2D rotation20.833.5+12.8

Sperry therefore generalizes beyond SperryBench to independent spatial-reasoning benchmarks, including rotation tasks outside its training data, while largely retaining Bagel's general abilities. We will highlight these results in the revised paper.

2. Matched capacity and budget in the Monet comparison (Weakness 2). We agree that the comparison with the latent-reasoning baseline Monet is meaningful only at matched capacity and training budget, and our comparison is matched on both.

MonetSperry
Model that writes every reasoning step and the answerQwen2.5-VL-7B7B understanding branch
Intermediate visual stateslatent tokensgenerated by a 7B image generator
Training data and budgetSperry-43K, official recipe with RL, budget matched to oursSperry-43K

Sperry's second 7B branch is the image generator, the very component whose effect the comparison measures. Under these matched conditions:

SperryBench (%)MazeBlockCountCubeViewLineTrackCountingJigsawOverall
Monet (with RL)2.516.251.034.561.648.935.8
Sperry SFT (no RL)41.826.851.842.362.136.243.5
Sperry70.031.454.056.759.240.251.9

Even without RL, Sperry is 7.7 points ahead of Monet with RL, and the margin is largest on the tasks that require simulating state changes (Maze, BlockCount and LineTrack). With matched capacity and budget, reasoning with explicit generated images therefore clearly beats reasoning with latent visual tokens. We will state both points explicitly in Sec. 5.2.

AuthorsOpenReview ↗

Rebuttal for reviewer zMUU (2/2)

3. Contribution of GSPPO vs. DiffusionNFT (Question 2). We agree that this split should be made explicit. The two RL variants share the same pipeline, data and reward and differ only in where the generator's prompts come from.

SperryBench (%)MazeBlockCountCubeViewLineTrackCountingJigsawOverall
SFT41.826.851.842.362.136.243.5
+ RL with dataset prompts (DiffusionNFT)59.727.954.655.964.341.050.6 (+7.1)
+ RL with self-written prompts (GSPPO)70.031.454.056.759.240.251.9 (+1.3)

Of the 8.4 points that RL adds, 7.1 come from generation-branch RL and 1.3 from self-prompting. GSPPO trains the generator on prompts written by the model itself, which are exactly the prompts it receives at inference, and its extra gain is concentrated on Maze and BlockCount, the two tasks whose answer is read off the generated image. Self-prompting therefore adds a further gain on top of an already strong image-RL stage. We will state this split explicitly in Sec. 5.3.

4. Per-image latency and inference cost (Question 3). Measured over our SperryBench evaluation runs:

Sperry on one GB200 GPUTime
One intermediate image (50 denoising steps)6.8 s
Image generation per problem (3.4 images on average)about 23 s
Full problem (text and images)35 s
Per problemText writtenTimeSperryBench (%)
Sperry, one H100a few hundred tokens and 3.4 images46.0 s51.9
Qwen3.5-9B with vLLMabout 24k thinking tokensabout 3 min36.1

Image generation takes about two thirds of Sperry's time, but Qwen3.5-9B has to spell out every intermediate state in words. A few generated images cost less time than the long textual chains of text-only thinking, so reasoning through images makes Sperry faster as well as more accurate. We will add the per-image latency to Sec. 5.3.

Thank you for the additional experiments. The new results help address my concern regarding cross-benchmark generalization. However, two important concerns remain:

  1. Matching the 7B understanding branch does not establish system-level capacity matching, as Sperry additionally employs a 7B image generator. Could the authors clarify the total parameter counts and actual training computational budgets of both systems? If they are not matched, the claim of superiority over latent visual reasoning should be appropriately qualified.

  2. Does GSPPO optimize only the image generation branch, or does it also update the text reasoning branch? If the latter receives no direct reward or process-level supervision, how does the method ensure the correctness of the textual reasoning that guides image generation?

While I appreciate the additional results, these concerns remain insufficiently addressed. I therefore maintain my current rating pending further clarification.

Reviewer_apSgOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Sperry, a unified multimodal model that reasons through interleaved text and generated images. It provides Sperry-43K and SperryBench, combines two stages of supervised training with GSPPO, and evaluates how generated visual states contribute to reasoning accuracy.

Critical strengths
  1. Clear motivation and useful data contribution. The paper addresses an important question: whether generating intermediate visual states can help solve problems that are difficult to reason about through text alone. The dataset provides explicit intermediate states, with simulator ground truth for four task families, making this question easier to study systematically.

  2. Convincing evidence. Table 8 includes useful controls using the same backbone. Accuracy drops from 43.5% to 33.6% with text-only SFT, and replacing generated images at inference causes further substantial degradation. These results provide evidence that the model benefits from learning and using intermediate visual states.

Critical weaknesses
  1. Generalization remains partially explored. The external evaluations are useful, but transfer to unseen transformation rules remains unclear. The second backbone experiment also validates only SFT, rather than the complete training pipeline.

  2. Modest additional benefit and unclear training stability. GSPPO improves overall accuracy from 50.6% to 51.9% over DiffusionNFT (Table 7), with gains mainly on Maze and BlockCount, while Counting drops from 64.3% to 59.2%. However, the paper does not provide training curves or clearly explain how the reported checkpoints were selected for either RL variant. Results across independent training runs are also missing. These details matter because the additional gain is relatively small and uneven across tasks, leaving its robustness and sensitivity to checkpoint selection unclear.

Questions for authors

Please see the concerns in Weaknesses. I would be happy to raise my score if the authors can address these concerns convincingly.

5 comments
AuthorsOpenReview ↗

Rebuttal for reviewer apSg (1/2)

Thank you for the constructive review, and for recognizing the data contribution and the evidence from our same-backbone controls. We address both concerns below.

1. Generalization beyond SperryBench and transfer to unseen rules (Weakness 1). We agree that transfer to transformation rules outside the training set is the right test of what Sperry learns. Below are all our external evaluations, including one where Sperry does not improve. BabyVision's 2D and 3D pattern completion require inferring the rule behind a visual pattern and completing it, and none of our six training tasks contains such rules.

BabyVision (%)Qwen3-VL-PlusQwen3-VL-235BBagelSperry
2D pattern completion (unseen rules, 20 items)30.025.060.075.0
3D pattern completion (unseen rules, 18 items)27.829.655.677.8
3D views (27 items)27.829.644.444.4
Maze (20 items)2.511.720.030.0
BenchmarkReferenceSperry
VisWorld-Eval Maze-I, complete correct route (%)37.1 (after SFT)52.7
VisuLogic (%), random choice 25.013.1 (Bagel)24.5
MMBench v1.182.4 (Bagel)83.5
GenEval0.82 (Bagel)0.78

On the unseen pattern-completion rules, Sperry answers 15 of 20 and 14 of 18 items correctly, against 12 and 10 for Bagel. VisWorld-Eval Maze-I applies our maze rule to independently generated and rendered mazes, so it measures transfer across renderings rather than to a new rule. We also evaluated SpatialViz-Bench (ICLR 2026), whose mental-rotation tasks use transformations that none of our training tasks contains.

SpatialViz-Bench (%)BagelSperrySperry - Bagel
Mental rotation25.233.3+8.2
of which 2D rotation20.833.5+12.8

Sperry therefore transfers to transformation rules outside its training tasks, most clearly to rotation, with a 12.8-point gain on 2D rotation, and to pattern completion, while Bagel's general abilities are largely retained. Transfer to rules far from our six task families, such as finding the differences between items or paper folding, remains much harder; closing that gap requires scaling the training data to more task families.

2. Full pipeline on the second backbone (Weakness 1). We agree that the pipeline should be shown on a second backbone beyond the SFT stage, so we ran the generation-branch RL stage on SenseNova-U1 as well, for only 10 RL steps.

SperryBench (%)Base+ our SFT recipe+ RL (DiffusionNFT)
Bagel19.143.5 (+24.4)50.6 (+7.1)
SenseNova-U117.863.8 (+46.0)67.2 (+3.4)
SenseNova-U1 (%)MazeBlockCountCubeViewLineTrackCountingJigsawOverall
+ our SFT recipe70.860.544.963.369.174.063.8
+ GSPPO, only 10 steps77.564.948.363.273.176.167.2

Our RL stage therefore generalizes to a new model: with only 10 RL steps, it adds a further 3.4 points on SenseNova-U1 and improves five of the six tasks by 2 to 7 points. Together with the SFT recipe, which gains even more than on Bagel, both training stages transfer to a second unified architecture. We will add these results to the revised paper.

AuthorsOpenReview ↗

Rebuttal for reviewer apSg (2/2)

3. Size, checkpoint selection and stability of the GSPPO gain (Weakness 2). We agree that the gain of GSPPO over DiffusionNFT has to be shown not to depend on checkpoint selection, so we give the selection rule for both variants and measure this sensitivity directly.

SperryBench (%)MazeBlockCountCubeViewLineTrackCountingJigsawOverall
SFT41.826.851.842.362.136.243.5
+ DiffusionNFT (dataset prompts)59.727.954.655.964.341.050.6 (+7.1)
+ GSPPO (self-written prompts)70.031.454.056.759.240.251.9 (+1.3)

Both variants keep the checkpoint with the highest training reward, a rule that never looks at SperryBench, so neither is tuned on the benchmark. They differ only in the prompts the generator is trained on: DiffusionNFT uses dataset prompts, while GSPPO uses the prompts the model writes itself, the ones it receives at inference. GSPPO's gain therefore appears on the two tasks whose answer is read off the generated image: +10.3 points on Maze and +3.5 on BlockCount. Checkpoint choice cannot explain these gains: two high-reward checkpoints of a GSPPO run (epochs 43 and 65) differ by only 1.2 points on Maze and 0.1 on BlockCount. The gain of GSPPO over DiffusionNFT is therefore robust to checkpoint selection, and on top of the large gain of the RL stage, self-prompting continues to improve the model. We will add the training curves of both variants and the selection rule to the revised paper.

Hi Reviewer apSg, as the discussion period is coming to a close, we would appreciate it if you could review our rebuttal and let us know whether it addresses your concerns. We would be happy to discuss any remaining questions or concerns. Thank you for your time and thoughtful feedback!

Reviewer_697hOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper constructs simulator-based interleaved text-image reasoning trajectories and combines staged supervised fine-tuning with generation-branch reinforcement learning to improve visual reasoning in a unified multimodal model. Its main contribution is an integrated data and training pipeline that uses visual intermediate states, rather than simply producing longer textual reasoning traces.

Critical strengths
  1. The data and overall improvements are valuable. Accuracy on the paper's own benchmark increases from approximately 19% to 52%, with additional validation on external benchmarks and a second backbone. This provides positive evidence beyond a single experimental setting.

  2. The intermediate-image ablations directly support the usefulness of visual reasoning states. Replacing intermediate images substantially reduces performance, and text-only SFT with matched training epochs is weaker. This is stronger evidence of their role in answering questions than qualitative visualizations alone.

Critical weaknesses
  1. The end-to-end advantage of staged training has not been sufficiently isolated. The joint-training comparison reports image reward rather than final-answer accuracy. Image reward cannot substitute for answer accuracy; an equal-budget joint-training comparison is needed to attribute the gains to the staged design.

  2. Self-prompting offers only a small incremental gain over DiffusionNFT, with declines on some tasks and no repeated-training evidence. It also uses reference history prefixes, leaving its effectiveness on model-generated histories, which may contain errors, unclear.

The data, overall gains, and image-use ablations provide substantial positive evidence. Novelty should be framed around data construction, branch-training schedules, and self-prompting rather than the first visual CoT or image-RL method. The main limitation is attribution of individual components, not evidence that the overall pipeline is ineffective.

Questions for authors

Can you report final-answer accuracy for equal-budget joint training and establish the repeatability of the self-prompting gain? Does that gain persist when the history is generated by the model rather than supplied from reference trajectories?

2 comments
AuthorsOpenReview ↗

Rebuttal for reviewer 697h

Thank you for the careful review, and for recognizing the value of our data, the gains on SperryBench and external benchmarks, and the image-use ablations. We also appreciate your conclusion: "The main limitation is attribution of individual components, not evidence that the overall pipeline is ineffective." We address the attribution questions below.

1. Answer accuracy of joint vs. staged SFT (Weakness 1, Question). We agree that image reward cannot replace answer accuracy, so we trained a one-stage joint SFT model on the same 31,474 traces with an equal training budget and evaluated its final answers with the same protocol.

Same 31,474 tracesAnswer accuracy (%)Traces passing every image check (%)BlockCount traces passing (%)
Joint SFT (one stage)42.922.018.7
Staged SFT (ours)43.541.280.5

These images are what Sperry reasons over. With the same SFT text policy:

GeneratorImage rewardAnswer accuracy (%)
Untrained (Bagel's original)1.725.2
Trained by staged SFT41.243.5

Staged SFT therefore reaches the same answer accuracy with far more reliable visual states, and we will update the joint-SFT row of Table 8 with this model's answer accuracy.

2. Repeatability of the self-prompting gain (Weakness 2, Question). We agree that the gain of self-prompting over DiffusionNFT has to be shown to be stable, so we give the checkpoint selection rule for both variants and measure how much the gain depends on the checkpoint.

SperryBench (%)MazeBlockCountCubeViewLineTrackCountingJigsawOverall
SFT41.826.851.842.362.136.243.5
+ DiffusionNFT (dataset prompts)59.727.954.655.964.341.050.6 (+7.1)
+ GSPPO (self-written prompts)70.031.454.056.759.240.251.9 (+1.3)

Both variants keep the checkpoint with the highest training reward, a rule that never looks at SperryBench, so neither is tuned on the benchmark. They differ only in the prompts the generator is trained on: DiffusionNFT uses dataset prompts, while GSPPO uses the prompts the model writes itself, the ones it receives at inference. GSPPO's gain therefore appears on the two tasks whose answer is read off the generated image: +10.3 points on Maze and +3.5 on BlockCount. Checkpoint choice cannot explain these gains: two high-reward checkpoints of a GSPPO run (epochs 43 and 65) differ by only 1.2 points on Maze and 0.1 on BlockCount. The gain of self-prompting over DiffusionNFT is therefore stable under checkpoint selection, and on top of the large gain of the RL stage, self-prompting continues to improve the model. We will add the training curves of both variants and the selection rule to the revised paper.

3. Self-prompting on model-generated histories (Weakness 2, Question). We agree that what matters is performance on the model's own histories, and all gains we report are measured on them: at test time every SperryBench trace, including all earlier images, is generated by the model itself. A step-by-step analysis confirms that training on reference prefixes carries over to the model's own histories. We scored every intermediate image of the full SperryBench rollouts (five samples per question) against the reference state of its step.

Images correct (%)Step 1Step 2Step 3Step 4
LineTrack, SFT generator45.244.542.638.9
LineTrack, GSPPO generator68.364.662.661.0
BlockCount, SFT generator89.490.493.697.9
BlockCount, GSPPO generator92.892.095.196.2

On LineTrack, the task with the longest traces, the GSPPO generator stays about 20 points ahead at every step, and on BlockCount it is ahead at the first three steps. On Maze, whose traces contain one image, its images are correct in 96.2% of cases against 72.6%. The improvement GSPPO brings to the generator therefore persists when the history is the model's own, and we add this per-step analysis to the appendix.

Hi Reviewer 697h, as the discussion period is coming to a close, we would appreciate it if you could review our rebuttal and let us know whether it addresses your concerns. We would be happy to discuss any remaining questions or concerns. Thank you for your time and thoughtful feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.