acceptodds
Desk rejected at ICLR 2027

InterleaveThinker: Reinforcing Agentic Interleaved Generation

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent image generators have demonstrated impressive photorealism and instruction-following capabilities in single-image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text-image sequence), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open-source Unified Multimodal Models (UMMs) exhibit limited performance in this regard. In this paper, we introduce InterleaveThinker, the first multi-agent pipeline designed to endow frozen image generators with interleaved generation capabilities. Specifically, we employ a planner agent to organize the image-text input sequence, instructing the image generator on the required execution at each step. Subsequently, we introduce a critic agent to evaluate the generator's outputs, identify samples that deviate from the planned instructions, and refine the instructions for regeneration. To implement this pipeline, we construct the Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k to perform a format cold-start. Then we develop Interleave-Critic-RL-13k to reinforce the instruction correction capability within a generation trajectory using GRPO. Since an interleaved generation trajectory may involve over 25 generator calls, full-trajectory optimization is computationally expensive and suffers from difficult credit assignment. We therefore propose **T**rajectory-**D**ecoupled **GRPO** (**TD-GRPO**), which performs single-step optimization with accuracy and step-wise rewards to provide effective learning signals throughout the generation process. InterleaveThinker shows consistent gains across two off-the-shelf image generators and achieves performance comparable to Nano Banana and GPT-5 on interleaved generation benchmarks. Beyond generation, it also substantially improves the base model on reasoning-based benchmarks; for example, on 4-step FLUX.2-klein, WISE increases from 0.47 to 0.73 and RISE from 13.3 to 28.9.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Generation, Intereaved Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vwDHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

InterleaveThinker employs a design consisting of a Planner, a Generator, and a Critic to produce coherent sequences of images and text. The authors propose a comprehensive training pipeline—covering all stages with well-structured data—and incorporate a sound reward mechanism for reinforcement learning (RL). Experimental results across several benchmarks demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper is well-written and organized.

  2. The overall framework is robust, with training data that thoroughly covers the error-correction process.

  3. The experimental evaluation is comprehensive, including performance testing on various benchmarks, as well as ablation studies across different models and configurations.

Critical weaknesses
  1. The scoring provided by Gemini may be unreliable—particularly in tasks like image editing or maintaining consistency for items such as clothing or products—as it often struggles with fine-grained details and is prone to hallucinations. This raises questions regarding the trustworthiness of the resulting labels and supervisory signals.

  2. Furthermore, given that Gemini is tasked with evaluation, generating training data, and providing rewards for RL, how do the authors address the inherent trade-off of having the "referee" also act as a "competitor" (or participant)? Is there any human-based evaluation?

  3. Could the evaluation be expanded to include more specialized editing tasks, such as virtual try-on?

Questions for authors
  1. Please see the Weaknesses.

  2. Is it possible to perform joint fine-tuning that includes the Generator? Currently, the Generator remains static, yet it is likely to produce various errors that could be addressed through such joint optimization.

Reviewer_P4oTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces InterleaveThinker, a framework that enables frozen image generators to produce interleaved text–image sequences. A Planner generates a global execution plan, while a Critic evaluates intermediate images and refines prompts for regeneration. The authors construct dedicated planning and critique datasets and propose Trajectory-Decoupled GRPO (TD-GRPO), which trains the Critic through single-step updates using judgment accuracy and image-quality improvement rewards.

Critical strengths

The approach addresses a meaningful limitation of existing image generators by enabling coherent, multi-step text–image generation without updating the generator. Improvements across both FLUX.2-klein and Qwen-Image-Edit provide encouraging evidence of compatibility across generator backbones.

Critical weaknesses

The methodological novelty of TD-GRPO is insufficiently established. Closely related work, particularly EditThinker, already combines frozen image editors, iterative critique and prompt refinement, step-wise training samples, and GRPO with evaluation-error and quality-improvement rewards. Although cited, this overlap is not adequately analyzed. The paper should clarify what TD-GRPO adds beyond adapting this recipe to interleaved generation and include a comparison using the same Planner, generator, training data, and inference budget.

Questions for authors
  1. What are the specific new mechanisms by which TD-GRPO goes beyond existing single-step GRPO-based correction methods?
  2. Lack of systematic analysis on inference cost and latency.
Reviewer_EUHzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a multi-agent pipeline for interleaved text to image generation with a frozen image generator. The pipeline has three parts: Planner - a LLM that reads input and writes a full plan. Each step has instruction, generator prompt and optional text Generator - An editing model that can edit images from previous step and an instruction Critic - A LLM that looks at the generated result and gives a T/F judgement and writes a refined prompt.

The training data is made using frontier models - Gemini 2.5 Pro and Nano Banana Pro using an elaborate data generation pipeline.

After the data is generated, the authors propose TD-GRPO, an approach for training the critic with two rewards: an accuracy reward and a step reward. Experimental results show good improvements in standard benchmarks.

Critical strengths

The paper proposes a clean and a modular design for multistep interleaved image-text generation. The approach for generating data is sound. Experimental results are strong.

Critical weaknesses

Planner-generator-critic loops with prompt refinement already exist, for example GenArtist, Idea2Img, Reflect-DiT, ReflectionFlow, and EditThinker. The paper cites these. The new part is the extension to multi-step sequences. TD-GRPO is the standard single-turn GRPO on step-level samples. The paper does not seem to solve the long horizon credit assignment problem. The fact the only critic model is updated with RL and the generator is frozen is somewhat weak. One of the main issues with long interleaved image-text generation is image generation starts error accumulation after a while. Without updating generator, this issue can't be addressed. Also, consider the case where generator model simply can't follow a particular edit instruction. No matter what RL you do on prompts, generator simply can't generate.

Including a prompt rewrite-only baseline is very powerful to have. The WISE and RISE gains may come from prompt rewriting since the rewriten prompts can check for factual accuracy.

Questions for authors

Can we report statistical significance of the runs? How much time does the entire approach take?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.