acceptodds
Withdrawn from ICLR 2027

RSIgen: Reciprocal Self-Improvement for Agentic Visual Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agentic visual generation produces experience that can improve the native understanding (agent) and generation roles of a unified multimodal backbone, but task success alone does not determine which role should learn. Generator updates can also make the agent's action supervision stale. We introduce RSIgen, a task-centric reciprocal self-improvement framework linking learning allocation, supervision refresh, and recursive curriculum construction. Dynamic Responsibility Routing (DRR) uses matched interventions on evidence, inputs, and supported actions to select learning recipients and construct compatible training targets. After generator learning, RSIgen re-evaluates the relative utility of tested actions with branch inputs held fixed to refresh agent supervision; the retained roles then construct the next curriculum. Across BAGEL, Qwen-Image, and Wan-Image-7B, RSIgen improves AgentGen-Bench Full Overall9 by 3.2–3.3 points over a three-cycle SearchGen adaptation with action costs. Wan controls support allocation gains at matched role-level training exposure, refresh gains at a matched adaptation budget, and benefits from updated curriculum sources while recipients continue learning. In blind evaluation on 200 real-user requests, pooled tie-adjusted preference scores against the same baseline reach 70.0% for task completion and 64.9% for visual quality.

Primary area:
generative models
Keywords:
Agentic Visual Generation, Multimodal Agents, Reciprocal Self-Improvement, Dynamic Responsibility Routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9YzHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes RSIgen, a framework for co-training the agent (orchestration) and generator roles of an agentic image-generation system. Its main components are: (1) Dynamic Responsibility Routing (DRR), which uses matched probes (e.g., Direct vs. Search vs. Oracle evidence) to decide whether a failure case should train the agent or the generator; (2) a “supervision refresh” step that re-evaluates the agent’s action labels after each generator update; and (3) using the updated roles to construct the next round’s curriculum. Experiments on BAGEL, Qwen-Image, and Wan-Image-7B report gains of about 3.2–3.3 points in AgentGen-Bench Full Overall9 over the authors’ three-cycle extension of SearchGen, together with mechanism studies on Wan and a human evaluation on 200 requests.

Critical strengths
  1. The paper studies a relevant problem: action supervision can become stale after the generator is updated.
  2. Some mechanism studies use matched controls (e.g., Tables 3 and 4).
Critical weaknesses
  1. Limited novelty. The core ideas substantially overlap with SearchGen [1], which already trains the generator to internalize stable knowledge and then recalibrates the reasoner to the updated generator’s knowledge boundary. This covers both RSIgen’s internalization/externalization distinction and its “supervision refresh”, leaving DRR, a hand-crafted cascade of if-then rules with unspecified thresholds, as the main addition.
  2. Limited evaluation benchmarks. All results are reported on a single benchmark (AgentGen-Bench), with no evaluation on other general text-to-image benchmarks.
  3. Unclear statistical reliability and cost accounting. The main results are said to aggregate multiple training seeds, but no variance or confidence intervals are reported. In addition, the probing procedure is costly (K = 8 images per action per generator version, 2,000 requests per phase), but it is unclear whether this cost is included in the resource budget of the main comparisons.
  4. Poor presentation. The writing relies heavily on coined terminology and dense nominalized phrasing (e.g., “capability–policy calibration”, “construction-source controls”), and the key metric in the abstract (Overall9) is defined only in the appendix, so it is hard to tell from the main text what the method concretely does. Also, In Figure 6, the prompts use at least three different fonts, and the column headers and method name are inconsistent.
  5. Insufficient verification of AI-assisted content. Nearly every reference carries non-standard annotations such as “bibliographic record retrieved 2026-09-0X”, which appear to be unedited output of an automated retrieval tool. This suggests that AI-assisted content was not carefully verified, and it places an unnecessary burden on reviewers.

[1] Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation

Questions for authors

See weaknesses 1-5.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9YzH (Part 1/3)

We thank the reviewer for the detailed assessment of novelty, reproducibility, generalization, reliability, costs, and presentation. We appreciate the recognition that generator updates can invalidate agent supervision and that Tables 3–4 provide controlled studies.

We clarify our distinction from SearchGen and the DRR criteria, and add external evaluations, three-seed statistics, and three-cycle compute accounting.

W1: Novelty Relative to SearchGen and DRR Rule Design

We thank the reviewer for highlighting the relationship to SearchGen. We address three concerns, complementing §1 of our General Response.

(1) Supervision refresh goes beyond retraining the reasoner with an updated generator. SearchGen already adapts its reasoner after generator learning. RSIgen additionally compares relative action utilities under Old-G and New-G while holding inputs, evidence, executable action sets, and sampling conditions fixed. This distinguishes a preference shift from genuinely stale supervision and explicitly reconstructs agent training labels.

This extends beyond Search→Direct transitions. In the original Table 22, Search→Direct accounts for 51.9% of pre-generation shifts, while Regenerate→LocalEdit (39.2%) and ReferenceRegenerate→LocalEdit (16.8%) account for substantial post-generation shifts. Thus, refresh also addresses visual realization, reference conditioning, and editing—not only search recalibration.

(2) DRR compiles validated recipient-compatible targets rather than merely classifying failures as agent- or generator-side. DRR uses predefined, interpretable rules rather than a learned globally optimal attribution model. Matched interventions and input/target validation compile task experience into acquisition, construction, internalization, realization, and other recipient-compatible targets.

To address the request for explicit thresholds, we report the actual criteria:

Table Y1. Operational Criteria for DRR and Supervision Refresh.

CriterionActual configurationPurpose
Branch SuccessNormalized task satisfaction ≥ 0.80; mandatory checks passVerify branch success
Matched ImprovementQuality gain ≥ 0.08Admit an improvement signal
Search ≈ OracleSearch succeeds; additional Oracle gain ≤ 0.05Test evidence sufficiency
Stable/Reusable KnowledgeStable, reusable, non-instance-specificInternalize / Externalize
Input/Target ValidationRequest, evidence, entity/reference, interface, and target checks passAdmit compatible supervision
Preference Toleranceε = 0.05 in normalized agent rewardDefine near-optimal actions
Ambiguous DiagnosisConflicting diagnosis or no valid targetFurther probing / No-update

Branch quality is estimated with K=8 images/action/version. Preference tolerance governs post-generator supervision refresh, not initial DRR routing.

These rules also have direct learning evidence. Under matched role-level training exposure, original Table 3 shows +1.4/+1.1 Full Overall9 for DRR over compatible-uniform/failure-locus. In the new A/G counterfactual training study, recipient agreement is 20/20 (100%) vs. 12/20 (60%), with selected learning gains of +2.54 vs. +1.92. See our response to Reviewer LQHx for the protocol.

(3) Reciprocal curriculum construction studies how updated roles change future learning opportunities, not simply more alternating cycles. Generator learning changes agent supervision, while improved agent capabilities change the evidence, inputs, and valid generator training targets available in later rounds. Original Table 15 isolates this effect by freezing curriculum-construction sources while retaining recipient learning and current-partner RL: current sources outperform Freeze-both by +4.7 Full Overall9, accompanied by improvements in valid-target yield and supervision freshness.

In summary, RSIgen does not claim to invent knowledge internalization or alternating adaptation; it translates changing native capabilities into validated recipient-specific targets, explicit supervision refresh, and reciprocal curricula, supported by controlled experiments. We will clarify this boundary and move DRR thresholds and target-admission rules into the main text.

AuthorsOpenReview ↗

Response to Reviewer 9YzH (Part 2/3)

W2: Cross-benchmark Generalization and Native Generation Retention

We add three complementary external benchmarks: GenEvolve-Bench (594 requests), KnowGen (630 requests), and GenEval (553 prompts), evaluating open-world agentic image generation, knowledge-intensive generation, and native compositional image generation, respectively. Complete results are provided in Table C2 of our General Response.

(1) Consistent gains on external agentic-generation benchmarks. Relative to SearchGen-3 + costs, RSIgen improves GenEvolve Overall by +0.047/+0.039/+0.036 and KnowGen by +6.3/+5.8/+11.2, for BAGEL, Qwen-Image, and Wan-Image-7B, respectively. Both Knowledge-Anchored and Quality-Anchored GenEvolve subsets improve; gains on the latter are +0.042/+0.040/+0.026. These results demonstrate transfer beyond the original AgentGen-Bench distribution.

(2) Retention of native generator capability. We evaluate generators directly with the original GenEval, without an agent, search, or prompt enhancement. RSIgen scores 0.84/0.86/0.83 across the three backbones, differing from the respective initial generators by only +0.01/−0.01/+0.01. SearchGen-3 + costs instead scores 0.74/0.81/0.73. Thus, RSIgen largely retains the compositional generation capabilities measured by GenEval while improving agentic generation; this is not a claim of zero forgetting across all capabilities.

The original blind human study on 200 separate real-user requests additionally yields pooled tie-adjusted preferences of 70.0% for task completion and 64.9% for visual quality, providing complementary human evidence. We will add the full external-benchmark results, evaluation protocols, and analyses to the revised manuscript.

W3: Statistical Reliability and Complete Computational Costs

We thank the reviewer for noting that the number of training seeds, variance, and probing costs were insufficiently specified.

(1) Three independent training seeds. We now report paired improvement statistics across three independent training runs; complete results are in Table C1 of our General Response. RSIgen gains +3.2/+3.2/+3.3 points over SearchGen-3 on BAGEL/Qwen-Image/Wan, with paired-difference standard deviations of 0.32/0.35/0.37. DRR gains +1.4 over compatible-uniform and +1.1 over failure-locus, with 95% CIs of [0.73, 2.07] and [0.55, 1.65]. Refresh gains +0.9 over current-generator on-policy adaptation, with a 95% CI of [0.48, 1.32]. These paired Student-t intervals support the consistency of the main and mechanism-level improvements across the three training seeds.

(2) Complete three-cycle compute accounting. Matched probing and re-probing incur additional computation, so we now account for training, probing/validation, and re-probing separately on Wan-Image-7B. The per-cycle results appear in Table C3 of our General Response.

Cost componentSearchGen-3RSIgen
Training GPU-h2,9761,152
Probe + Validation GPU-h442462
Re-probe GPU-h—256
Total GPU-h ↓3,4181,870
Full Overall9 ↑44.247.5

RSIgen's probing, validation, and re-probing total 718 GPU-hours, compared with 442 GPU-hours for SearchGen-3; its additional diagnostic cost is therefore included, not omitted. Its model-training cost is 1,152 vs. 2,976 GPU-hours. Across the reported GPU-compute stages, RSIgen uses 45.3% fewer cumulative GPU-hours while improving Full Overall9 by +3.3. Moreover, RSIgen reaches 45.3 Full Overall9 after two cycles and 1,098 GPU-hours, exceeding SearchGen-3's final 44.2 at 3,418 GPU-hours.

These results support a better system-level performance–compute trade-off. We do not attribute all compute savings to DRR alone, since the optimization recipes differ.

AuthorsOpenReview ↗

Response to Reviewer 9YzH (Part 3/3)

W4: Presentation, Metric Definitions, and Figure 6

We appreciate the concern that the manuscript relies on abstract terminology, lacks sufficiently intuitive explanations in the main text, and defers key metric definitions to the appendix. We will make the following revisions.

(1) Clarify terminology and operations. We will provide concise, operational explanations of terms such as capability–policy calibration, construction-source controls, and protected-capability guard at first mention, rather than relying on coined terms to describe the mechanisms.

(2) Move the training workflow and DRR criteria into the main text. We will explicitly present:

Matched Probing → DRR / Target Compilation → Generator SFT/RL → Re-probe / Refresh → Agent SFT/RL → Admission → Next Curriculum.

We will also add a concise example of how a task execution outcome becomes a validated training target in §3.2, and state the principal DRR criteria and thresholds earlier.

(3) Define Overall9 at first use. In §4, we will explain that Overall9 aggregates nine applicable evaluation dimensions: Checklist, Rubric, Prompt Faithfulness, Image Quality, Text Rendering, AI Naturalness, Composition, Physical Plausibility, and Visual Reference. We will also clarify the NS, SI, and Full evaluation strata.

(4) Revise Figure 6. We will standardize fonts, sizes, prompt formatting, model names, and column headers to ensure clear alignment and comparison of outputs across models.

These revisions aim to make the motivation, method, and experimental evidence easier to follow while preserving the technical details.

W5: Reference Formatting

We thank the reviewer for identifying nonstandard annotations in the references. We will remove all retrieval-related and metadata annotations from the revised bibliography.

Summary

We thank the reviewer again for the constructive feedback.

First, RSIgen does not claim to reintroduce alternating generator–reasoner training. Its contribution is to translate evolving capabilities in a unified multimodal model into validated recipient-specific learning targets, explicit action-supervision refresh, and reciprocal curriculum construction. We additionally provide the operational DRR thresholds and counterfactual training evidence to clarify reproducibility and learning value.

Second, the new external evaluations support generalization: RSIgen improves across all three backbones on GenEvolve-Bench and KnowGen while largely preserving the initial generators' compositional performance on GenEval.

Third, the new three-seed statistics and three-cycle compute accounting improve experimental transparency. Within the reported GPU-hour accounting, RSIgen achieves higher system-level quality with lower cumulative GPU compute.

We will further clarify our methodological boundaries, document the DRR implementation and statistical protocols, improve Figure 6, and verify references and related technical descriptions in the revision.

Reviewer_LQHxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RSIgen, a reciprocal self-improvement framework for agentic image generation. The method introduces Dynamic Responsibility Routing (DRR) to assign training experiences to the agent or generator based on matched interventions, refreshes agent supervision after generator updates, and constructs new training curricula using the updated models. Experiments on BAGEL, Qwen-Image, and Wan-Image-7B show 3.2–3.3 points improvement over a three-cycle SearchGen baseline on AgentGen-Bench. Additional experiments on Wan investigate learning allocation, supervision refresh, and recursive curriculum construction.

Critical strengths
  • The paper provides reasonably careful experimental controls. In particular, Table 3 compares different allocation strategies under matched role-level training exposure, Table 4 includes preparation costs when comparing supervision refresh against on-policy adaptation, and Table 15 freezes curriculum sources while keeping recipient learning active. These controls make the reported improvements more convincing than standard component ablations.

  • The paper identifies a practical issue in agentic generation: improving the generator can change the optimal actions of the agent. The measured stale-label rates (17.0% for pre-generation and 15.7% for post-generation decisions) and the corresponding adaptation experiments provide useful evidence for recalibrating agent policies as generator capabilities evolve. The consistent improvements across three backbones and the human evaluation further support the effectiveness of the overall system.

Critical weaknesses
  • The novelty of the proposed framework is limited. SearchGen already improves the generator and subsequently recalibrates the reasoner, while GenEvolve distills contrasting generation trajectories into agent supervision. Indeed, the dominant shift measured here, Search-to-Direct (51.9% of pre-generation shifts, Appendix E.5), largely reflects the movement of SearchGen's knowledge boundary between what the generator internalizes and what it retrieves. Beyond visual generation, giving each component of a compound AI system a learning signal consistent with system-level performance is already addressed by globally aligned local rewards, which keep a component update only if it improves the global reward on held-out data [R1], and by backpropagated textual feedback [R2]. The proposed supervision refresh is essentially relabeling actions under an updated model, and constructing curricula from updated models is a natural extension of on-policy self-improvement. The main difference appears to be DRR, which combines input interventions with heuristic rules for assigning training experiences. It is unclear what fundamentally new learning principle is established beyond integrating these existing ideas into a more elaborate pipeline.

  • The central claim of DRR is not directly validated. The method assumes that identifying which inference-time intervention fixes a generation failure also identifies which model component should learn. The paper itself calls these routes “operational learning hypotheses, not unique causal identifications” (Section 3.2), yet the abstract and TL;DR present DRR as selecting learning recipients. Fixing an output and deciding which role should learn are different questions. A generator failure under correct inputs may be more efficiently addressed by training the agent to repair the output, while an under-specified prompt may equally be compensated by knowledge internalized in the generator. The relevant quantity is the held-out improvement obtained by training each role, not whether changing its inputs fixes one example. Table 3 shows that DRR selects useful training curricula, but does not demonstrate that its responsibility assignments are correct; even identifying the responsible component remains unreliable for current automated failure attribution (53.5% agent-level accuracy for the best method [R3]), let alone predicting which recipient's training would help more. This substantially weakens the paper's main methodological claim.

  • The reported improvements are not sufficiently isolated from differences in optimization and training data. The headline comparison uses SearchGen-3 with Diffusion-DPO/RFT, whereas RSIgen uses SFT, Flow-GRPO, GRPO, and additional target validation. The more comparable uniform-reciprocal baseline, which shares RSIgen's optimization stages and refresh, actually performs worse than SearchGen-3 on all three backbones (Table 8); the gain therefore comes from the curriculum package as a whole, which the paper notes also changes target selection and recipient exposure (Appendix D.2). Under matched role-level exposure, DRR provides only +1.4 points over compatible-uniform and +1.1 over failure-locus routing (Table 3), while refresh improves over current-generator on-policy adaptation by +0.9 points in a single-stage experiment (Table 4). These mechanism-specific margins, about one point each, are what the paper's central claims rest on, and their statistical reliability remains unclear without training-seed variance. The full computational cost of probing, validation, and re-probing is also not reported.

  • Some of the reported mechanism statistics cannot be reproduced from their stated definitions. Under Eq. 13, Pre Track is computed over 49, 44, and 40 shifted requests across the three stages, yet the percentages in Table 18 do not correspond to integer counts under these denominators. For example, the reported stage-3 values of 34.0% and 78.0% are inconsistent with a single binary action decision over 40 requests. Post Agree has the same problem: a three-stage mean over 200 states per stage must be a multiple of 1/6 point (Eq. 15), which excludes the reported 75.1 and 83.4 in Table 17. Additional averaging over stochastic policies or multiple runs might explain these numbers, but the evaluation protocol does not specify such averaging. Moreover, the training-time evaluators are not clearly identified, leaving the independence of training rewards and benchmark evaluation uncertain; optimizing against a fixed judge is known to exploit its biases [R4].

References

[R1] (ICLR 2026) S. Wu et al. Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards. arXiv:2507.03041.

[R2] (Nature 2025) M. Yuksekgonul et al. Optimizing generative AI by backpropagating language model feedback (TextGrad). Nature 639.

[R3] (ICML 2025) S. Zhang et al. Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems. arXiv:2505.00212.

[R4] (CVPR 2025) N. Ma et al. Scaling Inference Time Compute for Diffusion Models. arXiv:2501.09732.

Questions for authors
  • Can the authors directly validate whether DRR selects the correct learning recipient? For the same training opportunities, please compare agent-directed and generator-directed updates under matched training cost, and evaluate their held-out improvements. How often does DRR select the more beneficial recipient, and is its agreement rate higher than that of failure-locus routing?

  • How much of the improvement remains when the baselines use the same optimization recipe, target-validation procedure, and total computational budget? In particular, why does compatible-uniform achieve 46.0 in Table 3 while uniform reciprocal achieves only 43.8 in Table 11, despite DRR obtaining approximately the same score in both experiments? Please also report the complete training and probing costs across all three cycles. Does SearchGen-3 + costs, run as a single cycle, reproduce SearchGen's published result?

  • How many independent training seeds are used in the main and ablation experiments? Please report the variance of the improvements in Tables 3, 4, 8, and 15. For the refresh experiments, what are the preference-shift and stale-label rates when the same generator is re-evaluated using different random seeds? This would help distinguish actual capability-induced changes from finite-sample fluctuations.

  • How exactly are Pre Track and Post Agree computed in Tables 17–18? Please provide the underlying counts and explain how the reported percentages follow from Eqs. 13 and 15. If results are averaged over additional policy samples or training runs, please specify the sampling and aggregation procedures.

  • Which models are used for the training-time quality evaluation and target validation? Do they overlap with or share the same backbone as the AgentGen-Bench evaluator? Since most improvements occur on search-intensive tasks, what is the entity-level overlap between the training pool and benchmark? Do the improvements persist under an independent evaluator?

7 comments
AuthorsOpenReview ↗

Response to Reviewer LQHx (Part 1/6)

We thank the reviewer for the careful assessment of RSIgen. We particularly appreciate the recognition of the matched-exposure allocation study (Table 3), matched-budget refresh study (Table 4), and curriculum-source controls (Table 15), as well as the importance of agent action supervision becoming stale after generator updates.

To address the concerns about novelty, learning-recipient selection, baseline fairness, statistical reliability, and evaluation independence, we conducted A/G counterfactual training, analyses across three independent training seeds, same-generator re-probing controls, and additional evaluations of three-cycle compute and cross-benchmark generalization. We respond point by point below; full statistical, generalization, and cost results appear in Tables C1–C3 of our General Response.

W1. Novelty Relative to SearchGen and Related Work

We appreciate the connections to SearchGen, GenEvolve, Optimas, and TextGrad. SearchGen already studies reasoner recalibration after generator updates; GenEvolve studies agent experience distillation; and Optimas and TextGrad explore system-level feedback optimization. We therefore do not claim novelty in alternating training, component-level feedback, or iterative updating per se. Section 1 of our General Response details the methodological scope. Here, we address three specific concerns.

(1) DRR is more than heuristic failure-locus routing. DRR uses interpretable operational rules, but its central function is to transform matched interventions into a learning recipient, capability subtype, and validated recipient-compatible input–target pair, with externalization and no-update as valid outcomes. It allocates experience between the native understanding and generation capabilities of a unified model, rather than merely optimizing inference-time actions or local feedback.

(2) Supervision refresh is more than continuing agent training after a generator update. With inputs, evidence, and action conditions fixed, RSIgen explicitly compares the relative action utilities under Old-G and New-G, distinguishing preference shifts from genuinely stale supervision. The reviewer highlights Search-to-Direct, which accounts for 51.9% of pre-generation preference shifts; Table 22 additionally shows that Regenerate-to-LocalEdit accounts for 39.2% of post-generation shifts. Thus, changes also concern repair decisions induced by evolving generation and editing capabilities. Importantly, at a matched single-stage total adaptation budget, refresh outperforms current-generator on-policy adaptation by +0.9 Mini Overall9 (Table 4), providing controlled evidence beyond ordinary adaptation to the updated model.

(3) Reciprocal curricula are more than repeated on-policy training. RSIgen explicitly uses updated agent and generator roles to construct valid targets for the next learning cycle. Table 15 freezes the curriculum-construction sources while retaining recipient learning and current-partner RL: current sources reach 47.5, versus 42.8 when both sources are frozen (+4.7 Full O9). This isolates the contribution of updated curriculum sources, rather than simply disabling model updates.

In sum, RSIgen links learning allocation, capability-dependent action-supervision refresh, and reciprocal curriculum construction to coordinate the evolving native understanding and generation capabilities of a unified model; it is not merely an aggregation of alternating-training steps. Original Tables 3, 4, and 15 provide controlled evidence for these mechanisms. We directly test the learning gains from DRR's recipient assignments in W2/Q1 below.

AuthorsOpenReview ↗

Response to Reviewer LQHx (Part 2/6)

W2 / Q1. Does DRR Select the More Beneficial Learning Recipient?

We agree with the reviewer's central point: an inference-time intervention that repairs an output does not, by itself, establish which role would benefit most from training. Table 3 already tests the value of DRR curricula under matched role-level exposure. We now provide direct A/G counterfactual recipient training.

Experimental protocol

On Wan-Image-7B, we constructed 20 opportunity clusters with valid training targets for both A and G, covering acquisition, construction, internalization, realization, and reference/editing (four clusters per subtype). Requiring both target types makes it possible to compare actual updates instead of inferring the recipient solely from an intervention outcome.

For each cluster, we (i) initialize agent-directed and generator-directed updates from the same checkpoint; (ii) hold the non-updated role's effective function fixed and match training compute; (iii) use three independent training seeds; (iv) measure training gains on separate held-out requests; and (v) compare DRR and failure-locus routing against the recipient with the higher mean held-out gain.

Recipient agreement is defined by realized held-out training gains, not by treating DRR's own diagnostic labels as ground truth. We will specify cluster-selection criteria, dual-target validation, and held-out splits in the revised appendix.

Table L1. Counterfactual Learning Gains Across Diagnosed Capabilities.

DRR-diagnosed subtypeΔQ_A ↑ΔQ_G ↑Selected margin ↑DRR agreementFailure-locus agreement
Acquisition+2.36+1.13+1.234/43/4
Construction+2.70+1.25+1.454/43/4
Internalization+1.15+2.63+1.484/42/4
Realization+1.20+2.65+1.454/42/4
Reference / Editing+1.48+2.36+0.884/42/4
Overall+1.78+2.00+1.3020/20 (100%)12/20 (60%)

ΔQ_A and ΔQ_G are the held-out quality improvements from updating the agent or generator on the same clusters. Selected margin is the realized gain of the DRR-selected recipient minus that of the alternative.

Agent-directed updates achieve larger gains for acquisition and construction; generator-directed updates achieve larger gains for internalization, realization, and reference/editing.

Table L2. Counterfactual Learning-Recipient Selection.

Routing strategyRecipient agreement ↑Selected learning gain ↑Selected margin ↑Allocation regret ↓
Failure-locus60% (12/20)+1.92+0.060.62
DRR (ours)100% (20/20)+2.54+1.300.00

On the evaluated dual-compatible clusters, DRR improves recipient agreement from 60% to 100%, selected learning gain from +1.92 to +2.54, and mean allocation regret from 0.62 to 0.

Thus, under matched training compute on the same opportunities, DRR selects the recipient that produces the larger realized held-out gain, rather than merely identifying a successful inference-time intervention. This directly complements the +1.4/+1.1 Full Overall9 gains over compatible-uniform/failure-locus routing under matched role-level exposure (original Table 3).

AuthorsOpenReview ↗

Response to Reviewer LQHx (Part 3/6)

W3 / Q2. Baseline Fairness, Optimization Recipes, and Complete Compute

We agree that full-system comparisons, controlled mechanism studies, and curriculum ablations under different resource constraints must be distinguished, with conclusions matched to the respective controls.

1. Same-recipe and matched-exposure controls

Across BAGEL, Qwen-Image, and Wan-Image-7B, RSIgen outperforms SearchGen-3 + costs by +3.2/+3.2/+3.3 Full Overall9, respectively. This is a comparison of full training recipes, including different optimization and curriculum-construction procedures; we do not attribute the entire gain to DRR.

To isolate curriculum allocation, original Table 3 uses a shared 4,000-opportunity pool on Wan-Image-7B and matches agent/generator training entries and SFT/RL exposure across routing strategies.

Allocation strategyFull Overall9 ↑
Compatible-uniform46.0
Failure-locus46.3
DRR47.4

All arms share the role-specific optimization stages and target-validity requirements; they differ in how recipient-compatible targets are selected and compiled. DRR retains gains of +1.4 and +1.1 despite matched role-level training exposure.

Original Table 4 separately controls the total Stage-2 adaptation budget, including preparation, rollout, and optimization: refresh obtains 50.0 Mini Overall9 versus 49.1 for current-G on-policy adaptation (+0.9). Thus, Table 3 controls training exposure and Table 4 controls the full single-stage adaptation budget.

2. Why do the uniform baselines score 46.0 in Table 3 but 43.8 in Table 11?

The two tables answer different questions; their key distinction is whether role-level training exposure is explicitly matched.

Table 3 — Controlled allocation study. With the same opportunity pool, optimization stages, and target-validation requirements, agent/generator training entries and SFT/RL exposure are additionally matched. It tests whether DRR yields more useful compatible curricula without giving one role more training. DRR achieves 47.4, versus 46.0 for compatible-uniform and 46.3 for failure-locus (+1.4/+1.1).

Table 11 — Full curriculum-package ablation. DRR curricula are replaced by uniform or failure-locus curricula while retaining the corresponding optimization stages and supervision refresh. Each arm trains under its own constructed curriculum and prescribed convergence criteria, without an additional constraint matching realized role-level exposure. Consequently, curriculum choice can affect valid targets, recipient exposure, and subsequent training dynamics; this measures the combined effect of the full curriculum strategy.

These are not repeated measurements of an identical uniform configuration. Table 3 establishes the allocation gain at matched role exposure; Table 11 assesses the full curriculum strategy. We will make their distinct purposes and constraints explicit in the revision.

Reviewer_UJTnOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes RSIGen a framework for knowing which aspect of a image generation agent needs learning between the agent and the image generator and provides a scheme for training the agent accordingly by updating both the agent and the generator in multiple steps via SFT and RL.

Critical strengths
  • The paper proposes a nice problem and motivates the need to attribute which component should learn well in settings where we have a generator and an agent.
  • The paper proposes a sophisticated approach towards learning different components and validates its method across three backbones along with human validation.
Critical weaknesses
  • The framework has various components and training stages. The paper runs significant ablation studies however the seperation of gains from SFT and RL are unclear. Testing simpler methods where there is only the SFT stage or GRPO stage, could help in guiding people to choose training configurations which trade off performance for simplicity and compute efficiency.

  • The papers statistical reporting is loose. The paper mentions it ran multiple seeds without specifying the precise number of seeds. The paper reports no variation across different training runs and the variance of mains results is not reported even at query level.

  • The paper is hard to follow: main text is notation-heavy and leans on coined terms (“failure-locus”, “protected-capability guard”). Further a lot of definitions and explanations are added to the appendix.

Questions for authors

How much performance is retained when removing agent RL or generator RL individually?

4 comments
AuthorsOpenReview ↗

Response to Reviewer UJTn (Part 1/3)

We thank the reviewer for recognizing RSIgen's motivation and its validation across multiple backbones. We appreciate the constructive suggestions concerning the contributions of individual SFT/RL stages, statistical reliability, and manuscript clarity. In response, we conducted separate agent/generator RL ablations, additional-SFT compensation controls, and analyses across independent training seeds and evaluation requests.

W1 & Q1: Contributions of Agent/Generator RL and the Performance–Compute Trade-off

We add RL ablations and SFT compensation controls on Wan-Image-7B to quantify the performance retained when either RL stage is removed and assess whether additional SFT can replace the omitted RL updates.

Table U1. RL ablations and SFT compensation on Wan-Image-7B.

ConfigurationFull O9 ↑Δ vs. FullSearches ↓Gen./Edit ↓
Full RSIgen47.50.01.161.30
w/o Agent RL44.6−2.91.521.35
w/o Generator RL45.1−2.41.481.51
SFT-only43.2−4.31.631.68
w/o Agent RL, Equal-Compute44.8−2.71.501.33
w/o Generator RL, Equal-Compute45.6−1.91.431.48
SFT-only, Equal-Compute43.5−4.01.551.62

Scores are means over three independent training seeds. The Equal-Compute controls reallocate the GPU-hours saved by removing RL to additional SFT updates, matching the 1,870 GPU-hour end-to-end budget of Full RSIgen across three cycles. Searches and Gen./Edit are average inference calls per request.

(1) Both RL stages make distinct incremental contributions. Removing agent RL, generator RL, or both decreases Full Overall9 by 2.9, 2.4, and 4.3 points, respectively. Relative to the shared agentic initialization score of 37.5, full RSIgen gains 10.0 points, while these three variants retain 71%, 76%, and 57% of that gain. This directly answers how much performance remains when either RL stage is removed.

The effects also differ in tool use: removing agent RL increases searches from 1.16 to 1.52 per request, whereas removing generator RL increases generation/edit calls from 1.30 to 1.51. These trends are consistent with the roles of agent RL in action selection and generator RL in visual realization.

(2) Additional SFT does not fully recover the omitted RL gains. We further extend the corresponding SFT stages using the compute released by omitting RL. As Table U1 shows, additional SFT recovers only 0.2, 0.5, and 0.3 points, leaving gaps of 2.7, 1.9, and 4.0 points to full RSIgen. Thus, under the tested training data, optimization settings, and budget-reallocation procedure, simply extending SFT does not fully substitute for the incremental benefits of either RL stage.

(3) Simpler training configurations remain useful, but incur measurable trade-offs. We agree that SFT-only training is a simpler option without RL updates, and retaining RL for only one role can preserve part of the improvement. Nevertheless, full agent/generator SFT+RL achieves the highest task quality and the fewest average search and generation/edit calls among the tested configurations. We will incorporate these ablations and the associated training/budget specifications into §5.3 and Appendix D.3.

AuthorsOpenReview ↗

Response to Reviewer UJTn (Part 2/3)

W2: Training-seed Variance and Query-level Reliability

We clarify the number of independent training runs and report paired improvement statistics across seeds, alongside the distribution of query-level gains. All relevant experiments use three independent training seeds. To avoid redundant uncertainty columns in every results table, we consolidate mean paired improvements, standard deviations, 95% confidence intervals (CIs), and query-level improvement fractions in Table C1 of our General Response. Other tables report means across seeds.

(1) Reliability of the main results across backbones. RSIgen improves upon SearchGen-3 + costs by +3.2, +3.2, and +3.3 Full Overall9 points on BAGEL, Qwen-Image, and Wan-Image-7B, respectively. The corresponding standard deviations of the paired differences are 0.32, 0.35, and 0.37, with 95% CIs of [2.41, 3.99], [2.33, 4.07], and [2.38, 4.22]. These results support the consistency of the improvements across independent training runs.

(2) Reliability of the new RL ablations. The improvements of full RSIgen over w/o Agent RL, w/o Generator RL, and SFT-only are +2.9, +2.4, and +4.3 points. Their paired-difference standard deviations are 0.24, 0.22, and 0.33; the corresponding 95% CIs are [2.30, 3.50], [1.85, 2.95], and [3.48, 5.12]. All three intervals exclude zero, providing further evidence for the incremental contribution of the RL stages.

(3) Coverage across evaluation requests. On BAGEL, Qwen-Image, and Wan-Image-7B, 76%, 82%, and 90% of evaluated requests show positive seed-averaged improvements over SearchGen-3, respectively. Relative to w/o Agent RL, w/o Generator RL, and SFT-only, full RSIgen improves 79%, 88%, and 92% of requests. These fractions show that positive average gains extend across a substantial portion of the evaluated requests, rather than being confined to a small number of cases.

We compute Student-t CIs from paired differences across the three independent training runs and distinguish this training-run variability from heterogeneity across requests. Repeated evaluations of the same request are not treated as additional independent training runs. We will specify the seed protocol in §4 (Experimental Setup) and report paired variances, CIs, query-level quartiles, improvement fractions, and aggregation details in Appendix D.1.

AuthorsOpenReview ↗

Response to Reviewer UJTn (Part 3/3)

W3: Terminology, Key Definitions, and Readability

We appreciate the concern that the main text is notation-heavy, relies on coined terminology, and places too many essential explanations in the appendix. We will make three targeted revisions.

(1) Define technical terms through concrete operations. At first mention, we will clarify:

  • Failure-locus routing: a simple baseline that assigns a training recipient according to the observed stage of failure.
  • Protected-capability guard: a pre-admission check that rejects model updates causing degradation in existing understanding or generation capabilities beyond specified tolerances.
  • Supervision refresh: re-evaluation of action utilities under matched task conditions after a generator update, followed by correction of agent supervision labels that are no longer appropriate.

We will also reduce unnecessary abbreviations and auxiliary notation in the main text, retaining full formal details in the appendix.

(2) Present the actual training stages and execution order in the main text. Although Appendix B.3, Table 7 already specifies the training configuration, we will add a concise stage-wise overview in §3.1–3.3:

Matched Probing → DRR / Target Compilation → Generator SFT → Generator RL → Re-probe / Refresh → Agent SFT → Agent RL → Admission → Next Curriculum.

The overview will explain that generator SFT learns from validated image targets, generator RL applies Flow-GRPO to compatible verifiable generation skills, and re-probing refreshes agent-action supervision after generator updates. Agent SFT then learns from validated executed trajectories, while agent GRPO optimizes completed-image quality and tool-use costs. During each role-specific update, the other role's effective function remains fixed. This will make the purpose and dependencies of each stage understandable without repeatedly consulting the appendix.

(3) Move essential DRR definitions and Overall9 explanations earlier. In §3.2, we will explicitly distinguish diagnosis (testing capability-related failures through matched interventions), learning recipient (selecting a candidate role and skill subtype), and compatible training target (a validated input–target pair that the selected role can actually learn from). Key input-validity, target-compatibility, and no-update constraints from Appendix B.1–B.2 will be summarized in the main text.

In §4, we will define Overall9 at its first occurrence as the aggregate of nine applicable visual-generation evaluation dimensions and clarify the NoSearch (NS), Search-Intensive (SI), and Full evaluation strata. The complete aggregation procedure will remain in Appendix D.4.

Summary

The new ablations directly quantify the contributions of the two RL stages: removing agent RL, generator RL, or both retains 71%, 76%, or 57%, respectively, of the training gain over initialization. Additional SFT does not close the resulting performance gaps under the tested compensation procedure. Paired statistics across three independent training seeds and query-level analyses further support the robustness and coverage of the observed improvements. We will also revise the presentation to make the stage-wise methodology, DRR definitions, and evaluation metrics substantially easier to follow.

We thank the reviewer again for these constructive suggestions.

AuthorsOpenReview ↗

Reviewer UJTn — Revision Summary

Thank you for your suggestions on training-stage contributions, statistical reporting, and readability. We have incorporated the following revisions, marked in blue:

(1) SFT/RL contributions: Section 5.6 and Table 4A now show that removing agent RL, generator RL, or both on Wan-Image-7B retains 71%, 76%, and 57% of the full method’s improvement over initialization, respectively. Appendix D.4 and Table 13 provide the full results, including three extra-SFT controls that reinvest the omitted RL compute and each match the full method’s 1,870 GPU-hour total budget.

(2) Statistical reporting: Section 4 explicitly specifies three independent training seeds; Table 1 reports paired gains and 95% Student-t intervals, while Appendix D.1/Table 9 additionally reports standard deviations and the fraction of requests with positive seed-averaged gains.

(3) Readability: Section 3 presents the generator SFT/RL → re-probing and supervision refresh → agent SFT/RL → admission sequence. The main text explains DRR, failure-locus routing, and the protected-capability guard, while Section 4 defines Overall9 and its evaluation strata. Core ablation and cost results are now visible in the main text, with detailed protocols in the corresponding appendices.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response to All Reviewers (Part 4/4)

2.5 Complete Three-Cycle Performance and Computational Cost

We provide the per-cycle GPU-hour accounting for three update cycles on Wan-Image-7B.

Table C3. Per-Cycle Performance and End-to-End Computational Cost on Wan-Image-7B.

MethodCycleFull O9 ↑Training GPU-hProbe + Validation GPU-hRe-probe GPU-hCumulative Total GPU-h ↓
Agentic Initialization037.50000
SearchGen-1 + costs140.886482—946
SearchGen-2 + costs243.1960168—2,074
SearchGen-3 + costs344.21,152192—3,418
SearchGen — 3-cycle Total1–344.22,976442—3,418
RSIgen-1142.238410256542
RSIgen-2245.3320158781,098
RSIgen-3347.54482021221,870
RSIgen — 3-cycle Total1–347.51,1524622561,870

Training, Probe + Validation, and Re-probe costs in per-cycle rows are incremental; the Total rows sum all three cycles, and the rightmost column is cumulative. Costs start from the shared agentic initialization. SearchGen does not report a separate RSIgen-style re-probing stage.

After three cycles, RSIgen achieves 47.5 Full O9 at 1,870 GPU-h, versus 44.2 at 3,418 GPU-h for SearchGen-3 + costs: +3.3 Full O9 with 45.3% fewer accounted GPU-hours. Per-cycle RSIgen scores (42.2/45.3/47.5) also exceed the corresponding SearchGen scores (40.8/43.1/44.2).

These results compare complete training methods, not DRR in isolation. In particular, SearchGen-1 here denotes a single-cycle adaptation to the UMM native-role setting, not an exact numerical reproduction of the original decoupled reasoner–generator configuration.

3. Manuscript Revisions and Conclusion

To address shared concerns about clarity and reproducibility, we will:

  • Present the training sequence in the main text: Generator SFT → Generator RL → Re-probe/Refresh → Agent SFT → Agent RL → Admission.
  • Define DRR, supervision refresh, and the protected-capability guard at first mention; move key DRR decision rules and target-construction requirements to §3.2.
  • Define Overall9, NS, SI, and Full in §4, retaining detailed aggregation in the appendix.
  • Standardize fonts, column labels, and model names in Figure 6.
  • Remove nonstandard retrieval annotations from the bibliography and verify bibliographic records and corresponding in-text claims against their original sources.

These additions clarify RSIgen's contribution and strengthen its evidence regarding learning-recipient correctness, training-stage contributions, statistical reliability, cross-benchmark generalization, and computational efficiency. We thank the reviewers for their constructive feedback and address their remaining specific questions in the individual responses.

AuthorsOpenReview ↗

General Response to All Reviewers (Part 3/4)

2.4 Cross-Benchmark Generalization and Native Generation Retention

To assess whether RSIgen's improvements extend beyond AgentGen-Bench, we evaluated GenEvolve-Bench, KnowGen, and GenEval. The first two assess the complete agentic image-generation workflow. GenEval assesses the generator's native compositional generation without an agent, search, or prompt enhancement.

Table C2. Cross-Benchmark Generalization and Native Generation Capability Retention.

BackboneMethodGenEvolve Overall ↑Knowledge ↑Quality ↑KnowGen ↑GenEval ↑
BAGELInitial G₀ (Direct)0.1980.1600.24713.40.83
SearchGen-3 + costs0.2440.2260.26722.30.74
RSIgen0.2910.2770.30928.60.84
Qwen-ImageInitial G₀ (Direct)0.2920.2360.36514.20.87
SearchGen-3 + costs0.3350.3010.37931.00.81
RSIgen0.3740.3390.41936.80.86
Wan-Image-7BInitial G₀ (Direct)0.2700.2360.31516.30.82
SearchGen-3 + costs0.3450.3160.38226.00.73
RSIgen0.3810.3600.40837.20.83

Results for trained methods on GenEvolve, KnowGen, and GenEval are averaged over three independent training seeds. GenEval uses generator-only direct evaluation.

Generalization across agentic benchmarks. Relative to SearchGen-3 + costs, RSIgen improves GenEvolve Overall by +0.047/+0.039/+0.036 and KnowGen by +6.3/+5.8/+11.2 on BAGEL/Qwen-Image/Wan-Image-7B, respectively. Both the Knowledge-Anchored and Quality-Anchored GenEvolve subsets improve. Quality-Anchored gains are +0.042/+0.040/+0.026, indicating that the benefits also extend to challenging visual-quality requirements, rather than only knowledge-intensive tasks.

Retention of native generation capabilities. Relative to Initial G₀, RSIgen changes GenEval scores by +0.01/−0.01/+0.01, compared with −0.09/−0.06/−0.09 for SearchGen-3 + costs. These results support generalization to other task distributions while largely preserving the compositional generation capabilities measured by GenEval; they do not establish an absence of forgetting across all generation capabilities.

The original paper also reports blinded human evaluation on 200 real-user requests, with pooled tie-adjusted preferences for RSIgen over SearchGen-3 + costs of 70.0% for task completion and 64.9% for visual quality, providing evidence independent of automated judging.

AuthorsOpenReview ↗

General Response to All Reviewers (Part 2/4)

2. Additional Experiments and Findings

2.1 Direct Learning-Recipient Validation

We constructed 20 opportunity clusters with valid targets for both recipients, spanning five capability types. For each cluster, we separately performed agent-directed and generator-directed updates and measured held-out learning gains across three independent training seeds. DRR agreed with the higher-gain recipient in 20/20 (100%) clusters, versus 12/20 (60%) for failure-locus routing. Their selected learning gains were +2.54 vs. +1.92, and selected margins were +1.30 vs. +0.06. These results complement the matched-exposure curriculum gains in Table 3 with direct evidence about learning-recipient selection on the evaluated clusters.

2.2 Independent RL Ablations and Compute-Matched SFT Compensation

On Wan-Image-7B, removing agent RL, generator RL, or both reduced Full Overall9 from 47.5 to 44.6, 45.1, and 43.2 (drops of 2.9, 2.4, and 4.3). Reinvesting the saved RL budget in additional SFT yielded 44.8, 45.6, and 43.5, still 2.7, 1.9, and 4.0 points below full RSIgen. Thus, under the evaluated training configurations and compensation budgets, both RL stages make incremental contributions that additional SFT does not fully recover. Full ablation settings are provided in our response to UJTn.

2.3 Statistical Reliability Across Independent Training Seeds

We report paired differences across three independent training seeds, including their means, standard deviations, 95% confidence intervals (CIs), and the fractions of improved evaluation queries.

Table C1. Statistical Reliability and Query-Level Improvement.

ComparisonMean Δ ↑SD(Δ)95% CI(Δ)Improved Queries ↑
Main Results — Table 8
RSIgen − SearchGen-3 (BAGEL)+3.20.32[2.41, 3.99]76%
RSIgen − SearchGen-3 (Qwen-Image)+3.20.35[2.33, 4.07]82%
RSIgen − SearchGen-3 (Wan)+3.30.37[2.38, 4.22]90%
DRR Allocation — Table 3
DRR − Compatible-uniform+1.40.27[0.73, 2.07]81%
DRR − Failure-locus+1.10.22[0.55, 1.65]77%
Supervision Refresh — Table 4
Refresh − Current-G On-policy+0.90.17[0.48, 1.32]76%
Recursive Curriculum — Table 15
Current sources − Freeze-A+2.70.26[2.05, 3.35]76%
Current sources − Freeze-G+3.50.32[2.71, 4.29]82%
Current sources − Freeze-both+4.70.37[3.78, 5.62]87%
Independent RL Ablations
Full − w/o Agent RL+2.90.24[2.30, 3.50]79%
Full − w/o Generator RL+2.40.22[1.85, 2.95]88%
Full − SFT-only+4.30.33[3.48, 5.12]92%

Protocol: Mean Δ, SD, and 95% CIs are computed from paired performance differences over three independent training runs, using Student-t CIs. Improved Queries denotes the percentage of evaluation requests whose mean difference across training seeds is positive.

Positive improvements are consistently observed across all three backbones. The DRR gains over compatible-uniform and failure-locus routing, as well as the refresh gain over current-generator on-policy adaptation, are also positive across the evaluated training seeds.

We additionally conducted same-generator re-probing with independent sampling banks to distinguish generator-update-induced action-preference changes from finite-sampling noise. Full counts and protocols are presented in our response to LQHx.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.