- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes RSIgen, a framework for co-training the agent (orchestration) and generator roles of an agentic image-generation system. Its main components are: (1) Dynamic Responsibility Routing (DRR), which uses matched probes (e.g., Direct vs. Search vs. Oracle evidence) to decide whether a failure case should train the agent or the generator; (2) a “supervision refresh” step that re-evaluates the agent’s action labels after each generator update; and (3) using the updated roles to construct the next round’s curriculum. Experiments on BAGEL, Qwen-Image, and Wan-Image-7B report gains of about 3.2–3.3 points in AgentGen-Bench Full Overall9 over the authors’ three-cycle extension of SearchGen, together with mechanism studies on Wan and a human evaluation on 200 requests.
Critical strengths
- The paper studies a relevant problem: action supervision can become stale after the generator is updated.
- Some mechanism studies use matched controls (e.g., Tables 3 and 4).
Critical weaknesses
- Limited novelty. The core ideas substantially overlap with SearchGen [1], which already trains the generator to internalize stable knowledge and then recalibrates the reasoner to the updated generator’s knowledge boundary. This covers both RSIgen’s internalization/externalization distinction and its “supervision refresh”, leaving DRR, a hand-crafted cascade of if-then rules with unspecified thresholds, as the main addition.
- Limited evaluation benchmarks. All results are reported on a single benchmark (AgentGen-Bench), with no evaluation on other general text-to-image benchmarks.
- Unclear statistical reliability and cost accounting. The main results are said to aggregate multiple training seeds, but no variance or confidence intervals are reported. In addition, the probing procedure is costly (K = 8 images per action per generator version, 2,000 requests per phase), but it is unclear whether this cost is included in the resource budget of the main comparisons.
- Poor presentation. The writing relies heavily on coined terminology and dense nominalized phrasing (e.g., “capability–policy calibration”, “construction-source controls”), and the key metric in the abstract (Overall9) is defined only in the appendix, so it is hard to tell from the main text what the method concretely does. Also, In Figure 6, the prompts use at least three different fonts, and the column headers and method name are inconsistent.
- Insufficient verification of AI-assisted content. Nearly every reference carries non-standard annotations such as “bibliographic record retrieved 2026-09-0X”, which appear to be unedited output of an automated retrieval tool. This suggests that AI-assisted content was not carefully verified, and it places an unnecessary burden on reviewers.
[1] Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation
Questions for authors
See weaknesses 1-5.
5 comments
Response to Reviewer 9YzH (Part 2/3)
W2: Cross-benchmark Generalization and Native Generation Retention
We add three complementary external benchmarks: GenEvolve-Bench (594 requests), KnowGen (630 requests), and GenEval (553 prompts), evaluating open-world agentic image generation, knowledge-intensive generation, and native compositional image generation, respectively. Complete results are provided in Table C2 of our General Response.
(1) Consistent gains on external agentic-generation benchmarks. Relative to SearchGen-3 + costs, RSIgen improves GenEvolve Overall by +0.047/+0.039/+0.036 and KnowGen by +6.3/+5.8/+11.2, for BAGEL, Qwen-Image, and Wan-Image-7B, respectively. Both Knowledge-Anchored and Quality-Anchored GenEvolve subsets improve; gains on the latter are +0.042/+0.040/+0.026. These results demonstrate transfer beyond the original AgentGen-Bench distribution.
(2) Retention of native generator capability. We evaluate generators directly with the original GenEval, without an agent, search, or prompt enhancement. RSIgen scores 0.84/0.86/0.83 across the three backbones, differing from the respective initial generators by only +0.01/−0.01/+0.01. SearchGen-3 + costs instead scores 0.74/0.81/0.73. Thus, RSIgen largely retains the compositional generation capabilities measured by GenEval while improving agentic generation; this is not a claim of zero forgetting across all capabilities.
The original blind human study on 200 separate real-user requests additionally yields pooled tie-adjusted preferences of 70.0% for task completion and 64.9% for visual quality, providing complementary human evidence. We will add the full external-benchmark results, evaluation protocols, and analyses to the revised manuscript.
W3: Statistical Reliability and Complete Computational Costs
We thank the reviewer for noting that the number of training seeds, variance, and probing costs were insufficiently specified.
(1) Three independent training seeds. We now report paired improvement statistics across three independent training runs; complete results are in Table C1 of our General Response. RSIgen gains +3.2/+3.2/+3.3 points over SearchGen-3 on BAGEL/Qwen-Image/Wan, with paired-difference standard deviations of 0.32/0.35/0.37. DRR gains +1.4 over compatible-uniform and +1.1 over failure-locus, with 95% CIs of [0.73, 2.07] and [0.55, 1.65]. Refresh gains +0.9 over current-generator on-policy adaptation, with a 95% CI of [0.48, 1.32]. These paired Student-t intervals support the consistency of the main and mechanism-level improvements across the three training seeds.
(2) Complete three-cycle compute accounting. Matched probing and re-probing incur additional computation, so we now account for training, probing/validation, and re-probing separately on Wan-Image-7B. The per-cycle results appear in Table C3 of our General Response.
| Cost component | SearchGen-3 | RSIgen |
|---|---|---|
| Training GPU-h | 2,976 | 1,152 |
| Probe + Validation GPU-h | 442 | 462 |
| Re-probe GPU-h | — | 256 |
| Total GPU-h ↓ | 3,418 | 1,870 |
| Full Overall9 ↑ | 44.2 | 47.5 |
RSIgen's probing, validation, and re-probing total 718 GPU-hours, compared with 442 GPU-hours for SearchGen-3; its additional diagnostic cost is therefore included, not omitted. Its model-training cost is 1,152 vs. 2,976 GPU-hours. Across the reported GPU-compute stages, RSIgen uses 45.3% fewer cumulative GPU-hours while improving Full Overall9 by +3.3. Moreover, RSIgen reaches 45.3 Full Overall9 after two cycles and 1,098 GPU-hours, exceeding SearchGen-3's final 44.2 at 3,418 GPU-hours.
These results support a better system-level performance–compute trade-off. We do not attribute all compute savings to DRR alone, since the optimization recipes differ.
Response to Reviewer 9YzH (Part 3/3)
W4: Presentation, Metric Definitions, and Figure 6
We appreciate the concern that the manuscript relies on abstract terminology, lacks sufficiently intuitive explanations in the main text, and defers key metric definitions to the appendix. We will make the following revisions.
(1) Clarify terminology and operations. We will provide concise, operational explanations of terms such as capability–policy calibration, construction-source controls, and protected-capability guard at first mention, rather than relying on coined terms to describe the mechanisms.
(2) Move the training workflow and DRR criteria into the main text. We will explicitly present:
Matched Probing → DRR / Target Compilation → Generator SFT/RL → Re-probe / Refresh → Agent SFT/RL → Admission → Next Curriculum.
We will also add a concise example of how a task execution outcome becomes a validated training target in §3.2, and state the principal DRR criteria and thresholds earlier.
(3) Define Overall9 at first use. In §4, we will explain that Overall9 aggregates nine applicable evaluation dimensions: Checklist, Rubric, Prompt Faithfulness, Image Quality, Text Rendering, AI Naturalness, Composition, Physical Plausibility, and Visual Reference. We will also clarify the NS, SI, and Full evaluation strata.
(4) Revise Figure 6. We will standardize fonts, sizes, prompt formatting, model names, and column headers to ensure clear alignment and comparison of outputs across models.
These revisions aim to make the motivation, method, and experimental evidence easier to follow while preserving the technical details.
W5: Reference Formatting
We thank the reviewer for identifying nonstandard annotations in the references. We will remove all retrieval-related and metadata annotations from the revised bibliography.
Summary
We thank the reviewer again for the constructive feedback.
First, RSIgen does not claim to reintroduce alternating generator–reasoner training. Its contribution is to translate evolving capabilities in a unified multimodal model into validated recipient-specific learning targets, explicit action-supervision refresh, and reciprocal curriculum construction. We additionally provide the operational DRR thresholds and counterfactual training evidence to clarify reproducibility and learning value.
Second, the new external evaluations support generalization: RSIgen improves across all three backbones on GenEvolve-Bench and KnowGen while largely preserving the initial generators' compositional performance on GenEval.
Third, the new three-seed statistics and three-cycle compute accounting improve experimental transparency. Within the reported GPU-hour accounting, RSIgen achieves higher system-level quality with lower cumulative GPU compute.
We will further clarify our methodological boundaries, document the DRR implementation and statistical protocols, improve Figure 6, and verify references and related technical descriptions in the revision.
Response to Reviewer 9YzH (Part 1/3)
We thank the reviewer for the detailed assessment of novelty, reproducibility, generalization, reliability, costs, and presentation. We appreciate the recognition that generator updates can invalidate agent supervision and that Tables 3–4 provide controlled studies.
We clarify our distinction from SearchGen and the DRR criteria, and add external evaluations, three-seed statistics, and three-cycle compute accounting.
W1: Novelty Relative to SearchGen and DRR Rule Design
We thank the reviewer for highlighting the relationship to SearchGen. We address three concerns, complementing §1 of our General Response.
(1) Supervision refresh goes beyond retraining the reasoner with an updated generator. SearchGen already adapts its reasoner after generator learning. RSIgen additionally compares relative action utilities under Old-G and New-G while holding inputs, evidence, executable action sets, and sampling conditions fixed. This distinguishes a preference shift from genuinely stale supervision and explicitly reconstructs agent training labels.
This extends beyond Search→Direct transitions. In the original Table 22, Search→Direct accounts for 51.9% of pre-generation shifts, while Regenerate→LocalEdit (39.2%) and ReferenceRegenerate→LocalEdit (16.8%) account for substantial post-generation shifts. Thus, refresh also addresses visual realization, reference conditioning, and editing—not only search recalibration.
(2) DRR compiles validated recipient-compatible targets rather than merely classifying failures as agent- or generator-side. DRR uses predefined, interpretable rules rather than a learned globally optimal attribution model. Matched interventions and input/target validation compile task experience into acquisition, construction, internalization, realization, and other recipient-compatible targets.
To address the request for explicit thresholds, we report the actual criteria:
Table Y1. Operational Criteria for DRR and Supervision Refresh.
Branch quality is estimated with K=8 images/action/version. Preference tolerance governs post-generator supervision refresh, not initial DRR routing.
These rules also have direct learning evidence. Under matched role-level training exposure, original Table 3 shows +1.4/+1.1 Full Overall9 for DRR over compatible-uniform/failure-locus. In the new A/G counterfactual training study, recipient agreement is 20/20 (100%) vs. 12/20 (60%), with selected learning gains of +2.54 vs. +1.92. See our response to Reviewer LQHx for the protocol.
(3) Reciprocal curriculum construction studies how updated roles change future learning opportunities, not simply more alternating cycles. Generator learning changes agent supervision, while improved agent capabilities change the evidence, inputs, and valid generator training targets available in later rounds. Original Table 15 isolates this effect by freezing curriculum-construction sources while retaining recipient learning and current-partner RL: current sources outperform Freeze-both by +4.7 Full Overall9, accompanied by improvements in valid-target yield and supervision freshness.
In summary, RSIgen does not claim to invent knowledge internalization or alternating adaptation; it translates changing native capabilities into validated recipient-specific targets, explicit supervision refresh, and reciprocal curricula, supported by controlled experiments. We will clarify this boundary and move DRR thresholds and target-admission rules into the main text.