acceptodds
Under review as a conference paper at ICLR 2027

Stride: Scale-Aware Multi-Agent Evolutionary Refinement for Infographic Generation

Abstract

Infographic generation challenges text-to-image models because a single image must satisfy numerous visual, textual, and semantic constraints. Iterative agentic refinement with a multimodal verifier is a natural remedy, but existing methods face a scale mismatch: local edits cannot correct global composition errors, while full regeneration may destroy already-correct regions. We introduce **Stride**, a constraint-grounded multi-agent framework that treats refinement over any frozen generator as a history-aware evolution strategy. Stride separates constraint evaluation—whether each requirement is satisfied—from visual grounding—where and how it is violated. It then routes global revisions to a composer subagent and local corrections to a retoucher subagent. Each subagent conducts internal planning and generates multiple candidates, while Stride accepts a candidate only when it improves the verifier score, reducing regression across iterations. Because stochastic generators and imperfect verifiers make any single step unreliable, it is this combination of scale-aware routing, per-step redundancy, and monotonic acceptance that yields stable improvement across iterations. With fully open-weight generator–verifier systems, Stride outperforms proprietary systems including Nano-Banana-2.0 and Seedream-5.0-Pro on BizGenEval, matches GPT-Image-2.0 and surpasses Seedream-5.0-Pro on IGenBench, and achieves a new state-of-the-art score on WISE, while substantially outperforming existing agentic methods across all evaluated benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic AI, Multimodal-Agent, Text-to-Infographic Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_eTcNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

STRIDE is a training-free, multi-agent refinement loop that wraps a frozen open-weight text-to-image generator (ERNIE-Image, Infographic-V2, or Qwen-Image-2512) for infographic generation. A rubric verifier (Qwen3.5-397B-A17B) scores each candidate against atomic yes/no constraints, which on BizGenEval and IGenBench are taken directly from the released evaluation checklists; a perceptor (Kimi-K3) returns OCR text boxes and flags garbled text. A lead editor (Kimi-K3) then routes each round either to a composer, which appends failure-derived constraints (optionally web-retrieved) to the prompt and resamples the generator, or to a retoucher, which masks one OCR box and pastes a re-rendered text/LaTeX string. Four candidates are proposed per round, and a (1+λ) rule replaces the incumbent only when the verifier score strictly improves. With best-of-6 initialization and up to three rounds, the paper reports BizGenEval hard scores of 51.1 (ERNIE, from 28.0) and 69.5 (Infographic-V2, from 41.1), IGenBench Q-ACC/I-ACC of 0.84/0.38 (from 0.72/0.17), and 0.92 WiScore on WISE. Supporting evidence includes component ablations (Tables 3, 9), an evolution-strategy comparison (Tables 4, 17), oracle runs that use the benchmark evaluators as verifier/perceptor (Tables 11, 12), and a 100-pair human study with the same atomic questions (Table 5).

Critical strengths
  1. The incumbent-retention mechanism is isolated cleanly and matters. Table 4 / Table 17 compare forced replacement with champion retention at matched candidate counts: (1,1) 39.4 vs (1+1) 45.6, and (1,4) 45.4 vs (1+4) 51.1. This is a well-controlled ablation and shows that rejecting verifier-non-improving updates is a large part of what makes iterative verifier-guided refinement stable. It is useful to anyone building such loops.

  2. The gains are large and survive human rubric evaluation. On the 100 paired prompt–generator cases in Table 5, human-scored BizGenEval hard scores rise by +20.3 (ERNIE) and +29.8 (Infographic-V2), and IGenBench Q-ACC by +0.16 and +0.12, with human–evaluator correlations of 0.65–0.85. This rules out the worry that the improvement exists only in the eyes of the internal verifier. The appendix also documents prompts, verifier inflation (Table 15) and routing statistics (Table 10) in useful detail.

Critical weaknesses

1. The paper's central methodological claim — that scale-aware routing between a global composer and a local retoucher drives the improvement — is not supported by its own ablations.

  • The main-text argument (l. 449–451) compares "retoucher-only" (50.2) with the full system (51.1). According to the ✓/– columns of Table 3 and Table 9, the 50.2 row is lead + retoucher without the perceptor, while 51.1 includes the perceptor. Holding the perceptor fixed, routing adds only +0.7 (lead + retoucher + perceptor = 50.4, Table 3 row 6 / Table 9 row 14).
  • Without the perceptor, adding the composer hurts: lead + retoucher 50.2 vs lead + composer + retoucher 48.9 (Table 3 rows 4 and 5).
  • At matched iteration budgets, retoucher-only is better than the full system after one round (46.9/47.0 vs 46.4) and after two rounds (48.5 vs 48.1); the full system leads only after round three (Table 9). On the easy track after three rounds, the full system is worse (73.7 vs 75.1 for both retoucher-only rows).
  • By capability (three rounds, full vs lead + retoucher + perceptor), the full system gains on attribute (+6.8) but loses on text (−3.8) and knowledge (−1.2), and gains only +0.8 on layout, even though Table 10 shows the lead editor sends 93.7% of layout decisions to the composer (ERNIE). The routing distribution varies by dimension, but this variation does not translate into gains on the dimensions where the composer is used.
  • No result in the paper reports repeated runs, seeds, or confidence intervals, so a 0.7-point difference cannot carry the novelty claim. What the evidence supports is "strong local text repair + multiple proposals + verifier selection + incumbent retention", not error-scale routing.

2. The comparisons with baselines and proprietary models are not made under the same information or compute conditions, so the headline claims cannot be read at face value.

  • Privileged access to the evaluation questions. On BizGenEval and IGenBench the rubric is taken directly from the evaluation checklists (l. 199–201; Appendix A). The composer prompt receives the failed hard questions and the benchmark's capability label (domain=… dimension=…, Appendix J.4, l. 1805–1812), and the retoucher is instructed to answer the failed checklist questions correctly and write those answers into the image (Appendix J.5, l. 1885–1901). For IGenBench, 1,529 of the 5,259 questions are expert-informed rather than prompt-derived (Appendix B), so this channel adds specification beyond the prompt. None of the T2I, proprietary, or agentic baselines receive this information. Appendix I (l. 1645–1646) itself notes that monotonic admission concerns the internal verifier score and does not guarantee improvement under independent evaluation.
  • Gains track evaluator alignment. The Qwen verifier inflates its own best-of-6 picks by +9.1 points relative to Gemini (Table 15), and replacing it with the official evaluator lifts BizGenEval from 51.1 to 64.4 (Table 17). This is consistent with a substantial part of the improvement coming from fitting the scoring model, and it means the oracle numbers should not be presented as "headroom" of the method.
  • Weak baselines and missing controls. Most agentic baselines score below the frozen generator they wrap (BizGenEval ERNIE: frozen 28.0 vs PE 24.2, GenEvolve 22.9, PromptEnhancer-7B/32B 12.0/14.3; Infographic-V2: frozen 41.1 vs PromptEnhancer 9.8/14.0; IGenBench Infographic-V2: frozen 0.72 vs Gen-Searcher 0.60, PromptEnhancer-32B 0.52), which suggests they are mis-configured or ill-suited to these generators. The most informative baselines are missing: best-of-N with the same Qwen verifier and the same number of generator calls (up to 6 + 3×4 = 18), and Kimi-K3 prompt rewriting (with search) + best-of-N. The planners of the baselines are 3B–32B models, whereas STRIDE uses Kimi-K3 and a 397B verifier; the Params column nevertheless lists STRIDE as "8B" (Table 1/2) while listing PE as "8B-3B". No wall-clock, token, or API-call cost is reported anywhere.
  • Selective comparison with proprietary systems. The abstract states that STRIDE outperforms Nano-Banana-2.0 and Seedream-5.0-Pro on BizGenEval and matches GPT-Image-2.0 on IGenBench. In the same tables, Nano-Banana-Pro is clearly stronger (BizGenEval 76.7 vs 69.5, knowledge 82.6 vs 54.0; IGenBench 0.90/0.49 vs 0.84/0.38), and the GPT-Image-2.0 match holds only for I-ACC (Q-ACC 0.88 vs 0.84).
  • WISE does not test the claimed mechanism. In Table 9, lead + composer with a single round already raises WISE from 0.58 to 0.89; the full system reaches 0.92. Almost all of the WISE gain therefore comes from Kimi-K3 making the implicit entity explicit in the prompt. Appendix A's "prompt parser" is in fact a prompt enhancer that may rewrite the prompt entirely and is told not to reveal search-engine use (l. 725–755). WISE evidence therefore speaks to LLM prompt enhancement, not to scale-aware evolutionary refinement.
Questions for authors

Clarifications that could change my assessment

  1. Please report mean ± std (or CIs) over at least 3 independent runs for Table 3 rows 4, 6 and 9 and for the BizGenEval/IGenBench main results. Is the +0.7 difference between lead + retoucher + perceptor (50.4) and the full system (51.1) statistically reliable?
  2. How does the retoucher obtain block indices in the "no perceptor" configurations (Table 3 rows 4–5)? The method section says it first queries the perceptor for OCR blocks (l. 256–260).
  3. Can you re-run BizGenEval and IGenBench with rubrics generated by your own prompt parser from the prompt only (as on WISE), with no access to the released checklists, the failed evaluation questions, or the domain/dimension labels? How much of the gain remains?
  4. Please add (a) best-of-N with the same Qwen verifier and the same number of generator calls as STRIDE, and (b) Kimi-K3 prompt rewriting (+ search) followed by best-of-N, on all three benchmarks, together with per-prompt wall-clock time, number of LLM/VLM calls and tokens.
  5. What is {rules_prior} in the lead-editor prompt (Appendix J.3)? Were these priors derived from runs on BizGenEval/IGenBench test items?
  6. Human study: how many annotators, were they blind to condition (pasted text may be visually identifiable), what is the inter-annotator agreement, and what are the paired CIs of the reported gains with 25 prompts per cell?
  7. Why do most agentic baselines underperform the frozen generator (Table 1/2)? Were their prompts or settings adapted to ERNIE-Image and Infographic-V2?

Suggestions (not required for my assessment)

  1. Report a visual-quality or artifact check for retoucher outputs (e.g., a holistic human preference test, or detection of pasted patches), since the rubric-based study cannot reveal such artifacts.
  2. Revise the abstract to mention Nano-Banana-Pro and to state that the GPT-Image-2.0 match is on I-ACC only, and report the full agent stack in the Params column.
  3. Explain how the verifier (Appendix F.1, chosen on all 400 BizGenEval items), the perceptor (20 BizGenEval images), and the thresholds c=0.7, r=0.8 were selected, ideally on held-out data.
14 comments
AuthorsOpenReview ↗

Response to Reviewer eTcN (1/4)

We thank the reviewer for the close reading of our ablations and evaluation protocols, and for recognizing the evidence from incumbent retention and human evaluation. Below, we address the incremental contribution of scale-aware refinement, the information and compute conditions of the comparisons, and the requested clarifications and checks.


Q1. Please report mean ± std (or CIs) over at least 3 independent runs for Table 3 rows 4, 6 and 9 and for the BizGenEval/IGenBench main results. Is the +0.7 difference between lead + retoucher + perceptor (50.4) and the full system (51.1) statistically reliable?

Three-run results. Table E1 reports the scores averaged over three independent runs and their 95% CIs for paper Table 3 rows 4, 6 and 9, and for the BizGenEval and IGenBench main results with both generators. The three configurations retain the ordering 50.2, 50.4 and 51.1, with Full Stride achieving the highest average Hard score.


Table E1. Results under three independent runs.

SettingScoreCIs
Table 3 row 4: Lead + Retoucher50.2[46.6, 53.9]
Table 3 row 6: Lead + Retoucher + Perceptor50.4[46.8, 54.1]
Table 3 row 9 / main: Stride (ERNIE-Image), BizGenEval51.1[47.3, 54.9]
Stride (Infographic-V2), BizGenEval69.5[66.1, 72.9]
Stride (ERNIE-Image), IGenBench0.55 / 0.03[0.51, 0.59] / [0.02, 0.04]
Stride (Infographic-V2), IGenBench0.84 / 0.38[0.81, 0.87] / [0.34, 0.42]

Interpreting the 0.7-point gap. Full Stride (51.1) and retoucher only with grounding (50.4) differ by 0.7 points. Taken alone, this gap is smaller than the approximately 3.7-point half-width of the individual-score intervals. However, it forms part of a consistent pattern across all router variants (Please see Table E2). Adding the composer and the scale-aware router to the retoucher raises the score from 50.2 to 51.1 (+0.9). Removing the grounding from that full configuration lowers it from 51.1 to 48.9 (−2.2-2.2), placing it below retoucher only. Keeping the grounding while fixing execution to the retoucher gives 50.4, which sits between the two. The rule-based, random and fixed routers (46.5 to 47.7) all sit further below. These shifts consistently support the value of scale-aware routing when paired with grounding. We therefore interpret the +0.7 as a modest positive average increment within this broader pattern, and do not claim statistical significance for that increment alone.


Table E2. Ablation of routing and history controls (BizGenEval Hard, ERNIE-Image).

ConfigurationHistory HtH_tGround EtE_tRoutingHard [95% CI]Δ\Delta
Text repair + (1+4)(1+4)––Fixed (R)47.3 [43.6, 50.9]−3.8-3.8
Composer only (Table 3 row 3)✓–Fixed (C)41.1 [37.4, 44.9]−10.0-10.0
Retoucher only (row 4)✓–Fixed (R)50.2 [46.6, 53.9]−0.9-0.9
Retoucher only + grounding (row 6)✓✓Fixed (R)50.4 [46.8, 54.1]−0.7-0.7
Controller, no grounding (row 5)✓–Scale-aware48.9 [45.1, 52.5]−2.2-2.2
Controller, no history–✓Scale-aware49.7 [46.1, 53.3]−1.4-1.4
Rule-based (error type)–✓Rule47.7 [44.0, 51.4]−3.4-3.4
Random, frequency-matched (58.5% C)––Random46.5 [42.7, 50.2]−4.6-4.6
Random, uniform (50% C)––Random47.2 [43.4, 51.0]−3.9-3.9
Fixed alternation (C/R/C)––C/R/C46.8 [43.2, 50.5]−4.3-4.3
Full Stride (Controller, row 9)✓✓Scale-aware51.1 [47.3, 54.9]–

Q2. How does the retoucher obtain block indices in the "no perceptor" configurations (Table 3 rows 4–5)? The method section says it first queries the perceptor for OCR blocks (l. 256–260).

The block indices come from the retoucher’s branch-local OCR queries. Each branch independently queries the perceptor module for indexed OCR blocks of the current champion, then binds a selected block index to a corrected string. In paper Table 3 rows 4–5, “no perceptor” disables only the controller-facing grounding EtE_t: the lead editor receives no OCR view, but the retoucher’s OCR queries remain active. A retoucher round therefore makes N=4N=4 independent OCR queries, one per branch, in both configurations. The method section describes these branch-local queries, which are distinct from the grounding supplied to the lead editor.

AuthorsOpenReview ↗

Response to Reviewer eTcN (2/4)

Q3. Can you re-run BizGenEval and IGenBench with rubrics generated by your own prompt parser from the prompt only (as on WISE), with no access to the released checklists, the failed evaluation questions, or the domain/dimension labels? How much of the gain remains?

Prompt-only setting. Yes. We reran both benchmarks using our Kimi-K3 prompt parser (paper Appendix A) to construct rubrics from the original prompt alone. No released checklist, failed official evaluation question, or domain/dimension label enters the refinement loop. Feedback comes exclusively from verification against the generated rubric. The official evaluators score only the final images.

How much gain remains. As shown in Table E3, prompt-only Stride retains 89.2% and 92.6% of the BizGenEval Hard gains for ERNIE-Image and Infographic-V2, respectively. On IGenBench, it retains 64.3% / 50.0% of the Q-ACC / I-ACC gains for ERNIE-Image and 75.0% / 76.2% for Infographic-V2. These results show that substantial improvements persist without access to the released evaluation questions.


Table E3. Gain retained with prompt-only rubrics. Retained = (Prompt-only - Frozen) / (Checklist - Frozen).

BenchmarkGeneratorFrozenChecklist StridePrompt-only StrideRetained
BizGenEval (Hard)ERNIE-Image28.051.148.689.2%
BizGenEval (Hard)Infographic-V241.169.567.492.6%
IGenBench (Q-ACC / I-ACC)ERNIE-Image0.41 / 0.010.55 / 0.030.50 / 0.0264.3% / 50.0%
IGenBench (Q-ACC / I-ACC)Infographic-V20.72 / 0.170.84 / 0.380.81 / 0.3375.0% / 76.2%

Retained percentages are calculated from the reported scores. IGenBench values follow the Q-ACC / I-ACC order.


Q4. Please add (a) best-of-N with the same Qwen verifier and the same number of generator calls as STRIDE, and (b) Kimi-K3 prompt rewriting (+ search) followed by best-of-N, on all three benchmarks, together with per-prompt wall-clock time, number of LLM/VLM calls and tokens.

Generation-budget controls. We added both requested controls on all three benchmarks with ERNIE-Image. Both use the same Qwen verifier and benchmark-specific rubric as Stride. We set N=12N=12 for BizGenEval and N=13N=13 for IGenBench and WISE, rounding up Stride’s average T2I calls of 11.2, 12.6 and 12.9. The controls therefore receive slightly larger generation budgets. Table E4 reports per-prompt wall-clock time, LLM/VLM calls and tokens.

Results. As shown in Table E4, Compared with Best-of-NN, Stride improves BizGenEval Hard by 14.9 points, IGenBench Q-ACC by 0.11, and WISE WiScore by 0.25. Against Kimi-K3 rewriting with search followed by Best-of-NN, the gains are 18.0, 0.10 and 0.06, respectively. These improvements persist at approximately matched generation budgets, with additional LLM/VLM, token and latency overhead reported in the table.


Table E4. Generator-budget comparisons on all three benchmarks with ERNIE-Image. Scores are BizGenEval Hard, IGenBench Q-ACC, and WISE WiScore.

BenchmarkMethodScoreT2I callsLLM/VLM callsTokensTime (min)
BizGenEvalBest-of-1236.2122429,2402.05
BizGenEvalKimi-K3 rewrite + search + Best-of-1233.1122534,5485.22
BizGenEvalStride51.111.227.7118,98316.65
IGenBenchBest-of-13130.44132632,4792.67
IGenBenchKimi-K3 rewrite + search + Best-of-13130.45132737,6656.06
IGenBenchStride0.5512.629.3134,42818.21
WISEBest-of-13130.67132620,7731.43
WISEKimi-K3 rewrite + search + Best-of-13130.86132722,4964.20
WISEStride (K=1K=1, tmax⁡=3t_{\max}=3)0.9212.931.848,46116.08

All resources are averaged per prompt. Generation budgets are approximately matched by rounding up Stride’s average T2I calls. WISE uses one initial sample and up to three refinement rounds.


Q5. What is {rules_prior} in the lead-editor prompt (Appendix J.3)? Were these priors derived from runs on BizGenEval/IGenBench test items?

{rules_prior} supplies two fixed collapse heuristics to the lead editor (Appendix J.3). They suggest global resampling when the image has either near-total constraint failure or extensive garbled text. The same heuristics are used for every item and benchmark, and the lead editor may override them based on the current evidence:

  • IF easy=0 AND hard=0 → likely structure_collapsed → redraw

  • IF garbled_rate>0.70 AND acc<=0.10 → likely text_collapsed → redraw

These heuristics were specified from general experience before the benchmark experiments and were not derived from or tuned on BizGenEval or IGenBench test runs.


AuthorsOpenReview ↗

Response to Reviewer eTcN (3/4)

Q6. Human study: how many annotators, were they blind to condition (pasted text may be visually identifiable), what is the inter-annotator agreement, and what are the paired CIs of the reported gains with 25 prompts per cell?

Three annotators scored all 200 images. They were blind to condition: images were shuffled and condition labels hidden. Each annotator answered the same atomic rubric questions used by the benchmark (the 10 hard questions for BizGenEval; all released questions for IGenBench). The human score HH for each image is the arithmetic mean of the three annotators’ scores.

Table E5 reports mean pairwise agreement on the original binary rubric judgments, ranging from 89% to 93%. The 95% CIs reports the 25 prompts within each cell, retaining both images and all three annotators’ scores for each sampled prompt. Human Δ\Delta denotes Stride minus the frozen generator. All four paired intervals exclude zero, supporting positive human-rated improvements with both generators on both benchmarks.


Table E5. Inter-annotator agreement and prompt-paired 95% CIs (25 prompts per cell).

BenchmarkGeneratorPairwise agreementHuman Δ\Delta95% CI
BizGenEval (Hard)ERNIE-Image0.92+20.3[16.2, 24.5]
BizGenEval (Hard)Infographic-V20.93+29.8[25.9, 34.0]
IGenBench (Q-ACC)ERNIE-Image0.89+0.16[0.09, 0.20]
IGenBench (Q-ACC)Infographic-V20.92+0.12[0.07, 0.18]

Q7. Why do most agentic baselines underperform the frozen generator (Table 1/2)? Were their prompts or settings adapted to ERNIE-Image and Infographic-V2?

Baseline settings. We used the released prompts and default settings of each baseline, without generator-specific adaptation to ERNIE-Image or Infographic-V2.

Why the baselines underperform here. The original infographic prompts already specify exact text, numerical values and layout requirements. The baselines introduce prompt rewriting or additional planning, which can alter these specifications and reduce constraint satisfaction. Our controlled comparison shows that rewriting can indeed hurt performance: with the same ERNIE-Image generator, Qwen verifier and 12-sample budget, Kimi-K3 rewriting with search scores 33.1 on BizGenEval Hard, compared with 36.2 for direct Best-of-12 (Table E4). This result is consistent with specification drift as an explanation for the declines, although it does not establish the failure cause of each individual baseline.

Limited correction of text errors. Text constitutes a substantial part of infographic content, including titles, labels and numerical annotations. The prompt-rewriting and resampling baselines evaluated here lack a dedicated text-repair step. They rely on the generator to render the text again, which can reproduce existing errors and introduce new ones in previously correct regions. Additional generation therefore consumes compute without reliably resolving the text errors that limit constraint-satisfaction scores.


Q8. Report a visual-quality or artifact check for retoucher outputs (e.g., a holistic human preference test, or detection of pasted patches), since the rubric-based study cannot reveal such artifacts.

Visual-quality check. We compared 25 final outputs whose last admitted update was a retoucher edit with their immediate pre-edit parents (50 images). Three annotators judged overall visual preference and disruptive patch artifacts on randomized A/B pairs with method labels hidden. Results use case-level majority judgments. Unclear includes no clear preference or no majority. Artifact rates use 25 images per condition.

As shown in Table E6, edited outputs were preferred in 72% of cases, versus 12% for parents, with 16% unclear. Disruptive patch artifacts were observed in 4% of images in each condition. These results support a visual-quality benefit from accepted retoucher edits, with a low observed incidence of disruptive patch artifacts.


Table E6. Visual preference and patch artifacts (25 paired cases).

MeasurePercentage
Preference: edited / parent / unclear72% / 12% / 16%
Disruptive patch artifacts: edited / parent4% / 4%

Q9. Revise the abstract to mention Nano-Banana-Pro and to state that the GPT-Image-2.0 match is on I-ACC only, and report the full agent stack in the Params column.

We will revise the abstract to mention Nano-Banana-Pro and clarify that STRIDE matches GPT-Image-2.0 on I-ACC only. We will also update the Params column to report the full agent stack.

Reviewer_wXgyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes STRIDE, a training-free multi-agent framework for improving infographic generation through iterative, scale-aware refinement. STRIDE employs a rubric-based verifier to assess constraint satisfaction and a perceptor to localize visual errors. A lead editor selects between global regeneration by a composer and local editing by a retoucher, while a (1+λ) evolution strategy preserves the current best candidate unless a better one is found. Experiments on BizGenEval, IGenBench, and WISE demonstrate consistent improvements over frozen text-to-image generators and competitive performance against existing agentic and proprietary methods. Ablation studies support the contributions of targeted refinement, iterative optimization, and incumbent retention, while human evaluations provide additional evidence for the reported gains. Overall, the paper presents a practical framework for improving constraint satisfaction in image generation without fine-tuning the underlying generator.

Critical strengths
  1. A practical and well-motivated refinement framework. STRIDE combines rubric-based verification, spatial grounding, and scale-aware action selection to choose between global regeneration and local editing. The incumbent-retention mechanism further reduces the risk of degrading previously accepted results. This integrated design is a meaningful strength for constraint-intensive image generation tasks.

  2. Broad empirical validation with informative ablations. Experiments on BizGenEval, IGenBench, and WISE demonstrate consistent improvements over base generators. Ablation studies provide evidence for the contributions of iterative refinement, candidate selection, and incumbent retention, while human evaluations offer additional support for the reported gains.

Critical weaknesses
  1. The verifier remains a significant bottleneck in the refinement process. Although the paper reports positive correlations between the default verifier and human judgments (ρ = 0.650–0.785), its oracle experiments show that replacing the admission decision with Gemini increases the BizGenEval Hard average from 51.1 to 56.9, while using Gemini for both candidate ranking and admission raises it to 64.4. This indicates that errors in candidate selection materially limit the framework's performance. However, the human evaluation study only compares STRIDE with the frozen generator, rather than evaluating the oracle variants. It therefore remains unclear whether the substantial oracle gains translate into corresponding improvements in human-perceived quality.

  2. The incremental benefit of the full architecture is not fully established relative to its additional complexity. The Retoucher-only ablation achieves a BizGenEval Hard average of 50.2, close to the full system's 51.1. Although the paper provides further ablations suggesting that spatial grounding benefits the combined Composer–Retoucher configuration, the robustness and statistical significance of these incremental gains remain unclear without repeated-run results or uncertainty estimates. Moreover, the lack of detailed end-to-end cost and latency measurements makes it difficult to assess whether the additional components and iterative candidate evaluation offer a favorable practical trade-off.

Questions for authors
  1. Verifier reliability and oracle gains. Can the authors quantify the frequency and types of errors in candidate ranking and admission? In particular, do the gains observed in the oracle experiments translate into better human-rated constraint satisfaction, or do they primarily reflect closer alignment with the benchmark evaluator?

  2. Reliability of prompt-derived rubrics. On WISE, the verification checklist is generated by Kimi-K3 rather than obtained from ground-truth annotations. How do the authors ensure that these automatically inferred rubrics adequately capture the intended requirements without introducing omissions or incorrect assumptions? An independent validation of the generated checklists would help clarify the reliability of this evaluation pathway.

  3. Sensitivity to confidence thresholds and routing decisions. The framework uses fixed confidence thresholds of 0.7 for the Composer and 0.8 for the Retoucher. How were these values selected, and how sensitive are the results to alternative thresholds? Do the routing decisions remain reliable across different generators and task distributions?

  4. Inference cost and practical efficiency. Can the authors report end-to-end latency, the number of model calls, and the computational cost under the default configuration? A comparison under matched inference budgets would help establish whether STRIDE's quality improvements justify its additional generation, editing, and verification overhead.

5 comments
AuthorsOpenReview ↗

Response to Reviewer wXgy (1/3)

We thank the reviewer for recognizing the practical value of the integrated refinement framework and for highlighting verifier reliability as a key bottleneck. Below, we examine whether the oracle gains extend to human judgments and address rubric reliability, threshold sensitivity, and inference efficiency.


Q1. Verifier reliability and oracle gains. Can the authors quantify the frequency and types of errors in candidate ranking and admission? In particular, do the gains observed in the oracle experiments translate into better human-rated constraint satisfaction, or do they primarily reflect closer alignment with the benchmark evaluator?

Gemini-referenced verifier diagnostics. We analyze Qwen and Gemini judgments on the hard track (Table W1, 400 items) and we quantify both incorrect candidate ranking and incorrect admission using Gemini as reference. (Incorrect candidate ranking): At initialization, Qwen’s best-of-6 pick is not the candidate with the highest Gemini Hard score on 40.5% of the 400 items. (Incorrect admission): Among the 85 archived accepted first-round updates, 5 (5.9%) decrease the Gemini Hard score, providing observed instances of incorrect admission.

The question-level results show both overly lenient and overly strict verification: Qwen passes 7.3% of hard questions that Gemini fails, and fails 5.6% that Gemini passes. Representative lenient errors include overlooking misspelled text, incorrect counts, or partially satisfied requirements. In addition, 14.0% of final outputs have lower Gemini Hard scores than their initial champions, showing that these verifier errors can lead to end-to-end regressions.


Table W1. Verifier frequency and type diagnostics against Gemini (BizGenEval, ERNIE-Image; hard track).

MeasureLayoutAttr.TextKnow.All
Non-optimal initial pick (t=0t=0)43%50%42%27%40.5%
Qwen pass / Gemini fail4.5%9.6%9.7%5.6%7.3%
Qwen fail / Gemini pass8.1%9.4%4.4%0.6%5.6%
Final-score regression21%20%13%2%14.0%
Admission regression (t=1t=1)0/42/90/203/525/85 (5.9%)

Do the oracle gains survive human rating? Yes. As shown in Table W2, we extend the protocol of paper Table 5 to both oracle variants. The same three annotators, blind to condition, score the outputs on the same 25 ERNIE-Image prompts using the same 10 hard questions (50 new images).

Replacing Qwen admission with Gemini raises the human score by 5.6 points, closely matching the official gain of 5.9. Using Gemini for both within-round candidate ranking and admission adds a further 7.9 human points and 8.0 official points. Overall, full oracle improves over default Stride by 13.5 human points and 13.9 official points on the same prompts. Thus, the additional oracle gains are reflected in human-rated constraint satisfaction, rather than appearing only under the benchmark evaluator. These results also confirm that the default verifier materially limits both candidate ranking and admission.


Table W2. Human and official scores for the oracle variants (BizGenEval Hard, ERNIE-Image).

VariantPick / AdmitHOΔ\DeltaHΔ\DeltaO
Frozen generator–30.132.8––
Stride (default)Qwen / Qwen50.453.8+20.3+21.0
Oracle admissionQwen / Gemini56.059.7+5.6+5.9
Full oracleGemini / Gemini63.967.7+7.9+8.0

Q2. Reliability of prompt-derived rubrics. On WISE, the verification checklist is generated by Kimi-K3 rather than obtained from ground-truth annotations. How do the authors ensure that these automatically inferred rubrics adequately capture the intended requirements without introducing omissions or incorrect assumptions? An independent validation of the generated checklists would help clarify the reliability of this evaluation pathway.

The WISE rubric is inferred because the ground-truth annotation of WISE is its answer key. An independent audit against that annotation on 200 prompts finds the correct subject in 97.5% of the rubrics, with 0.8% contradicting items, 1.4% unsupported items, and 3.5% of rubrics missing the key element (Table W3).

AuthorsOpenReview ↗

Response to Reviewer wXgy (2/3)

Why the rubric is inferred. WISE provides one ground-truth annotation per prompt, the official explanation. It states the intended subject and is the reference that the official GPT-4o evaluator judges against. Giving it to the refinement loop would hand Stride the answer that WISE asks the system to infer. The prompt parser therefore sees only the original prompt, and the parser prompt in paper Appendix A states that the checklist is an inference made without ground truth. The rubric guides refinement and admission only. The reported WiScore is computed afterwards by the official evaluator with the official explanations, so a rubric error can reach the reported score only through the image it leads to.

How rubric quality is controlled. The parser must name one primary subject and list its key features as explicit visual statements, so every item can be checked on an image. It also reports a confidence for its own inference and runs a web search when the confidence is below 0.7. Over all 1,000 WISE prompts the mean confidence is 0.912, and the search is triggered on 3.7% of prompts.

Independent validation. Three annotators compared each generated rubric with the official explanation of its prompt. The audit covers 200 prompts stratified over the six WISE categories, with 1,436 rubric items in total. For each prompt the annotators judged whether the inferred subject matches the explanation and whether the rubric omits its key element. For each item they judged whether it contradicts the explanation or lacks support in both the prompt and the explanation. Labels are aggregated by majority vote.

Incorrect are rare. Of the 1,436 rubric items, 0.8% contradict the official explanation and 1.4% have no support in the prompt or the explanation. The rubric omits the key element on 7 of the 200 prompts (3.5%). The inferred subject matches the official explanation on 195 of the 200 prompts (97.5%, 95% interval 94.3 to 98.9). Agreement among the three annotators is substantial (Fleiss’ κ=0.70\kappa=0.70).


Table W3. Independent audit of prompt-derived WISE rubrics against the official WISE explanations. (ERNIE-Image)

MeasurennResult
Prompt level
Inferred subject matches the official explanation20097.5% [94.3, 98.9]
Rubric omits the key element2003.5% [1.7, 7.0]
Item level
Items contradicting the official explanation1,4360.8%
Items unsupported by the prompt or the explanation1,4361.4%
Agreement and outcome
Inter-annotator agreement (Fleiss’ κ\kappa)–0.70
WiScore, matched / mismatched subject195 / 50.95 / 0.38

Q3. Sensitivity to confidence thresholds and routing decisions. The framework uses fixed confidence thresholds of 0.7 for the Composer and 0.8 for the Retoucher. How were these values selected, and how sensitive are the results to alternative thresholds? Do the routing decisions remain reliable across different generators and task distributions?

Threshold selection. Both thresholds were fixed before the main experiments and were not tuned on benchmark scores. The composer threshold of 0.7 follows the prompt parser’s confidence gate, while 0.8 was preset for retoucher branch filtering. They control external knowledge retrieval and edit-branch filtering, respectively. The lead editor selects the execution route from verification, grounding and history.

Threshold sensitivity. Table W4 shows low sensitivity within the tested settings. Lowering the composer threshold to 0.5 or 0.6, or the retoucher threshold to 0.5 or 0.7, leaves the logged decisions unchanged. Raising the composer threshold to 0.8 or 0.9 increases the search rate from 7.53% to 7.92% or 11.20%, while Hard remains 51.1 or rises to 51.2. Thus, the reported performance does not depend on precisely choosing the tested default values.


Table W4. Confidence-threshold sensitivity on BizGenEval Hard with ERNIE-Image. All scores use the same 400 prompts.

τc∗\tau_c^*τr∗\tau_r^*HardΔ\Delta HardSearchPass
0.50.851.107.53%100%
0.60.851.107.53%100%
0.70.851.1–7.53%100%
0.80.851.107.92%100%
0.90.851.2+0.111.20%100%
0.70.551.107.53%100%
0.70.751.107.53%100%

AuthorsOpenReview ↗

Response to Reviewer wXgy (3/3)

Routing across generators and tasks. Table W5 compares lead-editor-guided refinement with random routing using matched overall composer-selection probabilities. C share describes Full’s composer-selection frequency (paper Table 10). Random selects the composer with probability 58.5% for ERNIE-Image and 66.3% for Infographic-V2, retaining the same generator, verifier, checklist, initialization, round budget and admission rule. Random removes the lead editor’s route selection and execution guidance. Δ\Delta is Full minus Random.

Full scores higher in all four capability groups with both generators, improving average Hard by 4.6 points for ERNIE-Image and 9.1 for Infographic-V2. Both generators favor the composer for layout and the retoucher for text, while knowledge routing varies with the generator. These results support the effectiveness of the lead editor’s routing and guidance across both generators and the four BizGenEval task groups.


Table W5. Lead-editor-guided versus random routing (BizGenEval Hard).

ERNIE-ImageInfographic-V2
DimensionC shareRandomFullΔ\DeltaC shareRandomFullΔ\Delta
Layout93.7%59.264.4+5.294.6%66.279.2+13.0
Attribute80.7%47.050.8+3.883.7%59.665.8+6.2
Text15.9%46.055.2+9.218.1%65.679.2+13.6
Knowledge50.6%33.834.0+0.271.2%50.254.0+3.8
Average58.5%46.551.1+4.666.3%60.469.5+9.1

Q4. Inference cost and practical efficiency. Can the authors report end-to-end latency, the number of model calls, and the computational cost under the default configuration? A comparison under matched inference budgets would help establish whether STRIDE’s quality improvements justify its additional generation, editing, and verification overhead.

Table W6 reports the per-test-case resource comparison. On BizGenEval with ERNIE-Image, default Stride averages 13.4 candidates, 11.2 full text-to-image (T2I) calls, 27.7 LLM/VLM calls, 118,983 tokens, 16.65 minutes and $\0.54\ per prompt; 26.3% of cases stop early. The table includes generation, editing, controller and evaluation overhead as accounted for in the measurements.

Comparison at a similar generation budget. Best-of-12 uses 12 T2I calls and scores 36.2 on Hard, compared with 51.1 for Stride with 11.2 calls on average, a difference of 14.9 points. Best-of-14 produces 14 candidates and scores 36.6, compared with 13.4 candidates on average for Stride. These comparisons match approximately on different resource dimensions; the table also reports their other costs.

Cost, latency and local editing. Kimi-K3 rewriting followed by Best-of-12 costs $\0.56\ per prompt and scores 33.1, versus $\0.54\ and 51.1 for Stride. The rewrite baseline is faster in this setup (5.22 vs. 16.65 minutes). Grounded retoucher-only refinement scores 50.4 at $\0.61\ and 20.53 minutes, compared with 51.1 at $\0.54\ and 16.65 minutes for Stride.


Table W6. Average score and resources per test case on BizGenEval Hard with ERNIE-Image.

MethodHardCand.T2ILLM/VLM callsTokensEarly stopLatency (minutes)Cost ($\\)
Best-of-634.7661213896––0.20
Best-of-12 (similar T2I calls)36.212122429240––0.37
Best-of-14 (similar avg candidates)36.614142831351––0.48
Kimi-K3 rewrite + Best-of-1233.112122534548–5.220.56
Text repair + (1 + 4)47.313.2629.49853919.3%14.780.57
Retoucher only + grounding50.412.8630.813264226.7%20.530.61
Stride (Ours)51.113.411.227.711898326.3%16.650.54

Reviewer_DhN4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Stride refines infographics from a frozen image generator without additional training. A verifier checks individual requirements, while a separate perceptor locates text and identifies where corrections are needed. The lead editor uses this feedback and the editing history to choose between regenerating the composition and repairing text or formulas in place. Several candidates are produced at each round; the current image is kept unless one improves the verifier score. Results on BizGenEval, IGenBench, and WISE show sizable gains over the base generators. Human rubric scores also improve after refinement.

Critical strengths
  1. The distinction between global and local errors is useful here. Regenerating an infographic can repair its layout but lose correct text; a text repair tool has the opposite limitation. Stride brings both operations into one workflow and demonstrates gains with more than one frozen generator.

  2. The retention experiment is particularly helpful: with four candidates in both settings, keeping the incumbent raises the BizGenEval Hard score from 45.4 to 51.1. The human study and verifier diagnostics also give some support for the improvements beyond the internal scores.

Critical weaknesses
  1. It is still hard to identify the new contribution of the controller. AutoFigure already uses improvement-based retention in layout space and OCR-based text correction; T2I-Copilot can edit locally and switch between editing and generation. Table 7 needs narrower, more precise definitions of these capabilities. The controller evidence is also thin: the full system gets 51.1 on Hard, versus 50.4 for the lead-editor/retoucher/perceptor configuration, and does worse on Easy (73.7 versus 75.1). Without a history ablation, simple routing controls, or uncertainty estimates, this does not establish the value of the proposed history-aware routing.

  2. Two differences in the evaluation make the gains difficult to attribute. Stride receives released checklists, including expert-informed IGenBench questions, but it is unclear whether all competitors receive them too. The full system can also produce up to 18 candidate results, compared with Best-of-6 for the sampling control. Planning, perception, verification, and search add further work. These are useful results for the complete system, but they leave open how much comes from its design rather than extra task information and computation.

Questions for authors
  1. Which part of Stride goes beyond AutoFigure and T2I-Copilot? Please revisit the capability definitions in Table 7 and test a simple text-repair-plus-retention baseline with the same generator, checklist, and budget.

  2. What do routing and history each contribute? A rule based on error type, fixed or random routing, and a controller without history would help interpret the 0.7-point Hard-score difference. Is that difference consistent across runs? Also, what OCR inputs and calls remain in the Retoucher-on/Perceptor-off setting?

  3. Did every competing method receive the same released checklist? Please compare matched checklist access and a version of Stride that builds its rubric only from the original prompt. This matters for separating better refinement from access to a more detailed task specification.

  4. How does Stride compare with larger Best-of-K sampling, prompt refinement, and local editing when total resources are matched? Please give average candidate counts, model calls, early-stopping frequency, latency, and cost, including the controller and evaluators. The limit of 18 candidate results is not necessarily 18 full text-to-image calls.

7 comments
AuthorsOpenReview ↗

Response to Reviewer DhN4 (1/5)

We thank the reviewer for recognizing the value of combining global regeneration with local repair and for requesting more precise methodological comparisons and controlled baselines. Below, we clarify the differences from prior methods and address the contributions of routing and history, checklist access, and resource use.


Q1. Which part of Stride goes beyond AutoFigure and T2I-Copilot? Please revisit the capability definitions in Table 7 and test a simple text-repair-plus-retention baseline with the same generator, checklist, and budget.

We correct one entry of Table 7. T2I-Copilot does support local editing, through masked inpainting. Table 7 marked this as absent, and we have corrected it.

Table D1 gives the narrower definitions and comparisons among Stride, AutoFigure and T2I-Copilot. Stride goes beyond AutoFigure and T2I-Copilot in five places:

(1) Champion evolution:

  • Stride runs a (1+λ)(1+\lambda) evolution strategy whose candidate is always a complete image and whose parent is always the admitted incumbent. A candidate replaces the champion only when it improves the requirement vector, and a rejected candidate is recorded but never becomes a parent.
  • AutoFigure applies its improvement rule to a symbolic layout, and the illustration is rendered once after the loop ends, so the score the loop admits is never the score of the image that is delivered. Inspecting the released code confirms this at the level of the search itself: the best score is kept for export, while every round refines the latest layout rather than the best-scoring one, so a weaker iteration still becomes the parent of the next and later budget is spent recovering ground already gained.
  • T2I-Copilot compares an attempt with nothing before it, regenerating until a quality threshold or an attempt limit and then returning whatever it holds, so a regression in its final attempt is delivered as it stands.

Ours is the only one of the three (also the only one of the Table 7) whose admitted candidate is the image it reports, and that is what makes repairs accumulate. Because every round starts from the admitted champion, an edit that undoes an earlier admitted fix lowers the score and is rejected, so corrections compound across rounds instead of competing with one another, and each round builds on what the previous rounds secured.

(2) Stable local text editing:

  • The Stride’s retoucher replaces one OCR block with an explicit string through a local renderer. The replacement never passes through a generative model, so the glyphs delivered are exactly the specified string, and the edit cannot disturb the surrounding layout.
  • T2I-Copilot synthesises the edited region with an inpainting model, so the new pixels are a generation output and carry the same uncertainty as the original text.
  • AutoFigure generates its text from layout labels via a generative model, so the content of a text element depends on what the generative step produces and cannot be guaranteed to match a required value.

(3) Choice of scale per round:

  • The Stride’s lead editor reads the failure evidence and the history of prior attempts, then either sends the champion to the retoucher for a block repair or to the composer for a global resample.
  • AutoFigure’s two stages are fixed: Stage I always rewrites the layout and Stage II always overlays text, so which scope applies is decided by the pipeline, never by the failures of the current image.
  • T2I-Copilot selects between two generation models from the user’s prompt, so its choice is blind to whether the current image has a text error, a layout error, or neither.

(4) Rubric feedback:

  • Stride evaluates the atomic requirements of the current instance one by one, and returns for each failure a rationale together with the OCR block where it occurs. What reaches the loop is therefore actionable at either scale: a failed requirement can be answered by revising that local block, or by carrying the requirement as a constraint into a global resample, and the lead editor decides which from the evidence.
  • AutoFigure’s critic rates alignment, balance and overlap over the whole layout, so what reaches its loop is a verdict on the figure as a whole, which says that something is off without saying which requirement is unmet.
  • T2I-Copilot’s evaluator rates composition and subject presence and lists missing prompt elements, so what reaches its loop is a list of things the prompt failed to convey, with no statement of where in the image a requirement is violated. Feedback at that granularity admits only a whole-figure reply, and a loop built on it has one move: redraw, or rewrite the prompt and redraw.
AuthorsOpenReview ↗

Response to Reviewer DhN4 (2/5)

(5) Knowledge acquisition:

  • When a requirement calls for a world fact that the prompt does not supply, the Stride’s composer identifies the gap, issues a targeted search, grounds an explicit constraint in the retrieved evidence, and conditions the generator on it. The constraint stays tied to the requirement, so the loop can return to the same fact in a later round if it remains incorrect.
  • AutoFigure builds its generation context from the symbolic layout and the source text, with no search step in the loop, so any factual error in the layout is passed forward unchanged.
  • T2I-Copilot’s evaluator flags elements missing from the prompt, not facts missing from the world, so an implicit domain requirement that the user never stated is invisible to its loop.

Table D1. Comparison among Stride, AutoFigure and T2I-Copilot.

AutoFigureT2I-CopilotStride
What the candidate isSymbolic layout (SVG/XML)Rendered imageRendered image
How the loop evolvesOne proposal per round; latest layout becomes parentOne proposal; latest result becomes next inputλ=4\lambda{=}4 champions proposed; admitted only if score improves
Verifier granularityGlobal critic score (alignment, balance, overlap)Global quality score (composition, subject presence)Per-requirement rationale with OCR block
Verifier actionabilityFine-letter range of the layoutAesthetic and alignment suggestionsEach failed requirement with a rationale and an OCR block
Knowledge acquisitionNo search; layout rule violations propagateNo search; Missing element labels, no factual retrievalTargeted search, grounded in explicit constraint
How text is producedGenerative model renders from layout labelsInpainting model synthesises masked pixelsExplicit string rendered deterministically
Text editing scopeWhole figure; Stage II overlays all textMasked region chosen by prompt or feedbackSingle OCR block chosen by failure evidence
Choice of scale per roundFixed by pipeline stage orderSelected per attempt from prompt, blind to failureSelected per round from failure evidence: local repair or global resample

Text-repair-plus-retention baseline. As shown in Table D2, we test different text-repair-plus-retention with the same generator, under the same generator, checklist and experiment setups. The most basic ’Text repair + (1 + λ\lambda) retention’ baseline keeps the retoucher as its only sub-agent with four branches per round under the (1+λ)(1+\lambda) admission rule. It keeps the same Qwen verifier and the same K=6K=6 initialization, and it issues six text-to-image calls in total, all for the initial champion. It removes the lead editor, composer, and controller-facing grounding while retaining the retoucher’s internal OCR. The round and candidate budgets match those of full Stride.


Table D2. Ablation of text repair on BizGenEval with ERNIE-Image.

ConfigurationLead EditorSub-agentPerceptorHardEasy
Best-of-6–––34.759.0
Text repair + (1+1)(1+1) retention–Retoucher–44.366.2
Text repair + (1+4)(1+4) retention–Retoucher–47.371.4
Lead + Retoucher (paper Table 3, row 4)✓Retoucher–50.275.1
Lead + Retoucher + Perceptor (Table 3, row 6)✓Retoucher✓50.475.1
Full Stride (row 9)✓Scale-aware✓51.173.7

Table D2 separates the benefits of local repair, proposal diversity, and controller guidance. With one proposal per round, text repair plus retention reaches 44.3 / 66.2 (Hard / Easy). Increasing the proposal count to four raises the scores to 47.3 / 71.4 (+3.0 / +5.2), showing the benefit of candidate exploration even without a controller. At the same four-branch setting, adding lead-editor guidance further improves the scores to 50.2 / 75.1 (+2.9 / +3.7). Thus, the benefit of controller guidance is not explained by a larger proposal count. Full Stride reaches 51.1 / 73.7, exceeding the four-branch text-repair-plus-retention baseline by 3.8 / 2.3 points. The separate contributions of routing and history are examined in Q2.


AuthorsOpenReview ↗

Response to Reviewer DhN4 (3/5)

Q2. What do routing and history each contribute? A rule based on error type, fixed or random routing, and a controller without history would help interpret the 0.7-point Hard-score difference. Is that difference consistent across runs? Also, what OCR inputs and calls remain in the Retoucher-on/Perceptor-off setting?

We systematically evaluated routing and history to clarify their respective contributions. Below, we first describe the configurations compared.

Routing and history comparison setups. All controls share the generator, verifier, checklist, initialization, three rounds and (1+4)(1+4) admission. They differ in what the Controller (the lead editor) reads and in how the sub-agent is chosen:

  • Controller, no history: the Controller reads the verifier result and the grounding EtE_t but not the editing history HtH_t and not the collapse priors.

  • Controller, no grounding: the Controller reads HtH_t and the verifier result but not EtE_t. This is the Perceptor-off configuration of paper Table 3, row 5.

  • Rule-based (error type): no Controller. For each failed requirement, the system checks whether its target text appears in the current OCR output. It routes to the retoucher when most failed requirements are found there and to the composer otherwise. The rule has no tuned threshold and reads EtE_t.

  • Random, frequency-matched: no Controller. Each round draws the composer with probability 58.5%, the frequency the Controller itself produced on ERNIE-Image (paper Table 10). The route mix stays fixed, so this control tests whether the scale-aware router sends the right items to each route.

  • Random, uniform: no Controller. Each round draws the composer with probability 50%. The route mix is not held fixed.

  • Fixed: composer only, retoucher only, or alternation C/R/C. The composer-only and retoucher-only rows come from paper Table 3 and keep the guidance of the Controller.


Table D3. Ablation of routing and history controls (BizGenEval Hard, ERNIE-Image).

ConfigurationHistory HtH_tGround EtE_tRoutingHard [95% CI]Δ\Delta
Text repair + (1+4)(1+4) (Q1 baseline)––Fixed (R)47.3 [43.6, 50.9]−3.8-3.8
Composer only (Table 3 row 3)✓–Fixed (C)41.1 [37.4, 44.9]−10.0-10.0
Retoucher only (row 4)✓–Fixed (R)50.2 [46.6, 53.9]−0.9-0.9
Retoucher only + grounding (row 6)✓✓Fixed (R)50.4 [46.8, 54.1]−0.7-0.7
Controller, no grounding (row 5)✓–Scale-aware48.9 [45.1, 52.5]−2.2-2.2
Controller, no history–✓Scale-aware49.7 [46.1, 53.3]−1.4-1.4
Rule-based (error type)–✓Rule47.7 [44.0, 51.4]−3.4-3.4
Random, frequency-matched (58.5% C)––Random46.5 [42.7, 50.2]−4.6-4.6
Random, uniform (50% C)––Random47.2 [43.4, 51.0]−3.9-3.9
Fixed alternation (C/R/C)––C/R/C46.8 [43.2, 50.5]−4.3-4.3
Full Stride (Controller, row 9)✓✓Scale-aware51.1 [47.3, 54.9]–

What routing and history contribute. Table D3 reports every configuration as the mean over three runs with a 95% CIs. The half-width is about 3.7 points for every row. The two-scale design produces the largest gap. Composer only reaches 41.1 and retoucher only reaches 50.2, a difference of 9.1 points, and the upper end of the composer-only interval (44.9) lies below the lower end of the intervals for retoucher only, retoucher with grounding, both Controller variants and Full Stride.

Among the routers, Full Stride reaches 51.1. Removing history gives 49.7 and removing grounding gives 48.9, which are 1.4 and 2.2 points lower. The rule-based router (47.7), the two random routers (46.5 and 47.2) and the fixed alternation (46.8) are 3.4 to 4.6 points below Full Stride and also below retoucher only (50.2). The scale-aware router, the history and the grounding therefore each contribute: every degradation moves in the same direction, and the largest drops come from replacing the scale-aware Controller by a rule, by random draws or by a fixed schedule.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.