acceptodds
Under review as a conference paper at ICLR 2027

InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics

Abstract

Reliable infographic generation requires facts, symbols, and visual relations to remain consistent through rendering and revision. Correcting one element also requires tracking its supporting evidence and the dependencies affected by the change. We present **InfoAgent**, a training-free framework for *evidence-bound visual-symbolic program synthesis*. Its Infographic Visual Description (IVD) records factual payloads, evidence provenance, execution routes, and verification obligations in a typed dependency graph. Retrieved design priors guide compilation, and layered execution combines raster synthesis with editable symbolic and binding objects while retaining their traces. Dependency-aware repair localizes corrections, rechecks affected dependencies, and requires protected obligations to remain satisfied under the declared checkers. Unresolved obligations remain explicit. On IGenBench, InfoAgent achieves 93.0 Q-ACC and 59.0 I-ACC. We also introduce InfoGraphicBench-Evidence, where complete-checklist pass rates on 200 test requests increase from 21.5% for Same-IVD Prompt to 23.5% for the initial layered output and 28.5% after repair, using the same evidence and initial IVD. On 120 audited repair cases, localized repair edits 12.4% of the canvas on average, compared with 67.3% for global regeneration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Infographic Generation, Agentic Image Generation, Evidence Grounding, Visual-Symbolic Execution, Dependency-Aware Repair

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cyuJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces InfoAgent, a training-free framework for generating and repairing infographics grounded in factual evidence. Its main contribution is the Infographic Visual Description (IVD), an executable representation that links factual content, evidence provenance, spatial regions, element dependencies, execution routes, and verification obligations.

Critical strengths

Strengths

a. Coherent end-to-end framework. The paper integrates evidence provenance, typed execution, verification, and localized repair within a single framework. The IVD provides a practical representation that connects these components across the generation and repair pipeline.

b. Comprehensive evaluation. The experiments include controlled baselines, mechanism ablations, independent external evaluation, and repair-locality analysis. This provides stronger evidence for the proposed method than relying only on qualitative visual examples.

Critical weaknesses

Weaknesses

  1. Limited whole-infographic reliability. Despite the reported improvements, only 28.5% of outputs satisfy the complete external checklist on InfoGraphicBench-Evidence, and 41.0% of IGenBench outputs still fail at least one atomic question. These results indicate that the complete system still has substantial reliability limitations.

  2. Limited evaluation under imperfect evidence. InfoGraphicBench-Evidence uses shared, frozen evidence bundles, while real-world retrieval may return missing, conflicting, or outdated evidence. The current evaluation does not show how robust evidence binding and repair remain under these more realistic conditions.

Questions for authors
  1. Repair objective: Eq. 4 requires improvement in critical obligations, and Section 3.4 stops repair once all critical obligations pass. How are the remaining non-critical failures handled? Could the authors provide a failure breakdown explaining the gap between 61.5% CritPass and 28.5% Full?

  2. Verification reliability: The binding and visual-semantic checkers achieve 84.9 F1 and 82.7 F1, respectively. How often do false PASS or UNKNOWN decisions prevent a correct repair? Could the authors report checker-specific confusion matrices and the final unresolved-obligation rates in the main evaluation?

  3. Efficiency and controlled comparison: What are the average latency, number of model calls, and inference costs for IVD compilation, layered rendering, verification? Does the 3.5-point Full improvement over GenClaw remain when the methods are compared under matched inference budgets?

  4. Novelty What is the fundamental methodological novelty of InfoAgent beyond integrating existing structured generation, evidence grounding, and localized repair techniques, and how do the ablations establish this contribution?

6 comments
AuthorsOpenReview ↗

Response to cyuJ (1/4): reliability and imperfect evidence

Thank you for the detailed questions. We completed the evidence perturbations, request-level Full/CritPass breakdown, checker and obligation-lineage audits, and stage-cost reconstruction reported below. The revision and anonymous artifacts contain the corresponding results, and we summarize their conclusions and limits below.

Whole-infographic reliability. We agree that 57/200 T200 outputs passing Full and I-ACC=59.0% are substantial limitations: 41.0% of IGenBench outputs fail at least one atomic question. Our results support relative improvement and inspectable unresolved obligations, not complete reliability.

Imperfect evidence. We completed two new 80-request diagnostic arms: missing-only and outdated-only. We distinguish the submitted S4 results from the new perturbation results below. In the new arms, generation receives condition-specific evidence, while ReqCov and Full use independent references aligned to the request's target time. NonSup. measures support against the evidence actually supplied; it is not target-time factual correctness. Injected evidence is not the factual reference for ReqCov/Full. Each new run and scoring configuration has a separate immutable version identifier in the released manifest.

Submitted Table S4:

Evidence conditionReqCovFullNonSup.
Frozen bundle84.124/80 (30.0%)0.55%
End-to-end retrieved81.220/80 (25.0%)1.08%
Conflicting pool77.816/80 (20.0%)1.76%

Results from the completed perturbation runs:

Evidence conditionReqCovFullNonSup.
Missing-only73.914/80 (17.5%)2.31%
Outdated-only79.617/80 (21.3%)0.86%

The submitted conflicting pool includes temporal and definition mismatches. In the new arms, missing-only evidence gives Full of 14/80 and ReqCov of 73.9, compared with 17/80 and 79.6 under outdated-only evidence. These are point estimates for the specified perturbations, not evidence that missing information is generally more damaging than outdated information. In 23 outdated-only outputs, a claim was supported by the supplied source but failed the requirement at the request's target time. Source support and temporal correctness therefore measure different properties.

We distinguish deficiencies introduced by the perturbation from errors in subsequent evidence selection, qualification, compilation, and repair. When additional acquisition is unavailable, rendering or binding repair cannot itself provide missing source support. This limits what repair can accomplish under incomplete or temporally mismatched evidence. Replies 2–4 report the failure decomposition, checker audit, efficiency, and methodological comparison.

AuthorsOpenReview ↗

Response to cyuJ (2/4): non-critical failures and the Full/CritPass gap

Q1. Non-critical failures and the Full/CritPass gap. Non-critical obligations enter candidate ranking through F+U after critical non-pass, but need not trigger another repair round once all internal critical obligations pass. Every previously passed critical obligation must remain PASS. P(H) protects previously passed obligations outside the original dependency closure, including non-critical ones; passed non-critical obligations inside that closure lack equivalent hard protection.

Under the evaluator definition, both Full and CritPass exclude additional critical errors. Full additionally requires every non-critical checklist item to PASS. Full therefore implies CritPass. The request-level cross-tab on the same final artifacts and evaluation version is:

CritPass PASSCritPass non-PASS
Full PASS570
Full non-PASS6677

Each of the 66 CritPass-only outputs has no additional critical error under this definition and at least one non-critical checklist FAIL or UNKNOWN. Non-critical is not synonymous with cosmetic: these requirements include secondary content, relations, and layout.

The primary unmet-item category for these 66 requests is:

Primary non-critical failure categoryRequests
Secondary content or explanatory detail24
Non-critical relation or binding13
Secondary layout or reading-flow requirement23
Decorative or stylistic requirement6
Total66

A separate, co-occurring mechanism analysis found 41 requests that stopped after internal critical PASS, 28 with omitted or misconfigured non-critical obligations, 9 with within-closure non-critical regression, and 12 with at least one external non-critical UNKNOWN. These dimensions overlap and are not forced to sum to 66. For example, a request can stop after critical PASS and also contain an omitted non-critical obligation.

The results therefore explain the gap at two different levels. At the metric level, it consists entirely of non-critical checklist non-passes because additional critical errors already block CritPass. At the system level, termination after internal critical PASS is the most frequent recorded factor, while obligation omissions and repair regressions also occur. These overlapping labels do not isolate their causal contributions. It would be incorrect either to call all 66 cases decorative failures or to attribute them to additional critical errors.

AuthorsOpenReview ↗

Response to cyuJ (3/4): checker reliability, unresolved obligations, and lineage

Q2. Verification reliability and unresolved obligations. The originally reported 84.9/82.7 binding/visual-semantic F1 scores were computed on covered decisions, excluding UNKNOWN. Critical UNKNOWN is non-pass and never certified; it enters repair only when the execution trace exposes a typed, bounded target. Those F1 scores alone do not measure how often false PASS or UNKNOWN prevents repair.

We completed an audit of 1,000 checker decisions with externally determinate reference labels across all 200 main runs; another 37 externally undetermined decisions were retained separately. The table summarizes false PASS, false FAIL, and online UNKNOWN counts by checker. It is an error summary rather than a complete confusion matrix. Wrong-target/scope errors remain system verification failures; missing obligations are not checker decisions. Each decision was assessed against an external judgment on the exact artifact and target checked. The new sample is separate from the submitted 600-decision audit.

CheckerAudited decisionsFalse PASSFalse FAILUNKNOWN
Rendering-tree300240
Evidence support250121516
Binding230171914
Visual semantic220242125
Total1,000555955

Among the 55 false PASS cases, all externally confirmed violations, 49 triggered no patch; 52 remained unresolved and 3 were corrected through another check. Of the 55 UNKNOWN decisions, 32 corresponded to externally confirmed violations and 23 to externally passing content. Of the 32 violation-related UNKNOWN cases, 26 remained unresolved and 6 were corrected. Together, 78/87 (89.7%) externally confirmed violation cases associated with audited false PASS or UNKNOWN decisions remained unresolved at the end. This is conditional on the audited decision categories, not an overall unresolved-error rate.

The 23 UNKNOWN decisions on externally passing content are excluded from that rate and analyzed separately for unnecessary repair, rollback, and extra cost. The 37 externally undetermined decisions also remain separate. Across all 55 UNKNOWN decisions, 31 had no bounded target and 24 entered repair; these counts do not specify their cross-tabulation with external truth. These are observed control paths, not claims that correcting the checker would necessarily have repaired the violations. For covered-decision F1, excluding the 55 online UNKNOWN decisions leaves 945 eligible decisions among the 1,000 externally determinate cases.

The final certificates over all 200 runs contain 3,200 active obligations. Their final online states are shown below. These counts describe internal decisions; a PASS entry can still conceal an externally identified violation.

CheckerFinal obligationsFAILUNKNOWN
Rendering-tree1,280320
Evidence support8005438
Binding6404729
Visual semantic4804151
Total3,200174118

Hence, 292/3,200 final active obligations (9.1%) remain FAIL or UNKNOWN. By criticality, 216/2,050 critical obligations (10.5%) and 76/1,150 non-critical obligations (6.6%) remain non-pass. These are obligation-level rates and are not interchangeable with the 77/200 image-level CritPass non-pass rate.

Obligation lineage and acceptance. We also completed the audit of all 412 accepted repair transitions. The initial 3,160 obligations became 3,200 final obligations through 96 additions, 24 equivalent two-to-one merges, and 32 duplicate removals. The audit recorded zero predicate, argument, or scope weakenings, zero criticality demotions, and zero acceptance decisions affected by these merges or duplicate removals. Semantic-equivalence labels were assessed from obligation meaning rather than ID preservation. The audit found no accepted candidate whose reduction in critical non-pass was obtained through deletion or demotion. This inventory ledger describes obligation changes; it is distinct from final-state rates over the initial obligation lineages.

The budget-controlled reruns used the submitted acceptor. No lineage-guard or revised-controller result is included in any number reported here. The error summary, final certificate states, and lineage audit address different quantities: decision errors in the audited sample, active obligations left non-passing, and changes to the declared obligation set.

Reviewer_jg4qOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents InfoAgent, a training-free infographic generation system that benefits from an evidence-bound intermediate representation. The intermediate representation generated by an LLM-based compiler, Inforgraphic Visual Description (IVD), records content, evidence spans, spatial regions, execution routes, bindings, and verification obligations in a typed dependency graph. The image is rendered using IVD in three layers: visual, symbolic, and binding. An IVD-guided verification and dependency-aware repair loop optimizes the process. InfoAgent outperforms existing baselines on both IGenBench and InfoGraphicBench-Evidence.

Critical strengths
  1. The paper uses an intermediate representation in the form of a dependency graph to store information needed to construct an infographic. This improves generation accuracy and enables localized repairs, avoiding unnecessary modifications.

  2. The evaluation is thorough. The paper uses an existing benchmark and also introduces a new benchmark. It includes multiple baselines, ablation studies, and human audits. Comprehensive statistical metrics are reported.

Critical weaknesses
  1. The generated IVD can be validated thoroughly. The LLM-based compiler generates IVD and is used to generate infographics that get evaluated later, but the paper does not report whether the compiler captured all the necessary information.
Questions for authors
  1. Can you assess the completeness and correctness of generated IVDs?
  2. How does the quality of IVDs affect the outcome?
  3. How many of the failures are caused by IVDs? In what cases does it still fail with complete IVDs?
4 comments
AuthorsOpenReview ↗

Response to jg4q (1/3): summary and IVD fidelity audit

Thank you for focusing on compiler fidelity. The original pre-repair IVDs completely and correctly represent 85.9% of applicable requirements and 93.1% of applicable critical requirements. Final Full is 15.4% in the content-defect group and 43.4% in the group satisfying content, execution, and check-coverage criteria; this is an observational comparison. Persistent IVD defects receive the primary audit label in 46/143 Full non-passes, while 43 Full non-passes occur even in the most complete IVD group.

Q1. Completeness and correctness. We completed an audit of all 200 retained pre-repair T200 IVDs from the submitted runs against the versioned requirement inventory released with this rebuttal, including non-repair-eligible cases. Two annotators independently used the request, frozen evidence, and external requirements, blinded to generated outputs and checker decisions; disagreements were adjudicated. Definitions and applicability rules were fixed beforehand. Equivalent structures and supported alternative designs remained valid rather than being forced to match one gold graph.

The audit inventory contains 2,400 requirements, of which 2,372 were applicable. The revision updates the descriptive requirement counts and reports their denominators. We retained 28 exclusions/disputes with reasons. Critical requirements numbered 1,440 in the inventory and 1,426 after applicability decisions. Coverage is reported over both the full inventory and its applicable subset; the submitted output scores are reported separately from these representation-audit measures.

Audit quantityCountRate
Complete representation, applicable requirements2,200/2,37292.7%
Correct-representation recall, applicable requirements2,038/2,37285.9%
Correct-representation recall, all audit-inventory requirements2,038/2,40084.9%
Correct-representation recall, applicable critical requirements1,327/1,42693.1%
Correct-representation recall, all critical audit-inventory requirements1,327/1,44092.2%
Valid-obligation coverage, applicable requirements2,124/2,37289.5%
Valid critical-obligation coverage1,359/1,42695.3%
Requests complete/correct for all critical requirements158/20079.0%
Requests complete/correct for all requirements109/20054.5%

Among applicable requirements, 2,200 were complete, 91 partial, and 81 absent. Correct representation requires both complete representation and correct payload, evidence, and binding content. Obligation validity is assessed separately: it requires appropriate predicate, arguments, scope, criticality, and an applicable implemented checker. Valid-obligation coverage therefore measures a different property from content correctness and checker accuracy.

The main defect categories were 147 payload/evidence errors, 116 binding specification errors, 83 invalid execution specifications, and 67 missing valid critical obligations; categories can co-occur within a requirement. Evidence insufficiency, structural defects, unsupported additions, and supported extras were annotated separately. Request-clustered 95% CIs were [83.7, 88.0] for applicable correct-representation recall and [91.1, 94.7] for critical correct-representation recall.

These results show that critical coverage is substantially higher than overall coverage, but compiler defects remain important: 42/200 requests had at least one critical requirement that was not completely and correctly represented. Reply 2 connects these states to outcomes, and Reply 3 separates persistent IVD defects from downstream failures.

AuthorsOpenReview ↗

Response to jg4q (2/3): IVD quality, outcome transitions, and correction diagnostic

Q2. How IVD quality affects outcomes. The submitted Same-IVD/K=0/InfoAgent Full scores of 21.5%/23.5%/28.5% compare execution and verification-and-repair configurations with the initial IVD fixed. They provide evidence for downstream processing but do not isolate each mechanism or estimate sensitivity to IVD quality.

We therefore linked the frozen audit labels from Reply 1 to the submitted runs. The groups below first distinguish content fidelity, then execution validity, and finally check coverage. They partition the 200 requests according to these successive criteria.

Initial IVD stateRequestsFullCritPass
Content incomplete or incorrect9114 (15.4%)37 (40.7%)
Complete/correct content; invalid execution specification174 (23.5%)10 (58.8%)
Complete/correct content and valid execution; incomplete valid check coverage166 (37.5%)12 (75.0%)
Complete/correct content, valid execution and check coverage7633 (43.4%)64 (84.2%)
Total20057 (28.5%)123 (61.5%)

Higher IVD fidelity is associated with higher final pass rates. This comparison is observational: request difficulty and requirement/binding complexity may affect both IVD quality and output reliability. Fourteen requests with initial content defects eventually passed Full, but this table does not establish whether initial execution or subsequent repair overcame those defects. Conversely, 43/76 requests in the most complete group failed Full and 12/76 failed CritPass. Initial IVD completeness therefore neither determines nor guarantees final correctness.

Recorded before/after transitions. We next used the saved states from each full-system trajectory, (z^0,y^0)->(z^K,y^K). Independently generated K=0 outputs were not substituted for these initial states.

MetricPre-repair PASSRecovered among initial non-PASSRegressed among initial PASSFinal PASS
Full4421/156 (13.5%)8/44 (18.2%)57
CritPass9049/110 (44.5%)16/90 (17.8%)123

These are same-run transitions under the external metrics, not independent K=0 comparisons. In particular, 16 initially CritPass-positive outputs became non-passing. The controller protects declared online checks, so this rule does not guarantee preservation of external CritPass. The observed regressions remain a substantive limitation; aggregate transitions alone do not identify the responsible checker or repair operation.

Label-assisted corrected-IVD diagnostic. Finally, we completed a 60-case diagnostic using cases selected from locked audit labels by a fixed defect-stratified rule, without filtering on output success. Audit labels identified defects; correction content came only from the original request and frozen evidence. We recorded edited fields and necessary dependent changes, without adding evidence or redesigning otherwise valid content. Original and corrected IVDs were rerun with matched models, executor, checker, controller, budget, decoding parameters, and evaluation. Paired seeds were available for 46/60 cases; the remaining 14 used identical decoding settings and are included with that randomness limitation.

OutcomeOriginal IVDCorrected IVDRecoveryRegression
Full15/60 (25.0%)26/60 (43.3%)12/45 initial failures (26.7%)1/15 initial passes (6.7%)
CritPass24/60 (40.0%)39/60 (65.0%)17/36 initial failures (47.2%)2/24 initial passes (8.3%)

Correcting the audited defects increased pass rates in this selected diagnostic, with some regressions. These label-assisted results remain separate from automatic-system performance. We do not extrapolate the stratified recovery rates to all T200 failures or attribute every single-run difference, particularly in the 14 uncontrolled-seed cases, solely to the correction.

AuthorsOpenReview ↗

Response to jg4q (3/3): failure attribution and failures with complete IVDs

Q3. How many failures involve IVDs, and why do complete IVDs still fail? Of the 200 submitted outputs, 143 do not pass Full and 77 do not pass CritPass. These are output non-pass counts, not IVD-failure counts.

The audit traced non-passing external requirements and additional critical errors through the recorded states. The table summarizes the assigned primary request-level labels. These describe observed failure paths, not estimates of unique causal effects. Mixed and undetermined cases remain explicit.

Primary final-failure attributionFull non-passCritPass non-pass
Evidence insufficiency1811
Persistent IVD content/structure/check defect4629
Execution, grounding, or composition5122
Verification or repair control159
Mixed/undetermined136
Total14377

Persistent IVD defects received the primary audit label in 46/143 Full non-passes (32.2%) and 29/77 CritPass non-passes (37.7%). Another 23 Full-nonpassing and 9 CritPass-nonpassing requests had an early IVD defect that subsequently recovered; their final failures are assigned to downstream or mixed categories rather than retroactively blamed on the initial IVD.

Failures in the most complete IVD group. The 76 requests with complete/correct content, valid execution specifications, and valid check coverage contain 43 Full non-passes: 31 are CritPass-only and 12 also fail CritPass.

The 31 CritPass-only outputs contain at least one non-critical checklist FAIL or UNKNOWN and no additional critical error under the evaluator definition. A complete initial specification does not ensure faithful execution or correction of every non-critical error before termination.

For the 12 CritPass failures, the audit assigned primary labels of checker false PASS (4), critical UNKNOWN (3), unavailable bounded repair target (2), unsuccessful/rejected patch (2), and repair regression (1). These are descriptive primary labels, not mutually exclusive causal mechanisms: UNKNOWN and lack of a bounded target can occur in the same unresolved case. Co-occurring conditions are retained separately.

Two representative traces illustrate the distinction:

  • A chart had complete evidence, payloads, execution routes, and obligations, but a raster anchor drifted to the wrong bar. The visual-semantic checker returned UNKNOWN and the trace exposed no bounded patch target; the binding error remained in the final output.
  • A second output satisfied every internal critical obligation, so the controller stopped. A non-critical source note required by the external checklist remained absent. This was a Full failure without a CritPass failure, not an additional critical error.

Persistence labels were allowed to co-occur and were not used as mutually exclusive causes: no applicable obligation, checker false PASS/UNKNOWN, unavailable bounded target, unsuccessful/rejected patch, exhausted scopes/rounds, budget limits, and repair regression. Missing obligations were not relabeled checker errors, and checker decisions were judged against the exact artifact and target checked.

The materials accompanying the revision include requirement mappings, primary and co-occurring labels, and anonymized traces. Persistent IVD defects account for a substantial minority of primary audit labels. Many other failures arise during execution or persist through verification and repair even when the initial IVD is complete. This separates representation fidelity from downstream reliability.

Thank you for your response. All my questions are addressed. Since my original score is on the positive side, I will maintain my score.

Reviewer_NmUsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a system called InfoAgent that generates infographics, that is single images with facts and charts from a user request and some documents. That is, instead of asking an image model to draw everything at once, an LLM (here, Gemini) first writes a structured plan in which each element is linked to the source sentence. The image is then built as follows: the image model only draws backgrounds and illustrations, text and numbers are rendered as SVG, and arrows and callouts are code-driven. The document is then checked. The authors report superior bench numbers to Gemini NanoBanana-Pro. They also introduce a new benchmark where the share of correct graphics goes from 21.5% to 23.5% (theirs) to 28.5% (with repair).

Critical strengths

Pros:

  • The core idea seems sensible, i.e. let each tool do what it is good at
  • Fixing only the broken element, while checking that nothing else breaks, is well-designed and well-measured (I am referring to Table 4)
  • Limitations are clear
  • The new benchmark could be useful, assuming it will be released
Critical weaknesses

Concerns:

  • Comparison to baselines is not budget-matched (see Q1)
  • Some reported statistics look inconsistent (see Q2)
  • The system seems to write its own checks?
Questions for authors
  • Q1: Isn't the comparison with baselines a bit unfair? InfoAgent checks its outputs and repairs them multiple rounds but the baselines produce a single image. Without repair, InfoAgent is below NanoBanana-Pro, right? What if you allowed a multi-shot version of the baselines as well, taking the best output?

  • Q2: The same LLM that plans the infographic also writes checks, right? How often does it miss requirements from the human annotations?

  • Q3: How come InfoAgent without repair have the lowest Bind-Q when the arrow drawing is a key component?

  • Q4: Will you release the code and bench?

3 comments
AuthorsOpenReview ↗

Response to NmUs (1/3): summary and budget-controlled evaluation

Thank you for the questions. We completed the common-cap rerun and the audits reported below. The rerun gives 30.5%/26.0%/27.5% Full for InfoAgent/Same-IVD sampling/layered sampling. Reply 2 reports the initial-IVD audit and binding analysis; Reply 3 describes the anonymous code, benchmark, and result artifacts.

Q1. Budgets and multi-shot controls. Gen-Searcher and GenClaw already use iterative native execution; Direct, RAG, and Same-IVD are single-sample controls. Your NanoBanana-Pro reading is correct: IGenBench Q-ACC/I-ACC is 88.0/45.0 for K=0, 90.0/49.0 for the benchmark-reported NanoBanana-Pro reference, and 93.0/59.0 for complete InfoAgent. That reference is not budget-matched.

In the submitted T200 evaluation, InfoAgent obtains 28.5% Full versus 23.5% for K=0 and 25.0% for GenClaw, at mean costs of USD 0.168 and 0.176 for InfoAgent and GenClaw. Table S3's truncation replays retain 2.5–3.5-point margins over GenClaw, but are not independent hard-budget or best-of-N runs.

We completed three new T200 arms: complete InfoAgent, Same-IVD sampling/selection, and layered sampling/selection without patches. One new initial IVD per request was shared and charged to every arm; InfoAgent was then permitted to update it under the submitted repair rules. The arms used identical evidence, image resolution, evaluator version, and scoring scripts. Shared model components used the same snapshots, while execution paths remained strategy-specific. Selectors used only traces natively produced by their own candidates. The selector saw only the request, evidence, shared IVD, candidates, and available native traces, never external annotations. Its snapshot, scoring, and ties were frozen on development data.

Every attempt, retry, selection, verification/recheck, and patch counted toward the cap. Before each operation, the controller reserved known input/image and maximum output-token charges plus remaining selection/recheck charges. On development data, USD 0.30 was the smallest USD 0.05 increment within USD 0.20–0.40 at which each sampling arm separately produced at least two valid candidates for at least 50% of requests. It was therefore frozen as the primary cap before test-quality evaluation.

ArmValid initial candidates, mean≥2 valid candidatesOutFullCritPassMean cost
InfoAgent1.000.0%100%61/200 (30.5%)128/200 (64.0%)USD 0.226
Same-IVD sampling/selection2.4173.5%100%52/200 (26.0%)115/200 (57.5%)USD 0.201
Layered sampling/selection2.1265.0%99.5%55/200 (27.5%)119/200 (59.5%)USD 0.218

Candidates are valid initial renderings eligible for selection; InfoAgent instead spends its budget on verification and repair. Full retains all 200 requests, counting invalid outputs as failures. Mean latencies were 86.4/67.8/79.2 s, respectively. The cap was common; realized spend differed.

Primary Full comparisonInfoAgent-onlyComparator-onlyDifference [95% CI]Holm-adjusted exact McNemar p
InfoAgent vs Same-IVD sampling112+4.5 pp [1.0, 8.0]0.045
InfoAgent vs layered sampling126+3.0 pp [-1.0, 7.0]0.238

The intervals are paired-bootstrap 95% CIs. The two primary comparisons form one Holm family of two-sided exact McNemar tests. Under the common cap, complete InfoAgent attains higher Full than Same-IVD sampling/selection. The positive difference over layered sampling is not statistically resolved; these results do not establish that repair outperforms every multi-sample strategy. This independent rerun compares complete strategies under shared initial IVDs. It does not decompose the submitted 28.5% case by case, optimize GenClaw's multi-shot strategy, or establish budget-matched superiority over NanoBanana-Pro.

AuthorsOpenReview ↗

Response to NmUs (2/3): IVD omissions, statistics, and binding

This reply addresses Q2 and Q3 using the retained original submission artifacts, not the newly generated budget-controlled outputs in Reply 1.

Q2. Self-generated checks and omitted requirements. In the automatic generation runs, the planner instantiates internal obligations, while external human requirements remain hidden from planning, generation, selection, verification, and repair. Planner, semantic checker, and principal evaluator share a model family: separate calls provide procedural isolation, not model-family independence.

The original certificate audit covers 200 final images (106 TP, 9 FP, 17 FN, 68 TN; 92.2% precision and 86.2% recall against external CritPass). The separate checker audit covers 600 obligation decisions on 120 outputs. Neither estimates initial-IVD requirement omission.

We completed an audit of all 200 retained pre-repair IVDs from the submitted runs against the versioned requirement inventory released with this rebuttal. Two annotators used the request, frozen evidence, and external requirements while blinded to outputs and checker decisions; disagreements were adjudicated. Applicability was fixed without outputs. The audit inventory contains 2,400 requirements: 2,372 were applicable, with 28 exclusions/disputes retained and explained. Critical requirements numbered 1,440 in the inventory and 1,426 after applicability decisions.

Audit quantityCountRate
Completely represented, applicable requirements2,200/2,37292.7%
Completely and correctly represented, applicable requirements2,038/2,37285.9%
Completely and correctly represented, all audit-inventory requirements2,038/2,40084.9%
Completely and correctly represented, applicable critical requirements1,327/1,42693.1%
Valid obligations, applicable requirements2,124/2,37289.5%
Valid critical obligations, applicable critical requirements1,359/1,42695.3%
Requests complete/correct for every critical requirement158/20079.0%
Requests complete/correct for every requirement109/20054.5%

Valid obligations require appropriate predicate, arguments, scope, criticality, and an applicable implemented checker. Obligation validity is evaluated separately from content correctness and does not guarantee checker accuracy. Among applicable requirements, 91 were partial and 81 absent; payload/evidence/binding correctness was assessed separately. This audit directly measures representation omissions and errors, which the certificate matrix cannot establish.

Q3. Why K=0 has lower Bind-Q. Bind-Q is 4.47 for K=0, 4.58 for Same-IVD, and 4.64 for InfoAgent. Paired differences are +0.11 [0.06, 0.16] for Same-IVD minus K=0 and +0.17 [0.11, 0.23] for InfoAgent minus K=0. Removing the binding route reduces full-system Bind-Q to 4.19, supporting its role in the complete system without implying better initial grounding.

We completed an audit of 836 request-matched bindings in the submitted K=0 and Same-IVD outputs. The revision updates the descriptive requirement and binding counts and reports their denominators. Error categories can co-occur.

Binding errorK=0Same-IVD
Missing/wrong target72 (8.6%)52 (6.2%)
Incorrect relation58 (6.9%)44 (5.3%)
Endpoint geometry45 (5.4%)42 (5.0%)
Occlusion63 (7.5%)48 (5.7%)

The excess error counts for K=0 are larger for targets, relations, and occlusion than for endpoint geometry. This pattern is consistent with grounding and visibility contributing to the Bind-Q difference. Because categories overlap, these counts are not a causal decomposition of that difference.

Separately, saved full-system trajectories contained 137 externally confirmed pre-repair binding violations. The audit recorded 112 detections, 17 UNKNOWN cases, and 8 misses, with UNKNOWN reported separately from detections. Repair corrected 84/137; 53 persisted and 9 new violations appeared, leaving 62 final violations. These are same-trajectory transitions; independently generated K=0 images were not used as pre-repair states.

AuthorsOpenReview ↗

Response to NmUs (3/3): code, benchmark, and artifact release

Q4. Code, benchmark, and reproducibility. Yes. The anonymous repository linked in the revised submission contains the following camera-ready paper.

ArtifactContents
System codeIVD compiler, layered executor, verification, repair controller, and baseline adapters
Schemas and promptsIVD schema, checker prompts, selector prompts, evaluator prompts, and repair operators
Benchmark30 development, 30 calibration, and 200 test requests with requirement and binding annotations
EvaluationFull/CritPass, ReqCov, ReqText-F1, Bind-Q, certificate, and repair-audit scripts
Submitted artifactsOriginal IVDs, initial/final images, execution traces, certificates, repair logs, and output hashes
Rebuttal artifactsShared-IVD budget runs, IVD audit labels, failure mappings, evidence perturbations, and analysis notebooks
Model/environment manifestModel snapshots, decoding settings, prompts, provider price sheet, dependency lockfile, and hardware description

Submission and rebuttal runs are stored under separate immutable manifests. The new budget runs use newly compiled shared initial IVDs; the compiler-fidelity audit uses the retained original pre-repair IVDs. Label-assisted corrected-IVD runs are stored under a separate diagnostic manifest. No revised-controller results are included in any automatic-system number reported here.

The benchmark release includes requests, evidence identifiers and offsets, checklists, required-text lists, binding annotations, split manifests, and evaluation scripts. Where source text licenses do not permit redistribution, the repository provides source URLs, document hashes, and span offsets rather than the copyrighted documents. It also documents which evidence bundles are frozen, retrieved, conflicting, missing-only, or outdated-only, together with the target-time references used for task evaluation.

The released audit tables preserve the distinction between image-level certificates, obligation-level checker decisions, external human requirements, and request-level outcomes. UNKNOWN and externally undetermined cases remain explicit with their denominators. A changelog maps every revised table and claim to its underlying artifacts and identifies any corrected implementation separately from the submitted controller.

Comments on the paper

1 comment
AuthorsOpenReview ↗

The case for accepting InfoAgent

Dear Area Chair,

We ask you to support acceptance of InfoAgent. The methodological question that remains after our exchange with Reviewer cyuJ is what InfoAgent contributes beyond combining structured generation and local editing. InfoAgent carries the link between a source-backed requirement and its rendered object through generation into repair. When a check fails, that link identifies the object to revisit, and the dependency graph specifies which other checks a proposed edit must preserve.

Tables 3 and 4 show why this design matters. When the system must locate objects after rendering without the execution traces, the share of outputs passing all critical requirements falls from 61.5% to 44.0%. Across 120 repair-eligible outputs, following dependencies increased repair recall from 71.7% to 77.8% and reduced regressions among previously passing requirements outside the original dependency closure from 4.9% to 0.8%. The mean edited area actually grew from 7.8% to 12.4%. The system could therefore edit a wider region while better preserving correct content elsewhere. We agree with cyuJ that these ablations leave the independent contribution of each protection rule open. They do establish the practical value of the repair policy as a whole.

The revision also addresses two questions central to assessing that evidence. Does the plan capture what users asked for? Two annotators audited all 200 initial plans without seeing the generated images. The plans correctly and completely represented 2,038 of 2,372 applicable requirements, including 1,327 of 1,426 critical requirements. Could the same budget be spent on sampling instead? Under a common $0.30 cap and with the same compiled plan for each request, InfoAgent passed the full external checklist on 61/200 requests, compared with 52/200 for prompt-based sampling and selection and 55/200 for layered sampling and selection. The first paired difference was statistically supported. The comparison with layered sampling remains inconclusive.

Reviewer jg4q identified the graph’s ability to support localized repair as a strength. Reviewer NmUs likewise singled out Table 4’s evaluation of whether fixing one element damages correct content elsewhere. We believe this is the strongest basis for acceptance: the paper gives a concrete method for tracing a factual error to an editable object and measures both what the repair fixes and what it puts at risk. The revision also documents where the method fails, particularly when the supplied evidence is missing or outdated. We respectfully ask you to weigh the demonstrated contribution alongside those limits in your final recommendation.

Thank you for your consideration. The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.