acceptodds
Under review as a conference paper at ICLR 2027

IconEdit: Benchmarking and Improving Progressive SVG Editing with Change-AND-Preserve Reasoning

Abstract

Progressive SVG icon editing requires models to execute each requested change while preserving visual and structural properties established by previous edits. Yet existing SVG editing benchmarks primarily evaluate isolated edits rather than sequential editing, where each generated output becomes the input to the next instruction. This distinction matters because an incomplete or unintended modification can propagate through subsequent edits, producing cumulative drift or degrading previously established content. We formulate progressive SVG editing as a trajectory-level change-and-preserve problem and introduce IconEditBench, an underlying corpus of 2.45 million SVG icons paired with edit instructions, together with a controlled benchmark of 4,000 source SVGs undergoing five sequential edits each. We further introduce twelve target-free metrics for measuring edit completion, preservation, drift, locality, validity, and trajectory-level reliability. We propose AgentIconEdit, a verifier-guided multi-trajectory framework that verifies and refines candidate edits before propagating them to subsequent states. Across four SOTA closed-source models, AgentIconEdit preserves perfect SVG validity while improving mean Edit Coverage by 4.8%, Spatial Placement by 31.6%, reducing geometric and topological drift by 37.5% and 35.3%, respectively. A 25-participant human study preferred AgentIconEdit over baseline in 80.8% of 25k pairwise comparisons. Together, these contributions provide a benchmark, evaluation metrics and a framework for studying and improving reliable progressive SVG editing. Code and dataset will be made publicly available.

Primary area:
datasets and benchmarks
Keywords:
SVG-editing, progressive editing, vector graphics, icon editing, agentic editing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_n1kFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a progressive SVG editing benchmark with 4,000 source SVGs and five edits per sequence. AgentIconEdit generates, checks, and refines candidates before using an output for the next edit, improving edit completion and preservation across six model backbones. Additionally, 12 categories of metrics are proposed, covering instruction completion, validity, structural preservation, modification locality, and cumulative drift.

Critical strengths

1, Sequential editing is a practical framework; this benchmark explicitly evaluates cumulative error and the preservation of prior edits. 2. The evaluation process incorporates structural checks, semantic scoring, a variety of backbone models, and a human assessment . 3. By employing fixed evaluation criteria and performing validation prior to state propagation, it is possible to effectively reduce incomplete or destructive editing operations.

Critical weaknesses
  1. Core preservation metrics cannot reliably distinguish between necessary modifications and unintended damage; for instance, correctly moving an element increases geometric drift, while correctly deleting an element reduces structural preservation.
  2. AgentIconEdit consumes approximately 12–15 times more tokens and incurs 4–5 times the latency, yet it is primarily compared against a single-pass direct generation baseline. It has not been demonstrated that its advantages stem specifically from its "change-and-preserve" design, rather than from increased sampling, more detailed prompting, or general self-correction mechanisms.
  3. The scope of the editing process is limited. All trajectories are fixed at five steps, with prompts instructing the model to "enrich the icon" at each stage while avoiding the reversal of previous changes. Real-world design workflows often involve iterative color adjustments, undoing actions, replacing elements, and shifting requirements—scenarios that are not well represented by the current benchmark.
Questions for authors
  1. Are equal-token comparisons or measured quality-cost curves available?
  2. How are intended changes distinguished from unwanted drift, and how are preservation constraints updated when a new instruction overrides an earlier edit?
2 comments
AuthorsOpenReview ↗

Official Response to the Reviewer n1kF

We thank the reviewer for the feedback. Please see our responses below.

[W2 + Q1] Comparison with baselines

Please refer to our response to Reviewer DRWT, W3 + Q1, where we report budget-matched comparisons as requested.

[W1 + Q2a] Intended changes versus unintended drift

Please refer to our response to Reviewer DRWT, W2 + Q2, where we discuss the intended changes versus unintended drifts.

[W3 + Q2b] Trajectory realism and preservation updates

We thank the reviewer for highlighting this.

1. Trajectory realism and coverage. IconEditBench deliberately uses five-step trajectories to isolate cumulative error propagation and preservation under controlled conditions. Importantly, it covers 14 editing operations, including recolouring, scaling, translation, replacement and removal (§3.1). Our additional audit of 4,000 chains (20,000 steps) shows:

Editing patternChains
At least one colour edit3,977 (99.4%)
At least two colour edits761 (19.0%)
At least two colour or opacity edits1,806 (45.2%)
At least one replacement or removal edit54 (1.4%)

Thus, repeated attribute edits are represented, although they may target different objects. We acknowledge that explicit reversals and same-object revisions remain underrepresented.

2. Five-step horizon. Fixing trajectory length ensures fair comparisons across models at identical editing depths. While our pipeline supports parameterized trajectory lengths, we restrict empirical claims to the evaluated five-step setting.

3. Preservation under instruction overrides (Q2b). Our current framework (§4.1) carries forward previously established properties but does not explicitly resolve conflicting instructions. A natural extension would track active object-level properties and remove superseded constraints:

Ptactive=P<t∖OtP_t^{\mathrm{active}} = P_{<t} \setminus O_t

where OtO_t represents properties overridden by the current instruction. This would enable legitimate revisions while preserving unrelated content. We clarify that this is a proposed extension, not part of the current implementation.

Overall, the controlled setting is a deliberate design choice enabling reproducible evaluation of progressive editing and error propagation. We will clarify these limitations in the revision, while emphasizing that they do not diminish the contributions demonstrated within the evaluated setting.

AuthorsOpenReview ↗

Official Response to the Reviewer n1kF

We sincerely thank the reviewer for their thoughtful feedback and constructive suggestions, which have helped us further strengthen our work. We hope our detailed responses and additional results adequately address the reviewer’s concerns and we respectfully invite the reviewer to reconsider their assessment.

Reviewer_fFJWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a new benchamark for SVG editing task. The main different of this benchmark compared to others is multi-turn editing setting. It contains 4000 different SVGs, and each of them undergoing 5 sequential edits.

The authors also introduced a new framework, AgentIconEdit, for the sequential editing task. And it achives the best results on their proposed benchmark.

Critical strengths
  • The problem of the proposed benchmark aimed is practical.
  • The experiments conducted on 6 differnet models. And the results shows that the proposed AgentIconEdit brings improvement on all of them.
Critical weaknesses

I acknowledge the effort of this work, but I have to indicates these weaknesses of it:

  • While the benchmark is well motivated, the reliability of this benchmark is need to be validate. The design of the metrics are too heuristic. The author just state how to compute the metrics, but did not explain why evaluate in this way. For example, why the value range is set to [0.5, 3] instead of [0.7, 2]. (in line 291).
  • The proposed method AgentIconEdit is lack of novelty. It just simply combines the planing, multi-candidates, verification, and refinement together. It's too straightforward to make it contribution significant. It's not suprising to see that the final results improved with more tokens spent.
Questions for authors

I have no questions about this paper.

5 comments
AuthorsOpenReview ↗

Official Response to the Reviewer fFJW

We thank the reviewer for the feedback. Please see our responses below. No explicit questions were submitted; the weaknesses are converted into rebuttal questions below.

W1: Benchmark Reliability

We would like to emphasize that the Appendix 14 already presents extensive human validation involving 25 participants, 105,000 rubric judgments and 25,000 task level evaluations. Human evaluators found 86.9% of rubric items valid, rated rubric completeness at 4.52/5, and reported no missing requirements in 91.6% of cases. Interrater agreement was also strong (Fleiss κ = 0.90 on an overlapping subset). Furthermore, an independent study of 25,000 blinded comparisons showed that AgentIconEdit was preferred in 80.8% of cases versus 10.0% for Base, with significantly higher instruction satisfaction and preservation ratings (p < 0.001). These results provide substantial human evidence supporting our evaluation framework.

Regarding the [0.5, 3.0] threshold, this is a permissive nondegeneracy check, not a semantic correctness metric. It detects severely collapsed or expanded renderings while accommodating legitimate scaling and element additions. Importantly, it does not affect Edit Coverage, Structural Preservation, or Target Integrity.

To address threshold sensitivity, we evaluate intervals [0.5, 3.0], [0.6, 2.5] and [0.7, 2.0]. The comparative conclusions remain unchanged, with AgentIconEdit (GPT 5.6 Sol) outperforming Codex (GPT 5.6 Sol) by 1.431%, 1.433% and 1.430% (relative gains) respectively on Successful Edit Score. These results demonstrate that our findings are not dependent on the particular threshold choice.

Finally, our metrics separately measure instruction execution, validity, preservation, locality and trajectory reliability rather than collapsing these distinct properties into a single heuristic score.

AuthorsOpenReview ↗

Official Response to the Reviewer fFJW

W2: AgentIconEdit Contributions

We would like to emphasize that AgentIconEdit is not merely a straightforward combination of planning, candidate generation and verification. Furthermore, we do not claim that these components are individually novel. Our contribution lies in addressing a specific failure mode of progressive editing: an incorrect intermediate SVG can permanently alter the state on which all subsequent instructions operate. This requires more than simply generating multiple candidates or iteratively refining an output.

AgentIconEdit introduces a fixed set of instruction requirements and preservation constraints, evaluates candidates against these requirements and propagates only the selected verified state. Refinement is always rooted in the original SVG for the current instruction, rather than repeatedly modifying an imperfect candidate.

To distinguish these contributions from generic agentic orchestration, we additionally compare against Codex CLI and ClaudeCode, two contemporary general purpose coding agent harnesses (Refer to Review DRWT, Q1). Both are adapted to SVG editing and provided with equivalent source inspection, editing, rendering and validation tools. We use the same GPT-5.6-Sol and Claude Opus 4.6 backbone and matched inference token budgets.

AgentIconEdit outperforms Codex by 11.7% in Edit Coverage and 24.2% in Spatial Placement, and ClaudeCode by 2.1% and 9.4%, respectively, while reducing Excess Edit Distance by 3.6% relative to the strongest baseline (absolute gains). These comparisons are important because they control for the benefits of planning, tool use, iterative execution, rubric-based assessment and verification. The remaining distinction is our explicit change and preservation verification before state propagation.

The existing ablations provide further evidence. Removing the Verifier reduces Edit Coverage from 0.760 to 0.486, while removing the Rubric Agent reduces it to 0.573. Both perform substantially worse than direct GPT 5.5 editing, which achieves 0.749. Moreover, increasing candidate branches from three to five reduces Edit Coverage from 0.760 to 0.580 (Tables 3-4 in Main paper). Thus, additional generation alone does not guarantee improvement. We emphasize that the paper makes two complementary contributions. IconEditBench introduces a controlled benchmark for evaluating stateful editing and accumulated preservation errors, while AgentIconEdit provides a framework specifically designed to improve reliability under these conditions. The additional comparisons against recent agentic harnesses directly support the distinction between generic agent orchestration and our preservation constrained approach.

AuthorsOpenReview ↗

Official Response to the Reviewer fFJW

We sincerely thank the reviewer for their thoughtful feedback and constructive suggestions, which have helped us further strengthen our work. We hope our detailed responses and additional results adequately address the reviewer’s concerns and we respectfully invite the reviewer to reconsider their assessment.

Reviewer_DRWTOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies progressive SVG editing, where each new instruction must be executed while preserving content established in earlier steps. The authors introduce IconEditBench, consisting of a 2.45M SVG-instruction resource and 4,000 five-step editing trajectories, along with 12 target-free evaluation metrics. They also propose AgentIconEdit, which uses multiple editing candidates, rubric-based checking, and visual verification. Experiments across six models show improvements over direct LLM editing on several editing and preservation metrics.

Critical strengths

S1. A meaningful research problem. Progressive SVG editing is a relevant problem. Errors introduced in one step can affect subsequent edits, making trajectory-level evaluation useful for studying instruction following and content preservation. S2. A substantial benchmark contribution. IconEditBench is the paper's strongest contribution. Its 4,000 five-step trajectories provide 20,000 editing instances per system, supported by a large SVG-instruction resource and multiple evaluation metrics. This provides a useful benchmark for studying sequential editing behavior.

Critical weaknesses

W1. Insufficient discussion of closely related work. The related work section could better position the paper against Chat2SVG (CVPR 2025), EdiVal-Agent (ICLR 2026), and IntroSVG (CVPR 2026). These studies address related aspects of SVG editing, multi-turn preservation, or iterative refinement. A more direct discussion would help clarify the contributions of IconEditBench and AgentIconEdit. W2. Limitations of the Cumulative Drift metric. Cumulative Drift measures differences between the original and final SVG states. However, legitimate instructions involving substantial object movement can also increase this metric. Although the paper acknowledges this limitation, lower drift is still interpreted as evidence of better preservation. Distinguishing intended changes from unintended drift would make this conclusion more convincing. W3. Limited baselines and unequal inference budgets. AgentIconEdit is mainly compared with direct LLM editing, without comparisons to other agent-based editing frameworks or standard LLM agent harnesses. It also requires approximately 12-15 times more tokens and 4-5 times higher latency than Base. Without a budget-matched comparison, it remains unclear how much of the improvement comes from the framework design itself. W4. Questions about the verifier's sensitivity to local visual errors. In the blue-book example in Figure 5 (p. 25), the third and fourth instructions add a shadow to the calendar. However, AgentIconEdit substantially alters the small folded corner at the calendar's lower right, producing a result that looks less natural than Base. This raises questions about whether the Rubric Checker and Verifier can reliably detect fine-grained preservation errors before they propagate to later steps. W5. Large performance degradation in the ablations. In Tables 3 and 4, removing the Verifier or Rubric Agent reduces Edit Coverage to 0.486 and 0.573, respectively, compared with 0.749 for direct GPT-5.5 editing in Table 2. The substantial drop below Base deserves further explanation. Understanding why these partially agentic variants perform so poorly would help establish the role of each component.

Questions for authors

Q1. Given the substantial inference overhead, could the authors provide a budget-matched comparison against iterative direct editing or a generic agent harness? This would help establish whether the gains come from the proposed design or the additional inference budget. Q2. Could the authors separate intended transformations from unintended changes when measuring Cumulative Drift, for example by excluding explicitly targeted objects? How would this affect the conclusions about preservation? Q3. Could the authors examine the verifier decisions for Figure 5 and explain why the altered calendar corner was retained? A brief failure analysis of the ablated variants in Tables 3 and 4 would also help clarify what the Rubric Agent and Verifier actually prevent. I would be willing to reconsider my rating if the authors can satisfactorily address my main concerns during the rebuttal.

8 comments
AuthorsOpenReview ↗

Official Response to the Reviewer DRWT

We thank the reviewer for the feedback. Please see our responses below.

W3 + Q1: Comparison with baselines

As requested, we present budget-matched comparisons against two LLM-based agent harnesses as additional baselines. AgentIconEdit (Ours) consistently outperforms both baselines under matched inference budgets.

MethodSVG Validity↑Structural Preservation↑Geometric Cumulative Drift↓Colour Cumulative Drift↓Topological Cumulative Drift↓Failing Steps↓Trajectory Survival↑Edit Coverage↑Colour Adherence↑Spatial Placement↑Target Integrity↑Excess Edit Distance↓Style Consistency↑Successful Edit Score↑
Codex (GPT 5.6 Sol)1.0000.9030.00519.940.070340.9000.7020.8070.7390.9300.6300.8570.978
AgentIconEdit (GPT 5.6 Sol)1.0000.9240.00210.640.028180.9970.7840.8350.9180.9580.5010.8830.992
Claude Code (Opus 4.6)1.0000.8920.00318.550.143780.9010.7020.7040.7350.8230.7130.8370.962
AgentIconEdit (Opus 4.6)1.0000.8960.00317.340.054720.9140.7170.7170.8040.8700.6870.8760.980

Across both backbones, AgentIconEdit consistently outperforms the respective agent harnesses across all metrics, with SVG Validity remaining at 1.000 and Geometric Cumulative Drift matching Claude Code. With GPT 5.6 Sol, AgentIconEdit reduces failing steps from 34 to 18 (47.1%), improves Edit Coverage by 11.7% and Spatial Placement by 24.2%, and reduces Topological Cumulative Drift by 60.0%. Similarly, with Claude Opus 4.6, it reduces failing steps from 78 to 72 (7.7%) and Topological Cumulative Drift by 62.2%, while improving Spatial Placement by 9.4% and Target Integrity by 5.7%.

AuthorsOpenReview ↗

Official Response to the Reviewer DRWT

W2 + Q2: Intended versus unintended drift

We thank the reviewer for this observation. As acknowledged in §5.3, Cumulative Drift (CD) measures accumulated changes and may include legitimate transformations. To quantify this effect, we conducted two additional analyses.

1. Contribution of explicitly requested transformations: Across 20,000 editing steps, only 36 (0.18%), spanning 36 of 4,000 chains (0.90%), are labelled as geometric transformations (translate, scale, rotate, flip, or crop). For AgentIconEdit, these account for only 0.11% and 0.14% of geometric CD on GPT 5.5 and Claude Opus 4.6, respectively. Their contributions to topological CD are 2.2% and 0.5%. Thus, explicitly requested geometric transformations contribute negligibly to the overall geometric drift, although other edit types may also introduce legitimate changes.

2. Instruction aware drift analysis: We further implemented an edit type based attribution analysis using element correspondences and operation categories. We conservatively exclude the full drift contribution of elements modified during authorized geometric, colour, style, or replacement operations, with chain level exclusions for topological CD. Since object level target annotations are unavailable, these exclusions are permissive and may also remove unintended changes. The percentage of original CD remaining after exclusion is:

BackboneGeometric CDTopological CDColour CD
GPT 5.539.3%56.8%2.1%
Claude Opus 4.639.3%44.6%1.6%

These results show that substantial geometric and topological drift remains even after permissive exclusions. However, most colour drift is attributable to potentially intended operations, so we interpret colour CD descriptively rather than as direct evidence of preservation.

3. Evidence supporting preservation: Importantly, our preservation claims do not rely on CD alone. Table 2 shows that AgentIconEdit reduces geometric CD by 40.0% for GPT 5.5 and 70.0% for Claude Opus 4.6, and topological CD by 51.4% and 48.6%, respectively, while improving Edit Coverage, Structural Preservation, and Target Integrity. Independent human evaluation also confirms significantly better preservation (4.31 versus 3.68, Table 11).

We acknowledge that the attribution analysis is approximate and has only been applied to AgentIconEdit, so it does not establish an instruction aware CD advantage over Base. Nevertheless, the combined automated metrics and human evaluations support our conclusion that AgentIconEdit improves preservation, while CD serves as a complementary measure of accumulated change rather than a direct measure of unintended damage. We will highlight this discussion/analysis in the manuscript.

AuthorsOpenReview ↗

Official Response to the Reviewer DRWT

W4 + Q3a: Figure 5 analysis

We thank the reviewer for highlighting this example. The corner modification is partly instruction-driven: Edit 3 explicitly requests 8px rounded corners for the book cover and calendar overlay, while Edit 4 adds a shadow with 20% opacity and a 3px offset. Thus, preserving the original sharp boundaries would conflict with the requested edits.

Verifier trace analysis: We examined the original Claude Opus 4.6 execution trace for Figure 5 (icon 72):

VerificationEdit 3Edit 4
Frozen rubric criteria1319
Initial rubric coverage13/1317/19
Final rubric coverage13/1319/19
Structural score1.01.0
Judge score1.01.0
TerminationSuccess (round 2)Budget exhausted (round 5)

Why was the corner retained?: For Edit 3, the rubric explicitly checked the requested rounding and preservation of previously established properties. All 13 criteria passed. SVG inspection confirms that five surrounding shapes were redrawn with rounded corners while preserving their bounding boxes, and the folded-corner paths themselves remained unchanged.

For Edit 4, the checker initially identified two unsatisfied criteria (17/19), triggering further refinement despite a judge score of 1.0. The final candidate satisfied 19/19 criteria, with the folded-corner paths still unchanged. Importantly, unchanged paths do not guarantee identical visual appearance. The interaction between rounded boundaries and shadows can affect local rendering, which the verifier did not flag. We acknowledge this limitation, also illustrated by the failure cases in Figure 7. Thus, the trace demonstrates explicit preservation checking and refinement, while also identifying a limitation in detecting subtle visual artifacts. We will clarify this distinction in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.