- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a progressive SVG editing benchmark with 4,000 source SVGs and five edits per sequence. AgentIconEdit generates, checks, and refines candidates before using an output for the next edit, improving edit completion and preservation across six model backbones. Additionally, 12 categories of metrics are proposed, covering instruction completion, validity, structural preservation, modification locality, and cumulative drift.
Critical strengths
1, Sequential editing is a practical framework; this benchmark explicitly evaluates cumulative error and the preservation of prior edits. 2. The evaluation process incorporates structural checks, semantic scoring, a variety of backbone models, and a human assessment . 3. By employing fixed evaluation criteria and performing validation prior to state propagation, it is possible to effectively reduce incomplete or destructive editing operations.
Critical weaknesses
- Core preservation metrics cannot reliably distinguish between necessary modifications and unintended damage; for instance, correctly moving an element increases geometric drift, while correctly deleting an element reduces structural preservation.
- AgentIconEdit consumes approximately 12–15 times more tokens and incurs 4–5 times the latency, yet it is primarily compared against a single-pass direct generation baseline. It has not been demonstrated that its advantages stem specifically from its "change-and-preserve" design, rather than from increased sampling, more detailed prompting, or general self-correction mechanisms.
- The scope of the editing process is limited. All trajectories are fixed at five steps, with prompts instructing the model to "enrich the icon" at each stage while avoiding the reversal of previous changes. Real-world design workflows often involve iterative color adjustments, undoing actions, replacing elements, and shifting requirements—scenarios that are not well represented by the current benchmark.
Questions for authors
- Are equal-token comparisons or measured quality-cost curves available?
- How are intended changes distinguished from unwanted drift, and how are preservation constraints updated when a new instruction overrides an earlier edit?
2 comments
Official Response to the Reviewer n1kF
We sincerely thank the reviewer for their thoughtful feedback and constructive suggestions, which have helped us further strengthen our work. We hope our detailed responses and additional results adequately address the reviewer’s concerns and we respectfully invite the reviewer to reconsider their assessment.
Official Response to the Reviewer n1kF
We thank the reviewer for the feedback. Please see our responses below.
[W2 + Q1] Comparison with baselines
Please refer to our response to Reviewer DRWT, W3 + Q1, where we report budget-matched comparisons as requested.
[W1 + Q2a] Intended changes versus unintended drift
Please refer to our response to Reviewer DRWT, W2 + Q2, where we discuss the intended changes versus unintended drifts.
[W3 + Q2b] Trajectory realism and preservation updates
We thank the reviewer for highlighting this.
1. Trajectory realism and coverage. IconEditBench deliberately uses five-step trajectories to isolate cumulative error propagation and preservation under controlled conditions. Importantly, it covers 14 editing operations, including recolouring, scaling, translation, replacement and removal (§3.1). Our additional audit of 4,000 chains (20,000 steps) shows:
Thus, repeated attribute edits are represented, although they may target different objects. We acknowledge that explicit reversals and same-object revisions remain underrepresented.
2. Five-step horizon. Fixing trajectory length ensures fair comparisons across models at identical editing depths. While our pipeline supports parameterized trajectory lengths, we restrict empirical claims to the evaluated five-step setting.
3. Preservation under instruction overrides (Q2b). Our current framework (§4.1) carries forward previously established properties but does not explicitly resolve conflicting instructions. A natural extension would track active object-level properties and remove superseded constraints:
Ptactive=P<t∖Ot
where Ot represents properties overridden by the current instruction. This would enable legitimate revisions while preserving unrelated content. We clarify that this is a proposed extension, not part of the current implementation.
Overall, the controlled setting is a deliberate design choice enabling reproducible evaluation of progressive editing and error propagation. We will clarify these limitations in the revision, while emphasizing that they do not diminish the contributions demonstrated within the evaluated setting.