acceptodds
Under review as a conference paper at ICLR 2027

ProgressNet: Sketching and Prompting with a Frozen Text-to-Image Model

58 views

Abstract

Humans draw progressively: a few strokes, a look at the result, a stroke erased, a prompt revised. Image generators do not work this way. They typically take a finished sketch and produce the image in a single pass, so every edit starts the picture again, and the models that do keep state across turns are driven by text, cannot take a stroke, and are too slow to draw with. We present ProgressNet, a training-free framework that lets a frozen text-to-image model follow a drawing session as it unfolds: strokes are added and erased, the prompt is revised, and the image keeps up at about a second per turn. It needs no new parameters because the frozen model already has what a progressive generator needs, a pathway through which the previous turn can be remembered, layers that can carry appearance forward without freezing structure, and an internal signal of how far to trust an unfinished sketch; three inference-time mechanisms (Previous-Concept Memory, Layer-Selective K/V Injection and Banded Adaptive Control) use each in turn. As a sketch fills in, every existing method degrades, the FID of the FLUX+ControlNet baseline doubling between 10% and 100% completion on FS-COCO, while ProgressNet's barely moves; it maintains strong fidelity and progressive coherence across three sketch domains and is preferred by users over five competitors, most widely on erasure.

Primary area:
generative models
Keywords:
Controllable Generation, Diffusion Models, Sketch-Conditioned Generation, Training-Free Adaptation, Temporal Consistency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zVtbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents ProgressNet, a training-free framework for progressive sketch-conditioned image generation. It supports stroke addition, erasure, and prompt updates using frozen FLUX.1-schnell and Union ControlNet. The method combines previous-output image conditioning (PCM), selective reuse of previous-turn attention keys and values (LS-KVI), and adaptive memory-strength modulation (BAC). The authors evaluate generation quality and progressive consistency on several sketch datasets. They also provide qualitative examples, ablations, and user studies.

Critical strengths
  1. The paper addresses a useful interactive generation problem. Supporting additions, erasures, and prompt changes while preserving established content could improve sketch-based creative workflows.

  2. The framework requires no additional training and introduces little inference overhead. The reported latency of 0.829 seconds per turn is promising for interaction. The experiments also show substantial improvements over the FLUX+ControlNet baseline.

Critical weaknesses
  1. The quantitative evaluation does not sufficiently measure local editing effectiveness. Most metrics assess whole-image similarity, text alignment, or distributional quality. PFC combines inter-turn similarity with FID, but neither component directly checks whether the requested edit was performed. A simple counterexample is to generate an image from the initial sketch and prompt, then return it unchanged in every subsequent turn. This would achieve perfect inter-turn similarity and could retain a good FID despite ignoring all edits. The user study does evaluate editing, but important details are missing, including participant counts, numbers of judgments, and how responses were converted into the normalized scores. These limitations make it difficult to assess the claimed balance between content preservation and editing responsiveness.

  2. The additional benefit of LS-KVI and BAC over PCM alone is not sufficiently established. PCM is a straightforward way to introduce memory by conditioning on the previous output. It already accounts for most of the quantitative improvement. In Table 4, at full completion, PCM alone achieves a better FID-I than the full method (36.6 versus 37.3), with comparable DINOv2 similarity. The full method improves FID-C and text-alignment scores a bit, but the ablation does not directly evaluate its claimed advantages in progressive consistency and editing. The user studies also exclude ablation variants. A focused human comparison between PCM alone and the full method would help establish whether the additional components provide a meaningful benefit.

  3. The Sketch-a-Sketch baseline is inconsistently identified. Section 5.1 names Sketch-a-Sketch, but the cited Sarukkai et al. (2024) reference is titled “Block and Detail: Scaffolding Sketch-to-Image Generation.” These are distinct projects. This leaves the evaluated implementation and configuration unclear.

Questions for authors
  1. Can authors provide more details about the user studies, including participant counts, evaluation criteria, and the exact normalization procedure used for Table 3?

  2. Can authors directly compare PCM alone with the full method on editing success and preservation of unedited content? A focused user study on the existing editing examples would be particularly helpful.

  3. Which implementation was used for the S-a-S baseline?

7 comments

We thank the reviewer for recognizing the usefulness of our work in the sketch domain, as well as its little computational overhead and clear improvements over the baseline. We address your questions below.

Question 1 · Weakness 1 (user study details)

Question 1: “Can authors provide more details about the user studies, ... Table 3?”

Weakness 1: “… important details are missing, including participant counts, numbers of judgments, …”

Thank you very much for asking this question. We agree that these details of Table 3 (Section 5.3) were missing; they are now in Appendix D (Tables R6–R9), summarized below.

Participant counts: The user study has 49 participants across two evaluations; 25 fully answered the video-based one. They were recruited from open communities via an online Gradio interface (Appendix D, L1294–1295); both studies were double-blind, in randomized order.

Table R6. Demographics of the 49 participants (students: university).

GenderAgeOccupationArtistsNon-artistsTotal
Male20–25Students134
25–30Students11011
25–30Corporate055
50–60Corporate112
Male subtotal31922
Female20–25Students2810
25–30Students21315
25–30Corporate022
Female subtotal42327
Overall total74249

Table R7. Study 1 (overall); all 49 completed it.

CriterionFormatExamplesJudgment sets
Overall preference (Pref.)Ranking (1–6)10490
Temporal coherence (Coher.)Slider (0–1)10490
Generation quality (Qual.)5-point Likert10490
Total3 × 10 × 49—1,470

Table R8. Study 2 (operation-based), pairwise to focus on editing; all 49 completed it.

CriterionFormatExamplesPairwise judgments
Erasure (Erase)Pairwise vs. 5 baselines122,940 (49 × 12 × 5)
Prompt/sketch editing (Edit)Pairwise vs. 5 baselines3735 (49 × 3 × 5)
Total49 × 5 × (12 + 3)153,675

Table R9. Study 2 (video-based): 2 videos for each of the 6 methods (12 in total), 4 sliders each; 25 of 49 completed it.

CriterionFormatExamplesTotal ratings
Progression (Prog.)0–10 slider12300
Fine-grained drawing (F-Draw)0–10 slider12300
Fine-grained erasure (F-Erase)0–10 slider12300
Content preservation (Preserv.)0–10 slider12300
Total1,200

Normalization procedure. Within each criterion, we sum each method’s score over participants and examples:

Normalized score = ( method’s total / total of all six methods ) × 100

The score is: Pref., times placed first; Coher. and the four video criteria, the slider value; Qual., the Likert rating coded [0–4]; Erase and Edit, pairwise comparisons won.

Since ProgressNet appears in every pairwise comparison and each baseline in one fifth of them, we also give its head-to-head win rates against S-a-S / SDXS / CB / Flux / StableFlow: Erase 93 / 91 / 90 / 86 / 66%, Edit 90.5 / 88.4 / 79.6 / 73.5 / 55.8%. StableFlow is the closest competitor, especially on Edit (55.8%), as now clarified in Section 5.3 (L425–429).

Question 2 · Weakness 2 (PCM alone vs. the full method)

Question 2: “Can authors directly compare PCM alone with the full method on editing success and preservation of unedited content? …”

Weakness 2: “… The user studies also exclude ablation …”

Thank you very much for this suggestion. We now report the focused user study (20 participants, three examples per operation) in the main paper.

Table Reb. 1(c). PCM-only vs. the full system: % of judgments preferring the full system (counts).

CriterionAddition ↑Erasure ↑Prompt Editing ↑Overall ↑
Editing Success71.7 (43/60)65.0 (39/60)76.7 (46/60)71.1 (128/180)
Content Preservation73.3 (44/60)60.0 (36/60)68.3 (41/60)67.2 (121/180)

Overall 95% Wilson intervals: [64, 77] (editing success) and [60, 74] (content preservation). We also run the same comparison on editing metrics over 100 FS-COCO sequences.

Table R3. Component ablation on the editing metrics.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
PCM-only0.10100.138016.40
PCM + LS-KVI0.09500.127015.50
Full (Ours)0.09820.124314.27

Stroke adherence is the increase in Canny-edge recall of the new strokes between stages; unedited LPIPS and color drift (ΔE00) measure the change in the regions without new strokes. We observe a trade-off here: the full ProgressNet gives up about 3% of stroke adherence against PCM-only but attains 10% better unedited LPIPS and 13% less color drift. Moreover, the layer choice is what makes LS-KVI work.

Table R4. Layer choice for K/V injection, six layers each (random: mean ± SD over three draws).

MetricNon-vital (Ours)VitalRandom (mean ± SD)
Stroke adherence ↑0.09820.07190.0763 ± 0.0040
Unedited LPIPS ↓0.12430.15080.1527 ± 0.0026
Color drift ↓14.2722.7521.45 ± 2.15

Notably, vital or random layers are worse than no injection at all. LS-KVI builds on StableFlow but with different philosophy of injecting at the opposite end (L505–517 updated). BAC keeps sketch edges from bleeding in (Fig. 9) and recovers the FID-C cost of LS-KVI in Table 4 (0.162 → 0.143). On add → erase sequences (updated Table 8), all five metrics improve monotonically from PCM-only to PCM + LS-KVI to the full system.

Question 3 · Weakness 3

Weakness 3: “The Sketch-a-Sketch baseline is inconsistently identified. …”

Question 3: “Which implementation was used for the S-a-S baseline?”

Good observation. We used the publicly released Sketch-a-Sketch ControlNet checkpoint (vsanimator/sketch-a-sketch), and we have now fixed the citation in Section 5.1 (L319), Table 6 and Appendix D.

Weakness 1 (PFC and local editing)

Weakness 1: “… neither component directly checks whether the requested edit was performed. A simple counterexample is to … return it unchanged in every subsequent turn. …”

Thanks for highlighting this concern. We agree with the reviewer on both points. This is now limitation (ii) in Section 6, and we have revised the introduction (L125–127) from “frame-to-frame similarity alone rewards a method that ignores the new stroke” to “… ignores image quality”. We also test the reviewer’s counterexample, the baseline’s first-turn image repeated at every turn (Table Reb. 1(a)).

Table R2. Edit-following metrics on 100 FS-COCO sequences, including the frozen output.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
Frozen first turn (Flux at 10%)0.0000.0000.00
Flux+ControlNet0.1050.19222.0
StableFlow0.0600.11027.0
Ours0.09820.124314.27

With the output frozen, everything stays at zero. Especially, every compared method follows the strokes (0.105 / 0.060 / 0.0982), so Table 2 compares methods that respond to the edits. ProgressNet keeps 94% of FLUX+ControlNet’s adherence with 35% less change in the unedited regions and the lowest color drift, whereas StableFlow attains locking (lowest unedited LPIPS, highest color drift; Figs. 7–8, 13–35).

PFC was deliberately designed as a quality-adjusted temporal consistency metric (Appendix A.2, L858–859). Its motivation was empirical: frame-to-frame DINOv2 and LPIPS reward StableFlow more and more as it locks, while its FID-I collapses and ProgressNet’s stays flat (Table R1).

Table R1. PFC reconstructed from its underlying measurements on FS-COCO (StableFlow / Ours).

Metric (StableFlow / Ours)10→20%30→50%80→100%
FID-I ↓36.8 / 36.540.7 / 36.052.8 / 37.3
DINO ↑.809 / .819.863 / .840.888 / .874
LPIPS ↓.250 / .251.142 / .231.097 / .206
PFC-DINO ↑2.198 / 2.2442.120 / 2.3331.682 / 2.343
PFC-LPIPS ↑2.174 / 2.1902.151 / 2.2571.726 / 2.223

In Table 1, our DINOv2 rises from 0.400 to 0.413, whereas StableFlow’s falls from 0.397 to 0.359, and our FID-I ends at 37.3 against the baseline’s 67.9. Therefore, consistency counts only when quality is kept; we have now added this discussion to the appendix.

Finally, we think the reviewer has suggested meaningful studies about the paper: the user study of full ProgressNet vs. PCM-only (Table Reb. 1(c), Question 2). We are grateful for it. Together with parts (a) and (b), it now forms a new main-paper table, Table Reb. 1 (page 8), which Sections 5.1, 5.4 and 6 report from.

AuthorsOpenReview ↗

Following up - Reviewer zVtb

Dear Reviewer zVtb,

Thank you for your suggestions. We have tried to address your concerns, and we look forward to your further input and engagement during the rebuttal period.

Reviewer_offmOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a method to generate images while continuously working on a sketch. Users can add or erase lines and change the text description, while the system uses the previous image to keep the rest of the picture consistent. The authors compare it with existing methods on three sets of sketches and through user studies. They report more consistent results across edits, especially when erasing lines, with each update taking about a second on their hardware.

Critical strengths
  1. This paper aims to generate sketches similarly to how humans really sketch. Adding a line should build on the previous picture. Erasing a line should also remove the corresponding evidence from memory. This is an interesting topic of research.

  2. The authors find that carrying old information through some parts of the model can make it resist new strokes. Reusing that information more selectively gives a better balance between keeping the picture recognizable and following the drawing. This is a nice technical insight.

  3. The system uses recent changes inside the model as clues for adjusting how strongly the previous picture should influence the next one. This is a reasonable design choice.

Critical weaknesses
  1. It seems like evaluation can reward ignoring the user’s edits. The proposed scores favor good-looking images that remain similar across turns, without checking whether new strokes are followed. Following equations 9-10, if we repeat the baseline’s early images unchanged it would outscore ProgressNet despite ignoring every later edit (Tables 1–2 and Appendix A.2). These scores therefore do not establish the claimed responsiveness.

  2. The added mechanisms are not clearly justified beyond using the previous image as memory. Table 4 shows that memory alone provides most of the improvement, while the additional components give mixed results.

Questions for authors
  1. How do you justify using PFC to evaluate progressive generation when unchanged baseline images would score higher than ProgressNet?

  2. When each variant runs Table 8’s complete add -> erase sequences, how do PCM alone, PCM + LS-KVI, and the full system compare in recovery and changes outside the erased region?

7 comments

We are pleased that our core philosophy and insights have resonated with the reviewer. Building upon the previous picture is indeed the primal philosophy of this paper. We have run everything asked for on 100 FS-COCO sequences, and all results are now in the revised PDF (Table Reb. 1, page 8; Tables R1–R4 in the appendix). We have addressed your questions below.

Weakness 1 · Question 1

Weakness 1: “It seems like evaluation can reward ignoring the user’s edits. … if we repeat the baseline’s early images unchanged it would outscore ProgressNet despite ignoring every later edit (Tables 1–2 and Appendix A.2). …”

Question 1: “How do you justify using PFC to evaluate progressive generation when unchanged baseline images would score higher than ProgressNet?”

The reviewer has observed a serious limitation of PFC (Eq. 9–10, Appendix A.2). We address it in three steps: (a) the limitation, (b) the counterexample with edit-following metrics (Table R2), and (c) what PFC was designed for (Table R1).

(a) Limitation. We agree: PFC does not check that strokes are followed. This is now limitation (ii) in Section 6, and we have revised the introduction (L125–127) from “frame-to-frame similarity alone rewards a method that ignores the new stroke” to “… ignores image quality”.

(b) Frozen output. We repeat the baseline’s first-turn image at every turn and compare it with FLUX+ControlNet, StableFlow and ProgressNet (Table Reb. 1(a) in the main paper).

Table R2. Edit-following metrics, including the frozen output.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
Frozen first turn (Flux at 10%)0.0000.0000.00
Flux+ControlNet0.1050.19222.0
StableFlow0.0600.11027.0
Ours0.09820.124314.27

Stroke adherence is the increase in Canny-edge recall of the new strokes between stages; unedited LPIPS and color drift (ΔE00) measure the change in the regions without new strokes. The frozen output scores exactly zero. Notably, every compared method follows the strokes (0.105 / 0.060 / 0.0982), so Tables 1–2 compare methods that respond to the edits. ProgressNet keeps 94% of FLUX+ControlNet’s adherence with 35% less change in the unedited regions and the lowest color drift, whereas StableFlow locks (lowest unedited LPIPS, highest color drift; Figs. 7–8, 13–35). As an illustration only, weighting PFC-DINO by stroke adherence drops the frozen output from 3.04 to zero, while ProgressNet stays first (0.227 vs. 0.148 and 0.120).

(c) What PFC measures. PFC was deliberately designed as quality-adjusted temporal consistency (Appendix A.2, L858–859). To deliniate a small discussion into this, we had an interesting observation: frame-to-frame DINOv2 and LPIPS reward StableFlow more and more as it locks, while its FID-I does exactly the opposite. FID-I for StableFlow collapses, while ProgressNet’s stays flat. Therefore, we constructed PFC, which is driven not only by the consistency metric but also by the quality that the resultant image preserves. The table below (Table R1) (StableFlow / Ours) constructs PFC from the existing metrics in the literature.

Table R1. PFC reconstructed from its underlying measurements on FS-COCO (StableFlow / Ours).

Metric (StableFlow / Ours)10→20%30→50%80→100%
FID-I ↓36.8 / 36.540.7 / 36.052.8 / 37.3
DINO ↑.809 / .819.863 / .840.888 / .874
LPIPS ↓.250 / .251.142 / .231.097 / .206
PFC-DINO ↑2.198 / 2.2442.120 / 2.3331.682 / 2.343
PFC-LPIPS ↑2.174 / 2.1902.151 / 2.2571.726 / 2.223

Therefore, consistency counts only when quality is kept. We have now added this discussion to the appendix together with the new results.

Question 2

Question 2: “When each variant runs Table 8’s complete add -> erase sequences, how do PCM alone, PCM + LS-KVI, and the full system compare in recovery and changes outside the erased region?”

We thank the reviewer for suggesting this experiment. We now report all variants in the updated Table 8.

Table 8 (updated). Add → erase ablation with all variants (Appendix B.2).

ConfigurationReturn LPIPS ↓Recovery ↑DINOv2 ↑Erased L1 ↓Outside L1 ↓
PCM-only0.2570.1250.8670.1080.140
PCM + LS-KVI, invalidate0.2250.2400.8780.0960.132
Full system, invalidate0.2040.3030.8850.0850.123
Full system, retain K/V0.2950.0190.8540.1210.096

All five metrics improve monotonically from PCM-only to PCM + LS-KVI to the full system, which is best on 4/5. Retaining K/V is lower only on Outside L1 (0.096 vs. 0.123), but its Recovery is 0.019 against 0.303, which means the image barely moves back. Therefore, we keep invalidation, now clarified in the appendix.

Weakness 2

Weakness 2: “The added mechanisms are not clearly justified beyond ... components give mixed results.”

This is a really good observation by the reviewer. We agree that PCM, the memory, gives the largest gain in Table 4 (Section 5.4, L500–504). However, LS-KVI and BAC govern how the memory is used under editing, which Table 4 does not measure. We therefore measure them directly under editing and with users.

Why LS-KVI? PCM was designed to preserve the previous memory through external image conditioning (Section 4.1), whereas the primary job of LS-KVI (Section 4.2) is to update the internal attention features. To back this layer choice, we compare the non-vital layers against the vital and the randomly selected layers on 100 FS-COCO sequences (Table R4; also in Table Reb. 1(b)).

Table R4. Layer choice for K/V injection (random: mean ± SD over three draws).

MetricNon-vital (Ours)VitalRandom (mean ± SD)
Stroke adherence ↑0.09820.07190.0763 ± 0.0040
Unedited LPIPS ↓0.12430.15080.1527 ± 0.0026
Color drift ↓14.2722.7521.45 ± 2.15

Notably, vital or random layers are worse than no injection at all (PCM-only: 0.1010 / 0.1380 / 16.40), while the least-vital layers are best on all three.

Why BAC? As our paper explicitly mentions in L524–527, BAC’s job (Section 4.3) is beyond uniform metric improvement. Rather, its purpose is to reduce sketch artifacts. To double down on that, Fig. 9 visually shows that, without BAC, sketch artifacts bleed in under harsh sketches. Moreover, BAC does improve FID-C, CLIP and T5 quantitatively (Table 4 at full completion: 0.162 to 0.143, 0.300 to 0.303 and 0.653 to 0.655), which also recovers the small FID-C cost of LS-KVI (0.150 to 0.162). Mechanistically, BAC’s main functionality is to provide adaptive control, which handles the sketch artifacts based on the memory.

Table R3. Component ablation on the editing metrics.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
PCM-only0.10100.138016.40
PCM + LS-KVI0.09500.127015.50
Full (Ours)0.09820.124314.27

We can observe a clear trade-off here: 3% less stroke adherence than PCM-only for 10% better unedited LPIPS and 13% less color drift, the balance LS-KVI and BAC were designed for. To delineate this, we also performed a user study with 20 participants, three examples per operation (now in the main paper).

Table Reb. 1(c). PCM-only vs. the full system: % of judgments preferring the full system (counts).

CriterionAddition ↑Erasure ↑Prompt Editing ↑Overall ↑
Editing Success71.7 (43/60)65.0 (39/60)76.7 (46/60)71.1 (128/180)
Content Preservation73.3 (44/60)60.0 (36/60)68.3 (41/60)67.2 (121/180)

Overall, the full method is preferred in 71.1% of the editing-success judgments and in 67.2% of the content-preservation judgments.

We hope this resolves both weaknesses, and we would be grateful if the reviewer could reconsider the assessment in this light.

AuthorsOpenReview ↗

Following Up - Reviewer offm

Dear Reviewer offm,

Thank you for the suggestions. We have tried to address all of your concerns. We look forward to your further inputs during the discussion period.

Reviewer_9tUZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProgressNet, built upon a frozen FLUX model and a pre-trained Union ControlNet, enables iterative sketching, erasing, and prompt modification through three key mechanisms: Previous-Cycle Memory (PCM), Selective K/V Injection (LS-KVI), and Adaptive Memory Control (BAC). The paper evaluates generation quality across three sketching domains and provides experiments on erasing, a user study, and a wide range of interactive examples.

Critical strengths
  • The task and system have practical value. Supporting structural changes while preserving existing content and maintaining low latency is a meaningful problem in interactive generation.

  • The method substantially improves generation quality and is evaluated across diverse settings. The evaluation covers 11,750 samples across three sketch domains.

Critical weaknesses
  • Most image-quality gains come from PCM. At full completion, PCM-only achieves better FID-I than the full system (36.6 vs. 37.3), leaving the additional benefits of LS-KVI and BAC unclear.

  • The erasure ablation compares retaining versus clearing caches, rather than injection versus no injection. Clearing caches also increases changes outside the edited region.

  • BAC lacks a fixed-control baseline with matched average strength, so the benefit of adaptive modulation remains unverified.

Questions for authors
  • How do PCM-only, PCM with LS-KVI, and the full system perform on the same interaction samples? Do low-importance layers outperform vital layers and randomly selected layers when the number of injected layers is matched? Does BAC outperform fixed control with matched strength?

  • How many participants and valid judgments were included in the user study? Could you provide the raw ratings, normalization procedure, and confidence intervals? The current evaluation includes only 10 samples for the overall study and three prompt-editing samples, so the evidence remains limited.

3 comments

We are grateful to the reviewer for their positive assessment. We have run every experiments asked for on 100 FS-COCO samples and all results are in the revised PDF (Table Reb. 1, page 8; Tables R3-R9, appendix). We address all your concerns below.

Weakness 1 · Question 1, Part 1

Weakness 1: “Most image-quality gains come from PCM. … leaving the additional benefits of LS-KVI and BAC unclear.”

Question 1, Part 1: “How do PCM-only, PCM with LS-KVI, and the full system perform on the same interaction samples?” Thank you for this question. We agree, and we understand that this query emerges from the dominant performance of PCM in Table 4 (Section 5.4). However, our aim was to design LS-KVI and BAC for specific reasons.

Performance of LS-KVI: PCM uses the previous image as external conditioning (Section 4.1), whereas LS-KVI injects internal attention features at specific layers (Section 4.2), driven by Fig. 5 and Fig. 10.

Performance of BAC: Without BAC (Section 4.3), the static conditioning never changes while human drawing does, so sketch artifacts bleed into the image (Fig. 9). In Table 4 at full completion, LS-KVI costs a little FID-C (0.150 to 0.162) and BAC recovers it (0.143, better than PCM-only).

We agree that LS-KVI and BAC govern editing, which Table 4 does not measure, so we now compare all three variants on the same samples.

Table R3. Component ablation on the editing metrics.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
PCM-only0.10100.138016.40
PCM + LS-KVI0.09500.127015.50
Full (Ours)0.09820.124314.27

Stroke adherence is the increase in Canny-edge recall of the new strokes between stages; unedited LPIPS and color drift (ΔE00) measure the change in the regions without new strokes. We can observe a clear trade-off here: the full system gives up about 3% of stroke adherence against PCM-only for 10% better unedited LPIPS and 13% less color drift, the balance LS-KVI and BAC were designed for. Moreover, BAC recovers the FID-C cost of LS-KVI in Table 4 (0.150 → 0.162 → 0.143).

We provide a user study with 20 participants, three examples per operation, now in the main paper.

Table Reb. 1(c). PCM-only vs. the full system: % of judgments preferring the full system (counts).

CriterionAddition ↑Erasure ↑Prompt Editing ↑Overall ↑
Editing Success71.7 (43/60)65.0 (39/60)76.7 (46/60)71.1 (128/180)
Content Preservation73.3 (44/60)60.0 (36/60)68.3 (41/60)67.2 (121/180)

Overall 95% Wilson intervals: [64, 77] (editing success) and [60, 74] (content preservation).

Weakness 2

Weakness 2: “The erasure ablation compares retaining versus clearing caches, rather than injection versus no injection. Clearing caches also increases changes outside the edited region.” Thank you for raising this point. We agree, and Table 8 now adds the no-injection setting (Appendix B.2, L1142-1149 in the revised version).

Table 8 (updated). Add → erase ablation with all variants (Appendix B.2).

ConfigurationReturn LPIPS ↓Recovery ↑DINOv2 ↑Erased L1 ↓Outside L1 ↓
PCM-only0.2570.1250.8670.1080.140
PCM + LS-KVI, invalidate0.2250.2400.8780.0960.132
Full system, invalidate0.2040.3030.8850.0850.123
Full system, retain K/V0.2950.0190.8540.1210.096

All five metrics improve monotonically from PCM-only to PCM + LS-KVI to the full system, which is best on 4/5. The reviewer is right that clearing raises Outside L1 (0.123 vs. 0.096 retained). However, retaining gives a Recovery of 0.019 against 0.303: the image barely moves back, so Outside L1 alone rewards ignoring the erase. Among the settings without stale K/V, the full system changes the outside region the least (0.123 vs. 0.132 and 0.140). We have now clarified this in the appendix (“Erasure-specific LS-KVI state invalidation”).

Question 1, Part 2

Question 1, Part 2: “Do low-importance layers outperform vital layers and randomly selected layers when the number of injected layers is matched?” Insightful observation. We match the count at six layers in every setting; ours are {5, 6, 7, 10, 13, 14}.

Table R4. Layer choice for K/V injection (random: mean ± SD over three draws, chosen uniformly distributed across all layers).

MetricNon-vital (Ours)VitalRandom (mean ± SD)
Stroke adherence ↑0.09820.07190.0763 ± 0.0040
Unedited LPIPS ↓0.12430.15080.1527 ± 0.0026
Color drift ↓14.2722.7521.45 ± 2.15

The non-vital layers are best on all three metrics. Notably, the vital and the random injection fall below PCM-only (0.1010 / 0.1380 / 16.40), i.e., worse than no injection at all, so the selection of the layers is what makes the injection useful.

Weakness 3 · Question 1, Part 3

Weakness 3: “BAC lacks a fixed-control baseline with matched average strength, …”

Question 1, Part 3: “Does BAC outperform fixed control with matched strength?”

Good experimental suggestion. On 100 FS-COCO sequences, we compare against a fixed control whose matched average strength is derived from the BAC correction factors (Section 4.3), applied uniformly rather than layer- and step-wise (Table R5), additionally put in the main paper, page 8, Table Reb. 1(b). We also include the setting without BAC.

Table R5. BAC against a fixed control with matched average strength.

MethodStroke adherence ↑Unedited LPIPS ↓Color drift ↓
Without BAC (PCM + LS-KVI)0.09500.127015.50
Fixed BAC-matched0.09670.124915.20
Full BAC0.09820.124314.27

Despite the matched strength, full BAC is better on all three, and all three improve step by step from no BAC to fixed to full BAC. The gain therefore comes from applying the correction layer- and step-wise, not from its magnitude.

Question 2, Part 1

Question 2, Part 1: “How many participants and valid judgments were included in the user study? Could you provide the raw ratings, normalization procedure, and confidence intervals?”

We thank the reviewer for this question. All details are now in Appendix D (Tables R6–R9); we provide them below.

Table R6. Demographic distribution of the 49 participants (students: university).

GenderAgeOccupationArtistsNon-artistsTotal
Male20–25Students134
25–30Students11011
25–30Corporate055
50–60Corporate112
Male subtotal31922
Female20–25Students2810
25–30Students21315
25–30Corporate022
Female subtotal42327
Overall total74249

Table R7. Study 1 (overall); all 49 completed it.

CriterionFormatExamplesJudgment sets
Overall preference (Pref.)Ranking (1–6)10490
Temporal coherence (Coher.)Slider (0–1)10490
Generation quality (Qual.)5-point Likert10490
Total3 × 10 × 49—1,470

Table R8. Study 2 (operation-based); all 49 completed it.

CriterionFormatExamplesPairwise judgments
Erasure (Erase)Pairwise vs. 5 baselines122,940 (49 × 12 × 5)
Prompt/sketch editing (Edit)Pairwise vs. 5 baselines3735 (49 × 3 × 5)
Total49 × 5 × (12 + 3)153,675

Table R9. Study 2 (video-based): 2 videos for each of the 6 methods, 4 sliders each; 25 of 49 completed it.

CriterionFormatExamplesTotal ratings
Progression (Prog.)0–10 slider12300
Fine-grained drawing (F-Draw)0–10 slider12300
Fine-grained erasure (F-Erase)0–10 slider12300
Content preservation (Preserv.)0–10 slider12300
Total1,200

Normalization procedure. For each criterion, a method’s score is summed over participants and examples:

Normalized score = ( method’s total / total of all six methods ) × 100

The score depends on the format. Pref.: times ranked first. Coher. and the four video criteria: the slider rating. Qual.: the Likert rating, [0–4]. Erase and Edit: pairwise comparisons won.

Confidence intervals. Head-to-head win rates vs. S-a-S / SDXS / CB / Flux / StableFlow with 95% Wilson intervals (588 Erase and 147 Edit judgments per comparison): Erase 93 [91, 95] / 91 [88, 93] / 90 [87, 92] / 86 [83, 89] / 66 [62, 70]%; Edit 90.5 [85, 94] / 88.4 [82, 93] / 79.6 [72, 85] / 73.5 [66, 80] / 55.8 [48, 64]%. StableFlow is the closest competitor and not statistically separable on Edit, as now clarified in Section 5.3 (L425–429).

Question 2, Part 2

Question 2, Part 2: “… only 10 samples for the overall study and three prompt-editing samples, so the evidence remains limited.”

Thank you for raising this point. While we were designing the study, we had two directions:

  1. To include many more samples, making the study longer so that fewer participants complete it.
  2. To keep a compact, carefully selected set, so that more participants answer every question.

We chose the second option: each participant already answers 30 judgment sets in Study 1 and 75 pairwise comparisons in Study 2, and in the longer video part only 25 of 49 completed all responses. However, we totally agree that three prompt-editing samples is a small set, and we have noted this in the appendix.

We believe every point is answered as asked, and are glad to address anything that remains.

AuthorsOpenReview ↗

Following up - Reviewer 9tUZ

Dear Reviewer 9tUZ,

Thank you for your suggestions regarding our paper. We have done our best to address the points you've suggested. We look forward to further input during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.