acceptodds
Under review as a conference paper at ICLR 2027

PEAT: Progressive Editing via ASCII Art Transforms for Rendering in Pixels

Abstract

ASCII art represents object shapes and tones through spatial arrangements of characters. We introduce PEAT (Progressive Editing via ASCII Art Transforms), which uses ASCII art as a visual bottleneck for multi-turn image editing. This representation provides a discrete, inspectable editing state while keeping the original image as a separate appearance reference. At each turn, a large language model (LLM) updates the ASCII art and scene prompt from a user instruction. After the final turn, an image editing model synthesizes an RGB image from the updated state, the aggregated edit instruction, and the original image. Across four turns, PEAT shows the smallest decline in CLIP-I and AugCLIP among six editors and achieves the highest fourth-turn AugCLIP, MANIQA, and MUSIQ scores. Over a four-turn sequence, it uses 60.2% less inference time and 90.9% less GPU energy than sequential Qwen-Edit.

Primary area:
generative models
Keywords:
Instruction-Guided Image Editing, ASCII Art, Visual Bottleneck, Large Language Models, Controllable Image Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_euMHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces PEAT (Progressive Editing via ASCII Art Transforms), a multi-turn image editing framework that uses ASCII art as a discrete, inspectable visual bottleneck. At each turn, an LLM updates the ASCII art and scene prompt from a user instruction, while the original image is retained as a separate appearance reference. After the final turn, an image editing model synthesizes the RGB output from the updated ASCII state, aggregated instruction, and original image. Experiments report smaller CLIP-I/AugCLIP degradation over four turns, higher fourth-turn AugCLIP/MANIQA/MUSIQ than six editors, and substantial inference-time and GPU-energy savings over sequential Qwen-Edit.

Critical strengths
  1. The use of ASCII art as an intermediate editing state is novel and interpretable, providing a discrete, inspectable representation for multi-turn editing.

  2. Separating the structural/editing state from the original appearance reference is a reasonable design for preserving identity and appearance while allowing semantic edits.

  3. The efficiency results are promising, with reported reductions in inference time and GPU energy relative to sequential Qwen-Edit.

  4. The multi-turn evaluation and comparison with six editors provide useful initial evidence for the proposed paradigm.

Critical weaknesses
  1. The ASCII bottleneck has limited information capacity. ASCII art captures only coarse shape and tone, and may fail to preserve color, texture, fine geometry, and complex spatial relations. The paper should analyze when this bottleneck helps versus when it causes semantic loss, especially in crowded scenes, small objects, and precise spatial instructions.

  2. Evaluation emphasizes perceptual quality rather than edit correctness. CLIP-I, AugCLIP, MANIQA, and MUSIQ do not directly measure whether the requested edits are correctly performed or whether unrelated regions are preserved. Instruction-following accuracy, region-level edit success, identity preservation, and human evaluation are needed.

  3. The multi-turn protocol and efficiency comparison are insufficiently justified. It is unclear how the four turns are constructed, whether instructions are dependent, how errors accumulate, and whether the comparison with sequential Qwen-Edit uses matched hardware, batch size, LLM calls, and final synthesis cost. Full-pipeline efficiency accounting is needed.

  4. The reliability of LLM-generated ASCII states is not validated. LLMs may produce inconsistent or malformed ASCII updates, and the final editor must correctly interpret the ASCII state together with the original image. The paper should report ASCII update accuracy, robustness to LLM errors, and ablations separating the ASCII state, scene prompt, and original-image reference.

Questions for authors

Please refer to the weaknesses.

11 comments

Thank you for the constructive review. We appreciate your recognition of ASCII art as a discrete, inspectable intermediate editing state, and of the separation between the editing state and the original appearance reference. We begin with W2 below.

The additional experiments requested in the reviews have now finished running. We are compiling the results and will post each response as soon as it is ready.

AuthorsOpenReview ↗

W2. Evaluation of edit correctness

Thank you for the valuable comment. To evaluate edit correctness, we added a VLM-based evaluation with GPT-5.4 mini as the judge. We evaluated all methods on 110 source images from three benchmarks with four turns each, which is 440 turns per method (Table R1, Table R3). The judge labels each case as success, partial, failure, or unjudgeable, and we count only success as success.

Instruction-following accuracy. Edit SR is the ratio of turns where the current instruction is correctly performed, compared with the previous image. The overall Edit SR of PEAT is 85.8 / 87.5 / 90.0% on PIE / HAOC / CelebA-HQ, and it is at least 80% at every turn on all benchmarks. At turn 4, PEAT gets 83.3 / 82.0 / 83.3% and sequential Qwen-Edit gets 80.0 / 82.0 / 76.7%.

Preservation of unrelated regions. Cum. SR counts a turn as success only when all active instructions are satisfied and the unrequested content (background, composition, style, subject appearance) is preserved compared with the original image. At turn 4, PEAT gets 83.3 / 76.0 / 83.3% and sequential Qwen-Edit gets 73.3 / 78.0 / 63.3%. PEAT is 10.0 and 20.0 points higher on PIE and CelebA-HQ, and similar on HAOC. On PIE and CelebA-HQ, PEAT is higher than the four open-weight baselines and is second after Nano Banana. The difference between Edit SR and Cum. SR of PEAT at turn 4 is 0 / 6 / 0 points.

Identity preservation. CelebA-HQ is a face benchmark, and the Cum. SR criterion includes subject identity. At turn 4, the Cum. SR of PEAT is 83.3%, and the four open-weight baselines get 63.3–80.0%. PEAT also shows the smallest CLIP-I drop from turn 1 to turn 4 on CelebA-HQ (PEAT −0.056, Qwen-Edit −0.078, Nano Banana −0.093; Table R2).

Image-level metrics. On all three benchmarks, PEAT shows the smallest CLIP-I drop from turn 1 to turn 4 and the highest MUSIQ at turn 4 (Table R2).

AuthorsOpenReview ↗

W2. Table R1.

Table R1. Evaluation set (110 source images × 4 turns = 440 evaluated turns per method)

BenchmarkSource imagesEdit turns per imageEvaluated turns per method
PIE-Bench304120
HAOC504200
CelebA-HQ304120
Total1104440
Reviewer_PUWfOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PEAT proposes using ASCII art as an intermediate state to keep track of accumulated edits in multi-turn image editing. They fine-tune an LLM (Qwen2.5-3B) to generate and update the ASCII art and a scene prompt at each turn, and only at the end generate the final image with Qwen-Image-Edit + ControlNet, using the original image as reference. They compare against several baselines, including commercial models like Nano Banana, and show the smallest drop in CLIP-I and AugCLIP over 4 turns and the best final-turn scores on most metrics (Nano Banana is still best on CLIP-I).

Critical strengths
  1. Novelty and insight. I believe the idea is novel. Using ASCII art as an intermediate state to keep track of accumulated edits is a new way to approach multi-turn image editing. It also gives an interesting insight: LLMs are not designed to generate images, but with the right representation (ASCII art) they can produce and edit a coarse version of the image, and this coarse ASCII state is good enough to guide the final image editing model (Qwen-Edit with ControlNet). I think this insight could be useful beyond image editing and could push these models toward further applications. This is the main reason I lean toward acceptance.

  2. Representation and implementation. I like how they build the ASCII art representation. It is a standard and to-the-point design, from the frequency-based codebook to the patch mapping. The implementation is also described in depth (codebook, token initialization, training setup, data filtering), which I found insightful.

Critical weaknesses
  1. The evaluation set is small. Only 25 source images (25 four-turn sequences) are used for the multi-turn editing evaluation.

  2. My main concern is how well the LLM has learned to generate ASCII art in general, on more samples and in new scenarios. Stage-2 editing is trained only on human, human-object and human-animal scenes (Table 5) with five edit types, and the evaluation uses 25 sequences from the same distribution. So we don't know how robust it is to new scenes or new kinds of instructions.

  3. I don't think the current metrics (CLIP-I, CLIP-T, AugCLIP, MANIQA, MUSIQ) capture edit quality well. They are global scores, so they can't reliably show whether each edit was actually done and whether earlier edits were kept. A VLM-based judgment would help.

  4. It is not clear which edits PEAT is actually suitable for. The ASCII state only captures the shape and tone of the foreground, so global changes (style, color, background, etc.), and especially several global changes in one sequence, cannot be represented in it. The paper says these are handled by the scene prompt and the final editor. But by that argument, any edit can be applied through the prompt alone. The paper should make a clear distinction between the edits where the ASCII state really helps and the edits where the prompt alone is enough.

Questions for authors

Clarifications:

  1. After Stage 1, how well does the LLM generalize in generating ASCII art? Is it limited to the scope of the training data, or can it go beyond that scope thanks to the knowledge the LLM already has from pretraining? (see Suggestion 4)

  2. Can you explain the 60.2% less inference time and 90.9% less GPU energy in more detail? Table 14 suggests the per-edit numbers are the total sequence cost divided by 4, and the paper says GPU energy excludes the GPT-4o call. Is the time of the LLM updates and the GPT-4o call included? A breakdown of time and energy for each part (LLM updates, GPT-4o, final synthesis) would help.

  3. In Table 3, Sequential RGB and Prompt-only have exactly the same MUSIQ (73.324). Is this a copy error? PEAT's MUSIQ advantage over prompt-only depends on it.

Suggestions:

  1. Could you test the LLM on at least 100 samples, including new scenarios not seen in training, and ask a VLM to score how faithful the generated ASCII art is to the corresponding image?

  2. Qwen-Image-Edit-2511 and FLUX.2 [klein] 9B accept multiple input images. I am curious how they perform if you give them the source image, the grayscale ASCII rendering, and a text describing the edit.

  3. Could you show sequences with multiple global changes (e.g., style, then background, then lighting)?

  4. I would like to see more multi-turn editing examples in the supplementary.

9 comments

Thank you for your thoughtful review and suggestions. We are glad that you found the idea of using ASCII art as an intermediate state novel. Following your suggestions, we enlarged the evaluation set and added a VLM-based evaluation. We will include the additional experiments in the revised paper.

AuthorsOpenReview ↗

W1. The evaluation set is small.

To address the small size of the evaluation set, we enlarged it and ran the multi-turn evaluation again.

The new evaluation set has 110 source images from three benchmarks. It has 50 human–animal–object composition scenes from HAOC, 30 images from PIE-Bench, and 30 face images from CelebA-HQ. Each image is edited for four turns, so each method is evaluated on 440 turns. All six methods use the same sequences. Table R1 summarizes the set.

Table R1. Evaluation set (110 source images × 4 turns = 440 evaluated turns per method)

BenchmarkSource imagesEdit turns per imageEvaluated turns per method
PIE-Bench304120
HAOC504200
CelebA-HQ304120
Total1104440
AuthorsOpenReview ↗

W3. VLM-based judgment.

As you suggested, we added a VLM-based evaluation with GPT-5.4 mini as the judge on all 440 turns of each method (Table R3). The judge sees the original, previous and current images with the instructions up to the current turn. It gives one of success, partial, failure and unjudgeable, and we count only success.

Whether each edit was done: Edit SR checks the current instruction against the previous image. The Edit SR of PEAT is at least 80% at every turn on all three benchmarks, which is comparable to the sequential RGB editors.

Whether earlier edits were kept: Cum. SR requires that all instructions so far are satisfied and the unrequested content is preserved. At turn 4, PEAT is 10.0 points higher than sequential Qwen-Edit on PIE and 20.0 points higher on CelebA-HQ, and similar on HAOC. On PIE and CelebA-HQ, PEAT is higher than all four open-weight baselines and second after Nano Banana. We think this is because PEAT synthesizes the RGB image only once from the original image.

This is consistent with Table R2, where PEAT has the smallest CLIP-I drop over turns on all three benchmarks.

Reviewer_hsC7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PEAT maintains multi-turn image-editing state as a tone-based ASCII grid and a scene description. A LoRA-adapted Qwen2.5-3B updates a 1,024-token patch representation and the description at each turn. Final synthesis combines the source image, final description, GPT-4o-aggregated edit history, and an ASCII-conditioned ControlNet with a Qwen-Edit backbone. Experiments compare four-turn editing on 25 synthetic source images, rate PEAT's intermediate states, and evaluate extracted spatial representations in a separate SDXL reconstruction setup.

Critical strengths
  1. Separating an inspectable spatial state from the source appearance reference is a useful design. The grid renders directly without a learned image decoder, and the patch vocabulary connects this view to an LLM-editable sequence. The qualitative sequences demonstrate a plausible interactive representation, with shape and layout retained alongside a semantic description.

  2. The paper includes the important prompt-only synthesis baseline and acknowledges that deferring synthesis accounts for much of the preservation gain. It also separates per-edit and complete-task costs and explicitly notes that PEAT takes longer than prompt-only synthesis. The SDXL condition study uses matched training sets and sampling settings within that reconstruction experiment.

Critical weaknesses
  1. The independent benefit of the ASCII editing state is not yet established by the editing experiments. In Table 3, prompt-only slightly exceeds PEAT in CLIP-I/AugCLIP (0.847/0.882 versus 0.846/0.881); PEAT's advantages in CLIP-T and MUSIQ lack paired uncertainty or comparative final-RGB human evaluation. PEAT also updates a scene description and uses a trained ControlNet, so the comparison does not clearly isolate the ASCII state from the other conditioning changes. The box/mask/edge study reconstructs images from extracted conditions with SDXL rather than testing those states in the same multi-turn editing pipeline. Thus it supports reconstruction utility under that setup, but not the stronger causal conclusion about ASCII as an editing state.

  2. The limited evaluation and missing main-pipeline details make the reliability and cost claims difficult to assess. The editing test has only 25 synthetic four-turn sequences. The human study rates PEAT's own states and final-state consistency, rather than comparing final cumulative instruction satisfaction across methods. Appendix B.3 terminates the remaining turns after an invalid output, but does not explain the failure frequency or its treatment in all reported denominators. The main ControlNet training and Qwen-Edit synthesis settings are also underspecified: Table 9 leaves the GPT-4o snapshot unresolved, and Appendix C.4 introduces the cost table without the hardware, sampling, precision, and measurement details needed to judge comparability. The SDXL settings in Table 12 belong to a separate representation experiment.

Overall assessment. I lean toward weak rejection. The representation is interesting and the state/synthesis separation is sensible, but the central additional benefit of the ASCII state remains weakly isolated and validated. The evaluation and missing main-pipeline protocol currently make the evidence insufficient for the strength of the multi-turn editing and cost conclusions. A matched ablation and clearer accounting could materially change this assessment.

Questions for authors
  1. Can you clarify whether prompt-only and PEAT use matched final scene descriptions, instruction aggregation, synthesis checkpoints, and sampling settings? A text-state-only ablation, paired uncertainty on the existing sequences, or comparative final-RGB judgments would help establish the additional value of ASCII beyond one-shot synthesis and updated text.

  2. Are data splits grouped by the complete source scene and its edited descendants, including reverse augmentations? How many evaluation sequences terminate early, and how are failed and unexecuted turns included in the fixed 25-sequence results and human study?

  3. Please specify the main ControlNet training data/schedule and the final synthesis checkpoint, resolution, sampler, steps, guidance, precision, and hardware for each cost comparison. Can final RGB synthesis and state-update costs be reported separately, explaining the measured 6.61 Wh total relative to the 18.6 Wh prompt-only call? The question concerns missing configuration and measurement information, not an assumed arithmetic error.

13 comments

Thank you for the careful review. We’re glad you found it useful to keep the spatial state separate from the source image. Below, we address your questions about the evaluation scale and how we measured cost.

We’ve finished the additional experiments and are putting together the remaining replies. We’ll post them as soon as they’re ready. If the discussion closes first, we’ll include the remaining results in the revised paper.

AuthorsOpenReview ↗

W2-Q2,Q3. The limited evaluation and missing main-pipeline details

  1. Evaluation scale. We now evaluate on 110 source images: 50 from HAOC, 30 from PIE-Bench and 30 from CelebA-HQ. Each image is edited for four turns, so each method has 440 turns (Table R1). All six methods use the same images and instructions. The results are in Table R2 and Table R3.

  2. Cumulative instruction satisfaction across methods. We added a VLM-based evaluation with GPT-5.4 mini as the judge. Cum. SR counts a turn as success only when all instructions up to that turn are satisfied and the unrequested content is preserved. At turn 4, PEAT is 10.0 points higher than sequential Qwen-Edit on PIE and 20.0 points higher on CelebA-HQ, and the two are similar on HAOC.

  3. Data split and early termination (Q2). The data are split by source scene. We separate the test scenes before building the training data of Stage 1 and Stage 2, and the edited descendants of each test scene, including the reverse augmentations, are held out together. HAOC was built in the same way. No sequence was terminated early, so all turns are included in the results and in the human study.

  4. Cost (Q3). The 6.61 Wh in the submission was measured on an A100 80GB and counted only the four state updates. It did not include the final RGB synthesis, so it was smaller than the 18.6 Wh of one prompt-only call. The time had the same omission. We corrected this and measured all methods again on an RTX PRO 6000 Blackwell Server Edition GPU. Now one state update takes 1.26 Wh and about 21 seconds, and the final RGB synthesis takes 13.53 Wh and about 88 seconds. For four turns, PEAT takes 18.57 Wh and 174.7 seconds, and sequential Qwen-Edit takes 48.95 Wh and 303.0 seconds (Table R4, Table R5). The GPT calls are API calls, so they are included in the time but not in the GPU energy. We will replace the cost table in the revised paper.

  5. Measurement settings (Q3). We measure the energy with NVIDIA NVML through pynvml, using the difference of the cumulative energy counter between the start and the end of inference after CUDA synchronization. Model loading, warm-up and file saving are excluded, and only the GPU board energy is counted. All methods run on one GPU with batch size 1 and BF16, without quantization, CPU offload or torch.compile. The baselines use 1024×1024, 50 steps and true CFG 4.0 (Table R6, Table R7, Table R8).

  6. ControlNet (Q3). The final synthesis uses Qwen/Qwen-Image-Edit with a fixed revision. We trained only a newly initialized block-wise ControlNet with 60 layers, and the Qwen DiT, VAE and text encoder are frozen. The ControlNet takes the ASCII rendering as its condition, and the loss is the flow matching loss (Table R10). We will add the ControlNet training data and schedule and the GPT snapshot to the appendix of the revised paper.

Table R1. Evaluation set (110 source images × 4 turns = 440 evaluated turns per method)

BenchmarkSource imagesEdit turns per imageEvaluated turns per method
PIE-Bench304120
HAOC504200
CelebA-HQ304120
Total1104440

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.