acceptodds
Under review as a conference paper at ICLR 2027

SparseEdit: Stage-Aware Sparse Inference for DiT-Based Image Editing

Abstract

Instruction-based image editing with Diffusion Transformers repeatedly processes generated-latent and reference-image latent tokens at every denoising step, leading to substantial inference cost. We study how to organize sparse update patches over denoising under an explicit per-step update budget. A diagnostic based on velocity differences computed from full forward passes shows a transition from larger and more connected update patches early in denoising to increasingly fragmented masks later. This observation motivates SparseEdit, a training-free sparse inference method centered on stage-dependent spatial selection. Its Patch Selector shifts from priorities averaged over neighborhoods to pointwise priorities as denoising progresses while maintaining an exact update budget. A short dense warmup assigns request-level update and velocity-reuse budgets, and an Execution Controller realizes the selected updates through sparse computation, dense refreshes, and velocity reuse. Across GEdit-Bench and KontextBench, SparseEdit achieves – end-to-end speedups on FLUX.1 Kontext Dev, LongCat-Image-Edit, and Qwen-Image-Edit, with PSNR – dB, SSIM –, and LPIPS – relative to dense outputs.

Primary area:
generative models
Keywords:
Diffusion Transformers, Image Editing, Efficient Inference, Spatial Sparsity, Training-free Acceleration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8mtgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SparseEdit, a training-free method to accelerate DiT-based image editing by updating only a subset of tokens at each denoising step. Motivated by an oracle analysis showing that update regions are clustered early and become fragmented later, it selects tokens with a locality prior that weakens over denoising. It also uses a short dense warmup to set per-request budgets and switches among dense refresh, sparse computation, and velocity reuse at each step. Across three editors and two benchmarks, it achieves 1.83–2.56× speedups while staying close to the dense outputs.

Critical strengths
  1. The paper correctly identifies the cost structure specific to editing. The observation that reference tokens double the sequence length, and that the speedup of region-aware methods varies with the edited area, is well motivated. Assigning an explicit per-request update budget to secure consistent acceleration even for global edits is a practical design choice.

  2. The ablation directly targets the core claim. Table 2 fixes the speedup at about 2.24× and compares fixed locality (LPIPS 0.064) and fixed pointwise selection (0.058) against stage-aware selection (0.037). Since the computation is controlled and only the selection rule changes, the comparison is clean.

  3. The oracle analysis provides an intuitive motivation. The result that an oracle updating only 15% of tokens nearly reproduces the dense output (LPIPS 0.010) is a meaningful diagnostic that shows the upper bound of spatial sparsity in editing.

Critical weaknesses
  1. The analysis is insufficient. Apart from the observation in Figure 1 that the tokens selected by the top-15% oracle become more dispersed in later denoising steps, there is little analysis. Even this observation is based on a single representative example (four including the appendix) and is not compared against a random mask with the same budget. On a 64×64 grid, a random 15% mask has about 437 connected components, and the late-stage mean of 362 is close to this value. The fragmentation may therefore be a trivial consequence of velocity changes becoming flat in later steps, which makes top-K selection close to random. In addition, the fact that edited regions change substantially early while unedited regions stabilize early has already been analyzed from a trajectory perspective by RegionE. The rest of the paper is an engineering design of how to combine refresh, sparse computation, and velocity reuse, which makes the scientific insight offered by this work unclear.

  2. The novelty is limited. Sparse computation with cached K/V is the same as in RAS and ToCa, and the combination of reference-token caching and velocity reuse is nearly identical to RegionE. The budget-based velocity reuse is also structurally equivalent to TeaCache, which accumulates estimated changes and recomputes once a threshold is exceeded. The only new component is a rule that linearly interpolates between 5×5-averaged and pointwise scores according to the flow time. Moreover, although the paper targets image editing, it contains few components specific to editing. The core stage-aware selection relies only on velocity changes and could be applied directly to image generation, while editing information (the discrepancy from the reference) is used only as one term of the routing score.

  3. Editing quality itself is not evaluated. All metrics are PSNR, SSIM, and LPIPS relative to the dense output. Caching and reuse may yield outputs similar to the dense output, but this does not guarantee a successful edit. In particular, whole-image metrics are dominated by the large unedited regions, so failures in which the instruction is only partially applied within the edited region can be hidden by small numerical differences. Comparisons with the baselines on such editing-quality metrics are essential.

  4. There is no per-task analysis to support the claim against region-aware methods. The paper’s central argument is that region-aware methods lose acceleration on global edits, whereas SparseEdit maintains consistent speedup through its explicit budget. However, without a breakdown over the 11 tasks of GEdit-Bench and the 5 tasks of KontextBench, this claim cannot be verified. The background-editing and style-transfer examples in Figure 3 are insufficient, and a table reporting speedup and quality separately for local and global edits is needed.

Questions for authors
  1. Could you report the connected-component statistics of the oracle masks over the full benchmarks, together with a random-mask baseline under the same budget? In addition, could you show whether imposing a locality prior on the oracle actually improves quality, and how your analysis differs from the trajectory analysis in RegionE?

  2. Could you clarify which components are specific to image editing?

  3. Could you provide a comparison of editing quality across all methods, for example using GPT-4o-based evaluation, rather than only measuring fidelity to the dense outputs?

  4. Could you provide speedup and quality broken down by task, or at least separated into local and global edits, for SparseEdit and the region-aware baselines?

Reviewer_qf6kOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SparseEdit, a training-free sparse inference method for DiT image editing models. Through a velocity-difference oracle for full-scale inference, the paper finds that update regions in the early stages of denoising are more concentrated, while they gradually become more fragmented in later stages. Based on this observation, SparseEdit designs three components: Stage-Aware Patch Selector, Warmup-Based Policy Routing, and Execution Controller. The Stage-Aware Patch Selector interpolates between local neighborhood aggregation and token-wise selection depending on the denoising stage. Warmup-Based Policy Routing dynamically allocates computational budgets based on a brief full-range warm-up. The Execution Controller jointly performs sparse computations, velocity reuse, and periodic full-range refreshes. Experiments evaluated FLUX.1 Kontext Dev, LongCat-Image-Edit, and Qwen-Image-Edit on the GEdit-Bench and KontextBench datasets. The results show that, compared to the corresponding dense inference, the end-to-end acceleration ratio reaches 1.83–2.56×.

Critical strengths
  1. The contributions of this paper are clear, and the experiments are comprehensive. The method leverages both spatial sparsity and striding-time redundancy to achieve a good balance between speed and quality across three models and two benchmarks. It also compares the approach with various existing acceleration methods, demonstrating practical value.

  2. The authors separately validated the roles of phase-aware spatial selection, velocity reuse, and warmup routing. In particular, the ablation results comparing fixed local aggregation with fixed point-by-point selection support the design rationale for adjusting spatial selection strategies based on the denoising phase. The method design is clear, and the ablation experiments are comprehensive.

Critical weaknesses
  1. The core selector in the current paper primarily combines neighborhood average scores and per-token scores through linear interpolation, while also incorporating token age. The remaining components are also closely related to existing acceleration techniques such as token caching and cross-step reuse. The paper needs to more clearly demonstrate that the phase-aware mechanism provides new insights that go beyond the combination of existing techniques.

  2. The core validation in this paper remains insufficient. The argument for phased mask fragmentation is primarily based on a small number of visual examples, and the connectivity analysis of the online selector is also rather limited. Furthermore, quality metrics are all benchmarked against dense model outputs and primarily measure output fidelity. This does not directly prove that the ability to follow instructions is preserved in complex editing tasks.

Questions for authors

In addition to comparing SparseEdit with dense outputs using PSNR, SSIM, and LPIPS, have the authors evaluated SparseEdit’s command adherence success rate on tasks such as text editing or global style transfer? This experiment was needed to avoid situations where semantic errors occur after acceleration but the visual differences are minimal. Existing metrics may not adequately reflect this issue.

Reviewer_m1eFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SparseEdit is a training-free acceleration method for DiT-based image editing. Its selector combines historical velocity changes and token age, gradually shifting from neighborhood-averaged to pointwise priorities under a fixed token-update budget. Warmup routing selects request-level update and reuse budgets, while an execution controller combines sparse computation, dense refreshes, and scaled velocity reuse. Experiments on three editors and two benchmarks report 1.83–2.56× end-to-end speedups with improved fidelity to dense outputs relative to the tested baseline configurations.

Critical strengths
  1. Explicit spatial-selection design. The selector maintains a fixed token count at sparse steps, and equal-speed ablations show improved fidelity over fully local and fully pointwise selection.

  2. Evaluation across multiple editors. Results on three models and two benchmarks report favorable end-to-end quality–efficiency tradeoffs, with separate ablations of velocity reuse and request-level routing.

Critical weaknesses
  1. The central stage-aware design is insufficiently validated. The selector is compared only with fully local and fully pointwise scoring, without a tuned constant mixture, so the benefit of stage dependence itself remains unclear. The motivating connectivity analysis also relies on a few illustrated trajectories rather than dataset-level statistics.

  2. The baseline comparisons are insufficiently controlled and documented. Implementation, adaptation, and tuning details for RegionE and other baselines are not adequately specified, and matched-latency tradeoff curves are absent. Without selector comparisons under a shared execution framework, the gains cannot be clearly attributed to stage-aware selection rather than temporal reuse, budget settings, or implementation differences.

Questions for authors
  1. Does stage-aware selection outperform a tuned constant mixture of neighborhood-averaged and pointwise scores at the same budget?

  2. Does warmup routing outperform a fixed intermediate profile with the same average computation budget?

  3. What are the runtime and memory overheads of token scoring, Top-K selection, and cache indexing, particularly at higher resolutions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.