- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CATD-MoE, a conflict-aware dynamic K/V routing framework for instruction-guided image editing. The method addresses the trade-off between editing flexibility and source-image preservation by dynamically mixing source and target K/V representations across denoising timesteps and Transformer block groups. A lightweight router is trained using offline counterfactual candidates and Pareto-based supervision, while the pretrained diffusion backbone remains frozen. Experiments on PIE-Bench demonstrate promising preservation performance while maintaining competitive editing quality.
Critical strengths
The paper addresses an important trade-off between editing quality and source-image preservation. Dynamically adjusting the contributions of source and target K/V representations is an intuitive approach to improving this balance.
The proposed method introduces a small routing network without requiring retraining of the diffusion backbone. Incorporating denoising progress, mask information, and source-target K/V discrepancies provides a potentially flexible mechanism for controlling the editing process.
Critical weaknesses
Although the motivation is intuitive, the paper does not sufficiently demonstrate that the identified conflict between source preservation and target editing is a significant limitation of existing methods. Figure 1 shows attention responses across denoising timesteps, but it does not directly establish how these observations affect editing performance or why adaptive K/V mixing is necessary. While the ablation results provide indirect evidence, a more systematic analysis of the relationship between K/V conflict, denoising progress, and editing errors would strengthen the motivation.
Equation (1) introduces several routing features, but their exact computation is not fully explained. Although Section 3.3 provides definitions for selected K/V discrepancy statistics and Table B.2 lists the router inputs, the complete construction of the conflict representation remains insufficiently specified. In particular, the aggregation and normalization of the different features are not entirely clear, which raises concerns about reproducibility.
The counterfactual candidate generation procedure assumes that each training sample contains an editing mask . However, the paper does not clearly explain how these masks are obtained for MagicBrush, which is described as containing source images, editing instructions, and edited target images. Since the masks are used in router features, conflict computation, and candidate evaluation, their construction is an important part of the proposed training pipeline.
Section 3.5.1 divides denoising into Early, Middle, and Late stages without specifying the exact boundaries or the criterion used for this partition. Furthermore, Algorithm 2 defines the supervision target as , which does not explicitly depend on the block-group index . It is therefore unclear how distinct supervision targets are constructed for different block groups, despite fine-grained timestep-by-block-group routing being a central contribution.
Equation (10) assigns fixed weights of , , and to the ranking, smoothness, and preservation-budget losses. Although Appendix A.2 explains the individual loss terms, the paper does not provide a clear justification or sensitivity analysis for these coefficients. It remains unclear how robust the method is to alternative choices.
The proposed training procedure requires generating and evaluating multiple counterfactual editing candidates, potentially introducing substantial offline computational overhead. However, the paper does not provide sufficient quantitative information about candidate generation time, total training cost, or additional memory requirements. Moreover, Algorithm 1 constructs a Pareto frontier but subsequently selects the optimal trajectory using a weighted-reward maximization. The specific benefit of Pareto-frontier construction over directly maximizing the weighted reward is therefore unclear.
Questions for authors
Could the authors provide more direct empirical evidence that fixed K/V mixing negatively affects editing performance across different denoising timesteps? For example, it would be helpful to analyze the relationship between source-target K/V conflict, optimal mixing coefficients, and editing errors.
Could the authors clarify how each routing feature in Equation (1) is computed? In particular, could they provide a complete definition of the conflict representation function , including feature aggregation, normalization, and the construction of the final router input?
How are the editing masks used in counterfactual candidate generation obtained from MagicBrush? Are they manually annotated, automatically generated, or estimated from source-target image differences? If masks are estimated, how sensitive is the method to their accuracy?
How exactly are the Early, Middle, and Late denoising stages defined? Additionally, how are block-group-specific supervision targets constructed in Algorithm 2, given that the stated target interpolation depends explicitly on timestep but not on block group?
How were the loss weights in Equation (10) selected? Could the authors provide a sensitivity analysis to demonstrate whether the reported performance is robust to different weight configurations?
What is the exact role of the Pareto frontier in selecting the optimal routing trajectories? Could the authors compare Pareto-based supervision with direct weighted-reward selection under the same candidate budget? Additionally, reporting the number of candidates per sample, offline candidate-generation time, total training time, GPU-hours, and end-to-end inference overhead would help clarify the practical efficiency of the method.
4 comments
W4. Early/Middle/Late boundaries and group-specific supervision
The Early, Middle and Late labels refer to offline curve control points at normalized progress , and ; consecutive points are joined by piecewise-linear interpolation. They are not three logits produced by the online Router. During inference, the Router produces one coefficient for each executed denoising update and routing group .
The counterfactual generator evaluates seven candidate curves separately for each of six groups. For a selected group , the candidate controls that group while the other five groups use . For every training sample and preservation preference , the highest-reward curve index is , with runner-up . The group-specific target is
Algorithms 1 and 2 now explicitly carry the sample, group and budget indices, so distinct groups receive distinct supervised targets.
W5. Lack of loss-weight sensitivity analysis
The training objective remains
We added a one-factor-at-a-time sensitivity study: each auxiliary weight is varied while the other two stay at the default values.
Table R3-3. Loss-weight sensitivity on PIE-Bench (Appendix B).
| Setting | Structure ×10³ ↓ | PSNR (dB) ↑ | LPIPS ×10³ ↓ | Edit CLIP ↑ |
|---|---|---|---|---|
| Rank = 0.10 | 36.84 | 34.56 | 53.90 | 22.88 |
| Rank = 0.20 | 36.52 | 34.71 | 53.74 | 23.01 |
| Rank = 0.30 | 36.69 | 34.59 | 53.86 | 22.94 |
| Smooth = 0.025 | 36.77 | 34.63 | 53.92 | 22.93 |
| Smooth = 0.05 | 36.52 | 34.71 | 53.74 | 23.01 |
| Smooth = 0.10 | 36.73 | 34.61 | 53.87 | 22.90 |
| Budget = 0.025 | 37.01 | 34.50 | 54.04 | 22.76 |
| Budget = 0.05 | 36.52 | 34.71 | 53.74 | 23.01 |
| Budget = 0.10 | 36.76 | 34.59 | 53.88 | 22.87 |
The default configuration gives the highest PSNR and Edit CLIP in these tested variations. The sensitivity ranges are narrow: for example, varying the ranking coefficient from 0.10 to 0.30 changes PSNR within 34.56–34.71 dB and Edit CLIP within 22.88–23.01. The regression term remains the primary training target; ranking and the two bounded regularizers serve auxiliary roles. These results demonstrate local stability around the reported coefficients, rather than claiming invariance to arbitrary weight choices.
W6. Counterfactual cost and Pareto-frontier role
The exact design consists of seven candidate temporal curves and six routing groups, yielding single-group candidate-generation jobs per sample. Each of the three preservation budgets reuses these generated candidates, yielding group–budget label records rather than tripling generation cost. A separate coarse configuration has seven global candidate jobs. The fine-grained design is thus a group-wise intervention scheme, not an exhaustive joint search.
The Pareto-nondominated set is recorded to describe editing–preservation–boundary–cost trade-offs. The implemented winner and runner-up training labels are selected by scalarized reward ranking across the seven candidates for each :
\qquad j^*_{x,g,\rho}=\arg\max_j R_{x,g,j}(\rho).$$ Accordingly, **Pareto membership is descriptive, not a separate filtering stage**, and we do not attribute an additional label-selection gain to a Pareto filter that the implementation does not use. The Appendix documents candidate counts and Router arithmetic; the matched **48.51 s / 39.48 GB** end-to-end execution and the **800 ms / 480 MB** added component costs are reported in its efficiency tables.Q1. Direct empirical evidence for time-varying conflict and editing error
Tables R3-1 and R3-2 above provide direct sample-level and stage-wise evidence. The conflict–LPIPS Pearson correlation changes from +0.31 (Early) to +0.42 (Middle) and +0.28 (Late), while PSNR correlates −0.41 with discrepancy over samples. Controlled ablations agree: fixed K/V obtains 33.09 PSNR / 22.16 Edit CLIP, three-stage routing obtains 33.74 / 22.85, and full timestep-by-group routing obtains 34.71 / 23.01. These results show that the relationship between K/V state discrepancy and fidelity is not constant across denoising and that finer adaptive control improves the output trade-off.
Q2. Complete feature computation and normalization
The 21 inputs and their aggregation axes are specified in W2 and in Appendix B, Feature Aggregation and Normalization. The conflict component uses masked-token Key/Value cosine similarity, normalized L1 source–target discrepancies, four Key/Value norms and a source-region/background gap. Mask geometry, temporal/depth coordinates, pooled text moments and complete the vector. Per-feature means and standard deviations are computed on training states and reused unchanged at inference; near-zero standard deviations are replaced by one as shown above. No target edited image is supplied to the inference Router.
Q3. Mask source and mask-accuracy dependence
The masks are read from mask-provided data records in the training manifest; mask-free records are not used for masked Router supervision. The implementation does not generate its masks through source–target difference thresholding. The preprocessing is deterministic: binarize the image-space mask, align spatially with the source and map it to packed latent tokens. Thus the method uses an explicit externally provided editing-region mask at inference as well.For more details about the mask, see W3.
Q4. Exact temporal definitions and distinct group targets
The three offline control points are , linked by piecewise-linear interpolation. They do not specify three fixed online intervals. The six Double-Block groups follow the contiguous 3+3+3+3+3+4 partition of the 19 Double Blocks. Candidate rewards are ranked separately for each , so the interpolated targets explicitly depend on . At inference, only one scalar is evaluated and cached for each group at each denoising update.
Q5. Selection and robustness of loss weights
We retain weights 0.20 / 0.05 / 0.05 for ranking, smoothness and budget consistency, with Huber regression as the principal target. Table R3-3 reports all nine single-factor configurations. At the default values, the method obtains 34.71 PSNR and 23.01 Edit CLIP, compared with 34.56–34.59 / 22.88–22.94 for the other tested ranking weights, 34.61–34.63 / 22.90–22.93 for the tested smoothness alternatives, and 34.50–34.59 / 22.76–22.87 for the tested budget alternatives. This documents robustness within the explicitly tested neighborhoods of the default coefficients.
Q6. Pareto, candidate budget, training cost and inference overhead
For each sample, 42 fine-grained single-group candidates are evaluated, with 18 group–budget label records obtained by rescoring the same outputs for three budgets; seven additional global policies belong to the separate coarse variant. The winner is chosen by the scalar reward above, with Pareto membership recorded for analysis. Consequently, a Pareto-filtered versus unfiltered winner-selection comparison does not describe two distinct label-selection rules in the current implementation; the precise algorithmic distinction is stated in W6. End-to-end inference efficiency is measured under matched hardware and schedule: fixed K/V 47.71 s / 39.00 GB versus CATD-MoE 48.51 s / 39.48 GB, with the conflict/Router/fusion breakdown in Table R2-2. This separates one-time offline supervision generation from per-image online inference cost.
Dear Reviewer cRUn:
Thank you again for your careful reading of our method. We hope you will have a chance to read our response, particularly the discussion of how Algorithm 2 constructs a supervision target for each block group. We would be grateful if you would consider these details when revisiting your initial rating.
Kind regards,
The Authors
Response to Reviewer cRUn (Reviewer 3)
We thank the reviewer for the detailed technical questions. We respond separately to the six weaknesses and six questions, with the most relevant new tables reproduced below for direct inspection.
W1. Insufficient evidence linking K/V conflict to editing errors
We extend the motivation beyond the attention visualization with sample-level correlation analysis, denoising-stage statistics and controlled routing ablations.
Table R3-1. Sample-level source–target K/V discrepancy correlations on PIE-Bench (Appendix B).
Higher discrepancy is associated with worse non-edited-region preservation (negative PSNR and positive LPIPS correlations). The weaker positive correlation with Edit CLIP also indicates that some representation change accompanies useful semantic editing; discrepancy should not simply be minimized at every step.
Table R3-2. Denoising-stage conflict analysis (Appendix B).
The conflict–error association varies over denoising and is largest in the middle stage. Consistently, removing the conflict feature reduces PSNR/Edit CLIP from 34.71/23.01 to 32.97/22.58, while removing temporal routing yields 32.64/22.50. This connects conflict statistics and temporal adaptation to measurable output errors rather than relying solely on Figure 1.
W2. Incomplete definition of the 21-dimensional Router input
The revised Appendix B, Feature Aggregation and Normalization, gives the complete partition 21=3+5+9+3+1:
Cosine similarities are computed over feature channels for masked tokens and averaged across tokens and heads. Relative L1 difference is normalized by the corresponding source magnitude; each norm is a channel-wise L2 norm averaged over heads and masked tokens. The region/background gap summarizes differences between source K/V means inside and outside the edited region. All features use training-state normalization,
s_k=\begin{cases}\sigma_k,&\sigma_k\ge 10^{-6}, \\\\ 1, &\sigma_k<10^{-6}.\end{cases}$$ The training statistics are stored in the Router checkpoint and reused during inference. The implemented input does not include a separate attention-entropy feature. ## W3. MagicBrush mask provenance The mask-conditioned training pipeline reads an explicit editing mask from its **data manifest**, rather than estimating one from source–target pixel differences during candidate generation. MagicBrush has mask-provided and mask-free settings; only records with valid masks enter the masked routing pipeline. The Appendix describes image-space binarization at threshold 127, spatial alignment with the source, latent resizing and token packing, as well as exclusion of empty/full masks. The edited target is used for **offline candidate scoring**, not as an input to the inference Router. This specifies what annotations the training procedure consumes and prevents any ambiguity about target-image access during inference.