acceptodds
Under review as a conference paper at ICLR 2027

CATD-MoE: Conflict-Aware Timestep-Dynamic K/V Mixture-of-Experts for Instruction-guided Image Editing

Abstract

Instruction-guided image editing models based on diffusion Transformers generally face a trade-off between editing flexibility and source image fidelity. Existing approaches typically rely on fixed K/V injection or manually defined task labels, making them less capable of adapting to dynamic editing requirements across different denoising stages and network depths. To address this issue, we propose CATD-MoE, a Conflict-Aware Timestep-Dynamic K/V Mixture-of-Experts framework that requires no retraining of the diffusion backbone. Specifically, we model the source image K/V and target denoising-state K/V as two memory experts, while their continuous interpolation forms a controlled bridging region. A lightweight Router dynamically determines their relative contributions based on denoising progress, mask properties, K/V conflict, and text representations. All diffusion backbone parameters remain frozen, with only the routing network being trained. During training, counterfactual candidates are constructed offline using target images, and Pareto-optimal supervision is derived according to editing quality, source image fidelity, and boundary quality, enabling the model to learn dynamic K/V mixing ratios. Experimental results on the independent PIE-Bench benchmark show that CATD-MoE effectively improves non-edited region fidelity and reduces boundary errors while maintaining high semantic consistency with the editing instructions. Further fine-grained evaluations and ablation studies demonstrate that joint spatiotemporal adaptation and K/V conflict awareness are key contributors to the observed performance gains.

Primary area:
generative models
Keywords:
Diffusion Transformer, Image Editing, Conflict-Aware, Counterfactual Supervision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cRUnOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CATD-MoE, a conflict-aware dynamic K/V routing framework for instruction-guided image editing. The method addresses the trade-off between editing flexibility and source-image preservation by dynamically mixing source and target K/V representations across denoising timesteps and Transformer block groups. A lightweight router is trained using offline counterfactual candidates and Pareto-based supervision, while the pretrained diffusion backbone remains frozen. Experiments on PIE-Bench demonstrate promising preservation performance while maintaining competitive editing quality.

Critical strengths

1. Practical and relevant problem.\textbf{1. Practical and relevant problem.} The paper addresses an important trade-off between editing quality and source-image preservation. Dynamically adjusting the contributions of source and target K/V representations is an intuitive approach to improving this balance.

2. Lightweight and adaptive framework.\textbf{2. Lightweight and adaptive framework.} The proposed method introduces a small routing network without requiring retraining of the diffusion backbone. Incorporating denoising progress, mask information, and source-target K/V discrepancies provides a potentially flexible mechanism for controlling the editing process.

Critical weaknesses

1. Insufficient empirical justification of the motivation.\textbf{1. Insufficient empirical justification of the motivation.} Although the motivation is intuitive, the paper does not sufficiently demonstrate that the identified conflict between source preservation and target editing is a significant limitation of existing methods. Figure 1 shows attention responses across denoising timesteps, but it does not directly establish how these observations affect editing performance or why adaptive K/V mixing is necessary. While the ablation results provide indirect evidence, a more systematic analysis of the relationship between K/V conflict, denoising progress, and editing errors would strengthen the motivation.

2. Incomplete specification of feature extraction and conflict representation.\textbf{2. Incomplete specification of feature extraction and conflict representation.} Equation (1) introduces several routing features, but their exact computation is not fully explained. Although Section 3.3 provides definitions for selected K/V discrepancy statistics and Table B.2 lists the router inputs, the complete construction of the conflict representation ct=Φ(Kst,Vst,Ktt,Vtt,M)c^t = \Phi(K_s^t,V_s^t,K_t^t,V_t^t,M) remains insufficiently specified. In particular, the aggregation and normalization of the different features are not entirely clear, which raises concerns about reproducibility.

3. Unclear construction of training masks.\textbf{3. Unclear construction of training masks.} The counterfactual candidate generation procedure assumes that each training sample contains an editing mask MM. However, the paper does not clearly explain how these masks are obtained for MagicBrush, which is described as containing source images, editing instructions, and edited target images. Since the masks are used in router features, conflict computation, and candidate evaluation, their construction is an important part of the proposed training pipeline.

4. Ambiguity in temporal routing and fine-grained supervision.\textbf{4. Ambiguity in temporal routing and fine-grained supervision.} Section 3.5.1 divides denoising into Early, Middle, and Late stages without specifying the exact boundaries or the criterion used for this partition. Furthermore, Algorithm 2 defines the supervision target as yt,g=PiecewiseAlpha⁡(αj⋆(τ),t)y_{t,g} = \operatorname{PiecewiseAlpha}(\alpha_{j^\star}(\tau),t), which does not explicitly depend on the block-group index gg. It is therefore unclear how distinct supervision targets are constructed for different block groups, despite fine-grained timestep-by-block-group routing being a central contribution.

5. Insufficient justification of the training objective.\textbf{5. Insufficient justification of the training objective.} Equation (10) assigns fixed weights of 0.20.2, 0.050.05, and 0.050.05 to the ranking, smoothness, and preservation-budget losses. Although Appendix A.2 explains the individual loss terms, the paper does not provide a clear justification or sensitivity analysis for these coefficients. It remains unclear how robust the method is to alternative choices.

6. Unclear benefit and computational cost of Pareto-based supervision.\textbf{6. Unclear benefit and computational cost of Pareto-based supervision.} The proposed training procedure requires generating and evaluating multiple counterfactual editing candidates, potentially introducing substantial offline computational overhead. However, the paper does not provide sufficient quantitative information about candidate generation time, total training cost, or additional memory requirements. Moreover, Algorithm 1 constructs a Pareto frontier but subsequently selects the optimal trajectory using a weighted-reward maximization. The specific benefit of Pareto-frontier construction over directly maximizing the weighted reward is therefore unclear.

Questions for authors

1. Motivation and conflict analysis:\textbf{1. Motivation and conflict analysis:} Could the authors provide more direct empirical evidence that fixed K/V mixing negatively affects editing performance across different denoising timesteps? For example, it would be helpful to analyze the relationship between source-target K/V conflict, optimal mixing coefficients, and editing errors.

2. Feature extraction:\textbf{2. Feature extraction:} Could the authors clarify how each routing feature in Equation (1) is computed? In particular, could they provide a complete definition of the conflict representation function Φ\Phi, including feature aggregation, normalization, and the construction of the final router input?

3. Training masks:\textbf{3. Training masks:} How are the editing masks used in counterfactual candidate generation obtained from MagicBrush? Are they manually annotated, automatically generated, or estimated from source-target image differences? If masks are estimated, how sensitive is the method to their accuracy?

4. Routing granularity:\textbf{4. Routing granularity:} How exactly are the Early, Middle, and Late denoising stages defined? Additionally, how are block-group-specific supervision targets constructed in Algorithm 2, given that the stated target interpolation depends explicitly on timestep but not on block group?

5. Loss coefficients:\textbf{5. Loss coefficients:} How were the loss weights in Equation (10) selected? Could the authors provide a sensitivity analysis to demonstrate whether the reported performance is robust to different weight configurations?

6. Pareto supervision and computational efficiency:\textbf{6. Pareto supervision and computational efficiency:} What is the exact role of the Pareto frontier in selecting the optimal routing trajectories? Could the authors compare Pareto-based supervision with direct weighted-reward selection under the same candidate budget? Additionally, reporting the number of candidates per sample, offline candidate-generation time, total training time, GPU-hours, and end-to-end inference overhead would help clarify the practical efficiency of the method.

4 comments

Response to Reviewer cRUn (Reviewer 3)

We thank the reviewer for the detailed technical questions. We respond separately to the six weaknesses and six questions, with the most relevant new tables reproduced below for direct inspection.

W1. Insufficient evidence linking K/V conflict to editing errors

We extend the motivation beyond the attention visualization with sample-level correlation analysis, denoising-stage statistics and controlled routing ablations.

Table R3-1. Sample-level source–target K/V discrepancy correlations on PIE-Bench (Appendix B).

OutcomePearson rrSpearman ρs\rho_s
Structure Distance+0.32+0.30
Non-edited PSNR−0.41−0.39
Non-edited LPIPS+0.37+0.35
Edited-region CLIP+0.18+0.16

Higher discrepancy is associated with worse non-edited-region preservation (negative PSNR and positive LPIPS correlations). The weaker positive correlation with Edit CLIP also indicates that some representation change accompanies useful semantic editing; discrepancy should not simply be minimized at every step.

Table R3-2. Denoising-stage conflict analysis (Appendix B).

StageMean K/V conflictPearson rr with non-edited LPIPS
Early0.286+0.31
Middle0.241+0.42
Late0.193+0.28

The conflict–error association varies over denoising and is largest in the middle stage. Consistently, removing the conflict feature reduces PSNR/Edit CLIP from 34.71/23.01 to 32.97/22.58, while removing temporal routing yields 32.64/22.50. This connects conflict statistics and temporal adaptation to measurable output errors rather than relying solely on Figure 1.

W2. Incomplete definition of the 21-dimensional Router input

The revised Appendix B, Feature Aggregation and Normalization, gives the complete partition 21=3+5+9+3+121=3+5+9+3+1:

  • 3 temporal/depth features: normalized denoising progress, scheduler time coordinate and group-first-block depth.
  • 5 mask features: foreground-area ratio, bounding-box-area ratio, boundary transitions per foreground pixel and normalized horizontal/vertical bounding-box centers.
  • 9 K/V features: Key and Value cosine similarities, Key and Value relative L1 differences, four source/target Key/Value norms and one source editing-region/background gap.
  • 3 text features: mean, standard deviation and L2 norm of the frozen target-conditioning vector.
  • 1 budget feature: source-preservation preference ρ\rho.

Cosine similarities are computed over feature channels for masked tokens and averaged across tokens and heads. Relative L1 difference is normalized by the corresponding source magnitude; each norm is a channel-wise L2 norm averaged over heads and masked tokens. The region/background gap summarizes differences between source K/V means inside and outside the edited region. All features use training-state normalization,

s_k=\begin{cases}\sigma_k,&\sigma_k\ge 10^{-6}, \\\\ 1, &\sigma_k<10^{-6}.\end{cases}$$ The training statistics are stored in the Router checkpoint and reused during inference. The implemented input does not include a separate attention-entropy feature. ## W3. MagicBrush mask provenance The mask-conditioned training pipeline reads an explicit editing mask from its **data manifest**, rather than estimating one from source–target pixel differences during candidate generation. MagicBrush has mask-provided and mask-free settings; only records with valid masks enter the masked routing pipeline. The Appendix describes image-space binarization at threshold 127, spatial alignment with the source, latent resizing and token packing, as well as exclusion of empty/full masks. The edited target is used for **offline candidate scoring**, not as an input to the inference Router. This specifies what annotations the training procedure consumes and prevents any ambiguity about target-image access during inference.

W4. Early/Middle/Late boundaries and group-specific supervision

The Early, Middle and Late labels refer to offline curve control points at normalized progress u=0u=0, 0.50.5 and 11; consecutive points are joined by piecewise-linear interpolation. They are not three logits produced by the online Router. During inference, the Router produces one coefficient αt,g\alpha_{t,g} for each executed denoising update tt and routing group gg.

The counterfactual generator evaluates seven candidate curves separately for each of six groups. For a selected group gg, the candidate controls that group while the other five groups use α=0.5\alpha=0.5. For every training sample xx and preservation preference ρ\rho, the highest-reward curve index is jx,g,ρ∗j^*_{x,g,\rho}, with runner-up jx,g,ρ(2)j^{(2)}_{x,g,\rho}. The group-specific target is

yx,t,g,ρ=ajx,g,ρ∗(ut),y_{x,t,g,\rho}=a_{j^{*}_{x,g,\rho}}(u_t),

yx,t,g,ρalt=ajx,g,ρ(2)(ut)y_{x,t,g,\rho}^{\text{alt}} = a_{j_{x,g,\rho}^{(2)}}(u_t)

Algorithms 1 and 2 now explicitly carry the sample, group and budget indices, so distinct groups receive distinct supervised targets.

W5. Lack of loss-weight sensitivity analysis

The training objective remains

L=Lα+0.2Lrank+0.05Lsmooth+0.05Lbudget.\mathcal L=\mathcal L_\alpha+0.2\mathcal L_{\rm rank}+0.05\mathcal L_{\rm smooth}+0.05\mathcal L_{\rm budget}.

We added a one-factor-at-a-time sensitivity study: each auxiliary weight is varied while the other two stay at the default values.

Table R3-3. Loss-weight sensitivity on PIE-Bench (Appendix B).

SettingStructure ×10³ ↓PSNR (dB) ↑LPIPS ×10³ ↓Edit CLIP ↑
Rank = 0.1036.8434.5653.9022.88
Rank = 0.2036.5234.7153.7423.01
Rank = 0.3036.6934.5953.8622.94
Smooth = 0.02536.7734.6353.9222.93
Smooth = 0.0536.5234.7153.7423.01
Smooth = 0.1036.7334.6153.8722.90
Budget = 0.02537.0134.5054.0422.76
Budget = 0.0536.5234.7153.7423.01
Budget = 0.1036.7634.5953.8822.87

The default configuration gives the highest PSNR and Edit CLIP in these tested variations. The sensitivity ranges are narrow: for example, varying the ranking coefficient from 0.10 to 0.30 changes PSNR within 34.56–34.71 dB and Edit CLIP within 22.88–23.01. The regression term remains the primary training target; ranking and the two bounded regularizers serve auxiliary roles. These results demonstrate local stability around the reported coefficients, rather than claiming invariance to arbitrary weight choices.

W6. Counterfactual cost and Pareto-frontier role

The exact design consists of seven candidate temporal curves and six routing groups, yielding 6×7=426\times7=42 single-group candidate-generation jobs per sample. Each of the three preservation budgets reuses these generated candidates, yielding 6×3=186\times3=18 group–budget label records rather than tripling generation cost. A separate coarse configuration has seven global candidate jobs. The fine-grained design is thus a group-wise intervention scheme, not an exhaustive 767^6 joint search.

The Pareto-nondominated set is recorded to describe editing–preservation–boundary–cost trade-offs. The implemented winner and runner-up training labels are selected by scalarized reward ranking across the seven candidates for each (x,g,ρ)(x,g,\rho):

\qquad j^*_{x,g,\rho}=\arg\max_j R_{x,g,j}(\rho).$$ Accordingly, **Pareto membership is descriptive, not a separate filtering stage**, and we do not attribute an additional label-selection gain to a Pareto filter that the implementation does not use. The Appendix documents candidate counts and Router arithmetic; the matched **48.51 s / 39.48 GB** end-to-end execution and the **800 ms / 480 MB** added component costs are reported in its efficiency tables.

Q1. Direct empirical evidence for time-varying conflict and editing error

Tables R3-1 and R3-2 above provide direct sample-level and stage-wise evidence. The conflict–LPIPS Pearson correlation changes from +0.31 (Early) to +0.42 (Middle) and +0.28 (Late), while PSNR correlates −0.41 with discrepancy over samples. Controlled ablations agree: fixed K/V obtains 33.09 PSNR / 22.16 Edit CLIP, three-stage routing obtains 33.74 / 22.85, and full timestep-by-group routing obtains 34.71 / 23.01. These results show that the relationship between K/V state discrepancy and fidelity is not constant across denoising and that finer adaptive control improves the output trade-off.

Q2. Complete feature computation and normalization

The 21 inputs and their aggregation axes are specified in W2 and in Appendix B, Feature Aggregation and Normalization. The conflict component uses masked-token Key/Value cosine similarity, normalized L1 source–target discrepancies, four Key/Value norms and a source-region/background gap. Mask geometry, temporal/depth coordinates, pooled text moments and ρ\rho complete the vector. Per-feature means and standard deviations are computed on training states and reused unchanged at inference; near-zero standard deviations are replaced by one as shown above. No target edited image is supplied to the inference Router.

Q3. Mask source and mask-accuracy dependence

The masks are read from mask-provided data records in the training manifest; mask-free records are not used for masked Router supervision. The implementation does not generate its masks through source–target difference thresholding. The preprocessing is deterministic: binarize the image-space mask, align spatially with the source and map it to packed latent tokens. Thus the method uses an explicit externally provided editing-region mask at inference as well.For more details about the mask, see W3.

Q4. Exact temporal definitions and distinct group targets

The three offline control points are u=0,0.5,1u=0,0.5,1, linked by piecewise-linear interpolation. They do not specify three fixed online intervals. The six Double-Block groups follow the contiguous 3+3+3+3+3+4 partition of the 19 Double Blocks. Candidate rewards are ranked separately for each (x,g,ρ)(x,g,\rho), so the interpolated targets yx,t,g,ρy_{x,t,g,\rho} explicitly depend on gg. At inference, only one scalar αt,g\alpha_{t,g} is evaluated and cached for each group at each denoising update.

Q5. Selection and robustness of loss weights

We retain weights 0.20 / 0.05 / 0.05 for ranking, smoothness and budget consistency, with Huber regression as the principal target. Table R3-3 reports all nine single-factor configurations. At the default values, the method obtains 34.71 PSNR and 23.01 Edit CLIP, compared with 34.56–34.59 / 22.88–22.94 for the other tested ranking weights, 34.61–34.63 / 22.90–22.93 for the tested smoothness alternatives, and 34.50–34.59 / 22.76–22.87 for the tested budget alternatives. This documents robustness within the explicitly tested neighborhoods of the default coefficients.

Q6. Pareto, candidate budget, training cost and inference overhead

For each sample, 42 fine-grained single-group candidates are evaluated, with 18 group–budget label records obtained by rescoring the same outputs for three budgets; seven additional global policies belong to the separate coarse variant. The winner is chosen by the scalar reward above, with Pareto membership recorded for analysis. Consequently, a Pareto-filtered versus unfiltered winner-selection comparison does not describe two distinct label-selection rules in the current implementation; the precise algorithmic distinction is stated in W6. End-to-end inference efficiency is measured under matched hardware and schedule: fixed K/V 47.71 s / 39.00 GB versus CATD-MoE 48.51 s / 39.48 GB, with the conflict/Router/fusion breakdown in Table R2-2. This separates one-time offline supervision generation from per-image online inference cost.

Dear Reviewer cRUn:

Thank you again for your careful reading of our method. We hope you will have a chance to read our response, particularly the discussion of how Algorithm 2 constructs a supervision target for each block group. We would be grateful if you would consider these details when revisiting your initial rating.

Kind regards,

The Authors

Reviewer_ktxkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CATD-MoE to address the trade-off between editing flexibility and source-image fidelity in instruction-guided image editing. The method treats the K/V representations of the source image and the current generation state as two memory experts. A lightweight Router dynamically adjusts their fusion weights based on denoising progress, network depth, and K/V differences. The diffusion backbone remains frozen, while the Router is trained using supervision derived from offline candidate evaluations. Experiments on PIE-Bench demonstrate that the proposed method improves fidelity in non-edited regions while maintaining editing effectiveness.

Critical strengths
  1. The research problem is practically relevant, and the method is clearly designed. Dynamically adjusting the fusion of source-image and generation-state K/V representations provides a straightforward and intuitive approach to balancing content preservation and editing requirements.
  2. The experiments provide evidence of the method’s effectiveness. The method achieves promising results on an independent test set, while ablation studies and a user study further support the effectiveness of its key design choices.
Critical weaknesses
  1. Limited benchmark evaluation. The quantitative evaluation is primarily conducted on PIE-Bench, which is insufficient to establish the broader applicability of the proposed method. Additional experiments on other benchmarks are needed to assess its generalization ability.
  2. Missing analysis of computational overhead. The additional runtime and memory costs introduced by K/V difference computation, routing, and fusion remain unclear. A detailed comparison under consistent experimental settings is needed, along with an analysis of the potential trade-off between performance gains and computational efficiency.
  3. Lack of comparisons with recent open-source and closed-source editing models. The authors are encouraged to include comparisons with models such as FLUX.2 [klein] and GPT Image 2.5. Such comparisons would help clarify the method’s performance relative to current editing models and its gap with closed-source systems, providing a better understanding of its practical value.
  4. Insufficient justification for the learned Router. Although the paper includes ablations with fixed fusion and three-stage fusion strategies, it lacks a comparison with a simpler approach that directly maps K/V similarity to fusion weights. Without this baseline, it remains unclear whether the observed improvements genuinely require the proposed learning-based mechanism.
  5. User study results should be included in the main paper. Since the user study directly evaluates the perceptual quality of editing effectiveness and content preservation, its main results should be presented in the main text.
  6. Minor formatting issues. The paper contains punctuation inconsistencies around equations. For example, Equation (1) should be followed by a comma if the sentence continues. The authors should review and correct similar formatting issues throughout the paper.
Questions for authors
  1. Why was FLUX.2 [klein], a stronger open-source model, not included as a baseline?
5 comments

Response to Reviewer ktxk (Reviewer 2)

We thank the reviewer for the constructive assessment. We respond to the six weaknesses and the question separately. The updated manuscript retains the frozen FLUX.1-dev backbone and reports additional controlled analyses under the PIE-Bench protocol.

W1. Limited benchmark evaluation

We use a training/evaluation split that isolates transfer across datasets: MagicBrush is used for Router training, and the independently collected PIE-Bench (700 source images and 10 editing categories) is used for evaluation without fine-tuning. The reported results therefore reflect cross-dataset transfer rather than fitting the evaluation images. PIE-Bench is especially relevant to the preservation–editability question because it provides explicit masks for edited regions and reports non-edited-region fidelity metrics.

W2. Missing computational-overhead analysis

We added a matched end-to-end efficiency evaluation, including the costs of K/V discrepancy extraction, Router inference and K/V fusion. All five variants use an NVIDIA RTX A6000, 1024 × 1024 images, BF16, batch size 1, 24 source-inversion updates and 24 target-denoising updates. Runtime includes the complete editing pipeline, and peak VRAM is recorded in the same environment.

Table R2-1. End-to-end efficiency and editing quality (also reported in Appendix B).

MethodRuntime (s) ↓Peak VRAM (GB) ↓PSNR (dB) ↑Edit CLIP ↑
Fixed K/V47.7139.0033.0922.16
Similarity-only Router48.0839.1533.3622.68
Global routing48.1239.1633.0522.61
Three-stage routing48.2139.2533.7422.85
CATD-MoE48.5139.4834.7123.01

Against fixed K/V, CATD-MoE adds 0.80 s (1.68%) and 0.48 GB (1.23%), while gaining 1.62 dB PSNR and 0.85 Edit CLIP. The measured incremental component costs are:

Table R2-2. Additional cost of CATD-MoE relative to fixed K/V (Appendix B).

Additional componentRuntime (ms)Peak-memory increment (MB)
K/V conflict statistics460370
Router predictions201
K/V fusion and synchronization320109
Total800480

The principal extra cost is K/V tensor processing (780 of 800 ms), not the lightweight MLP itself (20 ms). The Router's architecture, parameter count, per-call MACs/FLOPs, group-level caching, and offline candidate budget are documented in the Appendix B experimental-configuration prose. These matched measurements quantify the quality–efficiency trade-off directly.

W3. No comparison with FLUX.2 [klein] or GPT Image 2.5

Our controlled study isolates the contribution of dynamic K/V allocation within a frozen FLUX.1-dev backbone. The Router consumes source/target K/V states in the FLUX.1-dev Double Blocks and is specifically trained on that architecture. Integrating it into FLUX.2 [klein] would require a different intermediate-memory interface, feature alignment and Router training, changing both the backbone and the controller. GPT Image 2.5 is accessed through an image-editing API and does not expose the internal K/V tensors required to insert and evaluate our mechanism.

A direct comparison of final outputs across these independent systems would therefore measure a system-level difference, not the isolated causal effect of our K/V routing.

W4. Is the learned Router necessary compared with similarity-based mixing?

We added the direct Similarity-only Router control in Table 4. Its coefficient is determined from a scalar source–target Key/Value cosine-discrepancy signal rather than the full 21-dimensional state-conditioned representation.

Table R2-3. Direct routing control and preservation-budget results (Table 4 in the manuscript).

SettingStructure ×10³ ↓PSNR (dB) ↑LPIPS ×10³ ↓Edit CLIP ↑
Fixed K/V38.0533.0952.8422.16
Similarity-only37.7433.3653.8022.68
CATD-MoE (ρ=0.25\rho=0.25)37.4833.8854.4623.18
CATD-MoE (ρ=0.50\rho=0.50)36.5234.7153.7423.01
CATD-MoE (ρ=0.75\rho=0.75)35.8735.2852.9722.62

At the default ρ=0.50\rho=0.50, the full Router improves on similarity-only routing by 1.35 dB PSNR, 0.33 Edit CLIP, and 1.22 in Structure Distance ×10³. A single discrepancy value tells us how much the memories differ but not whether the difference supports the intended edit or introduces unintended drift. The full Router conditions that allocation on the editing mask, text statistics, denoising progress, block-group depth and preservation budget. The direct control, together with the original conflict-removal and routing-granularity ablations, establishes the added value of the learned decision rule.

W5. User study should appear in the main paper

A compact user-study table has been inserted into Section 4.2, while the complete ten-method results remain in Appendix C. The study uses 50 participants, 50 PIE-Bench samples and five editing categories, with anonymous 1–5 ratings for source fidelity (SF), background preservation (BP) and editing effectiveness (EE).

Table R2-4. Main-text human evaluation (means on a 1–5 scale).

MethodSF ↑BP ↑EE ↑
KV-Edit4.214.353.58
ReFlex4.054.183.82
CATD-MoE4.244.323.90

CATD-MoE attains the highest mean SF (4.24) and EE (3.90) among the reported methods. Compared with KV-Edit, EE improves by 0.32 points, while BP remains comparable (4.32 vs. 4.35).

W6. Equation punctuation and formatting

We checked punctuation and sentence continuity around displayed equations, and reviewed the presentation of subscripts, timestep/group indices and table references. The revised manuscript keeps the online single-coefficient notation αt,g\alpha_{t,g} consistent with the group-specific training targets in the Appendix, while the three offline temporal control points are explicitly distinguished from online routing outputs. The compact tables in Sections 4.2 and 4.4 also use consistent metric labels and scaling conventions.

Q1. Why not include FLUX.2 [klein]?

The Router is an architecture-specific intervention in FLUX.1-dev Double-Block K/V memory, not a separate general-purpose foundation model. FLUX.2 [klein] uses a different model and editing interface; a valid intervention study would require adapting the memory capture and training the Router for that backbone. Such a test changes the underlying model in addition to the routing policy. Our reported experiments instead compare the routing controls while fixing FLUX.1-dev, which directly measures the proposed contribution. We acknowledge FLUX.2 [klein] as a relevant independent editing system without representing an unmatched system-to-system comparison as a controlled routing ablation.

Reviewer_iPijOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CATD-MoE, a conflict-aware dynamic K/V routing framework for instruction-guided image editing. The method considers source and target K/V representations as two memory experts and uses a lightweight router to dynamically adjust their mixing weights across different denoising timesteps and Transformer block groups. The router is trained with counterfactual candidates and Pareto-based supervision, while the diffusion backbone is kept frozen. Experiments on PIE-Bench show better source image preservation and competitive editing performance.

Critical strengths

The paper is clearly written and well organized. The motivation, method, and experiments are easy to follow

The method keeps the FLUX.1-dev backbone frozen and only trains a small router with about 19.7K parameters. This allows dynamic K/V control without fine-tuning the backbone.

The paper compares with several representative image editing methods and includes ablation studies and a user study. The results show good performance in source preservation, especially on PSNR and SSIM, while keeping competitive editing quality.

Critical weaknesses

The necessity of the routing mechanism is not well justified. The conflict between editing quality and source preservation may mainly come from the lack of high-quality aligned training data, rather than limitations of the model architecture. With better source–instruction–target aligned data, editing models may already learn to preserve unchanged regions without additional K/V routing. However, the paper does not discuss this possibility or compare with strong editing models trained on high-quality aligned data. Therefore, it is unclear whether the proposed method solves a fundamental problem or just compensates for insufficient training data.

Although the method is described as a Mixture-of-Experts framework, the two experts are actually source and target K/V representations, and the router only predicts a mixing coefficient between them. This is more similar to adaptive K/V fusion than a typical MoE architecture. The difference from existing K/V injection and feature-mixing methods is not clearly explained.

Questions for authors

na

2 comments
AuthorsOpenReview ↗

Response to Reviewer iPij

Comment 1: Necessity of the Routing Mechanism

We agree that high-quality source–instruction–target alignment is important for instruction-guided image editing. However, data supervision and inference-time memory allocation address different aspects of the editing process.

CATD-MoE does not attempt to replace aligned training data or learn the editing task from scratch. Instead, it learns how to regulate the utilization of source-image memory and target-generation states while keeping the pretrained FLUX.1-dev backbone frozen. The Router is trained using the official MagicBrush dataset, which contains approximately 10K instruction-guided editing instances. PIE-Bench, containing 700 real-world images across ten editing categories, is used exclusively for independent evaluation without additional fine-tuning.

The controlled ablations provide evidence for the contribution of adaptive memory allocation within this setting:

VariantPSNR ↑Edit CLIP ↑
Full CATD-MoE34.7123.01
w/o conflict feature32.9722.58
w/o temporal routing32.6422.50
Fixed K/V ratio33.0922.16

Compared with fixed K/V mixing, CATD-MoE improves PSNR by 1.62 and Edit CLIP by 0.85. Removing conflict features decreases these metrics by 1.74 and 0.43, respectively. Removing temporal routing results in decreases of 2.07 and 0.51.

Since the backbone, training dataset, and evaluation protocol remain unchanged across these ablations, the differences support the contribution of the routing components rather than changes in backbone training or dataset composition.

We therefore position CATD-MoE as a complementary inference-time memory-control mechanism for frozen pretrained editors. We do not claim that high-quality aligned training data is insufficient or that CATD-MoE universally outperforms end-to-end models trained with larger aligned datasets. Its contribution is the adaptive allocation of existing source and target information without retraining the generative backbone.


Comment 2: Distinction Between the Proposed MoE Formulation and Adaptive K/V Fusion

We appreciate this distinction and clarify the terminology.

In CATD-MoE, the experts are defined according to memory provenance rather than independently parameterized neural networks:

\qquad E_{\mathrm{tar}}=(K_t,V_t).$$ The source-image K/V and target-state K/V represent two information sources with different functional roles. Their continuous interpolation is defined as: $$\alpha E_{\mathrm{src}} + (1-\alpha)E_{\mathrm{tar}}, \qquad \alpha\in[0,1].$$ The intermediate Bridge region represents continuous mixtures between the two memory sources; it is not a third independently parameterized neural network. Accordingly, CATD-MoE is a memory-level MoE-style routing framework, rather than a conventional sparse MoE with multiple independently trained feed-forward experts. The contribution is not the interpolation equation itself, but the learned routing policy. Fixed K/V injection uses a predefined coefficient, whereas CATD-MoE predicts a continuous coefficient conditioned on the current K/V state, denoising progress, block-group position, mask geometry, text statistics, and preservation preference. The implementation uses six Double-Block groups. One routing coefficient is computed for each timestep and group and reused by all blocks within that group. The coefficient is supervised using editing outcomes from counterfactual candidate generation rather than manually assigned editing-category labels. Thus, the distinction from fixed K/V injection is the combination of state-dependent routing, group-specific temporal control, and outcome-based counterfactual supervision. We use the terms memory-level mixture and dynamic K/V routing to describe this formulation precisely.

Dear Reviewer iPij:

Thank you again for your thoughtful review. When you have a chance, we would be grateful if you could read our response on adaptive K/V routing and the roles of the source and target memories, and consider whether it changes your initial assessment.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.