acceptodds
Desk rejected at ICLR 2027

CARVE: Cascaded Adaptive Routing of Visual-Condition Embeddings for Multi-Reference Generation

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-reference controllable generation represents each reference with many visual-condition tokens. As the number of references grows, these tokens create substantial attention cost in existing transformer architectures. Yet many tokens contribute little to the current generation while still being processed at every layer and denoising step. Existing compression methods use fixed, hand-crafted rules to select or merge tokens. Such rules do not adapt to the current input or generation state and can remove reference-specific identity, texture, or color details. We propose CARVE, the first learnable and input-adaptive framework for visual-condition-token compression. Rather than predefining token importance or pruning locations, CARVE learns to preserve the tokens that contribute to the current generation at every denoising step and to remove tokens after their reference information has become redundant. The resulting compression adapts to each input and denoising computation while preserving the information required for faithful multi-reference generation. Across diffusion-transformer and unified multimodal generation backbones, CARVE achieves the strongest generation quality among compared compression methods while substantially reducing token processing and inference latency.

Primary area:
generative models
Keywords:
Diffusion Model, Image Generation, Compute Efficiency

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rcu7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets in-context generation pipelines that perform self-attention over conditioning and generation tokens in their forward pass. For in-context multi-reference generation, the paper claims that most reference tokens contribute nothing, because of redundancy and distracting details in reference samples. CARVE trains small routers that progressively delete these tokens along the depth of the network.

Critical strengths

S1. The methodology is novel, it is faster to keep the backbone frozen and to train small adapters that can reduce tokens at little cost to performance. Instead of masking tokens, the paper physically prunes them, making it easy for standard FA to work just as well.

S2. The DDS ablation gives us important insight on how DiTs use in-context learning and how layers influence generation and reference adoption.

S3. The paper presents a well ablated methodology section, with sufficient contribution from proposed elements. The ablations are correctly designed with each component switched off completely for analysis.

Critical weaknesses

W1. The paper in the abstract claims 2x performance improvement ("doubles inference efficiency") but measured speedup is 1.2x on bagel. Would be good if the authors can clarify this point.

W2. Close related works like ToPi and PKA are discussed in related works but not compared against. This comparison is very important to understand how CARVE compares with recent works in the field.

Questions for authors

See Weaknesses

3 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer rcu7

Thank you for your careful review and constructive feedback. We report the efficiency measurements and direct comparisons with ToPi and PKA in the responses below.


R3.Q1. Clarification of the speedup claim

Reviewer comment

The paper claims to double inference efficiency, but the measured speedup is approximately 1.2x. Could the authors clarify this point?

Response

We report measured DiT latency and attention-compute reduction separately, with each efficiency result scoped to its backbone and metric.

(1) Qwen-Image: wall-clock speedup

  • On Qwen-Image, the measured DreamOmni2Bench DiT latency is 40.645 s for full tokens and 33.472 s for CARVE, corresponding to 1.21×1.21\times speedup or a 17.6% latency reduction.
  • The attention computation is reduced from 833.5 to 574.3 TFLOPs, or 31.1%.

(2) BAGEL: compute reduction rather than latency speedup

  • For BAGEL, the reported change from 477.4 to 187.1 TFLOPs is a 2.55×2.55\times reduction in estimated attention computation.
  • It is not evidence of a 2.55×2.55\times wall-clock speedup.
  • The amount of acceleration depends on the backbone, attention implementation, and overhead outside reference-token attention.

(3) Manuscript wording

We will revise the abstract and introduction to report inference latency and attention-compute reduction separately for each backbone.


R3.Q2. New comparisons with ToPi and PKA

Reviewer comment

ToPi and PKA are discussed in related work but not compared experimentally. These comparisons are important for understanding CARVE relative to recent work.

Response

Neither PKA nor ToPi had publicly released official source code at the time of our experiments. We therefore independently reimplemented both methods following their paper descriptions on our Qwen-Image/MICo baseline.

Quality comparison. The following DreamOmni2Bench scores cover all 319 cases. PKA retains the previously reported epsilon=0.6 quality setting.

MethodMean ref. keepPFSCOverall
ToPi0.54.1884.9234.261
PKAn/a4.60824.73354.3333
CARVE0.4564.7055.1944.537

CARVE achieves the highest Overall score, 4.537, compared with 4.261 for ToPi and 4.3333 for PKA.

Efficiency configuration. For PKA timing, we use the paper's default KSA threshold epsilon=0.2, first-step condition KV caching, and previous-step mask reuse, as described in the PKA paper. Our Qwen-Image inference uses 25 steps, CFG=4.0, seed=0, and reference images of approximately 512 squared pixels. The complete 319-case run at this setting gives a mean DiT time of 37.544 s.

MethodTiming configurationMean DiT time (s)
ToPi25 steps; mean ref. keep 0.556.438
PKA25 steps; epsilon=0.2; condition KV cache37.544
CARVE25 steps; mean ref. keep 0.45633.472

Across these runs with different hardware configurations, CARVE records the shortest DiT time: 33.472 s, versus 56.438 s for ToPi and 37.544 s for PKA.

We will include this comparison and expand the descriptions of the reproduction protocols, experimental configurations, and evaluation metrics in the revised manuscript.

Reviewer_rcu7OpenReview ↗

Response to Authors

Thank you for your response. The discussion period allows you to revise and upload a new manuscript incorporating changes from the review process. It would be great if you can incorporate these changes.

AuthorsOpenReview ↗

Authors' Response to Reviewer rcu7

Thank you for the suggestion. We have uploaded a revised PDF with updated efficiency descriptions.

Reviewer_5Ak6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies visual-condition token compression for multi reference controllable generation. It argues that existing compression methods rely on fixed, hand-crafted heuristics that may not accurately capture each token’s contribution to the current denoising computation. The proposed framework uses lightweight, learnable routers that fuse a global summary of the noised target tokens with each visual-condition token’s hidden state to predict keep-scores. Each router retains the top-k tokens under a given budget and physically deletes the rest, progressively shortening the condition sequence across layers. The paper also introduces Dependency Drop Selection (DDS), which measures layerwise target-to-reference dependency in the frozen backbone and uses dependency drops to select router insertion locations. Only the routers are trained, using flow matching and full context distillation under random compression budgets. On DreamOmni2Bench with Qwen-Image, CARVE retains approximately 47% of the condition tokens and achieves an approximately 9.5% relative improvement over the full-token baseline. On OmniContext with the same backbone, it removes roughly half of the condition tokens with less than a 3% relative decrease in average Overall.

Critical strengths

Practical motivation and adaptive compression. The paper addresses the growing computational cost of visual-condition tokens as the number of references increases in multi-reference generation. Its routers combine a summary of the current target tokens with condition-token states to dynamically predict keep-scores, while cascaded deletion reduces computation in subsequent layers. Compared with fixed, hand-crafted heuristics, this design adapts token selection to the specific input and denoising state.

Parameter-efficient training and flexible compression budgets. CARVE freezes the generation backbone and trains only lightweight routers through flow matching and ful context distillation, without requiring token-level annotations. Random budget training enables a single set of routers to operate across different compression ratios without budget specific retraining, allowing flexible adjustment of the quality efficiency trade-off at inference time.

Critical weaknesses

The methodological novelty is relatively limited. The proposed Router’s scoring structure, which combines local token representations with a global summary, and its cascaded, irreversible pruning across layers closely resemble DynamicViT. Random-budget training has also been explored in prior work such as OFA. The main additions therefore appear to be the adaptation to multi-reference generation and the DDS layer-selection criterion, rather than a fundamentally new mechanism for learning to prune tokens.

The robustness of DDS-based layer selection is insufficiently evaluated. DDS is computed offline only once on a subset of 40 samples from MICo-150K, after which the Router insertion locations remain fixed. The paper does not report how sensitive the selected layers are to the calibration subset size, sample selection, or denoising-step sampling, nor does it evaluate their stability across different data distributions.

Questions for authors

Could the authors clarify the core differences between CARVE and methods such as DynamicViT, and provide a controlled comparison against a directly adapted DynamicViT-style baseline using the same backbone, training objectives, and compression budget to quantify the gains from the proposed additions?

How stable are the Router insertion locations selected by DDS when varying the calibration subset size and denoising step sampling strategy?

How does random budget training compare with fixed budget Routers trained separately for each budget in terms of the quality efficiency trade-off?

1 comment
AuthorsOpenReview ↗

Authors' Response to Reviewer 5Ak6

Thank you for your careful review and constructive feedback. We clarify CARVE's design for reference-conditioned generation and report the additional comparisons and sensitivity tests below.


R2.Q1. DynamicViT-style comparison

Reviewer comment

Could the authors clarify the core differences between CARVE and methods such as DynamicViT, and provide a controlled comparison against a directly adapted DynamicViT-style baseline using the same backbone, training objectives, and compression budget to quantify the gains from the proposed additions?

Response

(1) Target-conditioned scoring versus reference-only pooling

DynamicViT's PredictorLG combines each token's local representation with a pooled summary of surviving image tokens. CARVE scores each reference token using a summary of the current noisy target. Selection therefore responds to the target's evolving need for reference evidence, including object-attribute binding and detail reconstruction.

(2) DDS placement and generation supervision

DDS selects where reference tokens are removed across depth. Full-context distillation and the generation objective supervise the effect of selection on the predicted generation update. CARVE couples target-conditioned token selection with calibrated deletion locations and generation supervision.

(3) DynamicViT-style adaptation protocol

  • We trained a direct DynamicViT-style adaptation on the same Qwen-Image/MICo backbone and 10K training set, using the same DDS positions, generation/distillation objectives, and exponential budget framework.
  • The baseline retains PredictorLG and uses the shared budget-enforcing training machinery with generation-specific losses.

(4) Quantitative comparison with CARVE

We compare CARVE with the DynamicViT-style result at B=0.55 and Mean Keep=0.500. Results cover all 319 cases and use the All sample average:

MethodBMean keepPFSCOverall
DynamicViT-style0.550.5004.4335.0394.412
CARVE0.470.4564.7055.1944.537

CARVE achieves higher PF (4.705 vs. 4.433), SC (5.194 vs. 5.039), and Overall (4.537 vs. 4.412), while retaining fewer reference tokens on average (0.456 vs. 0.500).


R2.Q2. DDS stability across calibration settings

Reviewer comment

How stable are the Router insertion locations selected by DDS when varying the calibration subset size and denoising step sampling strategy?

Response

(1) Calibration sizes and independently sampled subsets

We tested 20, 40, and 80 calibration samples with three independently drawn subsets for each size:

SamplesSubsetsSelected zero-based blocksOverlapJaccard
203[21, 22, 43, 52, 57]100%1.000
403[21, 22, 43, 52, 57]100%1.000
803[21, 22, 43, 52, 57]100%1.000

(2) Stability across sample sizes and subsets

All three calibration sizes and their tested subsets select the same five blocks, with 100% overlap and Jaccard 1.000. DDS layer selection is stable across the tested calibration sizes and subsets.

(3) Denoising-step sampling sensitivity

We compared uniformly sampled 10-step and 20-step probe grids and obtained the same selected blocks, [21, 22, 43, 52, 57]. This demonstrates that DDS generalizes across the two tested timestep-sampling densities, with stable layer selection even when the number of probe steps is halved.


R2.Q3. Random-budget versus fixed-budget training

Reviewer comment

How does random budget training compare with fixed budget Routers trained separately for each budget in terms of the quality-efficiency trade-off?

Response

(1) Training strategies

Fixed-budget training holds B constant for each separately trained model. CARVE samples B during training. At inference, both variants deterministically retain the highest-scoring reference tokens at the requested B.

(2) Original component ablation

In the original ablation, removing random-budget training gives Macro5 Overall 4.401, while the complete method achieves 4.892, an improvement of 0.491 under that ablation protocol.

(3) Comparison at the same inference budgets

We compare the separately evaluated fixed-budget and random-budget checkpoints at B=0.50 and B=0.60. Each evaluation covers the same 319 cases. Overall below uses the same five-task Macro5 aggregation:

Inference BFixed-budget trainingRandom-budget training (CARVE)Random - fixed
0.504.17004.2474+0.0774
0.604.26794.2648-0.0031

Random-budget training improves Overall at B=0.50 and remains close to fixed-budget training at B=0.60. Together with the ablation, these results support one policy serving multiple inference budgets while maintaining quality.

Reviewer_crMmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the cost of multi-reference generation in models that concatenate reference tokens with the noised target, such as Qwen-Image and BAGEL. CARVE inserts five lightweight Routers into a frozen backbone. Each Router scores the alive reference tokens from a mean-pooled summary of the target hidden states and each token's own hidden state. It keeps the top fraction α^i of the original count and physically removes the rest from the sequence, the RoPE table and the attention mask; target and text tokens are never removed. Router positions are chosen once per backbone by Dependency-Drop Selection. On 40 samples, DDS measures how much target queries attend to, and draw output from, reference tokens at each layer, and it picks the five largest drops. Only the Routers are trained, using flow matching plus distillation from the full-token model, with α sampled from U[0.3, 1] so that one model serves several budgets.

Experiments use Qwen-Image (60 DiT blocks) and BAGEL (28 layers), with Routers trained on 10K MICo-150K samples. Evaluation is on OmniContext and DreamOmni2Bench, with a GPT-4.1 judge scoring prompt following and subject consistency. On DreamOmni2Bench with Qwen-Image, CARVE at an input keep ratio of 0.47 scores 4.537 Overall. This is above the full-token model (4.144) and the compression baselines at 0.5 (3.820 to 4.274). It also reduces attention FLOPs by 31% and DiT latency from 40.6 s to 33.5 s. On OmniContext it scores 7.216 against 7.385 for full tokens. On BAGEL, CARVE uses an input keep ratio of 0.65 and leads the compression baselines, but it drops from 8.101 to 5.443 on OmniContext relative to full tokens. Ablations remove flow matching, distillation, DDS placement or random budgets. A budget sweep from 0.9 to 0.2 shows CARVE ahead of the baselines at every tested keep ratio.

Critical strengths
  • The problem is practical and timely. In-context multi-reference conditioning is now standard in models such as FLUX.1 Kontext, Qwen-Image and BAGEL, and its cost grows with every added reference at every layer and denoising step. Replacing fixed merging or downsampling rules with a selection that depends on the current target state is a sensible direction. The Router maps in Figures 5 and 7 suggest that the retained tokens concentrate on the objects relevant to the instruction.

  • The method is light and easy to adopt. The backbone, text encoder and VAE stay frozen, and only about 84M (Qwen-Image) or 94M (BAGEL) Router parameters are trained. For Qwen-Image this takes roughly 157 optimizer updates on 10K samples. Random-budget training lets a single model operate across keep ratios, and the sweep in Figure 4(a) shows smoother degradation than the fixed baselines (4.45 at a 0.2 keep ratio against 3.92 for the best baseline). Deleting tokens physically while preserving their RoPE positions, and reporting measured latency rather than only FLOPs, is careful engineering.

Critical weaknesses
  • The main claim, that a learned and input-adaptive selection is what preserves quality, is not isolated from where and when tokens are deleted. With Routers at blocks 21, 22, 43, 52 and 57 and an input keep ratio of 0.47, the schedule keeps all reference tokens for the first 21 of 60 Qwen-Image blocks. It then drops to about 47%, 22%, 10%, 5% and 2%. This reproduces the reported mean keep of 0.456, and the same calculation for BAGEL gives the reported 0.440. The reference-token baselines instead keep about half of the tokens at every layer. The largest ablation effect also comes from placement (Overall 3.851 with uniformly spaced Routers against 4.892), not from the selection itself. The missing control is the same DDS layers and schedule with random selection, or with a simple score such as target-to-reference attention. A baseline that keeps all tokens early and then drops all reference tokens after a given layer is also missing. Without these, the gains may come mostly from deleting late rather than from learning which tokens to keep. That the Qwen-Image Routers are trained for only about 157 updates makes this question more pressing.

  • The advantage over simple baselines is small in several settings, and the efficiency gain is modest. On OmniContext with Qwen-Image, uniform downsampling (OminiControl2) reaches 7.195 against 7.216 for CARVE, and on DreamOmni2Bench with BAGEL, ToMeSD reaches 3.537 against 3.571. On BAGEL every compressed model, CARVE included, falls far below the full-token model (5.443 against 8.101 on OmniContext). Measured DiT latency on Qwen-Image improves by 1.21×, the same level as OminiControl2 (33.8 s against 33.5 s). On BAGEL, at its 0.65 input keep ratio, CARVE uses slightly more attention compute than the baselines (187.1 against 181.5 TFLOPs). The introduction's statement that CARVE doubles inference efficiency while improving performance by about 10% is therefore not supported in general. The latency gain is 1.21×, and the quality gain over full tokens appears only for Qwen-Image on DreamOmni2Bench.

  • The evaluation relies entirely on a single GPT-4.1 judge, with no seeds, confidence intervals or human study, and several per-reference subsets have only 21 to 39 cases. On DreamOmni2Bench with Qwen-Image, three of the fixed baselines (ToMeSD 4.274, ToMA 4.215, SiTo 4.183) also score above the full-token model (4.144). This suggests that differences of this size are within the noise of the judge. Standard reference-fidelity and text-alignment metrics, such as DINO or CLIP-I similarity to the references and CLIP-T, would make the comparison more robust. Some baselines also appear broken on BAGEL, where ToMA and CAT-Pruning score 0.220 and 0.474 out of 10 on OmniContext. CAT-Pruning also prunes target rather than reference tokens i.e. Appendix C, so it is not a like-for-like comparison.

Questions for authors

How does CARVE compare with random or attention-score selection at the same DDS layers and schedule, and with keeping all tokens up to block 21 and then dropping all reference tokens? What keep ratio does Table 5 use? How were the baselines adapted to BAGEL, and why do ToMA and CAT-Pruning collapse there? Can you report variance over seeds or repeated judge runs?

I am willing to adjust the score if the questions could be resolved.

2 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer crMm (Part 1/2)

Thank you for your careful review and constructive feedback. We address selection controls, retention budgets, and BAGEL below; Part 2 covers evaluation and efficiency.

R1.Q1. Learned versus random/attention selection

We evaluated random and target-to-reference attention selection at DDS blocks [21, 22, 43, 52, 57], with the same exponential cascade and all target/text tokens retained. B is the cascade factor: cumulative reference retention is B, B^2, ..., B^5. The following are 319-case All scores.

SelectionBMean keepPFSCOverall
Random0.500.4714.4614.7304.128
Target-to-reference attention0.500.4714.4984.9444.237
CARVE0.470.4564.7055.1944.537

Budget scope. These operating points have close but unequal budgets. Random selection gives Overall 4.128/4.493/4.501 at B=0.50/0.60/0.70; attention selection gives 4.237/4.305/4.339 at B=0.50/0.55/0.60. We report each operating point with its actual budget.

Selection mechanism. Attention magnitude measures current routing; CARVE supervises selection through the resulting generation prediction. A weakly attended cue can still be needed later for attribute binding or detail reconstruction. The pretrained generator enables short policy adaptation; approximately 157 updates alone do not isolate the benefit of learning, which is why we added these controls.

R1.Q2. Late-drop after block 21

We retained all reference tokens through zero-based block 21 and deleted them before block 22. Mean keep is 22/60 = 0.367; Editing/Generation/All Overall is 3.088/4.618/3.635. This tests terminating reference access after early blocks. Its smaller budget makes it a boundary control rather than a budget-matched comparison. At this operating point, early exposure alone does not preserve the quality obtained with selected reference evidence available later.

R1.Q3. Table 5: budget and aggregation

We will specify each Table 5 ablation's B, cumulative retention, and measured Mean Keep. B does not mean uniform retention at every block. Ignoring integer rounding and the minimum-token constraint, the five DDS blocks give:

Cumulative retention: B, B^2, B^3, B^4, B^5
Mean keep = (21 + B + 21*B^2 + 9*B^3 + 5*B^4 + 3*B^5) / 60

Thus, B=0.47 gives mean keep 0.456; B=0.50 gives 0.471. Table 5 uses five-task macro averages; the controls above use 319-case All averages:

Per-case Overall = sqrt(PF * SC)
All Overall = sum of the 319 per-case Overall scores / 319
Macro5 Overall = (Add Overall + Replace Overall + Global Overall
                 + Local Overall + Generation Overall) / 5

Each task Overall is the mean of its per-case scores. All weights cases equally; Macro5 weights tasks equally.

R1.Q4. BAGEL adaptation and baseline behavior

How the baselines are adapted. We insert their operations into BAGEL-MICo's packed MoT forward:

  • ToMeSD: bipartite matching and weighted averaging of visual-condition tokens before the transformer.
  • ToMA: facility-location representative selection and soft merging of the ViT/clean-VAE condition pool before the transformer.
  • SiTo: per-block condition-token pruning, attention, and nearest-retained-state recovery.
  • OminiControl2: spatial downsampling before both reference encoders, with target resolution fixed.
  • CAT-Pruning: five full warm-up steps, then selected noisy-target updates with cached velocities for inactive positions; reference tokens remain.

Sequence compaction also updates position embeddings, attention masks, and expert indexes from the retained-token mapping. Merged representatives keep their associated position/expert assignments.

Why ToMA can degrade. BAGEL combines ViT semantics with clean-VAE spatial/appearance detail (Appendix A). Merging this heterogeneous pool can mix different objects, attributes, and feature streams. Soft averaging can blur semantic identity and fine detail together; the reduced condition sequence carries this bottleneck through later blocks. Feature similarity alone does not preserve instruction-specific conditioning roles.

Why CAT-Pruning can degrade. The adapter removes inactive target tokens from the current forward and reuses their cached velocities. As latent states and denoising time change, these velocities can become stale; attention also loses part of its current target context. Errors can propagate through later updates and be amplified by classifier-free guidance, even with reference features retained.

AuthorsOpenReview ↗

Authors' Response to Reviewer crMm (Part 2/2)

This continues Part 1 with repeated judging, objective metrics, and efficiency.

R1.Q5. Repeated-judge stability

Reviewer comment

Can you report variance over seeds or repeated judge runs?

Response

(1) Repeated-judge protocol

We completed five judging rounds on each fixed set of 319 images for all seven methods, including OminiControl2 and CAT-Pruning: 22,330 PF/SC metric records in total. The sample standard deviations of the complete-round means are:

(2) Original scores and repeatability measurements

MethodPaper PFPaper SCPaper OverallRepeated-judge PF SDRepeated-judge SC SDRepeated-judge Overall SD
Full-token4.0755.1884.1440.01860.04620.0088
ToMeSD4.2925.4174.2740.02680.06900.0335
ToMA4.4334.9694.2150.03220.04710.0215
SiTo4.2955.0724.1830.03330.03010.0478
OminiControl24.1265.0824.0960.03880.07560.0278
CAT-Pruning3.7934.9033.8200.03150.06480.0268
CARVE4.7055.1944.5370.01930.04360.0366

(3) What the standard deviations establish

These are repeated-judge fluctuations for fixed outputs, not generation-seed variance or confidence intervals for the original paper scores. We keep the original point estimates and the new repeatability measurements separate. We distinguish small numerical differences from statistically established improvements.


R1.Q6. Objective metrics and interpretation of small gaps

Reviewer comment

Standard reference-fidelity and text-alignment metrics, such as DINO, CLIP-I, and CLIP-T, would make the comparison more robust. Several score differences are small.

Response

(1) Additional objective measurements

We additionally report reference-fidelity and text-alignment metrics. The rebuttal summary gives the following DreamOmni2Bench results (CLIP-T / CLIP-I / DINO):

MethodCLIP-TCLIP-IDINO
Full-token0.2625110.6996490.499510
ToMeSD0.2623950.6895100.489252
ToMA0.2609590.6823990.473579
SiTo0.2660660.6845420.476246
OminiControl20.2652680.6938960.490514
CAT-Pruning0.2602840.6889970.455825
CARVE0.2681850.7002500.500535

(2) Interpretation of CLIP and DINO

These measurements complement the judge scores: CARVE improves text alignment in this reported comparison while remaining competitive with full-token reference similarity. We do not infer statistical significance from the small CLIP-I/DINO differences, or treat similarity to an input as a complete measure of a requested edit.

(3) Treatment of small quality gaps

We treat the 0.021 Overall difference from OminiControl2 on Qwen-Image/OmniContext as a near tie. The revised presentation will distinguish such near ties from settings with larger reported differences and qualify claims by backbone and benchmark.


R1.Q7. Efficiency claims and measured speedup

Reviewer comment

The introduction's statement that CARVE doubles inference efficiency while improving performance by about 10% is not supported in general. The measured Qwen-Image latency gain is about 1.21-fold, and some quality gaps are small.

Response

We distinguish attention-compute reduction from measured inference speed and report both quantities separately.

(1) Qwen-Image: measured latency and attention cost

  • On Qwen-Image, DiT latency changes from 40.645 s to 33.472 s: a 1.21-fold speedup and a 17.6% latency reduction.
  • Attention cost changes from 833.5 to 574.3 TFLOPs, a 31.1% reduction.

(2) BAGEL: attention-compute reduction

  • On BAGEL, 477.4 to 187.1 TFLOPs is a 2.55-fold reduction in estimated attention computation, not a measured 2.55-fold latency speedup.
  • The fixed baselines' 181.5 TFLOPs are slightly lower than CARVE's 187.1 at the reported operating points.

(3) Revision of the general efficiency claim

We will revise the wording to report inference latency and attention-compute reduction separately for each backbone, and present quality results by benchmark.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.