acceptodds
Withdrawn from ICLR 2027

Carnator: Fast Text-to-Video Generation with Generation-Native Compatibility-Guided Cross-Request Reuse

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video diffusion transformers produce high-quality videos, yet iterative denoising incurs substantial inference latency, limiting interactive and large-scale serving. Most existing acceleration methods focus on individual requests, thereby restricting efficiency gains to redundancy within a single generation trajectory. Recent cross-request reuse offers an additional source of savings, but existing approaches often infer reusability from coarse semantic similarity. This conflates semantic relatedness with generation-level computational compatibility, so aggressive reuse may accept incompatible historical computation while conservative reuse leaves substantial acceleration unrealized. We present Carnator, a cross-request acceleration framework that addresses this challenge by extracting and using generation-native compatibility evidence directly from the model's evolving internal states. Specifically, Carnator performs a lightweight early probe to construct an Early Signature from internal diffusion states, assessing reuse validity through risk-aware compatibility decisions. The same evidence characterizes reuse scope by localizing target-specific computation and guiding joint reuse of historical latent trajectories and sparse attention connectivity. Across three text-to-video backbones, Carnator consistently achieves higher cache-hit end-to-end acceleration than the evaluated cross-request baselines despite more selective cache acceptance, reaching up to 2.17 speedup while maintaining competitive generation quality.

Primary area:
generative models
Keywords:
DiT, Video Generation, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7hjTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies cross-request computation reuse for text-to-video generation, focusing on whether historical computation is compatible with a new request and which parts can be reused. Carnator constructs early signatures from object regions and appearance descriptors extracted from attention and latent states during early denoising, then filters historical caches using calibrated thresholds. For accepted caches, it combines region-guided latent reuse with historical sparse-attention connectivity reuse while preserving selected target-specific computation. The authors report improved acceleration over the evaluated cross-request baselines while maintaining competitive generation quality.

Critical strengths
  • Clear formulation and coherent design. The paper distinguishes semantic similarity from computational compatibility and uses evidence from internal generation states to guide both cache selection and region-specific reuse.
  • Useful initial empirical evidence. Experiments across multiple backbones and component ablations demonstrate acceleration potential. Early-signature filtering improves text alignment over text-only retrieval. Separating threshold selection from independent certification is also more principled than relying solely on manually chosen thresholds.
Critical weaknesses
  • Limited workload representativeness. The evaluation uses a curated VidProM subset with a target-informed cache that guarantees noun-matched candidates for every test request. Its hit rates and speedups may not generalize to independently arriving requests.
  • Missing strong acceleration baselines. The paper does not compare against within-request sparse-attention methods such as Radial Attention, Sparse VideoGen2, RAPID, and DFSAttn, leaving the additional benefit of cross-request reuse unclear. Gains over reduced-step sampling or few-step distilled models are also unverified.

[1] Shangran Lin, Lu Lu, Jian Chen, and Qiang Liu. RAPID: Reusing Attention Sparsity with Inter-step Adaptation for Efficient Video Diffusion. CVPR, 2026.

[2] Jie Hu, Zixiang Gao, Yutong He, and Kun Yuan. DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation. arXiv preprint arXiv:2605.23445, 2026.

[3] Shuo Yang et al. Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation. NeurIPS, 2025.

[4] Xingyang Li et al. Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation. NeurIPS, 2025.

Questions for authors
  • Could cross-request reuse reduce generation diversity? In particular, does reusing historical latent states bias outputs toward cached videos and reduce variation across different random seeds for the same prompt?
4 comments

We thank the reviewer for the insightful feedback. We address each point below.

W1 Workload Representativeness

Cross-request reuse remains an emerging research direction, and standardized benchmarks for evaluating reuse compatibility, semantic preservation, and workload diversity are still limited. Our evaluation is designed as a controlled reuse-rich setting to examine compatibility assessment and acceleration when relevant historical computation is available. This follows a similar experimental protocol to Chorus, which also constructs its workload from a semantically clustered VidProM subset, using 1,000 historical requests as a warm cache.

We recognize that broader request distributions may result in different cache availability, hit rates, and overall acceleration. Within our evaluated workload, we ensure fair comparisons by using the same historical cache, request sequence, backbone, and generation configurations across all methods. Developing a dedicated cross-request reuse benchmark that systematically captures workload locality, reuse compatibility, and semantic correctness is an important direction for our future work.

W2 Complementarity with Intra-Request Acceleration

We appreciate the reviewer for highlighting recent intra-request acceleration methods. CARNATOR exploits computational redundancy across requests, whereas intra-request methods reduce redundancy within individual generations. These approaches target different sources of redundancy and are complementary, as also discussed in Chorus.

To demonstrate composability, we evaluated CARNATOR with and without TeaCache on paired requests across three models. CARNATOR+TeaCache reduced mean E2E latency by 17.3–28.0% and DiT latency by 31.8–49.0%, while maintaining relatively stable generation quality. The table reports latency changes from CARNATOR to CARNATOR+TeaCache and the corresponding quality differences (ΔCLIP-T / ΔVBench-Q):

ModelE2E latency (s)DiT latency (s)ΔCLIP-T / ΔVBench-Q
Wan2.2-TI2V-5B187.30 → 154.98 (−17.3%)159.65 → 81.47 (−49.0%)−0.0023 / −0.0053
Wan2.1-TI2V-1.3B525.58 → 378.63 (−28.0%)496.00 → 293.74 (−40.8%)+0.0067 / −0.0090
LTX-Video-13B65.88 → 54.49 (−17.3%)62.53 → 42.63 (−31.8%)−0.0040 / −0.0043

Q1 Generation Diversity under Cross-Request Reuse

We thank the reviewer for this insightful perspective. Cross-request computation reuse may introduce historical-cache bias and reduce generation diversity, particularly through latent reuse, which preserves parts of historical generation states. Sparse-attention reuse may also influence output variation by constraining attention connectivity. Beyond its technical implications, this is an important question concerning the long-term impact of generative AI on human society, as repeated reuse of historical content could gradually reduce the diversity and originality of AI-generated media.

CARNATOR primarily focuses on accelerating generation while maintaining generation quality, and diversity represents an additional dimension of this trade-off. We believe this is a valuable research direction, where adaptive reuse policies, such as selectively disabling reuse or introducing stochastic cache dropout, could help balance acceleration and generation diversity. A systematic evaluation of cross-seed diversity and the development of diversity-aware reuse strategies constitute important directions for future work.

We thank the reviewer again for the insightful review.

Thanks for the response. The TeaCache experiment demonstrates composability with feature caching, but does not address my concern about comparisons with within-request sparse-attention methods. As shown in Table 3, the speedup of CARNATOR largely comes from reusing the sparse attention patterns from historical requests. Within-request sparse attention methods, such as Radial Attention, Sparse VideoGen2, RAPID, and DFSAttn, also reduce attention computation, so their complementarity with CARNATOR remains to be demonstrated. Therefore, I would like to see direct speedup and quality comparisons against within-request sparse-attention methods.

We thank the reviewer for the suggestion. We are currently conducting the requested experiments and consolidating the speedup and quality results. We will include them in our revised response.

We have now completed a direct comparison with Radial Attention, Sparse VideoGen2, RAPID, and DFSAttn. We adapted their public Wan2.1-14B implementations to Wan2.1-T2V-1.3B, updating the model-specific integration while preserving each method's core sparse-attention procedure. All methods were evaluated on a single NVIDIA H100 under the same generation setup. For CARNATOR, we used the same settings as in the original paper.

MethodE2E latency (s)E2E SpeedupDiT time (s)DiT SpeedupCLIP-TΔCLIP-TVBench-QΔVBench-Q
Dense1138.61.00×974.21.00×0.2984+0.00000.6379+0.0000
CARNATOR525.582.17×496.01.96×0.2810-0.01740.6361-0.0018
Radial Attention731.001.56×697.281.40×0.2883-0.01010.6433+0.0054
Sparse VideoGen2 SAP708.911.61×672.011.45×0.2914-0.00700.6342-0.0037
RAPID703.751.62×666.871.46×0.2838-0.01460.6454+0.0075
DFSAttn704.011.62×668.171.46×0.2925-0.00590.6459+0.0080

CARNATOR achieves the highest speedup: 2.17× end-to-end and 1.96× for DiT, compared with 1.56–1.62× and 1.40–1.46× for the within-request methods. The quality results show a trade-off: CARNATOR has a small VBench-Q decrease (-0.0018), while its CLIP-T decrease (-0.0174) is larger than those of the sparse-attention baselines. These results provide the requested direct comparison and suggest that cross-request reuse captures additional acceleration in this workload, while its quality impact depends on the metric.

Reviewer_DPtoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Carnator proposes a cross-request caching framework for accelerating text-to-video diffusion models. It uses an Early Signature extracted from initial denoising states to assess cache compatibility, then combines region-guided latent reuse with historical sparse attention reuse to reduce computation.

Critical strengths

The motivation is clear. Semantic similarity between prompts does not necessarily imply that historical computation can be safely reused. Using early generation states to assess cache compatibility is a reasonable and potentially useful approach. The method combines latent and sparse attention reuse without additional training. Experiments across three video generation models show consistent end-to-end speedups, with ablations supporting the contributions of the reuse components.

Critical weaknesses
  1. The novelty is somewhat limited, and the comparison with Chorus II is incomplete. Chorus II already explores cross-request sparse attention reuse together with optional latent feature reuse. Carnator mainly introduces Early Signature for compatibility filtering and combines existing reuse mechanisms. However, the experiments appear to adapt only the sparse attention component of Chorus II, without evaluating its optional feature reuse mechanism. The paper should better clarify its contributions over prior work.

  2. The evaluation of reuse safety and generation quality is insufficient. The authors use CLIP-T and VBench-Q to define safe reuse, but Figure 1 itself shows that these metrics may fail to detect content inconsistent with the target prompt. Moreover, risk certification is based on only 70 samples, and CLIP-T drops noticeably on Wan2.2 compared with dense inference and other baselines. The paper needs stronger evidence of reliable reuse, particularly regarding semantic alignment and temporal consistency.

  3. The practical speedup and deployment costs are not sufficiently evaluated. Early Signature reduces the cache hit rate, while Table 2 shows no improvement in cache-hit latency. Although this improves generation quality, the overall quality–speed trade-off remains unclear. The paper also lacks separate cache-miss latency measurements and analysis of cache storage costs and different request similarity distributions. These factors are important for evaluating practical deployment.

Questions for authors
  1. Chorus II already supports sparse attention reuse with optional feature reuse. Why is only its sparse attention component adapted to T2V? Could the authors compare against the combined reuse approach under comparable settings, or explain why this is not feasible?

  2. How does Early Signature identify incompatible requests involving the same objects but different spatial positions or motions? If a historical object lies outside the target Active Region, how does the method prevent residual content from the historical generation?

  3. Could the authors provide larger-scale risk certification and more detailed evaluations of prompt alignment, object consistency, and motion consistency? In particular, could they explain the noticeable CLIP-T degradation on Wan2.2?

2 comments

We thank the reviewer for the careful and constructive feedback. We address each point below.

W1 & W3 & Q1 Novelty and Comparison with Chorus II

Chorus II primarily focuses on cross-request sparse-attention reuse, with downsampled latent feature reuse as an optional acceleration extension. Although it outlines their high-level combination, its implementation is not publicly available, and the construction of feature-reuse regions and their alignment with historical sparse-attention masks are not sufficiently specified for faithful reproduction. Latent reuse operates on token states, whereas sparse-attention reuse operates on query-key connectivity, requiring coordinated region selection and attention masking rather than a straightforward combination. Furthermore, Chorus relies on an external segmentation model to obtain object-region masks, while Chorus II does not specify a reproducible procedure for constructing its feature-reuse masks. These limitations prevent a faithful reproduction of the complete Chorus II configuration in our T2V setting. We therefore adapt its primary sparse-attention reuse mechanism under comparable generation settings.

CARNATOR introduces two tightly coupled methodological novelties. First, we propose a model-internal, generation-native compatibility mechanism that simultaneously supports reuse validity and reuse scope. Early Signature extracts noun-conditioned responses from the DiT's cross-attention and expands them through self-attention to obtain object-specific spatiotemporal regions, without requiring an external segmentation model. This evidence serves two purposes: it identifies compatible historical caches through quality-aware filtering and directly provides the region masks used to guide subsequent latent and attention reuse.

Second, we introduce an evidence-guided joint reuse mechanism that explicitly coordinates latent-state reuse and sparse-attention computation. For an accepted cache, latent reuse follows the region-weighted update zk+1=W⊙z^k+1q+(1−W)⊙zk+1j⋆z_{k+1} = W\odot\hat{z}^{q}_{k+1} + (1-W)\odot z^{j^{\star}}_{k+1} while historical sparse-attention connectivity is restricted to active-query and contextual-key regions in selected Transformer layers. Both mechanisms are governed by the same evidence-derived regions, ensuring that reused latent states and sparse attention computation operate over coordinated spatial domains.

This also addresses W3's first concern: Early Signature supplies the generation-native region evidence that guides joint reuse rather than independently reducing hit latency. Together, they couple reuse validity with acceleration.

W2 & Q3 Reuse Safety and Quality Evaluation

CLIP-T and VBench-Q capture text alignment and general video quality, but not whether historical content is preserved appropriately or target-specific changes are realized. Dedicated benchmarks remain limited; our 70-pair certification therefore bounds risk only under these proxy metrics. Table 2 shows Early Signature improves CLIP-T from 0.2725 to 0.2913 over text-only retrieval. Larger certification sets and tests of target correctness and historical leakage are future work.

W3 Cache-Miss Latency, Storage Cost, and Request Similarity

After rejection at the five-step probe, CARNATOR continues with 45 dense steps without restarting, so miss latency remains close to dense inference with retrieval and gate overhead. We further analyzed the storage requirements of each baseline:

MethodPer-source CacheAmortized per Matched Pair
NIRVANA-VID20.95 MB12.03 MB
Chorus561.88 MB322.43 MB
Chorus II659.64 MB378.88 MB
CARNATOR1.52 GB873.51 MB

Although CARNATOR requires more storage, historical caches are shared across requests, amortizing storage costs. The CLIP-matched pairs have a mean similarity of 0.7488 (SD = 0.0614), with 79.3% in [0.65, 0.80) and 20.7% at or above 0.80.

Q2 Spatial and Motion Compatibility

In our experiments, CARNATOR successfully reuses historical computation for moving objects, including a dog running from left to right, demonstrating that object motion does not inherently prevent reuse. However, substantial differences in motion trajectories, such as opposite running directions, can introduce incompatibility in latent reuse. The stored spatiotemporal masks retain position cues, although the current gate does not directly compare absolute positions or motion trajectories. These cues could support position- and trajectory-aware filtering.

To mitigate historical-object residuals, CARNATOR constructs its update region from the historical object's mask and spatially expands it to provide a safety margin. Target-specific computation updates this region, reducing residual content caused by imperfect localization. Nevertheless, large spatial or motion changes may exceed this region, motivating position- and trajectory-aware compatibility checks in future work.

We thank the reviewer again for the constructive comments.

Thank you for the explanations and clarifications. Your response addresses some of my questions about the Chorus II baseline, but two points are still unclear to me. On Wan2.2, CLIP-T drops from 0.3372 with dense inference to 0.2913 with Carnator. What kinds of generation failures are associated with this drop? Also, given that Early Signature lowers the cache hit rate, how much does enabling it change the average end-to-end latency across all requests, including both cache hits and misses?

Reviewer_GbhCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Carnator aims to speed up text-to-video generation by reusing computation from past requests. It first retrieves similar prompts. It then runs four denoising steps and checks the object regions and appearance in the early results. These signals form an Early Signature for cache selection. For an accepted cache, the method reuses saved latent states and sparse attention connections. It focuses updates on selected regions and recomputes target Q/K/V. Requests without a suitable cache use dense generation.Tests across several video models show lower latency. General video quality scores remain close to or above the dense baseline, but text alignment drops, especially on the main model. Ablations show that the signature filter improves quality scores. Combining latent and attention reuse further reduces latency.

Critical strengths

1.Early generation states provide a useful basis for cache selection. Similar prompts do not always produce compatible visual structures. Checking the target model’s early states is a reasonable way to address this gap. Object regions also guide selective updates after a cache is accepted. The method clearly distinguishes reusing attention connections from copying historical attention values.

Critical weaknesses

1.The speed benefit of stricter filtering is unclear. Section 4.2 links stricter filtering to greater savings per cache hit. Table 2 shows better quality, but almost no change in hit latency. The methods also differ in how they reuse the cache. The results do not show how much of the speedup comes from filtering itself.

Questions for authors

1.What supports the claim that stricter filtering saves more time per cache hit? Hit latency barely changes in Table 2.
2.Is the update region based on the target probe, the cache, or both? Section 3.3 and Appendix C.3 seem to differ from Algorithm 1.
3.How many accepted test requests show a drop of more than 0.02 from dense generation on either quality score? Do Tables 1, 3, and 5 use the same reuse settings and filter?

3 comments

We thank the reviewer for the careful and constructive feedback. We address each point below.

W1 & Q1 Early Signature and Joint Reuse

We would like to clarify that Early Signature and joint reuse play complementary yet tightly coupled roles in CARNATOR. Early Signature primarily improves reuse validity and generation quality, while joint reuse directly reduces inference computation.

Importantly, Early Signature also provides the essential region guidance for acceleration. It extracts noun-conditioned responses from cross-attention and expands them through self-attention to obtain object-specific spatiotemporal evidence. This evidence is used both for compatibility assessment and for constructing the Active and Context Regions that guide latent reuse and constrain historical sparse-attention connectivity. Unlike segmentation-based approaches such as Chorus, CARNATOR obtains these regions directly from the DiT without requiring an external segmentation model. Table 2 demonstrates the quality benefit of Early Signature, improving CLIP-T from 0.2725 to 0.2913 while maintaining comparable hit latency. Table 3 demonstrates the computational benefit of joint reuse, reducing latency to 155.85 s compared with 206.06 s for latent-only and 160.91 s for attention-only reuse.

Therefore, the two components jointly enable acceleration through shared generation-native evidence, although stricter filtering alone does not directly improve per-hit latency. We will clarify this distinction in Section 4.2.

Q2 Update Region Construction

We thank the reviewer for pointing out this ambiguity. The update regions are constructed from the object-specific Evidence Region stored in the accepted historical cache. The target Early Probe provides the generation-native evidence for compatibility assessment, while the accepted cache supplies the corresponding source Evidence Region for subsequent reuse. Specifically, the cached Evidence Region is spatially expanded to construct the Active and Context Regions, which guide boundary-aware latent blending and region-constrained sparse-attention reuse. The current pseudocode does not sufficiently distinguish the evidence used for compatibility assessment from that used for region construction. We will clarify this distinction in Algorithm 1 to make the region-construction procedure more explicit and consistent with Section 3.3 and Appendix C.3.

Q3 Quality Degradation and Evaluation Consistency

Following our discussion, we further revisited the role of the trade-off parameter (\Delta). We realized that the current fixed-(\Delta) selection mechanism may not provide a sufficiently robust and interpretable policy for balancing generation quality and inference acceleration across different requirements, suggesting that this aspect deserves further investigation. We will clarify the parameter settings and statistical evaluation in the revised manuscript.

We thank the reviewer again for the thorough review.

Thank you for the response, which addresses my concerns and clarifies the source of the updated regions. I will maintain my original score.

We sincerely thank the reviewer for maintaining the positive assessment and for the constructive discussion.

Following this discussion, we further examined the trade-off parameter (\Delta), which controls tolerated quality degradation and affects cache acceptance and inference acceleration. We also realized that the current (\Delta)-based selection mechanism may not be good for balancing quality preservation and acceleration.

We are conducting further experiments to better understand and design this quality-acceleration trade-off and investigate a more principled selection policy under different requirements. However, this investigation may take longer than the remaining discussion period. We are therefore considering withdrawing the current submission to allow sufficient time for a more thorough revision.

We sincerely appreciate the reviewer's time, constructive feedback, and consideration.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for your time and effort in reviewing our submission. As the discussion period is nearing its end, we would like to kindly ask whether there are any remaining concerns that we have not addressed. We would be happy to provide further clarification or address any additional feedback.

Thank you again for your consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.