acceptodds
Under review as a conference paper at ICLR 2027

CoFiRA: Compute–Fidelity Region Allocation for Efficient Video Generation

Abstract

Diffusion transformers have substantially advanced video generation, but their attention cost grows rapidly with the large spatiotemporal token sequences involved. Sparse attention is a promising way to alleviate this bottleneck, yet evaluated video attention remains broadly distributed even under oracle selection, suggesting diminishing returns from selector-only sparsification. Existing compensation methods recover non-exact regions with a single summary, but this binary choice between a fixed surrogate and exact attention limits the representations available for regional allocation. We introduce **CoFiRA**, a training-free framework for heterogeneous region-wise compute–fidelity allocation that couples regional representation design with compute allocation. Using query-conditioned contribution and within-cluster K/V variation, CoFiRA assigns each region a calibrated mean, a variable-capacity nested set of real K/V representatives, or exact attention, creating a scalable representation space matched to regional demand. A fidelity-aware runtime packs variable-capacity computations and schedules their irregular routes on device. On HunyuanVideo T2V 13B, CoFiRA achieves a 2.27× end-to-end speedup while reaching 33.09 dB PSNR. Code and demo samples are available at [Anonymous/CoFiRA](https://anonymous.4open.science/r/CoFiRA/).

Primary area:
generative models
Keywords:
video generation, sparse attention, diffusion transformers, inference acceleration, training-free methods

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xpSYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoFiRA, a training-free framework for region-wise compute–fidelity allocation in video diffusion transformers. Instead of restricting each region to exact attention or a single approximation, CoFiRA allocates calibrated means, variable-capacity nested sets of real K/V representatives, or exact attention according to query-conditioned contribution and within-cluster K/V variation. A two-stream runtime executes these heterogeneous representations and combines their outputs through global log-sum-exp normalization.

Critical strengths
  1. The central contribution is the combination of a multi-capacity representation space and sensitivity-guided allocation. Nested real K/V representatives provide intermediate operating points between a single centroid and exact attention. The comparisons against Exact/Mean, fixed capacities, and shuffled assignments support the value of allocating different capacities to different regions.

  2. The packed approximate stream and variable-length exact stream address the irregular workload introduced by heterogeneous capacities. Table 5 shows a substantial latency reduction from packing and device-side scheduling. Tables 10 and 15–17 further quantify representative construction, component latency, and memory usage, providing useful evidence that the added machinery remains practical.

Critical weaknesses
  1. Table 10 shows that residual first-moment selection improves attention-output MSE by approximately 2.2% over uniform real-token sampling, while increasing construction latency from 5.42 to 10.06 ms. The end-to-end cost increase is small, but the corresponding final-video fidelity comparison is not reported there. Thus, the evidence strongly supports the representation-and-allocation framework, while the practical advantage of the particular nested selection rule remains less clear.

  2. Sensitivity to the representation ladder and calibration parameters remains insufficiently explored. Table 6 establishes an aggregate capacity–error trend, and Table 9 supports adaptive allocation over fixed-capacity alternatives. However, these experiments do not establish robustness to changing the ladder itself, its maximum capacity, or its granularity. Similarly, Table 12 demonstrates the benefit of enabling mean-mass calibration, but does not show sensitivity to its coefficient and clipping threshold. A small parameter study would clarify how much tuning is required across models.

Questions for authors
  1. Under identical routes and capacities, how do uniform sampling and residual first-moment selection compare in final-video PSNR, SSIM, and LPIPS? Are the differences consistent across prompts?

  2. How do alternative capacity ladders and maximum capacities affect the quality–latency trade-off under comparable compute budgets? How sensitive are results to the mean-calibration coefficient and clipping threshold?

  3. Can the authors explain the Wan T2V discrepancy between improved paired-to-dense fidelity and lower VBench macro scores relative to SVOO and SVG-EAR? Are the differences concentrated in particular quality dimensions or content categories?

  4. If a compatible implementation was available before submission, could the authors include a comparison with a residual-reconstruction method such as SparsePR, or explain the practical barriers to a fair comparison?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xpSY (1/2)

Thank you for recognizing the representation, allocation, and runtime contributions. We address the selector comparison, parameter sensitivity, VBench dimensions, and SparsePR below.

Q1. Residual matching versus Uniform across prompts.

Thank you for suggesting a comparison between residual matching and simpler uniform selection. This is a useful control for understanding how much final-video fidelity depends on the representative selection rule itself. We therefore compared the two selectors across five prompts while keeping the rest of the method unchanged.

Before presenting the results, we would like to clarify the role of the selector in CoFiRA. Our core contribution is to provide multiple regional representation capacities and allocate them according to regional demand. The controlled ablations in Tables 2, 4, and 9 support this design as the main source of the fidelity gains. Residual matching and uniform selection differ in how they construct representatives for a given capacity, but both retain the same representation levels and demand-aware allocation policy. The comparison below thus isolates the role of the selector, while CoFiRA's core contribution is unchanged under either choice.

SelectorPSNR ↑SSIM ↑LPIPS ↓
Residual matching25.890.8580.140
Uniform selection25.960.8560.142

Uniform selection yields slightly higher mean PSNR, while residual matching yields slightly better mean SSIM and LPIPS. The differences are small on all three metrics, indicating comparable final-video fidelity. Residual matching is also efficient: it constructs one ordered sequence and reuses its prefixes across all capacities. In the end-to-end benchmark, the generation times differ by only about 1% (Table 10).

At comparable video fidelity and end-to-end runtime, residual matching offers stronger worst-case control of the first-moment error in the normalized K/V sketch space. Let HH be the maximum distance from a token to its cluster mean in this space. When at most half of the cluster's tokens are selected, fixed-position uniform selection can incur a squared first-moment error of H2H^2 in the worst case. In contrast, the deterministic bound in Appendix A.3 for a greedy residual-matching prefix of size mm is at most H2/mH^2/m. This bound holds regardless of token order and tightens as capacity increases.

Table 10 provides complementary empirical evidence: with attention inputs, routes, and capacities held fixed, residual matching reduces attention-output MSE by 2.2%, from 9.11 × 10⁻⁴ to 8.91 × 10⁻⁴. Given comparable video fidelity and generation time, the stronger first-moment guarantee and lower measured local error motivate our default choice. Both selectors preserve CoFiRA's core design: allocating representation capacity according to regional demand.

Q2. Capacity ladder, maximum capacity, and calibration sensitivity.

Thank you for asking about sensitivity to the capacity ladder and mean-mass calibration. We address these two settings separately below.

Capacity allocation. The 2/4/8/16 ladder supplies intermediate capacities through prefixes of one nested construction. Equal quartiles are a simple rank-based default for assigning these capacities to regions ordered by sensitivity.

We tested an alternative proportion allocation and ladders with maximum capacities 8 and 32 on one full video. The alternative 16/8/4/2 proportions of 30/15/20/35% preserve the default's nominal average capacity of 7.5 representatives. All variants use the same generation settings and attention backend. Exact regions are retained, and intermediate coverage is adjusted to fit the default row-wise extra representative-interaction cap; larger capacities generally cover fewer regions.

Capacity allocationPSNR ↑SSIM ↑LPIPS ↓
2/4/8/16, equal quarters25.290.8300.149
16/8/4/2, proportions 30/15/20/35%25.220.8320.145
2/4/8, equal thirds24.990.8170.149
2/4/8/16/32, equal fifths25.210.8350.143

The alternative proportions give similar fidelity on this video. The 2/4/8 setting yields lower PSNR/SSIM; the 2/4/8/16/32 setting yields better SSIM/LPIPS and slightly lower PSNR than the default. These results support 2/4/8/16 as a practical default among the tested alternatives. This is a fidelity comparison under a common interaction-budget cap; no per-ladder latency measurements are reported.

AuthorsOpenReview ↗

Response to Reviewer xpSY (2/2)

Mean-mass calibration. We use alpha=1 and clipping threshold 2. In the original Table 12, disabling calibration increases mean attention-output MSE from 6.750 × 10⁻⁴ to 7.217 × 10⁻⁴, an increase of 6.92% relative to the calibrated default, with Q/K/V, clustering, routes, and other approximations held fixed.

Separately, the fixed-state coefficient/clipping sensitivity check gives the following changes, each relative to the default within that sensitivity check:

AlphaClipping thresholdChange in mean output MSE
0.52+2.47%
120%
1.52−0.27%
11+0.04%
140%

These parameter changes have small effects on local output MSE around the chosen default in the evaluated states.

Q3. Wan T2V VBench discrepancy.

Thank you for highlighting the difference between the Wan T2V fidelity and VBench results. The goal of this task is to accelerate the original video generator while reproducing its dense outputs as closely as possible. In this setting, training-free sparse attention methods generally aim to approximate dense attention, and PSNR is a primary metric for evaluating how closely the resulting videos reproduce their dense references. VBench serves a different purpose: it evaluates properties of the completed video without using the paired dense output as a reference. Because the score is computed on the completed video, it does not provide direct guidance on which interactions to retain or approximate in an individual attention call. Dense attention, in contrast, provides a direct reference for those local decisions. Exact reproduction would preserve the dense video's VBench scores under the same evaluation, but higher reconstruction fidelity does not imply monotonically higher VBench scores.

VBench dimensionCoFiRASVOOSVG-EAR
Subject consistency95.7795.7095.70
Background consistency96.1996.2196.18
Motion smoothness98.7198.7198.71
Dynamic degree41.3343.1742.83
Aesthetic quality59.5359.6959.65
Imaging quality67.7468.1468.02
Six-dimension macro76.5576.9476.85

On Wan T2V, CoFiRA outperforms SVOO and SVG-EAR on all three paired-reference metrics (PSNR, SSIM, and LPIPS), while its six-dimension VBench macro is slightly lower. Table 13 shows that most of the gap comes from dynamic degree, which accounts for approximately 0.31 of the 0.39-point gap to SVOO and 0.25 of the 0.30-point gap to SVG-EAR. On Wan I2V, CoFiRA achieves both higher reconstruction fidelity and the highest VBench macro among the evaluated sparse methods: 81.14, compared with 81.00 for SVG-EAR (Table 1).

Q4. SparsePR.

Thank you for suggesting a comparison with a residual-reconstruction method. We added a comparison with the official SparsePR implementation on the jointly supported HunyuanVideo-13B backbone. Both methods use the same weights, prompts, sampler, guidance settings, video export, and paired Dense-reference pipeline, retaining their respective default operating points. Mean results over five prompts are:

MethodPSNR ↑SSIM ↑LPIPS ↓
SparsePR25.800.8350.174
CoFiRA30.010.9070.120

CoFiRA improves PSNR by 4.21 dB, SSIM by 0.072, and LPIPS by 0.054, with all three metrics improving on every prompt. These are paired-reference fidelity results at the stated operating points.

The author has addressed most of my questions; I am maintaining my current score and am inclined to accept this paper.

Thank you for taking the time to read our response and share your follow-up. We appreciate your thoughtful feedback and are glad that our response addressed most of your questions. If any further questions come up, we would be happy to discuss them.

Reviewer_8MzFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoFiRA is a training-free attention approximation framework for accelerating video DiTs. Instead of choosing only between exact attention and a single cluster-mean approximation, it assigns semantic query–key regions different representation capacities: a calibrated mean, a nested subset, or exact attention. Allocation combines estimated attention contribution with within-cluster K/V variation. Representatives are constructed through residual first-moment matching, while a noise-dependent schedule redistributes the exact-attention budget across denoising steps. A packed approximate stream and a variable-length exact stream are combined through global normalization. The paper evaluates several models and reports 1.75–2.27× end-to-end speedups over dense model and improved reconstruction metrics over sparse baselines. Validation includes capacity-allocation controls, representative-construction comparisons, routing diagnostics, runtime breakdowns, and prompt-level confidence intervals. The theoretical analysis explains limitations of mean-only representations and what the representative-selection procedure controls, while acknowledging the gap between first-moment matching and nonlinear attention accuracy.

Critical strengths
  • Intermediate representations directly address the stated limitation of exact/mean compensation. The method provides additional capacity for regions that cannot be represented adequately by one mean. Table 2 compares this design with binary and fixed-capacity alternatives at comparable computation. The resulting fidelity improvements align with the paper’s central motivation.

  • The main allocation controls support assigning different capacities to different regions. Table 4 tests whether the placement of computation matters beyond the available capacity levels. Shuffling intermediate capacities degrades fidelity while preserving their nominal distribution. This supports sensitivity-guided placement as a meaningful component of the method.

  • The execution design addresses the main systems obstacle introduced by heterogeneous capacity. Variable representative counts create irregular workloads that can erode theoretical savings. The packed approximate stream and device-side scheduling directly target this problem. The runtime ablation and end-to-end comparisons connect the representation design to practical acceleration.

Critical weaknesses
  • The benefit of the specific representative construction is not fully established. While the experiments support adaptive capacity allocation, the proposed selector offers only a modest attention-level improvement over simple uniform sampling. Its benefit for final-video fidelity remains unclear.
  • Configuration robustness is insufficiently demonstrated. The method depends on several model-specific allocation and clustering settings, but their selection process and sensitivity are not fully documented.
Questions for authors
  • With semantic partitions, computation, and execution held fixed, what advantage do CoFiRA’s representations provide over a pooled multilevel hierarchy?

  • With routes and capacities fixed, does residual first-moment matching improve final video fidelity over simpler real token representative selection?

  • How were the main configuration parameters selected?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 8MzF (1/2)

Thank you for recognizing the multi-capacity representations, sensitivity-guided allocation, and execution design. We address the three questions below.

Q1. Controlled comparison with a shared pooled hierarchy.

Thank you for suggesting a shared pooling control to isolate the effect of the intermediate representation. We implemented this hierarchy inside the existing semantic K clusters: 16 balanced bins in cluster-member order, coarsened to 8/4/2 bins by cardinality-weighted averaging, without additional clustering. Each pooled representative carries the attention mass of its constituent bin, using the actual bin cardinality in its logit bias. This gives a low-cost hierarchy shared across capacity levels, distinct from the separately fitted subcluster centroids in Table 10.

The five-prompt comparison uses the same generation settings and semantic partitions, with regional routing, allocated capacities, calibration, and the representative-attention backend held fixed. Only the intermediate constructor changes:

Intermediate representationPSNR ↑SSIM ↑LPIPS ↓
Shared pooled hierarchy24.930.8200.177
CoFiRA real K/V representatives25.890.8580.140

CoFiRA improves PSNR by 0.96 dB, SSIM by 0.038, and LPIPS by 0.037, with all three metrics improving on every prompt. This supports the value of the real-K/V intermediate representation relative to this shared pooling control under the common allocation and execution setup.

Q2. Representative selection and final-video fidelity.

Thank you for suggesting a comparison between residual matching and simpler uniform selection. This is a useful control for understanding how much final-video fidelity depends on the representative selection rule itself. We therefore compared the two selectors across five prompts while keeping the rest of the method unchanged.

Before presenting the results, we would like to clarify the role of the selector in CoFiRA. Our core contribution is to provide multiple regional representation capacities and allocate them according to regional demand. The controlled ablations in Tables 2, 4, and 9 support this design as the main source of the fidelity gains. Residual matching and uniform selection differ in how they construct representatives for a given capacity, but both retain the same representation levels and demand-aware allocation policy. The comparison below thus isolates the role of the selector, while CoFiRA's core contribution is unchanged under either choice.

SelectorPSNR ↑SSIM ↑LPIPS ↓
Residual matching25.890.8580.140
Uniform selection25.960.8560.142

Uniform selection yields slightly higher mean PSNR, while residual matching yields slightly better mean SSIM and LPIPS. The differences are small on all three metrics, indicating comparable final-video fidelity. Residual matching is also efficient: it constructs one ordered sequence and reuses its prefixes across all capacities. In the end-to-end benchmark, the generation times differ by only about 1% (Table 10).

At comparable video fidelity and end-to-end runtime, residual matching offers stronger worst-case control of the first-moment error in the normalized K/V sketch space. Let HH be the maximum distance from a token to its cluster mean in this space. When at most half of the cluster's tokens are selected, fixed-position uniform selection can incur a squared first-moment error of H2H^2 in the worst case. In contrast, the deterministic bound in Appendix A.3 for a greedy residual-matching prefix of size mm is at most H2/mH^2/m. This bound holds regardless of token order and tightens as capacity increases.

Table 10 provides complementary empirical evidence: with attention inputs, routes, and capacities held fixed, residual matching reduces attention-output MSE by 2.2%, from 9.11 × 10⁻⁴ to 8.91 × 10⁻⁴. Given comparable video fidelity and generation time, the stronger first-moment guarantee and lower measured local error motivate our default choice. Both selectors preserve CoFiRA's core design: allocating representation capacity according to regional demand.

AuthorsOpenReview ↗

Response to Reviewer 8MzF (2/2)

Q3. How configurations were selected.

Thank you for asking how we selected the configurations. We first used preliminary experiments to examine how the main parameters affect fidelity and runtime. Within the tested ranges, increasing Q/K cluster counts generally improved paired-reference fidelity while increasing clustering, routing, and execution overhead. Increasing the exact-attention budget or intermediate-region coverage likewise generally improved fidelity at additional runtime cost.

Guided by these observed trends, we progressively adjusted clustering granularity and representation budgets, using paired-reference fidelity and measured end-to-end generation time to select a practical quality–runtime operating point for each model. Once selected, each model's configuration was kept fixed across all evaluated prompts. The resulting configurations are listed in Table 7.

Thank you again for taking the time to review our paper. Your feedback has helped us identify ways to make the paper clearer and stronger. If you have any further questions or if anything in our response could use clarification, please feel free to let us know. We would be happy to address any points you raise.

Reviewer_ZqTxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CoFiRA is a training-free method for accelerating video diffusion through region-wise allocation of attention computation. It assigns each query–key region a calibrated mean, a variable number of real K/V representatives, or exact attention, using attention contribution and within-cluster K/V variation to guide the allocation. A nested representative sequence supports several capacities, while a packed approximate stream and a separate exact stream are combined through global normalization. Experiments on Wan2.1 T2V/I2V and HunyuanVideo report closer agreement with dense-model videos than the evaluated prior works and 1.75–2.27× generation speedups over the dense baseline.

Critical strengths
  1. The reported results show useful gains in preserving the original model’s outputs while reducing generation time.

  2. Controlled ablations and detailed runtime analysis provide substantial support for the regional-allocation approach and its implementation.

Critical weaknesses
  1. The benefits of the proposed representations over existing multi-level representations and simpler selection methods need clearer evidence.

  2. The rationale for the fixed capacity-allocation policy remains insufficiently explained.

Please see the comments for more details.

Questions for authors

Thank you for your submission to ICLR. Here are my comments/questions.

  1. PSA mentioned in this paper already allocates different pooled K/V resolutions to query–key regions. This paper acknowledges this distinction, but the full-method comparison also changes token grouping, routing, and execution, making it difficult to isolate the benefit of real K/V representatives. Table 10 evaluates separately fitted subcluster centroids, which does not cover a cheap, shared pooling hierarchy. A control using such a hierarchy under the same semantic partitions and allocation would help clarify the specific contribution of the proposed representation design.

  2. Appendix A.1 divides the highest-ranked regions into four groups and assigns 16, 8, 4, and 2 representatives. Before exact promotion, this fixes the nominal capacity distribution regardless of how sensitivities are distributed within a query-cluster row. Table 9 and the fixed-histogram analysis support assigning more capacity to higher-ranked regions, but do not explain why this particular distribution matches regional demand. It would be helpful to justify these capacities and proportions and compare an alternative distribution at comparable executed computation.

  3. In Table 10, uniformly spaced real K/V selection has only 2.2% higher attention-output MSE than residual matching, while reducing generation time by 1.2%. Since nested representative construction is a stated contribution, it would be helpful to report the final-video PSNR, SSIM, and LPIPS for this simpler alternative under the same allocated capacities. This would clarify whether residual matching provides a meaningful generation-level benefit beyond the multi-rate allocation itself.

  4. Sol-Attn [1] fuses exact attention and approximate correction into a single online-softmax pass, whereas CoFiRA supports multiple representation capacities through a packed approximate stream and a separate exact stream. Comparing their quality–latency trade-offs would help establish the practical value of CoFiRA’s additional representational flexibility.

[1] Li, Haopeng, et al. “Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification.” arXiv:2607.24027, 2026.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZqTx (1/2)

Thank you for recognizing CoFiRA's fidelity gains, controlled ablations, and runtime analysis. We address each question below.

Q1. Cheap shared pooling hierarchy.

Thank you for pointing out that the full-method comparison makes it difficult to separate the effect of the intermediate representation from differences in token grouping, routing, and execution. Comparing a shared pooling hierarchy within CoFiRA's semantic partitioning and allocation scheme provides a more direct test of the representation choice, so we added this control.

Within each existing semantic K cluster, we form 16 balanced bins in cluster-member order, then merge them into 8, 4, and 2 bins by weighting each average by its token count, without additional clustering. Each pooled representative uses its bin's actual token count in the logit bias. This gives a low-cost pooling hierarchy shared across capacity levels, distinct from the subcluster centroids fitted separately for each capacity in Table 10.

The five-prompt comparison uses the same generation settings and semantic partitions, with regional routing, allocated capacities, calibration, and the representative-attention backend held fixed. Only the intermediate constructor changes:

Intermediate representationPSNR ↑SSIM ↑LPIPS ↓
Shared pooled hierarchy24.930.8200.177
CoFiRA real K/V representatives25.890.8580.140

CoFiRA improves PSNR by 0.96 dB, SSIM by 0.038, and LPIPS by 0.037, with all three metrics improving on every prompt. This supports the value of the real-K/V intermediate representation relative to this shared pooling control under the common allocation and execution setup.

Q2. Capacity levels and fixed proportions.

Why fixed levels and proportions? Thank you for raising this question. Adapting capacity levels or proportions to each query-cluster row could better match regional demand. Doing so based on marginal gains would require estimating approximation errors at multiple capacities or running an additional gain predictor online. The goal of video generation acceleration is to reduce end-to-end generation time while preserving the original model's output as closely as possible. The cost of making allocation decisions therefore matters: even if a finer-grained policy improves reconstruction, its overhead can offset the time saved by sparse attention. We therefore use fixed capacity levels and nominal proportions to keep allocation inexpensive, while using the current sensitivity ranking to decide which regions receive each capacity.

Why four capacity levels? The geometric 2/4/8/16 ladder spans an eightfold capacity range with four levels, all exposed as prefixes of one nested construction. Equal quartiles provide a simple default allocation over this representation space.

On one full video, we compared three-, four-, and five-level ladders and tested an alternative allocation of proportions within the four-level ladder, using the same generation settings and attention backend. The alternative 16/8/4/2 proportions of 30/15/20/35% preserve the default's nominal average capacity of 7.5 representatives. At each call, we retain the current-state exact regions and adjust the number of intermediate regions so that each row's extra representative interactions do not exceed the default budget for that state. Larger capacities therefore generally cover fewer intermediate regions.

Capacity allocationPSNR ↑SSIM ↑LPIPS ↓
2/4/8/16, equal quarters25.290.8300.149
16/8/4/2, proportions 30/15/20/35%25.220.8320.145
2/4/8, equal thirds24.990.8170.149
2/4/8/16/32, equal fifths25.210.8350.143

Changing the allocation proportions gives closely comparable fidelity in this test. Removing the 16-representative level lowers PSNR by 0.30 dB and SSIM by 0.013. Adding a fifth level with 32 representatives yields very similar results across all three metrics. Under the tested interaction budget, removing one level reduces fidelity, while adding one offers limited additional benefit. These results support the four-level 2/4/8/16 ladder as a practical default.

AuthorsOpenReview ↗

Response to Reviewer ZqTx (2/2)

Q3. Why residual matching rather than Uniform?

Thank you for suggesting a comparison between residual matching and simpler uniform selection. This is a useful control for understanding how much final-video fidelity depends on the representative selection rule itself. We therefore compared the two selectors across five prompts while keeping the rest of the method unchanged.

Before presenting the results, we would like to clarify the role of the selector in CoFiRA. Our core contribution is to provide multiple regional representation capacities and allocate them according to regional demand. The controlled ablations in Tables 2, 4, and 9 support this design as the main source of the fidelity gains. Residual matching and uniform selection differ in how they construct representatives for a given capacity, but both retain the same representation levels and demand-aware allocation policy. The comparison below thus isolates the role of the selector, while CoFiRA's core contribution is unchanged under either choice.

SelectorPSNR ↑SSIM ↑LPIPS ↓
Residual matching25.890.8580.140
Uniform selection25.960.8560.142

Uniform selection yields slightly higher mean PSNR, while residual matching yields slightly better mean SSIM and LPIPS. The differences are small on all three metrics, indicating comparable final-video fidelity. Residual matching is also efficient: it constructs one ordered sequence and reuses its prefixes across all capacities. In the end-to-end benchmark, the generation times differ by only about 1% (Table 10).

At comparable video fidelity and end-to-end runtime, residual matching offers stronger worst-case control of the first-moment error in the normalized K/V sketch space. Let HH be the maximum distance from a token to its cluster mean in this space. When at most half of the cluster's tokens are selected, fixed-position uniform selection can incur a squared first-moment error of H2H^2 in the worst case. In contrast, the deterministic bound in Appendix A.3 for a greedy residual-matching prefix of size mm is at most H2/mH^2/m. This bound holds regardless of token order and tightens as capacity increases.

Table 10 provides complementary empirical evidence: with attention inputs, routes, and capacities held fixed, residual matching reduces attention-output MSE by 2.2%, from 9.11 × 10⁻⁴ to 8.91 × 10⁻⁴. Given comparable video fidelity and generation time, the stronger first-moment guarantee and lower measured local error motivate our default choice. Both selectors preserve CoFiRA's core design: allocating representation capacity according to regional demand.

Q4. Comparison with Sol-Attn

Thank you for suggesting a direct comparison with Sol-Attn. This comparison helps clarify how the two attention designs trade off fidelity and runtime in practice. Sol-Attn is contemporaneous work first posted on arXiv on July 27, 2026, while CoFiRA was under development. We had not completed a direct comparison at the time of submission. We have now evaluated both methods in the same Wan generation pipeline.

We used the official Sol-Attn kernel with Morton ordering and the released code's default threshold, τ=1.0. Both methods used 50 denoising steps, one dense layer, and the same 15-step dense warm-up. Fidelity metrics are averaged over five prompts.

MethodPSNR ↑SSIM ↑LPIPS ↓Generation time (s) ↓
Sol-Attn, τ=1.023.510.8020.1841049
CoFiRA25.890.8580.1401037

At nearly identical end-to-end generation times, CoFiRA matched the dense reference more closely on all three metrics, with 2.38 dB higher PSNR, 0.056 higher SSIM, and 0.044 lower LPIPS. Overall, CoFiRA offers a better fidelity–latency trade-off in this comparison.

Thank you again for taking the time to review our paper. Your feedback has helped us identify ways to make the paper clearer and stronger. If you have any further questions or if anything in our response could use clarification, please feel free to let us know. We would be happy to address any points you raise.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary and Thanks to the Reviewers

As the author–reviewer discussion period draws to a close, we would like to briefly summarize our response and express our appreciation to all reviewers. In response to your feedback, we added controlled representation and selector comparisons, parameter-sensitivity studies, and comparisons with Sol-Attn and SparsePR. We also clarified the configuration choices and the interpretation of our evaluation metrics.

We sincerely appreciate your constructive suggestions, which have helped us improve the paper. Thank you for the time and care you have devoted to reviewing our work and considering our responses. We are grateful for your thoughtful feedback and contributions to this discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.