acceptodds
Under review as a conference paper at ICLR 2027

Perception-Action Conditioned Efficient Inference for Interactive Video World Models

Abstract

Interactive video world models enable users and agents to continuously explore generated environments via action-conditioned visual generation, benefiting applications in game simulation, autonomous driving, and embodied intelligence. Nevertheless, the repeated Diffusion Transformer (DiT) denoising conditioned on the long visual history incurs substantial latency, severely hindering responsive interaction. Existing approaches decrease the latency through reducing redundant computations in generated context, attention and denoising, by exploring the relevance between historically generated contents and the one under the current action. However, they generally rely on limited guidance signals for compression and adopt fixed reduction schemes, overlooking perceptual redundancy and dynamiclly varying computation demands, thus suffering significant performance degradation. To address this issue, we propose Perception–Action Conditioned Efficient Inference (PACE), a training-free framework that synergizes the perceived visual evidence with action cues to dynamically allocate computation. Specifically, we firstly develop the Block-wise Context Compression method to filter historical blocks based on visual redundancy and target-view relevance. Subsequently, we employ Action-Conditioned Attention Sparsity by optimally distributing attention budgets between past and current tokens. Besides, we design the Interruption-Aware Cache Reuse strategy, by scheduling feature reuse based on visual stability while conservatively adjusting during abrupt action transitions. On HY-WorldPlay and Matrix-Game-3.0, our method achieves 2.77× and 1.74× DiT-core speedup, respectively, remarkably promoting both inference latency and peak memory overhead with improved generation fidelity in most cases, compared to the state-of-the-art approaches. We will release source code upon acceptance.

Primary area:
generative models
Keywords:
interactive video world models, efficient inference, diffusion transformers, sparse attention, diffusion caching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_NsF1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the high inference latency of interactive video world models with a training-free framework that jointly conditions computation allocation on perceived visual evidence and action cues. The central insight—that visual history reveals redundancy while the action signal indicates whether it remains exploitable—is sound and well-articulated. The three components (block-wise context compression, action-conditioned attention sparsity, interruption-aware cache reuse) are clearly motivated against concrete limitations of prior work. Results (2.77×/1.74× DiT-core speedups with improved fidelity and revisit consistency) are convincing on two backbones. Concerns include reliance on offline calibration that may not transfer across models, and the gap between DiT-core and end-to-end speedup (2.77× vs 2.15×) due to fixed VAE overhead. The absence of released code is a minor reproducibility concern. A solid, clearly-scoped contribution to efficient world-model inference.

Critical strengths

1.Visual history reveals the redundancy, while the upcoming action reveals where the model is heading. PACE provides a causally valid look-ahead signal for interactive world model that generic autoregressive accelerators do not have.

2.The paper cleanly separates DiT inference cost into sequence length n, attention interactions A, and number of forward passes S, and assigns exactly one mechanism to each (BCC, ACSA, ICR). This makes the speedup interpretable rather than a bag of tricks, and the training-free design lets it plug into existing backbones without retraining.

3.Evaluation covers two architecturally different backbones with 400 generations, reports wall-clock latency (not just FLOPs) alongside an end-to-end component breakdown, and the ablations isolate both each module and each internal design choice. Notably, PACE improves PSNR over Light Interaction at comparable latency (26.21 vs. 24.81 dB) .

Critical weaknesses

1.More evaluations on longer or shorter memory trunks are needed. All latency and memory numbers are averaged over fixed-length rollouts of roughly 15–16 chunks, so it is unknown whether the 2.77× speedup holds as the interaction history keeps accumulating. Since long-horizon interaction is the central use case of interactive world models, like in Longbot-World [1] and other long-horizon models.

2.The action signal is limited to camera geometry and is only tested on preset trajectories. Actions is injected solely as target camera pose and FoV, reduced to a binary exploration/revisitation state, leaving keyboard, object-interaction, and physics actions uncovered. Moreover, experiments uses only two fixed camera trajectories and the appendix states the exploration/revisitation schedule is precomputed offline, so robustness to more free-form user input is unverified.

3.The advertised 2.77× is DiT-core latency excluding VAE decoding, while the full end-to-end speedup is only 2.15×; the paper should make this distinction more prominent. Also, SVG and BSA become dramatically slower than Original on Matrix-Game 3.0 (0.22× and 0.15×), and the result is counterintuitive and unexplained.

[1] Advancing open-source world models. arXiv preprint arXiv:2601.20540

Questions for authors

What is the performance of the proposed method on different memory size? What is the behavior of the method on other types of user inputs, like object-interactions or so?

3 comments

Dear Reviewer, we sincerely thank you for the valuable comments. We organize our responses into 2 sections to address your concerns regarding robustness under different interaction history lengths and diverse user inputs (W1–W2 & Q1–Q2), as well as the inference speedup results and the performance degradation of baselines (W3).

(1/2) W1–W2 & Q1–Q2: On Robustness under Different Interaction Lengths and Diverse Actions.

1) Performance under different interaction history lengths (W1 & Q1).

Thanks for the concern regarding PACE’s behavior under growing interaction history. We would like to first clarify that the original experiments adopt fixed-length rollouts to exactly follow the evaluation protocol of Light Interaction, ensuring fair comparisons with existing acceleration methods. Structurally, PACE’s speedup does not diminish as the history accumulates. Both backbones maintain a bounded historical context (sink, selected history, and recent frames) rather than an ever-growing KV cache, so the per-chunk cost of Original, and hence the per-chunk savings of PACE, remain stable as the rollout extends. To verify this, we conduct additional experiments on WBench [1], whose cases contain 2–6 interaction rounds, corresponding to 12–36 generated chunks on HY-WorldPlay and 14–42 on Matrix-Game 3.0, i.e., more than twice the length of the original rollouts. As shown in Tab. R4, PACE achieves 3.11× and 1.76× DiT-core speedups, even higher than the 2.77× and 1.74× under the original fixed-length protocol, while outperforming Original in WBench overall scores (76.08 v.s. 75.84 and 55.02 v.s. 53.79). Moreover, compared with Light Interaction, PACE achieves lower latency and higher agreement with Original on both models.

Table R4: Comparison on the WBench navigation split. PSNR, SSIM, and LPIPS measure agreement with Original outputs. The best results within each backbone are shown in bold.

MethodPSNR ↑SSIM ↑LPIPS ↓DiT-core Latency (s) ↓Speedup ↑WBench Avg. ↑
HY-WorldPlay
Original–––176.081.00×75.84
Light Interaction20.310.51210.291560.852.89×76.05
PACE (Ours)20.760.52430.290256.633.11×76.08
Matrix-Game 3.0
Original–––25.111.00×53.79
Light Interaction16.420.49430.394514.991.68×52.97
PACE (Ours)18.140.55370.332714.251.76×55.02

2) Diverse actions and flexible user inputs (W2 & Q2).

Firstly, for more diverse camera actions, we evaluate PACE on the full navigation split of WBench [1]. This split contains 158 multi-turn navigation cases, covering six trajectory types: round-trip, progressive, repeat, L-shape, loop, and zigzag. Among them, the L-shape, loop, and zigzag trajectories involve continuous direction changes and scene revisits, enabling the evaluation of PACE's applicability under more complex camera motion patterns. As shown in Tab. R4 above, PACE achieves 3.11× and 1.76× DiT-core speedups on HY-WorldPlay and Matrix-Game 3.0, respectively, while maintaining good generation quality.

Secondly, for interaction scenarios beyond camera navigation, we adapt PACE to Zing-0.5 [2], which supports joint control with keyboard actions and online text instructions. As shown in Fig. 5, we further conduct a case study that generates a 60-second multi-turn interaction sequence, which involves object state changes (a stone statue opening its eyes and glowing) and character behaviors (moving forward, moving left, moving right, kneeling, standing up, jumping, and floating).

PACE successfully renders these object state changes and character actions, while, as shown in Tab. R5, reducing the DiT-core latency from 101.89 s to 43.83 s, achieving a 2.325× speedup. Notably, in the final stage of this case, Original generates an extra statue face and fails to render the character floating, whereas PACE avoids this visual artifact and successfully renders the floating action. This qualitative case demonstrates the feasibility of applying PACE to non-camera interaction scenarios. Detailed results are provided in Fig. 5, Tab. 9, and Appendix E.

Table R5: DiT-core latency and speedup on Zing-0.5 for a 60-second multi-turn interaction sequence.

MethodDiT-core Latency (s) ↓Speedup ↑
Original101.891.00×
PACE (Ours)43.832.325×

Overall, these supplementary experiments extend the evaluation of PACE beyond predefined camera trajectories, further demonstrating its applicability to diverse multi-turn navigation trajectories and non-camera interaction behaviors. We have added the relevant experimental results to Fig. 5, Tab. 2, Tab. 9, and Appendix E of the revised manuscript.

[1] WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation. arXiv’26.

[2] Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control. arXiv’26.

(2/2) W3: On Inference Speedup Results and Performance Degradation of Baselines.

1) Distinction between DiT-core and end-to-end speedups.

We agree that the distinction between the DiT-core speedup and the end-to-end speedup should be made more prominent. Therefore, we have reported both the 2.77× DiT-core speedup and the 2.15× end-to-end speedup on HY-WorldPlay in the abstract of the revised manuscript, making the statement of PACE's practical inference efficiency gains clearer.

2) Analysis of the slowdown of SVG and BSA on Matrix-Game 3.0.

Thanks for pointing this out. The slowdown stems from the limited room for attention sparsification on Matrix-Game 3.0, together with the layout-adaptation overhead of existing sparse-attention implementations. Firstly, attention sparsification alone has a low speedup ceiling on Matrix-Game 3.0. With a wide backbone (hidden dimension 5120) and short attention sequences, attention accounts for only 31%–36% of the DiT FLOPs. Under the 31% retention ratio, even a zero-overhead sparse attention is bounded at 1.27×–1.33× DiT speedup, whereas PACE reaches 1.74× by additionally compressing the context and reusing denoising computation. In contrast, attention accounts for 37%–76% of the DiT FLOPs on HY-WorldPlay, where BSA does achieve a 1.29× speedup. Secondly, the block layouts of SVG and BSA do not match Matrix-Game 3.0. Its 22×4022\times40 latent grid cannot be directly partitioned into the 4×4×84\times4\times8 blocks of BSA, requiring padding, valid-token extraction, and layout restoration; SVG similarly requires token extraction, QKV layout conversion, and per-head pattern selection. Our profiling shows that these operations add about 206 ms (BSA) and 124 ms (SVG) per attention call, far exceeding the theoretical saving of about 8 ms per call. Therefore, the slowdown reflects the strong dependence of kernel-level sparse attention on model-specific layouts and FLOP composition, whereas PACE operates at the context and cache levels and transfers across backbones more readily. We have added this analysis to the revised manuscript.

Thanks for replies from the authors. My concerns are largely solved, in terms of memory-sized inference stability and acceleration stability. No need to change the rating.

Reviewer_anXhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACE, a training-free framework for accelerating interactive video world models. It uses visual history and the upcoming camera action to remove less useful history, reduce attention computation, and reuse some denoising results. PACE reports 2.77x and 1.74x DiT-core speedups on HY-WorldPlay and Matrix-Game 3.0. On HY-WorldPlay, it also achieves a 2.15x end-to-end speedup and reduces peak memory. The results are promising, but the method's generality and evaluation setup need further clarification.

Critical strengths

The motivation is clear, and the three components address different sources of inference cost. Combining visual history with the upcoming action is intuitive and well motivated. The paper reports real latency, memory, and end-to-end speedup instead of only theoretical computation. The ablation studies help show how each component contributes to the final speedup. On HY-WorldPlay, PACE gives a strong efficiency-quality trade-off compared with the closest baseline.

Critical weaknesses

The experiments only use two predefined camera trajectories, so it is unclear whether the method works for more general actions or interactions. The full method is not applied in the same way on Matrix-Game 3.0, which weakens the cross-model validation. Although the method is called training-free, it still requires model-specific offline calibration. The cost of this calibration is not clearly reported. The current metrics mainly measure similarity and revisit consistency. They do not directly test whether actions are followed correctly or whether the generated world state is accurate.

Questions for authors
  1. How well does PACE work for actions beyond camera movement, such as object interaction or character control?
  2. Why are BCC and ICR implemented differently on Matrix-Game 3.0? Can the authors evaluate the complete method on another backbone?
  3. How much data, time, and computation are required for offline calibration? Does calibration need to be repeated for every model or setting?
  4. Were all baselines tested with the same models, inputs, random seeds, hardware, and metric code? Which results were rerun by the authors?
  5. Can the authors include metrics for action following or world-state correctness, rather than only similarity to the original output?
  6. Can the authors report result variance and explain why VBench decreases on Matrix-Game 3.0?
4 comments

Dear Reviewer, we sincerely thank you for the insightful and constructive comments. We organize our responses into 3 sections to address your concerns regarding generalization and cross-model applicability of the method (W1–W2 & Q1–Q2), offline calibration cost and experimental fairness (W3 & Q3–Q4), as well as evaluation reliability and generation quality (W4 & Q5–Q6).

(1/3) W1–W2 & Q1–Q2: On Generalization and Cross-Model Applicability.

1) More general actions and interactions (W1 & Q1).

Thanks for the concern regarding PACE's applicability beyond camera movement. We would like to first clarify the evaluation scope of our original experiments. The original experiments with only two predefined camera trajectories exactly follow the evaluation protocol of Light Interaction, ensuring fair comparisons with existing acceleration methods under consistent experimental settings.

To address the reviewer's concern, we conduct additional experiments on scenarios with object interaction and character control. Specifically, we adapt PACE to Zing-0.5 [1], which supports joint control with keyboard actions and online text instructions. As illustrated in Fig. 5 of the revised manuscript, we present a case study that generates a 60-second multi-turn interaction sequence involving object state changes (i.e. a stone statue opening its eyes and glowing) and character behaviors (i.e. moving forward, moving left, moving right, kneeling, standing up, jumping, and floating), all successfully rendered by PACE. Meanwhile, as shown in Tab. R1 below, PACE reduces the DiT-core latency from 101.89 s to 43.83 s, achieving a 2.325× speedup. Notably, in the final stage of this case, Original generates an extra statue face and fails to render the character floating, whereas PACE avoids this visual artifact and successfully renders the floating action. This qualitative case demonstrates the feasibility of applying PACE to non-camera interaction scenarios. Detailed results have been provided in Fig. 5, Tab. 9, and Appendix E of the revised manuscript.

Table R1: DiT-core latency and speedup on Zing-0.5 for a 60-second multi-turn interaction sequence.

MethodDiT-core Latency (s) ↓Speedup ↑
Original101.891.00×
PACE (Ours)43.832.325×

Furthermore, to assess PACE’s effectiveness across more diverse multi-turn navigation trajectories, we evaluate it on HY-WorldPlay and Matrix-Game 3.0 using the full navigation split of WBench [2]. This split contains 158 cases, each involving 2–6 interaction rounds and covering 6 trajectory types: round-trip, progressive, repeat, L-shape, loop, and zigzag. As shown in Tab. R2 below, PACE achieves 3.11× and 1.76× DiT-core speedups on HY-WorldPlay and Matrix-Game 3.0, respectively, while outperforming Original in WBench overall scores (76.08 v.s. 75.84 and 55.02 v.s. 53.79 on the two models, respectively). Moreover, compared with Light Interaction, PACE achieves lower inference latency on both models and exhibits higher agreement with Original outputs, as measured by PSNR, SSIM, and LPIPS.

Table R2: Comparison on the WBench navigation split. PSNR, SSIM, and LPIPS measure agreement with Original outputs. The best results within each backbone are shown in bold.

MethodPSNR ↑SSIM ↑LPIPS ↓DiT-core Latency (s) ↓Speedup ↑WBench Avg. ↑
HY-WorldPlay
Original–––176.081.00×75.84
Light Interaction20.310.51210.291560.852.89×76.05
PACE (Ours)20.760.52430.290256.633.11×76.08
Matrix-Game 3.0
Original–––25.111.00×53.79
Light Interaction16.420.49430.394514.991.68×52.97
PACE (Ours)18.140.55370.332714.251.76×55.02

Overall, these supplementary experiments provide further evidence for the applicability of PACE to non-camera interaction behaviors and diverse multi-turn navigation trajectories, extending the evaluation beyond the two predefined trajectories used in the original experiments. We have incorporated these additional experiments and results into Fig. 5, Tab. 2, Tab. 9, and Appendix E of the revised manuscript.

2) Cross-backbone adaptation and validation of the complete method (W2 & Q2).

Firstly, we would like to clarify that the implementation adjustments on Matrix-Game 3.0 are made to accommodate its architectural differences, while preserving PACE’s core principles. Specifically, these adaptations mainly involve the granularity of Block-wise Context Compression (BCC) and the scheduling of Interruption-aware Cache Reuse (ICR), detailed as below:

  • For BCC, the compression granularity depends on the temporal organization of the historical visual context. In HY-WorldPlay, historical information is organized in groups of four consecutive frames, so BCC can perform redundancy assessment and compression with 4×8×44\times8\times4 spatiotemporal blocks. In contrast, the historical context of Matrix-Game 3.0 consists of 1 anchor frame, 4 retrieved frames, and 4 recent frames, whose temporal sources and organization differ across parts, so directly grouping them into four-frame blocks cannot preserve their temporal and semantic relationships. Therefore, we adopt frame-level compression, retaining 1 anchor, 2 retrieved, and 3 recent frames, which reduces the historical context from 9 to 6 frames.

  • For ICR, the reuse schedule needs to adapt to the number of intermediate denoising steps of the model. Both backbones determine reuse opportunities based on changes in historical visual content, while always recomputing the first and last denoising steps. HY-WorldPlay adopts four-step denoising with two reusable intermediate steps, thus supporting the reuse of 0, 1, or 2 steps; Matrix-Game 3.0 adopts only three-step denoising with a single reusable intermediate step, thus supporting the reuse of 0 or 1 step.

Moreover, to further validate the applicability of the complete method on another backbone, we transfer all three components of PACE, i.e. BCC, ICR, and ACSA, to Zing-0.5 simultaneously. This model adopts four-step denoising, and its native attention window contains 97 latent frames. The entire transfer requires no retraining or modification of the pretrained weights, with the specific adaptations as follows:

  • BCC: We exclude the 9th sink frame from the block grouping candidates, so that the remaining sink frames and visible history can be aligned in groups of four latent frames. Through masked padding, the spatial token grid is aligned from 15×2615\times26 to 16×2816\times28, thereby preserving the 4×8×44\times8\times4 spatiotemporal block structure.

  • ICR: Zing-0.5 also adopts four-step denoising, so the intermediate-step reuse strategy of HY-WorldPlay can be directly inherited.

  • ACSA: We recalibrate the attention allocation between current and historical blocks for different layers and attention heads on the new backbone.

As shown in Tab. R1 above, on a single NVIDIA H100 GPU, the complete PACE achieves a 2.325× DiT-core speedup, reducing the DiT-core latency of the 60-second interaction sequence from 101.89 s to 43.83 s. In terms of generation results, PACE successfully renders object state changes such as the statue opening its eyes and glowing, as well as character behaviors such as kneeling, standing up, jumping, and floating, demonstrating that PACE ensures the correct rendering of object state changes and character behaviors while accelerating inference.

These results indicate that the core mechanisms of PACE can adapt to the historical context organization and denoising schedule structure of different models, further supporting the cross-backbone applicability of the complete framework. We have added the above adaptation details and experimental results to Fig. 5, Tab. 9, and Appendix E of the revised manuscript.

[1] Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control. arXiv’26.

[2] WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation. arXiv’26.

(2/3) W3 & Q3–Q4: On Calibration Cost and Experimental Fairness.

1) Computational cost and reusability of offline calibration (W3 & Q3).

The offline calibration of PACE is only used for ACSA, to determine the attention allocation policies for different attention heads and interaction states.

Regarding the specific calibration overhead, only 10 reference images are required, each paired with two action trajectories (i.e. forward–backward and left–right), resulting in 20 calibration sequences in total. Taking HY-WorldPlay as an example, the entire process takes approximately 3 hours on a single NVIDIA H100 GPU.

Regarding calibration reusability, due to the inherent differences in attention distributions and context organization across backbones, different models exhibit distinct head-specific preferences. Therefore, recalibration is required when switching to a different backbone, as is common practice in broad calibration-based methods [1,2]. Nevertheless, since calibration captures model-intrinsic computation patterns, its results can be directly reused across inference runs as long as the model remains unchanged.

Overall, we consider this one-time calibration cost practically acceptable, especially given that the calibrated allocation alone improves Mutual PSNR/SSIM/LPIPS from 26.19/0.6889/0.1238 to 26.44/0.6982/0.1148 over the uniform sparse baseline at matched DiT-core latency (Tab. 4 of the manuscript). Moreover, the calibration results are reused across all subsequent inference runs, so this cost is further amortized.

2) Fairness of experimental settings and baseline comparisons (Q4).

Firstly, we would like to clarify that PACE is implemented directly on top of the official source code of Light Interaction, and strictly follows its evaluation protocol. Except for the PACE-specific algorithmic configurations, we keep the base models, evaluation inputs, action trajectories, random seeds, sampling settings, and metric computation unchanged. Specifically, we evaluate generation quality and inference efficiency as follows.

For generation quality, given our limited computation resources, we follow prior practice [3] by aligning our experimental setup with that of the state-of-the-art method, thus ensuring fair comparison between PACE and the baselines reported therein. Specifically, we first faithfully reproduce the results of Light Interaction, the strongest competing baseline, using its official code, thereby validating the alignment of our evaluation setup with the state-of-the-art evaluation framework established by Light Interaction. On this basis, we directly adopt the results of Original and other weaker baselines reported in the Light Interaction paper (including PSNR, SSIM, LPIPS, Self-Comparison, and VBench), and further implement our PACE with all corresponding experimental settings unchanged.

For inference efficiency, given the sensitivity of inference latency and peak GPU memory usage to hardware environments, we re-evaluate all the compared methods (i.e. Original, all baselines, and PACE) using the aforementioned evaluation framework on NVIDIA H100 GPUs, measuring both efficiency metrics under identical experimental settings.

We have incorporated the above details regarding experimental settings and baseline comparisons into the revised manuscript.

[1] MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention. NeurIPS’24.

[2] Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering. ICML’26.

[3] FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search. CVPR’19.

(3/3) W4 & Q5–Q6: On Evaluation Reliability and Generation Quality.

1) Action following and world-state correctness (W4 & Q5).

Thanks for the suggestion. We would like to first clarify that our Self-Comparison metric already reflects world-state correctness: it compares pose-matched frames upon scene revisits, so any drift in the generated world state directly lowers the score. As shown in Tab. 1 of the manuscript, PACE outperforms Original on this metric on both backbones (e.g., PSNR 20.02 v.s. 18.60 on HY-WorldPlay and 16.95 v.s. 15.49 on Matrix-Game 3.0).

Moreover, following the reviewer's suggestion, we further adopt the Navigation Trajectory (NavScore) metric of WBench to evaluate the accuracy of action following. On Matrix-Game 3.0, Original and PACE achieve NavScores of 85.34 and 85.36, respectively. This indicates that PACE maintains action-following capability largely consistent with Original while accelerating inference.

Overall, Self-Comparison and NavScore provide complementary evidence from two aspects, i.e. revisit consistency of the world state (world-state correctness) and accuracy of action following, indicating that PACE preserves these important interactive capabilities while accelerating inference.

2) Result variance and stability (Q6).

To further evaluate the sensitivity of generation results to random seeds, we adopt the same 40 test cases as the ablation studies in the main paper on HY-WorldPlay, evaluate both Original and PACE with 3 different random seeds, and report the mean and standard deviation of each metric.

Table R3: Generation quality across three random seeds on HY-WorldPlay (mean ± std).

MetricOriginal (Mean ± Std)PACE (Mean ± Std)
Mutual PSNR ↑—25.54 ± 0.20
Mutual SSIM ↑—0.6606 ± 0.0073
Mutual LPIPS ↓—0.1481 ± 0.0081
Self-Comparison PSNR ↑18.99 ± 0.3019.95 ± 0.30
Self-Comparison SSIM ↑0.5545 ± 0.00720.5879 ± 0.0044
Self-Comparison LPIPS ↓0.1766 ± 0.00870.1524 ± 0.0061
VBench Average ↑0.8292 ± 0.00240.8303 ± 0.0024

As shown in Tab. R3, PACE achieves average improvements of 0.96 dB, 0.0334, and 0.0242 in Self-Comparison PSNR, SSIM, and LPIPS, respectively, with small standard deviations across random seeds for all metrics. The VBench overall score is largely consistent with that of Original, with both standard deviations being only 0.0024. Moreover, PACE outperforms Original on Self-Comparison metrics under all three random seeds, indicating that its advantage in revisit consistency does not depend on specific random seeds.

3) Reasons for the VBench decrease on Matrix-Game 3.0 (Q6).

We further analyze the VBench results of Matrix-Game 3.0 on the 400 evaluation cases, and find that the score decrease is mainly concentrated in two dimensions, Aesthetic Quality and Imaging Quality, which drop from 0.4863 to 0.4696 and from 0.6497 to 0.6352, respectively. The changes in the remaining three VBench dimensions are all within 0.003, indicating that the quality decrease mainly lies in per-frame perceptual quality, while the other dimensions remain largely stable.

We believe that one possible explanation lies in the fewer denoising steps of Matrix-Game 3.0. Unlike HY-WorldPlay, which adopts four-step denoising and allows finer-grained reuse scheduling across two intermediate steps, Matrix-Game 3.0 adopts only three-step denoising and thus has only one reusable intermediate step. Under this more constrained setting, reusing cached intermediate computation may introduce more noticeable approximation errors in visual details. Similar quality decreases also occur in other cache-reuse-based acceleration methods: TeaCache and Light Interaction obtain VBench scores of 0.7146 and 0.7350, respectively, both lower than 0.7432 of Original.

Nevertheless, we would like to highlight the effectiveness of PACE in broader evaluations. On the original 400 evaluation cases, PACE achieves a 1.74× DiT-core speedup while maintaining high agreement with Original outputs; on the 158 multi-turn navigation cases of WBench, PACE achieves a 1.76× speedup with an overall score of 55.02, higher than 53.79 of Original. Meanwhile, we also recognize that cache reuse in few-step denoising models remains a challenging problem, which has also been discussed in prior work DisCa [1]. Therefore, how to further reduce the perceptual quality loss caused by cache reuse in such settings remains worthy of in-depth future research.

[1] DisCa: Accelerating Video Diffusion Transformers with Distillation-Compatible Learnable Feature Caching. CVPR’26.

Comments on the paper

3 comments

Dear Reviewer, we sincerely thank you for the insightful and constructive comments. We organize our responses into 2 sections to address your concerns regarding the experimental fairness of generation quality comparisons (W1), as well as robustness to irregular user actions and offline calibration (W2).

(1/2) W1: On Experimental Fairness and Generation Quality Comparison.

1) Fairness of experimental settings and generation quality comparison (W1).

Thanks for the concern regarding the controls in the quality comparison. We would like to first clarify that PACE is implemented directly on top of the official source code of Light Interaction and strictly follows its evaluation protocol. Specifically, except for the PACE-specific algorithmic configurations, the base models, test cases, input images, action trajectories, random seeds, sampling settings, and metric computation are all kept unchanged. Under this unified framework, we have faithfully reproduced the results of Original and Light Interaction, confirming the consistency of our evaluation setup with the established state-of-the-art evaluation framework. On this basis, we follow the common practice [1] in adopting the quality scores of all compared methods reported in the Light Interaction paper.

Besides, to further substantiate the reliability and fairness of our quality comparisons, we additionally conduct a fully controlled evaluation on WBench [2], where Original, Light Interaction, and PACE are independently run and evaluated under identical settings. As shown in Tab. R6, PACE remains outperforming Light Interaction in PSNR (20.76 v.s. 20.31 on HY-WorldPlay and 18.14 v.s. 16.42 on Matrix-Game 3.0), SSIM, and LPIPS on both models, while also achieving lower latency (56.63 v.s. 60.85 s and 14.25 v.s. 14.99 s). This provides additional evidence that PACE’s quality advantage over the closest baseline stems from the method itself rather than differences in evaluation settings.

Table R6: Comparison on the WBench navigation split. PSNR, SSIM, and LPIPS measure agreement with Original outputs. The best results within each backbone are shown in bold.

MethodPSNR ↑SSIM ↑LPIPS ↓DiT-core Latency (s) ↓Speedup ↑WBench Avg. ↑
HY-WorldPlay
Original–––176.081.00×75.84
Light Interaction20.310.51210.291560.852.89×76.05
PACE (Ours)20.760.52430.290256.633.11×76.08
Matrix-Game 3.0
Original–––25.111.00×53.79
Light Interaction16.420.49430.394514.991.68×52.97
PACE (Ours)18.140.55370.332714.251.76×55.02

We have incorporated the above experimental details and results in the revised manuscript.

[1] FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search. CVPR'19.

[2] WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation. arXiv'26.

(2/2) W2: On Robustness to Irregular Actions and Offline Calibration.

1) Robustness under frequent direction changes (W2).

We appreciate your concern regarding robustness under frequent and irregular direction changes. We would like to first politely clarify that our original experiments with two predefined trajectories strictly follow the evaluation protocol of Light Interaction, ensuring fair comparisons with existing acceleration methods.

To further address this concern, we conduct additional experiments using the full navigation split of WBench [1]. This spilt comprises multi-turn navigation cases, each has 2–6 interaction rounds, covering 6 trajectory types: round-trip, progressive, repeat, L-shape, loop, and zigzag. In particular, L-shape, loop, and zigzag trajectories involve frequent direction changes, compound motions, and scene revisits, posing greater challenges to interaction-state determination and the intermediate computation reuse.

As shown in Tab. R6 above, PACE outperforms both Original and Light Interaction in WBench overall scores (76.08 v.s. 75.84/76.05 on HY-WorldPlay and 55.02 v.s. 53.79/52.97 on Matrix-Game 3.0), and exhibits higher fidelity to Original outputs than Light Interaction, as measured by PSNR, SSIM, and LPIPS. Notably, these quality advantages are achieved with even greater efficiency than the most competitive Light Interaction, with DiT-core speedups of 3.11× v.s. 2.89× and 1.76x v.s. 1.68× on the two models, respectively.

Overall, these results provide further evidence of PACE’s robustness beyond the original fixed trajectories, demonstrating effective acceleration and generation quality across diverse multi-turn interaction patterns and lengths. We have incorporated this supplementary experiment into Tab. 2 of the revised manuscript.

2) Offline calibration and online inference mechanism (W2).

Thanks for raising this point. For calibration, we use two types of action trajectories (i.e., forward–backward and left–right), with 10 sequences for each, randomly sampled from VBench and kept strictly separate from the evaluation set. We have clarified these details in the revised manuscript. To further ensure reproducibility, we will release the source code and exact calibration data upon paper acceptance.

[1] WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation. arXiv’26.

AuthorsOpenReview ↗

A Kind Reminder for Reading the Response

Dear Reviewer utJA,

We sincerely appreciate your thoughtful comments and suggestions on our work. We have carefully considered your feedback and provided detailed responses in our rebuttal.

We would be grateful to hear whether our explanations have sufficiently addressed your concerns. Should you have any remaining questions or suggestions, we would be glad to further discuss and clarify them.

Thank you again for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.