acceptodds
Withdrawn from ICLR 2027

StreamAccel: Automated Inference Acceleration for Streaming Causal Video Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive video generation requires fast, sustained causal inference to provide timely feedback to user inputs. Yet accelerating a new model still requires substantial manual tuning: practitioners must not only select suitable acceleration techniques but also determine their order and combination. Because earlier optimizations can alter the execution conditions of later techniques, choices that perform best in isolation may not be optimal when combined. Moreover, differences in model structure, streaming state, and deployment make existing configurations difficult to transfer directly, requiring these decisions to be revisited for new models. To address these challenges, we introduce StreamAccel, an automated inference optimization system for streaming causal video models. Its staged workflow defines the optimization tasks and prerequisites for each stage, guiding an agent to select, compose, and validate acceleration techniques. Steady-state measurements assess the benefits of these combinations for diffusion-transformer (DiT) generation and VAE decoding. Complementing this workflow, a reusable technique knowledge library captures applicability conditions, composition experience, and failure modes from prior tasks, guiding optimization on new models and enabling engineering experience to be reused across models. Experiments demonstrate that StreamAccel consistently reduces steady-state latency for both generation and decoding across different causal video models.

Primary area:
optimization
Keywords:
Video generation acceleration, Streaming causal video generation acceleration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pJAzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces StreamAccel, an automated inference optimization system for streaming causal video models. It proposes a workflow and a reusable technique knowledge library to guide optimization on new models. However, the experiments and ablation studies are insufficient to demonstrate its effectiveness.

Critical strengths
  • Inference acceleration for streaming causal video models is worth exploring.
  • An automated inference optimization system is valuable.
Critical weaknesses
  1. Although the paper emphasizes that StreamAccel is automated, it does not clarify how much time it saves compared with manual optimization, how many agent interactions and what costs are required to optimize a new model, or how much human expertise contributes to the optimization.
  2. The contributions are limited, and it's more like combining submodules into an agentic workflow.
  3. The paper lacks ablation studies.
  4. The paper does not include comparisons with other agentic/harness optimization methods.
  5. What does “Reference” Method refer to in Table 1?
Questions for authors

Please see weaknesses, especially W2,W3,W4.

Reviewer_L1pdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

StreamAccel uses an agent to optimize streaming causal video inference through a staged workflow covering execution-path preparation, DiT compilation and attention-backend selection, residual bottleneck optimization, graph replay, and streaming VAE decoding. A technique library records applicability conditions and prior experience. Candidates are evaluated within the current optimization stack for latency and affected generation behavior. On Solaris and SolarWM-Wan5B, the resulting implementations reduce DiT latency by 38.33% and 32.21%, respectively, with similar aggregate visual-quality and control scores.

Critical strengths
  1. The matched-workload experiments show substantial DiT acceleration on two causal models. Evaluation includes 60 paired long rollouts per model and control-response measures, providing relevant evidence for interactive generation.
  2. The workflow accounts for evolving streaming state. Separating denoising from KV-refresh capture and preserving decoder-cache updates addresses meaningful correctness constraints. The attention-backend example also connects local screening to a full-model benefit.
Critical weaknesses
  1. The experiments compare the final implementations only with native references. Without a matched agent baseline or workflow ablation, they do not isolate the benefit of stage ordering from standard acceleration techniques. The main methodological contribution therefore remains insufficiently supported.
  2. Knowledge reuse is unvalidated. Appendix A.1 states that both models informed workflow development; neither tests held-out transfer. There is also no library ablation or measurement of tuning effort, leaving reuse and reduced manual engineering largely untested.
  3. Evaluation is limited to single-GPU steady-state component timings. Multi-GPU execution and end-to-end action-to-frame latency, including tail latency, are untested. Communication, synchronization, and scheduling can change the bottlenecks, leaving practical responsiveness and applicability to multi-GPU causal-video deployment unclear.
Questions for authors
  1. What evidence isolates the benefit of dependency-aware staging? Are there controlled comparisons or optimization traces showing an advantage over the same agent and technique pool without the prescribed order?
  2. Is there an evaluation with the library frozen before optimizing a new target, or with and without library access? What records are available on optimization time, trial counts, and human intervention?
  3. Are measurements available for multi-GPU execution and end-to-end action-to-frame latency, including p95/p99? How do these affect the claimed benefit for interactive causal video generation?
Reviewer_L85rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

StreamAccel uses an AI agent to optimize streaming causal video inference through a staged workflow. The agent selects and combines compilation, attention backends, operator optimizations, and CUDA Graph replay, with guidance from a library of previous optimization experience. On Solaris and SolarWM-Wan5B, the resulting implementations reduce steady-state DiT latency, with model-dependent VAE improvements and similar measured visual quality and control response.

Critical strengths
  1. The workflow explicitly handles the changing state of streaming inference. Separating denoising from history updates, and checking that replay uses current controls and positions, addresses correctness risks that isolated kernel optimization would overlook.

  2. The evaluation measures recurring work, including cache updates and streaming decoding, and checks visual quality and control response over long rollouts. This makes the reported acceleration more relevant to sustained use.

Critical weaknesses
  1. The benefit of the proposed optimization workflow is not isolated. Comparisons are only against the models’ reference implementations. Without a simpler agent baseline or a comparison with an existing optimization workflow, the speedups do not establish how much the staged process contributes beyond applying standard acceleration techniques.

  2. Cross-model knowledge reuse is not demonstrated independently. Whether experience collected from one model helps optimize a new model is not clarified. Since reusable optimization knowledge is a central contribution, the absence of evidence separating its benefit from model-specific development substantially limits the assessment.

Questions for authors
  1. Is an anonymized implementation available?
  2. Do existing comparisons or optimization logs show what the staged workflow contributes beyond using the same agent with ordinary profiling and optimization instructions? Please clarify the optimization budget and human interventions involved.
  3. Which knowledge-library entries existed before optimization began on each model? Can the existing records identify decisions or avoided failures that resulted from experience collected on the other model?
  4. Based on the existing profiles and trial logs, what limits VAE acceleration on SolarWM-Wan5B, and which attempted optimizations were ineffective or rejected?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.