acceptodds
Under review as a conference paper at ICLR 2027

DepGraph-U: Dependency-Aware Execution Pruning for Unified Multimodal Inference

Abstract

Unified multimodal models (UMMs) bring understanding and generation into a single Transformer, reducing the need for separate specialist models. A shared inference implementation can nevertheless execute operators that the current request does not consume. Even when a valid image cache exists, or a caller requests pixels rather than text, unused producers and vocabulary projections can still execute. We use output-dependent liveness to identify work that remains executable but is unnecessary for the requested output or required future state. To address this, we propose DepGraph-U, a dependency-aware execution framework that uses audited output and state dependencies to guide guarded inference rewrites, without further weight modification. Cache-Aware Dependency Dispatch (CADD) bypasses upstream producers on safe cache hits, and an image-only interface omits unused vocabulary projection. For understanding, graph-compatible KV policies distinguish preserving dynamic layer selection from enabling earlier reuse under a stable graph. We evaluate DepGraph-U across two Show-o2 scales and eight benchmarks. The complete system achieves 2.88–3.21× generation speedup over native Show-o2, with quality trade-offs from the selected approximation policies. Against the reconstructed understanding reference, Exact-DLS preserves all 24,769 decoded responses with 1.15–2.34× speedup. Across the six tested understanding tasks, 80.8% of Exact-DLS examples finish before incremental reuse, explaining why cache compatibility alone does not determine realized latency.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
unified multimodal models, efficient inference, dependency-aware execution, execution pruning, cache reuse, KV caching, dynamic layer selection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7x6tOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose DepGraph-U, a training-free inference-efficiency framework for unified multimodal models. The authors argue that unified models may execute unnecessary computation for outputs that are not requested or consumed, and propose dependency-aware execution mechanisms to skip such computation. The main implementation and evaluation are centered on the Show-o2 architecture

Critical strengths
  • The farmwork accelerates the efficiency while can preserve the performance
  • The experiments and internal ablations are extensive.
Critical weaknesses
  • Limited novelty: The high-level motivation of training-free, task-aware acceleration for unified multimodal models has already been explored by Flash-Unified [1]. In fact, DepGraph-U builds on several FlashU mechanisms, including FFN pruning, dynamic layer selection, visual-token pruning, and diffusion-head caching. The primary additional contribution is therefore execution-level dependency analysis that removes computation remaining unnecessary after an acceleration policy has already been selected. While this distinction is valid, some proposed mechanisms, such as checking a valid cache before executing its unused producers or bypassing an unused vocabulary projection, appear relatively straightforward from a systems perspective.

  • Limited architectural generalization: Although the dependency-analysis principle is presented as general, the concrete mechanisms are strongly tailored to Show-o2 [2]. CADD relies on Show-o2's Head-A/Head-B caching structure, VocabBypass targets a vocabulary projection that other architectures may already avoid, and Exact-DLS is specifically designed around the selected dynamic layer-selection and pruning policy. Thus, applying the framework to another unified architecture requires a new architecture-specific dependency audit and potentially different execution rewrites.

  • Limited additional benefit on newer/different unified architectures. the new generation of unified model does not need this framwork. For exampl,in BAGEL[2], several targeted inefficiencies already do not exist: vocabulary projection is already avoided for image calls, native KV is available, and the architecture does not share the same two-part diffusion-head structure. Consequently, the applicable new generation rewrite alone provides almost no additional acceleration in the reported native BAGEL control. This makes it unclear whether DepGraph-U is a broadly applicable acceleration technique or primarily a systematic methodology for finding architecture-specific

  • Paper presentation and clarity. The paper is poorly presented and, in its current form, does not meet the presentation standards expected of a top-tier conference such as ICLR. For example, the motivation for the proposed approach is not clearly articulated.

[1] J. Ke, etal, Flash-unified: A training-free and task-aware acceleration framework for native unified models,CVPR, Findings, 2026.

[2] J. Xie, Z. Yang, and M. Z. Shou, “Show-o2: Improved Native Unified Multimodal Models,” NeurIPS 2025

[3] Ch. Deng, et. Al, ‘Emerging properties in unified multimodal pretraining’, 2025, arxiv.

Questions for authors

Please see the weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7x6t (1/4): W1, novelty

Thank you for the review. We respond to the four weaknesses in order.

W1. Limited novelty

We agree with your description of what the paper adds: an execution-level dependency analysis that removes computation which is still unnecessary after an acceleration policy has been chosen. FlashU and DepGraph-U answer different questions.

  • FlashU asks which computation can be approximated (FFN pruning, dynamic layer selection, visual-token pruning, diffusion-head caching, a shorter schedule). This changes the outputs (GenEval 0.722 → 0.692 at 1.5B) and gives 1.75–1.84× over native Show-o2.
  • DepGraph-U asks which executed operators still have a consumer once that policy is fixed (CADD, VocabBypass, graph-compatible KV reuse). This changes no output and gives a further 1.61–1.77× over FlashU, with pixel-identical images.

We also agree that the two generation rewrites are simple to state. That is not the same as being known, or easy to get right, for three reasons.

  1. The waste is large, and it sits in a published accelerator. In FlashU-accelerated Show-o2, 14 of 34 model calls recompute the features, trunk and projection and then discard the result in favor of a cached head output. Every trunk call also computes text-vocabulary logits that the image sampler never reads (16–18% of profiled time). Removing both gives 1.61–1.77× on top of FlashU with identical pixels. That is about as much as FlashU's whole approximation policy gives over native Show-o2, and it costs no quality. We do not mean this as a criticism of FlashU: choosing an approximation policy and executing it are separate problems.
  2. "Check the cache first" is wrong on some steps. Of the 16 cache hits per image, two fall on DLS-recalibration steps, where the trunk's hidden states are still needed to choose the next skip set. Skipping there would silently change later layer selection, and with it the image. The guard (Eq. 1) excludes these steps, as well as conditional-only guidance and the full-network region, and 3,570 replayed steps confirm equality on every branch. Finding which hits are safe to take is what the dependency audit is for.
  3. A valid cache does not imply a saving. On the understanding side, 80.8% of the 24,769 examples finish before any cached state is reused under dynamic layer selection, and the speedup grows only with reached reuse (Figs. 6, 7, 9). This is an empirical finding, and it applies to any policy that re-selects layers during decoding, not only to Show-o2.

The simplicity also has a practical side: the rewrites change no weights and no outputs, so they can be added to an existing accelerator without any quality risk. We will add a comparison with FlashU to Sec. 2 and lead the abstract with the fixed-policy factor.

AuthorsOpenReview ↗

Response to Reviewer 7x6t (2/4): W2, architectural generalization

W2. Limited architectural generalization

We agree that each rewrite has to be instantiated per architecture, and Sec. 5 says so. But the three patterns are less tied to Show-o2's structure than the main text made them look. Taking your three examples in turn:

  • CADD does not need the Head-A/Head-B split. The pattern is to test the guard of a cached or predicted state before running the producers that this state replaces. OmniGen2 (a separate MLLM and diffusion transformer) has no Head-A/B. There the same pattern skips the context refiners on TaylorSeer-predicted steps, which gives 1.12× on full GenEval and DPG-Bench with matching paired outputs (Table A13).
  • VocabBypass is not always already there. BAGEL does avoid the projection, and we credit no gain for it. Lumina-DiMOO does not: with its shared vocabulary, text logits are computed on every image call. Restricting the head to the image-token slice gives 1.06× with identical outputs, with or without ML-Cache (Table A12).
  • Exact-DLS is specific to policies that re-select layers during decoding. We agree, and we do not claim more. Stable-Graph KV covers static pruning (FFN and visual tokens) and does not depend on DLS.

In the table, "Lines" is the number of inserted lines of rewrite code, and the BAGEL number is from the 12-prompt control of Appendix I.4.

Model (accelerator)Pattern that appliedLinesGenEval / DPGPaired outputs
OmniGen2 (TaylorSeer)CADD (refiners on predicted steps)171.116× / 1.117×match
Lumina-DiMOO (ML-Cache)VocabBypass (image-token slice)131.062× / 1.061×match
BAGEL (none)skip an unconsumed CFG prefill331.0001×36/36 identical

So a new architecture does need its own audit. The audit follows the same four steps each time (Appendix A.1), and the resulting rewrite was 13–33 inserted lines per model. These results appeared only in Appendix I.5, and we will move them into the main text.

AuthorsOpenReview ↗

Response to Reviewer 7x6t (3/4): W3, newer architectures

W3. Limited additional benefit on newer or different architectures

BAGEL is the boundary case of the paper, and we included it for that reason. On your question of whether DepGraph-U is a broadly applicable acceleration technique or a methodology for finding architecture-specific inefficiencies: it is the second, with three rewrite patterns that recur across models. Its gain on a model equals the unconsumed work that model executes, and the audit reports that amount before any engineering is done. Of the four models we audited, three execute a measurable amount of such work. All gains below preserve the outputs.

Model (release)Unconsumed work found by the auditGain
Show-o2 (June 2025) + FlashUproducers before 14 of 34 hits; unused text logits1.61–1.77×
OmniGen2 (June 2025) + TaylorSeercontext refiners on predicted steps1.12×
Lumina-DiMOO (Oct. 2025)text-vocabulary slice on image calls1.06×
BAGEL (May 2025)one duplicate CFG prompt prefill1.0001×

We do not think "the new generation does not need this" follows, for two reasons.

First, release date does not predict it. BAGEL (May 2025) appeared before Show-o2 (June 2025), and both OmniGen2 and Lumina-DiMOO appeared after BAGEL and still execute unconsumed work.

Second, the waste tends to come back when a caching or prediction accelerator is added to a model. In both cases we studied (FlashU's head cache on Show-o2 and TaylorSeer on OmniGen2), producers keep running ahead of a state that replaces them. Such accelerators are a common way to speed up diffusion-based generation, so we expect the pattern to recur.

Finally, because every rewrite preserves outputs, applying the audit carries no quality risk even when the gain is small. On BAGEL the outcome is 1.0001× with 36/36 pixel-identical images, not a degraded model. We will state this scope in the abstract and introduction.

AuthorsOpenReview ↗

Response to Reviewer 7x6t (4/4): W4, presentation

W4. Paper presentation and clarity

We take this seriously and will restructure the paper. The motivation can be stated in three sentences, and the revised introduction will open with them.

A unified model shares one inference path between understanding and generation, so a request can execute operators whose outputs it never reads. In FlashU-accelerated Show-o2, an image request spends 38–44% of its time this way: on 14 of 34 denoising calls the trunk is run and its result discarded because a cached head output is used instead, and every trunk call computes logits over the text vocabulary although only pixels are requested. DepGraph-U finds such operators with an output- and state-dependency audit and removes them behind guards, which makes the system 1.61–1.77× faster without changing a single output pixel.

Concretely, we will (1) open the introduction with this example and a table that separates FlashU's factor from ours; (2) lead the abstract with the fixed-policy result and present 2.88–3.21× as its composition with FlashU; (3) move the transfer results and Table A8 into the main text; (4) replace the qualifications repeated through Sec. 4 with one limitations paragraph; and (5) add the applicability guidance for the understanding policies.

We hope this clarifies the contribution and its scope.

Reviewer_4djmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DepGraph-U removes avoidable execution in unified multimodal inference through guarded cache dispatch, image-only vocabulary bypass, and graph-compatible KV management with no additional training, while the complete system inherits calibrated pruning and approximation policies. On Show-o2, complete generation achieves approximately 3x speedup with quality reductions. Fixed-policy rewrites add 1.6~1.7× acceleration with matching images.

Critical strengths
  1. The distinction between approximation and execution savings is unusually careful.
  2. The reachable-reuse analysis provides valuable systems evidence.
  3. The transfer studies and negative results clarify applicability.
Critical weaknesses
  1. It is unclear which dependency descriptions, guards, and interfaces transferred unchanged versus required adaptation, and the existing implementation lacks a concise supported-caller contract.
  2. Given Table A8, the motivation and applicability of retaining FlashU-R's DLS policy are insufficiently justified.
  3. The identification of adapters, frozen artifacts, and environment segments, along with the relevant checks and state/RNG records, is incompletely documented, limiting independent verification.
Questions for authors
  1. Which dependency descriptions, guards, and interfaces transferred unchanged, and which required adaptation? Can the existing implementation supply a concise supported-caller contract?
  2. What motivates retaining FlashU-R's DLS policy given Table A8? If feasible, an optional, limited native-KV comparison on MMStar with matched projection, inputs, caps, RNG, and hardware could clarify applicability; it is not required for acceptance.
  3. Can the authors identify adapters, frozen artifacts, and environment segments, and summarize existing checks for changing skip sets, missing layer-zero KV, visual gather indices, tied maxima, and EOS? Which state/RNG records support independent verification?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4djm (1/3): Q1, transfer and caller contract

Thank you for the precise questions, and for the optional MMStar suggestion, which turned out to be a useful addition.

Q1. What transferred unchanged, what required adaptation, and a supported-caller contract

Short answer: the liveness rule and the acceptance test transferred unchanged. The guard predicate and the choice of which producers to skip were re-derived for each model, in 13–33 inserted lines each.

Unchanged in all three ports

  • The liveness rule of Appendix A.1: required outputs and future state, plus their data and control ancestors, are live.
  • The acceptance test: paired output equality under a fixed policy.

Adapted per model

  • Lumina-DiMOO. VocabBypass is adapted: the vocabulary is shared, so the head projects only the image-token slice, and sampled token ids are unchanged. Guard: eval-mode image generation with the slice set. CADD and KV are not ported.
  • OmniGen2. CADD is adapted: the accepted state is a TaylorSeer prediction instead of a Head-A cache hit, and the skipped producers are the context refiners. Guard: prediction-type step, TeaCache off, eval mode without gradients, cached factor present with matching shape, dtype and device. VocabBypass does not apply, because image denoising runs in a separate diffusion transformer with no vocabulary head. KV is not ported.
  • BAGEL. VocabBypass is already native (no gain credited), and there is no head cache for CADD. The audit found one new instance: an image-CFG prompt prefill that nothing consumes (two text prefills become one). Guard: stateless text-to-image, image-CFG scale 1, text-CFG > 1, TaylorSeer off; any other request goes to the unchanged official API. Native KV is kept.

Supported-caller contract. Yes. The contract below restates Appendix A.1–A.3 in compact form, and we will add it as a box in the appendix.

  • VocabBypass (skip_lm_head=True)
    • Supported: inference calls that use hidden states only through the diffusion head.
    • Effect: the call returns logits=None; hidden states are unchanged.
    • Not supported: text generation, text loss, or any caller that reads logits. These leave the flag off and get the original projection. Passing labels together with the flag raises an error.
  • CADD
    • A hit is accepted only if all of these hold (Eq. 1): the cache interval is positive; the step is not a refresh step; a Head-A output is cached; the step is outside the full-network region; it is not a DLS-recalibration step; guidance is not conditional-only; the selected configuration has positive FFN pruning (an implementation restriction).
    • Effect on a hit: Head B runs on the cached Head-A output with the current timestep embedding and modality spans. The sampler, guidance, cache contents and refresh schedule are unchanged. On any other step the reference path runs, including its cache and layer-selection updates.
    • Not supported: callers that read trunk hidden states or logits on hit steps (new hooks or consumers), training, or stochastic layers in the skipped region. These must disable CADD or extend the guard.
  • Exact-DLS / Stable-Graph KV
    • Supported: batch one, one image encoded once, fixed weights and pruning, the sampling rule of Sec. 3.2.
    • Effect: Exact-DLS replaces the whole cache at phases 0 and 1 of every DLS period and is validated to reproduce the no-KV reference (24,769/24,769 responses). Stable-Graph disables DLS and does not guarantee equal outputs.
    • Not supported: multi-request batches, or a pruning configuration that changes within a sequence.
AuthorsOpenReview ↗

Response to Reviewer 4djm (2/3): Q2, DLS policy and Table A8

Q2. Why retain FlashU-R's DLS policy, given Table A8?

We keep DLS in Exact-DLS because it is part of the published policy that we hold fixed, not because we recommend it.

The protocol of the paper is to fix an approximation policy and change only how it is executed. For understanding, the fixed policy is FlashU's (FFN pruning, visual-token pruning and DLS), as reconstructed in FlashU-R. Exact-DLS answers one narrow question: if a deployment runs this policy, what does valid KV reuse require, and what does it buy? Standard KV is not valid there, because a prefix cached under one layer set is not the prefix state of the next selected set. Exact-DLS rebuilds at every selection boundary. It reproduces all 24,769 reference responses and runs 1.15–2.34× faster than the reference execution of the same policy.

We agree with what Table A8 implies, and we see it as a finding of the paper. DLS forces two full-prefix calls per period (one to select, one to rebuild), and a full-prefix call costs 15–33× an incremental call: about 330 ms vs 21 ms on benchmark prompts (Table A9), and 789 ms vs 23.8 ms at a 1,536-token prefix (Sec. 4.3). Since 80.8% of examples end before any incremental call, DLS with KV cannot match fixed-graph KV on short answers. We will state the practical guidance explicitly:

  • For the highest accuracy: native Show-o2 + standard KV. Our understanding-side methods are not needed.
  • For the lowest latency: Stable-Graph KV, which keeps FFN and visual-token pruning and drops DLS.
  • For an existing DLS deployment whose outputs must not change: Exact-DLS KV.

The static part of the pruning policy pays off in a KV-enabled decoder, and the dynamic part does not. This is why the paper presents Exact and Stable as alternatives, and it is the point of the reachable-reuse analysis (Figs. 6, 7, 9). The accuracy gap to native comes from FlashU's pruning policy, not from the KV handling: Exact-DLS reproduces the pruned policy's responses exactly.

The MMStar comparison you suggest. We ran this comparison before submission but left it out of the paper. All three arms use the same GPU, the same 1,500 inputs, the same per-example seeds and cap 128:

MMStar, 1,500 examplesAccuracyMean latencySpeed vs native + KV (95% CI)
native Show-o2 + standard KV55.6%0.814 s1.00×
Stable-Graph KV50.7%0.467 s1.74× faster [1.73, 1.75]
Exact-DLS KV50.3%1.078 s0.76× [0.71, 0.80], i.e. slower

Stable-Graph is faster than native + KV on 1,499 of the 1,500 examples. Table A8 shows the same ordering on MMMU (native + KV 48.1% / 0.789 s, Stable-Graph 47.1% / 0.491 s, Exact-DLS 46.1% / 0.865 s), but there the native and FlashU-R blocks are separate paired runs, so we compare across them only qualitatively.

Matched projection (new). One difference remained in that run: the native arm computed logits at all positions on its prefill call, whereas Exact and Stable project only the last position. For this response we reran the native arm with both projections, on the same 1,500 inputs and seeds and on a GPU of the same model. The rerun reproduces the earlier native run token for token (1,500/1,500), and last-position projection makes native + KV 1.063× faster (0.808 s → 0.760 s; 95% CI [1.063, 1.064]). With the projection matched, Stable-Graph is 1.64× faster than native + KV instead of 1.74×, and Exact-DLS is still slower. The ordering does not change.

One note on reading these numbers. The accuracies above are overall accuracy (correct answers out of 1,500), while Table 3 reports MMStar as the mean over L2 categories (51.74 for Exact-DLS and 51.92 for Stable-Graph). In overall accuracy the Table 3 runs score 50.47 and 50.60. The matched comparison is a separate run with per-example seeding, so it differs from those values by up to 0.2 points. We will report overall accuracy for MMStar in the revision.

We will move Table A8, this comparison and the guidance into Sec. 4.2.

AuthorsOpenReview ↗

Response to Reviewer 4djm (3/3): Q3, verification records

Q3. Adapters, frozen artifacts, environment segments, existing checks, and state/RNG records

Everything below is in an anonymized repository (https://anonymous.4open.science/r/review-copy-2027-86F1/). Each file is covered by a SHA-256 manifest, and python3 scripts/verify_release.py rechecks every hash and recomputes the reported aggregates on CPU.

Adapters. Show-o2 generation: one patch against a pinned FlashU commit (early dispatch, cache predicates, projection bypass). Show-o2 understanding: the decoding adapter and the FlashU-R policy module (layer skips, visual selection, positions, gather maps). Lumina-DiMOO: two patches. OmniGen2: one adapter that rewrites forward at load time and saves the diff. BAGEL: a dispatch wrapper and a policy adapter. All are pinned to upstream revisions.

Frozen artifacts. One Wanda pruning mask, calibrated on 20 DPG images that are disjoint from evaluation and shared by all six understanding tasks; the rewritten GenEval prompt file; the prompt index lists of every controlled experiment.

Environment segments. All runs in the submission use A800-SXM4-80GB GPUs at batch one, with recorded package lists per model. There are three segments: (1) the original six-task Exact-DLS study; (2) the fresh cap-128 Stable-Graph pairs, with per-example control of the visual RNG; (3) the recovered Lumina-DiMOO and OmniGen2 runs. Every reported speedup pairs two arms inside one segment.

Existing checks for the five cases. The end-to-end check is response equality against the no-KV reference on all 24,769 examples (65,828 decode calls: 51,201 full-prefix and 14,627 incremental), plus token-id equality on the 20,781 examples that have token logs. For each case, how it is handled (Alg. A2, App. A.3) and how often that run exercised it:

  • Changing skip sets. The whole cache is replaced at phase 0 (select with all layers) and at phase 1 (rebuild under the new set), even if the set repeats. Exercised by 846 re-selections after the first period, in 439 examples; 714 of them changed the skip set.
  • Missing layer-zero KV. A new token's position comes from the unpruned mask length, never from a layer's cache length, and skipped layers hold no entry. Layer 0 was never selected naturally (25,615 selections used only layers 2–26, zero-indexed), so we forced it (new): with layer 0 in the skip set, at the first selection or at a re-selection, the KV path matches full recomputation at all 32 steps on 4 of 4 inputs.
  • Visual gather indices. The gather map is stored at each rebuild, extended with later text positions, and applied to the mask's key axis in deep layers. Exercised by all 14,627 incremental calls, since visual-token pruning is always on.
  • Tied maxima. Both arms keep all tied maxima and sample with torch.multinomial under the same RNG protocol. Ties were not logged in the original traces, so we forced them (new): with logits quantized so that up to 19 maxima tie, both paths sample the same token at all 32 steps and end in the same RNG state, on 4 of 4 inputs.
  • EOS. The loop stops after the EOS-producing call in any phase. EOS occurred during the first rebuild in 20,014 examples, during a later selection in 29, during a later rebuild in 24, and during an incremental call in 4,507; 195 ran to the token cap.

State and RNG records

  • Understanding: decoded responses for both arms (24,769 pairs), token-id sequences (20,781), per-call timings with phase labels (65,828).
  • Generation: PNG and decoded-pixel hashes for every paired image; prediction and future-state hashes for 3,570 replayed steps; nine interface records for retained logits and loss and for rejected conflicting calls.
  • BAGEL: CPU and CUDA RNG-state hashes before and after each of the 144 generations. They match across paired arms.

Limits we want to be upfront about. For Show-o2, RNG equality was checked in-process, but raw RNG states were not exported; we will export them in the released verifier. The generation replay does not cover the understanding KV boundaries, which rest on the end-to-end equality above. OmniGen2 pairs match within a process, but a cross-process bitwise check failed, so we claim no cross-process identity. The forced tests are 20 replays (4 MMStar inputs × 5 cases, 32 decode steps each, EOS suppressed) that compare Exact-DLS KV with full recomputation; predictions, DLS schedules and final RNG states are identical in all 20. The frozen pruning mask was not on the rebuttal machine, so they use a mask re-calibrated on 20 MMStar examples. They are equality tests, not quality or timing results.

We will add the case list, the forced tests and an artifact index to Appendix H. Thank you again for the constructive questions.

Reviewer_SB5hOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies execution pruning for unified multimodal models (UMMs), where a shared Transformer serves both understanding and generation yet a given request may still execute operators that no output or future state consumes. The authors diagnose two sources of waste—unused producers still running before a valid Head-A cache hit is dispatched, and vocabulary projections evaluated when the caller only requests pixels—plus an understanding-side gap where a compatible prefix cache saves nothing if decoding ends before any incremental call consumes it. DepGraph-U uses audited output and state dependencies to guide guarded inference rewrites without weight modification: CADD tests the cache guard before the producer stack, VocabBypass omits text-vocabulary projection for image-only calls, and two alternative KV policies handle understanding—Exact-DLS rebuilds at dynamic-layer-selection boundaries to preserve the selected policy, while Stable-Graph disables DLS for earlier reuse at the cost of changed outputs. The paper reports 2.88–3.21× complete-system speedup over native Show-o2 with quality trade-offs from the inherited approximation policy, 1.61–1.77× additional acceleration with pixel-identical images once that policy is fixed, Exact-DLS preserving all 24,769 responses at 1.15–2.34×, and 80.8% of understanding examples finishing before any incremental reuse. Results are established on Show-o2 with a single BAGEL transfer case; extension to other UMMs and batched serving is left open.

Critical strengths
  1. The paper separates the cost of the approximation policy from the cost of executing it, and shows with a fixed-policy control that the execution rewrites give 1.61–1.77× additional acceleration with matching paired PNG hashes and equal scores. This is a clean argument that "the cache is valid" and "the cache is actually consumed" are different questions, and it keeps the headline 3× number from being read as the rewrites' own contribution.

  2. The paper reports controls that bound its own claims—the native standard-KV MMMU comparison where native + KV is both faster and more accurate than Exact-DLS, and the 80.8% zero-reuse finding. These make the boundary of the contribution legible: cache compatibility alone does not determine realized latency, and a pruned policy with rebuild cost is not automatically better than native KV.

Critical weaknesses
  1. Limited transferability.

The proposed method is tailored to Show-o2: the structure is manually edited and the rewrites are implemented specifically for this architecture. Moving to another unified multimodal model requires re-editing dependencies and rewriting the code, and without such re-tuning will result in a perfortmance drop , as the results on BAGEL. Besides, the paper only provide a BAGEL transfer canse without showing how much effort this edit costs. I suggust the author provide more information or discussion to strengthen the transferability of this method.

  1. Weakness on effectiveness.

Tables 1 and 2 include some generative models' results, but as the author notes, "literature timings use different protocols and are not directly comparable". So, the only result that contributes to effectiveness is the one from Show-o2. In addition, the 3× speedup performance is the result of stacking multiple strategies (FFN pruning, DLS, cache interval, guidance adjustment, shortened sampling steps), and it is unclear how well the proposed method performs under different combinations of these strategies. If the authors could demonstrate the method's effectiveness across different strategy stackings, this would strengthen the claim of effectiveness.

3.The experimental setup limits the effectiveness of the method.

The experiment setup uses a fixed batch size of 1, and each complete understanding benchmark is run only once at the process level, so no uncertainty is provided. Besides, whether the method remains effective and stable in real deployment scenarios is therefore an open question, and I suggest the authors could provide some experiments and discuss this explicitly.

Questions for authors

See the Weakness.

3 comments
AuthorsOpenReview ↗

Response to Reviewer SB5h (1/3): W1, transferability

Thank you for the careful review. Your two strengths describe what the fixed-policy and native-KV controls were meant to show, and we are glad that came through. Results marked (new) were measured during the rebuttal period.

W1. Limited transferability

We agree with one part of this and would like to correct two others.

We agree that every rewrite has to be written for the architecture at hand. Sec. 5 says this, and we do not claim a plug-and-play method. What carries over from model to model is the dependency audit, and the code for each port is small.

(1) The submission reports three ports, not only BAGEL. We also ported to Lumina-DiMOO and OmniGen2 and evaluated both on full GenEval (553 prompts × 4 seeds) and DPG-Bench (1,065 prompts). These results are in Appendix I.5 (Tables A12–A13), which made them easy to miss. We will move them into Sec. 4. In the table, "Lines" is the number of inserted lines of rewrite code.

Model (accelerator)What we portedLinesGenEval / DPGPaired outputs
Lumina-DiMOO (ML-Cache)VocabBypass (image-token slice)131.062× / 1.061×match
OmniGen2 (TaylorSeer)CADD (refiners on predicted steps)171.116× / 1.117×match
BAGEL (none)skip an unconsumed CFG prefill331.0001×36/36 identical

On Lumina-DiMOO all 2,212 GenEval pairs are faster (1.05–1.08×). The BAGEL number is from the 12-prompt control of Appendix I.4.

(2) How much effort a port takes. It is the same four steps each time (Appendix A.1): (i) list the outputs the caller reads and the state that later calls read; (ii) mark everything they depend on, through data or control flow, as live; (iii) an operator outside this set may be skipped only if no branch depends on it, it modifies no live tensor, and it draws no random numbers; (iv) put the skip behind a guard that falls back to the original path, and accept the port only if paired outputs are identical. The result was 13, 17 and 33 inserted lines for the three models.

(3) The quality drop on BAGEL does not come from porting without re-tuning. A port changes no weights and has no tunable parameters, so there is nothing to re-tune. The BAGEL "+ DepGraph-U" rows in Tables 1 and 3 also apply a FlashU-style FFN-pruning policy, and that pruning is what changes the answers (about one point on MMStar; 1,157 of 1,500 responses match). The 2×2 control in Appendix I.4 separates the two effects: with the rewrite alone, all 36 paired images are pixel-identical to native BAGEL, while pruning alone changes every image. A skipped operator that was actually needed would show up as a failed equality check in step (iv), not as a quiet quality drop.

BAGEL's speedup is about 1.00× for a reason the audit reports before any code is written: its native path already skips the vocabulary projection on image calls and already reuses KV, so almost nothing is left to remove.

We will move Tables A12–A13 to Sec. 4 and add this table and the four-step procedure.

AuthorsOpenReview ↗

Response to Reviewer SB5h (2/3): W2, effectiveness

W2. Effectiveness

(a) Literature timings. We agree they are context only, and no claim in the paper relies on them. Every speedup we claim is a paired measurement on our own hardware, with weights, prompts, seeds, sampler and policy held fixed. Show-o2 is not the only model with such paired evidence: Lumina-DiMOO (1.06×) and OmniGen2 (1.12×) are measured in the same way on the same two full benchmarks (Appendix I.5; table under W1). We will remove the literature latencies from Tables 1–2 and bring the paired transfer results into the main text.

(b) The 3× comes from stacking several strategies. You are right, and as your first strength notes, only part of the 3× is ours. It splits into two factors ("+ FlashU" is FlashU's policy alone; "+ ours" adds our rewrites on top):

Benchmark, scaleNative (s)+ FlashU (s)+ ours (s)Policy factorRewrite factorTotal
GenEval, 1.5B10.6295.7833.3121.84×1.75×3.21×
GenEval, 7B11.5146.5683.7181.75×1.77×3.10×
DPG-Bench, 1.5B5.3102.9731.8441.79×1.61×2.88×
DPG-Bench, 7B11.5386.5753.7231.75×1.77×3.10×

FlashU's approximation policy gives 1.75–1.84× and changes the outputs. Our rewrites give a further 1.61–1.77× and change nothing. The second factor is measured with the whole policy stack held fixed, so the stacking does not produce it.

To answer your question directly, we changed the strategies you list one at a time and re-measured the rewrite factor (new). The protocol is that of Table 4: DPG prompts, seed 42, 20 timed prompts per run, three fresh-process repeats, mean ± SD. The default policy is 35 steps, CFG 10, FFN ratio 0.2, DLS 0.2, and the model is Show-o2-1.5B unless noted. "Hits / calls" is the number of accepted cache hits out of the model calls per image.

Change from the default policyHits / callsOff → on (s)Rewrite speedupIdentical
none (default)14 / 342.898 → 1.7631.644 ± 0.005×63/63
FFN ratio 0.114 / 342.914 → 1.7771.640 ± 0.008×63/63
FFN ratio 0.314 / 342.830 → 1.7431.624 ± 0.017×63/63
DLS off16 / 343.028 → 1.6931.788 ± 0.013×63/63
guidance adjustment off14 / 342.895 → 1.7611.644 ± 0.007×63/63
50 steps21 / 494.114 → 2.4451.682 ± 0.004×63/63
70 steps32 / 695.695 → 3.2401.758 ± 0.005×63/63
no FlashU policy (native, 50 steps)0 / 495.236 → 4.5461.152 ± 0.007×63/63
7B: default14 / 346.445 → 3.6241.778 ± 0.003×63/63
7B: DLS off16 / 346.902 → 3.4661.992 ± 0.006×63/63
7B: no FlashU policy (native, 50 steps)0 / 4911.368 → 9.7121.170 ± 0.005×33/33

For the cache interval, the submission already has the sweep (Table A10, 70 steps): Tc = 1 / 2 / 3 / 5 / 8 gives 1.16× / 1.44× / 1.61× / 1.74× / 1.73×, with 360/360 identical images at each setting.

The rewrite factor stays between 1.62× and 1.99× under every FlashU-style setting and is still 1.15–1.17× with no FlashU policy at all. One rule explains every row: the rewrites remove exactly the producers that the chosen policy leaves without a consumer, so the gain follows the number of accepted cache hits.

  • The FFN ratio and the guidance schedule do not change which steps hit the cache (14 hits), so the gain stays at 1.62–1.64×.
  • With DLS off, the two recalibration steps on which a hit cannot be used disappear. All 16 hits are accepted, and the gain rises to 1.79× (1.99× at 7B).
  • More steps or a longer cache interval give more hits and a larger gain, until Head B, the sampler and the final full-network steps dominate.
  • With no cache hits (native Show-o2, or Tc = 1), only VocabBypass applies: 1.15–1.17×.

We will add both tables to the paper.

AuthorsOpenReview ↗

Response to Reviewer SB5h (3/3): W3, batch size, uncertainty, deployment

W3. Experimental setup: batch size, uncertainty, deployment

(a) Batch size. We ran the batch study (new). Throughput under the default policy, without ("off") and with ("on") our rewrites:

ModelBatchOff (img/s)On (img/s)SpeedupIdentical images
Show-o2-1.5B10.3450.5671.644 ± 0.005×63/63
Show-o2-1.5B40.3800.6581.731 ± 0.002×132/132
Show-o2-1.5B160.4000.7011.753 ± 0.002×288/288
Show-o2-7B10.1550.2761.778 ± 0.003×63/63
Show-o2-7B40.1590.2941.852 ± 0.008×72/72

The gain does not shrink with batch size. It is slightly larger, and all paired images stay identical. This is what the design predicts: the CADD guard (Eq. 1) looks only at schedule-level flags (step index, cache interval, full-network region, recalibration, guidance mode) and never at the content of a sample, so every sample in a batch hits on the same steps. Each batch holds B images of one prompt with independent noise. Batch 4 is timed over 10 batches (5 at 7B) and batch 16 over 5, with three fresh-process repeats.

(b) Uncertainty. For quality there is nothing to put an error bar on: Exact-DLS returns the same response as its reference on all 24,769 examples, so the score difference is exactly zero. For latency, the controlled experiments already use three fresh-process repeats with small spread (Table 4: SD ≤ 0.04 s on means of 1.77–6.52 s). For the six full benchmarks we add 95% confidence intervals from a paired bootstrap over examples (10,000 resamples), computed from the per-example timings of the submitted runs:

BenchmarkExamplesReference → Exact-DLS (s)Speedup95% CIExact-DLS faster on
MME2,3740.823 → 0.7181.147×[1.138, 1.157]2,374
GQA12,5780.848 → 0.7151.185×[1.181, 1.190]12,578
MMB-EN4,3291.081 → 0.8171.323×[1.268, 1.382]4,329
MMMU9001.183 → 0.8651.367×[1.308, 1.429]899
MMStar1,5002.541 → 1.0872.338×[2.197, 2.473]1,500
AI2D3,0881.032 → 0.7811.321×[1.286, 1.357]3,088

Every interval is well above 1, and Exact-DLS is faster on 24,768 of the 24,769 paired examples. The bootstrap captures variation across examples. For variation between processes we have the three-repeat controls and one repeat of a full benchmark: Exact-DLS on MMStar was run twice, 17 days apart, with different RNG protocols. Accuracy was 50.5% and 50.3%, and mean latency 1.087 s and 1.078 s.

(c) Deployment. We have not run a production serving study and do not want to overclaim. Besides the batch result, three measured properties matter for deployment:

  1. The worst case is negligible. When no hit is accepted, the guard costs 0.03 ms per model call (+0.0145% over 510 paired calls, Fig. 8c).
  2. It fails safe. If any guard condition does not hold, the original path runs unchanged. 3,570 replayed steps covering accepted hits, cache refreshes, DLS recalibration, the full-network region and conditional-only guidance all match the reference with zero prediction error (Appendix I.2).
  3. The saving is predictable. Which steps hit is fixed by the schedule and not by the prompt (14 of 34 calls in the 35-step schedule). On Lumina-DiMOO, for example, all 2,212 paired images are faster, within 1.05–1.08×.

We do not claim serving throughput for understanding. Batching requests with different prefix lengths needs serving-level KV management, and under DLS each request selects its own layer set. This is outside the scope of the paper (Sec. 5), and we will state it explicitly in the experiments section.

We will add the batch table, the confidence intervals and a short deployment paragraph. We hope the new results address your concerns, and we are happy to run further checks.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers: summary of new results and a metric correction

General response to all reviewers

We thank all three reviewers for the careful and constructive reviews. Two concerns appear in more than one review, so we summarize our answers here. Details and tables are in the individual replies.

1. What DepGraph-U adds on top of FlashU (SB5h, 7x6t). The 3× over native Show-o2 is the product of two factors. FlashU's approximation policy gives 1.75–1.84× and changes the outputs. Our rewrites give a further 1.61–1.77× and leave every paired image pixel-identical. We claim only the second factor. It is this large because 38–44% of the latency of FlashU-accelerated Show-o2 goes to operators whose outputs are never read.

2. Transferability (all reviewers). The rewrites are written per architecture, as Sec. 5 states. What transfers is the dependency audit. The submission contains three ports (Lumina-DiMOO, OmniGen2, BAGEL), two of which were only in Appendix I.5. Each is 13–33 inserted lines, and paired outputs match. The gain on a model equals the unconsumed work it executes: 1.61–1.77× on Show-o2 + FlashU, 1.12× on OmniGen2 + TaylorSeer, 1.06× on Lumina-DiMOO, and about 1.00× on BAGEL.

New results in this response

  • The rewrite factor with each FlashU strategy changed one at a time, at 1.5B and 7B: 1.62–1.99×, and 1.15–1.17× with no FlashU policy at all. Outputs are identical in every case (reply to SB5h, W2).
  • Batch sizes 4 and 16: 1.73–1.85×, with identical images (SB5h, W3).
  • 95% confidence intervals for all six understanding benchmarks (SB5h, W3).
  • The matched native-KV comparison on MMStar suggested by Reviewer 4djm (4djm, Q2).
  • Forced tests for two edge cases, and an anonymized repository with the adapters and verification records (4djm, Q3).

A correction we want to flag ourselves. Table 3 reports MMStar as the mean over L2 categories (as declared in Table A3). The benchmark's standard metric is overall accuracy, and we will switch to it: Exact-DLS and its reference 51.74 → 50.47; Stable-Graph 51.92 → 50.60 (its reference 51.67 → 50.27); BAGEL 68.01 → 67.67 (native) and 66.69 → 66.67 (adapted). No model was rerun. Predictions, timings, response-equality counts and all speedups are unchanged. Where our replies quote MMStar accuracies, they use overall accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.