- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
S2I-dLLM is a training-free, task-aware accelerator for unified masked-diffusion models. Two diagnostics motivate it: tasks rank FFN channels differently, increasingly with depth, and evolve representations differently (Figure 1); target tokens drift more than conditioning tokens, with spatially correlated image drift in deep layers (Figure 3). Structurally, a Wanda-style saliency (Eq 2) keeps a per-task FFN prefix and a cosine criterion (Eq 4) bypasses layers. Token scheduling refreshes only high-drift image features, mixes MMU embeddings, and commits confident text tokens (App B.2). A final full-compute phase covers the last β of denoising (Eq 8). On MMaDA-8B-MixCoT (RTX 4090) it reports 2.39-2.43x on T2I (Table 1b) and, with token scheduling only, 15.12-30.04x on MMU (Table 1a) and 3.92-10.48x on text (Tables 2 and 6c), plus cross-model transfer (Table 3).
Critical strengths
-
The task-dependent-compute premise is measured, and a matched control at equal channel budget favors it. Channel-importance rankings of T2I and the two language-output tasks decorrelate with depth (Figure 1a), and at matched retained-channel counts a task-specific ranking scores above a shared one on all three control subsets (GSM8K 56.25 vs 50.78, Table 8a, L1094-1096) at similar latency. The subsets are small (128 GSM8K questions), but GenEval and AI2D-lite move the same way. FlashU already states this premise for unified models; the credit here is for testing it on a masked-diffusion model, where text is also generated by iterative denoising.
-
The depth-resolved analysis of image-target drift is a fresh diagnostic, and a small control favors the design it motivates. Image-target drift becomes spatially correlated in deeper layers, measured by Moran's I against a flat shuffled control (Figure 3b), and FlashU reports no such analysis. The paper turns it into depth-dependent refresh, token units in shallow layers and two-dimensional blocks in deep ones (App B.2.1, L880-883), which scores 0.5521 against 0.5208 for token-wise refresh on the 48-prompt GenEval control subset at similar latency (Table 8c). This is the most defensible novel piece of the paper.
-
The evaluation covers all three task types of a unified model and transfers to two other backbones. Task-routed configurations run every MMU, T2I, and text/code benchmark in Tables 1, 2, and 6 on MMaDA-8B-MixCoT against existing accelerators on the same backbone; recalibrated per model without training, S2I-dLLM reaches 2.07-4.71x on Dynin-Omni and Lumina-DiMOO on fixed subsets (Table 3). Latency is per-request wall-clock time at batch size 1 on an RTX 4090, with the timing protocol (App A.3) and hyperparameters (Table 5) disclosed. I weight this breadth because a unified model exists to serve all three tasks from one backbone.
Critical weaknesses
-
The headline MMU and text speedups come mainly from per-task recipes of the compared baselines, not from the structural and adaptive parts the paper proposes. Those runs use "full FFN width and all layers with token scheduling and prefix caching" (App B.4, L982-983): ReMix-style mixing for MMU (Eq 11) and Fast-dLLM-style commitment for text (Eq 12). Where adaptive layer selection is isolated, refreshing the skip set instead of freezing the first probe's set cuts latency by only 1.5-1.9%, with no score gain (Table 8b, L1097-1099). A component ablation on the full MMU and text benchmarks, with the structural half on, would show what the framework itself adds.
-
The structural half closely follows Flash-Unified, published in CVPR 2026 Findings and compared in no table. FlashU (Ke et al., 2026, arXiv:2603.15271) already ranks FFN neurons by activation L2 norm times summed absolute weight (the form of Eq 2), skips layers by input-output cosine recomputed at fixed intervals (cf. Eq 4-5), and switches between pruned and full FFNs by timestep (cf. Eq 8). Its introduction also claims "the first systematic dissection of the internal computational mechanisms of unified models", and Figure 1(b) follows its analysis (L1184). Running FlashU's structural parts on MMaDA-8B-MixCoT would show what this half adds, and L54 should credit that dissection.
-
Two headline T2I cells take different values in different tables for the same configuration. DPG generation is timed at 8.90s (2.43x) in Table 1b but 9.64s (2.24x) in Table 6b, although ImageReward "evaluates the generated DPG images without additional image generation" (App A.3, L821-822). Only S2I and dLLM-Cache move, each by the same amount, while Vanilla, Few-step Vanilla, and ML-Cache are identical; the fixed 35-call GenEval score also differs between Table 1b and Table 11. Reporting each table from one run with fixed seeds, or stating per cell which run it comes from, costs no experiments.
Questions for authors
- Can you compare Flash-Unified with your channel pruning and layer skipping alone on MMaDA-8B-MixCoT, in speed and quality? If not, what prevents running it on MMaDA?
- On the full MMU and text benchmarks, what do channel selection and layer skipping add on top of the token-only configuration with prefix caching? Figure 9 (L1473) reports one such run, on a single AI2D example.
- With your prefix caching, commitment thresholds (0.80 and 0.85, L983), and BF16 confidence (L815-816), how do ReMix on MMU and Fast-dLLM on text compare with S2I-dLLM in NFE, latency, and accuracy?
- Which runs produce the DPG latencies of 8.90s (Table 1b) and 9.64s (Table 6b), and the GenEval scores 0.5489 (Table 1b) and 0.5281 (Table 11)? How do 57.21 and 57.63 (Table 8b) arise from 96 and 128 questions?
- S2I-dLLM reaches 1.21-1.38x in the lighter 50-step development setting (Table 10). Can you time dLLM-Cache and ML-Cache at 35 NFE, to compare the 2.43x DPG speedup at matched calls and show the step-subsampling share?