- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes HASTE, an inference acceleration framework for diffusion-based image decomposition into editable RGBA layers. The approach exploits differences in convergence across output layers and spatial sparsity within each layer’s alpha support. A lightweight AlphaNet estimates opacity from cached clean latent predictions. An online scheduler uses support-restricted feature changes and layer-specific sensitivity calibration to determine which layers require fresh computation. Within selected layers, computation is restricted to the estimated alpha support. Skipped tokens bypass the Transformer and advance using cached clean endpoints, while persistent KV caches allow freshly evaluated tokens to access context from skipped positions. Warmup, periodic full refreshes, and final refinement mitigate accumulated errors
Critical strengths
Alpha support provides a task-specific cue for allocating computation. Restricting scheduling statistics to this support is particularly sensible for small foreground layers, where full-canvas averaging could dilute meaningful changes.
The diffusion backbone remains frozen, and the auxiliary opacity predictor is small. The additional pipeline experiments provide useful evidence of portability within layered image generation.
Critical weaknesses
The distinctive contribution of the hierarchical policy is not sufficiently isolated. Table 3 reports 3.07× speedup for temporal-only scheduling and 3.19× for complete HASTE. Adding spatial selection reduces latency from 31.74 s to 30.49 s, an incremental reduction of approximately 3.9%. However, Appendix D.5 states that temporal-only timing includes execution-audit overhead, whereas complete HASTE uses normal sampling times.
Can you compare global scheduling, per-layer scheduling without alpha-restricted scoring, and the proposed support-aware scheduling using the same endpoint/KV implementation and matched computation budgets? Please also measure temporal-only and complete HASTE with identical timing instrumentation.
How sensitive is the final decomposition to missed alpha-support regions? An oracle-support control would help separate support-estimation error from endpoint and KV staleness. Please also specify the full-update guard conditions and report their trigger frequencies and runtime costs.
Questions for authors
Please see weakness part.
4 comments
Response (Part II)
Weakness 3: How sensitive is the final decomposition to missed alpha-support regions? An oracle-support control would help separate support-estimation error from endpoint and KV staleness. Please also specify the full-update guard conditions and report their trigger frequencies and runtime costs.
Response: Thanks for the catch! Our controlled experiments show that HASTE is reasonably robust to missed alpha-support regions, while support-estimation achieves near-oracle-support fidelity at lower computational cost.
(1) Robustness to Missed Alpha-Support Regions.
Even with 20% additional missed support, HASTE's decomposition quality remains stable. Moreover, alpha-support achieves comparable fidelity to oracle-support while reducing latency by 23.3%. These findings suggest that moderate support-estimation errors have limited impact under HASTE's recovery mechanisms.
| Support Source | Latency (s) ↓ | GT αIoU ↑ | GT RGB L1 ↓ | LPIPS ↓ |
|---|---|---|---|---|
| Alpha-support (HASTE) | 30.72 | 0.88738 | 0.05071 | 0.01341 |
| Oracle-support | 37.23 | 0.88653 | 0.05121 | 0.01297 |
| Oracle + 10% missed | 37.38 | 0.88730 | 0.05086 | 0.01334 |
| Oracle + 20% missed | 37.11 | 0.88703 | 0.05074 | 0.01346 |
(2) Full-update guards frequencies and runtime costs are little.
Across 500 images (25,000 sampling steps), full-update guard triggered conditions occur only 0.68 times per image, accounting for 5.20% of total sampling time. In contrast, selective/cached updates cover 77.3% of sampling steps while consuming only 20.96% of runtime. These results demonstrate that HASTE achieves substantial computational savings with modest full-update guard overhead.
| Component | Total steps | Mean steps /image | Mean time/image | Runtime share |
|---|---|---|---|---|
| Warmup | 3000 steps | 6.00 | 12.786 s | 41.76% |
| Periodic refresh | 1830 steps | 3.66 | 7.683 s | 25.09% |
| Final refinement | 500 steps | 1.00 | 2.124 s | 6.94% |
| Selective / cached updates | 19330 steps | 38.66 | 6.416 s | 20.96% |
| Others | — | — | 0.018 s | 0.06% |
| Full-update Guard refresh | 340 steps | 0.68 | 1.592 s | 5.20% |
| Total sampling | 25000 steps | 50.00 | 30.617 s | 100.00% |
These findings suggest that HASTE's recovery mechanisms help limit the impact of missed support, while maintaining manageable runtime overhead.
Thank you for the reply.
Thank you for the detailed responses and additional experiments. The matched-budget comparisons, oracle-support analysis, and runtime breakdown substantially address my concerns about the effectiveness and efficiency of HASTE. I appreciate the clarification regarding the timing instrumentation. These results strengthen the paper's empirical evidence.
We are happy to hear that our response addressed your concerns. It was a pleasure to discuss with you! Wish you all the best and continued success in your scientific pursuits!
Response (Part I)
We sincerely appreciate the time and effort you have devoted to reviewing our manuscript.
Response: We thank the reviewer for the careful observation. We clarify two points.
1. Timing instrumentation is identical in Table 3.
Temporal-only and complete HASTE in Table 3 were measured with the same execution-audit instrumentation, and its overhead is included in both latencies. Appendix D.5 incorrectly stated that complete HASTE used normal sampling times. This was a documentation error, not a measurement inconsistency, and it has been corrected. The 3.07× vs. 3.19× comparison is therefore based on identical timing instrumentation.
2. Table 3 is not the right setting to isolate the spatial contribution.
Table 3 uses the long-trajectory T=50 setting, where temporal redundancy dominates. Layer-level/temporal skipping already removes most redundancy, leaving little spatial redundancy. The modest +3.94% gain is thus expected and does not undermine the two-layer design.
The independent value of Alpha-Guided Spatial Selection is clear when spatial redundancy is high. Under matched quality, adding spatial selection yields an additional 9.32%–16.35% latency reduction over layer-level scheduling. On Ming, spatial selection alone (1.41×) even outperforms layer-level scheduling alone (1.27×). Temporal and spatial sparsity target complementary redundancies; neither dominates universally.
These clarifications and ablations have been incorporated into the revision.
Response: We thank the reviewer for this valuable suggestion. Under nearly identical FLOPs(~ 4.41P) and query budgets (~ 29.44%), our support-aware scheduling consistently outperforms global and per-layer scheduling in both efficiency and decomposition quality. All three strategies share the same endpoint-reuse and KV-cache implementation, demonstrating the contribution of our proposed support-aware scheduling policy.