- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies training-free caching for accelerating diffusion-based 3D generation with structured latent representations. It is motivated by the observation that globally aggregated change metrics can overlook localized latent evolution, while direct token-output reuse can introduce attention-context inconsistencies, potentially degrading geometric fidelity. The proposed method introduces a spatially localized reuse criterion that averages anchor-relative token drift within 3D grid cells and takes the maximum across cells. This criterion is combined with established step-level and block-level residual reuse mechanisms to select among whole-step reuse, selective block recomputation, and full model evaluation.
Critical strengths
The paper identifies relevant challenges in transferring caching methods to structured 3D generation, particularly the risk of overlooking localized latent changes.
Critical weaknesses
The incremental contribution over closely related caching methods is insufficiently clarified, particularly the independent value of the cell-based reliability metric.
Questions for authors
- How does Cache3D differ from EasyCache, Δ-DiT, ProfilingDiT, and MixCache beyond the spatial reliability metric?
- Can the authors compare cell-based and global reliability metrics within the same execution framework at matched latency?
- Given the cell-level risk estimates, why not selectively reuse or recompute individual cells? Could this provide a better trade-off if recomputed tokens retain access to the full attention context?
6 comments
Response to Reviewer HDyz: Cell-Selective Recomputation
[Q3] Why not recompute selected cells with full attention context?
Cell-level risk identifies local input changes, while attention couples outputs across regions. Cache3D therefore uses cells to measure reliability and restores computation at the step/block level. To evaluate the proposed alternative, we select complete cells under a token budget and recompute their tokens with attention over all positions, using current K/V for selected tokens and cached K/V elsewhere. Sampling latency includes cell selection, data movement, and cache maintenance.
Table 3 compares Cache3D with cell-selective variants at different recomputation budgets. The local variants also differ in aggregation and anchor updates, so the comparison evaluates their overall implementation.
Table 3. Cache3D and cell-selective recomputation at different token budgets.
| Method | Speedup↑ | RMSE↓ | CD↓ | [email protected]↑ | LPIPS↓ | DINO↑ |
|---|---|---|---|---|---|---|
| Cache3D | 1.633× | 0.3167 | 0.009310 | 0.9637 | 0.0561 | 0.9854 |
| Local 5% | 1.564× | 0.5879 | 0.009485 | 0.9546 | 0.1130 | 0.9544 |
| Local 20% | 1.419× | 0.5918 | 0.009551 | 0.9514 | 0.1056 | 0.9569 |
| Local 35% | 1.286× | 0.5639 | 0.009327 | 0.9594 | 0.0875 | 0.9686 |
| Local 50% | 1.133× | 0.3920 | 0.009184 | 0.9622 | 0.0604 | 0.9800 |
| Local 70% | 1.030× | 0.1960 | 0.009033 | 0.9651 | 0.0272 | 0.9926 |
Cache3D is faster and achieves lower RMSE and LPIPS and higher DINO similarity than the 5–50% variants. At 70% coverage, quality improves but acceleration falls to 1.030×.
These results suggest that locating change and choosing which outputs to refresh require different considerations. Through attention, changes in one region can affect outputs in cells with little local input drift. Broader updates can improve fidelity, but reduce the computation saved and incur selection and cache-maintenance overhead. This supports our choice to measure risk locally while restoring computation at the step/block level.
Thank you for the response. The local recomputation experiments are convincing and have addressed most of my concerns. My remaining concern is the novelty and scope of the contributions. Could the authors clarify which parts of the mechanism build on prior work and which are newly proposed, and explain the incremental contribution of each component over its closest counterpart? I apologize for the late reply, which may leave insufficient time for additional experiments. Clarifying the key differences and the motivations or insights behind them would suffice. If the response establishes a clear and substantive incremental contribution, I would be inclined to raise my rating.
Response to Reviewer HDyz: Method Contributions and Spatial Pooling
Thank you for your careful review and constructive suggestions. Below, we clarify Cache3D's contributions, compare spatial and global reliability, and evaluate cell-selective recomputation.
[Q1] Differences from EasyCache, Δ-DiT, ProfilingDiT, and MixCache
Beyond the spatial metric, Cache3D couples online block selection with step-level anchor updates. EasyCache's basic policy switches between whole-step reuse and full evaluation; Δ-DiT uses stage-dependent block ranges; ProfilingDiT selects blocks by attention roles; and MixCache combines redundancy with profiled impact to choose caching granularity. Cache3D selects reusable blocks using EMA-smoothed residual variation along the current trajectory. After fallback, the updated prediction anchors subsequent step reuse, while block caches and stability scores are retained until a full refresh. Cache3D builds on established residual reuse; its contribution lies in combining spatial reliability with this computation recovery policy.
Table 1. Execution ablations from original Table 3. All variants retain spatial reliability.
Table 1 holds the spatial metric fixed. Removing fallback raises RMSE from 0.3167 to 0.3861 at similar acceleration, while fixed block selection gives 0.3213. These results show a substantial benefit from fallback and an additional gain from adaptive block selection.
[Q2] Cell-based versus global reliability at matched latency
We compare Cell-Mean-Max and Global Mean under the same residual reuse, block fallback, block selection, and cache-update rules, varying only the reliability metric and calibrated thresholds. Table 2 reports three operating points with pairwise speedup differences of at most 0.006×.
Table 2. Cell-based versus global reliability under the same execution framework at comparable measured speedups.
Cell-Mean-Max achieves lower RMSE and LPIPS and higher DINO similarity at all three points. At the highest acceleration, RMSE falls from 0.4246 to 0.3923, a 7.6% reduction. These results support the benefit of spatial reliability at closely comparable measured speeds.