acceptodds
Under review as a conference paper at ICLR 2027

Cache3D : Adaptive Caching for Faster Structured 3D Latent Diffusion

Abstract

Diffusion and rectified-flow models enable high-quality 3D synthesis, but their iterative inference remains computationally expensive. Transferring existing caching strategies to structured latent 3D generation poses reliability challenges: globally aggregated change estimates can obscure spatially concentrated latent evolution, while a straightforward token-output reuse adaptation mixes stale predictions with outputs recomputed under an altered attention context. We propose Cache3D, a training-free acceleration framework that separates the spatial granularity of cache-reliability estimation from the computational granularity of reuse. Cache3D groups sparse tokens into cells of the latent 3D grid, averages anchor-relative token drift within each cell, and uses the maximum cell response as a localized spatial risk metric. This signal guides adaptive selection among denoising-step reuse, transformer-block fallback, and full denoiser refresh, without treating individual tokens or cells as independent cache units. On the TRELLIS 2.0 example-image benchmark, Cache3D achieves 1.633 end-to-end speedup and reduces shape latent relative RMSE by 16.0% compared with EasyCache, while maintaining comparable geometric fidelity.

Primary area:
generative models
Keywords:
3D Generation Flow Matching Spatially Structured Caching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_HDyzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies training-free caching for accelerating diffusion-based 3D generation with structured latent representations. It is motivated by the observation that globally aggregated change metrics can overlook localized latent evolution, while direct token-output reuse can introduce attention-context inconsistencies, potentially degrading geometric fidelity. The proposed method introduces a spatially localized reuse criterion that averages anchor-relative token drift within 3D grid cells and takes the maximum across cells. This criterion is combined with established step-level and block-level residual reuse mechanisms to select among whole-step reuse, selective block recomputation, and full model evaluation.

Critical strengths

The paper identifies relevant challenges in transferring caching methods to structured 3D generation, particularly the risk of overlooking localized latent changes.

Critical weaknesses

The incremental contribution over closely related caching methods is insufficiently clarified, particularly the independent value of the cell-based reliability metric.

Questions for authors
  1. How does Cache3D differ from EasyCache, Δ-DiT, ProfilingDiT, and MixCache beyond the spatial reliability metric?
  2. Can the authors compare cell-based and global reliability metrics within the same execution framework at matched latency?
  3. Given the cell-level risk estimates, why not selectively reuse or recompute individual cells? Could this provide a better trade-off if recomputed tokens retain access to the full attention context?
6 comments
AuthorsOpenReview ↗

Response to Reviewer HDyz: Method Contributions and Spatial Pooling

Thank you for your careful review and constructive suggestions. Below, we clarify Cache3D's contributions, compare spatial and global reliability, and evaluate cell-selective recomputation.

[Q1] Differences from EasyCache, Δ-DiT, ProfilingDiT, and MixCache

Beyond the spatial metric, Cache3D couples online block selection with step-level anchor updates. EasyCache's basic policy switches between whole-step reuse and full evaluation; Δ-DiT uses stage-dependent block ranges; ProfilingDiT selects blocks by attention roles; and MixCache combines redundancy with profiled impact to choose caching granularity. Cache3D selects reusable blocks using EMA-smoothed residual variation along the current trajectory. After fallback, the updated prediction anchors subsequent step reuse, while block caches and stability scores are retained until a full refresh. Cache3D builds on established residual reuse; its contribution lies in combining spatial reliability with this computation recovery policy.

Table 1. Execution ablations from original Table 3. All variants retain spatial reliability.

VariantSpeedup↑RMSE↓LPIPS↓
Cache3D1.633×0.31670.0561
Fixed block selection1.608×0.32130.0575
Without block fallback1.600×0.38610.0713

Table 1 holds the spatial metric fixed. Removing fallback raises RMSE from 0.3167 to 0.3861 at similar acceleration, while fixed block selection gives 0.3213. These results show a substantial benefit from fallback and an additional gain from adaptive block selection.

[Q2] Cell-based versus global reliability at matched latency

We compare Cell-Mean-Max and Global Mean under the same residual reuse, block fallback, block selection, and cache-update rules, varying only the reliability metric and calibrated thresholds. Table 2 reports three operating points with pairwise speedup differences of at most 0.006×.

Table 2. Cell-based versus global reliability under the same execution framework at comparable measured speedups.

MethodSpeedup↑RMSE↓CD↓[email protected]↑LPIPS↓DINO↑
Lower-speed group
Global Mean1.472×0.25170.0092790.96280.04340.9874
Cell-Mean-Max1.477×0.24870.0092600.96270.04200.9877
Middle-speed group
Global Mean1.639×0.32380.0093490.96200.05960.9801
Cell-Mean-Max1.633×0.31670.0093100.96370.05610.9854
Higher-speed group
Global Mean1.827×0.42460.0094790.95830.09580.9535
Cell-Mean-Max1.822×0.39230.0094130.96010.08850.9605

Cell-Mean-Max achieves lower RMSE and LPIPS and higher DINO similarity at all three points. At the highest acceleration, RMSE falls from 0.4246 to 0.3923, a 7.6% reduction. These results support the benefit of spatial reliability at closely comparable measured speeds.

AuthorsOpenReview ↗

Response to Reviewer HDyz: Cell-Selective Recomputation

[Q3] Why not recompute selected cells with full attention context?

Cell-level risk identifies local input changes, while attention couples outputs across regions. Cache3D therefore uses cells to measure reliability and restores computation at the step/block level. To evaluate the proposed alternative, we select complete cells under a token budget and recompute their tokens with attention over all positions, using current K/V for selected tokens and cached K/V elsewhere. Sampling latency includes cell selection, data movement, and cache maintenance.

Table 3 compares Cache3D with cell-selective variants at different recomputation budgets. The local variants also differ in aggregation and anchor updates, so the comparison evaluates their overall implementation.

Table 3. Cache3D and cell-selective recomputation at different token budgets.

MethodSpeedup↑RMSE↓CD↓[email protected]↑LPIPS↓DINO↑
Cache3D1.633×0.31670.0093100.96370.05610.9854
Local 5%1.564×0.58790.0094850.95460.11300.9544
Local 20%1.419×0.59180.0095510.95140.10560.9569
Local 35%1.286×0.56390.0093270.95940.08750.9686
Local 50%1.133×0.39200.0091840.96220.06040.9800
Local 70%1.030×0.19600.0090330.96510.02720.9926

Cache3D is faster and achieves lower RMSE and LPIPS and higher DINO similarity than the 5–50% variants. At 70% coverage, quality improves but acceleration falls to 1.030×.

These results suggest that locating change and choosing which outputs to refresh require different considerations. Through attention, changes in one region can affect outputs in cells with little local input drift. Broader updates can improve fidelity, but reduce the computation saved and incur selection and cache-maintenance overhead. This supports our choice to measure risk locally while restoring computation at the step/block level.

Thank you for the response. The local recomputation experiments are convincing and have addressed most of my concerns. My remaining concern is the novelty and scope of the contributions. Could the authors clarify which parts of the mechanism build on prior work and which are newly proposed, and explain the incremental contribution of each component over its closest counterpart? I apologize for the late reply, which may leave insufficient time for additional experiments. Clarifying the key differences and the motivations or insights behind them would suffice. If the response establishes a clear and substantive incremental contribution, I would be inclined to raise my rating.

Reviewer_vUFGOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

Cache3D is a training-free acceleration framework that speeds up inference for structured-latent 3D generation models such as TRELLIS 2.0. Its key idea is to decouple the spatial unit used to estimate reuse confidence from the unit at which computation is actually reused. At each step where the latent is updated by the Euler sampler, it selects one of three modes: Full refresh, Block Fallback, or Step Reuse. Confidence is computed from the per-token relative drift, averaged within each cell, and then taken as the maximum across cells, and the mode is chosen by comparing it against configured thresholds. On TRELLIS 2.0 and Toys4K, it outperforms other diffusion model acceleration methods.

Critical strengths
  • Diffusion models require heavy computation, so attempting to accelerate them without any additional training is a well-motivated and important problem.
  • Dividing steps into three modes for acceleration is a clear and clever formulation.
Critical weaknesses
  • The set of comparison methods is insufficient. Beyond simple caching or reuse, I understand that TaylorSeer [1], which predicts features via basis functions, and its follow-up works achieve very strong performance and high acceleration ratios these days. A comparison against these and other more recent methods is needed.
  • Among the compared methods, EasyCache and TeaCache achieve comparable performance, but there is no clear evidence that the gap between their results and those reported for Cache3D is meaningful. For example, [email protected] differs only at the fourth decimal place and the DINO score at the third, and I doubt these differences are statistically significant. The authors should show, through a statistical test such as a t-test or McNemar’s test, that Cache3D is significantly better.
  • Most diffusion acceleration papers report FLOPs (floating-point operations) in addition to simple speedup. It would be good to measure and report them here as well. This would make it easier to judge whether the speedup comes from an actual reduction in computation or from some other factor.
  • The method figure (Figure 2) is somewhat unclear in what it conveys and needs aesthetic improvement. What does the “Warm-up: Full Refresh → Adaptive Cache 3D → Full Refresh” at the top mean? Is it a diagram of the overall Cache3D flow? In addition, in the label under C (“BLOCK FALLBACK”), only the K wraps onto a new line, and likewise only the H in “FULL REFRESH”. This seriously undermines the figure’s professionalism. The area to the right of step D that indicates what is reused and what is recomputed could also be improved. Please improve readability, aesthetics, and interpretability overall.
  • The proposed method seems unrelated to 3D. Reducing the problem to defining modes over arbitrary tokens and steps and selecting among them is a clever approach, but I am not sure the method is 3D-aware. Without anything view-aware or 3D-aware, it appears it could be ported to image generation as is.
  • The method has many hyperparameters (cell width, threshold, minimum interval, EMA momentum, block multiplier, etc.), yet there is no study of them. There is no explanation of how sensitive the method is to them, whether there are failure modes, how they were tuned, or on what validation data. The absence of such a study makes it hard to be confident in the method’s reliability.
  • In the qualitative results, regardless of the method (Cache3D included), the generated 3D objects show many holes and blotchy artifacts. If the output of an accelerated model is not satisfactory, acceleration has little meaning. I therefore find it hard to say that acceleration is yet very useful in 3D.
  • A paper should always be self-contained and reproducible. This submission lacks the evaluation setup, evaluation measures, and dataset descriptions, in both the main text and the Appendix, and no code is provided as supplementary material either. This is a very critical problem.
  • I could not find the paper’s limitations anywhere. I believe an academic paper should clearly present what it can and cannot do, and this requires dedicating a section to discussing limitations.

[1] Liu, J., Zou, C., Lyu, Y., Chen, J., & Zhang, L. (2025, October). From reusing to forecasting: Accelerating diffusion models with taylorseers. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 15853-15863). IEEE.

Questions for authors

This reviewer believes the issues raised under Weaknesses are important questions that must be answered for this work to stand as an academic paper. I therefore request the following experiments or clarifications.

  1. Please run at least one or two forecasting baselines such as TaylorSeer and report their performance.
  2. For the quantitative metrics reported, please run a statistical test on the performance gap between the first- and second-ranked methods and show the results. This should not require much computation or time, so it seems very feasible. Please report in detail which test was performed and on how many samples.
  3. Please measure FLOPs for each method and report them in the revised paper.
  4. Please revise the figure to make it easier to understand.
  5. Please explain why the proposed method is 3D-aware, or why it can only be applied to 3D, or discuss its applicability to general image-generation diffusion models.
  6. For each of the many proposed hyperparameters, please run a hyperparameter robustness study with at least 3 to 5 different values.
  7. Please discuss the limitations clearly in a new section.
9 comments
AuthorsOpenReview ↗

Response to Reviewer vUFG (Part 1): Baselines and Quantitative Evidence

Thank you for your careful review and detailed suggestions. Your comments helped us strengthen the baseline comparisons and statistical evaluation. Below, we address forecasting baselines, computational cost, and the significance of the quality differences.

[Q1 & Q3] Forecasting baselines and computational cost

We added TaylorSeer and HiCache, which use Taylor and scaled-Hermite feature forecasting, respectively. Table 1 reports their speed, quality, and computational cost alongside the caching baselines. The Cache3D speed and quality values are retained from the original manuscript.

Table 1. Baseline comparison with reported speedup, quality, and computational cost. TFLOPs include Shape/Texture denoiser computation and associated prediction and controller arithmetic. HiCache geometry and appearance metrics use valid outputs after a decoding failure.

MethodSpeedup↑TFLOPs↓RMSE↓CD↓[email protected]↑LPIPS↓DINO↑
Cache3D1.633×717.880.31670.0093100.96370.05610.9854
EasyCache1.594×776.010.37690.0093400.96310.06810.9806
TeaCache1.582×759.460.34350.0093100.96360.05970.9846
TaylorSeer1.446×895.930.43600.0094120.96150.06530.9820
HiCache1.494×865.280.54520.0094220.96140.07470.9745

FLOPs are profiled separately on three inputs, counting each multiply–add as two operations. Full inference requires 1670.12 TFLOPs under this accounting scope, which covers Shape/Texture denoising and associated prediction and controller arithmetic. Sampling latency also includes cache and forecasting maintenance; early stages, decoding, and rendering are outside the FLOPs scope.

At the reported operating points, Cache3D is faster and uses fewer FLOPs than both forecasting baselines, with lower RMSE and LPIPS. This comparison supports its practical speed–quality advantage over the tested forecasting alternatives.

[Q2] Are the quality differences statistically meaningful?

We use two-sided paired t-tests on 64 common input assets, averaging views within each asset, with Holm correction over ten metric–baseline comparisons. Table 2 also reports percentile 95% confidence intervals from 10,000 paired asset-bootstrap resamples, without multiplicity adjustment.

The tests use a separate Cell-Mean-Max rerun at 1.633× (RMSE 0.3081, LPIPS 0.0597, DINO 0.9788), compared with TeaCache at 1.684× and EasyCache at 1.647×. Table 1 retains the original manuscript settings; the statistical results apply to the rerun configurations and their reported speeds.

Table 2. Local-rerun paired differences on 64 assets: baseline minus Cache3D. Positive differences favor Cache3D for RMSE, CD, and LPIPS; negative differences favor it for F-score and DINO. Confidence intervals are not multiplicity-adjusted.

MetricTeaCache ΔTeaCache 95% CITeaCache p (Holm)EasyCache ΔEasyCache 95% CIEasyCache p (Holm)
RMSE+0.035464[0.030698,0.040453]2.92e-20+0.033257[0.028274,0.038341]2.74e-18
CD+0.000025[0.000006,0.000044]0.0372+0.000025[0.000008,0.000042]0.0190
[email protected]-0.000300[-0.000921,0.000261]0.5793-0.000360[-0.001029,0.000274]0.5793
LPIPS+0.003860[0.002527,0.005171]1.89e-6+0.002379[0.000880,0.003780]0.0109
DINO-0.002174[-0.002892,-0.001452]1.31e-6-0.001462[-0.002255,-0.000626]0.0050

Overall, these paired comparisons support Cache3D's quality advantage over both baselines, with significant improvements in RMSE, CD, LPIPS, and DINO after Holm correction. The small [email protected] differences remain statistically inconclusive.

AuthorsOpenReview ↗

Response to Reviewer vUFG : Method Clarifications and Revisions

Thank you also for your suggestions on the method explanation and presentation. This part addresses 3D awareness, parameter sensitivity, visual quality, limitations, and reproducibility.

[Q4] Figure 2 and presentation

We have revised Figure 2 with a clear sampling timeline, readable Block Fallback and Full Refresh labels, and a consistent legend for reused and recomputed features. The top row shows warm-up full evaluations, adaptive intermediate steps, and a terminal full refresh. The revised caption explains that the controller selects one of three execution paths at each intermediate step: Step Reuse, Block Fallback, or Full Refresh.

[Q5] Why is the method 3D-aware, and could it apply to images?

Cache3D is 3D-aware through its reliability controller, which groups sparse tokens by their coordinates in the latent volume. For token positions pᵢ ∈ ℤ³, we compute

\qquad D_{\mathrm{cell}} =\max_{c\in\mathcal C} \frac{1}{|\mathcal I_c|} \sum_{i\in\mathcal I_c}\delta_i,$$ where s is the cell width, C contains occupied cells, and δᵢ is the anchor-relative token drift. A small changing region may have little influence on a global average, even when it affects a thin structure or small component. Cell-Mean-Max retains that region's response while averaging token fluctuations within each cell. Redistributing the same drift values across 3D locations can therefore change the refresh decision while preserving the global average. Our controlled comparisons test this use of spatial organization against global and nonspatial controls under the same execution framework. The cells measure reliability, while step/block execution preserves attention across the full token set. The method's 3D awareness thus comes from latent-space geometry. The same principle could be extended to images through 2D neighborhoods and a calibrated controller; evaluating that extension is a direction for future work. **[Q6] Hyperparameter sensitivity and configuration selection** Table 3 reports one-factor sensitivity studies using three values for cell width, EMA momentum, and each block multiplier, and four values for the minimum interval. The two thresholds are varied jointly in a separate sweep. These measurements use the sensitivity-study reference configuration, separately from the original manuscript aggregate. **Table 3.** Reported sensitivity studies. Ranges summarize observed values across the tested settings; the threshold row is a separate joint sweep. | Parameter | Tested values | Observed speed / quality range | | :--- | ---: | ---: | | Cell width | 2, 4, 8 | 1.554–1.651×; RMSE 0.2896–0.3144 | | EMA momentum | 0.5, 0.8, 0.95 | 1.616–1.643×; RMSE 0.3085–0.3099 | | Minimum interval | 2, 3, 4, 5 | 1.510–1.800×; RMSE 0.2908–0.3981 | | Shape multiplier | 1.0, 1.10, 1.25 | 1.616–1.654×; RMSE 0.2973–0.3317 | | Texture multiplier | 1.0, 1.25, 1.50 | 1.601–1.668×; LPIPS 0.0551–0.0580 | | Full/fallback thresholds | 0.12/0.072, 0.30/0.18, 0.48/0.30 | 1.472–1.681×; RMSE 0.2500–0.3711 | EMA momentum has little effect on RMSE within the tested range. Increasing the minimum interval from 3 to 5 raises speedup from 1.616× to 1.800×, with RMSE increasing from 0.3090 to 0.3981. Together, these results identify the reuse interval and thresholds as controls for the speed–quality trade-off, while illustrating the relative stability of the EMA setting over the evaluated range. **[Q7] Limitations** We have added a dedicated Limitations discussion in Section B of the supplementary material. It covers the evaluated backbone and datasets, transfer to other latent representations, and the speed–fidelity trade-off, including possible degradation of fine structures under aggressive reuse. **[W7] Visual artifacts and practical usefulness** Cache3D aims to reduce sampling time while preserving the quality of the underlying generator. We compare accelerated outputs with their corresponding full-inference outputs to measure the additional degradation introduced by caching. Its practical benefit is faster generation with acceptable quality preservation; artifacts inherited from the generator and quality loss under aggressive reuse remain relevant limitations. **[W8] Self-contained evaluation and reproducibility** We have expanded Section E of the supplementary material with the TRELLIS example-image and Toys4K evaluation setups, generation settings, cache configurations, geometry and appearance metrics, result aggregation, and timing scope. We will publicly release the implementation, fixed configurations, and evaluation scripts upon acceptance.
Reviewer_vUFGOpenReview ↗

Comment to Author's rebuttal

I thank the authors for the effort they put into the rebuttal experiments. Below I respond point by point to the experiments they ran.

[Q1&Q3] Forecasting baselines and computational cost. Thank you for running these experiments. I have two follow-up questions, and answers to them would fully resolve my concern on this point.

  1. Forecasting baselines perform very well in image and video generation (video is, in a sense, also 3D, with time as the extra axis). In your results, however, they perform noticeably worse than methods such as EasyCache and TeaCache. Is there a reasonable explanation, or empirical evidence, for this?
  2. Could you show qualitative results for the forecasting baselines? I am curious whether they produce 3D objects that differ more from the full-inference output but are actually of higher quality in themselves.

⠀[Q2] Are the quality differences statistically meaningful? This experiment may be meaningful, but the relatively small sample (64 assets) and the lack of a prior normality check make me question the significance results. Did the paired differences pass a normality test? If not, could you run a different test, such as the Wilcoxon signed-rank test, and report the results?

[Q4] Figure 2 and presentation. My concern here has been resolved.

[Q5] Why is the method 3D-aware, and could it apply to images? I am still not convinced that this makes the method 3D-aware. It seems to apply to any data with a grid-like structure, and it does not use anything specific to 3D, such as view consistency. Taken together with the forecasting results in [W1], if Cache3D works well in 3D without being 3D-aware, one would expect it to work even better in 2D or video, which could broaden the paper's impact considerably.

[Q6] Hyperparameter sensitivity and configuration selection. Is there a particular reason why only RMSE is reported for most configurations and only LPIPS for one? Could you also report the speedup for each configuration? I strongly encourage the authors to include, in the revision, the speedup and all metrics for every setting in this hyperparameter study.

[Q7] Limitations. This has been resolved.

[W7] Visual artifacts and practical usefulness. I think our views differ on what counts as acceptable quality. I understand the authors' position. With images, however, acceleration does not punch holes in the output. In 3D, it produces holes that can make the asset unusable as is, and fixing them may cost more than the acceleration saves. On this point I still believe the practicality is limited.

[W8] Self-contained evaluation and reproducibility. This has been resolved.

Considering all of the above, I maintain my current recommendation of Weak Reject.

Reviewer_fHSFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Cache3D accelerates structured 3D latent diffusion by using spatially pooled feature changes to control cache reuse. It aggregates token drift within spatial cells and uses the resulting risk estimate to choose between step-level reuse, partial block recomputation, and full recomputation. The spatial cells determine the refresh decision, rather than serving as independent cache units. Experiments on TRELLIS examples and Toys4K report acceleration with relatively small deviations from full inference. The appendices further examine the risk statistic and alternative pooling rules.

Critical strengths
  • The separation between spatially local risk measurement and coarse-grained execution is a sensible response to the structure of sparse 3D latents. It avoids requiring every token to support an independent cache update while retaining sensitivity to local changes. The partial block fallback is also useful: Table 3 shows that removing it increases feature error substantially at a similar speedup.

  • The evaluation considers both efficiency and deviations in the generated geometry and appearance. Appendix C adds useful evidence beyond the main tables, including correlations between the risk score and velocity error and controlled comparisons with TokenMax and CellMeanTop5. These analyses help explain the controller's behavior and make the practical contribution more credible.

Critical weaknesses
  • The evidence for the specifically spatial contribution is still narrower than the overall caching results. EasyCache already uses reuse of the output-minus-input residual; consequently, the central additional contribution here is the spatial controller and its interaction with block fallback, rather than the residual reuse rule itself. Table 5 is a useful control, but its closest alternatives are also local/extreme-value aggregation rules, and the reported RMSE differences are modest. Table 3's no-cell variant runs at a substantially different speed. A global-mean controller, or a controller using nonspatial token groups, with the same execution hierarchy and a matched speed would more directly establish why spatial grouping is necessary.

  • The quality advantage over the nearest practical baseline is not consistent across metrics. On Toys4K in Table 2, Cache3D and TeaCache have nearly identical speedups; Cache3D slightly improves feature RMSE, while TeaCache has better LPIPS and DINO similarity. This does not negate the useful acceleration, but it makes the claimed improvement over existing caching less decisive than the comparison with full inference suggests. A speed-quality curve using the same threshold-selection protocol would show whether the proposed controller gives a meaningful operating advantage or mainly selects a different trade-off. The Fast3DCache-style comparison should also be interpreted within the explicitly disclosed partial adaptation, rather than as a complete comparison with the original system.

Questions for authors
  • At matched speed, how does spatial cell pooling compare with global averaging and nonspatial grouping when residual reuse, fallback, and all other controller settings are held fixed?
  • Can the comparison with TeaCache be shown across several operating points, with the evaluation sample count and threshold-selection protocol stated? Would the relative differences remain consistent under repeated sampling?
1 comment
AuthorsOpenReview ↗

Response to Reviewer fHSF : Spatial Controls and Speed–Quality Comparisons

Thank you for your careful review and constructive suggestions. We appreciate your recognition of the separation between spatial risk estimation and step/block execution. Below, we provide additional spatial controls and comparisons across acceleration levels, and clarify the scope of the Fast3DCache-style baseline.

[Q1] Does spatial pooling help under the same execution framework?

We compare Global Mean, Nonspatial Mean+Max, and spatial Cell-Mean-Max under the same residual reuse, block fallback, block selection, and cache-update rules. Only the reliability metric and calibrated thresholds vary. Nonspatial grouping preserves group sizes while randomizing membership, allowing us to assess the contribution of spatial locality separately from mean–max aggregation.

Table 1. Reliability controls under the shared execution framework. Results are reported at closely comparable measured speedups. Nonspatial grouping seeds are averaged within each input.

RuleSpeedup↑RMSE↓CD↓[email protected]↑LPIPS↓DINO↑
Lower speed
Global Mean1.472×0.25170.0092790.96280.04340.9874
Nonspatial Mean+Max1.479×0.24920.0092630.96260.04230.9875
Spatial Mean+Max1.477×0.24870.0092600.96270.04200.9877
Middle speed
Global Mean1.639×0.32380.0093490.96200.05960.9801
Nonspatial Mean+Max1.632×0.32090.0093340.96220.05850.9812
Spatial Mean+Max1.633×0.31670.0093100.96370.05610.9854
Higher speed
Global Mean1.827×0.42460.0094790.95830.09580.9535
Nonspatial Mean+Max1.815×0.41860.0094610.95910.09250.9547
Spatial Mean+Max1.822×0.39230.0094130.96010.08850.9605

Table 1 shows that spatial grouping improves quality at closely comparable speedups, with larger gains at higher acceleration. Near 1.8×, it reduces RMSE by 7.6% relative to Global Mean and 6.3% relative to Nonspatial Mean+Max, alongside lower LPIPS and higher DINO similarity. These results support the value of spatial locality in the reliability controller.

[Q2-1] Multiple operating points, sample counts, selection, and repeated sampling

We agree that the original Toys4K results show a trade-off across metrics. To examine the speed–quality relationship more broadly, Table 2 compares Cache3D, TeaCache, and EasyCache at three acceleration ranges on the example-image benchmark. Methods share evaluation inputs and rendering settings within each range. The middle group uses the original manuscript results, while the lower and higher groups add measurements at further operating points.

Section E of the supplementary material describes the evaluation data, generation settings, quality metrics, and timing protocol. Table 2 reports the achieved speedups for each method.

Table 2. Example-image comparison at three operating ranges. The middle group reproduces the original manuscript; the other groups are additional measurements. Speedup excludes decoding, rendering, and export.

MethodSpeedup↑RMSE↓CD↓[email protected]↑LPIPS↓DINO↑
Lower speed
TeaCache1.482×0.27400.0093270.96230.04750.9843
EasyCache1.459×0.26260.0093160.96260.04310.9873
Cache3D1.477×0.24870.0092600.96270.04200.9877
Middle speed: original manuscript
TeaCache1.582×0.34350.0093100.96360.05970.9846
EasyCache1.594×0.37690.0093400.96310.06810.9806
Cache3D1.633×0.31670.0093100.96370.05610.9854
Higher speed
TeaCache1.823×0.43850.0094830.95790.09990.9525
EasyCache1.837×0.44060.0094950.95760.10110.9508
Cache3D1.822×0.39230.0094130.96010.08850.9605

Across these example-image operating points, Cache3D achieves lower RMSE and LPIPS and higher DINO similarity, supporting a favorable speed–quality trade-off beyond feature RMSE alone.

[Q2-2] Clarification of the Fast3DCache-style baseline

Fast3Dcache accelerates the initial sparse-structure generation stage of TRELLIS, where its PCSC scheduler uses evolving voxel occupancy. Cache3D targets Shape-SLat and Texture-SLat denoising, whose sparse coordinates remain fixed. For a comparison on these denoisers, we adapt SSC-based ranking, subset transformer execution, and stitching of cached and recomputed predictions. The name Fast3DCache-style identifies this partial adaptation. Its results compare token-output reuse with our strategy within the same stages; a comparison of the complete systems would additionally require the original PCSC scheduler and sparse-structure stage.

Comments on the paper

3 comments
AuthorsOpenReview ↗

For All Reviewers

We thank all reviewers for your constructive suggestions. We have added the following evidence in response to your questions:

  1. Spatial contribution. We added global and non-spatial controls under the same execution framework at comparable speeds to assess the value of spatial grouping.

  2. Baseline evaluation. We expanded TeaCache and EasyCache comparisons across acceleration levels, added TaylorSeer and HiCache, and reported FLOPs alongside speed and quality.

  3. Further analyses. Paired statistical tests, cell-selective recomputation experiments, and parameter sensitivity studies provide additional evidence on quality differences and design trade-offs.

  4. Clarifications and presentation. We clarified 3D awareness, differences from prior work, and baseline scope, improved Figure 2, and expanded evaluation details and limitations.

The corresponding revisions in the main text and appendices are highlighted in blue.

Dear Reviewers fHSF & HDyz,

Could you take a moment to review the authors' responses and let them know whether any concerns remain or require further clarification?

Best,

AC

AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank the reviewers and the Area Chair for the time and care they devoted to our paper across several rounds of discussion. Their questions helped us clarify what Cache3D contributes and evaluate when its speed–fidelity trade-off is most favorable.

  1. Contribution and design choices. We clarify how Cache3D builds on residual caching and uses local changes in the 3D latent to guide step- and block-level computation. Near 1.8× speedup, spatial grouping reduced shape-latent RMSE by 7.6% relative to global averaging and 6.3% relative to nonspatial grouping. The block fallback ablations and cell-selective recomputation experiments further explain our choice to measure changes locally and restore computation at the step or block level.

  2. Baseline comparisons and visual quality. We compare Cache3D with TeaCache and EasyCache across three speed ranges and add comparisons with TaylorSeer and HiCache, including FLOPs and an analysis of how refresh intervals affect forecasting error. The new visual examples show both where forecasting produces smoother surfaces and where Cache3D better preserves colors and material details visible in the input.

  3. Statistical and geometric checks. In a paired evaluation on 64 assets, Wilcoxon signed-rank tests with Holm correction showed significant improvements over both caching baselines in RMSE, CD, LPIPS, and DINO. F-score differences remained inconclusive. At 1.633× speedup, Cache3D retained an average of 99.68% of the reference foreground interior across matched views. We also report cases with larger local losses.

  4. Sensitivity and reproducibility. Appendix C now reports speedup and all five quality metrics for each tested parameter value, making the trends across settings visible. We also revised Figure 2 to clarify the three execution paths and expanded the descriptions of the evaluation protocol, metrics, and limitations.

The individual responses provide the detailed results. The corresponding revisions in the manuscript and supplementary material are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.