- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes SoftCache, which uses additional information from the staleness signal that dynamic caching methods typically use to decide whether to reuse or refresh the cache. The authors observe that the accumulated signal correlates with the relative error caused by reusing cached residuals. They use the remaining margin below the refresh threshold to choose the sampling step size: a larger margin allows a longer step, which can skip scheduled timesteps and land outside the original grid. At comparable speedups, they report better fidelity to the uncached model’s outputs according to the evaluated metrics.
Critical strengths
The paper presents a clear, simple, and well-motivated idea: using information from the cache refresh signal as a measure of cache reliability to choose larger sampling steps. The framework is well structured, easy to apply, and requires no additional training. It achieves notable fidelity improvements over the evaluated baselines. For example, on HunyuanVideo at 6.05× speedup, it achieves 22.27 dB PSNR, 0.739 SSIM, 0.267 LPIPS, and 0.141 DreamSim. Compared with SeaCache at 6.11× speedup, this represents 1.89 dB higher PSNR and 0.082 lower LPIPS. The experiments are comprehensive, covering FLUX.1-dev for image generation and HunyuanVideo and Wan2.1-T2V-1.3B for video generation, with SoftCache achieving the best fidelity among the compared methods at similarly high speedups.
Critical weaknesses
- The benefit of headroom at 7× is not entirely clear (Table C.4.1). LPIPS and DreamSim show only small improvements, which are difficult to assess without uncertainty estimates, while PSNR worsens.
- It would be helpful to clarify when jumps happen and how big they actually are for different thresholds . The tables do not clearly show this behavior during generation. Figure 1 provides a schematic illustration, but does not show the actual jump locations and sizes observed during sampling.
Questions for authors
see weaknesses
We thank the reviewer for the constructive feedback.
Headroom benefit (W1): The reviewer is right to point out that at 7× the headroom does not improve the average scores. However, the headroom does matter at lower speeds, e.g. at 3× removing it costs 0.8 dB PSNR and 0.029 LPIPS (Tab. C.4.1). Our main contribution involves the headroom together with longer jumps, as further discussed in the response to W1 of reviewer jLNv.
Jump locations (W2): We added Sec. B.2 to the revised version. This subsection includes Tab. B.2.1 with the start time, length and headroom of the jumps at each speed of the main tables (Tab. 1-2) on all backbones, along with the τ values that are used for each speed.
It shows that a run takes three to seven jumps across the whole trajectory, with almost half the jumps with a size bigger than six scheduled steps. Moreover, it shows that the headroom is soft rather than binary, so jump lengths vary with it instead of switching between a regular step and the cap.