- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes GeoShrink, a training-free acceleration method for diffusion transformers that preserves the original solver grid while reducing model evaluations through sparse anchor queries. The method introduces a geometric retention rule to predict velocity fields from historical model outputs, along with a geometric anchor scheduling strategy to allocate exact model evaluations under a fixed computational budget. The authors provide theoretical analysis of the retention mechanism, anchor scheduling, and prediction error propagation. Extensive experiments across image, video, motion, audio, music, and 3D generation demonstrate strong acceleration performance and improved generation fidelity compared with existing training-free acceleration baselines.
Critical strengths
- The proposed solver-level velocity prediction is simple, training-free, and architecture-agnostic, offering an elegant alternative to model-specific feature caching methods.
- The method is supported by clear mathematical analysis and extensive experiments across multiple generative domains, demonstrating strong acceleration performance while preserving generation quality.
Critical weaknesses
The theoretical connection between geometric anchor scheduling and actual velocity-field prediction accuracy requires further clarification.
Questions for authors
Thank you for the solid work. The paper is well presented and supported by comprehensive experiments. My only question concerns the geometric anchor scheduling strategy. Could the authors further explain why geometric spacing is preferred and how it relates to velocity-field prediction accuracy? A clearer justification would strengthen the theoretical framework. I would like to raise my score to 4 if this concern be clearly clarified.
4 comments
Experimental protocol and schedule comparison
We evaluate the schedules on SD3.5-Medium at , using Pick-a-Pic-100, CFG 4.5, the original 28-step reference grid, and five random seeds. Each schedule uses seven exact model queries. The first three anchors and the terminal anchor are fixed; only the intermediate anchors change. All schedules use the same solver, sigma schedule, prediction rule, and sampling configuration.
The manuscript reports anchor indices in reversed order (28 to 1):
| Schedule | Anchor indices (28 → 1) | |
|---|---|---|
| Geometric | 28, 27, 26, 24, 21, 14, 1 | 2.333 |
| Cosine | 28, 27, 26, 22, 14, 5, 1 | 4.000 |
| Log-SNR | 28, 27, 26, 17, 5, 2, 1 | 9.000 |
| Uniform-Sigma | 28, 27, 26, 14, 8, 4, 1 | 12.000 |
| Karras | 28, 27, 26, 6, 3, 2, 1 | 20.000 |
To test whether the schedule-level bound is reflected in actual field prediction, we additionally evaluate the true model output at each skipped timestep using the accelerated trajectory's current latent, timestep, conditioning, and CFG. These diagnostic forward passes do not update the sampler and are excluded from the seven-query budget. We compare that output with GeoShrink's predicted field at the same state.
| Schedule | PSNR ↑ | SSIM ↑ | LPIPS ↓ | MSE ↓ | Cosine ↑ | Angle (°) ↓ | Relative L2 ↓ |
|---|---|---|---|---|---|---|---|
| Geometric | 19.851 | 0.7355 | 0.3606 | 0.1309 | 0.9716 | 12.43 | 0.2256 |
| Cosine | 19.484 | 0.6537 | 0.4386 | 0.1431 | 0.9689 | 13.13 | 0.2353 |
| Log-SNR | 18.108 | 0.7055 | 0.4523 | 0.1368 | 0.9699 | 13.08 | 0.2320 |
| Uniform-Sigma | 17.733 | 0.7023 | 0.4155 | 0.1315 | 0.9711 | 12.73 | 0.2266 |
| Karras | 16.496 | 0.6159 | 0.6380 | 0.1368 | 0.9710 | 12.73 | 0.2289 |
Geometric has the smallest gap-expansion ratio and the best result on each reported generation and same-state velocity metric. The MSE difference from Uniform-Sigma is small (0.1309 versus 0.1315); the evidence should therefore be read as a consistent but modest advantage in this five-seed diagnostic, not as a universal law that a smaller always produces lower empirical MSE.
Finally, Appendix E.12 makes the connection to the full solver trajectory explicit. For an augmented solver state and update maps satisfying the stated stability assumptions, Proposition 9 gives
\left\lVert \widetilde X_0-X_0^\star \right\lVert_2 &\le \sum_{n=1}^T \beta_n \left\lVert \epsilon_n \right\lVert_2 \qquad \cdot \prod_{k=1}^{n-1} \alpha_k. \end{aligned}$$ where $\epsilon_n$ is the field defect at stage $n$. This is a conditional perturbation bound: it connects local field errors to final solver-state deviation under explicit assumptions on the solver. It does not claim that $\gamma$ alone deterministically bounds final image error. The revision therefore clarifies the chain of reasoning at three levels: Section 2.3 (pages 4–5) derives the schedule-level exposure bound and the unique continuous minimax solution; Section 3.6 (Table 7, page 10) reports generation fidelity and same-state velocity diagnostics; and Appendix E.12 (pages 42–44) gives the full-grid error-propagation result. For completeness, the broader evaluation results in Appendix D.3 are organized as HPSv3 in Table 11 and the combined GenEval/Qwen-Image-Bench results in Table 12(a)–(b).Concerns Addressed
Thank you for the detailed response and additional experiments. The authors have fully addressed my concern regarding geometric anchor scheduling. The theoretical explanation and comparisons with alternative schedules provide convincing support for the proposed design. I have two minor suggestions for the final revision. First, I recommend including the new scheduling comparisons and velocity-field prediction results in the main paper, as they provide valuable evidence for the effectiveness of geometric anchoring. Second, it would be helpful to clarify the distinction between the theoretical bound on prediction horizon and the empirical improvement in prediction accuracy. But I still appreciate the authors' efforts and decide to raise my score to 4.
Thank you very much for your thoughtful follow-up. We are glad that our explanations and additional experiments have addressed your concerns.
Regarding your first suggestion, the table above includes both scheduling comparisons and velocity-field prediction results. Specifically, Cosine similarity, Angle, and Relative L2 measure the agreement between the predicted velocity and the reference velocity under different scheduling strategies, while PSNR, SSIM, and LPIPS measure the similarity between the generated images and their reference images.
We also appreciate your second suggestion. We will add a clearer discussion distinguishing the theoretical bound on the prediction horizon from the empirically observed improvement in prediction accuracy as soon as possible.
Thank you again for your constructive feedback and helpful suggestions.
Thank you for the positive assessment of GeoShrink's solver-level prediction rule and its experimental coverage. We address the question about the connection between geometric anchor spacing and velocity-prediction accuracy below.
Q1. Why is geometric spacing preferred, and how does it relate to velocity-field prediction accuracy?
The schedule controls how quickly the prediction horizon can grow after the three warm-up queries. This is the quantity that enters the geometric retention rule.
Let the anchors be ordered in an increasing progress coordinate s,
\qquad h_j=s_{j+1}-s_j.$$ The first three queries are consecutive, so $h_1=h_2=1$. Define the maximum adjacent gap-expansion ratio $$\begin{aligned} \gamma &= \max_{2\le j\le B-2}\frac{h_{j+1}}{h_j}. \end{aligned}$$ For a skipped update in the cell following $h_j$, Lemma 1 gives $$0\le |\rho_t^{(s)}| \le \frac{h_{j+1}}{h_j} \le \gamma.$$ Since the geometric exposure is $R_t^{(s)}=(\rho_t^{(s)})^2\sin^2\theta_t$, the retention rule satisfies $$R_t^{(s)}\le \gamma^2, \qquad \omega_t^{(s)} =\frac{1}{1+R_t^{(s)}} \ge \frac{1}{1+\gamma^2}.$$ Thus, a smaller $\gamma$ limits the worst-case normalized horizon and gives a larger lower bound on the retained fraction of the latest innovation. The paper also makes explicit the distinction between the progress coordinate used to construct the schedule and the physical $\sigma$ coordinate used by the solver. If $-\sigma=\psi(s)$ and $$0<m\le \psi'(s)\le M_\psi, \qquad K=\frac{M_\psi}{m},$$ then the corresponding physical-coordinate guarantee is $$|\rho_t|\le K\gamma, \qquad \omega_t\ge \frac{1}{1+(K\gamma)^2}.$$ For an affine grid, $K=1$. For a nonuniform grid, the bounded-distortion factor is necessary; an index-space gap ratio alone does not certify the physical prediction horizon. ### Why the minimax solution is geometric Under a fixed number of exact queries, the total coverage is fixed: $$\sum_{j=1}^{B-1}h_j=T-1, \qquad h_1=h_2=1.$$ The schedule objective is $$\min_{\\{h_j\\}} \max_{2\le j\le B-2}\frac{h_{j+1}}{h_j} \quad \text{subject to}\quad h_1=h_2=1,\quad \sum_{j=1}^{B-1}h_j=T-1.$$ Let $\gamma$ be the maximum ratio of a feasible schedule. Recursively applying $h_{j+1}\le \gamma h_j$ gives $$h_j\le \gamma^{\max(j-2,0)}.$$ Therefore, $$T-1 =\sum_{j=1}^{B-1}h_j \le \sum_{j=1}^{B-1}\gamma^{\max(j-2,0)}.$$ The right-hand side is increasing in $\gamma$. Let $r_\star\ge 1$ be the unique solution of $$\sum_{j=1}^{B-1}r_\star^{\max(j-2,0)}=T-1.$$ Every feasible schedule must have $\gamma\ge r_\star$, and equality is attained only when every recursive inequality is tight: $$h_j=r_\star^{\max(j-2,0)}.$$ Hence, the unique continuous minimax solution is $$\begin{aligned} h_1,h_2,h_3,\ldots &= 1,1,r_\star,r_\star^2,\ldots. \end{aligned}$$ The solver uses discrete indices, so the continuous anchor locations are projected onto the original grid with order-preserving guards. The resulting integer gaps are approximately geometric; the reported $\gamma$ values are computed from those discrete gaps.