- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The papers main idea appears to be simple and effective, however complex, over indexed, over use of different letters, or simply unclear notation makes it hard to understand. Let me note know that I did my best to understand it, however there could be parts which still not fully clear to me.
The goal of the paper is to accelerate the generation process of flow and diffusion models. The methods they suggest is to cache models outputs and trajectory states to skip some of the model evaluations along the generation trajectory. This is in contrast to other caching methods which suggest to cache the intermediates features of the model.
Given a pre-trained diffusion or flow matching model with parametrization (-prediction, -prediction, or -prediction), AutoTarget set a time parametrization , and a solver (e.g. Euler method). Denoting the states and model outputs along the trajectory by and (resp.), they propose a search algorithm to find a subset such that that trajectory can be approximated with for .
The method is validate empirically on Flux.1, PixArt-LCM, and HunyuanVideo, showing improvements compared to previous caching methods.
Critical strengths
The proposed AutoTarget method does not require to change the weights of the models, and only optimizes the inference algorithm.
The efficacy of AutoTarget is demonstrated on large scale models, both text-to-image and text-to-video. Additionally, though it is not stated clearly, my understanding is that all baselines were reimplemented and reevaluated for a fair comparison.
AutoTarget performance is quite consistently better than the considered baselines.
Critical weaknesses
I have two main concerns for the paper. First the presentation of the paper makes it very hard to follow. Second the even though AutoTarget is referred as caching method for diffusion and flow models, it is actually much more close to solver learning methods[1,2,3,4] and the paper does not discuss these at all.
Regarding the presentation issue, I apologize if my writing is not 100% clear, but this is only as I had a hard time following the method section.
Presentation:
-
The authors introduce the notation to denote a general solver step. But the only provided example is Euler solver I found only one use of this notation in equation 5. If this is indeed the case, adding redundant notation makes it harder for the reader to follow. If it is not the case, then currently it is unclear what other solvers are used.
-
The relation between -prediction, -prediction, and -prediction are well known and can be presented in a simple small table. Given the current state , current time, and one of the three prediction types the other two can be recovers, hence it is not clear why is it important which one is being cached. Additionally, the related model specific discussion at the beginning of the method section is hard to follow at this stage and it is also unclear. There is a use of both and , the difference between them is still not clear to me. Model specific discussion should probably place in or just before the experiment section.
-
While the index is clearly defined to to denote the times steps, between lines 162 to 264, there is a heavy use of the index even though its relation to the index is first defined in line 270.
-
It still not clear to me what does the letter index and what is its domain. It is first introduced by "Let r index a solver-facing cache target considered by AutoTarget", and the meaning of "solver-facing cache target" is still unclear to me. This seems important as later it is the parameter that the method minimize over in equation 13.
-
In equation 3 another two letters notations are introduced to the notation , and . The function is never explicitly defined, is it a conversion between the three prediction types? why not simply cache ? Later on, the letter is used to denote accumulated error.
-
In equation 4, the function is introduced, again this function not defined explicitly and I still do not understand it exactly what is does. The only thing clear is that the authors assume that the composition of and yields a an approximation to . This seems rather important point which is not presented clearly.
-
In equation 12, is introduced and again it is never explicitly defined in the text, and it is still not clear to me what does denotes. Again, it makes the text harder to follow.
-
In line 269 the supscript is introduce without any explanation, only later in line 285 in equation 13 the reader my understand that it is related to the minimizer w.r.t. . But this still does not make its meaning clear as it is already used inside the definition of the risk.
-
in the risk it is not clear how does is chosen.
learned solvers: Since the proposed method caches only the models outputs and states along the trajectory, and to my understanding the approximation for cached steps is a linear combination of previous states and model outputs, it is essentially a new solver. Thus it seems appropriate to include an extensive discussion about the relation of AutoTarget to such methods [1,2,3,4], and clearly state what its contribution on top of them. Additionally, it seems appropriate to include an empirical comparison with such methods.
[1] Zhao, Wenliang, et al. "Unipc: A unified predictor-corrector framework for fast sampling of diffusion models."
[2] Zheng, Kaiwen, et al. "Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics."
[3] Shaul, Neta, et al. "Bespoke non-stationary solvers for fast sampling of diffusion and flow models."
[4] Frankel, Eric, et al. "S4s: Solving for a diffusion model solver."
Questions for authors
As the paper already present a good empirical results, I would appreciate if the author could address the weaknesses I have raised in the "Weakness" section.
Also could the authors please clarify for which tasks were AI tools were used and for which tasks AI tools were not used? The ICLR recommendation is to clearly state both and I find the provided statement somewhat vague. For example does statement "AI-assisted tools to support academic writing, language editing" means AI model essentially written a draft of the paper (including equations) and the authors have verified it or the authors have written most of the paper and an AI model just polished the text, fixed type grammar, typos, and so on? Another example, do not state whether AI model was used to develop the suggest method or not?
5 comments
Reviewer response
In the authors' response they promised a revised version of the paper with improved presentation which was main concern at my initial review. Do the authors plan to upload this version before end of discussion period?
I want to stress that presentation was my main concern in the initial review because the paper is too hard to follow for me to properly assess the proposed method. Hence, I would not consider to increase the score before reviewing the revised version of the paper.
Moreover, the authors response regarding use of AI tools does not match the statement in paper: "We used AI-assisted tools to support academic writing, language editing, code development, debugging, and experiment implementation.". Following my inquiry, their response adds the use of AI tools for "mathematical derivations and formula development.". This suggest that indeed the Method section was heavily written by AI tools. While in itself, it is not a problem, I have strong concern that the authors did not verified the method section properly.
Yes—we have uploaded the revised manuscript to OpenReview prior to the end of the discussion period. Thank you for the candid feedback and for taking the time to re-evaluate our paper. Below, we summarize the presentation improvements made to Section 3 and detail how the authors verified the mathematical derivations and Method section.
1. Summary of Presentation Improvements in Section 3
To make the Method section substantially easier to follow, we restructured Sections 3.1–3.3 around concrete intuition, lighter notation, and clearer logical transitions:
- Concrete Four-Step Running Example (Section 3.1): Before introducing the general storage and reuse operators (), we added a self-contained four-step linear-flow Euler example () showing directly why caching velocity or clean endpoint yields the exact same update at step 2, whereas caching step displacement shifts the next state by .
- Reduced Subscript Overhead (Sections 3.1 & 3.3): We omitted the prompt index throughout Sections 3.1 and 3.2 when defining single-trajectory storage, reconstruction, and Euler equivalence, introducing only in Section 3.3 when averaging calibration risk across prompts.
- Denominator Conditioning & Solver Dependence (Sections 3.1 & 3.2): Under the linear-flow conversion table, we explicitly stated the positivity requirements ( for clean endpoints and for noise endpoints) and noted that small denominators can amplify tensor and finite-precision perturbations. In Section 3.2, we also brought forward the solver-comparison finding from Table 8 (Appendix D.1)—showing that switching from multi-step DPM-Solver++ to DDIM on PixArt-XL causes clean-endpoint caching to produce non-finite values and reverses the lower-ranked targets while remains best under both solvers.
- Clearer Calibration Motivation & Theoretical Scope (Section 3.3): We clarified why candidates residing in different parameterizations and numerical scales are mapped through and the frozen VAE into a shared decoded RGB space for comparison. We also kept the scope of the error-propagation bound (Eqs. 7–8) explicit: it characterizes error accumulation in the candidate-specific complete solver state rather than directly guaranteeing decoded RGB rankings (Eq. 6), which motivates fixing the separation-ratio acceptance gate () prior to held-out testing.
2. Consistency of AI Disclosure and Verification of the Method Section
We apologize for the inconsistency between our earlier response and the initial statement in the paper. We have updated the Declaration of AI Assistance (page 10) in the revised manuscript so that it explicitly and consistently states that AI tools assisted with mathematical derivations and formula development, in addition to code implementation, debugging, and linguistic editing.
Every derivation, formula, and algorithm in Section 3 and the appendices was independently checked, verified line-by-line, and cross-validated against our codebase by the authors:
- Analytical Verification of Euler Equivalence and Step-Size Coupling (Proposition 1, Eq. 4, Appendix B.1): The authors manually verified the induction proof establishing across velocity, clean-endpoint, and noise-endpoint reconstructions (Eqs. 10–17), as well as the exact single-step discrepancy for stored solver displacement (Eq. 19).
- Verification of Error-Accumulation Recursion and Scope (Eqs. 7–8, Appendices B.2–B.3 & E): We checked the complete-state local Lipschitz recursion and explicitly documented its boundaries in Section 3.3, Appendix B.3, and Appendix E so that state-space error bounds are not conflated with decoded RGB risk rankings.
- Code-Level Numerical Cross-Checks (Appendix A.1): All storage and reconstruction formulas () and pipeline conversions () were cross-checked directly against our implementation, including pre-set numerical equivalence checks on recorded calibration traces (e.g., BF16 terminal latent relative and decoded RGB MSE for PixArt-LCM's denoised and clean predictions).
We hope the revised Section 3 and these clarifications make the proposed method straightforward to follow and fully address your concerns. Please let us know if any remaining part of the derivation or presentation warrants further detail.
We thank the reviewer for the detailed comments. We reorganized the method section, made the cache formulas explicit, and expanded the discussion of fast and learned sampling solvers.
1. Method inputs, outputs, and procedure
The revised method starts by specifying its inputs: a pretrained model, a solver, sampling coordinates, and fixed positions for full model evaluations. AutoTarget compares cache representations under these conditions and returns a target and an acceptance decision.
Calibration first records uncached trajectories. All candidates are then tested from the same recorded states and preceding refresh positions. During generation, full evaluations refresh the selected cache; the remaining steps reconstruct the input required by the original solver from that cache.
2. Notation and explicit formulas
We now define notation before its first use:
A compact table now states what is stored and how it is reused for velocity, clean-endpoint, and noise-endpoint caching. Model-specific conversions are placed in the appendix. The native model prediction is itself included as a candidate.
We also explicitly define the scoring conversion Qj. It converts each reconstructed prediction into the latent estimate supplied to the frozen VAE. Calibration averages decoded squared RGB error over prompts, all planned reuse steps, and RGB elements. These steps are fixed before scoring. The lowest-error candidate is accepted when the runner-up/best error ratio reaches the preset threshold of 1.10.
3. Why compare representations that can be converted into each other?
The familiar conversions apply at a common state and time. Cross-step reuse additionally requires specifying how an earlier stored tensor is used with the current state and time. The revised text gives these storage and reconstruction formulas explicitly.
For representations that yield the same Euler trajectory, we provide a proof and group them together, testing one representative per group. For a stored solver displacement, we explain that it includes the anchor step size; reusing it when the current step size differs can change the update. This makes clear which candidates are equivalent and which implement different reuse rules.
4. Relation to sampling solvers and additional experiments
We added UniPC, DPM-Solver-v3, BNS, and S4S to the related-work discussion. They share the goal of reducing model evaluations. UniPC constructs predictor-corrector updates; DPM-Solver-v3 uses empirical model statistics; BNS and S4S fit solver coefficients to reference outputs. AutoTarget fixes the solver, coordinates, and evaluation positions and selects which tensor to retain between evaluations. Its calibration selects a discrete candidate without fitting solver coefficients.
We added PixArt-LCM experiments with 16 calibration prompts and 100 independent test prompts. All methods use four full model evaluations; PSNR and SSIM use the native eight-step output as reference. AutoTarget achieves 14.5864 dB PSNR, 0.5559 SSIM, and 0.5676 ImageReward, compared with 12.8336 dB, 0.4989, and 0.5091 for native four-step LCM sampling.
The experiment also includes UniPC and DPM-Solver++. Direct scheduler replacement on the LCM-distilled checkpoint produces texture artifacts, motivating an additional comparison on native PixArt-XL with model-appropriate sampling settings.
5. Declaration of AI assistance
The research idea, core methodology, and overall research direction presented in this paper were independently conceived and developed by the authors.
AI tools were used to assist with code implementation, experimental development, and debugging under the guidance and supervision of the authors. During manuscript preparation, the authors independently determined the main arguments, technical content, and overall structure, while AI tools were employed to improve linguistic clarity, readability, and academic expression.
AI tools also assisted in mathematical derivations and formula development. The resulting derivations and equations were subsequently reviewed and verified by the authors for correctness and consistency.
The authors take full responsibility for the scientific content, experimental results, mathematical correctness, and conclusions presented in this paper.