- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an extension of drifting models that considers higher-order statistics of the model and data distributions. This paper introduces the tilted kernel-local expectation, which is a generalization of the mean shift term in drifting models; its derivative corresponds to the cumulant tensors associated with the kernel-localized distributions. The proposed training loss is constructed from the discrepancy between the cumulant tensors, enabling models to take higher-order statistics into account. In addition, this paper provides a theoretical analysis of the correction effect of the proposed method from the perspective of spectral analysis of the drifting dynamics. Experimental evaluations are conducted on a Gaussian mixture dataset and image datasets, and the proposed method achieves better scores than the baseline drifting models in most cases.
Critical strengths
- This paper is well-organized. The motivation and the technical background are clearly explained.
- The proposed method is supported by theoretical foundations. The construction of the tilted kernel-local expectation is sophisticated and successfully achieves considering higher-order statistics. In addition, theoretical analyses of its behavior, such as Proposition 3, are provided, which deepen the understanding of the proposed method and drifting models.ss.
Critical weaknesses
- Some parts lack sufficient explanation. Some examples are listed as follows.
- In Eq. (12), the derivative of with respect to is taken. However, it is unclear what representes, as does not appear in the quantities involved in the computation of involves.
- Is missing from Eq. (13)?
- In Eqs (11) and (13), summation over anchor points is taken. However, how the anchor point set is constructed remains unclear.
- There seem to be incomplete sentences around Line 268:
- "only in the optional extension of" what?
- "The stop-gradient operator is denoted by" what?
- In experiments, explanations of methods denoted by "Parallel-only" and "Orthogonal-only" are largely omitted, alghough it can be presumed that these are related to the cosine gate in Eq. (18).
- This paper mainly focuses on the Gaussian kernel and ignores the Laplacian kernel, which is practically used [1]. In addition, the spectral analysis of [2] also suggests that the Laplacian kernel might be favored for drifting models. While Proposition 3 ensures the correction effect of the proposed method, it seems that the cut-off issue of the Gaussian kernel still remains, as the exponential bottleneck in the high-frequency region is unchanged. One critical concern is that changing the kernel might have a greater impact on model performance than considering higher-order statistics. Experiments regarding the Laplacian kernel are not conducted. Further experimental comparison using the Laplacian kernel or further discussion on this point would enhance the demonstration of the effectiveness of the proposed method.
[1] M. Deng et al. "Generative modeling via drifting." arXiv preprint arXiv:2602.04770 (2026).
[2] E. Turan et al. "Generative drifting is secretly score matching: a spectral and variational perspective." arXiv preprint arXiv:2603.09936 (2026).
Questions for authors
Regarding Weakness 2, could further experiments using the Laplacian kernel or further discussion on the Laplacian kernel be provided?
4 comments
Response part 2 to Reviewer YkBC
W2: Kernel choice / Theoretical discussion
We thank the reviewer for raising this important point. We agree that the Laplace kernel is widely used in drifting models and that changing the kernel can have a significant impact. We have addressed this in both theory and experiments.
- Theoretical analysis
Our original theoretical discussion in Section 4.5/4/6 was mainly developed for the Gaussian case, where the covariance discrepancy admits a clean interpretation as a spatial derivative of the smoothed log-density ratio. However, the method itself is kernel-agnostic: Eq. (14) is defined through kernel-weighted local moments and does not rely on Gaussianity. To address this, we have completely rewritten Section 4.6 and Proposition 3 to cover general normalized nonnegative radial kernels with finite covariance. We explicitly provide the spectral rate and the covariance multiplier for both the Gaussian kernel and the radial Laplace kernel (Appendix A.5.2 and Figure 4). We show that for the Laplace kernel, the covariance correction strengthens finite-frequency damping while preserving the leading algebraic tail, providing a theoretical basis for why the method is not limited to the Gaussian case.
In normalized frequency , with , the revised result separates the baseline rate from the covariance multiplier:
\lambda_{1,k}(u)=-u\psi_k'(u),$$ $$W_{2,k}(u)=1+c_2\bigl[\psi_k''(u)+\psi_k(u)\bigr],\qquad c_2=\frac{\alpha_2\sigma_K^2}{\ell}.$$ We now provide closed-form instances for both kernels used in the experiments. For the Gaussian kernel $k=G_h$, $\sigma_K=h$ and $\psi_G(u)=e^{-u^2/2}$, giving $$\lambda_{1,G}(u)=u^2e^{-u^2/2},\qquad W_{2,G}(u)=1+c_2u^2e^{-u^2/2},\qquad c_2=\frac{\alpha_2h^2}{\ell}.$$ For the radial Laplace kernel $k_b$, $\sigma_K^2=(d+1)b^2$ and, with $z=u^2/(d+1)$, $$\psi_L(u)=(1+z)^{-(d+1)/2},\qquad \lambda_{1,L}(u)=\frac{(d+1)z}{(1+z)^{(d+3)/2}},$$ $$W_{2,L}(u)=1+c_2\frac{z(z+d+4)}{(1+z)^{(d+5)/2}}.$$ In both cases $W_{2,k}(u)\to1$ as $u\to\infty$, so the correction strengthens finite-frequency damping while preserving the leading high-frequency tail of the underlying kernel. These instances show that the benefit is not specific to the Gaussian case. We have also added a scope statement in Section 4.6 and Remark 1 clarifying which proposition applies to which experiment. **2. Laplace kernel in the ImageNet experiments.** We would like to clarify that our ImageNet experiments **already use the Laplace kernel**. We continue training from the publicly released Drifting checkpoint, which originally used the Laplace kernel. We deliberately kept the kernel unchanged to isolate the effect of our higher-order correction. Therefore, the ImageNet improvements (FID 1.75 to 1.61) are obtained under the Laplace kernel, directly demonstrating the effectiveness of the covariance correction in the Laplace setting at scale.I thank the authors for their reply and revised manuscript. Since the Critical Weaknesses I previously raised have been adequately addressed, I'll raise my score accordingly. However, I think that several passages in the revised text still require writing improvements, and I would ask the authors to address them in the final version. For example:
-
The revised definition of the stop-gradient operator in Line 273 is appreciated. However, since the notation is already used in Section 3, it would be more appropriate to introduce this definition earlier. Please reconsider the placement of this explanation.
-
I am satisfied with the additional explanation about the weight in Eq. (13). That said, I feel that the description in Line 231: "Here denotes the normalized generated-side kernel weight for anchor and particle " remains difficult to understand. I would suggest that provinding the explicit form of or adding a cross-reference to Eqs. (3) and (4) would make this part easier to understand.
-
In Line 303, the current text states "... extend the local-homogeneity spectral analysis of Turan et al. (2026) to a class of radial kernels". However, the assumption on the kernel in the spectral analysis of Turan et al. (2026) appears to be that it is an even kernel, and radial kernels already satisfy this assumption. Therefore, I'm not sure that it is accurate to claim that the analysis has been extended to radial kernels. Rather, extending it to higher-order statistics would be a more suitable characterization of the provided theorem.
-
The current manuscript appears to heavily rely on Turan et al. (2026) for the definition and explanations of the linearized McKean-Vlasov dynamics and the spectral rates and . For the paper to be more self-contained, I would recommend adding brief but sufficient explanations of these concepts, either in the main text or in the appendix, rather than leaving these details to Turan et al. (2026).
Thank you very much for your thorough feedback
Thank you very much for raising your score and for your thorough feedback. We value all your suggestions and will revise the main text to improve readability and manuscript self-containedness.
Response part 1 to Reviewer YkBC
W1: Clarifications on formulas, terminology, and missing explanations
Thank you for identifying these ambiguities. We agree that the particle dependence, anchor construction, and directional ablations require clearer explanations. We have revised the relevant passages and marked the changes in blue. Equation numbers below refer to the original submission.
1. Meaning of X in Eq. (12)
We now define X=(x1,…,xn)∈Rn×d as the generated particle coordinates. The derivative acts on the finite-particle surrogate Em(X), whose generated-side local cumulants depend on X. Anchor locations, normalized weights, local scales ℓa, and data-side statistics are held fixed during this differentiation; the generated-side means and cumulants retain their dependence on X. This makes explicit the transition from the population hierarchy to the particle correction. We also retain the distinction between this surrogate and its independent-fold estimator in Appendix B.3.
2. Weight waiq in Eq. (13)
No additional factor is missing: the weight is already included in the weighted covariance,
\mu_{a,q}(X)&=\sum_j w^q_{aj}x_j,\\ C_{a,q}(X)&=\sum_j w^q_{aj}(x_j-\mu_{a,q}(X))(x_j-\mu_{a,q}(X))^\top. \end{aligned}$$ Writing $e_{ai}=x_i-\mu_{a,q}(X)$, fixed-weight differentiation gives $$dC_{a,q}=w^q_{ai}\left(dx_i e_{ai}^{\top}+e_{ai}dx_i^{\top}\right),$$ which produces the explicit $w^q_{ai}$ in Eq. (14). We have added these definitions and the differentiation step to make the role of the weight clear. ### 3. Anchor construction The revised finite-batch formulation explicitly specifies the indexed anchor collection $\mathcal A=(a_b)_{b=1}^{n}$, with $a_b=\operatorname{sg}(x_b)$. Anchors are refreshed at each training step and held fixed when differentiating the local surrogate. This distinguishes their role as neighborhood centers from the differentiable particle coordinates. The baseline affinity normalization and applicable self-pair masking are retained when defining the local weights. ### 4. Incomplete sentences We have completed the reference to the optional third-order extension in Section 4.3 and explicitly defined $\operatorname{sg}(\cdot)$ as the stop-gradient operator, which preserves its input value while blocking gradients through that input. ### 5. Parallel-only and Orthogonal-only We have added their definitions in the training-objective subsection and cross-referenced them in the experimental description. Let $H=\alpha_2G^{(2)}$, with $\alpha_2$ computed from the full correction. Using the batch Frobenius inner product, for $V\neq0$ we define $$H_{\parallel}=\frac{\langle V, H\rangle_F}{\|V\|_F^2}V, \qquad H_{\perp}=H-H_{\parallel}.$$ Parallel-only uses $V+H_{\parallel}$; Orthogonal-only uses $V+H_{\perp}$. The gate and scale are computed before projection and are not recomputed for the retained component. In particular, applying the cosine gate again to the orthogonal component would suppress it. We have also replaced the ambiguous cosine notation in Eq. (18) with an explicit normalized inner product.