- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a random subspace descent framework for probability flows to reduce the computational cost of high-dimensional distribution optimization. By projecting descent directions onto randomly sampled subspaces while preserving the original objective and geometry, the method provides theoretical guarantees and demonstrates promising empirical results across several probability flows.
Critical strengths
The proposed method is well-motivated and technically sound. The overall framework is natural and clearly presented, with a reasonably comprehensive theoretical analysis supporting its effectiveness. Although the underlying idea is relatively straightforward, the formulation is mathematically principled, and the reported results demonstrate its potential benefits.
Critical weaknesses
- Random subspace methods have been extensively studied in the optimization literature. Therefore, the core technical idea of this paper is not particularly novel. Nevertheless, extending subspace-based optimization to probability flows is a natural and reasonable direction, and the authors provide a relatively comprehensive theoretical analysis. Despite some concerns regarding novelty, I still consider this a technically solid and meaningful contribution.
- The manuscript appears somewhat unfinished in several places. For example, the generative modeling experiments lack qualitative visualizations of the generated images, which would help readers better understand the practical improvements. There are also noticeable editorial issues, such as missing appendix references in Section 2.2. These issues give the impression that the manuscript would benefit from another careful round of revision and proofreading.
- Although the experimental results are generally positive, the evaluation of the method on practical AI applications remains somewhat limited. Additional experiments on representative real-world AI tasks would help demonstrate the broader applicability and practical value of the proposed framework.
Questions for authors
- What are the key technical distinctions between the proposed framework and existing subspace-based optimization methods, particularly those involving variational projections or functional gradient approximations?
- The theoretical analysis suggests that convergence is influenced by the dissipation captured by the random subspace and the associated projection loss. Can this dissipation loss be reasonably estimated or characterized in the practical problems considered in the experiments?
- The paper aims to reduce computational costs, but the experiments do not seem to provide sufficient empirical evidence of computational efficiency. Could the authors provide runtime comparisons or other computational measurements to validate the claimed efficiency gains?
- The current baselines mainly focus on comparing projected and sliced variants of MMD, Sobolev descent, and LAWGD. Could the authors include at least one existing computationally efficient approximation method as an additional baseline to better demonstrate the competitiveness of the proposed approach?
Overall, I consider this a reasonably solid paper with a natural methodological design and meaningful theoretical analysis. My main concerns are the relatively limited novelty and the insufficient experimental validation, along with several presentation issues that suggest the manuscript would benefit from further refinement. Nevertheless, I appreciate the technical foundation of this work and believe these concerns are potentially addressable. I would be willing to increase my score if the authors can provide convincing clarifications and strengthen the experimental evaluation during the rebuttal process.
We thank the reviewer for these comments. We address Critical Weaknesses (CW) and Questions (Q) below.
CW1 — Novelty relative to random subspace optimization. We agree the Euclidean idea is classical and the appendix now says so directly, citing that literature. What changes is the setting: the "subspace" is a space of potentials under a non-Euclidean geometry, so the Gram is the flow's own metric rather than a restricted Hessian UᵀHU; the sampling law preconditions an infinite-dimensional space, with no finite-dimensional analogue; and the captured/lost split is exact in that geometry. We also claim the exact captured/lost decomposition with its trace bound, and three instantiations one of which needs no score, while conceding that the Gram solve and the projection identity themselves are standard. (Sec. 1; App. B, C.)
CW2 — Manuscript appears unfinished. Thanks reviewer for the careful reading. Both specifics fixed. The missing appendix pointers in Section 2.2 are restored; the unresolved references elsewhere pointed into a commented-out appendix, and the manuscript now has zero undefined references. We have also proofread the revised sections. Qualitative sample grids for all four superclasses are added, with real images alongside the sliced and projected arms. (Sec. 2.2; App. I.3, Fig. 3.)
CW3 — Limited evaluation on practical AI applications. We acknowledge this as a scope limitation and prefer to state it. The CIFAR-100 latent coupling task remains our only real-data application. This revision adds depth rather than breadth: seed variability, two coupling baselines, a mechanism ablation isolating which part of the method works, end-to-end timings, sensitivity studies on the regularizer and probe budget, and qualitative samples.
Q1 — Distinctions from existing subspace / variational-projection methods. See CW1.
Q2 — Can the dissipation loss be estimated in the practical problems? Yes, and we now report it rather than leaving it theoretical. Along the reported CIFAR-100 trajectories we assemble the Gram system both for the P sampled potentials and for a rich reference dictionary of 8192 potentials drawn from the same law, and report the captured fraction:
A 512-d trial space captures roughly a tenth of what an 8192-d one does — the regime the trace bound predicts — which sharpens the claim: the method helps not because the subspace is nearly complete but because the fraction it captures is spent well. Two caveats are stated in the text: the reference is itself a finite-dictionary estimate, hence a lower bound, and each column follows its own trajectory. We additionally report the companion quantity, the fraction of the available one-step decrease captured by each solve rule at a fixed dictionary: 0.14–0.40 for uniform slicing at its best global scale against 1 for the Gram solve. (App. I.2.)
Q3 — Runtime comparisons. Now reported, and split into the claim we can support and the one we cannot. We withdraw the general efficiency claim on the image task. CIFAR-100 coupling takes 3.6–5.8 s for uniform slicing against 22.9–39.5 s for the projection, downstream training identical at 16.2 s; per projected iteration the cubic solve is only 14–24 %, so cost is dominated by estimating the objective gradient, not by the projection. The cost discussion claims savings only where the accounting supports them: dense Sobolev descent, and LAWGD vs a full Poisson solve. We now also measure that saving. Timing full against projected Sobolev descent to a common objective (16-d target, m = 16, five seeds), the per-step saving grows with the feature count d_ψ as predicted — 1.4× at d_ψ = 768 to 7.8× at 3072 — but projection needs ~3× more steps, so the net speedup is 1.5–2.5× and appears only above d_ψ ≈ 10³; below that the projected arm is slower end to end. This also explains the CIFAR-100 timings: that study runs far below the crossover. (Sec. 3.3; App. D.)
Q4 — An existing efficient approximation as a baseline. Added: minibatch-OT coupling at three batch sizes, the standard coupling baseline for flow matching, alongside an independent-pairing control. GlobFID on the four groups is 21.9/8.1/16.9/15.3 for independent pairing and 16.4/5.8/15.4/14.3 for minibatch OT at batch 512, against 8.9/3.6/11.5/7.3 for the best flow. Minibatch OT improves monotonically with batch size but remains behind every flow coupling on every group, by 1.3–1.8× (worst flow per group). Uniform slicing, our existing comparison, is itself the standard cheap approximation in this family. (Sec. 5.4; App. I.)