- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Mixture-of-Depth Rectified Flow (MoD-RF), an adaptive extension of hierarchical rectified flow. The paper makes three main contributions: 1) a Gaussian-mixture analysis of hierarchical target directions; 2. a learned sequential depth selector which predicts stopping probabilities over hierarchy levels, and is trained using weights derived from a coarse reconstruction of the original velocity; 3. adaptive-depth inference where at each outer integration step, the method selects a hierarchy depth, fixes that depth and the sampled deeper source states, solves the selected hierarchy, and updates the outer state. Experiments cover two one-dimensional synthetic configurations, a two-dimensional two-moons task, MNIST, CIFAR-10, and ImageNet-32, and the paper reports better Wasserstein distances and FID scores than standard rectified flow and fixed-depth hierarchical baselines.
Critical strengths
- This paper's adaptive-computation strategy for hierarchical rectified flow is reasonable and useful
- The paper provides both solid theoretical analysis (both in Corollary 1 and Corollary 2), and also practical implementations for training (supervise the depth selector using the prediction quality achieved at each depth, as in Eq. (15))
Critical weaknesses
- The paper only reported total NFE budgets, which may accounts depth-specific predictor evaluations, it would be good to also report overall computational cost budget.
- The experiment in the paper mainly uses figures, it would be good to also provide numerical tables and uncertainty estimates for quantitative comparison.
Questions for authors
- In sec.3.3.1, the reconstruction error normalization factor (\sigma_d) is set as 0.5, how sensitive are generation quality to this parameter?
- The Image dataset for experiment is relatively small, how about the algorithms' performance on other dataset such as IMAGENET 256*256.
4 comments
Response to Reviewer rm4u (Part 2/2)
Q3. In sec.3.3.1, the reconstruction error normalization factor (\sigma_d) is set as 0.5, how sensitive are generation quality to this parameter?
To investigate the impact of the reconstruction error normalization factor , we conduct additional ablation studies with $\sigma_d \in $ {} on the 1to5 and 2to2 datasets, keeping the stopping threshold fixed at . The results are presented in Appendix D.4.2 (Figure 14 and Table 7) of the revised manuscript.
Generation quality and computational efficiency. As shown in Figure 14, consistently achieves the best quality-computation trade-off at moderate and high NFE budgets across both datasets. For example, at 800 NFEs, MoD-RF with achieves Wasserstein distances of 0.0103 and 0.0125 on 1to5 and 2to2, respectively, substantially outperforming both smaller () and larger () settings. Although larger values can be competitive at very low NFE budgets on 2to2, provides the strongest overall performance as inference computation increases.
Effectiveness of adaptive depth allocation. Importantly, Table 7 reveals that these performance differences are closely related to how computation is allocated across hierarchy depths. Smaller values produce sharper supervision signals and substantially increase depth-3 usage without corresponding improvements in generation quality. Conversely, larger values () lead to exclusively selecting depth 2, effectively removing adaptive depth variation. In contrast, achieves a more effective allocation by primarily using depth 2 while selectively invoking depth 3, demonstrating the benefit of allocating additional hierarchical computation only to a subset of states.
Overall, these results support our central motivation: effective generation does not require uniformly deeper computation, but rather an appropriate allocation of hierarchy depth. The consistent advantage of across both datasets motivates its use as the default setting in our experiments.
Q4. The Image dataset for experiment is relatively small, how about the algorithms' performance on other dataset such as IMAGENET 256*256
We thank the reviewer for highlighting the importance of evaluating MoD-RF on larger-scale image generation tasks. To answer, we have extended our experiments to ImageNet-256 ( resolution) in the revised manuscript and observed similarly outstanding performances across NFE ranges (Section 4.3, Figure 7, lines 421-425).
Experimental setup. Unlike our experiments conducted in pixel space, ImageNet-256 generation is performed in the latent space of the SDXL autoencoder, representing images as latents. We independently train HRF-1 and HRF-2 experts for 150K steps, freeze their EMA weights, and train a lightweight Transformer depth selector for 20K steps using the same one-step velocity-reconstruction objective. During unconditional generation, the selector adaptively chooses between depths 1 and 2 at each outer Euler step, with five inner steps for depth 2. Generated latents are decoded into images and evaluated using FID-50k, with RF and HRF-2 compared at matched NFE budgets.
Results on ImageNet-256. As shown in Section 4.3 (Figure 7), MoD-RF consistently achieves lower FID than standard RF and fixed-depth HRF-2 across all evaluated NFE budgets. While RF exhibits nearly constant FID as the inference budget increases, MoD-RF continues to improve and maintains the most favorable FID-NFE trade-off throughout the evaluated range. This demonstrates that adaptive depth allocation remains effective for substantially higher-resolution image generation.
Furthermore, we investigate whether the improvements persist with stronger underlying experts by comparing models trained for 100K and 150K steps (Section 4.4, lines 483-489; Figure 7). Although additional training improves all methods, MoD-RF consistently maintains the lowest FID at every evaluated NFE under both training budgets, indicating that adaptive hierarchy depth provides complementary benefits as the underlying experts become better optimized.
Together, these additional experiments provide evidence that MoD-RF's quality–computation advantages extend beyond low-resolution benchmarks to higher-resolution ImageNet-256 generation, supporting the scalability and practical relevance of adaptive hierarchical modeling.
Follow-up on our Responses
We thank the reviewer again for their constructive feedback and positive assessment. In our rebuttal and revision, we have provided wall-clock comparisons to assess overall computational costs, added numerical results with uncertainty estimates, clarified the sensitivity to the normalization parameter , and extended our experiments to ImageNet-256 using latent flow matching to demonstrate practical relevance.
We would greatly appreciate it if the reviewer could review these additional results and clarifications and let us know whether they address the main concerns. We would be happy to address any remaining questions.
Thanks the authors for detailed experiment results on component-wise computation analysis, numerical results with uncertainty estimates, and sensitivity analysis. My concerns have been addressed and I'll keep my positive rating
Response to Reviewer rm4u (Part 1/2)
We thank the reviewer for their time and feedback. We also thank the reviewer for highlighting our the applications of adaptive computation strategy, our solid theoretical analysis and proposed implementations. We answer questions next.
Great suggestion. We adopt the number of function evaluations (NFEs) as our primary computational cost metric, as NFEs constitute a major computational expense in flow-matching-based generative models. To provide a more comprehensive assessment of computational efficiency and answer the reviewer’s question, we have additionally included end-to-end wall-clock measurements and component-wise computation analysis in Appendix E of the revised manuscript.
1. Overall computational cost. In Appendix E.1 (Table 8, lines 1512-1520), we report the total batch inference time, the wall-clock shares of the selector and HRF experts, and their average execution time per call. These measurements include all forward passes and solver operations on MNIST, CIFAR-10, and ImageNet-32, using a batch size of 1,000. The numerical results are summarized below.
As shown, the selector accounts for only 0.37%–0.72% of total inference time, while the HRF experts account for 97.5%–98.3%. Moreover, each selector call is approximately 32-48 times faster than an expert call, confirming that adaptive depth selection introduces negligible additional overhead relative to nested integration.
2. Performance versus wall-clock time. Beyond the component-wise analysis, Appendix E.2 (Figure 15, lines 1503–1508) presents end-to-end FID versus wall-clock-time trade-offs on MNIST, CIFAR-10, and ImageNet-32. These comparisons account for all selector evaluations, expert computations, nested integration, and solver overhead. The results show that MoD-RF maintains a favorable quality-computation trade-off over standard RF and fixed-depth HRF at comparable inference times, consistent with the NFE-based comparisons.
Together, these additional results demonstrate that MoD-RF's performance improvements persist when evaluated using actual inference time, rather than relying solely on NFE as a computational proxy.
We thank the reviewer for this constructive suggestion. To facilitate quantitative comparisons and better assess the robustness of our results, we have added numerical FID scores with mean and standard deviation in Appendix D.1 (Table 4, lines 1080-1093) of the revised manuscript. These results complement the FID-NFE curves in Figure 6 and provide detailed comparisons between MoD-RF, RF, and HRF-2 across multiple inference budgets on MNIST, CIFAR-10, and ImageNet-32.
Numerical results. We reproduce the results from Table 4 below. All values are reported as mean FID ± standard deviation, where lower is better.
As shown, MoD-RF achieves the lowest mean FID at every reported NFE budget across all three datasets, with standard deviations comparable to those of the baselines. Furthermore, MoD-RF consistently improves as the inference budget increases, whereas RF remains nearly unchanged. These numerical results reinforce the findings in Section 4.3 (Figure 6) and provide additional evidence that the quality-computation advantages of adaptive depth allocation are consistent across the evaluated settings.
(Continue from the next comment)