- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that, when training flow map models (FMMs), the coefficient that balances stochastic conditional supervision and pretrained guidance should not be set globally, but should instead adapt to the transport state. To this end, the authors propose Risk-Adaptive Flow Map Learning (RAFL), which approximates the closed-form optimal balance coefficient through kNN-based risk estimation combined with an amortized predictor. Experiments on CIFAR-10 and class-conditional ImageNet-256 show that RAFL outperforms the baselines and support the effectiveness of the proposed risk-based adaptation.
Critical strengths
The motivation is clear and insightful, and it is partially supported by empirical observations. The method appears to have the potential for broad applicability, and its benefits are well demonstrated in the experiments. Overall, these points lead me to lean toward acceptance.
Critical weaknesses
I am not an expert on flow-map models, but the claimed derivation of "the optimal balance between stochastic conditional supervision and potentially biased pretrained guidance" (i.e., Prop. 3.1) appears to be a standard bias–variance trade-off result. This leaves me uncertain about how much of the theoretical analysis goes beyond results that are already classical in this field or in statistics. It may be more appropriate to frame the first contribution around demonstrating why λ needs to be modeled as a risk-adaptive coefficient.
In addition, several details of the experimental settings and results are not sufficiently explained (see Questions).
Questions for authors
- Which teacher model provides the v_φ signal during training?
- Since the estimation of V and B is one of the core contributions, is there any experiment that directly evaluates the accuracy of these estimates? The current experiments use a kNN-based metric to show that the estimates are self-consistent, but do not characterize their bias or variance.
- In Fig. 2(a), all curves appear to start at an FID of approximately 3.7 at iteration 0, whereas the shared pretrained checkpoint has an FID of 3.47 (Tables 1 and 10), and Fig. 3(b) starts at approximately 3.47 for the same model. Could the authors clarify this discrepancy?
- Could the authors report the additional training cost that RAFL introduces relative to standard training?
4 comments
Q2. Accuracy of the estimated risk components.
To directly assess the accuracy, bias, and variability of the estimated and , we added an independent oracle-based evaluation in Appendix A.4 (Table 7).
On ImageNet, the teacher bias cannot be directly evaluated because the true marginal velocity is unavailable. Squared target-space disagreement between teacher and conditional supervision has conditional expectation , so it does not directly measure . We therefore use an eight-component 2D Gaussian mixture whose analytic posterior allows direct computation of ground-truth and for a frozen MF snapshot.
An invertible representation gives population-level , isolating estimation and amortization errors. We evaluate four transport conditions using 20 independent evaluation banks per condition (16,384 samples each) and three predictor-training runs. Training data and calibration banks are separate from test queries and evaluation banks; oracle targets are used only for evaluation.
At :
| Method | bias | SD | RMSE ↓ | bias | SD | RMSE ↓ |
|---|---|---|---|---|---|---|
| Uncorrected kNN | 0.4139 | 1.6630 | 2.7450 | 0.3787 | 0.5166 | 1.1111 |
| Corrected kNN | 0.0330 | 1.7166 | 2.6092 | 0.1442 | 0.4340 | 0.7774 |
| Amortized | 0.0140 | 0.4670 | 1.1563 | 0.0881 | 0.0064 | 0.1575 |
Bias and SD are computed per query across independent banks for kNN or training runs for , then averaged over queries. All entries are arithmetic means of the four condition-level statistics; RMSEs are not pooled across conditions.
The finite-sample correction reduces RMSE by 30.0%, while the amortized predictor further reduces RMSE by 55.7% relative to corrected kNN. The predictor also achieves coefficient MAE of 0.02088 and normalized excess risk of 0.00326. These oracle-based measurements directly characterize the bias, variability, and accuracy of the estimated risk components. Ground-truth accuracy claims are restricted to this controlled setting; the ImageNet diagnostic evaluates policy alignment.
Q3. Consistency of the initial FID across figures.
We thank the reviewer for identifying this inconsistency. All strategies start from the same 300k-step MF-XL/2 checkpoint with 1-NFE FID 3.47, and iteration 0 denotes this checkpoint before any post-training updates. The approximately 3.7 starting value in the original Fig. 2(a) was incorrect. We have corrected the panel using the corresponding evaluation records; the revised Fig. 3(a) now shows the common initial FID of 3.47, consistent with Table 1.
Q4. Additional training cost.
For iMF-XL/2 on ImageNet-256, RAFL increases FLOPs per training step by approximately 6.64% relative to standard continued training. Both methods start from the same 1M-step checkpoint and use batch size 1024 for 40k additional steps.
| Method | Batch size | Additional steps | Compute (PFLOPs/step) | Total post-training compute (EFLOPs) |
|---|---|---|---|---|
| iMF-XL/2 + continued training | 1024 | 40k | 2.0201 | 80.804 |
| iMF-XL/2 + RAFL | 1024 | 40k | 2.1543 | 86.172 |
RAFL adds 0.1342 PFLOPs per step, totaling 5.368 additional EFLOPs over 40k steps. Under the same per-step accounting, the entire RAFL post-training stage requires approximately 4.27% of the original pretraining compute.
Thank you for the detailed response. Though I'm not an expert in this field, I learned something from this paper. I will maintain the positive rating.
It has been our honor to have you carefully review our work, and we sincerely appreciate the time and effort you devoted to evaluating our manuscript. We are truly grateful for your many valuable comments and constructive suggestions, which have helped us improve our work substantially.
We thank the reviewer for the constructive feedback and the suggestion to emphasize why and when supervision should be risk-adaptive. We address each point below.
W. Theoretical contribution and the motivation for risk-adaptive weighting.
We agree that the scalar bias-variance minimization in Proposition 3.1 is standard. We have revised the contribution statement to emphasize why and when supervision should be risk-adaptive, using this proposition as a building block.
First, we formulate reliability in the finite-time target geometry, Gξ=Aξ⊤Aξ. This accounts for how instantaneous uncertainty and teacher bias propagate into the supervision target actually optimized by the model.
Second, Corollary 3.3 quantifies the benefit available to fully state-dependent mixing over the best transport-condition-only coefficient:
\mathbb E\left[W_\xi(\lambda_\xi^\star-\lambda_\kappa^\star)^2\mid\kappa\right],$$ where $\kappa=(r,t)$ and $W_\xi=V_\xi+B_\xi$. Positive $H_{\mathrm{sup}}$ identifies when state adaptation can strictly reduce supervision-target risk. Third, Theorem 3.2 and Corollary 3.3 characterize the optimal coefficient under restricted information and decompose this opportunity into representation-resolved and unresolved components. Section 4 further quantifies the policy discrepancy introduced by unresolved teacher-bias variation in disagreement calibration. These results motivate state-conditioned risk prediction and calibration. The optimality statements concern supervision-target risk within the specified mixture family at a fixed target-network snapshot. ### Q1. Which teacher model provides the $v_φ$ signal during training? For each experiment, we use a frozen copy of the corresponding pretrained baseline as the teacher, and initialize the student from the same checkpoint. Thus, the teacher matches the backbone being refined: MF uses its pretrained MF baseline, iMF uses its pretrained iMF baseline, and similarly for the other evaluated FMMs. For example, the controlled MF-XL/2 experiments use the 300k-step MF-XL/2 checkpoint with one-step FID 3.47. For the MF formulation, instantaneous guidance is obtained through diagonal evaluation: $$v_\phi(z_t,t):=u_\phi(z_t,t,t).$$ The teacher parameters $\phi$ remain fixed throughout post-training, while the student parameters $\theta$ are updated by RAFL.