- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies soft-label bias in long-tailed dataset distillation. The authors argue that existing calibration methods, which shift logits by a class-dependent constant, are too rigid to undo the kind of distortion that class imbalance introduces into soft labels. Instead, they propose FMLT, which treats debiasing as a transport problem: a small MLP, conditioned on the synthetic image's features, learns a velocity field in logit space via conditional flow matching, and the biased logits are pushed toward a target distribution by integrating this field with an Euler solver. The method is a post-hoc module that plugs into existing distillation pipelines such as SRe2L, G-VBSM and EDC, and is evaluated on CIFAR-10/100-LT and ImageNet-1k-LT against the ADSA calibration baseline.
Critical strengths
Soft-label bias in long-tailed dataset distillation is a real and fairly under-explored problem, and I appreciate that the method is designed as a lightweight post-hoc step that does not require touching the distillation pipeline itself. It is cheap to run and the authors do try it on top of several different distillation methods, including at ImageNet scale. The general intuition that a per-sample, feature-dependent correction might do better than a single global shift is reasonable and could be worth exploring properly.
Critical weaknesses
The method does not actually do what the paper claims. The whole paper is built around the argument that a linear, additive logit shift is not enough and that a nonlinear, feature-conditioned transport is needed. But the target distribution in Eq. 2 is just the original prediction reweighted by the inverse of each class's mean confidence. In logit space, this is exactly a class-wise constant shift, i.e., the same kind of linear correction the paper criticizes, with the class prior replaced by the mean confidence. Since the reference path is a straight line between the original and target logits, the velocity the network is trained to regress is essentially the same constant vector for every sample, regardless of the image features or the time step. So there is no meaningful nonlinear trajectory for the MLP to learn, and the feature conditioning has nothing to adapt to. Following this through, the step-count ablation does not make sense either: for a constant velocity field, a single Euler step is already exact, so accuracy should not keep improving with more steps. The text also disagrees with itself about where the performance plateaus, and the default number of steps is not the best one in the table.
Key baselines and ablations are missing. The target in Eq. 2 can be computed in closed form for every synthetic image, and the flow is trained and applied on the same set of images. The obvious question is what happens if you simply use Eq. 2 as the soft label directly, without any flow. I suspect this would perform the same, which would mean that any gain over ADSA comes from using mean confidence instead of the class prior, not from flow matching. There is also no ablation removing the image-feature conditioning, even though this is the central selling point, and the repeated claims about "restoring inter-class semantic topology" are not backed by any measurement at all.
The experimental evidence is thin and has consistency issues. No standard deviations or numbers of runs are reported, and many of the improvements over ADSA are around one point or less, which is hard to interpret without variance estimates. The gains are also strikingly uniform across very different settings, especially in the ImageNet-LT table. In that table, some rows cannot be reconciled with each other: for SRe2L at one IPC setting, the head accuracy drops sharply after calibration while the overall accuracy goes up, and I could not find any split of head/mid/tail classes under which both the baseline and the calibrated rows would be consistent. In addition, the text makes claims that the tables do not support, e.g., that EDC with FMLT outperforms the other baselines in all settings. There is only a single ablation in the whole paper, and the main text does not even fill the page limit, so there was clearly room for the experiments that are missing.
The theory adds length but not insight, and contains errors. Most of the appendix restates textbook results (Picard–Lindelöf, Euler truncation error, universal approximation, Rademacher bounds) that say nothing specific about this method. Some of it is also wrong or inconsistent. Lemma 5 claims the target yields a uniform class marginal, directly contradicting the main text, which says it does not. The Fisher–Rao distance is misidentified as the square root of the JS divergence. Theorem 1 claims cosine similarity is invariant under a shift, and its proof then admits it is not. Theorem 3 assumes the MLP output is bounded, which does not hold for a ReLU/GELU network taking unbounded logits as input. Theorem 4 bounds a Euclidean kinetic energy using a Fisher–Rao argument without connecting the two.
Presentation is poor. Figure 1 is a conceptual illustration with no real data behind it, and Figure 2 is a low-resolution raster plot where the axis labels and legends are basically unreadable; its legend also says "Ours" instead of FMLT, and it uses fractions while the tables use percentages. The reference list contains several duplicated entries for the same papers, and citations are formatted inconsistently throughout (e.g., "Jiang et al. Jiang et al. (2025)"). Overall, the paper reads as an unfinished draft rather than a submission ready for ICLR.
Questions for authors
-
Since the target in Eq. 2 amounts to a class-wise constant shift in logit space, the regression target for the vector field should be (up to a softmax-invariant offset) the same constant for every sample. Can you show that the learned field is actually non-constant and genuinely depends on the image features? For example, how much does the predicted velocity vary across samples and time steps?
-
What happens if you use the closed-form target from Eq. 2 directly as the soft label, without training or integrating any flow? Please report this baseline in the main tables. If it performs about the same as FMLT, what does the flow-matching component contribute?
-
Please add an ablation that removes the image-feature conditioning h(x) from the vector field network. This is the central claim of the paper, but it is never tested.
-
With a straight-line reference path and a constant target velocity, one Euler step should already be exact. Why does accuracy keep improving as the number of steps grows in Table 3? Also, the main text and Appendix D.4 disagree on where performance plateaus, and the default setting is not the best one in the table. Which is correct?
-
In Table 2, the overall accuracy for SRe2L and SRe2L + ADSA at IPC=50 does not seem consistent with the reported head/mid/tail accuracies under any fixed class split. How exactly are the head, mid and tail groups defined, and how is overall accuracy computed? Could you double-check these numbers?
-
How many runs are the reported results averaged over, and what are the standard deviations? Many of the gains over ADSA are around one point or less, so variance estimates are needed to judge whether they are meaningful.
-
Were the ADSA results produced by your own runs with the tuning protocol described in Appendix D.3, or taken from the original paper? If the latter, were the experimental settings identical?
-
Lemma 5 states that the target produces a uniform class marginal, while Section 3.2 says it does not in general. Which statement is correct, and how should the proof be read?
-
The paper repeatedly claims that FMLT "restores inter-class semantic topology." Can you provide a quantitative measure that supports this, for example some comparison of inter-class similarity structure before and after calibration?