- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper takes the variational flow matching (VFM) framework and changes the neural network backbone to a tree-based gradient boosting model. Using VFM allows a reparameterization of the denoising objective, so the endpoint, instead of the velocity, can be predicted. The authors argue that this is a more suitable objective for tree-based models.
Critical strengths
The authors include many benchmarks and 10 different datasets in their experiments. They include extensive additional experiments about hyperparameter sensitivity and scaling in the appendix. Some insight is provided into why the endpoint prediction objective is better for tree-based model via likelihood geometry argument.
Critical weaknesses
The contribution of the paper is really marginal. The authors only change the backbone used in EF-VFM [1], keeping the entire rest of the generative model unchanged. This shows in section 3.2, which is just a reframing of the content in [1]. The idea of using a gradient-boosting model as the backbone is not novel either, as many models, such as ForestDiffusion [2] exist. The proposed memorization metric is of little value, as DCR-based metrics suffice and capture the same information. A threshold-based DCR for different chosen threshold could as be computed. Since the only contribution is the switch to a tree-based model, I would expected a more thorough comparison to other tree-based methods, instead of only Forest-Flow. Lastly, the proposed model, similar to Forest-Flow requires substantial memory. Currently, this fact is hidden in the appendix. It also implies that presented results only hold for small to medium-sized datasets, as the lack of scalability would not allow the generation of higher-dimensional datasets.
[1] Exponential Family Variational Flow Matching for Tabular Data Generation. https://arxiv.org/abs/2506.05940
[2] Generating and Imputing Tabular Data via Diffusion and Flow-based Gradient-Boosted Trees. https://arxiv.org/abs/2309.09968
Questions for authors
- I do not understand the added value of the proposed memorization metric over the popular Distance-to-closest-record (DCR) or DCR Share, e.g., as explained in https://arxiv.org/pdf/2504.01908. Instead of L2 distance, you are just switching to L1 distance with a difficult-to-choose threshold parameter. You could also report DCR@1%, which would have the same meaning?
- The training times for TabDiff and Tabbyflow seems very excessive for some datasets. I do understand that CDTD is particularly efficient, but it is still a diffusion model. Why do the other models train for more than 100 minutes on the adult datasets whereas for CDTD 3 minutes are enough? Are you training CDTD for long enough to converge to a good state? The high DCR values seem like overfitting to me?
- Should the component ablation in Table 2 not start from EF-VFM (TabbyFlow) model? Since you keep the specification the same and switch out the backbone?
- Appendix L lacks a deeper comparison to ForestDiffusion: What explains the time differences between ForestFlow and TabTreeVFM?
- How does the sample quality scale with method complexity? Mentioning that setting K=10 improves training time and memory scaling is meaningless if sample quality drops significantly relative to the other methods.
- Why do you use different samplers across the diffusion baselines? Would it not be a fairer benchmark if you used the Euler sampler for all?
7 comments
Response to Reviewer yRXo (Part 2 / 5)
Response to W (continued):
Memory and scalability. Peak training memory grows linearly with the number of rows and is directly controlled by : across the ten datasets, it is 4.4–57.3 GiB at and 3.1–10.7 GiB at (Appendix L). With , TabTreeVFM already matches the leading generators, including on large datasets (see our response to Q5). On Diabetes (101,766 total rows, including 61,059 training rows; 9 numerical and 28 categorical columns with an average cardinality of approximately 79 in the training split, including three diagnosis-code columns with 670–716 categories), it ranks first on Shape (0.80) and WD (0.0016) and second on C2ST (66.97, after TabDiff).
Q1:
I do not understand the added value of the proposed memorization metric over the popular Distance-to-closest-record (DCR) or DCR Share, e.g., as explained in https://arxiv.org/pdf/2504.01908 (https://arxiv.org/pdf/2504.01908). Instead of L2 distance, you are just switching to L1 distance with a difficult-to-choose threshold parameter. You could also report DCR@1%, which would have the same meaning?
Response to Q1:
We appreciate the opportunity to clarify this point.
Mem is not an L2-to-L1 replacement. Our benchmark DCR already uses L1 distance (Appendix H.6), following TabSyn. Mem instead requires exact agreement on categorical attributes and on every numerical attribute , where is the attribute range. Its numerical criterion is therefore a range-normalized threshold.
DCR share and Mem answer different questions. DCR share measures whether synthetic records are closer to training or holdout records; Mem measures the positive excess of synthetic-to-training full-row matches over held-out-to-training matches. On Churn, TabSyn and TabTreeVFM have DCR shares of 53.29% and 51.26%, but Mem scores of 2.26 and 0.44 percentage points. On NMES, CoDi has a higher DCR share than CDTD (94.93% vs. 86.80%) but much lower Mem (1.37 vs. 18.75 percentage points). Thus, relative training-set proximity does not determine the amount of tolerance-bounded full-row reproduction.
Relationship to the cited framework. Sidorenko et al. (2025, Section 2.3) compute L2 distances in a language-model embedding space and, alongside DCR share, report identical-match statistics against a holdout set. We build on this holdout-based assessment principle, while specifying near-copy matching directly in the original mixed-type attributes and reporting the excess matching rate. This extension matters empirically: in our additional three-seed evaluation, CDTD's average excess exact-match rate is 0.85 percentage points, compared with 8.43 for Mem@1%; on Credit-g, TabbyFlow has an excess exact-match rate of 0.00, but Mem@1% of 20.20. Exact matching alone therefore misses substantial reproduction under the tolerance-based criterion.
Regarding DCR@1%. DCR@1% would not have the same meaning. A first-percentile DCR is a distance, not a reproduction rate. The share of synthetic records whose DCR falls below a threshold is a rate, but it differs from Mem in two ways. First, a threshold on the summed L1 distance becomes stricter per attribute as the number of columns grows, whereas Mem bounds the deviation on each attribute. Second, it has no held-out reference: at , real held-out records already match training records at rates of 3.52% on Default and 2.50% on Adult. Mem explicitly measures the excess above this observed reference.
Regarding threshold selection. The tolerance specifies the maximum numerical deviation as a fraction of each attribute's range, while categorical values must match exactly. Across the tolerances evaluated in Table 19 (0.1%–5%), CDTD and TabbyFlow have the highest dataset-averaged Mem, while TabTreeVFM remains at 0.14–0.28 percentage points, below TabDiff and TabSyn throughout.
Response to Reviewer yRXo (Part 3 / 5)
Q2:
The training times for TabDiff and Tabbyflow seems very excessive for some datasets. I do understand that CDTD is particularly efficient, but it is still a diffusion model. Why do the other models train for more than 100 minutes on the adult datasets whereas for CDTD 3 minutes are enough? Are you training CDTD for long enough to converge to a good state? The high DCR values seem like overfitting to me?
Many thanks for examining the training budgets so carefully. We strictly followed CDTD's official configuration and completed 30,000 optimization steps on all ten datasets. CDTD's original paper also reports that 30,000 steps are sufficient for convergence in its evaluated settings ( Appendix M of CDTD paper).
The runtime difference reflects different training workloads and implementations. On Adult, TabDiff and TabbyFlow perform approximately 64,000 optimization updates, and their official training loops additionally evaluate EMA loss over the full training set after every epoch. They also use multi-worker PyTorch data loading, whereas CDTD employs a lightweight tensor-based loader.
Q3:
Should the component ablation in Table 2 not start from EF-VFM (TabbyFlow) model? Since you keep the specification the same and switch out the backbone?
We are grateful for this good question. Following your suggestion, Table 2 in the revised manuscript now starts from TabbyFlow, and a new paragraph in Section 4.4 (Estimator comparison) discusses this comparison (both highlighted in blue). We add TabbyFlow (Aligned), which keeps the network and training recipe of TabbyFlow but follows the specification of TabTreeVFM, including a probability path with the same residual terminal noise in training and sampling, a unit-variance Gaussian objective with a fixed weight for numerical columns rather than a time-dependent variance with a weight annealed to zero, and the sampling grid. The remaining differences thus lie in the estimator: one network amortized over continuous time and trained with Adam, or independent boosted-tree heads at each grid time fit by second-order boosting, whose categorical leaf updates are diagonal-Fisher-scaled (Section 3.5):
| Model | Estimator | C2ST | Shape | Trend | MIA | Mem@1% | |
|---|---|---|---|---|---|---|---|
| TabbyFlow (Aligned) | Network | 78.10 | 1.79 | 3.01 | 21.02 | 0.9636 | 5.38 |
| Direct velocity regression | Trees | 78.62 | 2.51 | 4.89 | 1.47 | 0.9843 | 0.07 |
| TabTreeVFM | Trees | 85.26 | 1.31 | 2.94 | 2.48 | 0.9837 | 0.14 |
With the specification fixed, switching to boosted trees raises C2ST from 78.10 to 85.26 and lowers Shape error on every dataset. TabbyFlow (Aligned), in contrast, has the largest DCR deviation and Mem@1% in Table 2, 21.02 and 5.38 percentage points against 2.48 and 0.14 for TabTreeVFM, and its DCR share is farther from 50% on every dataset. Lower reproduction thus accompanies higher fidelity rather than coming at its expense. Trees alone are not sufficient either: with direct velocity regression, C2ST stays at 78.62, and it reaches 85.26 only with the boosted-tree multinomial heads of TabTreeVFM (Table 2).
Q4:
Appendix L lacks a deeper comparison to ForestDiffusion: What explains the time differences between ForestFlow and TabTreeVFM?
We thank the reviewer for this question. The runtime differences in Table 21 largely reflect differences in training implementations. We use the scalable Forest-Flow implementation released by Cresswell and Kim (2024). The two implementations differ in their data handling and model organization. For example, Forest-Flow fits separate XGBRegressor models across timesteps, classes, and output columns, rebuilding the data matrix for each fit and repeatedly incurring preprocessing overhead, including quantile sketch construction and GPU data transfers. In contrast, TabTreeVFM constructs a QuantileDMatrix once per timestep and reuses it across prediction heads, with GPU data prepared directly in device memory. Moreover, TabTreeVFM uses a single multinomial classifier per categorical column rather than separate regressors for individual one-hot dimensions, reducing the number of independent model fits. These implementation and modeling differences help explain the observed runtime difference.
Response to Reviewer yRXo (Part 1 / 5)
W:
Response to W:
Thank you for the detailed comments. We address each point below.
Contribution beyond EF-VFM, ForestDiffusion, and Forest-Flow. Section 3.2 presents the endpoint formulation of EF-VFM, on which TabTreeVFM builds; our contributions are in Sections 3.3–3.6. Moving this formulation to boosted trees changes how the objective is optimized, not only the function class. EF-VFM trains a single time-conditioned network on all timesteps and columns jointly with Adam, a first-order method that rescales parameter gradients by running moment estimates; it notes that the Fisher information of its endpoint likelihoods is the Hessian of the log-partition function, but leaves natural-gradient optimization to future work. TabTreeVFM instead decomposes the objective into independent XGBoost regression and multiclass heads at each timestep, whose posterior means define the velocity analytically, and fits each head by second-order boosting, whose leaf updates use the per-sample curvature of the endpoint likelihood. This design also differs from ForestDiffusion and Forest-Flow, which regress scores or velocities coordinate-wise on one-hot encodings. Under the same probability path, solver, and preprocessing, the EF-VFM network reaches a C2ST of 78.10 and tree-based velocity regression 78.62, whereas TabTreeVFM reaches 85.26 (Table 2 and Section 4.4; see Q3).
For the categorical heads, this curvature is proportional to the diagonal Fisher information of the endpoint posterior, and Proposition 3.1 shows that the leaf update equals the diagonal-Fisher step at the leaf-mean prediction rescaled by 1/(1−ρjk), where ρjk is the within-leaf dispersion ratio, so the two coincide when the predictions within a leaf agree. This curvature matters in practice (Appendix D): across the 46 non-binary categorical heads of nine datasets, a constant-curvature update raises held-out endpoint cross-entropy roughly a hundredfold for t≥0.90 and worsens the categorical fidelity of generated tables, whereas the full Fisher matrix of natural-gradient boosting brings no consistent gain. The Fisher scaling thus comes from standard second-order boosting at no extra cost, without NGBoost's explicit Fisher inversion.
Comparison with other tree-based generators. Beyond Forest-Flow, we add ForestDiffusion and Adversarial Random Forests (Watson et al., 2023). Both are evaluated with the full Table 1 protocol on all ten datasets:
Avg. rank is computed over all 12 metrics of Table 1 among 14 generators (Table 1 plus the two new baselines). TabTreeVFM remains first overall and has by far the highest fidelity among tree-based generators. The other tree-based generators reproduce fewer training rows, but at a much lower fidelity (C2ST at most 27.73).
Mem versus DCR. Please see our response to Q1.