acceptodds
Under review as a conference paper at ICLR 2027

CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data

10 views

Abstract

Generative models for tabular data are typically trained separately for each dataset, limiting knowledge transfer and requiring the storage of many specialized models. In this paper, we introduce CDMD, a tabular diffusion model trained jointly across heterogeneous datasets with different schemas and variable numbers of numerical and categorical features. Unlike existing cross-dataset tabular diffusion models that operate in continuous representation spaces, CDMD defines diffusion directly over the mixed-type feature space and is trained end-to-end. To accommodate heterogeneous categorical domains, we introduce a schema-restricted reverse-process parameterization for masked diffusion models, in which the output space dynamically adapts to each feature’s vocabulary. We then compose numerical and categorical feature-level diffusion processes into a schema-dependent row-level process. A shared schema-aware Transformer denoiser captures dependencies between features and parameterizes the reverse process across varying schemas. On seven real-world datasets, a single jointly trained CDMD achieves the highest average generation quality among strong single-dataset and cross-dataset baselines, while using substantially fewer total parameters than the collection of separately trained models. Furthermore, pre-training on a corpus of 337 datasets improves generation on previously unseen datasets under both limited target data and limited adaptation epochs. These results demonstrate the potential of direct mixed-type diffusion for shared and transferable tabular data generation.

Primary area:
generative models
Keywords:
tabular data generation, mixed-type diffusion, masked discrete diffusion, cross-dataset generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4r45OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CDMD combines numerical Gaussian diffusion and categorical masked diffusion through a shared schema-aware Transformer to generate mixed-type data across heterogeneous table schemas; joint training on seven datasets demonstrates parameter savings and competitive generation quality, while pretraining on 337 datasets improves adaptation to unseen tables under limited target data and adaptation epochs.

Critical strengths
  1. A single shared model with approximately 10.7M parameters matches or exceeds the average quality of separately trained generators, providing convincing empirical support for cross-table parameter sharing.
  2. Pretraining improves generation on unseen tables under limited data and adaptation epochs, demonstrating useful transfer when target data are scarce.
Critical weaknesses
  1. The generalized masked diffusion largely retains SUBS’s reverse transitions and training objective, with schema-specific vocabulary restriction as the main addition, making this methodological contribution relatively incremental.
  2. The paper insufficiently specifies CTSyn’s two-stage training configuration and checkpoint selection, making the substantial performance gap against its closest cross-table diffusion baseline difficult to independently verify.
Questions for authors
  1. In the few-shot experiments, are categorical vocabularies constructed from the selected N rows or supplied by the complete schema, and do all methods receive the same categorical-domain information?
  2. Could the authors specify CTSyn’s training and adaptation budgets for both stages and its checkpoint-selection rules to clarify whether its substantial performance gap is affected by training configuration?
  3. On the same hardware, how much time do pretraining, target-table adaptation, and sampling require, and how many target tables are needed to amortize pretraining costs?
2 comments

We thank the reviewer for their valuable feedback. In the following, we address the reviewer's questions and concerns.

In the few-shot experiments, are categorical vocabularies constructed from the selected N rows or supplied by the complete schema, and do all methods receive the same categorical-domain information?

For a fair comparison between baselines, we construct the categorical vocabularies from the selected N rows in the few-shot experiments. Therefore, all methods received the same categorical domain information.

Could the authors specify CTSyn’s training and adaptation budgets for both stages and its checkpoint-selection rules to clarify whether its substantial performance gap is affected by training configuration?

For CTSyn, we used the implementation provided in their OpenReview submission [1]. While the implementation matches the description in their paper, we found discrepancies between the code and the paper in the hyperparameter settings. Therefore, we tried both settings and picked the best-performing checkpoint. For a fair comparison with our model, we trained CTSyn on the same datasets as our model in both our settings (the cross-dataset experiment and the pre-training experiment).

On the same hardware, how much time do pretraining, target-table adaptation, and sampling require, and how many target tables are needed to amortize pretraining costs?

On the same hardware, pre-training on the 337 datasets takes 2 days, while target-table adaptation takes between 15 minutes and one hour, depending on the number of samples. Sampling a batch of 10000 rows takes less than 1 minute on average. In our experiments, training the model from scratch on the target dataset required at least 2000 epochs to converge (consistent with TabDiff), whereas a pre-trained model converged after 50 epochs.

We hope that our response clarifies the questions raised by the reviewer and are happy to clarify any further concerns!

References

[1] https://openreview.net/forum?id=Sh4FOyZRpv

Reviewer_4r45OpenReview ↗

Response to the authors

Thank you for the helpful response, which has clarified the questions raised in my review. I maintain my recommendation of weak acceptance.

Reviewer_Z6GGOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This papers introduces a diffusion-based generative model in which the denoiser is conditioned on the schema of the data table. This allows a single model to be trained on multiple datasets, instead of a single one but requires meaningful labeling of features. The main contribution lies in changing the backbone to a transformer-based model and adding schema conditioning. Results show some marginal improvements over existing models, which are only trained on a single table.

Critical strengths
  • The problem of generating previously unseen datasets with only the schema given.
  • Multiple cross-dataset and single-table models are included in the benchmark.
Critical weaknesses
  • I find section 3.1 confusingly written: The assumption of factorization is fully described in Equation 8. That these diffusion processes act feature-wise is not novel but how most diffusion-based model are setup. Furthermore, the denoiser given by μθ\mu_\theta in Equation 6 and 7 is wrong, as it only conditions on a single feature, not the entire noisy data. This way, the denoiser cannot learn any feature dependencies. Judging from discussion in section 3.2, the denoiser does actually condition on the full data row. This is also nothing new, as many tabular generators define diffusion in exactly the same way. The only addition is the schema conditioning.
  • The description of the shared MLP in Equation 14 does not match Figure 3. Also, the notation is very sloppy as μθ\mu_\theta is reused again as the last layer of the denoiser.
  • I think the benchmark is not very fair. The number of datasets is limited, in particular compared to recent papers. CDMD uses a complex transformer-based architecture, whereas other diffusion baselines, like TabDDPM, TabSyn and TabDiff, use a much smaller, mostly non-transformer-based, architecture. This difference in model complexity may explain part of the performance benefit.
  • The paper uses a very unusual aggregation of the sample quality metrics. This may hide cherry-picked results. Why not display a ranking per metric as the main results? Also, currently not all individual metrics are weighted the same. DCR is not a valid privacy metric [1]. The most important metric for tabular sample quality, the classifier-two-sample-test (C2ST) is not included at all. This makes me suspect that mostly AI was used to run the experiments and pick the metrics.
  • There is no information given on training and sampling times, also not for pre-training. There is very little information on the pre-training in general, so the reader does not know how the final model was trained. I suspect that this hides some major inefficiencies, given the large transformer architecture relative to the included baselines.

[1] The DCR Delusion: Measuring the Privacy Risk of Synthetic Data. https://arxiv.org/abs/2505.01524

Questions for authors
  • How was the final model used for the main results trained? Does this include pre-training? How long is the training time compared to the baselines?
  • What happens if we use the same transformer-backbone for the baseline diffusion models?
  • Why did you not include the C2ST score?
  • Is the problem of not being able to generate previously unseen datasets for which we only observe the schema really a practical problem? Should we not almost always expect a better performance by just training a new model?
10 comments
AuthorsOpenReview ↗

Rebuttal (Part 1)

We thank the reviewer for their valuable feedback, which helped us improve the paper's quality and clarity. In the following, we address the reviewer's concerns and clarify different aspects of our paper.

First, we would like to clarify the contributions and main experimental results mentioned by the reviewer in the paper summary:

[...] The main contribution lies in changing the backbone to a transformer-based model and adding schema conditioning. [...]

Our main contribution is to define and train the diffusion model directly on the mixed-type data space rather than on a latent space, as done by previous cross-dataset tabular generative models. To achieve this, we extend masked diffusion to heterogeneous vocabularies and use a transformer-based model that adapts both its input and output to the target schema, which may have different numbers of columns, different categorical vocabularies, and, in general, different distributions.

[...] Results show some marginal improvements over existing models, which are only trained on a single table.

This refers to only one out of three main experiments presented in the paper. In this experiment, we showed that a single model checkpoint trained on a collection of datasets can match or exceed the performance of single-dataset models trained separately on each dataset (Section 4.1, Table 1). In the second and third experiments (Section 4.2), we show that CDMD can significantly improve generation performance in constrained settings by leveraging pre-training on a bigger dataset corpus (337 datasets). We emphasize that the two sets of experiments have distinct setups and serve different goals.

AuthorsOpenReview ↗

Rebuttal (Part 2)

Weaknesses

I find section 3.1 confusingly written [...] the denoiser given by μθ\mu_\theta in Equation 6 and 7 is wrong, as it only conditions on a single feature [...]

We wish to clarify that the structure of our method section, particularly Sections 3.1 and 3.2, follows the structure of seminal works on discrete diffusion models (see, e.g., [1,2]). Following these papers, we first assume a single feature to, more easily but without loss of generality, describe the noising process, the true denoising process, and the objective function (Section 3.1). These aspects are separate from the denoising model architecture, as different architectures can be trained with the same objective. Then, we extend to multiple features by factorizing the forward process and defining a denoising network that is indeed conditioned on the full row (Section 3.2). For this reason, the denoiser in Equations 6 and 7 (and also 2) is conditioned on a single feature, and Equation 10 shows that the final objective function has the same overall form as the feature-level objective functions (Equations 2 and 7), so the denoiser can be extended directly to multiple features.

[...] The only addition is the schema conditioning.

We would like to emphasize that the critical differences from existing tabular generators are not only the schema conditioning but also the masked diffusion process, which can natively handle heterogeneous vocabularies (Section 3.1), and the corresponding output modules in our architecture (Section 3.3). All these components are crucial to unlock cross-dataset training for existing mixed-type tabular diffusion models.

The description of the shared MLP in Equation 14 does not match Figure 3. Also, the notation is very sloppy, as μθ\mu_\theta is reused again as the last layer of the denoiser

In Figure 3, we used the same color to denote shared blocks. So, the two MLPs in Figure 3 are the same and simply applied twice, once for each numerical feature in the processed row. Also, we used μθ\mu_\theta to describe the last layer because the output of μθ\mu_\theta is the same as the output of its last layer. We will clarify both points in the revised version of the manuscript.

[...] CDMD uses a complex transformer-based architecture, whereas other diffusion baselines, like TabDDPM, TabSyn and TabDiff, use a much smaller, mostly non-transformer-based, architecture. [...]

We emphasize that we deliberately designed our model to match the number of parameters of the diffusion baselines. Specifically, CDMD, TabDiff, and TabSyn each have 10.7M parameters, while TabDDPM has 11.7M parameters. Also, TabSyn and TabDiff both use transformers in their denoising model [3,4].

[...] This difference in model complexity may explain part of the performance benefit.

Additionally, to separate the effect of model architecture from cross-dataset training, in the experiments of Section 4.2 we compare against a version of CDMD trained only on the target dataset, matching the setup of the single-dataset baselines.

The paper uses a very unusual aggregation of the sample quality metrics [...] DCR is not a valid privacy metric [...]

While we present the aggregated metrics in the main text, due to the large number of metrics/datasets/baselines, we report the detailed results for each metric in Appendix D. In Table 1, we also show two aggregation schemes: with and without the privacy proxy metrics. We agree with the reviewer that DCR is not a valid privacy metric, which is why we also used Authenticity and labeled both as "Privacy proxies". We highlight that the experiments and metric choices were fully designed by us.

There is no information given on training and sampling times, also not for pre-training [...]

On the same hardware, pre-training on the 337 datasets takes 2 days, while target-table adaptation takes between 15 minutes and one hour, depending on the number of samples. Sampling a batch of 10000 rows takes less than 1 minute on average. We will add these details to the revised manuscript.

AuthorsOpenReview ↗

Rebuttal (Part 3)

Questions

How was the final model used for the main results trained? Does this include pre-training? How long is the training time compared to the baselines?

The CDMD model in Table 1 is trained jointly on the seven benchmark datasets without pre-training. On the other hand, the models from Figures 1 and 2 were pre-trained and then fine-tuned in the specified setup. On a single dataset, the training time is the same as TabDiff, which is expected given the similar training objective and backbone architecture and size. However, training our model on the seven benchmark datasets was faster than training seven TabDiff instances, as it converges faster when trained on multiple datasets.

What happens if we use the same transformer-backbone for the baseline diffusion models?

This can be seen from the "CDMD (from scratch)" baseline in Figures 1 and 2: our training objective (Equation 10) generalizes TabDiff's objective, and when we remove cross-dataset training, the two are nearly equivalent. This is also reflected in both models behaving similarly.

Why did you not include the C2ST score?

In earlier experiments, we found that the C2ST score was almost perfect for most baselines, as reported in [3,4]. Because this metric depends on the choice of classifier, we decided to exclude it. However, we will include it again in the revised manuscript.

Is the problem of not being able to generate previously unseen datasets for which we only observe the schema really a practical problem? Should we not almost always expect better performance by just training a new model?

We agree that we should expect better performance by training a new model. But this holds only if the target dataset has enough data to train the new model from scratch. As our experiments show, when only a small amount of data is available, training a new model from scratch yields limited performance, and pre-training helps in this case. We argue that cross-table pretraining is a prerequisite for further scaling the diffusion backbone, which is likely to improve performance even further. The extreme "zero-shot" case, where we observe only the schema, can be useful for data augmentation or for training foundation models, but we leave this exploration to future work.

We hope that our clarifications help the reviewer reassess the quality of our work, and we are happy to address any remaining concerns.

References

[1] Austin, Jacob, et al. "Structured denoising diffusion models in discrete state-spaces." Advances in neural information processing systems 34 (2021): 17981-17993.

[2] Sahoo, Subham S., et al. "Simple and effective masked diffusion language models." Advances in Neural Information Processing Systems 37 (2024): 130136-130184.

[3] Zhang, Hengrui, et al. "Mixed-type tabular data synthesis with score-based diffusion in latent space." International Conference on Learning Representations. Vol. 2024. 2024.

[4] Shi, Juntong, et al. "Tabdiff: a mixed-type diffusion model for tabular data generation." International Conference on Learning Representations. Vol. 2025. 2025.

Reviewer_KJr6OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

CDMD is an end-to-end diffusion generator shared across tabular datasets with different schemas. It combines Gaussian diffusion for numerical features and masked diffusion for categorical features with a schema-conditioned Transformer. Its categorical interface uses a vocabulary-aware masked-token representation and a shared decoder over each column's valid categories. The paper evaluates joint generation on seven datasets and adaptation after pretraining on 337 datasets. Its main empirical result is the highest reported average generation-quality score among the evaluated generators, with substantially fewer aggregate trainable parameters than separately trained generators.

Critical strengths
  1. The parameter-sharing result is meaningful. A single 10.7M-parameter checkpoint serves seven heterogeneous datasets, compared with 74.8M parameters for the collection of independent TabSyn or TabDiff models. This is approximately an 86% reduction in aggregate trainable parameters, excluding the frozen text encoder. Average generation quality is 85.09, compared with 84.27 for TabSyn and 84.22 for TabDiff (Table 1; Section 4.1). CDMD also has the highest reported mean generation-quality score among the evaluated generators on six of the seven datasets. The storage benefit is supported by the reported results and does not require a large improvement in quality to be useful.

  2. The architecture provides a coherent interface for sharing a mixed-type generator across different column counts and categorical vocabularies. Type-specific modules are shared across tables, while schema information identifies the fields and constrains categorical predictions. Direct mixed-type diffusion permits joint optimization of the trainable input, denoising, and output modules without the separate autoencoder stage used by latent approaches such as CTSyn.

  3. The evaluation includes useful transfer evidence. Pretrained and randomly initialized CDMD are compared, and the limited-data experiments use the same target subsets across methods. Numerical preprocessing is fitted only on the available training rows. The appendix provides component metrics and generation-quality curves that exclude privacy proxies, helping interpret the aggregate results.

Critical weaknesses
  1. The predictive benefit over a real-only baseline remains untested in the few-shot setting. Section 4.2 concludes that pretraining "supports effective generation from limited target data", but Figures 1 and 5 report only aggregate S and Q scores. Both scores include predictive utility, yet utility is not reported separately and no predictor trained on the same N real target rows is included. The aggregate gains therefore do not establish whether synthetic-data training improves prediction over using those rows directly. Table 4 provides a real-data reference for the full-data synthetic-only evaluation, but does not resolve this few-shot comparison. The paper also does not report a real-plus-synthetic versus real-only comparison. Please report utility separately for XGBoost trained on the same N real rows alone, synthetic rows alone, and real-plus-synthetic rows, with uncertainty and several synthetic-to-real ratios. Use the same held-out real test set and comparable predictor tuning, and report each method's access to external pretraining data.

  2. Computational cost is not measured. The introduction motivates cross-dataset modeling partly through reduced per-dataset training and storage costs. The paper reports parameter counts and epoch-based adaptation curves, but no measured training time, GPU-hours, peak memory, or generation throughput. The categorical decoder processes all candidate-category embeddings of each column as target tokens (Equation 15). Parameter counts alone do not quantify the computational cost of this operation, and equal epoch counts do not imply equal compute budgets. Quality-versus-cost comparisons would clarify the practical tradeoff. Pretraining cost should be reported separately from target adaptation, since Section 4.2 fine-tunes a separate model for each target dataset.

  3. The full-data in-domain comparison lacks a matched single-dataset CDMD control. Section 4.2 includes CDMD trained from scratch under constrained settings, but Section 4.1 does not compare joint CDMD with the same architecture trained separately on each dataset to comparable convergence. This control would establish whether single-dataset CDMD also outperforms single-dataset baselines, and whether joint training improves or compromises its quality. A comparison of independent CDMD, joint CDMD, and single-dataset baselines should specify model capacity, data exposure, and optimization budgets. The missing control limits attribution of the quality differences to the architecture versus cross-dataset sharing.

  4. The key categorical modules lack ablation evidence. The paper does not isolate the vocabulary-aware mask representation in Equation 11 or the Transformer categorical decoder in Equations 15-16. Two particularly informative controls are replacing the attention output in the masked branch of Equation 11 with a single learned mask embedding used without attention to the column's categories, and replacing the categorical decoder with a simpler shared candidate scorer conditioned on the field representation. Both controls should retain the schema, type, and time embeddings, parameter sharing, and the same valid-category output support. Reporting their quality and computational cost would show whether the added mechanisms justify their complexity. The pretrained-versus-scratch experiment already addresses pretraining and should be distinguished from these missing architectural controls.

  5. The generator comparison omits relevant recent baselines. The evaluation includes strong single-dataset methods such as TabSyn and TabDiff, and the cross-dataset method CTSyn. However, TabNAT (ICML 2025) [1] is a directly relevant omitted comparator: it supports unconditional generation of numerical and categorical columns and evaluates six datasets also used here. NRGBoost (ICLR 2025) [2] would additionally provide a complementary tree-based joint-density generator that can sample synthetic rows. Both have public official implementations predating this submission. Please compare them under the same data splits, synthetic sample counts, and evaluation protocol, with fidelity, utility, and measured computational cost reported separately. These are generation baselines for Section 4.1; they do not provide a matched comparison of cross-dataset pretraining. Their omission limits the scope of the empirical comparison.

Questions for authors
  1. For each target-data budget N, how does held-out predictive utility compare when XGBoost is trained on the N real rows alone, synthetic rows alone, or both, and how does it change with the synthetic-to-real ratio?

  2. Can the authors report training time, hardware, peak memory, and sampling throughput, with data exposure and the cost of pretraining made explicit?

  3. In the full-data in-domain setting, how does CDMD trained separately to comparable convergence perform against joint CDMD and single-dataset baselines?

  4. What changes when vocabulary-aware mask attention and the categorical Transformer decoder are individually replaced by the simpler controls described above?

  5. How does CDMD compare with TabNAT and NRGBoost under the same generation and evaluation protocol?

2 comments

We thank the reviewer for their valuable feedback, which helped us improve the quality and the clarity of the paper.

We would like to start by clarifying our main contributions, the major results we presented in the paper, and the experiments we conducted to support these claims. Then we address the reviewer's questions and concerns.

Main contributions

Our main contribution is to define and train the tabular diffusion model directly on the mixed-type data space rather than on a latent space, as previous cross-dataset tabular generative models do. To achieve this, we extend masked diffusion to heterogeneous vocabularies and use a transformer-based model that adapts both its input and output to the target schema, which may have different numbers of columns, different categorical vocabularies, and, in general, different distributions.

Main results

Main Result 1: CDMD can be effectively trained on a collection of heterogeneous datasets to jointly learn their distributions using one shared model. It significantly outperforms CTSyn, which is also trained on all datasets but relies on a latent space. CDMD even outperforms state-of-the-art single-dataset baselines, which are trained separately on each dataset. While the performance difference between CDMD and the single-dataset baselines is less pronounced than with CTSyn, the benefit of CDMD is that it requires only a single model across all datasets rather than one per dataset. For this experiment, we used seven real-world datasets, compared against nine baselines (seven single-dataset and two cross-dataset models), used the real dataset as a reference, and reported seven distinct synthetic data quality metrics (Section 4.1).

Main Result 2 & 3: Pre-training CDMD on a collection of 337 datasets improves its generation quality on previously unseen datasets when the number of training samples or training epochs for the target dataset is constrained. In these experiments, we compare to CTSyn, which we pre-train on the same 337 datasets, and to the best-performing single-dataset baselines from the previous experiment, which, however, can only be trained on the target dataset. Crucially, to separate the impact of knowledge transfer from that of the model architecture, we also compare against CDMD when trained from scratch on the target dataset. These results show that CDMD indeed benefits from pretraining when the target data are limited or when the number of epochs is limited (Section 4.2).

With this perspective, we hope to clarify that adding more baselines and/or more metrics would not change the main conclusions we drew from this work.

Weaknesses & Questions

In the following, we address the reviewer's concerns and questions:

(W1, Q1) While we agree that extensively evaluating the utility of synthetic data for downstream model training is useful, we think that it provides an orthogonal direction to our work. We do include utility scores in all our experiments in the paper.

(W2, Q2) On the same hardware, pre-training on the 337 datasets takes 2 days, while target-table adaptation takes between 15 minutes and one hour, depending on the number of samples. Sampling a batch of 10000 rows takes less than 1 minute on average. In our experiments, training the model from scratch on the target dataset required at least 2000 epochs to converge (consistent with TabDiff), whereas a pre-trained model converged after 50 epochs.

(W3, Q3) As discussed above, we do not claim that cross-dataset training across seven datasets improves generation quality on the individual datasets; rather, we show that a single shared CDMD model can match the performance of single-dataset models. Indeed, training CDMD separately on each dataset achieves an average generation quality of 85.10, which is very close to the shared model's score of 85.09. The benefit of the shared model is that it requires fewer model instances and hence fewer total parameters.

(W4, Q4) When developing our model architecture, we tested different variations of the model and found that having the categorical decoder attend to the allowed categories was important for high-quality generation, whereas the mask representation was significantly less important. We will add the detailed ablation results to the revised manuscript.

(W5, Q5) We thank the reviewer for the suggested additional baselines. Due to limited time during the rebuttal period, we were unable to evaluate them in our setup, but we will include them in the revised manuscript.

We hope that our clarifications help the reviewer reassess the quality of our work and are happy to address any remaining concerns.

Thank you for the detailed rebuttal and clarifications. I appreciate the authors' efforts in addressing my concerns. In particular, the additional single-dataset CDMD comparison (85.10 vs. 85.09) substantially addresses W3 and provides stronger evidence for the benefits of parameter sharing. The additional computational cost information and architectural clarifications are also helpful.

I still have a few reservations regarding the empirical evidence:

1. Downstream utility (W1): I understand that data augmentation is not the primary focus of this work. The synthetic-only predictive utility evaluation and real-data reference in Table 4 are valuable. However, these results do not establish whether adding synthetic data improves predictive performance over using the available real data alone. Since data augmentation is discussed as a motivating application, this distinction remains relevant, particularly in limited-data settings. Previous studies, including TabEBM [1] and Curated LLM [2], have explicitly evaluated such downstream benefits.

2. Computational efficiency (W2): The additional timing information is helpful. However, without direct computational cost comparisons against existing methods, the overall efficiency advantage remains unclear. Reducing the total parameter count does not necessarily translate into lower computational costs.

3. Ablation studies (W4): I appreciate the explanations regarding the relative importance of the categorical decoder and vocabulary-aware mask representation. However, their individual contributions remain insufficiently supported by quantitative evidence, making it difficult to assess the benefits of the proposed architectural choices.

Overall, I appreciate the authors' careful and constructive responses. The rebuttal has clarified the paper's contributions and meaningfully addressed several of my original concerns. While some limitations in the empirical validation remain, I acknowledge the value of the additional evidence and the authors' efforts to strengthen the work.

References

[1] Margeloiu, A., Jiang, X., Simidjievski, N., & Jamnik, M. (2024). TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models. Advances in Neural Information Processing Systems, 37, 72094–72144. https://proceedings.neurips.cc/paper_files/paper/2024/hash/8488454077cb0fd9d31772274c78115d-Abstract-Conference.html

[2] Seedat, N., Huynh, N., van Breugel, B., & van der Schaar, M. (2024). Curated LLM: Synergy of LLMs and Data Curation for Tabular Augmentation in Low-Data Regimes. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 44060–44092. https://proceedings.mlr.press/v235/seedat24a.html

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.