- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors propose a diffusion-based model for tabular prediction. It is trained by denoising noisy versions of the output target embeddings of a frozen teacher TFM. This kind of distillation is trained on datasets from TabLib. At inference time, it starts from random noise and denoises it toward embeddings that can be used for predictions. The denoising is conditional on the input's table encoding. Predictions are made with a post-trained output head. The idea is evaluated on the TabArena datasets with a custom validation protocol.
Critical strengths
The one critical strength of the paper is the combined idea of distillation and denoising to produce a generally suitable approach to obtain a diffusion-based model for tabular predictions. The idea is very interesting and leaves a lot of room for follow-up questions and research. The idea is worth sharing.
Critical weaknesses
I see two critical weaknesses: the work lacks quality and rigor. The presentation is generally confusing; the related work is minimal; several findings lack sufficient evidence; and the main validation protocol is flawed and provides insufficient positive evidence.
In short: in several places, the presentation lacks reasoning or detail (e.g., why pretrain on TabLib instead of a synthetic prior, why MotherNet is an alternative regime in the related work, how many datasets were filtered by the approach in Appendix A1). In its current form, the manuscript and its argumentative flow are unnecessarily hard to follow. Moreover, the authors claim to follow the official TabArena protocol but deviate significantly (wrong splits, wrong baselines, wrong metrics, wrong aggregation). As a result, the reported validation is extremely untrustworthy, and their methods' results across metrics and aggregation (unweighted vs rank) suggest bias from the self-chosen aggregation. Neither the deviation nor the new metric choices are justified. Moreover, they omit stronger TFM baselines, and their argument for improving the teacher is underdeveloped, especially since a basic fine-tuned version of Limix already seems competitive with the denoiser. A more rigorous evaluation and alternative post-training baselines are needed to justify the method based on predictive performance alone. Furthermore, the presentation needs significant improvement, including the results and the paper's argumentative flow.
Questions for authors
Given the paper's current state, there are no questions that would affect my assessment.