- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces PAVES, a framework that leverages large language models (LLMs) to propose candidate relationships based on table column names, data descriptions, and domain-specific information. These relationships are then verified against actual data, with their parameters estimated accordingly. The retained information is structured into an executable specification, which is subsequently used to generate synthetic data through quota allocation, conditional sampling, and constraint solving. The authors aim to narrow the space of relationships requiring verification, improve relationship estimation when data is limited, and enhance the transparency and controllability of the data generation process. The paper provides a theoretical analysis of the trade-off between candidate space reduction and relationship estimation error, and evaluates the framework across multiple tabular datasets—assessing statistical quality, constraint satisfaction, downstream predictive performance, and scalability as the volume of synthetic data increases.
Critical strengths
I particularly appreciate how the author integrates relationship extraction, data validation, and generation/execution into a well-defined workflow, while storing the validated information in a specification that can be reviewed and modified. This design is highly practical for tabular data that must adhere to specific business rules; it allows users to trace which relationships the generation process relies upon and makes it easier to identify unreasonable constraints. Furthermore, the fact that a single specification can be used to generate datasets of varying scales opens up a compelling area for further research: how to control data volume—without the need for repeated model training—and whether increasing the amount of synthetic data can improve performance on downstream tasks.
Critical weaknesses
I am curious about the scope of applicability of this approach when applied to complex, real-world tables. I would like to see a more comprehensive comparison against representative state-of-the-art (SOTA) table generation methods—specifically, an evaluation of whether this approach maintains its edge in terms of generation quality, downstream performance, and computational overhead across varying data scales, higher column counts, and significant rates of missing values. While increasing the number of generated rows demonstrates a certain degree of scalability, high-dimensional tables introduce a larger set of candidate relationships and more complex joint dependencies; the performance of the method in these scenarios remains unclear.
Furthermore, the approach relies on an LLM to extract candidate relationships from table headers and metadata. In real-world business contexts, however, tables frequently contain abbreviations, internal codes, or incomplete field descriptions. Some relationships may even hinge on specialized industry knowledge, unique business exceptions, or definitions known only to domain experts.
I am also skeptical about the LLM’s ability to accurately interpret and propose candidate relationships involving complex multi-column statistical dependencies, non-linear formulas, or conditional mathematical constraints. While subsequent data validation can filter out some erroneous relationships, it is unclear whether there is a mechanism to recover relationships that were not identified in the first place. This issue is further exacerbated by missing data, which reduces the number of valid samples available for relationship verification. Consequently, I would like the authors to provide a clearer explanation of the method’s dependence on metadata quality, domain knowledge, and data completeness, and to include targeted experiments that demonstrate how performance shifts when these conditions vary.
Questions for authors
Could the authors clarify the rationale behind the current choice of baselines? Furthermore, given a standardized setup—consistent data partitioning, preprocessing, and generation scale—in what specific metrics and data conditions does this approach demonstrate a clear advantage over recent, representative state-of-the-art (SOTA) methods? How do the quality of candidate relationship identification, the quality of generated data, and the computational overhead (in terms of time and memory) vary when dealing with tables that have few rows but many columns, versus tables with a large number of both rows and columns? In scenarios involving varying degrees and patterns of missing data, how does the method validate cross-column relationships, and how does it ensure that the original missingness patterns—along with the dependencies between missing values and other variables—are preserved in the generated data? I am also interested in how the model’s performance is affected when table headers consist of abbreviations or internal codes, or when metadata is incomplete. Additionally, if correctly interpreting a specific relationship requires specialized domain knowledge, how does the system acquire such information? Regarding complex multi-column statistical relationships, intricate mathematical formulas, and constraints involving business-specific exceptions, could the authors provide concrete examples of both successful and unsuccessful cases? Specifically, which types of relationships can be automatically discovered by the current pipeline, and which require additional field definitions, domain-specific rules, or human input? These insights would help me assess the extent to which this method is applicable to real-world tables.