acceptodds
Under review as a conference paper at ICLR 2027

TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data

Abstract

Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark for evaluating synthetic tabular data with SQL-shaped analytical queries. TabQueryBench distills 12 public query sources into 44 reusable query templates and grounds them to each dataset through a policy-guided template-to-SQL pipeline. Across 49 datasets and 11 generative models, it produces more than 100 executable queries per dataset while preserving cross-model comparability. Our evaluation reveals a substantial gap between conventional and query-centric fidelity. Even the strongest model, RealTabFormer, reaches only query-centric fidelity compared with for real data. Failures are most pronounced for high-cardinality attributes and local or tail queries, while model quality also trades off against generation cost. These results show that synthetic data can appear statistically realistic yet still fail to preserve the analytical behavior of real data, motivating query-centric fidelity as a necessary complement to existing evaluation metrics. TabQueryBench is open-sourced and available for anonymous review.

Primary area:
datasets and benchmarks
Keywords:
Synthetic Data, Benchmarking, Data Fidelity, Query-Centric Evaluation, Generative Models.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GFAkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TabQueryBench, a query-centric benchmark for evaluating the quality of synthetic tabular data. The method constructs 44 query templates from 12 public SQL sources, covering five categories of analytical tasks, and measures Query Fidelity by comparing the results of SQL queries run on real data with those run on synthetic data. The authors conducted experiments on 49 datasets and 11 generative models and found that traditional statistical similarity metrics fail to adequately reflect query fidelity; existing models struggle in particular to preserve high-cardinality attributes, local conditional structures, and tail distributions.

Critical strengths
  1. The paper addresses a meaningful limitation of existing synthetic data evaluation approaches.

  2. The proposed benchmark provides a relatively comprehensive and automated evaluation framework, and the experiments are extensive and provide useful insights into existing generative models.

Critical weaknesses
  1. The validity of Query Fidelity still requires further validation. This paper proposes evaluating generation quality by comparing the results of SQL queries on real data and synthetic data, which is a reasonable research motivation. However, even if the generative model has correctly learned the distribution of real data, randomly generated data may still produce different query results. Therefore, it remains unclear to what extent these differences reflect shortcomings in the generative model. I recommend that the authors include a “Real-vs-Real” baseline and further validate whether Query Fidelity can effectively reflect the performance of synthetic data in practical analytical tasks.

  2. Some experimental results and scoring methods require further explanation. First, Section 3.3 mentions that each dataset contains more than 100 SQL queries, but Section 4.6 reports a total of only 2,763 queries, and Appendix Table 17 also shows that some datasets have fewer than 100 queries. I recommend that the authors explain why these numbers do not match. Second, the authors evaluate the generative models by calculating the average score across all SQL queries. However, since the number of queries may vary by type, query types with higher counts may have a greater impact on the final score. I recommend that the authors test whether the final model rankings would change if queries of different types were assigned equal weights. I am willing to increase my score if my concerns are addressed, especially the second one.

Questions for authors
  1. Could the authors include a Real-vs-Real comparison experiment? Specifically, randomly select two subsets of the same size with no overlap from the same real dataset, execute the same SQL query on both subsets, and observe how much variation there is in the query results themselves. This would help determine whether the differences in query results between synthetic and real data stem from poor performance of the generative model or are simply due to variations that would naturally occur with random sampling.

  2. Could the authors explain the inconsistencies in the number of SQL queries reported in the paper? Section 3.3 mentions that over 100 executable SQL queries were generated for each dataset, but Section 4.6 reports a total of only 2,763 queries, and Appendix Table 17 also shows some datasets with fewer than 100 queries. Do these figures stem from different experimental setups or counting methods?

  3. Have the authors considered the impact of the number of different query types on the final score? For example, if one type of query is very common while another is rare, the more common type may have a greater influence on the final score. Could the authors try assigning equal weights to all five query types and examine whether the final rankings of different generative models would change?

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer GFAk

We thank you for finding our paper clear and recognizing the breadth and practical insights of our automated SQL benchmark. Below, we address your questions on Real-vs-Real sampling validation, query-count reconciliation, and equal query-family weighting, all updated in our revised PDF.

W1/Q1: Real-vs-Real Split-Half and Resampling Validation Following your exact recommendation, we evaluated three Real-vs-Real sampling baselines across all 49 datasets—including partitioning each real table into two non-overlapping 50% halves and running the identical SQL suite on both (full calibrated table in our reply to Reviewer NUsw, W1):

  1. Disjoint 50/50 Split-Halves (zero overlap): 0.829 mean Query Fidelity.
  2. Independent 50% Subsamples (without replacement): 0.881.
  3. Size-Matched Bootstrap Resampling (ntrain rows with replacement): 0.906.

Even at half the sample size with zero row overlap (0.829), real data subsets outperform every non-memorizing generator trained at full ntrain scale (0.443–0.636 raw; 0.480–0.695 calibrated against the 0.906 bootstrap oracle). This confirms that the reported deficits reflect genuine generative modeling limitations rather than natural random-sampling variance.

W2/Q2: Reconciling SQL Query Counts (Section 3.3, Section 4.6, Appendix Table 17) Thank you for catching both discrepancies, which stem from an unedited prose reference to an early prototype and multi-run API cost logging:

EnginePer-Dataset (Min/Med/Max)TotalRole in Paper
Template83 / 606 / 68126,548Early prototype (unused)
GLM-5.318 / 57 / 962,763Paper suite (all results)
  • Section 3.3 vs. Section 4.6: All experiments use the single frozen GLM-5.3 workload (2,763 queries across 49 datasets, mean 56.4). The phrase "over 100 executable SQL queries" in Section 3.3 was a holdover from Template and now reads "18–96 executable SQL queries per dataset (median 57; 2,763 total in frozen GLM-5.3 version)."
  • Appendix Table 17: The Appendix API-usage table logs cumulative LLM grounding calls across three builder stability runs (2,761+2,760+2,763=8,284; e.g., c1 =263=78, c3 =183=54), not a single benchmark pass. We have clarified the Table 17 headers and caption accordingly.

W3/Q3: Equal Weighting Across the Five Query Families We implemented equal weighting across active query families within each dataset (added in revised Appendix R.4):

GeneratorQuery-EqRankFamily-EqRank
RealTabFormer0.75810.8001
ARF0.63620.7202
BayesNet0.60530.6773
TabDiff0.59540.6234
TabSyn0.57550.6165
TabbyFlow0.56660.5937
CTGAN0.55070.5908
TVAE0.53780.5709
TabPFGen0.52990.6086
TabDDPM0.461100.48211
ForestDiffusion0.443110.49810

The top five generators remain completely unchanged, confirming our conclusions are robust to query-family frequency. TabPFGen rises from 9th to 6th (0.529 to 0.608) because its per-family performance is balanced whereas cardinality—its weakest family—contributes the most queries under default averaging. Because many real datasets lack NULLs or high-cardinality columns (making only 3 or 4 families active and shifting family weights between 33.3%, 25.0%, and 20.0%), we report both query-equal and family-equal scores side-by-side in our revised leaderboard.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Additional Results

Since the rebuttal period is approaching the end, we’d like to kindly ask whether you’ve gone through our new results and whether they’ve addressed your previous concerns. We understand that you might be busy with other reviewing tasks and your own submissions, but we sincerely appreciate your latest comments on our rebuttal contents and your willingness of updating the score. Thank you so much!

Reviewer_r525OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

TabQueryBench evaluates synthetic tables by asking whether they preserve the answers to analytical SQL queries. The motivation is that matching individual column distributions may still leave filtered groups, rare events, or relationships between columns poorly represented. The benchmark uses an LLM-assisted grounding pipeline to apply 44 query templates to 49 single-table datasets. Its five query families cover subgroups, conditional relationships, tails and rarity, missingness, and cardinality and range. Answer-specific scorers compare numeric aggregates, count distributions, rates, and ranked identities.

Experiments evaluate eleven generators and report rankings that differ from those obtained with marginal distribution metrics. The paper also reports losses in high-cardinality support and local-slice fidelity, compares generation costs, and examines stability across workload rebuilds and structural overlap with public SQL workloads. The main contribution is an executable query suite for examining how synthetic data supports analytical use.

Critical strengths
  1. The evaluation target is useful and concrete. Comparing analytical answers reveals errors that marginal distribution summaries can hide. The paired local and global queries are particularly informative because removing the filter preserves the rest of the operation. This provides a practical way to examine how synthetic data behaves in downstream analysis.

  2. The benchmark offers substantial reusable infrastructure. The suite covers 44 templates, 49 datasets, and eleven generators, with explicit answer types and deterministic scoring rules. The workload-rebuild analysis also retains the same four leading generators across seven builder runs. This breadth and attention to the evaluation instrument are meaningful strengths.

Critical weaknesses
  1. The scores do not yet support the claimed diagnosis of analytical fidelity. REAL is a self-comparison rather than a fresh sample. For the c3 column with 2,541 distinct values in 2,551 rows, same-size empirical resampling would retain roughly 1,607 to 1,609 distinct values on average. Six reported generators produce 1,535 to 1,649, so this loss of support alone does not establish a modeling failure. Separately, Appendix E.2 normalizes grouped counts before scoring. Counts of 100 and 200 therefore receive the same perfect score as counts of 10 and 20, despite a 90% loss of slice mass. These issues affect the benchmark's central conclusions about support and subgroup preservation.

  2. The leaderboard mixes generator behavior with configuration and coverage differences. Appendix B.2 gives CTGAN an embedding size of 4, hidden layers of 8 units, batch size 2, and 50 epochs. BayesNet has explicit category and training-row caps, which can directly limit the support it generates. The paper does not establish how much of the reported failure comes from these choices. Technical failures also change the evaluated dataset set across models. For example, TabDDPM fails on eight of 49 datasets, and its mean over the remaining datasets is about 0.461 versus 0.386 if failures count as zero. Neither convention alone is sufficient, but the difference shows why common-dataset and failure-aware comparisons are needed before interpreting the ranking.

Questions for authors
  1. How do the query scores, distinct counts, and local-slice gaps compare with repeated same-size samples from the empirical training distribution? Which findings remain beyond that sampling variation? A copying baseline or memorization diagnostic would also clarify what high fidelity rewards.

  2. Is the perfect score for proportionally reduced group counts intended? Can slice magnitude and conditional composition be reported separately? Please also resolve whether the continuous-tail threshold in Appendix C.4.3 denotes total mass or mass per side, since the written definition and formula give different self-comparison values.

  3. How sensitive are the main findings to CTGAN's small configuration and BayesNet's category caps? Comparisons with justified configurations and matched training-row exposure would determine whether the conclusions concern the generators or the benchmark adapters.

  4. Which dataset and query weighting, failure policy, and workload version define the leaderboard? Please reconcile Tables 18 and 20, define the reported uncertainty, and provide a ranking on the common successful dataset set alongside failure rates.

3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer r525

We thank you for recognizing the diagnostic value of our paired local–global design and the breadth of our benchmark infrastructure.

Your detailed feedback motivated additional empirical sampling analyses and a clearer explanation of our scoring and aggregation protocols.

W1/Q1: Empirical Resampling, Memorization, and High-Cardinality Support

  • Resampling & Memorization: As detailed in our response to Reviewer NUsw, three Real-vs-Real references yield 0.906 (size-matched bootstrap), 0.881 (independent 50% subsamples), and 0.829 (disjoint split-halves). The paired local–global fidelity gap remains larger for generators (0.047) than for bootstrap resampling (0.008), with a larger gap on 22/29 datasets. We also conducted a 49×11 exact-row overlap audit and 51 rejection-sampling sensitivity experiments to examine the influence of training-row overlap on Query Fidelity.

  • High-Cardinality Support: We agree that raw distinct-count reductions must account for finite-sample variation. On c3, the real column contains 2,541 distinct values, whereas same-size empirical resampling yields approximately 1,609 on average. Six generators produce 1,535–1,649 distinct values, suggesting that part of the apparent support reduction is attributable to sampling effects rather than necessarily to model failure. We therefore interpret raw cardinality comparisons with reference to empirical resampling, while retaining the original Query Fidelity scoring protocol.

W1/Q2: Slice Mass vs. Composition and Tail Threshold (τ\tau)

  • Composition vs. Mass: Proportional scoring (1-TVD) is intentional (Appendix E.2) and applies only to grouped-count queries (43.1%, 1,192/2,763), whereas SUM/AVG aggregates (34.6%, 955) penalize scale errors via sMAPE. Rather than multiplying composition and mass into an entangled scalar, revised Appendix E reports per-family mass-similarity diagnostics and flags queries below 0.5 (catching 44/1,046 RTF cardinality queries caused by row-count violations on n15/n17, now fixed).

  • Continuous-Tail τ\tau (Appendix C.4.3): Our code and all submitted figures use symmetric τ\tau and 1−τ1-\tau quantile cutoffs (τ\tau mass per side, 2τ2\tau total), giving exact self-comparison of 1.000 with zero numerical changes. The C.4.3 phrase "total probability mass τ\tau" was a prose typo (which would cap a perfect model at 0.5) and is now corrected alongside our one-sided penalty rationale.

W2/Q3: Adapter Configuration Sensitivity

Because datasets span orders of magnitude in scale under a 7-day wall-clock cap, compact adapters were used only on the largest tables to complete the 49×11 grid.

Across a 176-run hyperparameter sweep over all 11 models, scores are highly stable:

  • CTGAN: On c2, sweeping epochs 50–200, batch 32–128, embedding 8–32, hidden widths (16,16)–(64,64), and pac 1–10 yields scores of 0.976, 0.977, 0.978, 0.976, 0.977 (range 0.0015; maximum spread 0.034 across all swept datasets).

  • BayesNet: On c2, sweeping category caps 32–128 plus an uncapped variant, bins 4–8, fitting rows 2K–120K, and structure rows 1K–25K yields scores of 0.994, 0.994, 0.994, 0.993, 0.993, 0.995 (range 0.0020; maximum spread 0.002 overall).

Full sweep provenance is added to the revised Appendix.

AuthorsOpenReview ↗

Rebuttal to Reviewer r525 (Continue)

W2/Q4: Reconciling Tables 18 & 20 and Failure-Aware Rankings

Your check caught a real bug: submitted Tables 18 and 20 came from two scoring passes. Both are now reconciled onto frozen v11.2.2 (2,763 queries, 49 datasets; ±\pm denotes cross-dataset SD).

We compare five policies:

  • (a) Default dataset-equal averaging.
  • (b) Equal weighting across query families.
  • (c) Restriction to 35 common-success datasets.
  • (d) Restriction to 1,717 common-valid queries.
  • (e) Failure-aware evaluation (technical failures scored as zero).
Generator (Valid)(a) Default(b) Fam-Eq(c) Common-35(e) Fail=0
RealTabFormer (48/49)0.7580.8000.7670.743
ARF (49/49)0.6360.7200.6420.636
BayesNet (49/49)0.6050.6770.6240.605
TabDiff (44/49)0.5950.6230.6170.534
TabSyn (48/49)0.5750.6160.6050.563
TabbyFlow (47/49)0.5660.5930.6110.543
CTGAN (49/49)0.5500.5900.5560.550
TVAE (49/49)0.5370.5700.5510.537
TabPFGen (47/49)0.5290.6080.5380.507
TabDDPM (41/49)0.4610.4820.4660.386
ForestDiffusion (49/49)0.4430.4980.4630.443

Policy (d) matches (c) within 0.002 on every model.

The top three generators retain their rankings across all five policies, and the top four remain unchanged under policies (a)–(d). Failure-aware evaluation (e) explicitly penalizes models with technical failures (e.g., TabDiff drops from 4th to 8th).

Both common-dataset and failure-aware columns now appear in the main leaderboard.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Additional Results

Since the rebuttal period is approaching the end, we’d like to kindly ask whether you’ve gone through our new results and whether they’ve addressed your previous concerns. We understand that you might be busy with other reviewing tasks and your own submissions, but we sincerely appreciate your latest comments on our rebuttal contents and your willingness of updating the score. Thank you so much!

Reviewer_NUswOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a benchmark to evaluate synthetic tables using SQL queries. The same queries are run on the real and synthetic tables and the answers are compared. They test 11 generators on 49 datasets

Critical strengths

I think the problem is relevant. Many evaluations only look at column distributions, while this paper checks whether the answers to real queries are kept.

Critical weaknesses
  1. My main concern is how the reference is computed. In Table 18, real data gets 1.00 everywhere because the training table is compared with itself. This is a correct upper bound, but I do not think it is a realistic one. For example, two random samples from the same data will give slightly different answers, so even a perfect generator would get a score below 1. This difference can be larger for small slices and rare values, which is where many of the findings in the paper come from. So I am not sure how much of the reported gap is actually due to the generators.
  2. Another issue is that all the scores are computed against the training data. A generator that simply copies the training rows can get a very high score, and the benchmark may not tell this apart from a model that has actually learned the data. For example, I noticed that on some datasets RealTabFormer is much higher than all the other models in Table 19. I think this needs to be checked.
  3. I also doubt how the count queries are scored. In Appendix E, it is mentioned that the counts are first converted to proportions and then compared. This is fine when the query is on the full table, but for filter queries a much smaller synthetic slice can still get a perfect score. For example, if the real table has 100 and 200 rows in two groups while the synthetic table has 10 and 20, the score is still 1 even though the synthetic slices are ten times smaller. I think the scores for these queries, including the ones used for Finding 3, may look better than they actually are.
Questions for authors

check weaknesses

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer NUsw

We thank you for recognizing the practical relevance of evaluating synthetic tables via analytical SQL queries rather than marginal column distributions alone. Below, we address your concerns regarding the empirical reference ceiling, training-data memorization, and count-query scoring, with all updates incorporated into our revised PDF.

W1: Empirical Reference Ceiling and Calibrated Scoring We agree that comparing the training table against itself (REAL = 1.000) represents a noiseless identity bound rather than an attainable finite-sample ceiling. To isolate natural sampling variance from generative error, we evaluated three empirical Real-vs-Real baselines across all 49 datasets: (1) Bootstrap Resampling (ntrain rows with replacement): 0.906; (2) Independent 50% Subsamples (without replacement): 0.881; and (3) Disjoint 50/50 Split-Halves: 0.829.

Even an ideal empirical sampler scores ~0.91 due to finite-sample variance on small slices and rare groups. We adopt Bootstrap Resampling (1) as our primary oracle—matching the exact sample size and evaluation protocol of the generators—and report a Calibrated Score (raw score divided by per-dataset bootstrap reference, where 1.000 indicates parity with an ideal empirical sampler):

Model / ReferenceRawCalibratedDatasets ≥ Ref
REAL (Self-Comparison)1.0001.104—
Bootstrap Oracle0.9061.000—
RealTabFormer (RTF)0.7580.82512
ARF0.6360.6951
BayesNet0.6050.6610
TabDiff0.5950.6480
TabSyn0.5750.6250
TabbyFlow0.5660.6130
CTGAN0.5500.6000
TVAE0.5370.5840
TabPFGen0.5290.5761
TabDDPM0.4610.5000
ForestDiffusion0.4430.4800

Self-comparison exceeds the attainable ceiling by ~10%. After calibration, large modeling gaps remain: nine of eleven generators never reach the empirical reference on any dataset, ARF attains 0.695 of oracle fidelity, and RTF exceeding the reference on 12 datasets stems directly from memorization (W2).

W2: Memorization Audit and Zero-Copy Rejection Sampling Your observation regarding RealTabFormer's outlier scores in Table 19 pinpointed a critical issue. Auditing the exact-match fraction (synthetic rows identical to training rows) across the full 49×11 grid reveals a sharp divide:

GeneratorValidCopy > 0%Copy > 50%Max Copy
RealTabFormer48/4929171.000
ARF49/491150.955
TVAE49/491030.709
TabbyFlow47/49830.710
TabPFGen47/49920.898
TabDiff44/49720.731
BayesNet49/49920.632
CTGAN49/49910.589
TabDDPM41/49710.521
TabSyn48/49610.646
ForestDiffusion49/49510.641

For ten of eleven models, exact copies occur almost solely on small categorical tables (c1–c11) via natural discrete collisions and drop to 0.000 across nearly all medium/large tables (m1–m11, n1–n19). Conversely, RTF copies heavily across all regimes—exceeding 80% on 14 datasets and reaching 100% verbatim copies on c3, c10, c17 and 88.6%–97.4% on m1, n12, n15, n17.

To evaluate fidelity without memorization, we re-ran all 51 high-copy pairs using rejection sampling until exact matches reached 0.000 (full 49×11 matrix and all 51 pairs added to the revised Appendix; representative cases below):

CellPre CopyPost CopyPre ScorePost Score
RTF-c40.79300.9890.978
RTF-c90.87400.8770.819
RTF-m80.56700.9600.883
RTF-m10.97400.5850.619
RTF-n170.88600.6380.661
ARF-c20.75100.9400.821
ARF-c70.80800.9550.877
BayesNet-c20.63200.9780.864
TVAE-c20.70900.9510.836
TabDDPM-c10.41800.7890.918

Removing exact copies moderates inflated scores on heavily copied tables while improving fidelity in mode-collapsed runs (RTF-m1, TabDDPM-c1) by restoring sample diversity.

W3: Count Scoring Scope and Finding 3 Validation Proportional normalization is not applied universally; our 2,763 queries route by SQL contract to four distinct scorers:

  • Grouped counts (43.1%, 1,192): 1-TVD on proportions (composition); slice mass logged separately in Appendix E.
  • Numeric aggregates (34.6%, 955): Scale-sensitive normalized sMAPE (SUM/AVG, where 10× scale error is fully penalized).
  • Rates/proportions (21.4%, 592): 1-MAE.
  • Top-k rankings (0.9%, 24): Rank-Biased Overlap (RBO).

For Finding 3, comparing paired queries with vs. without WHERE filters against our bootstrap oracle shows generators suffer a 0.047 local–global drop vs. only 0.008 for empirical resampling, exceeding sampling variance on 22/29 datasets.

Thanks for the clarifications regarding the realistic reference, copy check on the generators, and count query scoring. However, I would like to maintain my score

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Author Response (All Reviewers & Area Chair)

We thank the Area Chair and Reviewers (NUsw, r525, and GFAk) for their constructive reviews and for recognizing TabQueryBench's SQL-grounded evaluation, paired local–global design, and breadth across 49 datasets and 11 generators. We have uploaded a revised manuscript and expanded Appendix (changes highlighted) addressing all shared and individual concerns.

Empirical Oracle Calibration To separate finite-sample variance from modeling error (raised by all reviewers), we added three Real-vs-Real baselines across all 49 datasets: size-matched bootstrap resampling (0.906), independent 50% draws (0.881), and disjoint split-halves (0.829). Calibrated against our bootstrap oracle, non-memorizing generators reach only 0.480–0.695 (ARF leads; nine models never match the oracle on any table), confirming clear modeling gaps beyond sampling noise. Paired WHERE queries also show a 0.047 local–global drop for generators versus 0.008 for resampling, exceeding sampling variance on 22 of 29 tables.

Summary of Manuscript Revisions

  • Memorization Audit: An exact-match audit across the full 49×11 grid exposes verbatim RealTabFormer copying across regimes (up to 100%), while 51 zero-copy rejection-sampling runs isolate true generalization (NUsw, r525).

High-Cardinality Analysis: We compare distinct counts against same-size bootstrap references to distinguish expected sampling effects from additional support loss (r525).

  • Leaderboard Robustness: Reconciling Tables 18 and 20 onto frozen release v11.2.2 (2,763 queries), top-3 rankings hold across five coverage/failure policies and top-5 hold under family-equal weights (r525, GFAk, App. R.4).
  • Adapter Sensitivity: A 176-run hyperparameter sweep across all generators confirms adapter stability, bounding score variation to at most 0.034 for CTGAN and 0.002 for BayesNet (r525).
  • Scorer Diagnostics: We added per-family slice-mass diagnostics for grouped counts (App. E), fixed the per-side tail prose (App. C.4.3), and aligned Section 3.3 and Table 17 query counts (NUsw, r525, GFAk).

Complete tables and point-by-point analyses appear in our individual responses below, and we welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.