- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 3(+1)
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper argues that when you compare a reference dataset to a synthetically generated dataset, auditing that dataset for fidelity and diversity is important (because they mirror precision and recall). Fidelity is whether the dataset measures the same properties that the reference dataset measures. Diversity is ensuring the synthetically dataset (or the augmented dataset) is representative. They formulate this as an optimal transport problem: representing the samples with a Gaussian or an embedding and measuring how much mass to move from the synthetic/augmented set to the reference set. Their experiments show that their fidelity metric is able to capture different concepts/topics and their diversity metric is able to truly measure diversity (they do this with a synthetic setting in which they increase/decrease the diversity).
Critical strengths
The idea is well formulated, and the writing is well structured/reads well. Their experiments are also quite interesting, as they seem to be working as intended.
They also back their empirical claims with significance tests.
Critical weaknesses
I think this paper has much more potential than what is presented. Currently, the empirical contribution is weak.
While reading the paper, I thought of two potential directions that this can be applied to make this paper much stronger and impactful: (1) using these metrics to select a meaningful set of training data for fine-tuned or post-trained models, and (2) measuring whether popular benchmarks do indeed have high fidelity and diversity compared to a particular training task. To clarify, if the community used for training and for as the set of testing sets, you can use these metrics to test whether (1) is a good training set compared to the test sets, or (2) the testing sets truly cover .
Questions for authors
See Weaknesses.
2 comments
While I appreciate your answers to my questions (especially the correction to my misunderstanding in "Training data selection") and your general efforts in the rebuttal, I think there are a significant number of changes to be made from the original submitted work. I believe it would be beneficial to submit to another cycle and would result in a stronger submission. I can raise my assessment scores, but I will keep my rating the same.
Thank you for the positive comments and the thoughtful suggestions. We are encouraged that you see potential in this method, and we find both of your proposed applications exciting.
(1) Training data selection. Our GSM8K experiment (Section 4.2) is a first case study of exactly this direction. There, we score a family of synthetic finetuning datasets against a reference before any training, and find that our diversity score strongly correlates with the downstream accuracy of the finetuned models (r=+0.973, p=0.0002), while fidelity does not. This finding holds after removing the human-written seed problems as a potential confounder (r=+0.968, p=0.0003; see our response to reviewer MbcM). Since our metrics are inexpensive relative to finetuning (embedding cost is linear in corpus size, and the transport cost depends only on the number of atoms), this suggests a practical workflow in which practitioners may cheaply screen candidate training sets with our metrics. We will make this framing explicit in the revision.
(2) Benchmark auditing. Our framework also applies to this setting. Taking the target task data as the reference and the benchmark as the candidate, diversity measures how well the benchmark covers the task distribution, while fidelity measures whether benchmark is on-distribution. Because the two metrics are separate, they distinguish two distinct failures: whether the benchmark is overly concentrated on a few modes of the task (low diversity), or contains too much out-of-distribution material (low fidelity). Our new controlled experiments on mode collapse and outlier injection (see our response to reviewer MbcM) validate precisely these two cases. We will add a discussion of this application to the paper.
On the empirical contribution. Our primary goal was to establish that the metrics behave as intended, which we view as a prerequisite for trusting them in downstream applications such as the ones you propose. During the rebuttal, we have substantially extended this validation with experiments on candidate sample size, duplication, mode collapse in both the reference and the candidate, outliers, randomly sampled (non-nested) subdomains, and a comparison against DCScore. In each case, the metrics respond as expected. We believe these results, together with the GSM8K study, strengthen the empirical case considerably, and we will include them in the revision.
We hope these clarifications address your concerns, and would be grateful if you would consider them in your assessment.