- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposed an algorithm with the goal of sampling a diverse batch of responses under distortion constraints. They first define the notions of distortion and diversity in the probability space, specifically, the original model distribution and the joint distribution for batch sampling. Based on these definitions, they derive the best-diversity can be achieved under distortion constraints, and the lowest distortion under diversity constraints. The method is applied at test time. In practice, the algorithm samples redundant independent responses, and calculate the empirical desired joint distribution and subsample the batch of responses. They also propose an adaptive sampling algorithm along with the non-adaptive version. Through experiments, they show the effectiveness of their method.
Critical strengths
- The method is mathematically correct. And the motivation of explicitly controlling the diversity and distortion bound can be useful in some scenarios.
Critical weaknesses
-
The main concern is that the notion of distortion is defined in the probability space, i.e., regarding the model's distribution as the anchor, measuring the distance with it. This does not align with the usual motivation in this area where people care about the quality of responses when encouraging diversity. Controlling the distortion is just to stay close with the model's distribution, it does not capture the quality of the responses generally and heavily depends on the model's original distribution. From this point of view, I think the overall method framework is not that applicable.
-
Another concern is the gap between theoretical and practical finite data setting. In the perfect setting, one can derive the bound on distortion and diversity. However, in the finite data setting, the results are not guaranteed even for a weaker version. Proposition 3.2 only holds when is infinity. And there is no guidance on how to select the hyper parameters, for example, given a model and task, it is not clear which is appropriate. In practice, the method is basically doing subsampling at test time: first sample a lot of responses and then subsample a batch of response. I didn't see the advantage of the proposed method compared with simple submodular methods.
Questions for authors
No
We thank the reviewer for the comments. A revised PDF is uploaded, with changes in blue. We respectfully disagree with the assessment and respond to its four concerns below. In the tables, values in brackets are 95% confidence intervals and pp means percentage points.
Response to the comment that distortion does not capture the quality of responses.
Measuring distortion against the model's distribution lets the method preserve quality without committing to one quality score: at ρ = 0 every returned response has exactly the model's distribution (Proposition C.3), so the expected value of any quality measure of a response (accuracy, a reward, a judge's score) is the same as under ordinary sampling. A positive ρ is a budget that the user may spend for more diversity. Every method that samples from a fixed model, including MMR and k-DPP, depends on that model's distribution.
The experiments at ρ = 0 are consistent with this. The table gives Release (N = 64) minus independent sampling for the eight metrics of Table 32.
Every difference is in the favorable direction or within 0.13 pp of zero. After Holm correction over all comparisons of that table, the differences in diversity and in constant-reward groups hold on both models, and those in pass@8 and majority@8 on Llama (Appendix E.7). Qwen already has 96–97% pass@8.
Response to the comment that the results are not guaranteed with finite data.
The guarantee on distortion is a finite-sample statement. For a feature map fixed in advance, D ≤ ρ holds for every pool size N (Proposition 3.1) and every response limit T (Proposition 3.2). On every finite pool, Release returns a selection that satisfies the pool-level constraint and approximates the most diverse one (problem (8)), exactly for class features (Appendix C.1) and with a certified gap below 10⁻⁶ in the embedding experiments (Appendix D.2). Only the comparison of this diversity with its supremum on the support of the model's features is asymptotic, as the paper says. In a check added in the revision (Appendix F.13; observed class distribution of each prompt as the population), the mean diversity of Release at ρ = 0 is within 0.88% (B = 8) and 1.1% (B = 16) of the mean supremum at N = 64 on Llama. The revision states "for every N" and "for every T" in both propositions.
Response to the comment that there is no guidance on how to select the hyperparameters.
ρ = 0 needs no selection and is the setting of the comparisons above. For more diversity, ρ is a budget with a direct meaning: how far the returned responses may move from the model's distribution, in units of the model's own spread. With class features, ρ = √(2ε(1 − 1/N)) guarantees that on every prompt Release moves at most a share ε of the probability mass between classes, so the accuracy of a returned response changes by at most ε relative to its pool. No such bound exists for the temperature or for the λ of MMR. The revision adds this rule (Section 3.1).
Response to the comment that the method is subsampling at test time and has no advantage over simple submodular methods.
Release, MMR, and k-DPP all select a batch from a pool, and the paper compares them on the same 64-response pools (Section 4.3). The contribution of the paper is what the selection does to the responses. MMR and k-DPP place no constraint on the distribution of the selected responses, and Release selects for diversity subject to a bound on distortion. The table shows the difference on Llama (Tables 3 and 38).
Release at ρ = 0 leaves the accuracy of the responses within 0.1 pp, whereas MMR shifts it by between −5.3 and +4.3 pp depending on λ and k-DPP by −2.1 and −2.8 pp. In majority@8, Release at ρ = 0 is within 0.6 pp of the best setting of MMR on each dataset, with no λ to choose. On Qwen, where pass@8 is 96–97%, MMR and k-DPP shift the accuracy by at most 0.6 pp.