acceptodds
Under review as a conference paper at ICLR 2027

Selection Is Not Validation: Auditing Proxy-to-Behavior Reversals in Language-Model Data Curation

Abstract

Data-selection methods for language-model training are often validated with the same proxy used to construct the training set. This creates a circular evaluation shortcut: a selector can improve its target score without improving the trained model. We introduce a staged claim-advancement audit that treats selection scores as hypotheses rather than outcomes. It specifies random and credible simple-baseline comparisons, audits training parity, and organizes source-clustered uncertainty, paired reference-loss and generation analyses, retention checks, and mechanism diagnostics, while marking missing evidence explicitly. Unlike a pass/fail checklist, the audit maps each result to the strongest claim supported by its evidence and identifies the next experiment required to advance that claim. In linked discovery cases in mathematical pretraining and instruction tuning, stored evaluation records reveal source-dependent uncertainty, aggregation-dependent likelihood estimates, and disagreements between reference likelihood and decoded reference overlap on shared example IDs. These paired analyses and a single-annotator, conflict-enriched 20-pair calibration provide bounded diagnostic evidence; useful fitted-model results are retained even when broader claims remain unresolved. The resulting evidence map turns proxy optimization into an actionable validation program: reject unsupported downstream claims, qualify surface-specific gains, or advance only those effects that survive independent behavioral and mechanism tests.

Primary area:
datasets and benchmarks
Keywords:
data selection, proxy validation, language-model evaluation, uncertainty, behavioral auditing

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_DJgrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This article explores to what extent the data-selection results support claims about trained models. Its eight-stage audit treats selection scores as hypotheses, then checks for training parity, uncertainty, reference loss, generated outputs, and possible mechanisms. In pre-training and instruction tuning, conclusions sometimes change depending on the resampling unit or the way NLL is averaged; reference likelihood and decoded reference overlap can also disagree.

Critical strengths

The authors focus on an important problem and the results are interpreted cautiously.

Critical weaknesses

(1) It is unclear what the eight-stage procedure adds beyond rigorous standard evaluation. Random baselines, matched budgets, multiple seeds, clustered uncertainty, and behavioral testing seem to be familiar methods. It is useful to organize them into a claim-evidence ledger, and Table 5 shows how the audit limits claims in these cases. The case studies illustrate the rules, but do not show whether the framework yields more reliable or actionable decisions than careful conventional evaluation. No prospective case reaches the later stages, and Section 6 notes that the impact of the audit on research practice has not yet been measured. Therefore, I believe the added methodological value of this framework has not yet been fully established.

(2) These cases demonstrate the audit process but only support narrow empirical claims. The three Pythia-70M contrasts and the Qwen2.5-0.5B instruction-tuning comparisons each represent one training seed (Section 3; Tables 2-3). The 998 SFT comparison rows cover only 498 distinct prompts, with some baselines shared across comparisons. Therefore, the bootstrap intervals quantify uncertainty over evaluation examples, rather than across new selection or training runs. Training exposure also does not fully match. P3 uses about 1.32% fewer non-padding tokens than its parent and changes filtering and frequency capping; I would consider its worse loss a result of the combined recipe, not just frequency capping. More importantly, S1 reports 24,452 supervised completion tokens for random selection and 39,699 for MMR (Section 3.1; Appendix B). These are not total training-token counts, but matching the number of selected examples clearly does not match the supervised completion-token budget. This complicates attributing the S1 difference solely to the selector.

S1 is an important example, but its interpretation remains limited. Token-weighted reference NLL improves, while ROUGE-L declines; the prompt-macro NLL result remains unresolved (see Table 3). NLL uses Qwen's chat template, while generation uses a literal Instruction/Input/Response format (Section 4.2), so matching example IDs does not give the model matching contexts. The target-test text and references are also used for duplicate-risk control during selection (Section 3.1). This is not necessarily problematic and is not evidence of outcome-based tuning. However, the paper should clarify whether this information was used exclusively for decontamination or also influenced other selection, model, or hyperparameter decisions. ROUGE-L measures reference overlap rather than overall answer quality; human checks cover only 20 deliberately selected pairs judged by one annotator (Section 4.4). Under the saved decoding protocol, MMR also increases the empty outputs from 12/200 to 22/200. This is a concrete behavioral diagnostic, and its robustness under matched templates has not been tested. Earlier scale, placebo, and cross-family numerical claims are associated with a synthetic data generator, and are explicitly excluded from empirical support. The earlier reported training and evaluation runs remain unverified.

The authors acknowledge these limitations and do not claim a general quality reversal. I believe these cases are useful diagnostics, but they are not yet independent, matched-protocol behavioral validation or mechanism tests for the broader framework.

(3) The manuscript contains statements that read as unresolved notes to the authors rather than finished prose. Line 717 states, “Before publication the authors must confirm the annotator’s age category, relationship to the authors, recruitment and consent procedure, and applicable institutional review or exemption determination.” Line 583 states, “The author reports that the annotator was an unpaid linguistics-Olympiad candidate; age, authorship relation, informed consent, and ethics review or exemption remain author-confirmation items. No claim of clearance is made.”

Questions for authors

Please see the weaknesses.

Reviewer_iES9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
  • Selection score, reference NLL and generated answers are different claims. The paper proposes an eight-stage audit for which claim the evidence supports. Evidence: single-seed Pythia-70M and Qwen2.5-0.5B runs, plus 20 pairs from one annotator.
Critical strengths
  • Table 5 clearly separates what is supported and what is not, and the paper is honest about its missing evidence. But this does not make up for the weaknesses.
Critical weaknesses
  • Not human-readable. Uncommon terms are used without definition, or long before it: "pass/fail checklist" of what? stored evaluation "rows"? "claim-advancement audit"? "source-clustered" uncertainty? Results appear before their setups, and most Ch. 2.1 notation is never used again. I had to ask AI sentence by sentence to get the core message.
  • The conclusion is weak. "Validated with the same proxy used to construct the training set" → any real examples? It seems to be a trivial mistake. After all the decoding, the message is just "a proxy is not the outcome," from single-seed 70M/0.5B runs. The pretraining gain is only about 0.05%.
Questions for authors
  • Any real example of a selector validated with the same proxy used to build its training set?
Reviewer_rk4oOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Highly Suspect as a Low-Quality AI-Generated Paper.

I could not understand what this paper is about. It seems almost entirely AI-generated, with very little coherence, undefined terms, and puzzling claims and word choices. For example, it includes text that appears to come from an AI addressing the authors directly: "These disclosures require author resolution..." (Ethics Statement, line 369).

Critical strengths

Highly Suspect as a Low-Quality AI-Generated Paper.

I could not understand what this paper is about. It seems almost entirely AI-generated, with very little coherence, undefined terms, and puzzling claims and word choices. For example, it includes text that appears to come from an AI addressing the authors directly: "These disclosures require author resolution..." (Ethics Statement, line 369).

Critical weaknesses

Highly Suspect as a Low-Quality AI-Generated Paper.

I could not understand what this paper is about. It seems almost entirely AI-generated, with very little coherence, undefined terms, and puzzling claims and word choices. For example, it includes text that appears to come from an AI addressing the authors directly: "These disclosures require author resolution..." (Ethics Statement, line 369).

Questions for authors

Highly Suspect as a Low-Quality AI-Generated Paper.

I could not understand what this paper is about. It seems almost entirely AI-generated, with very little coherence, undefined terms, and puzzling claims and word choices. Did a human verify all text written in this submission?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.