acceptodds
Under review as a conference paper at ICLR 2027

Which Environments Should LLMs Learn From? Predicting RL Utility Across Scale

Abstract

Reinforcement learning with verifiable rewards can improve language-model reasoning. Yet equal compute can help, do little, or degrade a policy across mathematical prompt distributions. We ask whether measurements from the starting policy can predict a distribution's held-out utility before observing its RL outcomes. We run 32 controlled Qwen3 trajectories from 1.7B to 14B parameters in a 22 design that varies group-signal availability and the admission-cost profile. We evaluate 196 policies on a fixed, disjoint 1,000-prompt OpenR1 evaluation panel. Environment choice changes held-out pass@1 by up to 11.4 percentage points at matched generated-action-token budget. Reward variation alone does not explain this spread. We measure admitted advantage mass per generated token, , and its effective coverage across prompts, . A compact - utility model is fit through 8B and frozen before inspecting 14B outcomes. At 14B, it achieves 2.29-point curve RMSE, 77.8% pairwise rank accuracy, and 11/12 cell-budget means inside its empirical envelope. A four-scale leave-one-scale-out audit yields 2.09-point RMSE. Fixed-label and crossover rules remain competitive for allocation, while a frozen uniform-mixture test identifies the current boundary of composition transfer. Together, these results provide evidence that pre-RL measurements combined with earlier-scale outcomes can help predict how environment choice changes held-out improvement across model scale and compute.

Primary area:
reinforcement learning
Keywords:
RLVR, post-training, environment selection, data selection, compute allocation, language-model reasoning, GRPO, scaling laws, predictive modeling

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_q7mgOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether measurements of a starting policy can predict which prompt distribution deserves RL compute. It constructs four environments, fits training gains using the density and coverage of usable advantage signals, and tests prediction across model scales. Environment choice matters a lot, but the predictor does not beat simple selection rules and fails on a uniform mixture.

Critical strengths
  1. The question is useful. The authors match budgets by generated tokens, freeze predictions in advance, and report the 8B failure and competitive simple baselines openly.

  2. The environment effects are substantial: the best and worst 4B environments differ by 11.4 points at matched token budget. Figure 3's finding that updates can concentrate on a small fraction of prompts is also informative.

Critical weaknesses
  1. I don't see a practical gain in choosing environments. In Table 1, mean regret is 0.48, the same as always choosing HL and worse than H alone at 0.33. Simple rules also make the same choices at 14B. More concerning, the only candidate outside the four constructed environments—the uniform mixture—is predicted to lose utility and ranked last at all three budgets. It actually gains at all three and beats every individual environment at the largest budget (Table 4C). This matters more for selection than reducing curve RMSE from 2.40 to 2.09. Measurement is also costly: Stage D alone uses 30% of the tokens of the eight 14B RL runs, and regret does not charge this cost. The results show that environment choice matters, but do not yet show that this predictor helps. Independent candidate tests or a narrower framing are needed.

  2. The explanation of usable signal needs more work. In the frozen 14B model, rho has coefficient -0.077 (Appendix D): at fixed H, more signal density predicts less gain. H is almost identical to the fraction of groups with mixed outcomes, with correlation 0.999. What does the extra coordinate explain? The setup truncates 38%–75% of responses, disables dynamic sampling, and couples response length with truncation, so their effects cannot be separated. Please explain the negative coefficient and whether the pattern holds with less truncation before presenting this as a general signal-survival principle.

Questions for authors
  1. On the frozen 14B test, how do budget/scale-only, H-only, and cost/admission predictors compare in RMSE and ranking? Does rho–H still help once measurement costs are included?

  2. What are the uniform mixture's rho and H? Are they outside the fitting range, and why is the prediction reversed? Please include uniform mixing as a simple selection baseline.

  3. How do you interpret the negative rho coefficient? Is there an analysis controlling for response length, or evidence under a higher length cap, that separates length from truncation?

Also, please clarify whether the main conclusions use token- or step-matched budgets; Table 2 and Figure 2 give different orderings. The drop in pass@4 for every 14B arm also needs to qualify what “utility” means here.

3 comments
AuthorsOpenReview ↗

Author rebuttal to Reviewer q7mg - Part 1/2

We thank the reviewer for separating the controlled effect, curve prediction, allocation, and composition transfer. The result is substantially larger than a 0.31-point regression improvement.

Under one warm start and trainer, environment choice produces an 11.4-point token-matched utility spread at 4B. The four effects replicate on Polaris with correlation 0.991. The untouched 14B curves are predicted with 2.292-point RMSE and 0.288-point mean regret. Finite-budget RL returns therefore depend materially on the environment and can be forecast within the controlled family. The uniform-mixture result identifies the boundary beyond it.

W1/Q2 summary: The core predictor improves curve RMSE only modestly over budget/scale and does no better than simple rules on allocation regret.

W1/Q2 response: Revised Section 7.2 fits every requested baseline on the same 1.7B, 4B, and 8B rows and scores 14B without refitting. These are post-hoc comparators on the original prospective fold, not additional prospective forecasts.

Model14B RMSERank accuracyMean regret
ρ\rho-HH2.29277.8%0.288
Cost/admission2.14077.8%0.288
HH only2.55061.1%0.288
Budget/scale only2.47750.0%1.358

The direct conclusion is that environment information improves prospective ranking over a model that knows only scale and budget. Under the fixed tie-break needed for the environment-agnostic model, mean regret falls by 1.07 points. Cost/admission has slightly better RMSE than ρ\rho-HH and equal rank/regret, so the evidence supports environment-aware prediction rather than the unique superiority of one representation.

The task is not reducible to retaining the four labels. In a leave-one-environment-type-out audit, each fold removes one complete HH, HL, LH, or LL type at all four scales. ρ\rho-HH obtains 2.45-point RMSE and 0.48-point regret, compared with 2.73 and 4.20 for budget/scale only. Cost/admission obtains 2.58 and 0.48. This remains a within-family test, but the held-out label is absent from fitting.

W1/Q3 summary: The only out-of-design candidate reverses the predicted ordering. Testing the claim requires independently constructed candidates and simple baselines.

W1/Q3 response: Revised Section 7.3 reports the 14B uniform-mixture miss and protected candidate test. The mixture has ρ=13.600\rho=13.600 and H=0.418H=0.418, within the 16-condition ranges and inside their convex hull in (log⁡ρ,H)(\log\rho,H). The miss is therefore not simple range extrapolation. It identifies a composition-transfer boundary. The revised title, How Training Environments Shape RL Utility Across Model Scale, makes this scope explicit.

Before review, we also froze a protected candidate-selection experiment with two identity-disjoint uniform prompt banks and one nonuniform mixture, all disjoint from the evaluation panel. Candidate identities, two seeds per candidate, forecasts, and scoring rules were fixed before any RL outcome. All six runs and 50 evaluation shards passed integrity, covering 25 policies and 25,000 responses.

The selected cost/admission model predicts uniform B>uniform A>nonuniform\text{uniform B}>\text{uniform A}>\text{nonuniform}. It selects the observed best candidate at the first two of four budgets, then misses crossovers to nonuniform and uniform A. Mean regret is 0.740 points, compared with 1.042 for budget/scale only, 1.250 for composition-only rules, and 1.760 for raw mixedness or starting success. Curve RMSE is 1.131 versus 1.045 for budget/scale only, and both reach 50.0% rank accuracy. Thus the measurements reduce decision regret on independently constructed candidates but do not improve full-curve calibration or recover later crossovers. This extends the evidence beyond the four named factorial environments while preserving a clear composition-transfer boundary.

W1 summary: Measurement cost is not included in the submitted allocation regret.

W1 response: Submitted regret measures RL tokens and does not include probe cost. Appendix E now reports it separately. A 64-prompt-per-environment probe uses one quarter of the submitted measurement and preserves all three frozen 14B decisions in 99.54% of 5,000 resamples, with 0.950 mean pairwise-order agreement. At 128 prompts, decision preservation is 100%, pairwise agreement is 0.980, and median curve-deviation RMSE is 0.311 points. This is retrospective sensitivity, not independent validation.

Continued in Part 2/2.

AuthorsOpenReview ↗

Author rebuttal to Reviewer q7mg - Part 2/2

W2/Q1 summary: The negative ρ\rho coefficient conflicts with the paper's monotone interpretation of usable signal.

W2/Q1 response: Revised Section 3.2 calls ρ\rho admitted advantage density, and Section 6 separates measurement from causal interpretation. It is admitted leave-one-out coefficient mass per generated token, not a claim that increasing it must improve utility.

The negative fitted coefficient is conditional on HH, scale, and budget in a small correlated design. Admission, response cost, mixedness, ρ\rho, and HH are not independently varied. The current design therefore does not isolate truncation masking, longer reasoning, or admission as the cause of the low-admission endpoint wins. The revision removes language implying that separate causal result. The supported mechanism statement remains consequential: reward variation, admitted coefficient mass, and its concentration are distinct under the fixed loss contract, while raw reward variation alone does not order held-out utility.

The predictive comparison remains strong. Removing environment information lowers same-fold prospective 14B rank accuracy from 77.8% to 50.0% and raises regret from 0.288 to 1.358 points under the fixed tie-break.

W2/Q4 summary: The truncation-heavy contract, disabled dynamic sampling, and small warm start may determine the observed ordering.

W2/Q4 response: Revised Section 4 and Appendix A specify the 12,288-token cap and overlong filtering as part of the controlled contract. NeMo treats overlong filtering and dynamic sampling separately: the former masks capped responses, while the latter replaces zero-variance groups. We disabled dynamic sampling because replacement would change the realized environment distribution. This does not imply invariance to DAPO-style replacement or a higher cap.

The existing trajectories provide a stronger diagnostic. Cumulative truncation is not uniformly increasing during RL. From step 16 to 128 it falls from 65.5% to 53.2% in 1.7B HL, 58.3% to 40.4% in 4B HL, 54.8% to 48.0% in 8B HL, and 57.4% to 35.7% in 14B HL.

The degraded 1.7B LH seed is different. Degeneration begins around steps 49–56. At step 56, mean output length is 11,310 tokens, 87.5% of responses are capped, and only 4/32 responses are loss-admissible. By step 72 its reward is zero under sustained cap pressure. We found no coincident KL, importance-ratio, checkpoint, tokenizer, or infrastructure anomaly. This supports an association between cap pressure and one learned failure mode, but not a separate causal effect of truncation.

The 2,048-example SFT warm start is fixed across all environments, so its size cannot explain the within-policy treatment spread. It does define the policy-relative measurements, however. A different warm start could change which prompts are mixed, admitted, or costly, and the revised scope statement now says so.

Q5 summary: Are comparisons aligned by actual generated tokens, and does the result extend beyond pass@1?

Q5 response: Sections 3.1 and 5 make generated-action-token matching primary. Step-128 values are descriptive because environments consume different tokens per step. The 11.4-point spread and Equation 7 use generated-token budgets.

Pass@1 is primary. Revised Section 7.2 reports the independent native-G=4G=4 audit over 36,000 responses. Its per-response pass@1 agrees with the G=1G=1 surface, r=0.956r=0.956, and HL again leads at 14B. Native pass@4 remains below the warm start for every environment. The benefit is improved single-rollout reliability, not expanded four-sample coverage.

The protected candidate test directly addresses a central request for independently constructed environments. Candidate identities, forecasts, and scoring rules were fixed before any RL outcome. The selected model lowers allocation regret relative to the environment-agnostic and simple feature baselines, while the later crossover defines the current claim boundary. In light of this direct test and the revised analysis, we would appreciate the reviewer's reassessment of the paper and overall score.

Reviewer_q7mgOpenReview ↗

Reply to author's rebuttal

Thank you for the clarification. After considering your response, I believe the manuscript still requires substantial revision. I will therefore maintain my original score.

Reviewer_PC2JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether measurements collected before RL can predict the benefits of training on different prompt distributions under a fixed generation budget. The authors construct four mathematical training environments at four Qwen3 model scales. They introduce usable signal density ρ\rho and effective coverage HH, then fit a utility predictor using RL results from smaller models. The experiments show that environment choice substantially affects held-out accuracy. The revised predictor achieves some predictive success on 14B, whose RL outcomes were excluded from fitting.

Critical strengths
  1. Practical research problem. Selecting training prompts directly affects RL performance under a fixed budget. The paper holds the training procedure fixed and compares environments at matched generated-token budgets. At 4B, the best and worst environments differ by approximately 11.4 percentage points in held-out utility, providing concrete evidence that environment selection matters.

  2. Prospective evaluation. The authors freeze predictions before evaluating transfer to a new model scale and retain the failed initial 8B forecast. This design helps distinguish retrospective fitting from prediction of unseen training outcomes. The experiments also evaluate curve prediction and environment selection separately.

Critical weaknesses
  1. Poor readability. The order in which key concepts are introduced makes the paper difficult to follow.

    The introduction uses HH, HL, LH, and LL, but the full textual definitions of the two letters appear only in Section 4. Figure 1 labels the axes as high or low, but the second axis combines admission and cost, which vary in opposite directions. This makes the labels difficult to interpret on first reading.

    Section 3.3 introduces the utility predictor before explaining the experimental construction. The main text briefly describes its terms, while variable transformations and fitting details are distributed across the appendix. Readers must repeatedly consult different sections to understand how the environments are constructed, how the measurements are obtained, and how the predictor is fitted. I suggest presenting the experimental workflow before the prediction model and adding a concrete example connecting measurements, predictions, and observed utility.

  2. The predictor's benefit for training decisions is not sufficiently established.

    In Table 1, the joint ρ\rho and HH model achieves lower curve prediction error, but its mean allocation regret is 0.48 percentage points. This matches the fixed-HL rule. The crossover rule and the HH-only model achieve 0.36 and 0.33 percentage points, respectively.

    These results support more accurate estimation of utility curves. The paper should explain when this accuracy translates into better training decisions. For practitioners, the central questions are whether the predictor improves budget allocation and whether it reduces the large-model experiments needed to identify suitable training environments.

Questions for authors
  1. How far can a fixed predictor extrapolate across model scales?

    The two prospective tests use 1.7B and 4B results to predict 8B, then use 1.7B, 4B, and 8B results to predict 14B. The predictor is also revised before the second test. This makes it difficult to assess the extrapolation range of a fixed predictor.

    If the predictor is fitted using only 1.7B and 4B results, how do prediction error and allocation regret change when forecasting 8B, 14B, and larger models without refitting? Can the authors identify the scale range over which prediction remains reliable? This would clarify how much the method can reduce expensive large-model experimentation.

  2. Which training decision benefits most from utility-curve prediction?

    For selecting the best environment, simple rules already achieve comparable or lower allocation regret. Where should the joint predictor provide its main advantage: selecting environments, choosing training duration, or allocating a budget across multiple environments?

    Could the authors identify a decision setting that demonstrates this advantage and compare final held-out performance against simple strategies under a fixed total budget? This would connect prediction accuracy to practical training benefits.

1 comment
AuthorsOpenReview ↗

Author rebuttal to Reviewer PC2J

We thank the reviewer for separating curve prediction from allocation. We completed the requested fixed-extrapolation and same-fold baseline analyses.

W1 summary: The order of presentation makes it difficult to reconstruct how environments are built, measured, and mapped to utility.

W1 response: Revised Figure 1 and Sections 3 and 4 define HH/HL/LH/LL before use, explain the coupled admission/cost axis, and present construction before the response surface. A worked example follows one prompt group from rewards and admission masks to ρ\rho, HH, and predicted utility.

W2 summary: The predictor's benefit for training decisions is not sufficiently established because simple rules obtain comparable or lower allocation regret.

W2 response: Revised Section 7.2 evaluates budget-conditional improvement against every requested baseline on the same prospective fold.

We fit the requested baselines on exactly the same 1.7B, 4B, and 8B rows and scored the untouched 14B outcomes without refitting. These are post-hoc comparators on the original prospective fold, not additional prospective forecasts.

Predictor14B RMSERank accuracyMean regret
ρ\rho-HH2.29277.8%0.288
Cost/admission2.14077.8%0.288
HH only2.55061.1%0.288
Budget/scale only2.47750.0%1.358

Environment-aware models improve prospective rank accuracy from 50.0% to 77.8%. Under the fixed tie-break required for a model that assigns equal scores to all environments, regret falls from 1.358 to 0.288 points. Cost/admission has slightly lower RMSE than ρ\rho-HH, while both obtain the same rank accuracy and regret. The revision reports this comparison directly rather than implying that one environment representation dominates every metric.

The most directly supported decision is whether and how long to train a measured candidate under a fixed token budget. A fixed selector cannot indicate whether its expected margin is meaningful, whether further training is worthwhile, or whether uncertainty exceeds that margin. Fixed HL and the crossover rule remain competitive on mean regret, but they do not provide calibrated utility curves. Selection among arbitrary datasets is the direct next test addressed in Q2.

Q1 summary: If the predictor is fit using only 1.7B and 4B, how do error and regret change at 8B and 14B without refitting?

Q1 response: We applied the original 1.7B+4B cost/admission model unchanged at both larger scales:

Test scaleRMSERank accuracyMean regret
8B3.06372.2%0.632
14B3.87577.8%0.288

The 8B row is the original prospective test. The unchanged 14B application is a post-hoc extrapolation diagnostic, not a second prospective claim. Decisions remain low-regret, but absolute calibration worsens. Adding the intermediate 8B outcomes lowers 14B RMSE from 3.875 to 2.292. The evidence supports sequential recalibration when an intermediate scale is available. It does not establish a reliable range beyond 14B.

The complete machine-readable audit is included in the anonymous supplementary material.

Q2 summary: Which decision benefits most from curve prediction, and can it improve selection among candidate environments under a fixed total budget?

Q2 response: Revised Section 7.3 and Appendix C report a three-candidate experiment frozen before review. It contains two identity-disjoint uniform prompt banks and one nonuniform mixture, all disjoint from evaluation. Candidate identities, two seeds, numerical forecasts, and scoring rules were fixed before any RL outcome.

All six runs and 50 evaluation shards passed integrity, covering 25 policies and 25,000 responses. The selected cost/admission model predicted uniform B>uniform A>nonuniform\text{uniform B}>\text{uniform A}>\text{nonuniform} at every budget. It selected the observed best candidate at the first two of four budgets, then missed crossovers to nonuniform and uniform A. Its mean regret is 0.740 points, compared with 1.042 for budget/scale only, 1.250 for composition-only rules, and 1.760 for raw mixedness or starting success. Curve RMSE is 1.131 versus 1.045 for budget/scale only, and both have 50.0% pairwise rank accuracy.

The protected result therefore gives a precise answer: pre-RL measurements reduce allocation regret by 0.302 points on independently constructed candidates, but they do not yet improve full-curve calibration or recover later budget-dependent crossovers. This is decision value beyond the four named environments, not a universal allocator.

In light of the same-fold baselines, unchanged-model extrapolation, and protected candidate test, we would appreciate the reviewer's reassessment of the quantitative concerns and overall score.

Reviewer_Dtb3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates whether measurements collected from a starting policy can predict the held-out benefits of reinforcement learning on different mathematical prompt distributions. It summarizes each distribution using usable advantage density ρ\rho and effective prompt coverage HH, and fits a five-coefficient utility model using these statistics, model scale, and generated-token budget.

Critical strengths

The paper investigates a practically relevant problem: how to predict the effectiveness of different training environments for LLM reinforcement learning under a fixed compute budget.

Critical weaknesses
  1. The methodological contribution is limited, and the presentation makes it difficult to assess. The core prediction method combines two summary statistics with a five-coefficient linear regression over the specified scale and budget features. The paper needs to establish more clearly what these measurements contribute beyond existing statistics and simple predictors. In particular, for nonzero admitted advantage mass, the coverage measure in Equation 6 is

    {n_E\sum_g\left(A_g^{\mathrm{adm}}\right)^2},$$ which is exactly Jain's fairness index applied to the admitted advantage masses. This established connection is not acknowledged or cited. Moreover, Section 6 reports a correlation of 0.999 between $H$ and raw mixedness across the 16 evaluated cells, leaving its additional informational value unclear in this experimental design. The writing compounds this concern. Extensive abstract terminology, insufficient intuitive explanation, and weak positioning against prior work obscure a relatively simple pipeline: probe the starting policy, compute summary statistics, and fit a low-dimensional predictor. I found it unnecessarily difficult to distinguish the empirical contribution from the terminology and statistical definitions used to describe it.
  2. Limited seed replication and inconsistent reporting undermine confidence in the environment rankings. Each scale-by-environment condition uses only two training seeds, despite substantial variability. For example, Section 7.2 reports a 13.9 gap between the two 14B HH runs.

    There is also a concrete inconsistency in the reported seed-level decisions. Section 7.2 states that the seeds agree on the winning environment at 24M generated tokens. However, Table 4B lists HL as the winner for Seed 1 and LL for Seed 2 at that budget. The table therefore contradicts the text on the budget at which seed-level agreement is claimed.

Questions for authors

N/A

2 comments
AuthorsOpenReview ↗

Author rebuttal to Reviewer Dtb3

We thank the reviewer for the close reading. The revision makes the experimental contribution easier to assess, adds a stricter held-out-environment audit, and resolves the attribution and seed-reporting points below.

W1 summary: The paper needs to establish what these measurements contribute beyond existing statistics and simple predictors. Equation 6 is Jain's fairness index, and HH correlates at 0.999 with mixedness.

W1 response: For positive admitted mass, Equation 6 is Jain's fairness index applied to prompt-level admitted advantage masses. Revised Section 3.2 cites Jain, Chiu, and Hawe (1984) and identifies the equation directly. We do not claim a new concentration formula. Our use of the statistic is to measure how broadly a fixed GRPO loss contract distributes admitted group-relative coefficient mass across prompt groups.

In this deliberately extreme 2×22\times2 grid, corr(H,qmixed)=0.999\mathrm{corr}(H,q_{\mathrm{mixed}})=0.999. These 16 conditions therefore do not identify an incremental causal effect of HH over mixedness, and the revision removes language suggesting otherwise. Mixedness records reward variation before admission, while HH measures the concentration of admitted leave-one-out advantage mass across prompts. Figure 3 shows how admission and token weighting redistribute nominal signal. It does not establish an independent causal effect of HH.

Revised Section 7.3 reports a stricter leave-one-environment-type-out audit. Each fold removes one complete HH, HL, LH, or LL type at all four scales. The ρ\rho-HH model obtains 2.45-point RMSE and 0.48-point mean regret, compared with 2.73 and 4.20 for budget/scale only. Cost/admission obtains 2.58 and 0.48. This shows that continuous environment information predicts an environment type absent from fitting, although it remains a within-family test and does not make HH uniquely necessary.

This correction narrows the representation claim, not the controlled result. Environment choice changes held-out utility by 11.4 points at equal generated-token budget, and raw mixedness alone does not order the observed outcomes.

W2 summary: Abstract terminology and the order of presentation make a relatively simple pipeline difficult to follow.

W2 response: Revised Figure 1 and Sections 3 and 4 organize the main text around the actual workflow:

starting policy→fixed probe→four environments→(ρ,H)→utility curve→prospective test.\text{starting policy} \rightarrow \text{fixed probe} \rightarrow \text{four environments} \rightarrow (\rho,H) \rightarrow \text{utility curve} \rightarrow \text{prospective test}.

It defines HH/HL/LH/LL at first use, states immediately that the second letter denotes a coupled admission/cost profile, presents environment construction before the predictor, and adds one worked prompt-group example. We also replace abstract terminology where a direct description is clearer.

The five-coefficient model is intentionally small. With 16 scale-by-environment experimental units, a larger surface would be weakly identified. The contribution is the controlled intervention, generated-token estimand, policy-relative measurement pipeline, and two temporally separated prospective tests, not a claim of a novel regression method.

This also differs from online variance filtering or curriculum methods. Those methods change which prompts are sampled during training. We hold the trainer and environment fixed, measure the starting policy, and predict the held-out return to a generated-token budget across model scale. The prospective 8B miss and untouched 14B test evaluate that prediction rather than only showing that a selector can improve its own training reward.

W3 summary: Two seeds provide limited evidence under substantial variability, and the text's 24M seed-agreement claim conflicts with Table 4B.

W3 response: Two seeds do not estimate a detailed variance model. We retain both trajectories and resample whole seed trajectories rather than treating checkpoints as independent. The 13.9-point 14B HH gap is why we report seed-level outcomes, curve envelopes, and regret rather than only a mean winner.

The reviewer identified an inconsistency in one sentence. Table 4B is correct: at 24M tokens, Seed 1 selects HL and Seed 2 selects LL. The seed-level argmax differs at all three displayed budgets. Revised Section 7.2 states this directly beside the prospective result. This correction does not change the mean surface, 2.292-point RMSE, 77.8% pairwise rank accuracy, 11/12 envelope coverage, or 0.288-point mean regret.

The revised presentation makes the simple pipeline and the scope of each claim substantially clearer. If the attribution, corrected seed statement, stricter held-out-environment audit, and revised organization address the reviewer's main concerns, we would appreciate reconsideration of the current assessment.

Thanks to the authors for their clarifications. However, I believe the paper still falls short of ICLR's standards, so I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.