- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces a method for measuring the amount of local adaptation an environment benefits from. It does so through a counterfactual measure, where an agent capable of local adaptations is rolled out on an environment, then compared against the same agent being rolled out but with its local adaptation mechanism disabled. The authors show how pairing this metric with an incentive to keep it at the frontier of what is attainable leads to gains over other curriculum design approaches that prioritize regret or learning progress.
Critical strengths
- The idea of measuring adaptation through this intervention is intuitive and well motivated, and the authors did a good job of contrasting this objective from prior work.
- The comparison to the Post experiment shows that AdaptGain carries additional information relevant for the task and combining both can lead to improvements
Critical weaknesses
- Each round requires rolling out adaptive and static executions. However, the gains given this additional cost do not seem to be that meaningful, and there do not seem to be seeds or anything determining if these gains are even significant. Comparing all the baselines at the same amount of environment interactions would make it clear if the extra compute is worth the moderate gains.
- Parts of the evaluation are unclear. Freezing the initial state could push recurrent policies out of distribution, so AdaptGain could reflect errors there rather than meaningful adaptation. For agents like in-context LLMs, it’s also not clear how this approach will extend, since removing interaction history could remove information needed to track the environment state.
Questions for authors
- How many total environment interactions does PACE use compared to the baselines? Do the results change when all baselines have the same amount of environment interactions?
- For recurrent learners, does freezing the hidden state keep the policy within its training distribution?
- How does the static intervention work for in-context learners like LLM agents?
- Could you report seed counts, confidence intervals, and significance tests for all the results where applicable?
We thank the reviewers for their thoughtful and constructive feedback. Building on the original matched-Post experiment, we have added noise-stratified retraining analyses and intervention controls to further examine the training relevance of AdaptGain. In the matched-Post experiment, task groups are closely matched on initial Post and retrained from the same checkpoint under equal budgets. Across 12 learner–environment settings, the high-AdaptGain groups achieve an average held-out Post advantage of +0.00693 over the matched low-AdaptGain groups (Tables 2 and 9). The state-matched and native-suffix controls further show that the adaptation contrast persists under alternative evaluation protocols (Table 12).
We have also evaluated PACE under equal total interaction budgets, including the cost of teacher evaluation. PACE improves over the Post frontier by +0.0011 in the standard setting and +0.0012 under high drift, while the high-noise comparison shows no clear advantage (Table 6). The noise-stratified retraining analyses show smaller matched-group contrasts at higher noise levels, while the task-selection results show fewer selections from higher-noise strata in a balanced candidate pool (Tables 13, 18, and 19). These findings provide evidence for the training relevance of adaptation dependence while clarifying the modest gains and limitations of using this signal for task selection.