acceptodds
Under review as a conference paper at ICLR 2027

Aspire: Can Models Self-Evolve from Vague Goals?

Abstract

Many important forms of human learning begin with a vague goal, such as “become a better physicist” or “improve at research.” Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce Aspire, a benchmark for vague-goal-driven self-evolution. Aspire provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. Aspire supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation. Current agents routinely complete training and harness-editing loops, but weight-level gains remain sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference. Agents often train on mismatched data and trust narrow self-evaluations, so local gains fail to transfer to hidden evaluation and continued search and training can erase earlier improvements.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-evolution, vague goals, LLM agents, agent harness, benchmarks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qS48OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLM agents can translate broad capability goals into effective self-improvement without access to downstream evaluation task specifications. The proposed ASPIRE benchmark supports model-weight and agent-harness updates, with 520 hidden, expert-authored evaluation items across six goals. Experiments compare vague-goal and explicit-task settings, examine weight updates under final-only and adaptive-feedback protocols, and test harness evolution with fixed weights. Agents generally complete update workflows, but retained gains over the base model are rare, and successor harnesses remain below the reference in mean score. The main contributions are formalizing goal operationalization, introducing the benchmark, and analyzing the gap between executing updates and achieving capability gains.

Critical strengths
  1. Valuable problem formulation and benchmark. ASPIRE makes goal operationalization in autonomous learning measurable by connecting agent-directed data selection, updates, and self-evaluation with hidden evaluation, providing a useful setting for studying this challenge.
  2. Useful outcome accounting and preliminary failure evidence. The paper distinguishes local recovery, above-base improvement, and retained gains. Its trajectory analyses provide concrete, though preliminary, evidence of data-selection mismatch, narrow self-evaluation, and output-pipeline failures, helping interpret apparent progress and identify directions for further investigation.
Critical weaknesses
  1. Insufficient direct evaluation of goal operationalization. RQ2 mainly reports checkpoint scores and retained gains, without systematically presenting or assessing the proxy tasks, learning objectives, and agent-constructed validation criteria. The reported cases provide partial evidence, but insufficiently distinguish poor goal operationalization from downstream training failures. This limits insight into the paper’s central question of deciding what to learn and how to verify progress.
  2. Limited evidence of robustness. The adaptive-feedback protocol uses one run per configuration–goal cell, and RQ3 uses one harness-creation trajectory per condition. RQ2’s trainable models are also limited to two sizes from the same family. These choices leave the sensitivity of key findings to search randomness and model selection unclear.
Questions for authors

Could the authors systematically report the proxy tasks, learning objectives, and validation criteria selected or constructed in RQ2, including how often agents performed their own validation? How well do these artifacts align with the capability goals, and how do agent-validation scores relate to hidden-evaluation scores? This evidence would help distinguish goal-operationalization failures from downstream training failures.

Could the authors provide independent evolution repeats for a small set of key configurations, particularly the sole retained improvement in RQ2 and the strongest harness creator in RQ3, and report across-run variability? This would help assess whether the observed gains and gaps to the reference are stable.

Which specific completion requirement did the Qwen3.5-4B Self science run fail, and why did this require rejecting its otherwise verified, above-base checkpoint?

5 comments
AuthorsOpenReview ↗

Response to Reviewer qS48 (1/5)

We thank the reviewer for the comments on process-level evaluation and robustness. First, the definitions of the two RQ2 protocols used below:

  • Final-only: no hidden-set scores are returned during training, and one final model is submitted. Qwen3.5-4B Self and Qwen3.5-9B Self each run twice independently on each of six goals, for 2 × 6 × 2 = 24 runs. ("Self" means the frozen initial model acts as its own decision model.)
  • Adaptive-feedback: a limited number of aggregate hidden-set scores may guide further training. There are five configurations (4B/9B Self plus the external decision models GPT-5.6 Luna, Terra, and Sol), each run once on each of six goals. This gives 5 × 6 = 30 configuration–goal cells, i.e., 30 runs, and 54 runs across both protocols.

Adaptive outcomes are reported at three levels:

  • best evaluated: the highest-scoring intermediate model;
  • eligible: passes the predeclared completion and other requirements;
  • retained: eligible and above the initial base, so it is kept rather than rolled back.
AuthorsOpenReview ↗

Response to Reviewer qS48 (2/5)

W1 / Q1: Direct evaluation of goal operationalization; systematic reporting of proxy tasks, objectives, self-validation, and alignment

W1: Insufficient direct evaluation of goal operationalization. RQ2 mainly reports checkpoint scores and retained gains, without systematically presenting or assessing the proxy tasks, learning objectives, and agent-constructed validation criteria. …

Q1: Could the authors systematically report the proxy tasks, learning objectives, and validation criteria selected or constructed in RQ2, including how often agents performed their own validation? How well do these artifacts align with the capability goals, and how do agent-validation scores relate to hidden-evaluation scores? …

Why the hidden-set score is the primary measure. Under a vague goal, the final hidden-set score is the most direct end-to-end test of goal operationalization, i.e., of turning a broad goal into training data, learning objectives, and validation criteria. The agent cannot optimize against hidden items, so whether its chosen data and validation work is ultimately shown by whether the target capability improves (§2.2, p.3). We do not use intermediate artifacts as the primary metric, for three reasons:

  • No reference plan: the same broad goal can be decomposed in many valid ways, so there is no reference learning plan to grade against.
  • Incomparable validation: self-built validation varies across runs (checklists, held-out sets, scripts) and cannot be placed on a common scale.
  • Subjective grading: human grading of alignment would introduce a subjective scale.

We therefore treat the hidden-set score as primary and use process statistics to diagnose failures.

Process statistics already in the paper. The appendix reports full-coverage extraction over 63,973 events, 124 training plans, and 95 evaluation attempts (Appendix C.5, p.17). Examples (Appendix C.4, p.17):

  • 4B Self (30/32 imports) and 9B Self (11/15 imports) concentrate on mathematics data;
  • SFT/GRPO ratios per configuration;
  • search profiles of Luna, Terra, and Sol.

New results from the discussion period. We systematically audited and back-tested all 54 RQ2 runs.

Audit dimensionScopeResultNotes and log limits
Rate of declared self-validationAll 54 runs (querying the hidden aggregate excluded)Final-only: 11 / 24 (45.8%); Adaptive: 5 / 30 (16.7%)A lower bound from controller logs: agents with external feedback build their own validation less often. A declaration does not imply successful validation.
Self-validation vs. hidden scoreAll 54 runsNo common-scale self-score seriesSelf-built validation metrics are heterogeneous, and the portable logs redact raw messages, so a pooled correlation is not meaningful. RQ3 cases show narrow self-evaluation diverging sharply from the hidden set (e.g., Luna scores 8/8 on its own checklist but 19.32 vs. the reference 28.64).
Blind labeling of proxy task vs. goalAll 54 runs, hidden scores maskedThree-way independent AI blind labeling, agreement 46 / 54 (85.2%)Off-domain mathematics imports and the 13 single-digit-label collapses (989 single-digit answers) are the main front-end operationalization errors.
Public-benchmark results when the data match the goal42 pairs of trained model vs. matching public test set (GSM8K / MATH-500 / MMLU / SciQ)38 / 42 below base; mathematics goal 18 / 18 below baseEven on the mathematics goal, where the agent chose matching GSM8K/MATH data, none of the 9 models exceeds base on the public test sets.

These numbers separate the two failure types:

  • Front end: more than half of the final-only runs and more than 80% of the adaptive runs never even declare self-validation, and off-domain data choices and single-digit label mismatches are frequent.
  • Back end: even with matching mathematics data, continued training quickly collapses the model without a reliable intermediate self-check.
AuthorsOpenReview ↗

Response to Reviewer qS48 (3/5)

W2 & Q2: Robustness; independent repeats of key configurations

W2: Limited evidence of robustness. The adaptive-feedback protocol uses one run per configuration–goal cell, and RQ3 uses one harness-creation trajectory per condition. RQ2's trainable models are also limited to two sizes from the same family. …

Q2: Could the authors provide independent evolution repeats for a small set of key configurations, particularly the sole retained improvement in RQ2 and the strongest harness creator in RQ3, and report across-run variability? …

What the paper already reports.

  • The final-only protocol runs each model–goal pair twice and reports the mean, Avg@2 (§3.3 p.6; Table 4 p.17).
  • The overall negative finding is consistent across both protocols, five decision models, and 54 runs (Figure 3 p.6; Figure 4 p.7).
  • The paper states that the adaptive protocol uses one run per cell (§3.3 pp.6–7; Appendix C.6 p.18; §5 p.9).
  • It also states that 4B vs. 9B Self changes the decision model, trainable base, and initial score together, so it is not a pure scale effect.

Our conclusions concern the Qwen3.5 family of trainable models and are not directly extrapolated to other model families.

New stability results. For the configurations the reviewer singled out, the RQ2 positive gains (especially the sole retained Terra 4B mathematics gain) and the RQ3 harnesses (especially Sol's H1, the closest to the reference), we added repeated re-evaluation, paired item-level bootstrap (10,000 resamples), and independent repeated creation of the Sol harness.

1. RQ3: standard deviation over three executions and paired item-level bootstrap for all frozen harnesses.

HarnessTask macro (mean ± s.d.)Example micro (mean ± s.d.)Diff. vs. H0 [95% CI]Conclusion
Qwen-Agent reference (H0)28.64 ± 0.6927.65 ± 0.57—Reference is highly stable (s.d. < 0.7)
Luna H119.32 ± 0.9818.33 ± 1.11−9.32 [−16.28, −3.07]Significantly below reference
Terra H120.76 ± 0.7420.14 ± 0.88−7.88 [−15.59, −1.82]Significantly below reference
Sol H127.22 ± 2.2325.97 ± 2.58−1.42 [−5.04, +1.46]No significant difference (Sol reverted to the H0 policy)

Standard deviations across executions are generally small (0.69 for H0, 0.74–0.98 for Luna and Terra), enough to clearly detect the degradation of Luna and Terra. Sol does not differ significantly from H0 (its CI includes 0) because, as Appendix D notes, Sol restored the original Qwen-Agent policy after rejecting complex designs and added only a non-empty check that never fired. The two harnesses share the same runtime logic.

Reviewer_b5XrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ASPIRE introduces a benchmark for automated improvement from broad goals, where agents select their own data, update methods, and validation signals and are assessed on hidden tests. The paper studies changes in goal specification, model-weight updates, and agent-harness modifications. Agents frequently complete the update procedures, but above-base weight improvements are rare, and the strongest generated harness remains numerically below the fixed Qwen-Agent reference. All reported runs keep the decision model fixed, and harness evaluation covers only one-step generation for writing tasks. The contribution is primarily an evaluation protocol and descriptive failure analysis, the extent to which goal vagueness itself accounts for these failures remains unresolved by the current comparisons.

Critical strengths
  1. The failure records provide useful reference material. Results across multiple goals and concrete trajectory examples offer useful observations, and the authors explicitly disclose several important limitations.

  2. The experimental protocols are clearly described. Explicit information boundaries, candidate-selection rules, and rollback procedures help readers interpret the results.

Critical weaknesses
  1. The distinct evaluation contribution of goal operationalization is not sufficiently established. RQ1 uses unmatched historical performance references, while RQ2/RQ3 include familiar failures such as answer-format collapse and incomplete outputs. The experiments therefore do not adequately establish what distinct difficulty the setting reveals beyond conventional automated post-training failures.

  2. I think the alignment between broad goals and hidden evaluation is insufficiently validated. Expert authorship and difficulty screening support item quality, but do not establish adequate coverage of the intended goal. Reasonable agent-selected subgoals may be underrepresented, making a lack of score improvement difficult to interpret as failed goal operationalization.

Questions for authors

1&2 See weaknesses

  1. Could the authors quantify losses attributable to format and execution errors and test whether the main findings persist under minimal repair controls? Five numeric-label SFT checkpoints produce single-digit outputs, while two incomplete answers account for approximately 70% of the decline in one inspected Terra execution.

  2. Do the few positive results and harness differences exceed evaluation and search variability? Please provide uncertainty estimates, independent confirmation of selected checkpoints, and independent repetitions of harness creation.

6 comments
AuthorsOpenReview ↗

Response to Reviewer b5Xr (1/4)

We thank the reviewer for the concrete suggestions. Two notations recur below:

  • RQ2 has two weight-training protocols:
    • Final-only: no hidden-set scores during training and one final model is submitted. Qwen3.5-4B/9B Self × 6 goals × 2 runs gives 24 runs.
    • Adaptive-feedback: a limited number of aggregate hidden-set scores may guide search. 4B/9B Self plus the external decision models GPT-5.6 Luna, Terra, and Sol make 5 configurations; × 6 goals gives 30 runs. A run counts as a retained gain only if it passes the completion requirements and exceeds the initial base model.
  • RQ3: Qwen3.5-4B weights are fixed. Each creator makes a single edit to the original Qwen-Agent reference H0, producing a successor H1, which is frozen and executed three times on the 20 hidden writing task bundles. We report task macro (mean over tasks) and example micro (mean over scored examples).

Below we answer point by point and report the new results.

W1 / Q1: Distinct contribution of goal operationalization

The distinct evaluation contribution of goal operationalization is not sufficiently established. RQ1 uses unmatched historical performance references, while RQ2/RQ3 include familiar failures such as answer-format collapse and incomplete outputs. …

RQ1 is aligned with PostTrainBench. RQ1 is the vague-goal version of PostTrainBench: apart from replacing the benchmark name in the prompt with a broad capability description, the core conditions match official PostTrainBench (§2.1 p.3, §3.2 p.5). These are the evaluator, training interface, tasks, base models, and 10-hour GPU budget.

  • Due to local cluster constraints, the host CPU model and memory size are not identical to the official machines. Both have ample headroom and are not a bottleneck, so we regard the change in goal specification as the main source of the performance difference.
  • For rigor, §3.2 notes that environment metadata may differ.
  • The search-process comparison uses 48 strictly matched pairs on the same local cluster, with the same decision model, base model, task, and trial index (Appendix B, p.15). It shows that vague goals shift time toward understanding the goal and shorten active training (§3.2 and Figure 2(b), p.5).

The "familiar failures" are symptoms of goal operationalization. The decision model cannot see the downstream task specification, so it must decide on its own what to train on and how to validate.

  • In the explicit-task setting, the prompt states the task format, metric, and success criterion (Appendix A.3, p.14), so the agent would not train a free-response task on single-digit classification labels.
  • Under a vague goal, the agent chose MMLU single-digit labels for SFT and did not check the output format, so its evaluation answers collapsed to single digits (§3.3 p.8; Appendix C.3 p.16).
  • RQ3 shows the same pattern (Appendix D, p.18):
    • Terra's H1 lacks a final-answer check, and its self-built validation did not catch this.
    • Luna stopped after testing one prompt with an 8-item checklist.

The failure modes look familiar, but their root cause is that the agent must choose its own training signal and validation method. This goal-operationalization ability is what distinguishes Aspire from conventional automated post-training benchmarks (§1, p.2).

AuthorsOpenReview ↗

Response to Reviewer b5Xr (2/4)

W2 / Q2: Alignment between broad goals and hidden evaluation

… Expert authorship and difficulty screening support item quality, but do not establish adequate coverage of the intended goal. Reasonable agent-selected subgoals may be underrepresented, making a lack of score improvement difficult to interpret as failed goal operationalization.

The coverage limits of the six goals and the item bank are stated in §5 (p.9) and Appendix E.2 (p.20). However, "subgoals not covered by the hidden set" cannot explain the observed results:

  1. Insufficient coverage would at most leave scores flat. It cannot explain sharp drops below base: 9B mathematics falls from 25.16 to 0.714 and 2.540, and 4B mathematics falls to 0.79 (Table 4 p.17; Figure 5(c) p.7).
  2. Many choices are simply off-domain, such as importing mathematics data under the science, logic, and writing goals (§3.3 p.8; Appendix C.4 p.17).
  3. On the mathematics goal, the agent's choice of GSM8K/MATH closely matches the hidden mathematics items, yet both 4B and 9B scores still fall sharply.

During the discussion period we added two checks.

1. Evaluation on the agent's own chosen public benchmarks. We compared each candidate model with its initial base model on the public test set matching its training data. 38 of 42 pairs fall below base, and none of the 18 mathematics-goal pairs (9 models) exceeds base. Models collapse even on their own chosen in-distribution benchmarks, so the failure is not a coverage gap in the hidden set.

Public test setPairsAbove : below baseObservation
GSM8K / MATH-50015 / 153 : 12 / 1 : 14All 18 / 18 mathematics-goal pairs drop
MMLU / SciQ6 / 60 : 6 / 0 : 6All drop sharply (−35.20 to −96.90 points); SciRIFF also drops from 57.81% to 19.67%
Total424 : 38Only 4 non-mathematics pairs show tiny gains of +0.23 to +0.53 points

2. Subskill coverage and blind labeling of training choices.

  • Coverage: We annotated all 520 hidden items with subskills independently, and the annotators were broadly consistent. Of the 64 subskills the agents named in their plans, 46–55 (71.9%–85.9%) are covered by hidden items.
  • Training choices: Three independent LLM annotators classified the training choices of all 54 runs with scores hidden, with three-way agreement on 46/54 (85.2%). The failures trace to off-goal data selection, format mismatch, or training degradation, not to reasonable subgoals going uncovered.
AuthorsOpenReview ↗

Response to Reviewer b5Xr (3/4)

Q3: Quantifying format and execution losses, with minimal-repair controls

Could the authors quantify losses attributable to format and execution errors and test whether the main findings persist under minimal repair controls? …

We audited all run logs, then recomputed results with the affected runs excluded and ran a minimal repair control.

1. Exclusion of all single-digit-label collapses. Across both RQ2 protocols we found 13 affected candidate models and 989 single-digit evaluation outputs, including the 5 final-only models and 279 outputs reported in the paper. Excluding them leaves the conclusions unchanged:

Protocol and metricAll runsExcluding single-digit collapsesConclusion changed?
Final-only (runs above base)3 / 24 (12.5%)3 / 19 (15.8%)No (still only 9B science A/B and 9B writing A)
Adaptive-feedback (retained gains)1 / 30 (3.3%)1 / 30 (3.3%)No

2. Minimal repair of the Terra harness. We changed exactly one thing in Terra's H1, leaving all other code untouched. If the response after a tool call is empty, the harness now requests the final answer once more within the same context and call budget, instead of submitting the partial draft. With the same Qwen3.5-4B model, decoding, and judge, the patched harness was executed three times on all 20 hidden writing tasks. A concurrent re-run of H0 scored 27.79, consistent with the original 28.64 ± 0.69.

HarnessTask macroExample microDiff. vs. H0 [95% CI]Two originally failing tasks
Qwen-Agent reference (H0)28.64 ± 0.6927.65 ± 0.57—75.00 / 52.33
Original Terra (H1)20.76 ± 0.7420.14 ± 0.88−7.88 [−15.59, −1.82]2.57 / 2.91
Patched Terra H124.35 ± 1.6423.52 ± 1.63−4.29 [−10.00, +0.82]32.97 / 25.78 (mean of 3)

What the repair shows:

  • Across the 60 task executions of the original Terra H1 (20 tasks × 3), empty responses occur 5 times. The patch fires in 2 of 60 executions and recovers a complete answer each time.
  • Task macro rises by 3.60 points, closing about 46% of the gap to H0.
  • The "about 70%" in Appendix D was a single-execution arithmetic upper bound that assumed both tasks would fully recover to their H0 scores. Measured on these two tasks, the patch recovers about 31% of that execution's total decline.
  • Even after the repair, Terra's H1 does not exceed H0: 13 of 20 tasks remain below H0, and the two originally failing tasks remain well below.
  • Sol's H1 (27.22), which is effectively "H0 + non-empty check", also does not exceed the reference.

The conclusion that no evolved harness exceeds the reference therefore holds.

Reviewer_5nmPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLM agents can turn vague capability goals into effective self-improvement strategies. It introduces an environment for model-weight and agent-harness evolution, together with a hidden evaluation set of 520 expert-authored items spanning six goals. Three studies examine how goal specification changes post-training behavior, whether self-directed weight updates produce retained improvements, and whether one-step harness evolution improves performance with fixed runtime weights. Agents frequently complete the optimization procedures, but retained weight-level gains are rare, and the evaluated successor harnesses remain below the Qwen-Agent reference on the writing tasks.

Critical strengths
  1. Important and clearly defined problem. The paper investigates an important question: whether agents can decide what and how to learn from broad capability goals.

  2. Clear organization. The three research questions provide a coherent progression from goal specification to weight updates and harness evolution. The experiments follow this structure consistently, making the setup, findings, and limitations easy to locate.

  3. Concrete and interpretable failure evidence. The paper supplements aggregate results with checkpoint trajectories and specific failure cases, including answer-format collapse and incomplete final responses. These examples help explain how agents can complete training or harness-editing procedures while producing worse outputs.

Critical weaknesses
  1. The description of the experimental setup lacks clarity. For instance, the paper does not provide concrete examples of the tasks (including inputs and outputs), the specific rubrics used for evaluation, or the full prompts employed for the optimization models. The absence of such examples makes the work difficult to follow. Furthermore, the setup of the "agent harness" mentioned in relation to RQ3 is not clearly explained.

  2. The experimental results consist primarily of a collection of behavioral statistics, with limited discussion on the underlying causes or a deeper, fundamental analysis. There is a lack of targeted ablation or corrective experiments to verify the impact of specific factors on the results. Regarding RQ1 and RQ2, for example, it remains unclear what the primary bottleneck is that causes performance to be worse when training with "vague goals" compared to "clear goals." It is difficult to discern whether this is because the optimizer’s perceived path toward capability improvement—and its interpretation of the objective—deviates from the actual ground truth, or whether the ambiguity simply leads to longer convergence times during training.

Questions for authors
  1. Regarding RQ1 and RQ2, where exactly does the failure lie? Is it that the model fails to translate a vague goal into a concrete learning plan, or is it simply that standard post-training is inherently difficult to succeed with under this specific setup?
  2. Why was a writing task chosen for RQ3? Furthermore, why do the authors believe that the writing task used in RQ3 requires such a sophisticated and intricate "harness" design? To what extent is there room for further optimization of this harness?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 5nmP (1/4)

We thank the reviewer for the careful reading. The experimental setup, prompts, evaluation protocol, and failure analysis asked about are documented in the main text and appendices. Below we point to the exact locations and add several clarifications, including new results on public benchmarks obtained during the discussion period.

W1: Clarity of the experimental setup

The description of the experimental setup lacks clarity. For instance, the paper does not provide concrete examples of the tasks (including inputs and outputs), the specific rubrics used for evaluation, or the full prompts employed for the optimization models. … Furthermore, the setup of the "agent harness" mentioned in relation to RQ3 is not clearly explained.

Prompts. The prompt structure for the optimization (decision) model is given in Appendix A.3 (p.14).

  • All six goals share one versioned prompt template; only the goal sentence differs.
  • Under the vague-goal setting, the prompt contains only:
    • the natural-language goal;
    • the model to be trained;
    • the action protocol;
    • the compute budget;
    • the rule that the hidden evaluation set returns only an aggregate score.
  • It reveals no evaluation items, answers, rubrics, routing identifiers, or per-item feedback. For example, the mathematics goal asks the agent to improve "difficult multi-step reasoning, precise calculation, and clear justification" without naming any benchmark.
  • The explicit-task setting additionally discloses the task format, difficulty anchor, metric, and success criterion.
  • Every run checks its setting identifier, prompt version, and the SHA-256 of the rendered prompt at creation and on recovery.

Evaluation and rubrics. Construction and scoring are described in Appendix E.2 (pp.19–20) and Figure 6 (p.20).

  • Domain experts write every item from scratch, and other experts review it.
  • Items are difficulty-graded by blind screening with three models, deduplicated, and checked for overlap with public benchmarks.
  • Each item is then bound to a deterministic checker or a fixed-rubric judge, and the set is version-frozen.
  • Item counts per goal are in Table 2 (Appendix A.1, p.14).

We do not print raw items because Aspire's core premise is that evaluation items stay hidden from the agent. Releasing items and answers would break this information boundary, so items and raw trajectories are under controlled access (Reproducibility Statement and Ethics Statement, p.10).

Agent harness (RQ3). The harness is defined in §2.1 (p.3) as the runtime instructions, tool-use policy, workflow, memory, and validation logic around a fixed-weight model.

  • Editable scope and freezing rules: §2.3 (p.4).
  • Fixed vs. editable components: Table 6 (Appendix E.1, p.19).
  • The edit–freeze procedure: Figure 7 (Appendix E.3, p.21).
  • The RQ3 design: §3.4 (p.8), summarized in our answer to Q2 below.
AuthorsOpenReview ↗

Response to Reviewer 5nmP (2/4)

W2 & Q1: Where does the failure lie? Translating the goal into a plan, or post-training difficulty / slower convergence?

W2: … it remains unclear what the primary bottleneck is that causes performance to be worse when training with "vague goals" compared to "clear goals." It is difficult to discern whether this is because the optimizer's perceived path … deviates from the actual ground truth, or whether the ambiguity simply leads to longer convergence times during training.

Q1: Regarding RQ1 and RQ2, where exactly does the failure lie? Is it that the model fails to translate a vague goal into a concrete learning plan, or is it simply that standard post-training is inherently difficult to succeed with under this specific setup?

The three RQs are designed to localize the failure step by step. One clarification first: the "vague vs. explicit goal" comparison appears only in RQ1. RQ2 has no explicit-goal arm; it compares each trained model against its own initial instruction-tuned base model.

RQ1 takes PostTrainBench and replaces only the benchmark name in the prompt with a broad capability description; the evaluator, training interface, tasks, and budget are unchanged (§2.1 p.3, §3.2 p.5). Across 48 matched pairs with the same model, task, and trial index (Appendix B, p.15), vague goals change how the agent spends its time (§3.2 and Figure 2(b), p.5):

  • the agent reads task materials 2.98× as often;
  • it inspects proxy evaluation scripts 2.39× as often;
  • active training time falls by 1.27 hours.

Much of the budget goes into working out what to learn.

RQ2 (§3.3, p.6) tests weight self-evolution under two protocols:

  • Final-only protocol: no hidden-set scores during training, and a single final model is submitted. Qwen3.5-4B Self and Qwen3.5-9B Self each run twice on each of six goals, for 2 × 6 × 2 = 24 runs. ("Self" means the frozen initial model acts as its own decision model.)
  • Adaptive-feedback protocol: a limited number of aggregate hidden-set scores may guide further training. There are five configurations (4B/9B Self plus the external decision models GPT-5.6 Luna, Terra, and Sol), each run once on each of six goals, for 5 × 6 = 30 runs.

These 54 runs fail mainly at two steps:

  • Choosing the wrong training data and label format.
    • 4B Self imports GSM8K/MATH in 30 of 32 data imports, including under the science, logic, and writing goals (§3.3 pp.7–8; Appendix C.4 p.17).
    • Across both protocols, 13 candidate models were SFT-trained on single-digit classification labels, turning 989 evaluation answers into a single digit. Five of these models and 279 of these outputs are the final-only cases (§3.3 p.8; Appendix C.3 p.16).
  • No effective self-check, so continued training degrades the model.
    • Terra's mathematics run reaches 20.10, then drops to 2.78 with further training (Appendix C.4 p.17; Appendix F.2 p.22).
    • 4B Self mathematics goes 0.79 → 1.75 → 2.38, far below its base of 17.86 (§3.3 and Figure 5(c), p.7).
    • During the discussion period we also evaluated each trained model on the public test set matching its own training data. The models collapse even on the proxy tasks they chose themselves:
Public test set matching training dataPairsAbove baseBelow baseObservation
GSM8K15312Mathematics goal: 0 / 9 models above base
MATH-50015114Mathematics goal: 0 / 9 models above base
MMLU606All drop sharply (−38.32 to −60.20 points)
SciQ606All drop sharply (−35.20 to −96.90 points)
Total42438All 18 mathematics-goal pairs fall below base; SciRIFF also drops from 57.81% to 19.67%

RQ3 fixes the model weights and edits only the harness. It also stalls at self-evaluation (Appendix D, p.18):

  • Luna validated on one prompt with an 8-item checklist and stopped after about 40 minutes.
  • Terra's harness lacks a final-answer check, so some tasks submit only a partial draft.
AuthorsOpenReview ↗

Response to Reviewer 5nmP (3/4)

Is standard post-training simply too hard here? No. In the science goal, both independent 9B Self runs improve from 45.33 to 48.00 (§3.3 p.6; Table 4 p.17), so the training pipeline itself works.

Is it slower convergence? The data do not support this. Each run has a 40 GPU-hour cap, yet agents stop on their own after using only about 10–30% of it on average, and the three longest-training runs collapse to near zero:

Protocol / caseBudget cap per runTraining compute usedOutcome vs. baseSource
Final-only (24 runs)40 GPU-hmean 6.22 GPU-h (15.5%)3 / 24 runs above baseTable 5, Appendix C.6 (p.18)
Adaptive-feedback (30 runs)40 GPU-hmean 10.75 GPU-h (26.9%)1 / 30 retained gain (passes completion checks and exceeds base)Table 5, Appendix C.6 (p.18)
Three longest-training runs (4B math B / 9B math B / 9B writing B)40 GPU-h26.57 / 32.29 / 30.53 GPU-h (16–20 training jobs)0.159 / 2.540 / 0.000 (bases: 17.86 / 25.16 / 22.93)Appendix C.3 (p.16)

The bottleneck is not training time. The agent's decomposition of the goal goes off-target, and without reliable self-checks, more training only accelerates degradation. The observational nature of this evidence is stated in §3.2 (p.5) and §3.3 (p.8).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.