acceptodds
Under review as a conference paper at ICLR 2027

From Context to Skills: Can Language Models Learn from Context Skillfully?

Abstract

Many real-world tasks require language models (LMs) to reason over complex contexts that exceed their parametric knowledge. This calls for context learning, where LMs directly learn relevant knowledge from the given context. An intuitive solution is inference-time skill augmentation: extracting the corresponding rules and procedures from context into explicit, natural-language skills. However, constructing such skills faces two fundamental challenges: the prohibitive cost of manual skill annotation for long, technically dense contexts, and the lack of external feedback for automated skill construction. In this paper, we propose Ctx2Skill, a self-evolving framework that autonomously discovers, refines, and selects context-specific skills without human supervision or external feedback. At its core, a multi-agent self-play loop has a Challenger that generates probing tasks and rubrics, a Reasoner that attempts to solve them guided by an evolving skill set, and a neutral Judge that provides binary feedback. Crucially, both the Challenger and the Reasoner evolve through accumulated natural-language skills: dedicated Proposer and Generator agents analyze failure cases and synthesize them into targeted skill updates for both sides, enabling automated skill discovery and refinement. To prevent adversarial collapse caused by increasingly extreme task generation and over-specialized skill accumulation, we further introduce a Cross-time Replay mechanism that identifies the skill set achieving the best balance across representative cases for the Reasoner side, ensuring robust and generalizable skill evolution. The resulting skills can be plugged into any language model to obtain better context learning capability. Evaluated on four context learning tasks from CL-bench, Ctx2Skill consistently improves solving rates across backbone models, e.g., lifting GPT-4.1 from 11.1% to 16.5%, GPT-5.1 from 21.1% to 25.8%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Language models, context learning, agent skill evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6eD7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the problem of extracting knowledge that lies outside the pretraining data from a given context C and converting it into natural-language skills. To this end, the authors propose Ctx2Skill, which divides the roles among five LLM agents (Challenger, Reasoner, Judge, Proposer, and Generator) and updates the target skill set in an adversarial manner. The experimental results show that Ctx2Skill consistently improves the problem-solving ability of LLMs compared to the baselines.

Critical strengths
  • The role of each agent is described in detail, which helps the reader understand the proposed scheme.

  • The experiments are extensive. The authors report results on various models and baselines.

  • The experimental results are consistent. The extensive results suggest that Ctx2Skill improves performance in a consistent way.

  • The ablation study is thorough. Questions that came up while reading the paper (e.g., what happens if the Generator and the Proposer are merged into a single model, and why using both the passed and the failed sets to update both the Challenger and the Reasoner skills is harmful) were answered, which was satisfying.

Critical weaknesses
  • It is doubtful whether the iterative self-play actually contributes much. For GPT-4.1, the performance with the Iter-1 skills is 15.9%, while the performance with Cross-Time Replay is 16.5%. Figure 4 also gives the impression that selecting the final skill set only among early iterations might be more cost-efficient.

  • There is too much repetition in the writing. For example, the second-to-last paragraph of the Introduction (starting with "In this paper") overlaps considerably with Sections 3.3 and 3.4, which gives the feeling of reading the same text twice and weakens the delivery of the paper.

  • The cost of model usage, the number of tokens, and the time required for self-play are not stated in the main text. Since five agents are run, the cost should not be negligible, and it is a pity that this point is omitted from the main text. The same holds for the time required for self-play.

  • Figure 1 does not carry enough information. If it is meant to illustrate Ctx2Skill, the part where the agents perform self-play should be explained, but this is lacking.

  • It is not stated in the main text what the initial skill sets S_0^C and S_0^R are.

Questions for authors

Included in weakness part.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6eD7

Thank you for the careful review and the positive assessment of our experiments and ablations.

W1. Iterative self-play may contribute little (Iter-1 15.9% vs. 16.5%); selecting among early iterations may be more cost-efficient.

A1: Iter-1 already contains one self-play round (Prompting 12.3% → Iter-1 15.9%), and 65.8–79.0% of contexts select a later iteration, accounting for 76% of the net gain on GPT-4.1 (General Response C1). Replay selects Iter-4/5 for 27.6% (GPT-4.1) and 36.0% (GPT-5.1) of contexts, so stopping early would remove these candidates. Moreover, extra compute spent on single-pass skills does not substitute for self-play: compute-matched Best-of-12 reaches only 13.3% (General Response C2). We agree that the cost trade-off deserves explicit treatment, and we will discuss lighter settings (smaller N) for budget-constrained use.

W2. Repetition between the Introduction and §3.3–3.4.

A2: We will condense the second-to-last paragraph of the Introduction into a short overview and leave the details to §3.3–

W3. Cost and time are not stated in the main text.

A3: Ctx2Skill makes at most 175 calls per context (≤90 read the context, roughly 1M input tokens), paid once, with one call per query at deployment (General Response C2). We will move this to the main text and report wall-clock time; self-play runs in parallel across contexts.

W4. Figure 1 lacks the self-play part.

A4: We have detailed the self-play part in Figure 2. We will redesign Figure 1 to show the loop: Challenger → Reasoner → Judge, the routing of failed and solved cases to the two sides, and Cross-Time Replay.

W5. S0CS^C_0 and S0RS^R_0 are not stated in the main text.

A5: Both are empty, S0C=S0R=∅S^C_0 = S^R_0 = \emptyset (Algorithm 1, line 1); in Iter-1, both agents rely only on their prompts and the context. We will state this in §3.3.

We thank the reviewer again for the constructive feedback and the kind recognition of our work.

AuthorsOpenReview ↗

Gentle Reminder on the Discussion Period

Dear Reviewer,

We sincerely thank you for your time and effort in reviewing our paper and for your valuable comments. As the discussion period is coming to an end, we would greatly appreciate it if you could take a moment to check whether our responses have addressed your concerns. If they have, we would be grateful if you would kindly consider updating your score. If anything remains unclear, we would be happy to provide further clarification before the deadline.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_gfEEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a multi-agent self-play framework to “evolve” a set of skills that are relevant on a given context document and will become useful at solving tasks in that context. The main loop develops a separate set of skills for a Challenger and Reasoner where at each iteration, the Challenger comes up with practice tasks, which the Reasoner needs to solve. The skill sets for the two agents are updated based on the Judge and Proposer’s feedback. After 5 iterations, the Reasoner chooses a skill set from one of the five iterations based on a replay mechanism: it attempts to solve the easiest and hardest practice tasks from previous iterations and chooses the skill set from the best snapshot.

While the framework is interesting and there are many ablations, the paper lacks some important baselines and fails to sufficiently prove the effectivenss of different components of the proposed pipeline, especially under compute-matched budget.

Critical strengths
  1. Well-motivated problem and framework: extracting interpretable procedural knowledge / skills from a novel context is important for LLMs to generalize to domains far beyond their training distribution. Self-play and co-evolution between the Challenger and Reasoner is a well understood method and seems relevant to this problem.

  2. Comprehensive ablations: Table 2 covers a long list of ablations, removing different components of the framework, justifying the choice of each. In Appendix D, the authors also touch on some validity concerns, replacing the Judge with a different model to account for potential self-collapse.

Critical weaknesses
  1. (Major) Insufficient comparison against compute-matched baselines:
  • Because of the five iterations of self-play, each requiring the Reasoner to solve practice tasks, there is a significant compute overhead in deriving the final skill set. Even though the authors justify this framework by saying that this overhead is amortized across all test-time tasks (line 297), it is worth noting the CL-Bench only contains 3.8 tasks per context on average (line 838). On the other hand, based on my understanding, this framework will need to at least use 50 task-worth of compute just for the Challenger and Reasoner (25 each) plus any additional compute for the Judge, Challenger-Proposer, Reasoner-Proposer, Challenger-Generator, and Reasoner-Generator. The experiments do not sufficiently establish whether the improvements are from the skill extracting mechanism or simply from more test-time compute.

  • For the baseline approaches, I would like to see comparisons against direct test-time scaling methods such as repeated skill extraction, self-refinement, or even best-of-k.

  1. (Major) Limited evidence for the effectiveness of iterative self-play – The results in Table 3 and Figure 4 show that the skill sets generated in the later iterations of the framework result in weaker performance in general, raising concerns about the effective of the proposed self-play. Specifically, GPT-4.1 already reaches 15.9% accuracy with the skill set from the first iteration, compared to 16.5% from the full framework. Since the base model has a 11.1% accuracy, the first iteration already accounts for about 90% of the total improvement.
  • These results suggest that most of the improvement might be coming from generating multiple candidates for skills and selecting among them on some held-out tasks generated by the Challenger. However, the paper does not compare against a natural baseline: run the first self-play iteration five times (or more if you match the compute) independently, select the best skills using a similar replay mechanism. This baseline would test whether sequential co-evolution provides any advantage over independent sampling of skills. Without this comparison, the necessity and effectiveness of self-ply is not sufficiently established.
  1. (Minor) Generalizability of the method not tested sufficiently
  • The method is tested on one benchmark, only on broadly GPT models.
Questions for authors
  1. Can the authors compare against compute-matched baselines? Or, report on the API cost per run, and report the number of tasks per context where amortization would favor the proposed method above other methods?

  2. Can the authors clarify the effectiveness of iterative self-play by comparing against the baseline of generating independent skill sets and selecting from them?

2 comments
AuthorsOpenReview ↗

Response to Reviewer gfEE

Thank you for the constructive review, and for recognizing the motivation of our work and the comprehensiveness of our ablations.

W1 & Q1. Compute-matched baselines and API cost; CL-bench has only 3.8 tasks per context.

A1: We agree that this is important. Ctx2Skill makes at most 175 calls per context, of which ≤90 read the context; this cost is paid once, and deployment needs one call per query (General Response C2). CL-bench's 3.8 tasks per context reflects how the benchmark was built, whereas in practice a manual, a regulation, or a codebase is queried many times. Skills also transfer across models (GPT-5.1 skills lift GPT-4.1 to 16.1%, Table 3), so one construction can serve several models.

As suggested, Table R2 adds Self-Refine and Best-of-k, which also covers repeated extraction with selection. Even the compute-matched Best-of-12, which uses 25 calls per query, reaches only 13.3%, versus 16.5% for Ctx2Skill with one call per query. More test-time compute on single-pass skills therefore cannot replace the knowledge discovered through self-play.

W2 & Q2. Effectiveness of iterative self-play; compare with independent Iter-1 runs plus selection.

A2: As the reviewer observes, later-iteration skill sets are weaker on average. This is the adversarial collapse we identify in §3.4, and it is precisely why we let self-play explore and use Cross-Time Replay to choose a stopping point per context, since the best stopping point varies across contexts. On the "~90%" attribution: Iter-1 already contains one self-play round, while Prompting reaches only 12.3%, so 75% of Iter-1's gain comes from self-play. Moreover, 65.8–79.0% of contexts select Iter-2–5, and on GPT-4.1 these contexts account for 76% of the net gain (Table R1).

For the independent-sampling baseline, the closest existing control is w/o Challenger Skills Evolving. It uses the same N, the same compute, and the same Replay, but the Challenger never evolves, so every iteration samples tasks from the same distribution as Iter-1. It drops from 16.5% to 13.8% on GPT-4.1 and from 25.8% to 22.5% on GPT-5.1, showing that the evolving Challenger matters beyond re-sampling tasks. We will add the exact independent-run baseline in the revision.

W3. Generalizability: one benchmark, GPT models only.

A3: We thank the reviewer for this suggestion. As shown in Table R3 (General Response C3), we run the full Ctx2Skill pipeline with a non-GPT backbone, Claude Opus 4.8, and on a new benchmark, CL-bench-Life, which is built from messy real-life contexts whose domain differs substantially from CL-bench. Ctx2Skill raises Claude Opus 4.8 from 24.8% to 27.8% on CL-bench and from 16.8% to 19.8% on CL-bench-Life, and raises GPT-5.1 from 17.3% to 20.2% on CL-bench-Life. The gains therefore hold across both backbone families and both benchmarks.

We thank the reviewer again for the valuable suggestions, which have helped us clarify the cost and the generality of our framework.

AuthorsOpenReview ↗

Gentle Reminder on the Discussion Period

Dear Reviewer,

We sincerely thank you for your time and effort in reviewing our paper and for your valuable comments. As the discussion period is coming to an end, we would greatly appreciate it if you could take a moment to check whether our responses have addressed your concerns. If they have, we would be grateful if you would kindly consider updating your score. If anything remains unclear, we would be happy to provide further clarification before the deadline.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_qX3KOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Ctx2Skill, a framework that turns a long context into a natural-language skill file that any LM can use at inference time, without human annotation or external feedback. The paper frame this as Self-Play: A Challenger generates tasks and rubrics from the context, a Reasoner answers them using its skill set, and a fixed Judge (GPT-5.1) gives binary verdicts. Failed cases update the Reasoner's skills, and solved cases update the Challenger's skills, both through a Proposer–Generator pair. Because later iterations drift toward extreme tasks ("adversarial collapse"), a Cross-Time Replay step picks the best skill set across iterations using small hard and easy probe sets. On CL-bench (500 contexts, 1,899 tasks), Ctx2Skill improves GPT-4.1 from 11.1% to 16.5%, GPT-5.1 from 21.1% to 25.8%, and GPT-5.2 from 18.2% to 21.4%, beating single-pass prompting and AutoSkill4Doc.

Critical strengths
  1. The paper has thorough ablation and analysis. The paper ablates nearly every component and also addresses the obvious concerns head-on: a different judge in the loop (Table 11), n-gram contamination against test tasks (Table 12), and a skill-swap test (Table 13). The skill-swap result is especially convincing: mismatched skills drop performance from 16.5% to 12.5%, so the gains come from context-specific content, not generic answering tips. One caveat is that results come from single runs, and several component ablations differ by under 1 point, which is within noise for 1,899 tasks.
Critical weaknesses
  1. Framing issue: the "self-play" loop in this paper has no mechanism that makes the two roles co-evolve.:
  • In the usual sense, self-play has two ingredients. First, both players optimize an explicit, shared objective, typically zero-sum, so one side's gain is exactly the other's loss. In LLM self-play, the challenger is usually rewarded for tasks the solver can learn from, not just for tasks it fails. Second, there is an update rule that reliably moves each player toward that objective, usually gradient updates on model weights. The self-play works the paper cites (SPIRAL, Search Self-Play) follow this pattern: models are trained with RL or distillation against an explicit reward. That is why improvement on one side forces improvement on the other.
  • Ctx2Skill has neither ingredient. The weights are frozen, which on its own would be fine. But nothing replaces the missing optimization: there is no objective the skill sets are meant to maximize, and no check that an edit actually helps. Each iteration, an LLM rewrites a skill file based on a few failed or passed cases, and the new version is accepted unconditionally. The Challenger is pushed only toward "make the Reasoner fail". It gets no credit for tasks staying valid, solvable or representative of the context, and the Judge never checks whether its rubrics are correct.
  • The paper's own results show what this leads to. Tasks and rubrics keep growing (GPT-5.2's tasks double in length), GPT-5.2's solve rate falls across iterations, and downstream performance gets worse with more iterations (GPT-4.1: 15.9% at Iter-1 to 14.7% at Iter-5). The authors call this "adversarial collapse" and fix it after the fact by selecting an early iteration (Cross-Time Replay). Most of the gain seems to come from the first round of skill extraction, not from co-evolution. I'd suggest either describing the method as iterative skill refinement with self-generated probes, or adding an objective (for example, rewarding the Challenger for tasks of intermediate difficulty, or accepting a skill edit only if it improves a held-out probe set) that would make the self-play framing hold.
  1. Missing comparisons with closely related skill-learning methods.
  • The paper compares only against single-pass prompting and AutoSkill4Doc, but several existing methods tackle almost the same problem: learning or refining natural-language skills or memories for a frozen LLM. EvoSkill [1] refines skills based on failures, CoEvoSkills [2] co-evolves skill generators and verifiers, and ACE [3] evolves a context "playbook" over time. All three are cited in the paper but never compared against. Two other highly relevant works are not discussed at all. ReasoningBank [4] builds reusable reasoning memory from the agent's own self-judged successes and failures, with no ground-truth labels. SkillOpt [5] optimizes a skill document for a frozen model and accepts an edit only if it improves a validation score.
  • The paper argues these methods need external feedback that isn't available in context learning. I don't find this convincing. ReasoningBank already works without ground truth. More importantly, Ctx2Skill's own Challenger tasks and Judge produce exactly the kind of feedback signal these methods need, so they could run in the same setting. Without these comparisons, it's hard to tell whether the gains come from the self-play design or simply from iterative skill refinement, which several of these methods already do. A compute-matched comparison also matters here, since Ctx2Skill makes many more LLM calls than single-pass prompting.

[1] Alzubi et al. EvoSkill: Automated Skill Discovery for Multi-Agent Systems. https://arxiv.org/abs/2603.02766

[2] Zhang et al. CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification. https://arxiv.org/abs/2604.01687

[3] Zhang et al. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. https://arxiv.org/abs/2510.04618

[4] Ouyang et al. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. https://arxiv.org/abs/2509.25140

[5] Yang et al. SkillOpt: Executive Strategy for Self-Evolving Agent Skills. https://arxiv.org/abs/2605.23904

Questions for authors
  1. Clarification: Iter-1 Reasoner skills are built from tasks generated with empty Challenger skills, so they involve no Challenger evolution, yet they reach 15.9%. Why does "w/o Challenger Skills Evolving" drop to 13.8%, when its candidate set should contain an equivalent Iter-1 skill set? How large is run-to-run variance?
  2. Clarification: Can you compare against EvoSkill- or ReasoningBank-style refinement, using your own self-generated tasks and Judge as the feedback signal, at matched API cost?
  3. Clarification: Did you check how often Challenger-generated rubrics are wrong or unsupported by the context, especially in later iterations? Is there any mechanism that keeps the Challenger's tasks valid, apart from selection afterward?
2 comments
AuthorsOpenReview ↗

Response to Reviewer qX3K(1/n)

Thank you for the detailed and insightful review, and for finding our ablations thorough and the skill-swap result especially convincing.

W1. Framing: the self-play loop has no shared objective and no update rule that reliably moves each player toward it.

A1: Prior self-play methods operate where outcomes can be verified. SPIRAL relies on game rules, Search Self-Play on ground-truth answers with retrieval-based verification, and learnability rewards on executable or majority-vote checks. Context learning has no environment or verifier: every signal is produced by LLMs inside the loop, and optimizing such a signal directly invites reward hacking. We therefore use the Judge signal only in its most robust forms, namely to route updates and to compare skill sets across time.

The two ingredients of self-play are instead instantiated in text space. The objective is zero-sum: J=E[y]J = \mathbb{E}[y], where y=1y = 1 iff all rubrics pass; the Reasoner maximizes JJ and the Challenger minimizes it. Each side is updated only from the instances it lost, and the Proposer–Generator pair acts as a text-space best response (cf. TextGrad). Cross-Time Replay then serves as the acceptance test the reviewer suggests, applied at the trajectory level. Since SiRS^R_i extends Si−1RS^R_{i-1}, selecting SiRS^R_{i} amounts to accepting the first ii edits and rejecting the rest. This decision is made on probes from all past Challengers, much like evaluating against past opponents in Fictitious Self-Play or PSRO. We do not test each edit separately because each iteration adds at most one hard and one easy probe, which makes per-edit tests noisy.

Finally, the gain does not mainly come from the first round of extraction (General Response C1), and it is not a matter of extra compute either: compute-matched test-time scaling stays at or below 13.3% (General Response C2). We will add the formal objective to §3.3, and we will describe Ctx2Skill as skill-level self-play without monotonic-improvement guarantees.

W2 & Q2. Related methods (EvoSkill, CoEvoSkills, ACE, ReasoningBank, SkillOpt): ReasoningBank needs no ground truth, and the Challenger/Judge already provide feedback. Can you compare at matched cost?

A2: We will discuss all five works in the revision. These methods rely on feedback that a static context does not provide. EvoSkill and SkillOpt accept or select skills using labeled validation sets, and ACE and CoEvoSkills rely on execution feedback. ReasoningBank is label-free, but its judge decides success from the interaction trajectory and the final environment state (e.g., the final webpage), and it accumulates memory over a stream of task queries. Context learning offers neither an environment state nor a task stream.

We agree that our Challenger and Judge produce exactly the signal these methods need, but this signal only exists once our Challenger–Judge loop creates it. Turning a static context into such a learning environment is our core contribution. Given this environment, the key question is whether the gains come from self-play or simply from iterative refinement. The closest instance in our paper is w/o Challenger Skills Evolving. At identical cost, it performs failure-driven skill refinement with selection, in the spirit of EvoSkill, on non-evolving self-generated tasks with the same Judge. It drops from 16.5% to 13.8% on GPT-4.1 and from 25.8% to 22.5% on GPT-5.1, showing that co-evolution matters beyond iterative refinement.

Our goal also differs from per-trajectory memory. Ctx2Skill produces one consolidated, human-readable skill document per context that is reusable for any unseen task, transfers across models (Table 3), and adds no calls at deployment. SkillOpt (May 2026) is concurrent work.

AuthorsOpenReview ↗

Response to Reviewer qX3K(2/n)

Q1. Why does "w/o Challenger Skills Evolving" drop to 13.8% when its candidates include an Iter-1-equivalent set (15.9%)?

A3: We agree that this ablation's candidates include an Iter-1-equivalent skill set; the gap lies in selection rather than in candidate quality. Without Challenger evolution, every iteration samples tasks from the same distribution, so the probes concentrate on the same parts of the context. Each hard probe was also used to build the subsequent skill sets, which favors later candidates on QhQ_h, while the homogeneous easy probes no longer penalize regressions (Eq. 4). Replay therefore tends to pick later, redundant skill sets. Challenger evolution thus plays a dual role: it pushes the Reasoner toward new knowledge, and it diversifies the probe pool that Replay relies on.

Regarding variance, Table 11 is an independent full re-run with a different loop Judge and yields 21.2% vs. 21.4%. The main GPT-4.1 gain is significant: a context-clustered paired bootstrap over all 1,899 tasks gives +5.4 points, 95% CI [+3.6, +7.4]. In addition, the three largest ablation drops are the same on GPT-4.1 and GPT-5.1.

Q3. How often are Challenger rubrics wrong or unsupported by the context? Is there any mechanism beyond selection?

A4: A human audit is impractical: CL-bench contexts average 10.4K tokens (up to 65K) and were built by domain experts. This annotation bottleneck is exactly what motivates Ctx2Skill. Instead, the evaluation itself provides external validation. All gains are measured on expert-written tasks and rubrics that share no 50-gram with Challenger tasks (Table 12). If Challenger rubrics were frequently wrong, the distilled skills would inject wrong knowledge and hurt performance on these tasks; instead, gains are consistent across all four backbones (Table 1 and Table R3) and both benchmarks (Table R3), and swapping skills across contexts drops performance from 16.5% to 12.5% (Table 13).

Beyond selection, three mechanisms keep tasks valid. First, the Challenger prompt requires tasks to be context-grounded and rubrics to be verifiable and specific (Figure 6). Second, the Challenger never sees failed tasks: it is updated only from solved ones, and its Proposer checks whether they "could be answered without carefully reading the full context" (Figure 7), so invalid rubrics that cause failures are never reinforced. Third, the Reasoner Proposer reads the full context and attributes each failure to information in it (Figure 9). We will add an LLM-based rubric validity audit by iteration in the revision.

We thank the reviewer again for the insightful comments, which have helped us sharpen the framing and strengthen the empirical support of our work.

Comments on the paper

5 comments
AuthorsOpenReview ↗

General Response (1/n)

We thank all reviewers for their constructive feedback. We are glad that the reviewers found the problem well-motivated (gfEE), the paper clear (qX3K, gfEE), the results consistent (6eD7), and the ablations thorough (all), with the skill-swap test "especially convincing" (qX3K). We address the shared concerns below.

C1. Does iterative self-play help beyond Iter-1? (all reviewers)

Iter-1 is not single-pass skill extraction. It already contains one full self-play round: Challenger probing, Judge feedback, and failure-driven refinement. On GPT-4.1, single-pass extraction (Prompting) reaches only 12.3%, while Iter-1 reaches 15.9%, so 75% of Iter-1's gain (3.6 of 4.8 points) comes from self-play itself.

Beyond Iter-1, the per-context picture differs from the averages over fixed iterations. Cross-Time Replay selects Iter-2–5 for 65.8% / 79.0% / 67.2% of contexts on GPT-4.1 / 5.1 / 5.2. On GPT-4.1, these contexts gain +5.8 points versus +4.2 for the rest, and account for 76% of the net increase in solved tasks (Table R1). The strongest backbone relies on later iterations the most: on GPT-5.1, which gives our best result (25.8%), selections are nearly uniform across Iter-1 to Iter-5 (86–112 contexts each), and the in-loop solve rate keeps rising until Iter-4 (Table 5).

Table R1. GPT-4.1 results grouped by the iteration selected by Cross-Time Replay.

Selected iteration#ContextsNo skill (%)Ctx2Skill (%)ΔShare of net gain
Iter-11719.613.8+4.224%
Iter-2–532911.917.7+5.876%
All50011.116.5+5.4100%

Solving rates follow the same evaluation protocol as Table 1. Share of net gain = each group's share of the additional tasks solved by Ctx2Skill over the base model.

C2. Cost and compute-matched baselines (all reviewers)

Ctx2Skill makes at most 175 LLM calls per context: 5 Challenger, 25 Reasoner, 25 Judge, ≤20 Proposer/Generator, and ≤100 for Cross-Time Replay. Only ≤90 of these read the context, since the Judge and the Generator do not, which amounts to roughly 1M input tokens per context given the 10.4K-token average context. This cost is paid once per context, and at deployment each query needs a single call, exactly as without skills.

To separate the effect of our skill-learning mechanism from extra compute, we add test-time scaling baselines on GPT-4.1 (Table R2): Self-Refine, and Best-of-k, which also covers repeated skill extraction with selection, including a compute-matched k=12. All three stay between 12.7% and 13.3%, on par with Prompting (12.3%) and AutoSkill4Doc (13.2%), whereas Ctx2Skill reaches 16.5%. Even Best-of-12, which uses 25 calls per query, gains only 1.0 point over Prompting. Spending more compute on single-pass extraction yields little because, without probing, the model cannot discover which contextual knowledge it is missing.

Table R2. Compute-matched comparison on GPT-4.1 (CL-bench overall, %).

MethodCalls / context (once)Calls / queryOverall
No skill0111.1
Prompting1112.3
AutoSkill4Doc≥17113.2
Prompting + Self-Refine (K=5)11112.7
Best-of-501113.1
Best-of-12 (≈ compute-matched)02513.3
Ctx2Skill≤175116.5

Self-Refine: the LM critiques and rewrites its single-pass skill set K times. Best-of-k: for each query, k skill sets are generated with Prompting, one response is produced with each, and an LLM selector without access to the test rubrics keeps one.

AuthorsOpenReview ↗

General Response (2/n)

C3. Generalization to new backbones and benchmarks (gfEE)

We further run the full Ctx2Skill pipeline with a non-GPT backbone, Claude Opus 4.8, and on a new context-learning benchmark, CL-bench-Life [1]. CL-bench-Life is built from messy, fragmented real-life contexts (405 context–task pairs across 3 categories and 9 sub-categories), a domain that differs substantially from the professional contexts in CL-bench. Ctx2Skill consistently improves both backbones on both benchmarks (Table R3).

Table R3. Ctx2Skill across backbones and benchmarks (overall, %). All results except GPT-5.1 on CL-bench (Table 1) are new.

MethodCL-benchCL-bench-Life
GPT-5.121.117.3
GPT-5.1 + Ctx2Skill25.820.2
Δ+4.7+2.9
Claude Opus 4.824.816.8
Claude Opus 4.8 + Ctx2Skill27.819.8
Δ+3.0+3.0

CL-bench has 1,899 tasks; CL-bench-Life results are computed over all 405 tasks. In the self-play loop, the Judge is GPT-5.1 and all other agents use the listed backbone (e.g., Claude Opus 4.8 for the Challenger, Reasoner, Proposer, and Generator). For evaluation, CL-bench uses the same GPT-5.1 judge as Table 1, and CL-bench-Life follows its official protocol with GPT-5.1 (high reasoning effort) as the judge.

Planned revisions: a formal self-play objective (§3.3), a cost table in the main text, a redesigned Figure 1, an explicit definition of S₀, and a condensed Introduction.

[1] Dou et al. CL-bench Life: Can Language Models Learn from Real-Life Context?

Reviewer_gfEEOpenReview ↗

Remaining/Additional concerns regarding C.1

I thank the authors for a detailed response. I will first comment on the authors' response C.1, since it was a shared concern among all reviewers. I hope other reviewers can chime in with their own responses.

Just to recap, the main concern that I raised (and I believe was raised independently by Reviewer 6eD7) is: it is unclear what the contribution of iterative self-play is. To correctly show this, its contribution needs to be separated from any claims about the cross-time replay. This is because the cross-time replay can also be applied on the skill sets generated from 5 indepedent runs of Iteration 1, not necessary from {Iter 1, Iter 2, ..., Iter 5}.

So, is {Iter 1, Iter 2, ..., Iter 5} + cross-time replay better than {Iter 1 run 1, Iter 1 run 2, ..., Iter 1 run 5} + cross-time replay?

To specifically comment on the authors' comments:

on GPT-5.1, which gives our best result (25.8%), selections are nearly uniform across Iter-1 to Iter-5 (86–112 contexts each)

Uniform distribution is the least we can expect. This is what would happen if you have 5 independent, identically sampled copies of skill sets. This further corroborates my concern that the benefit of 5 iterations seems to be just coming from having access to 5 different copies of skill sets to sample from, not from an "evolution" viewpoint. A true "evolving" pipeline would assign a heavier probability mass to the later iterations. However, it is good to note that the stronger baseline is able to benefit a little more from the "evolution".

in-loop solve rate keeps rising until Iter-4 (Table 5)

This does not mean much. If the models indeed collapse (as the authors describe), then the in-loop solve rate may be decoupled from the downstream performance.

On GPT-4.1, [performance gains on the tasks where skills from Iter 2-5 are used] account for 76% of the net increase in solved tasks.

This is not the right metric to use. A more appropriate metric is: how much marginal improvement over Iter 1 (or any previous Iterations) do these skills contribute. Recall that using the skills from Iter 1 throughout all contexts already account for most of the solved tasks. At any subsequent iterations, the goal should be to "evolve" and outperform Iteration 1, not the baseline of no skills.

When reading the authors' response, I realized that it would be nice to have the following statistics to assess the effectiveness of the cross-time replay. The theoretical upper bound measured as: averaged across each context, the performance of the best skill set in hindsight. For GPT-4.1, this should be readily available without any additional experiment, since all skill sets have been evaluated on the downstream performance. All the authors would have to do is average across the max() of each row. This would show what would have been the theoretical maximum that the cross-time replay would have been able to achieve with an oracle, and the actual performance of the cross-time replay would be measured as the performance recovered from the Iteration 1 baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.