- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
INFO-SKILL compresses a retrieved skill and the current state into eight soft-prefix embeddings for a frozen LLM with LoRA. Training combines RL, outcome prediction, a KL-rate penalty, and grounding supervision. Skill-library edits are screened by predictive gain, rate and redundancy, then checked through paired rollouts. Experiments cover ALFWorld, Search-based QA and WebShop.
Critical strengths
- Useful topic. A retrieved procedure can contain steps that do not matter yet or have already been completed. Figure 1 shows that clause influence on the selected action’s probability changes with state, while retrieval frequency is a weak proxy for usefulness in the tested setting.
- Robustness results. After skill-library contamination, the coded interface achieves 87.3% success before repair, versus 63.4% for full text. After five repair rounds, misleading-skill retention is 9.8% with the utility gate and 45.6% without it.
Critical weaknesses
- The distinction from close prior work needs clarification. [1] combines reasoning states and retrieved text into latent memory. [2] uses PPO-style candidate verification and gain-based pruning. The paper should clarify what its shared predictive-rate criterion adds beyond these existing components.
- The ALFWorld results need a clearer evaluation account. With a fixed task mix and consistent aggregation, overall success should equal the weighted task-family rates. Table 1 cannot be reconciled under this usual aggregation model, even allowing for rounding. This could reflect sampling or aggregation; it does not uniquely identify different test sets.
- The library rule needs clearer controls and implementation details. Table 4 supports the utility gate, but does not isolate the predictive screen's added value. A utility-only comparison at equal total interaction cost would help.
References:
[1] MemGen: Weaving Generative Latent Memory for Self-Evolving Agents. In ICLR 2026.
[2] Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents. In ICML 2026.
Questions for authors
- Figure 1(a) shows the influence on the selected action's probability, and Section 5.1 reports unstable full-text prompting in some settings. How can interference from irrelevant clauses be distinguished from effects of skill quality or policy training, to support the motivation for state-conditioned selection?
- What distinct role does the rate penalty play? The authors interpret the top-1 objective as a finite-capacity predictive-rate surrogate. What benefit does the rate penalty add over a state-conditioned encoder with only an auxiliary prediction loss?
- Table 3 already compares token pruning. What advantage does a soft prefix offer over text compression that uses the current state as its query [1]? How should this benefit be weighed against text readability and ease of reuse across backbones?
References:
[1] LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. In ACL 2024.
6 comments
Response for W3 and Q1
Response for W3
1. Controls and Implementation Details
For admission, paired rollouts compare candidate-enabled and matched null-code conditions. For pruning, they compare skill-retained and skill-masked conditions. Each pair shares the task, random seed, initial state, horizon, and decoding settings, with the policy, encoder, retriever, and predictors fixed (Appendix A.3). We collect the paired trajectories before gate testing, apply predictive screening first, and reuse the same trajectories for utility verification when screening passes. An edit requires both tests to pass.
2. Ablation: Utility-Only Maintenance at Equal Interaction Cost
Table 2 compares information-based scoring with frequency ranking. To isolate the predictive screen’s contribution, we additionally compare the full method with utility-only maintenance at equal total interaction cost.
Task: Follow natural-language instructions to complete household tasks in ALFWorld.
Setup: Using Qwen2.5-3B-Instruct, we remove only predictive screening. Both variants share the initial skill library, policy and encoder checkpoint, candidate-generation settings, paired utility test, and all other maintenance settings. The utility-only variant applies utility verification directly. Both use the same independent test set.
Interaction budget: Both variants receive the same total environment-interaction budget. We count all steps from both paired-rollout branches, including assessments of rejected candidates.
Table A. Predictive Screening vs. Utility-Only Maintenance at Equal Interaction Cost
| Maintenance rule | Selection criteria | Macro SR (%) |
|---|---|---|
| Full method | Predictive screen + utility gate | 96.0 |
| Utility-only | Utility gate | 93.6 |
In Table A, retaining predictive screening improves macro SR by 2.4% with the utility gate and total environment-interaction budget fixed. Considering predictive gain, coding cost, and redundancy alongside rollout utility thus helps select more effective library edits without increasing environment interactions.
Response for Q1
Info-Skill improves task performance by making learned skills more useful at each decision, complementing advances in policy optimization and skill quality.
1. Stronger Policies and Skills Still Require State-Relevant Skill Use
The baselines in Table 1, including GRPO, Skill-GRPO, OPID, RLSD, SDAR, Skill-SD, and ADRS, advance policy optimization, skill-based supervision, or skill distillation (Section 2). Our work builds on these directions by addressing a further bottleneck: a useful skill can describe an entire procedure, while only part of it is relevant to the current decision. A better-trained policy still needs to identify that part. Section 5.1 documents the inconsistent benefits of full-text skill prompting across backbones and tasks. Info-Skill addresses this skill-use bottleneck through state-conditioned compression, seeking further task-performance gains from the skills available to the policy.
2. Figure 1(a) Compares Clause Contributions Within the Same Policy and Skill
The useful part of a skill changes with the decision state, even when the policy and underlying procedure remain the same. A correct procedure can contain steps that are already complete or not yet needed.
Figure 1(a) uses frozen full-text skill-prompting agents. Each comparison removes one clause while keeping the current state, evaluated action, remaining skill text, and policy fixed. We then compare clause contributions across states for the same procedure.
3. State Conditioning Improves Task Performance at the Same Prefix Budget
Table 2 shows lower performance when state conditioning is removed. With fixed retrievals and matched context budgets, Table 3 compares Info-Skill with a state-agnostic soft code using the same eight-prefix budget. Info-Skill improves ALFWorld success from 94.1% to 96.4%, Search-based QA accuracy from 43.4% to 45.6%, and WebShop score from 86.3 to 88.7.
Figure 1(a) and Tables 2–3 support an additional source of task-performance improvement: using the state-relevant content of a skill more effectively, alongside advances in policy training and skill quality.
Response for Q2 and Q3
Response for Q2
1. Reducing Redundant Information in Skill Encoding and Its Interference with Decision-Making
The rate penalty aims to reduce redundant skill information and its interference with decision-making. Prediction-only training rewards accurate outcome prediction but does not charge for retaining additional skill details. The second term of Eq. (5) in Section 4.1 adds a KL cost for deviations of the skill-conditioned code distribution from the state-only reference. Together, the two terms encourage the encoder to preserve useful skill guidance without carrying unnecessary detail. The rate penalty therefore adds a coding-cost constraint to the finite-capacity objective, rather than another prediction target.
2. Ablation: Removing Only the Rate Penalty
Task: Follow natural-language instructions to complete household tasks in ALFWorld.
Setup: Using Qwen2.5-3B-Instruct, we remove only the rate penalty by setting . Both variants retain the same state-conditioned encoder, prediction loss, grounding loss, RL, eight soft-prefix embeddings, fixed skill library, and training budget. Both use the same independent test set.
Table A. Ablation of the Rate Penalty
| Training objective | Rate weight | Macro SR (%) |
|---|---|---|
| Full predictive-rate | 0.001 | 89.2 |
| Prediction-only | 0 | 87.4 |
In Table A, retaining the rate penalty improves macro SR by 1.8% with the state-conditioned encoder, prediction loss, and eight-prefix budget fixed, showing that constraining the code’s KL cost improves task performance even when the encoder already receives outcome-prediction supervision.
Response for Q3
1. From Long Prompts to a Matched Skill Budget
- Long-context regime. LongLLMLingua ranks source tokens by query-conditioned contrastive perplexity. Its latency study compressed about 10,000 tokens by 2–6×, retaining roughly 1,700–5,000 [1].
- Same ratios on skills. ALFWorld skills average 92.4 tokens. The same ratios leave about 15–46 tokens per skill. We select 16 tokens (5.8×) as a relaxed control within this range, giving text twice the prefix budget.
- Equal skill budget. Eight text tokens correspond to 11.6× compression. The matched run fixes the library and retriever, uses the task, recent history and observation as the query, and compresses only the selected skill. We count added positions with the Qwen2.5-3B-Instruct tokenizer and pair runs by task and seed.
Table B. LongLLMLingua Comparison at Matched and Relaxed Skill Budgets
| State-queried LongLLMLingua | Mean skill positions | ALFWorld success (%) | Info-Skill minus text (%) |
|---|---|---|---|
| 16-token target | 16.0 | 94.3 | +2.1 |
| 8-token target | 8.0 | 92.9 | +3.6 |
In Table B, Info-Skill has 2.1% higher success than LongLLMLingua when text receives twice the input budget and 3.6% higher success at equal budget. The equal-budget result favors a learned skill representation even when text selection is also state-conditioned.
2. Readability and Reuse in a Skill Library
In long-context QA, readers can inspect the compressed passage and pass it to another model. Our agent can consult the same skill while locating, cleaning and placing an object (Figure 1(a)). The library stores the skill's principle and conditions of use in editable text (Table 8), so users can review, revise and transfer the procedure to another backbone. Outcome prediction under a KL penalty and rollout returns guide how the agent encodes this text (Eqs. 5–6). The resulting prefix can link a prerequisite in one clause to the action it governs in another. Its vectors require a compatible learned projector (Appendix A.3). Text compression is preferable for auditing the exact policy input or using a text-only API. A fixed agent can reuse its projector at every eight-position skill call.
References
[1] Huiqiang Jiang et al. LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. ACL, 2024.
Response for W1 and W2
Response for W1
Thank you for your careful review and constructive feedback.
Info-Skill weighs a skill code's predictive gain over the state alone against its KL coding cost. This trade-off guides what to encode at each decision and which library entries warrant a rollout test.
1. State-conditioned compression.
MemGen learns when to invoke memory and can combine reasoning states and retrieved text into latent memory. In Fig. 1(a), the clause whose removal most changes next-action probability varies with the state. We use clause masking only for this diagnostic. Our encoder learns a continuous code from the skill and state by balancing outcome log likelihood against KL cost on a frozen snapshot (Eq. 5). [1, Sections 4.2–4.3]
2. Library screening.
Skill-Pro's PPO gate compares new and old skills using historical action likelihood ratios and return advantages. It prunes by empirical gain during skill use. Our score δk,z measures the code's held-out gain in outcome log likelihood over a state-only predictor matched for capacity and training budget. The rate τk,z is its KL cost relative to a state-conditioned reference (Eqs. 7–8). [2, Sections 4.2–4.3]
3. What sharing the criterion adds.
On a fixed snapshot, measuring the outcome log score in Eq. (5) relative to the state-only baseline shifts the encoder objective by a constant. For library edits, we weight δk,z−βlibτk,z by usage and penalize redundancy (Eqs. 9–12). Even a skill used on high-return trajectories can face a pruning test if the state alone predicts the outcome equally well and the code incurs a KL cost. Paired rollouts with the skill retained or masked then determine removal.
At fixed capacity and update frequency, replacing our criterion with frequency ranking lowers mean success across the six ALFWorld task types from 96.0% to 92.5% (Table 2). With fixed retrievals and eight prefix positions, Info-Skill achieves 96.4% success versus 94.1% for a state-agnostic code (Table 3), with higher held-out gain at lower rate (Fig. 3).
We will add a methodological comparison with MemGen and Skill-Pro to Sec. 2. In Sec. 1 (lines 72–73), we will distinguish our predictive-rate screen from library maintenance based on reward advantage.
References
[1] Guibin Zhang, Muxin Fu, and Shuicheng Yan. MemGen: Weaving Generative Latent Memory for Self-Evolving Agents. arXiv:2509.24704, 2025.
[2] Qirui Mi et al. Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agents. ICML, 2026.
Response for W2
1. Table 1 Combines Published Baseline Aggregates with Our Overall Success Rate
For the Qwen2.5-7B results in our Table 1, we quote both the six task-family scores and the aggregate from each source. The Vanilla–SDAR rows match Table 1 of SDAR [1]. OPID matches Table 1 of [2], and ADRS matches Table 2 of [3]. Code repositories for SDAR, OPID, and ADRS are linked in their respective papers [1–3]. We also inspect these official repositories but do not find the task-family weights needed to reconstruct all quoted aggregates. Info-Skill's 97.1% is the overall success rate across individual tasks (136/140).
The original “Avg.” column therefore contains source-reported baseline aggregates alongside our overall success rate.
2. Supplementing Evaluation Results with Macro-Average as a Unified Metric for Fair Comparison
For a fairer comparison under consistent task-family weighting, we additionally report the six-family macro-average as a common metric for all methods. We compute it as the arithmetic mean of the six displayed task-family scores in our Table 1, giving every family equal weight and retaining the original reported aggregates. Each row still corresponds to its original evaluation sample and protocol.
ALFWorld Results from Our Table 1 (Qwen2.5-7B-Instruct, %)
References
[1] Zhengxi Lu et al. Self-Distilled Agentic Reinforcement Learning. arXiv:2605.15155, 2026.
[2] Shuo Yang et al. OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning. arXiv:2606.26790, 2026.
[3] Ranxu Zhang et al. Agentic Reinforcement Learning with Self-Distilled Reward Shaping. arXiv:2608.03223, 2026.