acceptodds
Desk rejected at ICLR 2027

𝜋Skill: High-Density Knowledge Extraction from Single Trajectories via Circular Step-Level Analysis

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal agents can reuse past interactions through external memory, yet trajectory-derived memories are often both coarse and unreliable: a rollout-level summary can miss locally useful decisions, while unverified experiences may remain in the prompt after they become irrelevant or repeatedly harmful. We introduce πSKILL, a training-free framework that separates experience extraction, maintenance, and use. First, Circular Step-level Knowledge Distillation (CSD) combines cross-rollout circular analysis of useful operations and observed failure points to identify operational lessons at different granularities. A deterministic admission rule limits overly specific, redundant, and non-actionable candidates. Second, Confidence-aware Memory Updating (CMU) associates each admitted experience with outcome statistics and a lifecycle state, allowing unreliable entries to be suppressed as feedback accumulates. At inference time, the agent decom- poses the task into method-oriented retrieval queries and rewrites a small set of selected experiences into task-specific guidance. Using Qwen2.5-VL-7B-Instruct, πSKILL improves the four-benchmark macro-average Average@4 from 12.67 to 15.32 and Pass@4 from 25.28 to 33.33 relative to state of the art. The gains are largest on benchmarks that require repeated procedural decisions, while the ablations show that inference-time contextualization is essential and that the two memory-construction components have task-dependent effects.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Agents, Training-free Continual Learning, Agent Memory, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_VQCwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This article studies how multimodal tool agents can reuse historical experience without training. It is believed that existing methods summarize the entire trajectory as a single memory, with too coarse granularity, and once the experience is stored, reliability is no longer tested. The author proposes π Skill, which breaks down experience reuse into three steps: extraction, maintenance, and utilization. In the extraction stage, in addition to comparing across rollouts, step level analysis is also performed on individual trajectories, and rule-based scoring is used to filter out overly specific or vague experiences; During the maintenance phase, record confidence and lifecycle status for each experience, and update them based on subsequent execution results; In the usage phase, the task is decomposed into multiple sub queries to retrieve experience, and then combined with the current image and problem to rewrite and inject context. On Qwen2.5-VL-7B, the macro average of π Skill on four benchmarks Average@4 and Pass@4 Compared to XSkill, it has increased by 2.65 and 8.05 points respectively, and has also improved on other backbone models.

Critical strengths

The system considers the complete lifecycle of experiential memory and evaluates it on four different types of benchmarks, multiple backbone models, and cross task transfer. The author also candidly expresses the limitations of the results, such as component effects varying by task and limited transfer.

Critical weaknesses
  • The Ď€ Skill version that removes external memory during ablation has reached 14.19/31.03, higher than all baselines including DC, while the complete system only improves by about 1 point on this basis, indicating that most of the benefits come from the different configurations of the base agent, rather than the memory mechanism proposed in the paper, which directly weakens the core contribution and is the main reason why I tend to reject it.
  • CMU updates the empirical confidence level based on the results provided by the benchmark evaluator during testing, which is equivalent to using test labels during the evaluation process; At the same time, the absolute scores of each benchmark are very low (such as MMSearch Plus about 5%), and the main table does not provide the variance of multiple runs. TIR Bench also uses separately adjusted search hyperparameters, making it difficult to determine whether the differences between 1-3 points are reliable.
Questions for authors
  • What is the difference between Ď€ Skill w/o Memory and the w/Tools baseline in Table 1? What are the advantages of Ď€ Skill if all memory baselines use the same basic agent configuration?
  • Is the confidence level updated during the testing phase using the true label of the benchmark? What would be the result if testing updates were turned off and only accumulated memory was used? Did the baseline method also receive the same feedback?
  • What is the standard deviation of the main table results between multiple runs?
1 comment

Thank you for your careful review and helpful comments. We address your questions and concerns below.

1. The contribution of memory

The w/ Tools baseline uses tools without external experience. The πSkill w/o Memory setting removes external experience from the πSkill pipeline while keeping the base agent and tools. We use this ablation to measure the gain from memory within the same pipeline.

Table R1. Original macro averages.

MethodAverage@4Pass@4
w/ Tools11.4025.47
XSkill12.6725.28
Ď€Skill w/o Memory14.1931.03
Dynamic Cheatsheet DC14.4331.88
Ď€Skill15.3233.33

DC scores above the no memory variant in the original table, and complete πSkill scores above both. πSkill gains 1.13/2.30 points over no memory, 0.89/1.45 over DC and 2.65/8.05 over XSkill in Average@4/Pass@4.

CSD extracts local lessons, and CMU maintains their confidence and availability. Adding them separately to XSkill raises Average@4 from 12.67 to 13.25 and 12.82, respectively. Complete πSkill reaches 15.32/33.33, as shown in Table R1.

The new controls in Table R2 share the backbone, base prompt, tools, decoding settings and executor limits. The no memory and guidance only controls remove both experience and skill. Guidance only keeps the same output limits for the two auxiliary stages as the memory pipeline.

2. Test feedback and frozen evaluation

Online CMU uses the benchmark evaluator's success signal. In the new online experiment, updates occur after all four attempts on a task finish and affect later tasks only.

Frozen evaluation uses outcomes only for scoring. The bank, confidence metadata and skill remain unchanged. Four attempts share guidance and use different executor seeds. Unchanged hashes and zero updates confirm that earlier attempts and tasks do not change the memory used later.

Table R2. New VisualToolBench controls.

SettingAverage@4Pass@4Recorded confidence updates
No historical memory, with tools9.2323.420
Guidance without historical memory9.0115.320
Frozen bank, decomposition and rewriting enabled8.0020.270
Same initial bank, online updates enabled9.4622.97231

These experiments use an updated executor and the same initial confidence for all entries. Their scores are reported separately from the original results.

All frozen controls follow the same policy of no updates. Online feedback gives an observed gain of 1.46/2.70 points over the frozen bank. Frozen evaluation and online adaptation are reported separately.

3. Uncertainty and task performance

We calculate paired confidence intervals from the saved task outcomes. We use 10,000 bootstrap samples, keeping each task's four outcomes together and comparing methods on the same sampled tasks. These intervals describe uncertainty from sampling tasks, rather than the standard deviation across independent runs of the main experiments.

Table R3. Paired differences between πSkill and its no memory variant in saved runs.

BenchmarkΔ Average@4 [95% CI]Δ Pass@4 [95% CI]
VisualToolBench+1.94 [-0.97, +4.86]+1.11 [-6.11, +8.89]
TIR-Bench+0.88 [-3.00, +4.63]+1.00 [-6.50, +8.50]
MMSearch-Plus+0.12 [-1.79, +2.14]-0.48 [-4.29, +3.33]
AgentVista+2.06 [-1.38, +5.50]+8.26 [+0.92, +15.60]

VisualToolBench has 222 test tasks. The paired analysis uses the 180 common tasks with complete saved outcomes. The intervals describe uncertainty from sampling tasks.

AgentVista Pass@4 improves by 8.26 points, with a 95% interval of [0.92, 15.60]. Average@4 differences are positive across the benchmarks, although their intervals include zero.

On MMSearch-Plus, πSkill scores 5.71/13.33 and XSkill scores 4.62/11.00. Table R3 shows the smaller difference from πSkill's own no memory variant.

4. TIR-Bench retrieval settings

The original TIR-Bench setup uses threshold 0.5, retrieves five entries and allows a skill budget of 300 words. The other benchmarks use threshold 0.35, retrieve one entry and allow 1,000 words. These settings control retrieval relevance, the number of entries and context size. The backbone, tools and limit of 20 interaction turns are shared.

The existing sensitivity study changes one parameter at a time. Table R4 gives the means of three runs on TIR-Bench when only the number of retrieved entries changes.

Table R4. Existing TIR-Bench sensitivity results.

Retrieved entriesAverage@4Pass@4
127.2551.50
327.3353.33
526.7951.50
Reviewer_1dh4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper combines cross-rollout critique with local analysis of individual trajectories to extract procedural lessons for multimodal agents. A deterministic admission rule filters candidates, and outcome-based confidence and lifecycle metadata control later retrieval. At inference time, the system decomposes a task, retrieves experiences, and rewrites them for the current query and image. Experiments on four benchmarks, additional backbones, and a transfer setting show improvements over XSkill, although the advantage over Dynamic Cheatsheet is smaller and the effects of the proposed components vary by task.

Critical strengths
  1. The paper treats retrieval mistakes and stale procedural guidance as concrete problems. Limiting admission and suppressing unreliable entries are sensible safeguards when one trajectory can produce several plausible but misleading lessons.
  2. The comparison includes several memory baselines under shared accumulation splits and tool limits. The main improvement over XSkill is substantial: macro-average Pass@4 rises from 25.28 to 33.33. Results with additional backbones suggest that the complete pipeline is not tied to the smallest model.
  3. The paper reports counterevidence rather than hiding it. CSD hurts AgentVista when added alone, Dynamic Cheatsheet remains better on MMSearch-Plus and cross-task transfer, and the reported turn reduction explicitly excludes auxiliary calls. These qualifications are useful.
Critical weaknesses
  1. The ablations leave the distinctive memory contribution uncertain. The no-memory version already reaches 14.19/31.03 in macro-average Average@4/Pass@4, compared with 15.32/33.33 for the full method and 12.67/25.28 for XSkill (Table 2). Removing decomposition and rewriting returns Average@4 exactly to the XSkill level. Thus much of the improvement could come from task conditioning or inference scaffolding. Decomposition and rewriting should be separated, with a control that produces comparable guidance without retrieved experience.
  2. Evaluation feedback may introduce dependence between trials. Appendix A says that completed-task success signals update lifecycle metadata after evaluation rollouts, while the paper also describes four rollouts as independent. It is unclear whether a result from rollout one changes the memory seen by rollout two, or whether earlier evaluation tasks affect later ones. This is particularly consequential for Pass@4 and for a benchmark-evaluator signal unavailable during ordinary deployment. A frozen-memory test and an explicitly separate online-learning protocol are needed.
  3. The evidence for improved extraction density is indirect. CSD adds only a small macro-average gain on its own, and the manuscript does not report useful lessons per trajectory, memory size, retrieval utilization, or independently judged rule validity. The central single-trajectory motivation also deserves a controlled one-rollout accumulation experiment, since the default system includes cross-rollout critique.
  4. The strongest baseline gap is modest: 0.89 points in Average@4 and 1.45 in Pass@4 over Dynamic Cheatsheet. The main table gives no uncertainty, and the appendix does not state the evaluation task counts per benchmark explicitly. The three-run sensitivity study does not quantify uncertainty in the main comparison. Fewer agent turns also need to be balanced against accumulation, decomposition, and rewriting costs before an overall efficiency advantage can be assessed.
Questions for authors
  1. Are memory snapshots reset between the four evaluation rollouts? When do success labels update confidence, and which later predictions can depend on them? Please report a fully frozen-memory comparison.
  2. How do decomposition-only, rewriting-only, and guidance-generation-without-memory controls perform under matched inference budgets? What remains of the gain with only one accumulation trajectory per task?
  3. Can you provide evaluation sample counts, paired uncertainty for the comparison with Dynamic Cheatsheet, memory-quality/utilization statistics, and total token costs including the auxiliary stages?
2 comments

Thank you for your careful review and helpful comments. We address your questions and concerns below.

1. The contribution of memory

CSD changes how local lessons are extracted, and CMU changes how their confidence and availability are maintained. Decomposition and rewriting use the stored memory to guide the current task.

Table R1. Original macro averages.

MethodAverage@4Pass@4
XSkill12.6725.28
Ď€Skill without memory14.1931.03
Dynamic Cheatsheet DC14.4331.88
Ď€Skill15.3233.33

The complete πSkill gains 2.65/8.05 points over XSkill and 1.13/2.30 points over its own no memory variant in Average@4/Pass@4.

The new VisualToolBench controls separate decomposition and rewriting while keeping the bank, skill, tools and executor limits fixed. The no memory and guidance only controls remove both experience and skill. Guidance only keeps the same output limits for the two auxiliary stages. Actual token use is reported in Table R5.

Table R2. New VisualToolBench controls.

SettingDecompositionRewritingAverage@4Pass@4
No historical memory, with toolsNot usedNot used9.2323.42
Guidance without historical memoryTask planningGuidance generation9.0115.32
Frozen bank D0R0OffOff8.2220.72
Frozen bank D1R0OnOff10.4722.07
Frozen bank D0R1OffOn8.3318.47
Frozen bank D1R1OnOn8.0020.27
Same initial bank, online updatesOnOn9.4622.97

2. How feedback is isolated

Frozen evaluation keeps the bank, confidence metadata and skill unchanged. Four attempts share task guidance and use different executor seeds. Outcomes are used only for scoring. Unchanged hashes and zero updates confirm that earlier attempts and tasks cannot change later memory.

Online evaluation updates confidence using benchmark outcome feedback after all four attempts on a task finish. Updates affect later tasks only. This run records 231 updates and scores 1.46/2.70 points above frozen D1R1 in Table R2. All frozen controls follow the same policy of no updates. Online adaptation is reported separately.

3. Local extraction and the memory it produces

CSD's local branch reads one trajectory and its outcome. It can therefore propose lessons without comparing several trajectories. When several trajectories are available, cross rollout critique adds those comparisons.

In the original experiments, adding CSD raises macro Average@4 from 12.67 to 13.25. Adding CMU raises it to 12.82. These gains use the default accumulation pipeline with several rollouts per task.

Admission requires a clear condition and an executable action. It filters advice that is vague or tied to a particular sample, and removes near duplicates. The threshold is 0.72. At most one local item is admitted per rollout group, with further limits on additions within a batch.

Table R3. Statistics of the supplied memory snapshots.

BenchmarkStored experience entriesMean words per entryTotal experience words
VisualToolBench13220.172,663
TIR-Bench12818.812,408

4. Size of the gain and paired uncertainty

Against the strongest baseline DC, πSkill improves Average@4/Pass@4 from 14.43/31.88 to 15.32/33.33, a gain of 0.89/1.45 percentage points. The original ablation also shows a gain of 1.13/2.30 points over the same pipeline without memory. This comparison isolates the added benefit of experience within the πSkill pipeline.

The test sets contain 222 VisualToolBench tasks, 200 TIR-Bench tasks, 210 MMSearch-Plus tasks and 109 AgentVista tasks, with four attempts per task. Each benchmark has 100 separate accumulation tasks. All methods share the same split.

To assess the reliability of the memory gain, Table R4 reports paired intervals against πSkill without memory. We use 10,000 bootstrap samples of tasks, keeping each task's four outcomes together and comparing both methods on the same sampled tasks. This preserves dependence between attempts on the same task.

Table R4. Paired differences between πSkill and its no memory variant in saved runs.

BenchmarkΔ Average@4 [95% CI]Δ Pass@4 [95% CI]
VisualToolBench+1.94 [-0.97, +4.86]+1.11 [-6.11, +8.89]
TIR-Bench+0.88 [-3.00, +4.63]+1.00 [-6.50, +8.50]
MMSearch-Plus+0.12 [-1.79, +2.14]-0.48 [-4.29, +3.33]
AgentVista+2.06 [-1.38, +5.50]+8.26 [+0.92, +15.60]

5. Costs including auxiliary calls

The reported 37% reduction refers to agent interaction turns. Table R5 counts decomposition, rewriting and execution calls. CSD runs when building memory, and CMU updates numeric metadata. Neither adds a model inference stage beyond the existing pipeline.

Table R5. Average inference cost per task on VisualToolBench.

SettingModel callsInput tokensOutput tokensSummed generation time in seconds
No Tools5.3619,5921,16620.55
No historical memory, with tools6.7931,8112,08041.14
Guidance without historical memory10.9458,2483,51467.95
Frozen bank D0R013.2275,0652,95060.29
Frozen bank D1R015.0286,1283,00460.51
Frozen bank D0R112.0064,5143,49262.58
Frozen bank D1R114.4579,9173,57566.92
Same initial bank, online updates14.0976,7613,60669.12

Memory construction produces a reusable bank whose cost is shared across later tasks. Table R5 measures the guidance and execution work after construction.

Reviewer_RMZVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents πSKILL, a training-free framework for reusing experience in multimodal agents. It combines Circular Step-level Knowledge Distillation (CSD), which extracts local operational lessons from trajectories, with Confidence-aware Memory Updating (CMU), which adjusts memory confidence and availability using execution outcomes.

At inference time, the framework decomposes tasks into retrieval queries and rewrites retrieved experiences into task-specific guidance. Evaluation covers four multimodal benchmarks, component ablations, additional backbones, and cross-task transfer.

With Qwen2.5-VL-7B-Instruct, πSKILL achieves macro-average Average@4/Pass@4 of 15.32/33.33, compared with 12.67/25.28 for XSkill and 14.43/31.88 for Dynamic Cheatsheet, the strongest baselines in Table 1.

Critical strengths
  1. The paper addresses a meaningful problem and provides evidence beyond a single benchmark.
  • The Gemini-2.5-Pro results are particularly encouraging: Ď€SKILL achieves 27.73/48.02 Average@4/Pass@4, compared with 23.82/40.96 for Agent-KB (Appendix C, Table 5). These results suggest that the benefits are not restricted to the small backbone used in the main experiments.
  1. The component analysis exposes meaningful limitations.
  • Table 2 shows that CSD and CMU have task-dependent effects, rather than uniformly improving performance. The paper also acknowledges weaker results on search-oriented tasks and limited cross-task transfer. These observations help identify the conditions under which procedural memory may be useful.
Critical weaknesses

The novelty over XSkill is limited, and the experiments do not sufficiently isolate the contribution of the new components.

  • XSkill already includes task-decomposed retrieval and visually conditioned experience rewriting (§§2.3.1–2.3.2). Consequently, the large degradation from removing decomposition/rewriting in Table 2 mainly establishes the importance of mechanisms inherited from prior work.
  • The actual additions—local trajectory analysis and confidence/lifecycle management—have modest standalone effects. Adding CSD increases macro-average Average@4 from 12.67 to 13.25, while adding CMU increases it to 12.82. The paper does not report removing each component individually from the complete inference pipeline, which would establish their marginal contributions more directly.
  • The interpretation of these components also needs stronger support. CSD uses outcome-conditioned reflection without external step-level supervision (§3.3), while CMU updates memory confidence using overall execution success (§3.4). The authors appropriately acknowledge that this is not causal credit assignment. Nevertheless, the evidence does not establish whether CMU identifies harmful memories or instead penalizes useful memories associated with difficult tasks or executor failures. This distinction is central to the claimed improvement in memory reliability.

The evidence for practically meaningful, robust, and cost-effective improvement remains insufficient (Old model part is critical to me!)

  • Against the strongest baseline, DC, the main Avg@4 improvement is only 0.89 percentage points (14.43 → 15.32), rather than the 2.65-point improvement highlighted against XSkill. Avg@4 averages individual rollout outcomes and therefore estimates single-attempt success under the evaluation protocol. The full system also exceeds its own no-memory variant by only 1.13 points (Table 2). Without uncertainty estimates for the main comparisons, it is difficult to assess the stability of these gains.
  • The main experiments rely on an older 7B backbone. Although model age alone is not a methodological flaw, the additional results do not establish consistently improved performance over simple alternatives: with Qwen3-VL-32B, Ď€SKILL scores 13.84/26.66, below 14.26/28.11 for No Tools (Table 5). The positive Gemini results are important, but the broader applicability claim should reflect this mixed evidence, and also, it's pretty old model already (it's like 4 generation ago for Gemini..)
  • The reported 37% reduction in interaction turns excludes decomposition and rewriting calls (§4.7). This supports a narrower interaction-efficiency claim, but does not establish lower total inference cost. The small main-table gains should be evaluated alongside the additional model calls, tokens, and latency.
Questions for authors
  • Do the four rollouts start from identical, isolated memory snapshots, or can feedback from one rollout affect another attempt on the same task? Do all memory-based baselines receive equivalent evaluation-time feedback?

  • What are the total inference calls, token usage, and latency, including decomposition and rewriting, relative to DC and XSkill?

2 comments

Thank you for your careful review and helpful comments. We address your questions and concerns below.

1. What CSD and CMU contribute

Ď€Skill adds CSD and CMU to XSkill. CSD extracts local lessons from trajectories. CMU checks their reliability as they are reused. Together, they address how experience enters memory and how it is maintained. Decomposition and rewriting provide the existing mechanisms for using that memory.

Table R1. Original component additions and complete system.

SettingMacro Average@4Gain over XSkill
XSkill12.67Reference
XSkill + CSD13.25+0.58
XSkill + CMU12.82+0.15
Ď€Skill15.32+2.65

Adding CSD or CMU alone improves Average@4. The complete πSkill reaches 15.32/33.33 in Average@4/Pass@4, a gain of 2.65/8.05 points over XSkill. The separate additions show each module's effect on XSkill, while the complete result shows the performance of the full pipeline.

CSD reads the recorded actions and final outcome. It does not require a correct answer for each step. A failed trajectory can still contain useful operations from which CSD extracts a lesson and the conditions for using it.

CMU reviews entries using their accumulated outcomes. An entry starts at confidence 0.7 and drops to 0.6 after one failure. Review requires at least three failures and more failures than successes. This avoids judging an entry from a single failed task. The score guides retrieval and maintenance. It does not identify the cause of failure, which can also involve task difficulty or execution errors.

2. How large and stable the gains are

In the original results, πSkill gains 0.89/1.45 points over DC, which scores 14.43/31.88. It gains 1.13/2.30 points over its no memory variant, which scores 14.19/31.03. Average@4 measures success across individual attempts. Pass@4 measures whether at least one of four attempts succeeds.

We also compare πSkill with its no memory variant using 10,000 paired bootstrap samples. Each task's four outcomes stay together, and both methods are compared on the same sampled tasks.

Table R2. Paired differences between πSkill and its no memory variant in saved runs.

BenchmarkΔ Average@4 [95% CI]Δ Pass@4 [95% CI]
VisualToolBench+1.94 [-0.97, +4.86]+1.11 [-6.11, +8.89]
TIR-Bench+0.88 [-3.00, +4.63]+1.00 [-6.50, +8.50]
MMSearch-Plus+0.12 [-1.79, +2.14]-0.48 [-4.29, +3.33]
AgentVista+2.06 [-1.38, +5.50]+8.26 [+0.92, +15.60]

3. Results with other backbones

To examine performance beyond the main 7B backbone, we also evaluated πSkill on Gemini-2.5-Pro and Qwen3-VL-32B without updating model weights. On Gemini-2.5-Pro, πSkill reaches 27.73/48.02 in Average@4/Pass@4, compared with 23.82/40.96 for Agent-KB, a gain of 3.91/7.06 percentage points. This shows that experience reuse can also benefit a stronger backbone.

On Qwen3-VL-32B, πSkill scores 13.84/26.66, compared with 14.26/28.11 for No Tools. Together, these results show that πSkill can deliver gains beyond the main 7B model, while its effectiveness varies across backbones.

4. Whether attempts share feedback

The paper allows completed task outcomes to update lifecycle metadata. We separate this online setting from frozen evaluation in the new VisualToolBench experiments.

In frozen evaluation, the memory bank, metadata and skill stay unchanged. Four attempts share the same task guidance and use different executor seeds. Outcomes are used only for scoring. Unchanged hashes and zero updates confirm that earlier attempts and tasks do not change the memory used later.

In online evaluation, confidence is updated only after all four attempts on a task finish. Feedback can affect later tasks, but not another attempt on the same task.

Table R3. New VisualToolBench results under frozen and online evaluation.

SettingAverage@4Pass@4Recorded confidence updates
No historical memory, with tools9.2323.420
Frozen bank, decomposition and rewriting enabled8.0020.270
Same initial bank, online confidence updates enabled9.4622.97231

These experiments use an updated executor and the same initial confidence for all entries. Their scores are reported separately from the original results.

The online run records 231 updates and scores 1.46/2.70 points above the frozen bank. All frozen controls follow the same policy of no updates. Online adaptation is reported separately.

5. What the cost figures include

The reported 37% reduction refers to agent interaction turns. Table R4 also counts the calls used to generate guidance. CSD runs when building memory, and CMU updates numeric metadata. Neither adds a model inference stage beyond XSkill's decomposition and rewriting pipeline.

Table R4. Average model use per task, including four attempts.

SettingModel callsInput tokensOutput tokensSummed generation time in seconds
No Tools5.3619,5921,16620.55
No historical memory, with tools6.7931,8112,08041.14
Guidance generated without historical memory10.9458,2483,51467.95
Frozen bank, decomposition and rewriting enabled14.4579,9173,57566.92
Same initial bank, online confidence updates enabled14.0976,7613,60669.12

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.