acceptodds
Under review as a conference paper at ICLR 2027

Daedalus: Bootstrapping Agent Memory from Self-Generated Tasks

15 views

Abstract

LLM agents often lack the operational knowledge to act reliably in new environments, as they must discover specific tool behaviors or environment conventions on their own. Without memory of past attempts, they repeat the same mistakes across tasks, leading to more task failures and longer trajectories. To address this, agentic systems typically rely on human-written guidelines or on procedural memory built from training tasks and an oracle verifier, both of which require prior knowledge of the environment. We present Daedalus, a method for bootstrapping reusable agent memory from self-generated practice without existing tasks or oracle verifiers. Daedalus pairs two agents: an explorer that interacts with the environment to generate challenging yet solvable tasks, and a solver that attempts them. A heuristic is derived from each solver failure and accepted only after the solver repeatedly succeeds with that heuristic in context. These outcomes also provide feedback for the explorer to refine the difficulty of future tasks. The accepted heuristics are then consolidated into a memory bank for test-time use. Across AppWorld, -bench, and AutomationBench, Daedalus improves mean success rates by up to 15.9 percentage points and pass^5 by up to 2.2 over a no-memory baseline, and is competitive with methods using training tasks, at a lower inference cost than most. We show that performance gains already emerge with a small exploration budget, and that its heuristics also benefit agents from other model families. Our ablations further reveal that solver traces provide the key information needed to derive effective heuristics, while factorizing early discoveries makes exploration more cost-efficient. Beyond memory construction, we find that the tasks generated by Daedalus can serve as a proxy for benchmark tasks when ranking models by performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent memory, self-improvement, task generation, environment exploration, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5pi4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces DAEDALUS, a framework for constructing reusable procedural memory before deployment, without curated task instructions or an oracle verifier during memory generation. An Explorer proposes and executes synthetic tasks, while a Solver attempts them in reset environments. Following failures, an Extractor produces or revises textual heuristics. A heuristic is retained after the Solver achieves three consecutive judged successes. Solver outcomes also guide task-difficulty refinement. Accepted heuristics are consolidated into a frozen memory bank for downstream use. Experiments cover AppWorld’s normal split, the retail domain of τ²-bench, and the Operations split of AutomationBench. The improvement validates the efficiency of the proposed methods.

Critical strengths
  1. The presentation and writing are good and easy to follow.

  2. The investigating problem that constructing reusable procedural memory is very important.

  3. Table 1 shows improvements over no memory across three different environments, including gains in repeated run reliability, which strengthens the methods' credibility.

  4. The reported deterioration at larger generation budgets is also informative and appropriately limits the paper’s scaling claims.

Critical weaknesses
  1. DAEDALUS uses substantially more solver rollouts than most baselines, leaving its advantage under comparable computation unclear. So could the authors provide the computation cost comparison?

  2. The ablations change retries, heuristic revision, and validation together, making the specific benefit of repeated-success filtering unclear.

Questions for authors
  1. What are PREPING’s total memory construction cost and environment interaction budget? Under comparable construction budgets, does DAEDALUS retain its advantage?

  2. How much do the main AppWorld result and the solver loop ablation vary across independently generated and consolidated banks?

3 comments

We thank the reviewer for their feedback, and for appreciating the clarity of the paper, the importance of reusable procedural memory, the consistent gains across three environments, and the scaling experiment. We address each concern below. All changes in the revised paper are marked in red.

W1. Computation cost

We agree that the comparison needs the cost of every method, which the revision now reports, together with solver rollouts (Appendix F). DAEDALUS costs more to generate than the baselines: 142 USD on AppWorld for 1,500 judged solver rollouts, against at most 18 USD for the baselines. This cost is paid once, however, whereas inference is paid on every task. DAEDALUS is the cheapest method at inference on AppWorld and τ², as cheap as the no-memory agent (Table 1), so its total cost per solved task falls below that of every baseline after enough task runs (Table 16).

Changes to the paper. Section 4.2; Appendix F, Tables 15–16 (costs, rollouts, break-even).

W2. The ablation changes three factors together

We agree that row D of Table 3 does not isolate repeated-success filtering. These components are hard to separate, because they only work together: a heuristic can only be revised after a failed retry, and only validated through further retries. We therefore evaluate the Solver loop as a whole.

Changes to the paper. Section 5.1 now attributes the gain to the Solver loop as a whole.

Q1. PREPING's cost and comparison at equal budget

PREPING runs 100 practice tasks, for at most 16 USD on AppWorld. To compare the two methods at equal cost, we stop DAEDALUS's generation on AppWorld once it reaches PREPING's budget, and consolidate only the heuristics accepted up to that point, for each of three banks. DAEDALUS then matches PREPING with a higher mean (51.1 ± 1.1 vs. 47.1 ± 5.3 MSR) and five times less variation across banks.

Changes to the paper. Appendix F (comparison at equal budget).

Q2. Variance across independently generated banks

We now build 3 independent memory banks for every method and benchmark, and Table 1 reports the mean ± SE across banks. On AppWorld, the main result of DAEDALUS is 56.8 ± 2.7 MSR across banks. DAEDALUS still scores above PREPING on AppWorld and τ² (56.8 ± 2.7 vs. 47.1 ± 5.3, and 70.7 ± 2.0 vs. 58.8 ± 1.4), and matches it on AutomationBench (34.6 ± 1.6 vs. 32.7 ± 1.1). Due to budget constraints, we could not also re-run the full ablation across banks, so Table 3 still reports one bank per row.

Changes to the paper. Table 1 and Section 4.1.

We thank the reviewer again for their positive assessment and useful feedback. We hope that the cost comparison, the comparison at equal budget and the results over three banks address their remaining concerns.

Reviewer_5pi4OpenReview ↗

Response to the Rebuttal

Thanks for your careful rebuttal. My concerns are addressed and I am willing to maintain my positive score to support this paper to be accepted.

We thank the reviewer again for their support throughout the discussion. Their questions on cost and comparable budgets led to the generation-cost comparison of Appendix F, which clarifies the trade-offs of our method.

Reviewer_oqVNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper “Daedalus: Bootstrapping Agent Memory from Self-Generated Tasks” introduces a training-free method called DAEDALUS. The method builds a memory bank of useful context information in an environment with no curated training tasks or oracle verifier, with the intention of increasing success, reliability and reducing the cost, for tasks carried out in that environment. In DAEDALUS, the Explorer proposes tasks with success conditions, the Solver attempts them, and the LLM Judge grades the attempts. The primary contribution of DAEDALUS is the construction of the memory bank exclusively from traces of tasks the Solver attempted and first failed to carry out correctly, but was able to solve with additional contextual information provided by the Extractor several times in a row. DAEDALUS is evaluated against baseline memory methods on three benchmarks and emerges as the method with the highest scores among methods that use self-generated tasks and is competitive with methods that use curated training tasks. Furthermore, the transfer of a memory bank built by one model family to other model families in the downstream application is shown, by comparing the transfer of memory banks to the baseline and same model combinations. Finally, the ablation study dissects the design choices of DAEDALUS and measures the contribution of each pipeline component. To this end, the components are evaluated cumulatively on the benchmark AppWorld.

Critical strengths

The proposed method is explained in full detail. The ablation study provides clarity about the contribution of the main pipeline components. By comparing to competitor methods, the article shows that DAEDALUS, using only self-generated tasks, comes close to the best methods that use curated training tasks. This suggests that curated training tasks are not strictly necessary to build memory for agents in unknown environments.

Critical weaknesses

W1: The reported errors underestimate the uncertainty of the results. They only include the deviation between repeated task runs, but not the randomness of building the memory bank, which is done only once in the presented experiments, nor the uncertainty from the small test sets. Even with these optimistic errors, several conclusions rest on differences below 2σ. Examples are the gain over no memory on AutomationBench (+4.3 MSR, 1.2σ), most comparisons to PREPING, and the step that adds validation in the ablation study (Table 3, C→D: +4.5 MSR, 1.4σ), which the article counts among its "two substantial jumps". Therefore, the claims that DAEDALUS "consistently outperforms PREPING" and that validation is "the main reason behind this improvement" are not supported by the experiments.

W2: All main experiments use a large model for the auxiliary agents and a small model for the Solver and the base agent. When all agents run on the small model, MSR on AppWorld drops by 9.0 points, and the gain over no memory shrinks to "about half of the gain obtained with GPT-5.4" (Table 5). The article attributes most of this drop to the auxiliary roles other than the Extractor (Table 8), i.e., the Surveyor, Explorer, Judge and Consolidator, most of which generate and judge the tasks that are the core of the method. The article claims that DAEDALUS "remains useful when a larger model is not available", but this setup is tested only for DAEDALUS. It is therefore unknown whether DAEDALUS still outperforms the baselines there. The performance of GPT-5.4 itself as the agent is also never reported, so it is unclear how much of the gain is knowledge transferred from the larger model rather than memory the agent builds "by practicing". Therefore, the experiments cannot fully separate the effect of the method from the effect of the setup.

Questions for authors

Q1: Could you build several memory banks independently for DAEDALUS and the baselines, and report the variance across banks?

Q2: Could you run the baselines in the small/small setup of Table 5, to test whether DAEDALUS still outperforms them when a larger model is not available? Could you also report GPT-5.4 itself as the agent, with and without DAEDALUS memory?

3 comments

We thank the reviewer for their feedback, and for appreciating our pipeline analysis, the clarity of our ablations, and the finding that tasks can be generated instead of curated. We address both concerns below, merging each weakness with the related questions. All changes in the revised paper are marked in red.

W1 & Q1. Uncertainty and variance across banks

We agree that our original error bars covered only repeated inference runs, not the randomness of building the memory bank. We now build 3 independent memory banks for every method and benchmark, and Table 1 reports the mean ± SE across banks. With these errors, DAEDALUS still scores above PREPING on AppWorld and τ² (56.8 ± 2.7 vs. 47.1 ± 5.3, and 70.7 ± 2.0 vs. 58.8 ± 1.4), matches it on AutomationBench (34.6 ± 1.6 vs. 32.7 ± 1.1), and remains level with ExpeL on AppWorld and τ² (57.3 ± 1.7 and 70.5 ± 2.8).

We also rephrased the two claims that the reviewer points out.

Changes to the paper. Table 1 and Sections 4.1, 4.2 and 5.1.

W2 & Q2. Separating model size from the method

We agree that the small-model setup was tested only for DAEDALUS, and that GPT-5.4 itself was missing as the agent. We ran both experiments.

Small models only. We re-ran PREPING, ACE and ExpeL on AppWorld with every auxiliary role on GPT-5.4-mini, as in Table 5, over 3 memory banks each.

MethodTraining tasksMSR
No memory–44.3 ± 1.1
ExpeLyes57.4 ± 1.8
ACEyes63.8 ± 2.0
PREPINGno48.2 ± 2.8
DAEDALUSno50.9 ± 0.2

Without a larger model, DAEDALUS still improves the no-memory agent by 6.6 points and matches PREPING with a higher mean. ACE and ExpeL, which learn from 90 human-written training tasks, depend less on the auxiliary model, because in DAEDALUS this model also writes and judges the practice tasks.

GPT-5.4 as the agent. We give the first DAEDALUS bank of Table 1 to GPT-5.4 itself (AppWorld, 5 runs). It raises the MSR of GPT-5.4 from 69.6 ± 1.1 to 77.6 ± 1.0, and its pass^5 from 38.7 ± 3.8 to 57.1 ± 3.8. If the bank only held GPT-5.4's own knowledge, GPT-5.4 would gain nothing from it: the bank therefore holds knowledge that the Solver gained by practicing on the generated tasks.

Changes to the paper. Table 5 adds the baselines; Appendix H adds the GPT-5.4 agent.

We thank the reviewer again for their useful feedback. With errors across three banks and the small-model and GPT-5.4 comparisons, we hope the revision now separates the effect of the method from that of the setup, and we would be grateful if the reviewer reconsidered their score.

I thank the authors for the rebuttal, and my questions have been clarified. I updated my rating to weak acceptance.

We thank the reviewer for their reply and for adjusting their score in light of the revised paper. Their request for variance across memory banks led to the three-bank evaluation now used throughout Table 1, which makes the paper's conclusions more robust.

Reviewer_kE3UOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Daedalus, a method that builds reusable memory for an LLM agent from practice tasks it generates itself, without explicit raining tasks or an oracle verifier. During memory construction, an Explorer agent probes the environment and proposes a set of feasible tasks with explicit success conditions, which a Solver then attempts while an LLM judge checks the result. After each failure, an Extractor writes or revises a heuristic, which is kept only once the Solver succeeds several times in a row with it in context, and tasks that prove too easy or too hard go back to the Explorer to adjust their difficulty. The accepted heuristics are then consolidated into one bank that the base agent receives at the start of every test task. The authors show that Daedalus improves success over a no-memory baseline on AppWorld, τ²-bench retail, and AutomationBench, outperforms PREPING, which also uses self-generated tasks, and is competitive with methods that learn from training tasks. They also show that the heuristics transfer across model families and that the generated tasks rank models consistently with the benchmark test set.

Critical strengths
  1. The task-generation loop is the most interesting part of the paper: an Explorer probes the environment and tunes task difficulty to the Solver, and a heuristic is kept only once it turns a failure into repeated success.
  2. Consistent improvements across benchmarks: Daedalus improves success over the no-memory agent on all three benchmarks, with no extra inference cost on AppWorld and τ²-bench.
  3. The analysis and ablations are well designed: they show that memory from exploration alone hurts, that giving the agent the whole bank up front works better than retrieving heuristics at each turn, and that the heuristics transfer across model families.
Critical weaknesses
  1. Baseline results are inconsistent with the original papers, and there are no reruns to separate signal from noise. The five baselines that use training tasks are reimplemented with changes (p. 19); for example, ACE runs with add-only updates and no success signal, and ExpeL uses L = 3 instead of 4–8. The rankings also differ from the original papers: PREPING scores below ACE here (56.0 vs. 60.5), while PREPING's own paper reports the reverse (83.7 vs. 82.7, with a different model). Each method is also evaluated with a single memory bank, so the error bars reflect only inference runs, and it is hard to tell how much of the gap between methods is noise.
  2. The setting is less task-free than the framing suggests. Each AppWorld session runs in "the initial world of one training task", AutomationBench generation "uses only the worlds of the 30 training tasks" and AppWorld generation excludes the two apps that appear only in the challenge split. These seeds may steer the Explorer toward the test distribution and there isn't any setting which brings out true task-free setting - which makes it hard to close the imagination on what kind of tasks a data-free techique like this can solve/
  3. The positioning omits prior work that generates tasks from environment interaction. The paper calls PREPING "the only existing method that also builds memory from self-generated tasks instead of training tasks" (p. 4), yet Learn-by-Interact [1] and BAGEL [2] also synthesize tasks through interaction, without human supervision, and use them in context.

References [1] H. Su et al. Learn-by-Interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments. ICLR 2025. https://arxiv.org/abs/2501.10893 [2] S. Murty et al. BAGEL: Bootstrapping Agents by Guiding Exploration with Language. ICML 2024. https://arxiv.org/abs/2403.08140

Questions for authors

Q1. Could you run ACE or ExpeL on the tasks Daedalus generates, with the LLM judge as their success signal, to separate the generated tasks from the Solver loop? Why was a prompt optimizer such as GEPA [3] not considered for the memory part? It also proposes and tests prompt updates on rollouts, and could use the same tasks and judge.

Q2. What happens if generation starts from a single default world, or from worlds not tied to training tasks, and without excluding the challenge-split apps? PREPING's paper separates a pre-task setting, limited to environment documentation and self-generated interaction, from a task-seeded variant. A similar comparison would show what Daedalus needs.

Q3. What are the generation costs (dollars, and rollouts including Explorer and judge calls) of each baseline, including PREPING? What are Daedalus's generation costs on τ²-bench and AutomationBench? Does consolidating the baselines' memories with your Consolidator change their scores?

Q4. How were the baseline settings chosen, and on which data? Why does PREPING rank below ACE here when its own paper reports the reverse? Is the Table 1 AppWorld bank the 90-session checkpoint in Fig. 6? What is its mean success rate without the seven sessions swapped in from the scaling run (p. 19)?

Q5. Table 4 measures the judge on success conditions that the Explorer restated from each benchmark's oracle requirements. This is likely easier than judging against the conditions it writes for its own tasks. How often are the Explorer's own conditions for generated tasks correct and complete, for example in a manual spot check?

References [3] L. A. Agrawal et al. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. ICLR 2026. https://arxiv.org/abs/2507.19457

4 comments

We thank the reviewer for the detailed review, and for highlighting the novelty of our Explorer–Solver loop, the consistency of our results across benchmarks, and the clarity of our ablation. We address each concern below, merging each weakness with the related question. All changes in the revised paper are marked in red.

W1 & Q4. Baseline settings and variance

ACE. We follow ACE's official AppWorld release. Every released configuration accepts only ADD operations (update, merge and delete are commented out), and the variant without a success signal is the one documented for reproduction and released with a playbook. We also ran ACE with its ground-truth reflector on all three benchmarks: it is not stronger on any of them (on AppWorld, 43.1 MSR with ground truth against 60.5 without, where the reflector infers wrong grading rules from the reference solutions).

ExpeL. We now use L = 4, the value of the ExpeL paper on WebShop. Re-extracting insights with L = 4 from the same rollouts raises its MSR by +1.0 / +5.0 / +1.7 points on AppWorld / τ² / AutomationBench, without changing our conclusions.

Other settings. We use the settings of the original papers and code wherever they apply (Appendix E), without tuning on our benchmarks. PREPING ranks below ACE here, unlike in its own paper, which uses DeepSeek-V3.2 without reasoning: the ranking of memory methods need not transfer across backbones, since their gains depend on what the base agent already solves.

Variance. We now build 3 independent memory banks per method and benchmark, and Table 1 reports the mean ± SE across banks. DAEDALUS still scores above PREPING on AppWorld and τ² (56.8 ± 2.7 vs. 47.1 ± 5.3, and 70.7 ± 2.0 vs. 58.8 ± 1.4), and matches it on AutomationBench (34.6 ± 1.6 vs. 32.7 ± 1.1).

Changes to the paper. Table 1 (3 banks, ExpeL with L = 4); Appendix E (ACE and ExpeL settings); Appendix H, Table 18 (ACE with ground truth).

W2 & Q2. Seed worlds and held-out apps

Two facts limit how much the seed worlds can steer the Explorer. First, a seed world is almost task-independent: all AppWorld worlds share one base database, a world adds only its user's profile and the few records its task needs, and AppWorld splits tasks by scenario, so no test scenario appears among the training worlds. Second, the Explorer is never given a task, solution or evaluation: it explores the world's data and writes its own tasks. A seed world thus plays the role of a user's account at deployment, since it holds data but no tasks.

We tested this on AppWorld, as the reviewer suggested, by running all 90 sessions in a single world, 82e2fac_1, the first world of the training split. Its initial state holds AppWorld's shared base database and its user's profile, with no records for its own task: it is the closest AppWorld offers to a default world. A single world also removes any steering that the 90 worlds could carry through how their databases are instantiated. Gmail and Amazon stay held out, so that they remain out-of-distribution for the challenge split, as AppWorld intends. We also ran one generation in the same world with no app held out.

Both variants match our default setup with the 90 training worlds (56.8 ± 2.7 MSR, Table 1):

  • single world: 55.9 ± 2.4 (three generation runs);
  • single world, no app held out: 57.1 ± 0.8, although Gmail and Amazon appear in 28 of its 88 tasks.

DAEDALUS therefore relies neither on the training worlds nor on the held-out apps, and stays well above PREPING (47.1 ± 5.3) even with a single generic world.

Changes to the paper. Section 4.1 and Appendix E describe the seeding; Appendix C, Table 9 adds these experiments.

W3 & Q1b. Related work and GEPA

We thank the reviewer for these references, and we removed the claim that PREPING is the only existing method that builds memory from self-generated tasks. Learn-by-Interact and BAGEL also synthesize tasks through interaction, but they keep a different kind of memory: they store synthetic demonstrations and reuse them as examples in context, whereas DAEDALUS stores short heuristics, each kept only after it turns a judged failure into repeated judged success. We therefore compare against methods that, like DAEDALUS, extract memory from trajectories.

GEPA is also relevant, since it proposes and tests prompt updates on rollouts. The difference is again in what is validated: GEPA scores whole-prompt candidates on a validation set, whereas DAEDALUS tests each heuristic on the failure it was written for. GEPA could indeed use our generated tasks and judge, but it is costly on agentic benchmarks: on AppWorld, the ACE authors report 204.1M input and 1.9M output tokens for GEPA, against 39.3M and 0.3M for ACE, with lower accuracy on average. We therefore did not add it to our baselines.

Changes to the paper. Section 2 discusses all three methods.

Q1a. Baselines trained on DAEDALUS-generated tasks

We ran this experiment as the reviewer suggested, to separate the generated tasks from our Solver loop. We train ExpeL and ACE on the 81 generated tasks behind the first DAEDALUS bank of Table 1, with our LLM judge as their only success signal, and evaluate them as in Table 1 (AppWorld, 5 runs, ExpeL with L = 3). The first row recalls each method trained on the benchmark's training tasks; the second row uses our generated tasks instead.

Trained onDAEDALUSExpeLACEACE with ground truth
Training tasks + oracle verifier60.8 ± 0.9 (curated)59.0 ± 1.960.5 ± 0.843.1 ± 0.5
Generated tasks + LLM judge60.2 ± 0.950.8 ± 1.451.9 ± 1.153.2 ± 0.6

Both baselines stay above the no-memory agent (44.3), so the generated tasks are useful to other methods. But with the same tasks and judge, DAEDALUS is 9.4 points above ExpeL and 8.3 above ACE, whereas the three methods are within 1.8 points on the training tasks. The gain therefore comes from the Solver loop, which keeps a heuristic only once it has fixed the failure it was written for.

Changes to the paper. Appendix H, Table 19.

Q3. Generation cost and consolidation

Cost. The revision now reports the generation cost and solver rollouts of every method on all three benchmarks (Appendix F). Generation costs 1–18 USD for the baselines, and 142 / 206 / 85 USD for DAEDALUS on AppWorld / τ² / AutomationBench, for 1,500 / 2,430 / 740 judged solver rollouts (100 for PREPING). This cost is paid once, and DAEDALUS is the cheapest method at inference on AppWorld and τ², as cheap as the no-memory agent (Table 1), so its total cost per solved task falls below that of every baseline after enough task runs.

We also evaluated DAEDALUS under the same generation budget as PREPING on AppWorld. It achieves a higher mean success rate (51.1 ± 1.1 vs. 47.1 ± 5.3 MSR) and five times less variation across banks.

Consolidation. As the reviewer suggested, we applied our Consolidator unchanged to PREPING's three AppWorld playbooks, since PREPING is the other method without training tasks. It halves their size but moves their MSR only from 47.1 ± 5.3 to 49.8 ± 5.0, whereas it adds 10.7 points to DAEDALUS's heuristics (Table 6). DAEDALUS stays 7 points ahead, so consolidation alone does not explain its advantage.

Changes to the paper. Section 4.2; Appendix F, Tables 15–16 (costs, rollouts, break-even, equal budget); Appendix H (Consolidator on PREPING).

Q4. The Table 1 bank

The first DAEDALUS bank of Table 1 is indeed the 90-session checkpoint of Figure 6, also used in Table 3 (row F) and Table 6. It contains no heuristic from the seven replacement sessions: it is consolidated only from the 81 heuristics of the original run, so its MSR without them is the reported 60.2 ± 0.9. The replacements affected only the acceptance rate in Table 3, and were unnecessary there: the seven filtered sessions had passed the Solver loop, so the original run alone accepts 88 of 90 tasks (97.8%).

Q5. Quality of the Explorer's success conditions

We agree that Table 4 uses conditions restated from the oracle, and so does not test the conditions that the Explorer writes for its own tasks. We therefore audited these conditions on every accepted task behind the first bank of each benchmark (153 tasks, 428 conditions). A GPT-6.1-Sol verifier checked each condition against the starting environment with read-only tools (correct: faithful to the instruction and true in the environment; complete list: covers every requirement), then reread every judge-accepted attempt on an incomplete list to check whether it would still be accepted with the missing requirements.

AppWorldτ²-benchAutomationBenchAll
Conditions correct95.7%99.3%90.4%96.0%
Lists complete95.1%28.8%60.0%68.0%
Accepted attempts failing a missing requirement0.3%19.6%4.7%7.2%

Of the 17 incorrect conditions, 11 are only over-strict, which can only make the judge reject valid solutions. Incomplete lists concentrate on τ², where the conditions check the database but not what the agent tells the customer: all 43 τ² attempts that fail a missing requirement left the database in the required state and missed only a conversational requirement. The conditions therefore verify the environment, which is what the memory captures. Overall, only 5 of the 663 accepted attempts left the environment in a wrong state. An author also labeled 10 tasks and agrees with the verifier on 25 of 29 conditions and on all 10 lists; in all four disagreements, the verifier was the stricter one.

Changes to the paper. Appendix G, Table 17, summarized in Section 5.1.

We thank the reviewer again for their precise feedback, and would appreciate it if they reconsidered their score in light of the revised paper.

Thank you for the detailed response and the additional experiments. The multiple memory banks, the ExpeL and ACE runs on your generated tasks, and the single-world experiment answer most of my questions.

My main remaining concern is PREPING, the only baseline that, like Daedalus, works without training tasks. On AppWorld it ranks below ACE, the opposite of what its own paper reports, and the gap grew after the reruns. The backbone explanation is plausible, but it has not been tested. Since there is no time left in the discussion to resolve this, I won't hold it against the paper, but please address it in the final version, ideally by running PREPING with its original backbone. I would also be more careful about crediting the Solver loop alone, since the Consolidator was only applied to PREPING. Overall, I am raising my score.

We thank the reviewer for their careful reading of our rebuttal and for adjusting their score in light of the revised paper. We agree on both remaining points, and will address them in the final version: we will run PREPING with its original backbone, to ensure faithfulness to the original paper’s results, and we will temper the attribution of our gains to the Solver loop, stating that consolidation was tested only on PREPING. We are grateful for a review that substantially improved the paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

We thank all three reviewers for their feedback, and for recognizing the clarity of the paper and the consistency of its gains across three benchmarks. We invested significant new resources to run the experiments they requested and to address every weakness raised. We answered each reviewer individually and uploaded a revised paper in which all changes are marked in red. This message summarizes what the revision adds.

On variance across memory banks

All three reviewers ask how much our results vary across independently built memory banks.

Table 1 now reports every method on every benchmark over 3 independently built banks, each evaluated with 5 runs. Scores are MSR, as mean ± SE across banks (across runs for no memory).

Without training tasksAppWorldτ2\tau^2-benchAutomationBench
No memory44.3 ± 1.157.5 ± 1.631.7 ± 3.2
PREPING47.1 ± 5.358.8 ± 1.432.7 ± 1.1
DAEDALUS56.8 ± 2.770.7 ± 2.034.6 ± 1.6

Our conclusions hold: DAEDALUS scores above PREPING on AppWorld and τ2\tau^2-bench, matches it on AutomationBench, and stays competitive with methods that use training tasks. We rephrased the two claims that reviewer oqVN found too strong.

On training worlds

Reviewer kE3U asks whether starting generation from the worlds of training tasks, and holding out two apps, steers the Explorer toward the test distribution.

We ran all 90 AppWorld sessions in a single generic world, with and without the held-out apps (Table 9).

Generation worldsHeld-out appsMSR
90 training worlds (default)Amazon, Gmail56.8 ± 2.7
Single worldAmazon, Gmail55.9 ± 2.4
Single worldnone57.1 ± 0.8 (one bank)

The three setups reach the same mean success rate, which indicates that DAEDALUS does not rely on knowledge from the training worlds and that it works even in the presence of distractor apps that are not used in the test evaluations. A seed world gives the Explorer the environment's data, but never a task, solution or verifier (Section 4.1, Appendix E).

On model size

Reviewer oqVN asks whether the gain depends on the larger auxiliary model, and whether it is knowledge transferred from that model.

Small models only. We re-ran the baselines on AppWorld with every auxiliary role on GPT-5.4-mini, over 3 banks each (Table 5).

MethodTraining tasksMSR
No memory–44.3 ± 1.1
ExpeLyes57.4 ± 1.8
ACEyes63.8 ± 2.0
PREPINGno48.2 ± 2.8
DAEDALUSno50.9 ± 0.2

DAEDALUS still gains 6.6 points over no memory and stays above PREPING on average. ACE and ExpeL, which learn from human-written training tasks, depend less on this model, since in DAEDALUS it also writes and judges the practice tasks.

GPT-5.4 as the agent. The DAEDALUS bank raises the MSR of GPT-5.4 itself from 69.6 ± 1.1 to 77.6 ± 1.0 (Appendix H). It therefore holds knowledge that GPT-5.4 does not have, which the Solver gained by practicing.

Other new experiments

  • Generation cost (kE3U, 5pi4). DAEDALUS costs more to generate: 85–206 USD, against 1–18 USD for the baselines (Appendix F). Since this is a one-time cost and DAEDALUS is the cheapest method at inference on AppWorld and τ2\tau^2-bench, its total cost per solved task falls below that of every baseline after enough task runs (Table 16). Stopped at PREPING's generation budget, it still reaches 51.1 ± 1.1 MSR on AppWorld, against 47.1 ± 5.3 for PREPING.
  • Generated tasks or Solver loop (kE3U). Applied to our generated tasks with our judge, ExpeL and ACE reach 50.8 and 51.9 MSR, against 60.2 for DAEDALUS, whereas the three methods are within 1.8 points on the training tasks (Table 19). The gain comes from the Solver loop.
  • Consolidation (kE3U). Our Consolidator moves PREPING only from 47.1 to 49.8 MSR, still 7 points below DAEDALUS (Appendix H).
  • Baseline settings (kE3U). ExpeL now uses L = 4 as in its paper, and ACE follows its official release. ACE with its ground-truth reflector is not stronger on any benchmark (Table 18).
  • Success conditions (kE3U). We audited the conditions that the Explorer writes for its own tasks (153 tasks, 428 conditions, Table 17): 96.0% are correct. Their lists are sometimes incomplete, mostly on τ2\tau^2-bench where they omit conversational requirements, but only 5 of the 663 judge-accepted attempts left the environment in a wrong state.

Other changes to the paper

  • Section 2 discusses Learn-by-Interact, BAGEL and GEPA (kE3U).
  • Section 5.1 attributes the ablation gain to the Solver loop as a whole (5pi4). Due to budget constraints, Table 3 still reports one bank per row.
  • Appendix H adds a failure analysis on AutomationBench.

Overall

We believe these additional results and clarifications strengthen the paper, and we hope they address the remaining concerns and provide grounds for revising your ratings. Please let us know if any concern needs further discussion.

We briefly summarize the end of the discussion period to facilitate the AC’s assessment of our work.

After the discussion, all reviewers consider their concerns and questions mostly, or fully addressed. Reviewer 5pi4 “maintains [their] positive score to support this paper to be accepted”, while reviewers kE3U and oqVN both raised their scores from 2 to 3 in light of the revision.

Reviewers highlight the importance of the problem (5pi4), the novelty of our Explorer–Solver loop (kE3U), and consistent gains across all three benchmarks (kE3U, 5pi4). They also note the well-designed ablations (kE3U, oqVN), the transfer across model families (kE3U), and the clarity of the paper (all reviewers).

To address the comments of the reviewers, we invested significant resources in new experiments, including three independent memory banks for every method and benchmark, a full comparison of generation costs, and controls on the training worlds, the generated tasks and the auxiliary model size.

In the revised version, the core conclusions are unchanged and are now supported by stronger evidence. Without any training task or oracle verifier, DAEDALUS improves agents on three benchmarks, outperforms the only other method in this setting, and is competitive with methods that rely on curated training tasks.

We believe the paper offers a well-supported answer to a practical question, how an agent can build useful memory in a new environment before any task is available.

We thank all reviewers again for their appreciation of our work, the constructive engagement and for considering our reviewed version and rebuttals in depth.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.