- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the capabilities and limitations of self-improving personal agents, which adapt to user preferences by evolving their control frameworks while keeping the underlying large language model (LLM) unchanged. The results indicate that: (1) context-based constraint frameworks can effectively handle simple preferences, while more complex preferences require external computation and execution control; (2) increasing memory size initially improves preference compliance but may ultimately lead to a decline in performance; (3) Existing self-evolutionary methods improve performance during early interactions but often reach a plateau, still showing a significant gap compared to the “Oracle” constraint framework. The authors further elucidated these observations through theoretical analyses.
Critical strengths
- Easy to follow.
- Important and timely study.
- Comprehensive experimental results and complementary analysis.
Critical weaknesses
The main weakness of this paper is that its key observations are largely intuitive and consistent with existing knowledge, offering limited novel insights. For example, the performance degradation observed with increasing memory size is not particularly surprising, as it could be attributed to various factors, such as long-context interference, hallucinations, or redundant information, rather than memory scaling itself. However, the paper does not sufficiently disentangle these factors or establish a clear connection between the empirical observations and the proposed theoretical explanations. Overall, the main findings are largely expected, which limits the novelty and significance of the paper's contributions.
Questions for authors
please see weakness
4 comments
rebuttal (part 2, finished)
Entries are violations divided by applicable episode–rule pairs. Repair improves long memory in all three orderings, particularly SMS greetings and sign-offs. Short memory worsens, largely through sign-off omissions in one ordering despite its unchanged sign-off sentence. The pooled length effect reverses after repair, with the repaired length comparison varying across orderings. These different responses suggest an interaction between content refinement and memory organization. Appendix C.2 reports higher violations for longer memory in the post-hoc non-SMS analysis under both content conditions. A separate analysis of expected task–rule pairs, excluding pre-identified task–preference conflicts, likewise gives 123/623 (19.74%) for repaired versus 94/504 (18.65%) for repaired . These comparisons highlight content quality and task–rule configuration alongside memory length. The repair completes missing specifications while preserving statement count and revising sentence wording and token length.
Q3: Feedback quality improves learning, and execution diagnostics reveal further targets for improvement. Holding ACE and the interaction budget fixed, explicit corrective feedback reduces final violations from 43/102 (42.16%) to 35/113 (30.97%), with improvement in all three seeds; the contemporaneous oracle is 8/110 (7.27%). Each condition has 54 final evaluation episodes. Audits locate failures despite relevant stored instructions: 12 payment episodes lack an effective description, including one placing formatted text in an ignored argument. Eleven contain successful payments; one contains only failed attempts, scored under the original applicability definition. We also observe cross-rule update errors: correcting SMS behavior corrupts a payment-initials example; later feedback repairs it but temporarily corrupts the SMS signature, which is subsequently repaired. These observations distinguish update and execution failures and motivate different interventions.
3. Making the theory–experiment connection precise
Theory and interventions connect the learning formulation to specific design questions. The architecture analysis describes attainable behavior; the memory theorem relates feedback allocation to statement reliability; and the pool experiment examines expansion from accumulated experience. Section 4.2 and Appendix C connect these perspectives through the memory-repair results.
Proposition B.6 in Appendix B.5 gives a concrete repair–corruption mechanism, illustrated by the three-step ACE case in Appendix C.3. For two harness states with risk difference , fixed corruption and repair probabilities give stationary excess risk . The proposition explains how repairable errors can coexist with a residual and identifies both correction and preservation as improvement objectives. The ACE trajectory shows the corresponding cross-rule editing failure and its subsequent repair. Together, these analyses guide information acquisition, preservation of valid content, and reliable execution of stored preferences.
We hope these additional results and clarifications make the paper's contribution and its implications for personal-agent design more concrete. We kindly refer you to our discussion with other reviewers, where we have largely enriched our work both empirically and theoretically.
We would also like to emphasize the motivation behind this work. The difficulties we study are familiar to many researchers building personal agents, and their recurrence makes understanding them especially important. Knowing that longer memories or repeated self-updates can fail still leaves open why they fail and which parts of the system should be improved. Our contribution is to introduce a learning perspective on harness evolution that connects these observations to approximation, generalization, and optimization. By combining controlled experiments with theoretical analysis, we provide a systematic interpretation of these phenomena and identify mechanisms that can produce them. This understanding translates recurring practical failures into concrete design questions about execution knowledge, reliable memory, and the preservation of correct content. We sincerely believe that explaining widely encountered phenomena is itself a meaningful scientific contribution and an essential step toward building more reliable personal agents.
Dear Reviewer s4hW,
With less than 6 hours remaining before the rebuttal deadline, we would like to know whether our response has addressed your concerns. Should you have any further questions, we would be happy to provide additional clarification.
Thank you once again for your time and effort.
Sincerely,
The Authors
We have posted a general response titled "Response After Rebuttal", in which we clarified our motivation. We hope this clarification can help the reviewers and ACs to understand our work better during the final assessment.
rebuttal (part 1)
We thank the reviewer for recognizing the importance and timeliness of this study, its readability, and its comprehensive experiments and complementary analysis. Your concern about the specificity of the insights and the connection between theory and experiments has motivated additional controlled comparisons and trajectory audits. We address the weakness and the corresponding question together below.
1. Contribution: a learning formulation that guides diagnosis
Our contribution is to formulate personal-agent harness evolution as a learning problem and use this formulation to identify what to inspect and improve. With the underlying model frozen, the learned object includes the surrounding memory and execution mechanisms. This separates three questions: which behaviors the harness can realize, how reliably interaction evidence supports its contents, and how updates correct errors while preserving valid information.
AppWorld-P supplies a common executable setting for studying these questions through preference checks on actual actions. The resulting contribution combines this evaluation setting, theoretical mechanisms, and interventions that identify practical improvements. Section 3.3 organizes the three learning questions, Section 4 develops their empirical and theoretical analyses, and our newly enriched Appendix C reports the additional comparisons and diagnostics.
2. Additional experiments sharpen the actionable insights
The following summarizes what the new comparisons reveal beyond the aggregate performance trends:
Q1: Accurate preference specifications and execution knowledge play distinct roles. We completed a new 228-episode comparison of Base context, Procedural, Few-shot, and Runtime support. Base context supplies accurate preference descriptions and, for habitual-card inference, the shared 20-observation raw history. Procedural adds execution steps, Few-shot adds two worked examples to those steps, and Runtime support augments Base context with external execution mechanisms. For SMS checksums, violations decrease from 6/12 with Base context to 2/12 with Few-shot, matching Runtime support's 2/12 (using up to three attempts). For running totals, Procedural and Few-shot each reduce violations from 3/7 to 1/8. Runtime support achieves 0/6, across six of nine expected payment opportunities. Habitual-card results also favor Procedural (2/15) over Runtime support (3/14). All four habitual-card conditions share the same raw history, aligning Base context with the original From history reference's input.
These findings separate preference specification, execution knowledge, and the mechanism that supplies that knowledge. Procedures and examples enrich context with useful implementation steps. Q1's assertion-selection updater learns by selecting from a fixed statement bank; acquiring richer procedures is a distinct learning objective. Section 4.1 develops this distinction, and Appendix C.1 reports the behavioral gains from supplied execution guidance.
Q2: The effect of expanding learned memory depends on its content and use. Interaction-derived memories capture the errors, incomplete specifications, and repetition that arise during preference learning. All lengths draw from the same learned pool, holding upstream evidence fixed. We audited and semantically repaired the memories while preserving statement count, positions, orderings, and exact repetition relationships. Each condition contains 360 evaluation episodes: