- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Program-Centric Memory, an experience reuse framework for agents based on the representation for function principle. It compiles trajectories into predictive beliefs for estimation and executable programs for control. Experiments across multiple benchmarks show significant performance gains and token reductions.
Critical strengths
-
Innovative perspective by introducing the representation for function principle, moving beyond traditional language-centric memory.
-
Harmonious design combining probabilistic beliefs with executable programs to balance accuracy and efficiency.
-
Strong empirical performance achieving substantial success improvements and massive token reductions across diverse benchmarks.
Critical weaknesses
-
Heavy reliance on LLM fallback when tasks fall outside program coverage, which may cause performance bottlenecks.
-
Strong dependence on offline construction and validation, potentially limiting its adaptability in highly dynamic scenarios.
Questions for authors
The authors should clarify the robustness of the program revision mechanism in highly stochastic environments and discuss potential extensions toward online real-time updating.
5 comments
Response to Reviewer opWF (2/3)
W2 & Q1(a): "Strong dependence on offline construction and validation, potentially limiting its adaptability in highly dynamic scenarios." "The authors should clarify the robustness of the program revision mechanism in highly stochastic environments ..."
Answer:
Thank you for asking about robustness in stochastic and dynamic environments. We would like to clarify PCM’s existing revision mechanism and discuss potential enhancements for these settings.
Robustness of the current mechanism to stochasticity and environmental changes. PCM determines whether to accept a Program Update or roll back to the previous version based on the candidate program’s overall performance on evaluation tasks during Online Execution, together with whether the intended revision objectives have been achieved. A version that previously performed well may be rejected or rolled back based on new evaluation results if its performance deteriorates under changed conditions. These changes are also reflected in execution trajectories and feedback, which guide subsequent revisions. The LLM uses this feedback to modify the program, and the candidate undergoes validation and execution evaluation before another acceptance decision. This cycle allows program revision and version selection to continually incorporate feedback from the current environment.
Enhancements for highly stochastic and dynamic environments. For highly stochastic environments, we could evaluate the current and candidate programs multiple times on the same tasks and compare their average performance, reducing the influence of individual random outcomes on version selection. For highly dynamic environments, we could apply time-decay weights to Online Execution evaluation results, giving recent performance greater influence on version selection. Predictive Beliefs could similarly use an exponential moving average (EMA) to emphasize recent observations, while Executable Programs could use adjustable feedback windows and revision frequencies to respond more promptly to new feedback. Together, these extensions could balance resistance to random fluctuations with responsiveness to environmental changes.
Response to Reviewer opWF (3/3)
Q1(b): "... discuss potential extensions toward online real-time updating."
Answer:
Thank you for raising this important question. We agree that real-time memory updating is an important direction for future work. We would like to outline a possible extension with two concurrent processes: task execution and background program updating.
Task execution and feedback collection. At the beginning of each task, the execution process would use the latest accepted program and keep that program version fixed throughout the task. It would continuously collect execution trajectories and feedback. Meanwhile, Predictive Beliefs could be updated incrementally after each interaction by incorporating the observed action–outcome pair into the accumulated statistics and probability estimates. Updated beliefs would become available for subsequent decisions.
Background program updating. A separate process could initiate program revision when sufficient feedback has accumulated, similar execution problems recur, or recent task performance declines. Each revision round would use the current accepted program and a fixed batch of recent feedback to identify revision objectives and generate candidates within a bounded budget. Feedback arriving during that round would continue to accumulate for subsequent revisions. Candidate programs would be developed separately from the program used by ongoing tasks, allowing execution and revision to proceed concurrently.
Candidate evaluation and version switching. Candidates would undergo interface and behavioral tests, followed by task-level comparison with the current version on the same evaluation tasks. Acceptance would depend on overall execution performance and whether the intended revision objectives have been achieved. An accepted version would be made available to subsequently started tasks, while ongoing tasks would continue using their existing version. If a candidate fails validation or does not meet the acceptance criteria, the current version would remain in use.
This design would allow the agent to incorporate new experience through incremental belief updates and background program revision, without requiring task execution to wait for program revision and validation to finish. We would welcome your insights and suggestions on this direction and would be happy to discuss it further.
Thank you again for recognizing the value of our work. Your recognition and thoughtful feedback mean a great deal to us. We hope these clarifications and additional results address your concerns and encourage you to reassess our work. We look forward to your further feedback and would be happy to discuss any remaining questions.
Response to Reviewer opWF (1/3)
Dear Reviewer opWF,
Thank you for your thoughtful review and for recognizing PCM's representation-for-function perspective, its combination of probabilistic beliefs and executable programs, and its strong empirical performance.
We added longitudinal construction data and mechanism clarifications to address your concerns:
Below, we provide detailed responses to each of your questions, with the utmost sincerity, to fully address every concern.
Answer:
The original paper did not explain this process clearly enough, which may have caused confusion. We would like to further clarify it here. Although program revision and validation take place offline, each revision is guided by feedback from online execution. PCM iteratively updates its programs to improve task performance. Each iteration consists of four stages:
Program Revision → Program Validation → Program Update → Online Execution.
During Program Revision, the LLM revises the program using trajectories and feedback from previous executions. The revised program then undergoes Program Validation. If it passes validation, Program Update applies the candidate version for the next execution. During Online Execution, the program interacts with the environment, producing new trajectories and feedback. We then determine whether to accept the update based on the overall execution results and whether the intended revision objectives have been achieved. The collected feedback guides subsequent revisions.
We report longitudinal TravelPlanner results from 52 construction iterations, comprising one initial program-generation attempt and 51 revision attempts. Across these iterations, 41 evaluations were completed on the same 45 training queries. The table reports mean success over completed evaluations, together with revision, evaluation, and overall token consumption for each iteration group.
Although individual revisions do not always improve task success, the evaluated versions show an overall upward trend, supporting continued improvement through execution feedback. We discuss extensions for adaptation to environmental changes, stochastic robustness, and online updating in our responses to W2 & Q1(a) and Q1(b) below. We will include further development results for TravelPlanner, ALFWorld, and WebShop in the revised paper.
Answer:
Thank you for raising this concern. Although uncovered cases require LLM fallback, our results show that PCM achieves higher task success rates with substantially lower online token consumption than the compared baselines, demonstrating overall gains in both task performance and token efficiency.
The table below quantifies program and LLM usage. For TravelPlanner, both components participate in each query; we report their shares of calls.
Furthermore, with beliefs held fixed in Figure 4, the share of program-executed steps increases from 0% to 60.1% over successive iterations. This demonstrates that iterative program revisions expand coverage and reduce reliance on LLM fallback.