- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how an LLM trading agent's previous experience affects its future decisions. The authors build AgenticFolio, where agents can maintain portfolios, retrieve past decisions, and update investment plans and theses.
The main experiment starts agents from the same portfolio but gives them different versions of their trading history. The authors find that agents retaining their memory trade considerably less than those starting with an empty history. Keeping previous investment judgments seems to reduce trading further. Most of the difference appears in the first session, but the resulting portfolio differences remain over subsequent sessions. The paper also includes longer-history experiments and a condition where agents explicitly review their investment beliefs.
Critical strengths
-
I like the basic experimental idea. Starting agents from exactly the same portfolio and changing only their inherited history is a reasonable way to study the behavioral effect of memory. Otherwise it would be hard to tell whether different decisions come from different experiences or simply different holdings. The paper handles this distinction fairly well.
-
The experiments also go beyond just comparing final returns. Looking at turnover, portfolio distance, and especially what happens right after the intervention makes the results more informative. I found the first-session effect particularly interesting. It suggests that memory may have a much larger influence at certain decision points than during ordinary day-to-day trading.
Critical weaknesses
- The behavioral finding is interesting, but its significance is not very clear.
The main observation is that agents with memory trade less and stay closer to their previous portfolios. But what should we make of this? Is the agent making more consistent decisions because it has learned something useful from experience? Or is it simply more reluctant to reconsider its earlier choices?
I think this distinction matters quite a lot. The paper mostly treats continuity as something worth understanding, which is fair, but it does not really tell us whether the continuity reflects useful learning, anchoring on past decisions, or just a tendency to maintain the status quo. The return results do not help much here either, since the authors do not find a significant improvement from retaining memory.
So I am not sure how to interpret the main finding beyond the fact that different histories lead to different trading behavior. What does this tell us about how memory should be designed or evaluated in LLM agents?
- I am not fully convinced that the paper explains why memory has this effect.
The comparison between retaining all history, keeping only factual records, and restarting is useful. But there is still a fairly large gap between observing that saved judgments reduce trading and understanding how these judgments influence the agent's decisions.
For example, does the agent actually rely on its previous investment thesis when deciding to hold? Would it behave similarly if it received an equally detailed but unrelated history? Or is simply having a longer, more structured context enough to make the agent less willing to trade?
These possibilities seem quite different to me, but the experiments do not really separate them. The paper talks about agents carrying forward their beliefs, while the evidence seems to establish a behavioral difference caused by supplying different memory content. I would be more convinced if there were a clearer connection between the two.
- The long-term interpretation is somewhat limited.
One result that stood out to me is that 81% of the turnover difference comes from the first continuation session. After that, the trading rates become much closer. The portfolio differences persist, but this is partly expected: once two agents make different trades, their holdings will remain different even if neither agent makes another decision.
The frozen-position analysis actually makes this point quite clear. It is useful evidence about path dependence, but I am less sure it demonstrates a persistent influence of memory on the agent's decision process.
This leaves an interesting question unanswered. Does accumulated experience continuously shape how the agent reasons and acts, or does it mainly affect the initial decision after the memory intervention? The paper seems to lean toward the former interpretation, while much of the evidence points to the latter.
Questions for authors
-
How do the authors distinguish useful experience-based consistency from simple anchoring on previous decisions? Is there any evidence that the retained investment judgments help agents make better decisions, rather than just making them less likely to change their portfolios?
-
Could the observed effect be driven by the amount or structure of the supplied context, rather than the specific content of previous investment beliefs? For example, would unrelated or shuffled investment histories produce a similar reduction in trading?
-
Given that most of the turnover difference occurs immediately after the intervention, what evidence supports an ongoing influence of memory on subsequent decisions, beyond the mechanical persistence of the initial portfolio difference?
2 comments
Thank you for the detailed rebuttal and additional clarifications. I maintain my original rating of 2 (Weak Rejection).
Response to Reviewer a1A3
Thank you for emphasizing the first-session result and its interpretation. The added controls and full-sample audit clarify both the content response and its persistence.
1. Continuity can sustain active trading
Continuity includes active rebuilding.
These cases broaden the interpretation beyond simply keeping existing positions.
Decision quality is evaluated through financial outcomes. Exact first-action accounting across all 108 note continuations shows that retaining exposure can avoid exit opportunity costs, while new purchases can capture rebounds in other windows. The aggregate return comparisons are non-significant. Appendix G reports these outcomes alongside the original return decomposition, whose largest component is invested-capital exposure.
2. Content and length controls
At 12 checkpoints, genuine, neutral, and unrelated notes are each repeated K=3 with a common account, factual transaction buffer, tools, and note structure. First-request token counts differ by at most 1.18%.
Appendix G reports both inferential levels. The newest 108-run probe fixes archived asset facts and initial news before adding evaluation and policy, with token differences at most 0.0288%.
The revised text separates the full inherited-state effect from attribution to an individual note component (Appendix H).
The audit also tracks what agents read and store.
Twelve of 14 genuine-note thesis reconstructions retain every archived exit condition. These records connect supplied history to new state alongside the observed actions.
3. Initial action and lasting account consequences
Section 5.2 explicitly identifies the lasting separation as account-state transmission. The evidence establishes a lasting consequence of the initial memory-sensitive order; additional direct effects on later reasoning remain a separate identification question. The Discussion translates this time structure into checking the first decision and subsequent account path after a memory operation.