acceptodds
Under review as a conference paper at ICLR 2027

Benchmarking In-Context Experiential Learning across Heterogeneous Action and Feedback Channels

Abstract

Real-world agents regularly contend with a sequence of tasks under a shared partially observable environment. Partial observability requires the agent to elicit feedback strategically, synthesize observations into actionable experience, and thereby refine its actions; shared environment requires the agent to not myopically act for the task at hand, but also for the subsequent tasks that may benefit from accrued experience. We refer to this adaptive cross-episode learning as experiential learning. Recent agent evaluations have begun to measure experiential learning in conversational or coding settings, where interactions occur only over a single channel. In contrast, agents in the wild tackle far more complicated scenarios, where experiences arrive through multiple senses and various distinct modes of actions are available. Web-based product recommendation is one such rich setting: a recommender agent must learn a customer's preferences over many shopping episodes, acting through natural-language dialogue and dynamic on-site product displays, while the customer reveals their preferences both in their replies and in logged browsing behaviors. Note that the latter signal is far noisier and frequent than text, so a performant agent must infer preferences by parsing the abundant yet opaque logs alongside informative yet sparse user responses. Evaluating state-of-the-art LLM agents in this setting, we found that agents can leverage multiple interaction channels to learn within a task; yet show no improvement across tasks. Decomposing experiential learning into synthesis, belief, and action, we identify three failures modes in the agents: (1) difficulty extracting useful information from the noisier channel, (2) forming low quality beliefs, both about the current customer modeling and the informativeness of its actions, and (3) failure to explore in proportion to its uncertainty. Our results suggest closing this gap toward robust experiential learning will require agents that overcome these failure modes.

Primary area:
datasets and benchmarks
Keywords:
in-context experiential learning, LLM agents, web-based benchmarking, product recommendation, interactive evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_yuEsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a benchmark for in-context experiential learning, in which a recommender agent serves the same simulated customer over several shopping episodes. It extends the chat-only BELA setup with a live web store. A persona agent (Claude Opus-4.7 with browser-use) answers questions and browses the catalog, which produces click logs. In each turn the recommender asks one question and updates a three-item sidebar. Log reliability has two levels: a clean regime with over 95% target-adjacent clicks, and a noisy regime with 30% target-adjacent and 70% random clicks. Metrics are the bag-of-words cosine between sidebar items and the target, and the number of turns until the target first appears. Each arm uses five personas over five episodes. The authors report that agents improve within a category, that clean logs help while noisy logs hurt unless a harness is added, and that no agent improves across episodes under default prompts.

Critical strengths
  1. The environment adds a behavioral channel and a second action (the sidebar) to experiential-learning evaluation, with adjustable log reliability, simulator checks. This is the main reason I see value in the paper: chat-only benchmarks cannot test whether an agent handles cheap but noisy evidence well, and this one can.
  2. Some diagnostics are well designed. Table 1 fixes the candidates and history and changes only the agent's own likelihood and certainty labels.
Critical weaknesses
  1. At the current scale, the across-episode and harness comparisons are not reliable. Each arm has five personas with no repeated runs reported, and some conditions use different persona sets. The main conclusions rest on these comparisons, so this weighs most in my rating.
  2. How informative each channel is comes from the simulator design rather than from data. The persona knows the target, its clicks are steered toward products that share the target's traits (App. E.2), the noisy regime mixes in 70% random clicks, and chat replies may not name the target or quote its attributes. The finding that clean logs help and noisy logs hurt is therefore largely built in, and the simulated logs are never compared with real browsing behavior.
Questions for authors
  1. What explains the gap between the unharnessed and summary arms in Fig. 7 and between Direct and full context in Fig. 8b ? Do all arms share the same personas, episode windows, and simulator settings?
  2. How many personas, categories, and runs enter each curve?
  3. Which model is used in Table 1?
1 comment

We are grateful for the reviewer’s constructive review and we are glad that they see value in an environment that can test whether an agent handles cheap but noisy evidence well. We hope the general response addresses your concerns for experimental reliability, benchmark validity, and the relevance of the behavioral channels. We respond to your other points below:

The simulated logs are never compared with real browsing behavior.

We fully agree with the reviewers that the behavioral logs are not a perfect model of how a real customer responds to each recommendation. However, we note that such realism is not the goal of the benchmark. Instead of realism, we aim to support across-episode learning by verifying that a customer preference is persistent and learnable across episodes, which is verified by the specialized harness and ideal trajectory experiments. We will state the lack of realism as a limitation.

What explains the gap between the unharnessed and summary arms in Fig. 7 and between Direct and full context in Fig. 8b?

In Fig. 7, the summary harnesses help to denoise and extract meaningful information from noisy experiences. First, the extractor only keeps the durable traits of the repeat customer. As such, many random clicks of the noisy regime never enter the carried context, whereas the unharnessed agent re-reads every random click verbatim. Secondly, the consistent profile provides a prior that downweight random clicks within an episode.

In Fig. 8b, the direct harness carries the history of prior episodes compressed into a fixed schema of traits. Thus, the agent sees these traits at full salience with the synthesis already done. The full context harness replays the raw history verbatim, including all chats, logs, and recommendations. The traits are buried in it, and the agent must redo the synthesis at every turn, negatively affecting performance.

We thank the reviewer for raising these concerns, and will add this discussion to the main text.

How many personas, categories, and runs enter each curve?

Each curve in the submission covers five personas, five episodes per persona, and three categories per episode (75 categories per arm), from a single run. Error bars are standard errors over personas. The expanded experiments cover 15 personas across 3 disjoint five-episode windows (225 categories per arm). We will add a table with these numbers for every figure.

Which model is used in Table 1?

Table 1 evaluates GPT-5.4. We thank the reviewer for noting this, and will state it in the caption.

Reviewer_UfSWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a continual learning setting studying whether an LLM recommender can learn a customer’s preferences across repeated shopping episodes. It extends previous work, BELA, which studies in-context continual learning through repeated product recommendations. The extension (i) constructs synthetic customers and shopping targets and (ii) creates an ierative interactive search, allowing the agent to gather feedback through both questions and recommendations.

Each sample has a hidden target product paired with a persona. The simulated customer purchases it immediately when it appears in the sidebar, and the agent has ten turns to find it. Performance is measured by similarity between recommended products and the target, and by turns until the exact target is recommended. It benchmarks several models and memory/prompt configurations.

The findings are primarily:

  • Clean browsing feedback generally helps within-category performance, while noisy feedback can hurt relative to chat alone. Memory summaries and agent scaffolding mitigate some of this harm.
  • Default agents show no consistent improvement across episodes, although a specialized prompt and customer-memory ledger help (5.22 -> 3.33 avg. turns).
Critical strengths
  • The paper addresses a relevant problem: learning persistent preferences from direct conversation (sparse but rich) and implicit behavioral signals (cheap but noisy).
  • The finding that additional noisy feedback can hurt performance is interesting.
  • The environment provides a useful extension of dialogue-only evaluation, with active information gathering.
Critical weaknesses

W1. Exact-target recovery is an imperfect measure of preference learning.

  • Several products may satisfy a customer equally well. An agent could learn meaningful preferences without recovering the particular product. The human study supports my worry: Humans pick the target from only 5 candidates 55.6% of the time, showing atbest weak alignment between personas and targets. This makes the benchmark mostly a search task of identifying a hidden item rather than the motivated claim of understanding user preferences.

W2. The evaluation does not isolate learning from prior experience.

  • Performance across episodes mixes the effects of memory with changing task difficulty.
  • Same task with no previous memory to check whether (and how much) experience helps.
  • Simple non-repeating search and a full-information oracle would also help establish the benchmark’s difficulty and achievable performance. The previous work BELA paper includes random, popularity, and full-information oracle baselines, making their absence here particularly noticeable.

W3. The evidence for the prompt improvement and the broader negative conclusion is limited.

  • The specialized GPT-Astra experiment uses only five customers over five episodes. Moreover, the GPT-5.4 prompt comparison uses different persona sets, confounding prompt effects with customer difficulty.
  • Self-summary and logs-only summary harnesses as changing only what is carried between episodes, then why do they lead to huge gaps in within-category cosine?

W4. What is shown beyond BELA?

  • BELA already studies repeated recommendation and reports stronger within-task than across-task improvement. The added behavioral channel is potentially interesting, but what new conclusions does it show?
Questions for authors

Please address weaknesses.

1 comment

We are grateful for the reviewer’s constructive review and we are glad that they find it interesting that additional noisy feedback can hurt performance. We hope the general response addresses your concerns for scale (W3) and benchmark validity (W2). We respond to your other points below:

Exact-target recovery is an imperfect measure of preference learning.

We thank the reviewer for their astute observations here. Apart from cosine similarity, we check the main results under three other metrics (Appendix B.1) and find no significant differences. Furthermore, even under exact recovery, the ideal trajectory and specialized harness show gains in performance across episodes. This shows preference learning is possible and can be quantified.

Human annotators only recover the target 55.6%.

While annotators only recover the target 55.6% of the time, they land on one of the top 2 products 83.08% of the time. Thus, the persona thus points to the target's neighborhood even when it does not single it out, which suffices with a three-product sidebar. Indeed, the addition of the oracle in the general response shows that with full information of the persona, it is possible to recover the ideal product in 1.67 ± 0.15 turns on average.

Why do the summary harnesses lead to huge gaps in within-category cosine?

The summary harnesses help to denoise and extract meaningful information from noisy experiences. First, the extractor only keeps the durable traits of the repeat customer. As such, many random clicks of the noisy regime never enter the carried context, whereas the unharnessed agent re-reads every random click verbatim. Secondly, the consistent profile provides a prior that downweight random clicks within an episode. We thank the reviewer for raising this, and will add this discussion to the main text.

What is shown beyond BELA?

We agree that the null result echoes that of Yang et al. (2025). However, their benchmark is chat-only, which leaves open whether chat is simply too sparse to carry preferences across episodes. Our inclusion of web-based interaction is a step change in the scale and richness of the information that modern agentic systems need to contend with.

Furthermore, we provide additional results such as how reasoning harnesses help in filtering through noisy data to extract pertinent information, how a specialized harness is able to demonstrate learning across episodes, and how a text only version of the specialized harness fails to achieve learning. This markingly demonstrates the text only setting in Yang et al. (2025) is insufficient.

Reviewer_aq9tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper extends the in-context experiential learning benchmark of Yang et al. (2025) to settings with two feedback channels and two kinds of action.
  • In a simulated web shop, a recommender agent serves the same customer over several episodes, each covering three product categories. At each turn it asks a question in chat and updates a three-product sidebar. The customer replies through two channels:
    • chat messages, which are sparse but informative;
    • browsing logs (clicks, views, searches), which are frequent but noisy.
  • The customer is a Claude Opus-4.7 browser agent conditioned on a persona, a three-day activity window and a hidden target product. Log reliability is controlled by a "noisy" regime (30% target-adjacent clicks) and a "clean" regime (>95%).
  • The simulator is validated with three small human studies and two automatic checks.
  • Main findings:
    • Agents improve within a category, and clean logs help, but noisy logs hurt unless a harness is used.
    • Agents show no improvement across episodes.
    • The authors break failures into three stages: synthesizing evidence, forming beliefs about the customer (an LLM judge scores how similar each stored belief is to the hidden persona), and acting on uncertainty (a perturbation study shows agents exploit their stated beliefs but do not explore).
  • A hand-curated history and a purpose-built "identity ledger" memory with a matching prompt suggest that cross-episode improvement is possible.
Critical strengths
  • Relevant, well-motivated extension with controllable difficulty. Real agents receive feedback through mixed channels. A benchmark that requires combining cheap-but-noisy logs with sparse-but-rich dialogue, and actively eliciting both, fills a real gap. The adjustable log-reliability dial lets the authors show a clear and useful result: an unreliable channel can be worse than no channel. Success drops from 0.53 for text-only to 0.31–0.36 with noisy logs (Appendix C.1), and a harness recovers most of the loss.
  • Careful diagnostic analysis beyond headline scores.
    • The breakdown into synthesis, belief and action is supported by targeted experiments. The most informative is the label-perturbation study (Table 1): agents act on their stated likelihoods but never switch to an uncertain candidate.
    • Appendix C.4 checks the null result across episodes against alternative explanations: the opening slate does not improve, stored memory ranks targets only slightly above chance, and agents re-ask questions instead of recalling.
    • Code is released, and the results are checked against three alternative metrics.
Critical weaknesses
  • It is unclear whether the main finding reflects the agents or the benchmark.
    • The central claim, that agents do not learn across episodes, is only meaningful if the benchmark rewards such learning. The evidence that it does is weak:
      • Annotators identify the target from the persona only 55.6% of the time (Sec. 4.3). Targets are chosen mainly for the current three-day activity, so persistent persona traits may carry little information about later targets.
      • The positive demonstration of learnability is a single hand-curated customer (Sec. 5.3).
      • The identity-ledger result rests on five personas with overlapping errors (5.22 ± 0.97 → 3.33 ± 0.77). The comparison against the base prompt uses different persona sets, which the authors acknowledge mixes the prompt effect with the persona sample (Appendix C.3).
    • Without an upper bound, such as an oracle with access to the persona, or a measure of how much information about future targets the persona carries, the flat curves cannot be attributed to the agents.
    • The cross-episode null also replicates Yang et al. (2025) and is therefore not a new finding.
  • The behavioral channel is synthetic, and the experiments are small.
    • The log stream comes directly from an LLM that knows the hidden target, mixed with a fixed share of random clicks (30% / >95%). Its informativeness is a design parameter, not a property of realistic behavior, and it is never compared with real clickstream data. The headline findings about noisy versus clean logs therefore partly reflect how the noise was constructed.
    • Each condition uses about five personas × five episodes. Error bars are computed over five personas, and several comparisons rely on single runs.
    • Two key measurements are not validated: belief quality is scored by an LLM judge (Gemini-2.5-Pro) with no human check, and the exploration analysis has no normative reference. The paper explicitly does not characterize the optimal exploration–exploitation balance, so "failure to explore" is judged against intuition, not an optimal or Bayesian baseline.
    • There are no non-LLM baselines (e.g., a bandit or a classical recommender using the logs).
    • Reproducing the experiments costs about $8,000 and depends on a closed, non-deterministic simulator, which limits the benchmark's practical value.
Questions for authors

Clarification

  1. What is the cross-episode performance of an oracle recommender given the full persona profile (but not the target)? This would establish how much room for learning across episodes the benchmark actually offers.
  2. How well do earlier episodes' targets and persona traits predict later targets? Please quantify this (e.g., rank of the target under a persona-only scorer) to show that persistent preferences carry over across episodes.
  3. Can the identity-ledger and base-prompt conditions be rerun on the same personas, with more than five, to confirm the 5.22 → 3.33 trend is not caused by which personas were sampled?
  4. Has the LLM-judge belief-similarity score been checked against human ratings?
  5. In the perturbation study (Table 1), which model was evaluated? Is choosing the uncertain low-likelihood candidate actually the better action in your environment, e.g., under a simple value-of-information calculation?
  6. How sensitive are the conclusions about noisy logs to the specific noise model (30% target-adjacent, uniformly random otherwise)? Could more realistic noise (e.g., popularity-biased clicks) change them?

Suggestions

  1. Calibrating the log simulator against real browsing data would substantially strengthen the realism claim.
  2. Adding non-LLM baselines and a cheaper open-weight persona simulator would make the benchmark more useful and more accessible to the community.
1 comment

We are grateful for your constructive review and for recognizing that the benchmark fills a real gap. We hope the general response addresses your concerns on the scale and reliability of experiments, the validity of the benchmark, and the relevance of the behavioral channels. We respond to your other points below:

The behavioral channel is synthetic. Its informativeness is a design parameter, not a property of realistic behavior.

We fully agree with the reviewers that the behavioral logs are not a perfect model of how a real customer responds to each recommendation. However, we note that such realism is not the goal of the benchmark. Instead of realism, we aim to support across-episode learning by verifying that a customer preference is persistent and learnable across episodes, which is verified by the specialized harness and ideal trajectory experiments. We will state the lack of realism as a limitation.

Has the LLM-judge belief-similarity score been checked against human ratings?

Yes. In our human validation studies, we also scored 100 (agent belief, true persona) pairs covering the full 0–100 range of judge scores. Human and judge scores agree closely, with Spearman ρ = 0.93 ± 0.02 overall, 0.74 to 0.95 per facet. We thank the reviewer for suggesting this check, and will add it to the main text.

In the perturbation study (Table 1), which model was evaluated? Is choosing the uncertain low-likelihood candidate actually the better action in your environment?

Table 1 evaluates GPT-5.4. Thank you for catching this missing information, we will state this in the caption. We echo the point in Sec. 2 that it is difficult to exactly characterize optimal exploration (e.g. compute the value of information) exactly in our setting. Thus, the experimental setup in Table 1 fixes the stated likelihood and varies only the stated certainty. Among products of equal likelihood, we expect the less certain products to be selected as they have the highest upside, according to the agent’s belief. Nevertheless, the agent picks each option at a rate close to random, which is incongruent with the agent’s stated posterior.

How sensitive are the conclusions about noisy logs to the specific noise model? Could more realistic noise change them?

We note that the noise model based on random click is among the hardest noise models, since it provides no information about the customer. Despite this, we find that agent harnesses can help provide useful information from the pure noise. Since a more realistic noise model would reflect some information on the customer and therefore constitute an easier setting, we believe our main conclusions would hold.

The cross-episode null also replicates Yang et al. (2025) and is therefore not a new finding.

We agree that the null result echoes that of Yang et al. (2025). However, their benchmark is chat-only, which leaves open whether chat is simply too sparse to carry preferences across episodes. Our inclusion of web-based interaction is a step change in the scale and richness of the information that modern agentic systems need to contend with.

Furthermore, we provide additional results such as how reasoning harnesses help in filtering through noisy data to extract pertinent information, how a specialized harness is able to demonstrate learning across episodes, and how a text only version of the specialized harness fails to achieve learning. This markingly demonstrates the text only setting in Yang et al. (2025) is insufficient.

There are no non-LLM baselines (e.g., a bandit or a classical recommender using the logs).

We thank the reviewer for this suggestion. We have a new experiment with a recommender that fills the sidebar uniformly at random, and this recommender finds the ideal product within ten turns about 12% of the time, for about 9.5 turns to ideal. We will work to add this and explore other non-LLM baselines for the camera-ready version of the paper.

Reviewer_cnLLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a web-based product recommendation benchmark to study in-context experiential learning across heterogeneous action and feedback channels. Extending the dialogue-based recommendation setup of Yang et al. (2025), the benchmark pairs a recommender agent with a web-browsing persona agent across multiple shopping episodes, each defined by a stable persona profile and a three-day activity window covering three product categories. At each turn, the recommender asks a question in the chat and updates a three-product sidebar, while observing explicit chat replies and implicit browsing logs under clean and noisy behavioral regimes. Measuring mean sidebar cosine and turns to ideal, the authors find that agents leverage clean behavioral logs for within-category learning, but show no across-episode learning under default prompts. The paper then decomposes experiential learning into synthesis, user modeling, and action to diagnose where agents fail.

Critical strengths

Targeted experiments consistently validate the paper’s hypotheses. The authors decompose experiential learning into synthesis, user modeling, and action, and design controlled ablations for each stage. Holding the base model and noisy behavioral stream fixed while varying only the agent harness cleanly shows that an unharnessed agent performs worse than the text-only baseline, whereas self-summary, logs-only summary, or ReAct recovers performance (Section 5.2.1), with the ordering reversing in the clean regime (Appendix C.2). Similarly, the belief similarity analysis (Section 5.2.2) and the two-pass harness perturbation study on stated likelihood and certainty labels (Section 5.2.3) consistently show that agents recommend based on point estimates of product suitability, but fail to form transferable beliefs or explore in proportion to their uncertainty.

Critical weaknesses
  • The implementation of the behavioral channel does not support the paper’s core premise. The paper assumes that browsing logs are an implicit signal of customer preferences and the three-product sidebar is an active lever for eliciting behavioral reactions (Sections 1 and 3). The actual implementation conflicts with both:
    • Browsing logs reflect the hidden target itself. The persona agent is directly given the hidden target product and generates browsing logs by rule (Section 4.2). Because over 95% of clicks in the clean regime point to target-adjacent products, the finding that clean behavioral evidence helps (Section 5.1) largely means that a channel containing near-target answers helps recover the hidden target.
    • The sidebar barely elicits behavioral feedback. Under Appendix E.2, the persona agent browses the catalog grid by its own browse rule and only changes its behavior when the hidden target appears in the sidebar, which immediately ends the category. Non-target sidebar picks rarely induce informative reactions, undermining its role as an active information-acquisition lever.
  • The across-episode null result is not separated from the environment’s upper bound. The paper attributes the lack of across-episode learning to the recommender agent (Section 5.1), but the environment itself may allow very little across-episode improvement:
    • The hidden target depends on unobserved context. Each hidden target is determined by a three-day activity window that the recommender does not observe (Section 4). Even human annotators who see the persona profile only recover the target 55.6% of the time in a 5-way choice (Section 4.3).
    • The paper’s own calculations point to a tight ceiling. Appendix C.4 shows that stored memory only moves the hidden target from roughly rank 128 to rank 107 out of 256 products, far from entering the three-slot sidebar.
    • Asking questions has no cost. Because asking questions in chat is costless and reliably resolves the current need in a few turns, the agent simply re-elicits rather than recalls, leaving little benefit to relying on prior memory (Appendix C.4).
Questions for authors

please see weakness

1 comment

We thank the reviewer for the close reading of our implementation, and we are glad that you find the targeted experiments of Section 5.2 well controlled. We hope the general response addresses your concern on the benchmark’s validity on the null result and the relevance of the behavioral channels. We respond to your other points below:

The sidebar barely elicits behavioral feedback

We hope the general responses address your concern here. Specifically, our new experiment with the text only specialized harness shows without the behavioural channel, learning is absent. Furthermore, we point to the noisy regime as quantitative evidence. Even though only 30% of clicks reflect the target product, agents with a reasoning harness are able to outperform their textual only counterparts. This reflects how the sidebar provides behavioral feedback that is necessary to successful recommendations.

The across-episode null result is not separated from the environment’s upper bound.

We hope the general responses address your concern here. In addition, we would like to clarify Appendix C.4. The measurement provided by the section is not a reflection of the tight ceiling of the benchmark, but rather, is the cosine similarity between the persona description with the ideal product. We hope the newly provided oracle model provides a better reflection of the benchmark’s ceiling. Finally, we note that the number of questions asked is a tunable parameter. By default, it is set to one question per turn, restricting how much agents can rely on purely textual evidence.

Human annotators only recover the target 55.6% of the time.

While annotators only recover the target 55.6% of the time, they land on one of the top 2 products 83.08% of the time. Thus, the persona thus points to the target's neighborhood even when it does not single it out, which suffices with a three-product sidebar. Indeed, the addition of the oracle in the general response shows that with full information of the persona, it is possible to recover the ideal product in 1.67 ± 0.15 turns on average.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

General Response

We thank the reviewers for their thoughtful comments. We are grateful that all four reviewers recognize the value of our central goal, which is evaluating across-episode learning from heterogeneous feedback, as well as the strength of our diagnostic analysis. The main concern raised is whether there is a consistent customer preference to be learned across episodes. We agree that this is central to the validity of the benchmark, and are hopeful that we can address this concern and other questions on benchmark details satisfactorily below:

  1. Benchmark Validity (Reviewers cnLL, aq9t, UfSW)

The reviewers’ concerns here are valid and pertinent. To address that the null result is an indicator of agent’s inability to learn across episodes rather than of the benchmark itself, we point to the specialized harness, ideal trajectories, and oracle model.

  • Specialized Harness (Section 5.4). Our specialized harness forces agents to seek information and use past episodes to produce recommendations. When evaluated on the same personas and environment, we find that agents augmented with this harness demonstrate cross-episodic improvement. This shows that improvement in the benchmark is possible, and the null result is a reflection of agent inability.

  • Ideal Trajectories (Section 5.3). We repeatedly run an agent in increments of episodes, keeping only the episodes that demonstrate learning from prior history. In these trajectories, the agent learns a consistent latent preference that informs the ideal product across diverse categories. As all actions are generated from the agent itself, learning is possible across episodes. We are happy to provide additional examples of these ideal trajectories if requested.

  • Oracle. To quantify an upper limit on benchmark performance, we run an agent (Claude Opus 4.7) that is given the full persona profile minus the target products. Across the same set of personas and episodes of other experiments, we find the oracle required 1.67 ± 0.15 turns on average to reach the ideal. This is far fewer than any agent in the paper, including that of the specialized harness. Thus, the environment leaves substantial room for across-episode learning. We thank Reviewers aq9t and UfSW for the suggestion and will add this into the paper.

  1. Consistency of Experiments (Reviewers aq9t, UfSW, yuEs)

The reviewers would like clarifications on whether the experiment setup is consistent. We confirm here that all conditions in the main text share the same personas, activity windows, and simulator settings. As reviewers point out, the one exception was the specialized harness (Appendix C.3). We have rerun it on the shared persona set, and the main result of declining turns to the ideal product across episodes remains (6.27 ± 0.83 → 6.20 ± 0.86 → 5.07 ± 1.01 → 5.20 ± 0.78 → 4.60 ± 0.92).

  1. Scale of Experiments (Reviewers aq9t, yuEs)

To further confirm the results of the experiments, we expanded the experiments from 5 personas to 15 personas repeated 3 times (45 total runs). Results are plotted here and show no changes to results. With the exception of the specialized harness, agents do not show any indication of improvement across episodes.

  1. Informativeness of Behavioral Channels (Reviewers cnLL, aq9t, yuEs)

Reviewers have raised questions on whether the behavioral channels can reflect the persona, or merely reflect the target product within the episode. We want to first point to the fact that both specialized harness and ideal trajectory experiments demonstrate learning across episodes. Furthermore, we conduct a new experiment with a text-only version of the specialized harness. We find that the agent both consistently performs worse in each episode and is unable to demonstrate learning across episodes (results). Thus, the specialized harness agents must have learned preferences relevant to the persona from the behavioral channel.

We hope the above satisfactorily addresses your main concerns, and we would be happy to provide raw materials or further analysis to address any that remain. Below, we address your other points in point-by-point responses to individual reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.