acceptodds
Desk rejected at ICLR 2027

Do Agents Learn the User? Evaluating Personalization Across Heterogeneous LLM Agents

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

A personalized language-model agent should use earlier user feedback in later responses and actions. Users should not have to repeat a preference for shorter answers on every task. Task completion alone cannot show whether an agent has learned such preferences. We introduce AgentBake, a framework and benchmark for comparing personalization methods across agent frameworks with shared tasks and execution interfaces. Simulated users provide feedback from written profiles hidden from the agents. Separate evaluators check later responses and actions, judging task correctness separately. We compare reusing learned user state, learning it from scratch, and continuing to update it. With a shared feedback extractor, agents complete most tasks but often miss answer-style preferences; further updating shows no clear style benefit. Using released learning procedures, we compare continued updating with state frozen after eight feedback opportunities. Under a secondary judge from the host's model family, PAHF improves style matching and task success, while CIPHER shows no clear style gain. Accurate instructions improve style but lower task success relative to no instruction. Training on simulated-user conversations improves preference extraction. However, some preferences are missing from saved instructions, and saved instructions are not always supplied to the host on later tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, personalization, agent evaluation, preference learning, online adaptation, user simulation, cross-agent transfer.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_JxTWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AgentBake, a framework and benchmark for testing whether LLM agents use earlier user feedback in later tasks, in which the hidden user profile is only seen by the simulator and the judges. Compared to prior personalization benchmarks, which mainly evaluate text responses, AgentBake evaluates tool-using agents across different agent frameworks. Compared to execution benchmarks such as tau-bench, it also checks whether user preferences are followed, and judges task correctness separately from preference matching. The paper also builds a shared interface that connects personalization methods to various agent SDKs.

The main experiments run 12 personalization components and one baseline on the same calculation tasks across 7 agent SDKs. The results show that agents complete most tasks but fully match the preferred style in less than half of the responses. The paper also traces preferences from feedback to host input and finds that many are lost.

Critical strengths
  1. The paper records what is extracted, stored, retrieved, and actually sent to the host, and separates this from the final behavior. It shows that many preferences are lost before the host sees them, which an evaluation based only on task success or on stored memory would not show.

  2. The paper covers a setting existing benchmarks miss. Existing benchmarks mainly evaluate text responses, or check task outcomes but not user preferences. AgentBake tests tool-using agents that must learn preferences from feedback, and checks both task success and preference matching.

  3. Careful experimental design. The 2 ×\times 2 update/exploration design with identical starting states separates the effect of updating from exploration, task correctness is judged separately from preference matching, and unobservable or missing judgments are reported instead of counted as failures.

Critical weaknesses
  1. The paper misses many details. Most preferences are lost at retrieval, but the paper does not say how retrieval works, e.g. what counts as "relevant". It is also not reported how much of the loss is due to no_memory, how much to relevance filtering, and how much to the budget. FABLE and its variant FABLE-Diag are not explained in detail, and each adapted method gets one or two lines. In addition, the paper states that the conversation dataset will not be publicly released, and it does not say whether the code will be released. All of this makes the paper hard to reproduce.

  2. The adapted methods are quite far from the originals. For example, CIPHER no longer learns from user edits. A shared interface makes the comparison fairer, but the methods being compared are no longer the original methods.

  3. The scope is narrower than the title and Figure 1 suggest. The main comparison uses only one type of calculation task. The title says "heterogeneous" agents, which sounds like many different kinds of agents, but in practice it mainly means 7 different SDKs. The underlying model is the same, and so are the tasks. Memory is unobservable in all judged calls under Astra (Table 3), and clarification is mostly not applicable.

Questions for authors
  1. How does retrieval work? Can the authors provide more details such as relevance criterion?

  2. What is the form of the extractor's confidence, and what threshold counts as "high-confidence"?

  3. Could the authors provide more details about FABLE-Diag, and explain how exactly it differs from FABLE?

  4. How the adapter is implemented for each SDK?

4 comments
AuthorsOpenReview ↗

Replies to Reviewer JxTW - Part 1

Reply 1/3 — Source methods, learning and task scope (W2, W3)

We now separate the common-interface controller comparison from experiments retaining CIPHER's and PAHF's released learning routines. Both released learners improve style within eight feedback opportunities, under two judges and on two task families. This directly addresses whether our earlier adaptations removed the mechanisms needed to learn.

What the adaptations establish. You are right that they differ from the source systems. Scalar CIPHER uses selected-versus-default setting contrasts instead of learning from edits; these contrasts remain zero when initialized empty under the tested conditions. PUMA and PEToolLLaMA update configuration scores without training their source models. We now state this distinction in §4.2 and enumerate the retained mechanisms and departures for every method in Table 4. The matched comparison evaluates these implemented controllers (revised manuscript, p. 5, lines 231–242; p. 17, lines 864–906).

Learning with the released routines. The added calculation study retains CIPHER's edit inference, retrieval and aggregation, and PAHF's retrieval, questions and feedback updates. It bypasses the shared extractor and tests seven SDKs, four fixed synthetic users and ten replicate trajectories. Held-out evaluations disable learning and questions. For continued learning, both the updated state and a copy frozen after eight opportunities execute the held-out tasks afresh (p. 9, lines 448–454; p. 40, lines 2145–2159).

Full style-match change, percentage pointsCIPHERPAHF
Initial → eight opportunities, Astra+30.0 [28.6, 31.5]+10.8 [8.0, 13.6]
Initial → eight, Sonnet+26.0 [25.1, 26.9]+9.1 [6.8, 11.5]
Continue to 32 − freeze after eight, Astra−1.0 [−2.5, 0.5]+14.7 [10.7, 18.8]
Continue to 32 − freeze after eight, Sonnet−0.4 [−1.3, 0.5]+9.7 [5.8, 13.6]

These are 97.5% intervals, adjusted over each study's two primary style contrasts; Astra is primary. Early gains concentrate in one profile per learner, and worst-case missing-outcome bounds remain positive. Style improvement does not ensure correctness: native task success falls from 97.8% to 93.3% for CIPHER and 69.8% to 62.5% for PAHF. Continued updating later raises PAHF's native success by 9.9 points [7.5, 12.1] (pointwise 95%), alongside its style gain; CIPHER shows no clear further style gain. (p. 41, lines 2169–2195).

Learning beyond calculation. Both learners train on the same eight support tickets in the same per-seed order, then face four held-out tickets (four users, five seeds; Sonnet 4.6/LangGraph). Astra full matches rise from 20/80 to 45/80 for CIPHER and 55/80 for PAHF; Sonnet's gains are +36.25 and +25.00 points. Correct first submissions remain 80/80 for CIPHER and 79–80/80 for PAHF. Further feedback gives no clear CIPHER gain; PAHF improves further only under Astra. (p. 42, lines 2233–2263).

The scope is now explicit. The title now specifies “Across Agent SDKs.” Figure 1 and §4.1 clarify that seven integrations share one host model; all-controller comparisons remain confined to calculation, and the added studies establish narrower instruction and learning effects beyond it (p. 4, lines 200–211). The SDK axis still matters: accurate instructions raise style matching over no instruction by 8.3 points under CrewAI but 35.5 under LangGraph (Sonnet; 26.3/40.4 under Astra) (p. 9, lines 443–444). On two other hosts, DeepSeek V3.2 shows no detected host × method interaction, whereas MiniMax M2.5 shows a fixed-state style interaction under both judges (pp. 45–46, lines 2428–2442).

To test persistent-memory operations absent from the calculation tasks, we added a cross-task test: recall succeeds on 54/56 tasks with memory versus 0/56 without. This directly demonstrates runtime memory capability, reported separately from the preference-learning experiments above. Clarification evaluation mainly covers the concise profile; the revised manuscript states this scope (p. 8, lines 395–398; p. 31, lines 1650–1664).

AuthorsOpenReview ↗

Replies to Reviewer JxTW - Part 2

Reply 2/3 — Retrieval, confidence and reproducibility (W1, Q1, Q2)

Thank you for asking how relevant instructions are selected, how confidence affects updates, and what can be reproduced. We explain these mechanisms below and trace where saved instructions are lost before reaching the host. Card construction accounts for the largest measured loss.

What “relevant” means. The shared text path retrieves the current user's general rules and rules matching the task's declared type. It does not rank rules by semantic similarity to the current request. Eligible persistent instructions require valid fields, relevant feedback, confidence ≥0.70, nonempty text and a general or task-specific scope; task-completion, factuality and “other” targets are removed. General rules need two supporting records. For each scope/target, one rule survives ranking by confidence rounded to two decimals, support, explicitness and recency. The renderer deduplicates wording and packs whole rules within 512 tokens; the compiler allows eight instructions of at most 500 characters each. Selecting no_memory clears this list while the other selected settings remain active (revised manuscript, p. 16, lines 819–832).

Where instructions are lost. Of 2,241 newly accepted instructions in the update/exploration study, 232 have no later task. The remaining 2,009 reach the following furthest stages:

Outcome before a later host callInstructions
Removed during card construction1,333
Excluded by task-type routing125
Retrieved only under no_memory365
Entered a compiled instruction list186

Card losses comprise 1,309 rules outranked for the same target/scope and 24 general rules lacking a second supporting record. Deduplication, token budgets and compiler limits remove no additional instruction. The initial study has the same ordering: 919 card losses, 495 suppressed by no_memory, 162 routing losses and 619 compiled instructions among 2,195 with later opportunities. Each origin is counted once at its furthest later stage (Table 17; revised manuscript, p. 27, lines 1426–1455).

The omissions can remove useful content. Blind judgments find additional content in 59–82% of lost origin–target pairs, although agreement is only fair (κ 0.33–0.35) and answer-level effects were not tested. Routing excludes rules tagged as problem-solving/proof even though each judge finds 283/287 applicable to calculation (280 jointly). This supports information loss through selection and routing (revised manuscript, p. 27, lines 1454–1457; p. 28, lines 1461–1464).

Confidence and the update threshold. Confidence is a generated JSON field in [0,1]. The extractor learns profile-blind teacher labels. The inclusive 0.70 cutoff is an operating choice; all valid scores in the eight shared-extractor studies were at least 0.75, so it excluded no additional valid label. Valid feedback from a successful update-on call can be stored at any confidence. A numerical update additionally needs relevant, polarity-consistent feedback: positive with rating 4–5 or negative with rating 1–2. Mixed feedback can supply text instructions but no numerical update. The selected, executed action receives (rating − 3)/2, without confidence weighting (revised manuscript, p. 20, lines 1071–1078; p. 21, lines 1080–1087).

The source-learner experiments use separate retrieval: released CIPHER retrieves up to three contexts by BERT similarity, while PAHF retrieves up to ten entries from the current user's memory with DRAGON and summarizes them. These instructions bypass the shared card packing and memory filter (revised manuscript, p. 16, lines 833–839).

What can be reproduced. An anonymous package is prepared with derived ledgers for the two instruction-path studies, recomputation code and synthetic checks. These run without private conversations or model calls; rebuilding the histories requires the saved private inputs. Upon acceptance we will release benchmark and evaluation code, including interfaces, controllers, rule processing, SDK integrations, configurations and tests. Private conversations, hidden profiles and unreviewed checkpoint weights are excluded. The revision states this release scope rather than implying that the conversation dataset is public (revised manuscript, p. 36, lines 1906–1921).

AuthorsOpenReview ↗

Replies to Reviewer JxTW - Part 3

Reply 3/3 — FABLE-Diag and SDK implementation (Q3, Q4; W1)

Thank you for asking how FABLE-Diag differs from FABLE and how each SDK adapter is implemented. We clarify how diagonal precision changes both posterior means and uncertainty, then show how the adapters pass the selected memory, tool and style settings to each SDK.

FABLE versus FABLE-Diag. Both score each permitted memory/tool/style setting as b + φᵀθ, using fixed default utility b and the same full feature vector φ (265 coordinates for the controlled catalog). Both start with a zero-mean unit Gaussian prior, observation variance 0.25, and update from the residual y − b. FABLE updates full precision Λ ← Λ + φφᵀ/σ² and h ← h + φ(y − b)/σ², giving mean Λ⁻¹h. FABLE-Diag keeps diagonal precision d, giving mean h/d and covariance diag(d)⁻¹. With exploration off they use the mean; with exploration on they take one unit-scale Gaussian draw (revised manuscript, p. 17, lines 908–916; p. 18, lines 918–929).

All features remain in FABLE-Diag. For repeated observations of the same feature vector with k nonzero coordinates and residual r, the full model's residual prediction approaches r, whereas the diagonal prediction approaches kr.

Does heuristic promotion explain its behavior? Only the separate scripted configuration-choice diagnostic supplies numerical onboarding. FABLE-Diag can then add a weight-0.8 observation from its own prediction after eight supporting updates and a working sign probability of 0.9 (or 0.1 for the opposite sign), once per direction. Source FABLE's confidence-sequence writeback is different and leaves its posterior unchanged. In the host-execution studies there is no numerical onboarding, promotion never triggers, and its recorded sentences never enter the host's instructions (revised manuscript, p. 18, lines 931–939).

In the five-seed scripted diagnostic, disabling promotion changes each of five measured outcomes by at most 0.0005, with 95% intervals containing zero and lying within ±0.0013. FABLE-Diag still has higher configuration alignment but lower adequacy and reward, and higher regret and cost, than FABLE (Table 5). Promotion therefore does not explain that trade-off; the diagonal variant is not uniformly better (revised manuscript, p. 18, lines 941–955).

How each SDK receives a selected configuration. A shared compiler turns the selected memory/tool/style settings into a control block and a permitted tool catalog; the task request itself is unchanged. The control block contains one instruction per setting and, except under no_memory, up to eight stored preference instructions. Table 8 now gives the concrete entry points:

SDKAgent and field carrying control text
AutoGenAssistantAgent, system_message
CrewAIOne Agent/Crew, Agent backstory
LangChaincreate_agent, system_prompt
LangGraphcreate_react_agent, prompt
LlamaIndexFunctionAgent, system_prompt
PydanticAIAgent, system_prompt
StrandsAgent, system_prompt

LangChain's entry point runs on LangGraph. These are SDK integrations with distinct loop behavior. The controlled runs share Sonnet 4.6, temperature zero, 2,048 output tokens and a twelve-request relay limit, with stated exceptions for host/budget sensitivity tests. Tools enforce user/task-bound records, memory keys and commit permissions; memory deletion requires separate authorization. (revised manuscript, p. 20, lines 1026–1060).

Delivery is checked separately. In the accurate-instruction study, all 17,356 forwarded requests are checked for the supplied instruction. Six SDKs preserve the system text and function schemas; CrewAI embeds the text and parses textual actions. For 6,622 successful executions, the last forwarded request matches logged tool events in order, arguments and return text p. 20, lines 1062–1070).

Thank you for your detailed responses, which resolve some of my concerns. I am keeping my original rating because the remaining issues are not changed by the rebuttal.

  • Reproducibility is still limited: the code is released only upon acceptance, and rebuilding the histories requires private inputs.
  • Finally, most compared methods remain adaptations of the originals.
Reviewer_Y5zoOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AgentBake, a benchmark which assesses whether agents that receive information about a user's preferences behave in accordance to those preferences over the course of a full, potentially long conversation.

Critical strengths

The high level conceptual idea behind this benchmark is sound and interesting. The authors do a good job in clarifying how the question they seek to answer is distinct from existing work in this space.

Critical weaknesses

The experiments section is greatly underspecified.

The actual tasks are not sufficiently described in the main body of the paper, and are barely expounded up on in the Appendix. Task completion scores hover near 100% which indicates a lack of diversity around task difficulty. I can imagine both task difficulty and the general "type" of the task influencing style match scores, but this is not assessed.

From what I can make out from the paper, it seems that a simple "calculation" task is the only one studied. From how this task is described, it's not clear that one would expect substantial variation in personalization to arise in this type of task. It's not clear why this task was chosen over the types of tasks that the authors used to motivate the problem in the intro (i.e. trip planning).

The personalization methods studied are also underspecified, and the authors point out they are adaptations of these methods. Without going through and reading all of the papers, it's hard to understand what the differences between the methods are and how that might be influencing the results. More space should be dedicated to this subject in the main body of the paper.

Only one model (Sonnet 4.6) is evaluated across personalization methods. I imagine there are non-trivial interaction effects between the personalization methods tested and the models, these are not analyzed but should be.

Once the above issues are fixed, it would also be helpful to add a qualitative analysis section to better understand failure modes of the different methods.

Questions for authors

What are the tasks that you evaluated models on?

3 comments
AuthorsOpenReview ↗

Replies to Reviewer Y5zo - Part 1

Reply 1/3 — What tasks do we evaluate, and why use calculation?

Thank you for asking which tasks we evaluate and whether calculation provides enough room for personalization. We use calculation to separate a fixed, checkable answer from the user’s preferred explanation style. We also test the itinerary setting used in our original introduction and report early style-learning results on itineraries and support tickets.

What agents do. The main calculation comparison fixes the required answer while varying how the user wants it explained: read three site records, inline or through lookup; calculate total units and remaining capacity; identify the largest site; submit the exact result; then explain it. For example, the correct result can be 164 units, 100 remaining capacity and site B, whether the user wants a one-line answer or a detailed derivation. The revision gives the task in §4.1 (p. 4, lines 206–211) and a feedback-to-later-answer example in Appendix A (p. 14, lines 704–746).

We added or clarified the following distinct evaluations:

TaskRequired result and personalization
Support-ticket routingApply seven ordered rules and submit the first matching queue/rule ID; adapt the explanation. Eight training and four held-out tickets.
Constrained itinerariesFind the cheapest feasible itinerary under budget and indoor-activity constraints; adapt its explanation. Eight standard and four complex tasks per family in the matched comparison; four new held-out itineraries in the learning test.
Harder calculationIncrease records from three to 48 sites and vary the output budget; test correctness and instruction responsiveness.
Application requests200 held-out generated request slots, 188 distinct request–goal pairs from 66 projects; compare preference-instruction formats on saved adapters.

These supplement the matched twelve-controller calculation comparison. The separate scripted memory study tests authorized operations, while 50,000 simulated conversations train the extractor (revised manuscript, pp. 42–44, lines 2232–2375; p. 5, lines 223–227).

High correctness leaves measurable style variation. On the same Sonnet 4.6 host, four profiles and two SDKs, own rather than opposing instructions increase full-style matches as follows:

Standard task, explanation requestAstra judgeSonnet judge
Calculation+29.7 [26.6, 31.2] pp+18.8 [15.6, 21.9] pp
Itinerary+28.9 [22.7, 36.7] pp+4.7 [0.8, 9.4] pp

The 99% task-cluster intervals adjust five contrasts. Both judges detect responsiveness in both families; only Sonnet establishes a smaller itinerary gain. These completed-schedule estimates include a continuation decided after the first results; the initial run has the same directions, but its missing-outcome bounds include zero. Neutral wording increases instruction gains under both judges, with changes in both generated answers and judging context. Thus the explanation request affects the measured effect without eliminating room for personalization (Table 36, revised manuscript, p. 44, lines 2326–2346).

Learning on the motivating task. With eight training and four held-out itineraries, four synthetic users and five seeds, released CIPHER-N gains 45.0 Astra points [36.2, 53.8] and 22.5 Sonnet points [20.0, 25.0] from initialization to eight feedback opportunities; all 240 evaluations are correct. Intervals are 97.5%, adjusted over two learners. PAHF's gain is undetected, and its tool-free adapter fails on 70/120 lookup evaluations, so that result does not isolate its learning rule (revised manuscript, p. 44, lines 2348–2358).

On held-out tickets, CIPHER-N and PAHF also improve early style under both judges: Astra gains are +31.25 [18.75, 43.75] and +43.75 [36.25, 53.75] points, with descriptive 95% five-seed intervals and near-ceiling correctness. Further gains are weaker and judge-dependent (Table 34, p. 42, lines 2232–2263). These controlled, simulated-user studies establish preference variation and early learning beyond calculation; itinerary personalization here concerns communicating a fixed correct plan.

AuthorsOpenReview ↗

Replies to Reviewer Y5zo - Part 2

Reply 2/3 — How do the methods differ, and where does personalization fail?

Thank you for highlighting the need to explain the adapted methods and analyze their failure modes. We now describe their mechanisms and differences from the source systems, then trace preferences from storage to later host input and behavior. These results distinguish a failure to deliver preferences from a failure to learn them.

What the adapted methods actually do. All twelve personalization controllers share a fixed feedback extractor and text-instruction path, maintain separate user states, and select among 108 permitted memory/tool/style configurations. Their numerical mechanisms differ:

  • FABLE uses full-precision Gaussian residual updates and sampling; FABLE-Diag changes both means and uncertainty with diagonal precision.
  • PUMA retrieves similar structured contexts to score actions; PEToolLLaMA learns tool-mode logits; APOLLO applies a one-step policy-gradient update.
  • SmartAgent uses reward means; ODYSSE uses local/episode rewards; PARPO keeps base and user rewards separately.
  • PersonalAlign/HIM uses context clusters; PAHF uses global/task-specific numerical notes; CIPHER uses retrieved action-feature contrasts; TAPS uses component tags and a fixed gate. These four do not explore.

The adapters do not train the source language or GUI models. In particular, compact CIPHER omits edit-to-preference inference and compact PAHF omits textual preference retrieval and clarification. The revision makes these departures explicit in §4.2 and in the per-method Table 4, so readers can understand the comparison without reconstructing the source papers (revised manuscript, p. 5, lines 229–248; p. 17, lines 864–906).

We separately tested released learning routines. CIPHER-N retains edit-to-preference inference, BERT retrieval and aggregation; PAHF retains retrieval, pre-action questions and feedback writes, with adapted runners and tasks. By eight feedback opportunities, style rises under both judges: CIPHER-N +30.0 Astra/+26.0 Sonnet points and PAHF +10.8/+9.1. Early gains concentrate in one profile per learner and accompany lower task success. At 32 opportunities versus the state frozen after eight, PAHF improves style and joint success, while CIPHER-N's changes are undetected. (Appendix I and Table 32, revised manuscript, p. 40, lines 2145–2159; p. 41, lines 2169–2203).

Where preferences are lost. The new analysis follows actual instructions and contrasting replies (Table 37, revised manuscript, p. 45, lines 2376–2399):

  • Storage/routing: card construction excludes 1,333/2,009 new instructions that have a later task. In one APOLLO trace, a request for essential metrics loses its slot to a higher-confidence historical rule.
  • Delivery: one FABLE call retrieves seven rules but compiles none because it selects no_memory. Both judges still match its detailed reply: a matching answer alone does not demonstrate reuse.
  • Updating: updated and frozen compiled messages are identical in 320/320 CIPHER, TAPS and HIM pairs and 319/320 PAHF pairs. A state update need not change the next host input.
  • Exploration: post hoc matched pairs show more style losses than gains when the default concise setting is replaced, but a reverse SmartAgent example improves under both judges. On sixteen prospective new tasks, the eight exploring controls lose 14.5 more style points than the five others under Astra (adjusted 95% interval [−16.8, −12.1]; p. 43, lines 2305–2312).
  • Host behavior and judging: Sonnet violates the compact-output contract in 702/1,719 successful fixed-state replies versus DeepSeek's 0/1,720. The same structured reply can also receive partial/full verdicts from Astra/Opus.

An intervention tests the delivery explanation. At 1,760 saved no_memory positions, forcing packed rules, preference memory and its tools raises style matching by 13.9 Astra points [12.1, 15.7] and 7.4 Sonnet points over same-host re-execution. Native task success changes by −0.4 [−3.7, 2.8] points. The nonlearning host gains similarly (+14.4), and the intervention changes rules and memory/tool availability together; it supports improved delivery. Nine positions read memory and none writes it (revised manuscript, p. 44, lines 2362–2375; p. 45, lines 2398–2399).

AuthorsOpenReview ↗

Replies to Reviewer Y5zo - Part 3

Reply 3/3 — How do task difficulty and the host model affect the results?

Thank you for highlighting task difficulty and possible interactions between personalization methods and the host model. We now distinguish first-attempt correctness from final completion, test harder tasks, and extend the host comparison to DeepSeek V3.2 and MiniMax M2.5. MiniMax shows a method-dependent style effect under both judges; DeepSeek substantially changes which profiles are matched.

Why completion was near 100%. Final completion allows a rejected submission to be corrected within the same call. The three-mode comparison has 5,720/6,240 correct first submissions versus 6,141 final successes; the update/exploration comparison has 6,817/8,320 versus 8,085. We now separate these outcomes and describe the tasks explicitly (revised manuscript, p. 4, lines 206–211; p. 43, lines 2268–2272).

Difficulty and personalization. We vary structural complexity and record volume under four profiles, two SDKs and two replicates. At a 2,048-token output limit, increasing calculation records from three to 48 sites reduces first-submission value correctness from 384/384 to 233/384 and final completion to 289/384. The own-versus-opposing instruction gain changes by −17.2 Astra points [−24.2, −10.9] and −9.5 Sonnet points [−16.4, −2.5], with 95% instance-bootstrap intervals.

With 8,192 output tokens and longer execution limits, all 384 large-record tasks complete, and neither judge detects a smaller instruction gain; 43 first submissions still contain value errors. The jointly changed token/time budget prevents attributing recovery to tokens alone. DeepSeek's complex itineraries similarly improve from 134/192 to 192/192 completed tasks. Structural complexity alone is less consistent: the Sonnet host's instruction gain falls by 3.9 points under Sonnet judging, but Astra's interval includes zero. Thus difficulty affects the interpretation of personalization through task, host, judge and execution budget (Table 35, revised manuscript, p. 43, lines 2274–2296).

Method comparisons on harder tasks. We reused decisions saved by twelve methods and the host control, without further learning. Neither judge found a statistically supported switch in which method was better (75 pairs). All thirteen controls matched user style less often when each task contained data for 48 simulated sites. This replay used a 2,048-token output limit and different execution phases; it does not establish rankings after further learning.

Profile cards versus direct instructions. Separately, both judges found lower style matching with profile cards for knowledge explanation and research/analysis requests. Astra found no clear effect for planning (revised manuscript, p. 43, lines 2298–2312).

The benchmark now includes three host models. Sonnet 4.6, DeepSeek V3.2 and MiniMax M2.5 are compared across seven SDKs, thirteen controls, four profiles and five seeds: 2,080 calculation positions per arm. Fixed-state comparisons hold learned state, settings, instructions, tools and first input constant; later turns depend on the host. The alternatives pass task-disjoint compatibility screening with the documented adapter adjustment. DeepSeek also continues learning from shared states and separately re-learns eight warm-up events.

We test whether paired host effects vary across methods after removing their mean, requiring both Astra and Sonnet support for a style interaction:

Host comparisonAstra style pSonnet style pTask p
DeepSeek, fixed state.27.66.71
DeepSeek, continued learning.45.15.66
DeepSeek, re-learned warm-up.90.62.90
MiniMax, fixed state.046.025.44

Primary Astra/task p-values are Bonferroni-adjusted. MiniMax's APOLLO and PUMA effects are more negative than the average control: Astra residuals −16.1 [−30.7, −1.5] and −18.9 [−32.9, −4.9] points, with the same directions under Sonnet. Only PUMA also passes the adjusted five-seed sensitivity under both judges; neither task interaction nor the joint three-host style criterion is met.

DeepSeek's profile effects are large and have agreeing directions under both judges, despite opposite pooled style signs; its wide control intervals leave moderate interactions unresolved. MiniMax was evaluated later through a separate route and only with fixed state. Appendix N now states these results and their scope together (Table 38, revised manuscript, p. 45, lines 2404–2429; p. 46, lines 2430–2442).

Reviewer_7gneOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces AgentBake, a framework and benchmark for evaluating different personalization methods. It formalizes a personalized agent with (1) a personalization component that learns a user's preferences from simulated user feedback, and (2) a host agent that carries out the task given a user's preferences. While keeping the personalized component constant, this work explores various approaches for the host agent.

Critical strengths

The paper presents an interesting formalization for personalization and a benchmark based on simulated user feedback, and conducts various experiments to gain a better understanding of what works best.

Critical weaknesses
  • The paper was extremely difficult to understand. The main contributions and motivations are not clear, and many parts of the paper did not provide sufficient context to readers. For example, many acronyms are used but never clearly defined (CIPHER, PAHF, FABLE, FABLE-Diag), and there is no justification behind why personalization is formalized in the way that it is, or why the authors chose to evaluate the specific methods that they did. There is no clear problem formulation, or descriptions of the different methods. Information is also scattered across different sections, and the author's would benefit from better organizing it. My ability to review was largely impacted by missing information and poor clarity.
  • The evaluation uses 4 hand-crafted simulated-user profiles based on a small set of generic preferences, which is a very small number relative to existing benchmarks and may not be very representative of real users. It is insufficient for a personalization benchmark.
Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Replies to Reviewer 7gne

W1. Motivation, contribution and problem formulation.

Thank you for identifying where the presentation prevented a clear assessment. We have reorganized the revision around one question: when feedback states a user's preference, does the agent use it on a later task? A correct calculation can still violate a request for a concise answer; storing that request does not show that it reached the host or changed its response. The introduction now develops this example before introducing the framework (revised manuscript, p. 1, lines 33–51).

What AgentBake contributes. The revised introduction states three contributions together: (1) an interface separating the personalization component from the executing host, with records of how feedback becomes a later host input; (2) matched comparisons of twelve personalization controllers and a non-learning host control, including separate tests of updating and exploration; and (3) evidence locating failures between preference recognition, storage, delivery and execution. This lets us distinguish successful task completion from successful personalization, and a learning failure from a failure to deliver what was learned (p. 2, lines 75–90).

One clarification to our original presentation is essential: the main comparison varies the personalization controller, while matching the host model, tasks, users, extractor and seed. The SDK is assigned once per user–seed block. Separate supplementary comparisons vary the host or evaluate transfer. Table 1 now states what changes and what is held fixed in each study, so these comparisons need not be inferred from scattered descriptions (p. 2, lines 54–68).

Why separate a component from its host? A user preference may be recognized correctly but omitted from the host input; conversely, the host may happen to produce a matching answer without learning anything. An external component makes these stages observable while leaving the host's model, planner and task permissions unchanged. It chooses supported memory, tool and response settings, and supplies accepted preference instructions to the host. The host still executes the task. This is the motivation for the split, now stated before the mathematical formulation (Section 3.1, p. 3, lines 141–148).

The learning problem. For user uu, the component maintains state SuS_u: learned scores, feedback records and instructions. Given task xtx_t, it chooses an allowed setting ata_t; the compiler forms host input pt=C(xt,at,It)p_t=C(x_t,a_t,I_t) from that setting and the accepted instructions. The unchanged host returns a response and execution trace. A simulated user reacts, and a profile-blind extractor turns that reaction into the record used to update SuS_u. The controller does not receive the hidden profile or evaluator judgments. Section 3.2 now defines this loop and its 108 supported settings together (p. 3, lines 152–159; p. 3, lines 160–161); Algorithm 1 gives the steps in execution order (p. 15, lines 781–797).

The worked example then follows one request from feedback to stored instruction, later host input and response (Section 3.3, p. 4, lines 188–194). We evaluate task success and observed preference matching separately: a selected setting or a stored instruction is not counted as a matching response, and missing evidence remains unknown (Section 4.3, p. 5, lines 254–264; pp. 5–6, lines 265–276). Together, the motivating example, formulation and worked loop provide the context that the submission lacked.

AuthorsOpenReview ↗

Replies to Reviewer 7gne - Part 2

W1, continued. Method definitions, selection and organization.

The introduction now expands the acronyms at first use, and Section 4.2 introduces the mechanisms before the results (revised manuscript, pp. 1–2, lines 52–73; p. 5, lines 229–242).

  • CIPHER (“Consolidates Induced Preferences based on Historical Edits with Retrieval”; Gao et al., 2024) infers preferences from user edits, retrieves relevant earlier preferences and combines them for a new request.
  • PAHF (“Personalized Agents from Human Feedback”; Liang et al., 2026) maintains per-user preference memory, can ask clarifying questions before acting, and updates memory from feedback.
  • FABLE (“Factorized Adaptive Bandit Layer for Execution”; Jin et al., 2026) is a policy outside the host that chooses memory, information-acquisition and response settings and learns user-specific residual preferences through Bayesian contextual Thompson sampling.
  • FABLE-Diag is our diagnostic variant, rather than a separate published source method. It uses diagonal posterior precision and a promotion rule. Diagonalization changes the posterior mean as well as uncertainty; promotion does not trigger in the execution studies. The revised description makes these distinctions explicit (Appendix C.3, pp. 17–18, lines 908–939).

What is actually compared? The main study adapts selected learning rules to the same observations and 108 host settings. All twelve personalization controllers keep accepted instructions in their own state; their numerical scores, update rules and use of exploration differ. For example, our adapted CIPHER learns scalar-signed action-feature contrasts instead of inferring preferences from user edits, while adapted PAHF uses numerical notes instead of text retrieval and clarifying questions. Thus this study compares adapted controllers under shared conditions; it does not reproduce the complete source systems.

Table 4 now puts the implemented mechanism beside its departure from each source, including PUMA, SmartAgent, PEToolLLaMA, APOLLO, ODYSSE, PARPO, PersonalAlign/HIM and TAPS. Its caption explicitly states that source language-model checkpoints, training and full application pipelines are not reproduced (p. 17, lines 864–894). This table and the common loop make the names interpretable without requiring readers to reconstruct implementations from the result tables.

Why these methods? We select methods whose per-user learning can be mapped to the memory, tool and response controls exposed by the hosts. They span the properties being tested: which settings receive learned scores and whether the controller explores. Eight support exploration and four do not; the non-learning host provides a common control. The selection rationale and method overview now precede the comparisons (p. 3, lines 131–135; Section 4.2, p. 5, lines 229–242). Table 4 identifies these properties for every controller.

Does the finding extend to the released learners? We test released CIPHER-N and PAHF learning routines separately, retaining their edit-based inference or question-and-feedback procedures. Both improve style matching within eight feedback opportunities under two judge families, mainly for one profile each, while native task success is lower at that point. Continued PAHF updating then improves both style and native task success relative to its state frozen after eight opportunities; CIPHER-N's additional-learning effect remains uncertain. These results limit any claim that the main adapted-controller findings characterize the complete source methods (Section 5.4, p. 9, lines 448–465; Appendix I, Table 32, p. 41, lines 2180–2195).

The revised reading order is now: motivating problem and contributions; component–host formulation and worked example; method definitions and selection; controlled results; then detailed mechanisms and follow-ups in the appendices. The main text points directly to the relevant table or appendix. This addresses both the missing context and the fragmentation highlighted in your review.

AuthorsOpenReview ↗

Replies to Reviewer 7gne - Part 3

W2. More users and the connection to real preferences.

We agree that four handmade profiles do not establish a representative personalization benchmark. They provide controlled cases for tracing the feedback-to-execution loop. We therefore expanded the study from four to 80 profiles: the four originals, 40 derived from PRISM participants' self-written instructions, and 36 answer-style configurations, and separately examined which preferences expressed by PRISM's 1,500 participants the interface can represent (revised manuscript, Appendix O, pp. 46–47).

Does the larger study support the central finding? Under Astra, supplying each profile's own instructions gives 291/640 (45.5%) style matches, versus 138/640 (21.6%) without instructions and 133/640 (20.8%) with another profile's instructions: gains of 23.9 and 24.7 percentage points. Task success remains similar—629/640, 627/640 and 631/640—while joint success is 284/640, 135/640 and 132/640. Thus the instructions change preference matching (Table 39, p. 46, lines 2461–2472).

The own-versus-no-instruction gain appears in both new groups: 22.5 points [12.5, 33.4] for PRISM-derived profiles and 23.3 [12.8, 34.4] for the combinations, with 95% user-cluster bootstrap intervals. Judging the same answers against the target and a mismatched profile also gives a positive instruction-specificity effect in all three groups. These checks go beyond adding more copies of the original four profiles (p. 46, lines 2475–2477; Table 40, p. 47, lines 2484–2493).

What does feedback-based learning achieve? At the final checkpoint, released CIPHER-N reaches 134/320 (41.9%) style matches, compared with 73/320 (22.8%) for the non-learning host; task success is 310/320 versus 305/320, and joint success 134/320 versus 67/320. PAHF reaches 86/320 (26.9%) style matching, but task and joint success are lower, at 218/320 and 64/320 (Table 39, p. 46, lines 2461–2472).

CIPHER-N's early-learning gain is positive in each new group: 21.3 points [10.0, 33.8] for PRISM and 18.1 [7.6, 30.6] for the combinations, using 97.5% intervals for the two learner contrasts. Its gain after accounting for non-learning-host drift is also positive in both. PAHF's early gain is not established in either new group, and neither learner shows a detected continued-learning gain over its four-feedback frozen state. CIPHER-N's early-learning specificity is positive for the combinations, 25.0 [12.5, 39.6], but remains uncertain for PRISM, 11.9 [−5.0, 29.4]. We retain these mixed outcomes: early improvement does not establish continued or target-specific learning everywhere (p. 46, lines 2477–2482; Table 40, p. 47, lines 2484–2493).

How representative are the supported preferences? Both model coders identify at least one in-channel preference for 68.7% [66.3, 71.0] of PRISM participants, most often tone or style. However, among those participants, the authored levels cover all their stated in-channel settings for only 51.9% [48.9, 55.0]. A further 20.8% of all participants express only out-of-channel preferences. Factuality, content and values are among the excluded dimensions. These are model-coded results without manual validation, and the crowdworker sample does not establish population representativeness (p. 46, lines 2446–2452).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.