acceptodds
Withdrawn from ICLR 2027

Remember the Alternatives: Choice-Context Memory for Personalized Decision Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

An embodied agent receives the instruction sit somewhere in a living room. Several seats may all be reachable, safe, and consistent with the instruction. Once these explicit constraints are satisfied, the remaining ambiguity is personal: which seat does this user prefer? Past choices can help resolve it. Choosing a nearby wide chair over a distant chair may suggest that distance matters in that comparison, while choosing the same chair over a nearby narrow seat may instead provide evidence for a preference over seat width. These observations do not uniquely identify the user's preference, but they carry different information about it. If memory stores only the selected wide chair, the two interactions become indistinguishable. Recent personalized agents increasingly use persistent memory to adapt across interactions. Most work focuses on how stored experiences are organized, updated, and retrieved. We instead ask what information a choice event should preserve before it enters memory. We identify a failure we call choice-evidence aliasing: different choice contexts can collapse into the same stored outcome, even when they provide different evidence about user preferences and can support different future actions. We formalize the information lost by this compression and show that it can change a later optimal decision. This motivates choice-context memory, which preserves the selected target together with the alternatives. Building on this principle, we introduce AffordanceMem, a personalized decision agent for human-scene interaction. It combines current instruction and feasibility constraints with preference beliefs inferred from past choice context, then decides whether to execute a target, ask for clarification, or defer. Clarification responses will be stored as new choice-context memories. We evaluate the framework on HUMANISE-ScanNet and LINGO with controlled preferences and repeated interactions. Matched comparisons and targeted interventions show that preserving choice context improves later personalized decisions and can change whether the agent acts or asks. These effects persist across choice mechanisms, interaction sessions, and scene constructions. Our results support choice context as a decision-relevant memory unit for personalized agents.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Agent Memory, Personalized Agents, Preference Learning, Selective Decision Making, Choice Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zMNjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AffordanceMem, a personalized decision agent that preserves both selected targets and their alternatives as choice-context memory. It uses historical choices to infer user preferences and an expected-regret policy to decide whether to execute, ask, or defer. Experiments on HUMANISE–ScanNet, LINGO, and a two-session human study demonstrate improvements over chosen-only memory.

Critical strengths

S1. Clear motivation and formulation.

The paper identifies a meaningful limitation of chosen-only memory and connects retained choice evidence to preference inference and downstream decisions.

S2. Comprehensive empirical evaluation.

The parameter-matched comparisons, targeted interventions, and cross-session human study provide useful evidence for the benefits of preserving choice context.

Critical weaknesses

W1. Limited novelty and technical contribution.

The importance of choice alternatives is already well established in discrete choice modeling and preference inference. Proposition 1 follows from a standard information-comparison argument, while the preference models and regret-based policy mainly rely on existing techniques. The overall technical contribution appears relatively limited.

W2. Lack of selective long-term memory management.

The framework retains historical choice contexts without investigating which interactions are worth remembering, how redundant or outdated memories should be handled, or how to operate under limited memory budgets. Given the emphasis on persistent personalization, the short interaction histories (6 or 10 events) leave this important aspect insufficiently explored.

W3. Limitations of the evidence-erasure experiment.

The 890 evaluation pairs are preselected based on reference EXECUTE-to-ASK transitions, limiting the generality of the reported results. More importantly, the intervention in Section 4.3 replaces all unchosen candidates' features with those of the selected candidate. Under the Bank model, this makes all candidate utilities identical, yielding p(y∣S,h)=1/∣S∣p(y \mid S,h)=1/|S| in Eq. (3) and consequently q(h∣Herased)=p0(h)q(h \mid H_{\mathrm{erased}})=p_0(h). Thus, the intervention necessarily eliminates all historical preference information under this model, making the observed increase in uncertainty largely expected. Less degenerate interventions and evaluations of downstream regret on the full test distribution would provide stronger evidence.

Questions for authors

Q1. Given that Eq. (3) allows recursive Bayesian updates of the preference posterior, what additional benefit does retaining historical choice-context records provide over maintaining only the updated posterior?

Q2. Under a limited memory budget, how would selectively retaining informative choice contexts compare with retaining all available contexts? Could the authors evaluate this setting with longer interaction histories?

Q3. Could the authors report decision changes and downstream regret before and after evidence erasure on the full test distribution, rather than only on preselected EXECUTE-to-ASK pairs? Do the conclusions hold under less degenerate interventions?

Reviewer_DLZ8OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers an embodied AI task in which an agent receives a request and must choose among several possible options. The agent must autonomously decide, based on past experience, whether to execute an action, ask for additional information, or defer.

The paper identifies a failure mode, termed choice evidence aliasing, in the standard outcome only strategy, where the agent evaluates past experiences based on the actions taken, without considering the alternatives available at the time of each decision. The key observation is that the information that can be inferred from an agent’s final decision depends heavily on the context in which that decision was made.

To address this issue, the authors introduce choice context memory, which preserves not only the action ultimately selected by the agent but also the entire set of options available at the time of the decision. The proposed agent, called AffordanceMem, employs a Bayesian approach to update its beliefs based on past decisions.

Critical strengths

The problem addressed in the paper is somewhat interesting, and I appreciate the formalization of the agent’s beliefs.

Critical weaknesses

The choice evidence aliasing failure is rather obvious, and it is quite clear why choice context memory works better, particularly in a setting where the correct candidates are provided as input. Furthermore, the formalization of Proposition 1, and even more so its formal proof in Appendix A, resemble an informal argument rather than a rigorous formalization of the problem. Specifically, the proof is largely discursive and relies on notation that is never properly defined: \pi_G, \pi_X, the function g, and the hypothesis h are only described in words.

Moreover, the proposed approach relies on very strong assumptions. Specifically, candidate proposals, feasibility checks, and motion execution are all provided to the agent a priori. Moreover, the features relevant to the decision-making process are themselves directly provided to the agent as inputs. These assumptions substantially limit the generality of the proposed method, to the point that I question how applicable it would be in more general settings.

The authors explicitly acknowledge some of these assumptions and justify them in the conclusion by arguing that they allow the effect of memory content to be isolated, leaving the learning of candidate construction to future work. However, I see three major issues with this justification.

First, the authors do not acknowledge the assumption that the features available to the agent are precisely those needed to make the decision. Second, I do not believe that simply stating that these assumptions will be relaxed in future work is sufficient. A more thorough discussion of how the authors intend to relax them, including the potential challenges involved, would be necessary.

Third, I find the decision to isolate the effect of memory from the effect of learning to be particularly problematic, as these two aspects are closely intertwined. Specifically, both the theoretical arguments and the experimental analyses focus on a setting in which the options available to the agent are perfectly known. However, when moving to a setting involving learning, one should expect some degree of noise.

I believe it is quite plausible that the proposed method is substantially more sensitive to noise than the outcome only strategy. In the former case, noise may affect every individual candidate, whereas in the latter, its impact is arguably more limited, as it only concerns the action actually selected by the agent. For example, suppose the agent observes a chair in the distance and incorrectly perceives it as wide, whereas it is actually narrow. This could lead the agent to incorrectly conclude that the decision was driven by distance rather than width.

More generally, I believe the relevant research question is not whether choice context memory outperforms outcome only memory when the candidates are correctly identified, but rather how it behaves in settings where candidate generation is learned and potentially imperfect. In my view, this question should be investigated both theoretically and experimentally.

Questions for authors
  • How do you intend to extend the approach in context where the model is learned end-to-end?
    • And what are the potential risks when moving in those kind of scenarios?
  • How do you formalise an hypothesis h? How it is actually implemented?
  • In proposition 1, what is the difference between G=g(X)G = g(X) and YY? To me, they seem to carry the same information (i.e., the chosen item)
Reviewer_qV4cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that a personalized agent's memory should store each user choice together with the alternatives that were offered, not only the selected item, because two choices with the same selection but different offered sets carry different evidence about the user. Contributions are: (i) a Blackwell type proposition that the chosen only record can only increase achievable Bayes loss, with a finite example where it strictly does; (ii) AffordanceMem, which keeps pairs, infers a belief over linear preference weights on eight geometric features, and uses expected regret to “execute”, “ask”, or “defer”; (iii) a controlled evaluation. Validation: Humanize-Scannet episodes with synthetic preferences, where a parameter-matched normalizing flow gains 18.9 points of hidden-target accuracy from seeing the alternatives; an evidence-erasure intervention; a selective-action cost analysis; a user study; and a frozen-threshold transfer to LINGO.

Critical strengths
  1. The two normalizing-flow models in Table 1 share the same parameter count, decoder, training objective and current candidate set; the only difference is whether the history encoder sees the unchosen alternatives. The accuracy gap is large, it holds across six non-MNL choice mechanisms, the splits are scene-disjoint, and the intervals come from a scene-level bootstrap. This is the evidence the central claim needs, and it is the reason I consider the paper sound on its core point despite the issues below.
  2. The memory-to-action framing with real-user validation. Tying memory content to the execute/ask/defer decision and providing and extensive user study, with a paired CI, plus a LINGO transfer with frozen thresholds, goes beyond the synthetic-only evaluation common in this space. The selective policy beating both always-act and always-ask on human data a meaningful practical result.
Critical weaknesses
  1. The core idea is not new, and the method the paper builds on top of it does not add much. A choice only makes sense relative to what was on offer; that is the starting point of every random-utility model since Luce and McFadden, and Freedman et al. (2020) and Jonnavittula & Losey (2021) have already worked out what happens in reward inference when the choice set is wrong or exploited. Proposition 1 is Blackwell's theorem applied to a deterministic garbling (stated in the paper). So the chosen-only baseline is a baseline nobody in choice modeling would build, and the comparison that would actually tell something, against a choice-set-aware learner, is missing??. The paper cites Seshadri et al. (2019) and KnowNo but compares to neither, and opens with SeCom, A-MEM and AdaMem without running any LLM memory baseline. The authors' own numbers then show that the proposed estimators are not the point either: Laplace ties Bank on the human data (55.19 vs 54.88, Table 3), and is better on dense and drifting preferences, and on ask AUPRC at short histories and high corruptio. What is left is "keep the offered set", shown in an eight-dimensional geometric feature space with no semantic attributes. And in a real embodied setting the scene is observed, so the offered set can be rebuilt from a stored scene; the paper never says whether aliasing is a problem in practice or only when memory is reduced to a symbol for the chosen object.
  2. Two of the four experiments do not show what the text says they show. The erasure study replaces every unchosen feature vector with the selected one. That makes all items in S identical, the likelihood in Eq. 3 becomes 1/|S| for every hypothesis, an the posterior is the prior (for the NF trace in Figure 4, the encoder receives chosen only equivalent input: selected features and zero differences). Regret going from 0.001 to 0.552 and the agent switching to ask is just what no memory does; it is not evidence about memory content. Appendix B.2 also confirms the reference policy is a 256-particle posterior of the same model family, so the 92.13% “recoery” is bank-versus-particle agreement on the preselected execute-to-ask transitions, and the lower Laplace and MAP numbers say how well they approximate that posterior, nothing more. The selective-action result has a different problem: “Bank + regret” against “MAP + entropy” (Fig. 5a, 8.19% vs 15.03% at 70% coverage) changes the estimator and the gating rule in one step (Sec. 4.4 says so itself, calling them complete strategies), and Table 7 only compares estimators under the same rule, so the regret gate is never isolated. Table 2 makes me think calibration is driving a lot of it, since Laplace covers 6.1% at cost 0.05 where Bank covers 46.6%. The abstract and conclusion rest on both of these results, so the claims are stronger than the evidence. Both are fixable: a chosen-only versus choice-context act/ask comparison on the same episodes, and the full estimator x gating grid.

Minor: • Inconsistent: "AffordanceMem" (Sec. 3 opening sentence, Sec. 5 paragraph 4) versus small caps elsewhere. (Double Check for more) • The chair A, B and C example is repeated three times in the first two pages; Figures 1 and 2 convey the same pipeline. • Figure 6 shows motion executions that are supplied components, not contributions; the space would be better spent on confidence intervals for Tables 1-3 or the missing ablation grid. • Table 5 mixes evaluation populations across columns within one row (ID on the reference-execute subset, non-MNL on all free choice sources); the authors disclose this, but it invites misreading. • Table 7's Laplace > Bank results on ask AUPRC belong in the "no estimator dominates" discussion of Sec. 4.5 rather than only in the appendix. • "Hidden-utility" (Table 2, top part) is used without definition in the main text. • The reproducibility statement contains no mention of code or data release, including the user-study data

Questions for authors
  1. In Sec. 4.3, replacing unchosen vectors with the selected vector makes the Eq. 3 likelihood constant in h, so the posterior is equal to the prior. How does this differ from "no memory"? Appendix B.2 shows the reference policy is a 256-particle posterior of the same model family, so the 92.13% recovery measures bank-vs-particle agreement. Why not compare the act/ask decisions of the choice context model against the matched chosen-only model on the same episodes?
  2. Please report the full estimator x gating grid (Bank / Laplace / MAP x regret / entropy) for Fig. 5a. Table 7 holds the gating rule fixed, so the paper currently cannot say whether the Fig. 5a gap is due to the estimator or to the rule.
  3. User study (C.3): how many of the 96x28 = 2,688 Session B responses were kept after platform-side exclusion, and is the exclusion independent of task difficulty?
  4. Please define "Acceptable (%)" in Table 3 (B.2 refers to an "acceptable set" without defining it). Which platform was used, and was the study ethics-approved with informed consent and compensation?
  5. Fig. 4 uses a fixed threshold of 0.123 and B.2 a reference threshold of 0.15. Are these different models' validation-calibrated thresholds? For Table 2, was a separate lambda calibrated per clarification cost?
  6. B.3 states the matched chosen only NF differs only in the history-encoder input. Does the chosen only encoder receive context-standardized features (which require the offered set) or raw features? If raw, feature normalization is a second difference between the "matched" models.
  7. For LINGO (C.4): are the historical choices generated by the controlled preference functions over clustered human endpoints, or are they the participants' actual endpoint choices? If synthetic, the transfer is of the episode-construction pipeline rather than of the principle to real behavior.
  8. Which model is "Matched chosen only" in Table 3: the NF of Table 1 or an MNL variant?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.