acceptodds
Under review as a conference paper at ICLR 2027

GCE: How much of a user's experience do language models actually use

Abstract

Persistent LLM assistants build up a long record of one user's decisions. Yet when such an assistant mispredicts a new decision, existing memory benchmarks cannot tell whether the history lacked the answer or the system failed to use it: they score against noisy labels, with no known bes t achievable loss. We introduce Generative Cognitive Evaluation (GCE), which measures how much of the predictive information in a user's experie nce a system actually uses. Each synthetic user is an executable preference model drawn from a declared finite family. Its choices are rendered as prose sessions that never reveal the schema, and exact enumeration gives the Bayesian posterior predictive for every held-out decision. This reference splits a system's loss into choice noise, uncertainty the history cannot resolve, and the system's own deviation, and defines an infor mation-use ratio : for the constant forecast , for exact Bayesian inference, and negative for worse than the constant. Across six preference families, the best of three frontier LLMs reading the full history (at low reasoning effort) realizes about three fifths of the available gain (), while the other two do no better than the constant ( and ). Summary, profile and retrieval memo ries do not close the gap, and two off-the-shelf memory systems add nothing beyond them: Mem0 behaves like episode retrieval, and Letta like in- context reading, since its agents rarely write memory. Using the LLM only to parse prose into features and fitting a simple statistical model re aches and on the two GPT backbones, close to logistic regression on the true features (). This suggests that the inform ation is recoverable from the prose and that much of the gap lies in how models aggregate evidence. GCE is a controlled diagnostic conditional o n the declared family, not a measure of performance on human preferences.

Primary area:
datasets and benchmarks
Keywords:
LLM memory evaluation, personalization, Bayesian inference, benchmark, in-context learning, preference learning, long-term memory, retrieval-augmented generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pPKPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper examines whether the LLMs can infer user preferences from past choices and use it to predict new choices. The prediction error can be divided into two parts: (a) the history did not contain enough information to make an accurate prediction (b) the LLM failed to infer the necessary information from the history to make the prediction.

To isolate the second type of error, the authors define a controlled task consisting of synthetic user profiles who pick one of two options based on well defined attributes for the two options. The LLMs get visibility into both the attributes and the options in a prose format instead of a structured schema. The selection of the options itself is non-deterministic and is based on a known Bayesian model. The authors define a metric called information-use which measures the information used by the LLM to make the prediction relative to a known Bayesian model. They conduct empirical experiments with different kinds of user profiles and model families to assess the information-use. The empirical studies also show that the use of LLM to infer latent attributes followed by a logistic regression model to predict the next choice outperforms direct LLM predictions. They further extend the experiments to examine sensitivity of preference specification regarding wording, attribute names etc. Finally they also examine whether the use of memory systems can improve predictions

Critical strengths
  1. The authors present a valid argument that an aggregated prediction error does not reveal the true cause of the prediction error. This paper presents an interesting tool to decompose the error into three parts (a) errors arising from incomplete information in the history (b) errors arising from an LLMs inability to infer relevant information from the history (c) inherent error due to randomness. By formulating a controlled experiment where the exact Bayesian preferences are known the tool allows us to quantitatively decompose the error into two parts.
  2. The extract-then-fit experiment provides an interesting result. Separating language extraction as one task followed by preference learning as a second task improves prediction over direct LLM predictions.
Critical weaknesses
  1. I found the paper very difficult to understand and there is room for improvement on the presentation side. The paper is not self-contained and there are several terms which are not well defined. In line 084, it is not immediately clear what misspecified logistic regression is. Line 87 mentions a parse-then-fit pipeline without describing the details until much later. In lines 122, it is not immediately clear what the variables a, b are. Similarly ϕF\phi_F is not defined clearly, the details of hidden-triple family is difficult to understand. I recommend that the authors make substantial improvements to the readibility of the paper
  2. One of the central claims of the paper is that direct LLM predictions perform worse than an extract-then-fit approach. However, it is not clear whether the errors are arising because the LLMs are unable to infer information from the history or because they are known to be poorly calibrated at generating numerical probabilities [1] [2]
  3. Finally, it is not clear if these findings transfer to real applications. Real users do not follow predefined bayesian preference rules and do not describe their preferences in the same templated language. It might be helpful for the authors to describe the practical relevance of these findings.

[1] Xiong et al. “Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs.” ICLR, 2024. [2] Gu et al. “Probabilistic Medical Predictions of Large Language Models.” npj Digital Medicine, 7, 367, 2024.

Questions for authors

N/A

Reviewer_AGVGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how a persistent agent may leverage user histories in order to make predictions about users preferences. The authors propose GCE, an evaluation metric that measures how much information from the histories a model uses to make a prediction. Experiments are conducted using GCE to evaluate three state-of-the-art LLMs on synthetic user preference data.

Critical strengths
  • The problem being addressed (models making inferences based on user history) is timely and likely very interesting to the community.
Critical weaknesses
  • The presentation of the paper, in particular the writing, lacks clarity. For example, consider the contributions listed on lines 74-89. The first contribution listed is "a measurement protocol for prediction after natural-language experience [...]" -- a prediction isn't a metric, so it's not clear what you're measuring exactly, and "natural-language experience" is also very vague. (Yes, some context is provided with careful reading of paragraph 2, but paragraph 2 does not present these ideas clearly either.) The sentence continues to enumerate components of the protocol, which are also relatively enigmatic ("an exact enumerated reference, a decomposition that separates inferability from use, and a ratio with a prespecified reporting rule"). The second contribution is "an environment that renders latent decisions as prose without revealing the schema [...]" -- what latent decisions? What kind of schema would the reader assume is revealed and why? Again, the sentence continues to list specifics that are jargon-heavy and difficult for the reader to parse without context. In the interest of space and time I will not continue, but can list more examples if needed.

  • Many key details are also unclear, and the paper is also missing a conclusion or discussion section that synthesizes the results. When describing the session in Section 3.3 (note: it seems this is also referred to as a "natural-language experience" at times; I recommend using one consistent term), it's unclear how and why these eight attributes are selected, what the four base sentences are, what the 24 flavor sentences are, and what the domains are. I also suggest references Fig 1 in this section, and editing Figure 1 to indicate where the context cues, attribute-related sentences, etc. can be found.

Questions for authors
  • What would you write in a conclusion and/or a discussion section?
Reviewer_KAkGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents Generative Cognitive Evaluation (GCE).

Contributions as per the authors (copy pasted from the paper as I did not understand any of it):

(i) a measurement protocol for prediction after natural-language experience, with an exact enumerated reference, a decomposition that separates inferability from use, and a ratio with a prespecified reporting rule; (ii) an environment that renders latent decisions as prose without revealing the schema, using randomized templates, distractors and a novel-vocabulary variant that replaces familiar attribute words with invented ones; (iii) six preference families on a common discrete grid, among them preference drift and a family in which stated and revealed preferences conflict; (iv) an evaluation of three frontier backbones under eight prespecified conditions (a ninth was withdrawn) that reimplement summary, structured-profile and retrieval memory with frozen prompts and measured byte budgets, plus four exploratory conditions including two off-the-shelf memory systems, next to exact and misspecified statistical references; and (v) findings across the six families.

Critical strengths

None

Critical weaknesses

The paper appears to have been written by and for a non-human audience.

It starts with a questionable statement from the first sentence: "A persistent assistant builds up a long record of one person’s decisions." - this not my experience with such tools.

The paper has no conclusion, does not report any baseline from the literature in the results. Data is purely synthetic and templated as per the Figure 1. Terms are not defined (e.g. "decision", "prose"...).

Questions for authors

In one sentence that a human can understand, what problem does this paper solve and how does it solve it better that the previous literature?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.