- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper examines whether the LLMs can infer user preferences from past choices and use it to predict new choices. The prediction error can be divided into two parts: (a) the history did not contain enough information to make an accurate prediction (b) the LLM failed to infer the necessary information from the history to make the prediction.
To isolate the second type of error, the authors define a controlled task consisting of synthetic user profiles who pick one of two options based on well defined attributes for the two options. The LLMs get visibility into both the attributes and the options in a prose format instead of a structured schema. The selection of the options itself is non-deterministic and is based on a known Bayesian model. The authors define a metric called information-use which measures the information used by the LLM to make the prediction relative to a known Bayesian model. They conduct empirical experiments with different kinds of user profiles and model families to assess the information-use. The empirical studies also show that the use of LLM to infer latent attributes followed by a logistic regression model to predict the next choice outperforms direct LLM predictions. They further extend the experiments to examine sensitivity of preference specification regarding wording, attribute names etc. Finally they also examine whether the use of memory systems can improve predictions
Critical strengths
- The authors present a valid argument that an aggregated prediction error does not reveal the true cause of the prediction error. This paper presents an interesting tool to decompose the error into three parts (a) errors arising from incomplete information in the history (b) errors arising from an LLMs inability to infer relevant information from the history (c) inherent error due to randomness. By formulating a controlled experiment where the exact Bayesian preferences are known the tool allows us to quantitatively decompose the error into two parts.
- The extract-then-fit experiment provides an interesting result. Separating language extraction as one task followed by preference learning as a second task improves prediction over direct LLM predictions.
Critical weaknesses
- I found the paper very difficult to understand and there is room for improvement on the presentation side. The paper is not self-contained and there are several terms which are not well defined. In line 084, it is not immediately clear what misspecified logistic regression is. Line 87 mentions a parse-then-fit pipeline without describing the details until much later. In lines 122, it is not immediately clear what the variables a, b are. Similarly is not defined clearly, the details of hidden-triple family is difficult to understand. I recommend that the authors make substantial improvements to the readibility of the paper
- One of the central claims of the paper is that direct LLM predictions perform worse than an extract-then-fit approach. However, it is not clear whether the errors are arising because the LLMs are unable to infer information from the history or because they are known to be poorly calibrated at generating numerical probabilities [1] [2]
- Finally, it is not clear if these findings transfer to real applications. Real users do not follow predefined bayesian preference rules and do not describe their preferences in the same templated language. It might be helpful for the authors to describe the practical relevance of these findings.
[1] Xiong et al. “Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs.” ICLR, 2024. [2] Gu et al. “Probabilistic Medical Predictions of Large Language Models.” npj Digital Medicine, 7, 367, 2024.
Questions for authors
N/A