- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
Across roughly 325K trials, the paper studies wealth-conditioned recommendations by 13 personal agents. Eight models recommend pricier options to wealthier personas across covered domains. Some gaps persist under explicit cost preferences and email-based inference. This paper is interesting, but a bit far from the real world complex setting.
Critical strengths
There are several key points for the positive side, which are listed below.
-
Fixed inventories and matched personas separate recommendation steering from price changes. This gives a new insight to the research problem.
-
Multiple domains, access channels, and explicit-intent conditions provide broad coverage, though still a bit far from the reality.
-
Attribute blocking and access restrictions test interventions, while retrieval analyses help distinguish access from steering.
Critical weaknesses
The major concern is the fidelity of the emulation. There are other weaknesses as follows.
-
Intent-pooled gaps may reflect legitimate personalization. Cheap-intent and price-ceiling violations provide stronger evidence of misalignment, but need model-specific uncertainty and violation rates. Smaller gaps are not evidence of absence.
-
Mean recommended price measures the five-item set, but not whether the first recommendation is cheapest. Top-1 price regret, cheapest-item inclusion, and ceiling compliance would clarify practical consequences. This is really critical in the real world.
-
GPT-5.5's insurance amplification is reported as both 171 and 151. Table 7 also contains increases above the advertised 40% maximum. Reconcile these and report uncertainty, including potential wealth-dependent inventory exclusions.
-
Binary wealth extremes and a ten-email inbox establish a controlled effect, not its prevalence in realistic personal contexts.
Questions for authors
Authors are encouraged to clarify some questions below, which are important to improve the readability of this paper.
-
What are the cheap-intent and ceiling-violation rates, with uncertainty, for each model? Emulating the real world is challenging, but closer to it is still valuable.
-
Do top-1 regret and cheapest-item coverage support the mean-price findings? This is the major concern from the reading of this submission. Please clarify.
-
Can you reconcile the amplification numbers and test a direct anti-profiling instruction?
2 comments
- Binary wealth profiles and external validity: We agree that our study establishes wealth-conditioned steering under controlled conditions rather than its real-world prevalence. As noted in §5 (lines 442–457), "We use personas and mock inventories rather than real users or fieldwork," and "wealth is binary rather than continuous, which may make the signal cleaner than in real populations." We also acknowledge that further validation with real email data and continuous variables remains important. We will emphasize these limitations more clearly in the revision, particularly that our results do not establish the prevalence of this behavior in real-world settings.
(comment 2 of 2)
We sincerely thank the reviewer for the detailed feedback and for recognizing the strengths of our experimental design. We agree that wealth-conditioned price differences do not necessarily imply misalignment, which is why we distinguish general recommendation steering from violations of explicit user objectives.
Reconciling the amplification figures and inventory exclusions: We thank the reviewer for catching the inconsistency in §5. For GPT-5.5 insurance, blocking employment information increases the wealth-conditioned gap from $122/month to $171/month (+40%, 95% bootstrap CI [129, 211]). Blocking demographic information instead produces a gap of $131/month (+7%). The $151 figure and reference to demographic blocking in §5 were reporting errors, which we will correct.
Regarding increases above 40% in Table 7, the 40% figure refers to the maximum insurance amplification among models with reliable inventory-valid evaluations and meaningful baseline gaps. Larger percentages arise when the baseline gap is near zero (e.g., $17 for Qwen3.5-2B) or when few trials pass inventory-validity checks (e.g., approximately 18–22% for Gemini 3/3.1 Flash). We will clarify this distinction and report with confidence intervals.
We also checked for wealth-dependent inventory exclusions by comparing gate-pass rates across wealth groups. These were largely symmetric, with a maximum difference of 2.2% points. For instance, GPT-5.5 insurance had 100% validity for both groups, while Gemini 2.5 Flash had 98.7% versus 99.7%. We thank the reviewer for raising this point and will include this analysis in the revision, as it helps assess potential selection bias introduced by filtering invalid model outputs and strengthens the interpretation of our findings.
Cheap-intent violations, top-1 regret, and cheapest-item coverage: We agree that the mean price of five recommendations may not capture the practical consequences of an agent's decision. We therefore computed the additional metrics suggested by the reviewer. First, we measured top-1 price regret, defined as the price of the first recommendation minus the cheapest available option. For insurance under explicit cheap intent, wealthy personas experience higher regret than low-income personas, with differences of $344/month for Claude Opus 4.8, $337 for Gemini 2.5 Flash, $330 for GPT-5, and $236 for GPT-5.5. The cheapest available plan is the same $85/month for both groups.
Second, we measured cheapest-item coverage among the five recommendations. For Gemini 2.5 Flash flights, the cheapest option appears in only 33% of wealthy-persona trials versus 83% of low-income trials, despite identical requests and inventories. Thus, the difference is not merely in ranking; the cheapest option is often excluded entirely.
Finally, we examined explicit price-ceiling violations. While most frontier models respect these constraints, some exhibit wealth-dependent violations. Claude Opus 4.8 breaches the $220 insurance ceiling in 29% of wealthy-persona trials, with plans exceeding the ceiling averaging $569 (2.6× the cap), compared to a 4% breach rate and an average of $322 (1.5× the cap) for low-income personas. For Gemini 2.5 Flash, the corresponding violation rates are 17% versus 4% for insurance and 21% versus 0% for flights.
These results provide stronger evidence of misalignment beyond potentially legitimate personalization. We will include model-specific results and uncertainty estimates in the revision.
Direct anti-profiling instructions: We thank the reviewer for this suggestion. Our objective is to investigate whether agents infer financial status from personal context and steer recommendations, even when users explicitly request otherwise. In our setting, users simply delegate tasks such as finding the cheapest flight, without asking agents to consider their wealth.
An explicit anti-profiling instruction would introduce an additional safeguard against the behavior we study. While testing such instructions would be a useful mitigation experiment, our focus is on whether this behavior emerges in ordinary delegation settings without these safeguards.
We do examine a related intervention through attribute blocking, where restricting financial information reduces the GPT-5.5 insurance gap [gap = mean(rich) − mean(poor)] from $613.79 − $491.53 = +$122/month to $545.67 − $559.79 = −$14/month (both gaps reported in Table 7, Appendix A.8).
(comment 1 of 2)