acceptodds
Under review as a conference paper at ICLR 2027

UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents

Abstract

Agentic recommendation pursues shopper goals through sequences of context-dependent decisions. Developing and evaluating such policies requires faithful recommendation world models that capture both the interactive shopping environment and users’ behavioral responses over long interactions. Existing resources rarely provide long-horizon, fine-grained real-user trajectories or support trajectory-level evaluation and offline validation against production A/B results. We introduce UserArena, a real-world e-commerce interactive benchmark containing 64,729 session trajectories from 11,010 users, spanning 566,102 items and 13 action types. UserArena pairs these trajectories with an interactive environment and supports offline validation against outcomes from a controlled production A/B test, enabling five-level evaluation from output validity and local decisions to sequential behavior, personalization, and outcomes. To improve simulator fidelity over these long-horizon interactions, we further propose On-Policy Trajectory Distillation (OPTD), which uses trajectory-level intent to provide dense teacher targets on student-generated rollouts. Experiments show that OPTD achieves the strongest overall fidelity among the evaluated simulators under UserArena’s evaluation protocol and most closely reproduces the production policy effect when the same ranking-policy pair is evaluated offline in simulation.

Primary area:
datasets and benchmarks
Keywords:
User Simulation, Recommender System, Large Language Models, Agentic Reinforcement Learning, GUI Agent

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_8fUAOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces UserArena, a benchmark for long-horizon shopping user simulation that supports both single-session and cross-session evaluation across five levels of behavioral fidelity. It also proposes On-Policy Trajectory Distillation, which uses trajectory-level intent and token-level teacher supervision to improve user simulation. Experiments show that OPTD outperforms baselines on multiple fidelity metrics and more closely reproduces real-world recommendation-policy effects in a production A/B test.

Critical strengths
  1. Valuable dataset. UserArena provides a large-scale real-world dataset with rich user profiles, visual observations, and interaction histories across multiple sessions. Both its scale and detailed behavioral information could make it a valuable resource for the community.

  2. Comprehensive evaluation. The benchmark supports multiple levels of evaluation, from next-action prediction to sequential behavior and population-level outcomes. The validation against a real production A/B test is particularly valuable for assessing the practical usefulness of user simulators.

Critical weaknesses
  1. Limited individual-level evaluation. Although the paper proposes a multi-level evaluation framework, I have concerns about whether the metrics adequately capture individual-level user simulation quality. L1 evaluates next-action prediction, which makes sense, and L3 evaluates behavioral differences across user groups. However, if I understand correctly, L2 and L4 primarily compare aggregate statistics between real and simulated populations, rather than computing fidelity for each individual user and then averaging across users (please correct me if I am mistaken). This makes the evaluation less informative about individual user behavior. For example, a simulator could achieve a perfect terminal-state distribution score (T-JSD = 0) even if it predicts purchases for entirely different users or different products than those actually purchased. Similarly, trajectory error (TE) only compares average session lengths, meaning that individual trajectories could differ substantially while still achieving a perfect score. A model that learns population-level behavioral distributions could therefore achieve strong L2/L4 scores without accurately simulating individual users, which is particularly concerning given the paper's emphasis on personalized, long-horizon user simulation.

  2. Reliability of trajectory-level intent supervision. Although the ablation study shows that trajectory-level intent is useful, I have concerns about how reliable the extracted intent is and whether it could introduce bias into the teacher's supervision. In the cross-session setting, intent is summarized from multiple sessions, but user preferences and intentions may change over time. A user may not have certain intentions in earlier sessions, and even within a single shopping session, their intent may shift as they explore different products. Therefore, a single intent summary extracted from the complete trajectory may not accurately reflect the user's intent at every step. It would be helpful to further validate the quality of the extracted intents and investigate how OPTD handles evolving user intentions.

Questions for authors
  1. Session definition. How are sessions defined in the dataset, and how is the "End Session" action determined? Is it based on a time interval, an explicit user action, or some other criteria? I ask because session boundaries may affect the validity of the extracted intents. For example, if sessions are split based on inactivity thresholds, a single shopping intent might span multiple sessions, while a user could also develop different intents within the same session. It would be helpful to clarify how sessions are segmented and whether this affects intent extraction and evaluation.

  2. User profile construction and effectiveness. How are user profiles obtained? The appendix lists several user attributes, but it is unclear whether these are self-reported by users, inferred from their behavior, or generated by an LLM. Additionally, have the authors conducted any ablation studies on the contribution of user profiles? Given that the listed attributes are relatively simple, I wonder how informative they are for predicting user behavior and how much they actually contribute to simulation performance. A comparison with and without user profiles would be helpful.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer 8fUA (Part 1)

Dear Reviewer 8fUA,

Thank you for your suggestions on individual-level evaluation and the reliability of intent supervision. We address these points and your questions about sessions and user profiles below.

W1. Per-user evaluation.

Reply: Thank you for highlighting the importance of individual-level behavioral fidelity. Our multi-level evaluation already provides evidence relevant to this question. L1 compares the simulator’s next decision with the real user’s recorded decision at the same state, while L3 examines whether behavioral differences across user groups are preserved. L2 and L4 complement these assessments by evaluating sequential behavior and aggregate outcomes during free-running simulation. Together, these levels assess different aspects of behavioral fidelity, with local decisions, user-group differences, and rollout statistics providing complementary evidence.

To further examine whether aggregate results conceal discrepancies within individual users, we computed the L1, L2, and L4 metrics separately for each user on the cross-session test set and then averaged users equally. For TE, we first take the absolute difference between each user’s real and simulated mean session lengths, then average across users. L3 retains its cohort-based definition because PF measures how well behavioral differences across user groups are preserved.

ModelF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
Shop-R10.36550.21650.37071.750.87390.11270.44670.3084
Customer-R10.34710.21220.32212.080.86280.08840.33220.2147
AlignUSER0.34020.21340.29912.200.87320.09000.31430.1666
STA0.31420.22050.28131.720.89110.08370.31190.1822
OPTD0.29190.22400.26861.750.89290.07900.30720.1643

As we can see from the table, OPTD leads on six of the eight reported metrics. With equal weighting across users, it achieves the highest Acc. and the lowest DM-JSD, ΔCTR, ΔACR, and T-JSD. These results show its strengths in exact next-action prediction, decision-mix fidelity, and outcome fidelity. It also retains the highest cohort-level PF. Shop-R1 achieves the highest F1, while STA has the lowest TE. These complementary strengths illustrate why UserArena evaluates multiple levels: each captures a distinct aspect of simulation quality. OPTD performs best across most evaluated dimensions, while the remaining gaps provide concrete targets for further improvement. We have added this evaluation to Appendix C.2 of the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer 8fUA (Part 2)

W2. Contribution of complete-trajectory intent.

Reply: Thank you for raising this concern. First, we would like to clarify how OPTD uses the extracted intent. The intent is a structured summary of preferences, concerns, information needs, and conditional decision tendencies, providing behavioral context for the teacher. Although the original intent record is shared across sessions, the teacher also conditions on the current observation, interaction history, and student-generated prefix. This allows the teacher’s guidance to vary with the current interaction context. A per-token KL budget limits the influence of intent, while the grounding loss anchors supervision to recorded behavior. Together, these mechanisms allow context-dependent guidance while constraining the influence of the trajectory-level summary.

To reduce step-specific answer leakage, the extraction and validation rules exclude concrete actions, specific products and identifiers, exact prices, trajectory outcomes, future-page content, and supporting step indices from the intent summary. These rules are designed to keep the supervision focused on high-level behavioral context.

To further examine whether a shared trajectory-level summary overlooks changes in user intent, we compared three alternatives: session-level intent from each complete session, centered-window intent from the current decision and up to four decisions on either side within the same session, and historical-prefix intent from interactions strictly before the current action. Session-level and centered-window summaries provide more temporally localized context, while historical-prefix intent excludes subsequent interactions from intent extraction. These comparisons assess how the temporal scope of intent supervision affects behavioral fidelity. The table reports cross-session results.

MethodF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
OPTD0.34560.29690.02840.720.89290.03570.13990.0077
Session-level intent0.34340.29220.03530.430.90020.03570.13060.0127
Centered-window intent (±4)0.34310.29090.03270.480.90000.03380.12940.0133
Historical prefix0.34440.29650.03070.580.89860.03590.08120.0125

As shown in the table, OPTD achieves the highest F1 and Acc. and the lowest DM-JSD and T-JSD among the compared extraction scopes. One possible explanation is that complete trajectories help distinguish persistent preferences from temporary exploration and reveal recurring tendencies across sessions. With this broader context, intent inference may draw on more complete and consistent evidence, supporting coherent teacher supervision across interactions. More temporally localized or historical-only summaries also offer benefits: session-level intent achieves the lowest TE and highest PF, centered-window intent achieves the lowest ΔCTR, and historical-prefix intent achieves the lowest ΔACR. These results suggest that the temporal scope of intent supervision affects different aspects of behavioral fidelity, with complete-trajectory context benefiting local prediction, decision-mix fidelity, and terminal outcomes, and alternative scopes improving selected behavioral statistics.

Taken together, none of the alternative extraction scopes consistently outperforms OPTD across the evaluated dimensions, supporting complete-trajectory intent as an effective source of teacher guidance. OPTD’s contribution lies in translating this behavioral context into bounded, behaviorally grounded supervision on student-generated responses. We have added these comparisons to Appendix C.4 of the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer 8fUA (Part 3)

Q1. Session definition and intent extraction.

Reply: Thank you for raising this question. A session starts when a user enters the shopping scenario in the app and ends when the user leaves it, rather than being split by an inactivity threshold. We mark the end of each recorded session with an “End Session” action.

For intent extraction, we do not assume that sessions and shopping goals correspond one-to-one. In the cross-session setting, we extract a structured summary from the user’s complete training trajectory. It captures multiple preferences, concerns, information needs, and conditional decision tendencies, rather than imposing one fixed goal on every session. The teacher uses this context alongside the current state and interaction history to guide each decision. This preserves relevant evidence when a goal spans multiple sessions.

Session boundaries determine trajectory lengths and termination points. We use users’ actual entry into and exit from the shopping scenario to define these boundaries. Our evaluation measures how closely simulated interactions and stopping behavior match the recorded behavior, rather than whether a shopping goal has been completed. This behavioral evaluation therefore does not require a one-to-one correspondence between sessions and shopping goals. We have clarified the session definition in Appendix A.2 of the revised manuscript.

Q2. User profile construction and effectiveness.

Reply: Thank you for raising this question. We use the profile information (e.g., age, gender, and city) reported by users on the platform as static profile attributes. These attributes are neither inferred from their behavior nor generated by an LLM. Recent clicks, purchases, and interaction histories come from recorded user behavior. To assess the contribution of these static attributes, we conducted an ablation removing only these attributes while retaining recent clicks, purchases, and interaction history. The table reports single-session results.

MethodF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
OPTD0.41780.32040.01561.720.86920.00260.04300.0387
w/o static profile0.41010.31160.01751.920.87870.00180.03790.0419

These results suggest that age, gender, and city provide limited additional benefit when behavioral history is retained. These static attributes may not capture specific shopping habits, whereas recent clicks, purchases, and interactions may provide more direct evidence of user preferences and decision patterns. We have clarified that user-profile attributes are self-reported in Appendix A.2 of the revised manuscript.

Thank you again for your thoughtful questions. We hope the additional evaluations and clarifications address your concerns.

Reviewer_JYoLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents UserArena, an interactive benchmark for long-horizon e-commerce user simulation, and proposes On-Policy Trajectory Distillation (OPTD). By aligning student models with frozen teachers via token-level trajectory supervision and trust-region regularization, OPTD significantly improves behavioral fidelity and accurately mirrors real-world outcomes under varying recommendation policies.

Critical strengths

This work is well-written and easy to follow. It contributes UserArena, a long-horizon interactive benchmark preserving cross-session histories beyond session-level scopes (e.g., OPeRA, USB), and OPTD, a method integrating trajectory intent with on-policy distillation for fine-grained supervision. Unlike SFT or sparse-reward RL, OPTD provides denser signals, while its trust-region mechanism mitigates privileged teacher bias, ensuring robust simulator training.

Critical weaknesses

[1].Although the paper emphasizes long-horizon simulation, OPTD samples training states from real trajectories and performs token-level distillation on individual student-generated responses, rather than continuously training on states encountered through autonomous multi-step interactions. Therefore, the method primarily addresses exposure bias during response generation, while its effectiveness in mitigating state distribution shifts and error accumulation over long interaction sequences remains insufficiently validated.

[2].OPTD extracts user intent from complete training trajectories and uses it as privileged information for the teacher model. Although this information is not directly provided to the student during inference, it remains unclear to what extent the performance improvements result from learning generalizable long-term user preferences. An additional comparison using intent extracted only from historical prefixes would help clarify the necessity and actual contribution of complete-trajectory information.

[3].Production A/B validation is an important strength of the paper, but the evaluation considers only a single pair of ranking policies. This makes it difficult to determine whether the simulator can consistently predict the direction of policy improvements and correctly rank different recommendation policies. Furthermore, the paper lacks an uncertainty analysis of the estimated policy effects.

[4].The recorded user actions are paired with first-person rationales generated by a large language model rather than explanations provided by actual users. Consequently, the model may learn post-hoc rationalizations instead of the underlying decision-making processes. The paper does not examine the effects of removing synthetic rationales or varying their quality, making it difficult to assess how this supervision component influences the effectiveness of distillation.

[5].OPTD incorporates several components, including intent interpolation, KL constraints, JSD-based distillation, and ground-truth behavioral supervision, but the experiments primarily rely on fixed hyperparameter settings. A systematic sensitivity analysis of the trust-region constraint, grounding weight, and distillation objective is missing. As a result, it remains unclear how sensitive the reported improvements are to hyperparameter choices or under what conditions overly strong intent guidance may negatively affect behavioral calibration.

Questions for authors

Check the weakness.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer JYoL (Part 1)

Dear Reviewer JYoL,

We appreciate your positive assessment of UserArena and OPTD, and your detailed suggestions for examining long-horizon training, intent and rationale supervision, and sensitivity to design choices. We address each concern below.

W1. Long-horizon state shifts and error accumulation.

Reply: Thank you for the suggestion. OPTD uses intent from complete trajectories to guide each response, partially addressing the need for long-horizon behavioral guidance. The original OPTD performs one interaction step per training rollout. Our existing free-running evaluation assesses the resulting simulator over complete sessions, where its own actions determine subsequent states. The L2–L4 results therefore provide evidence of behavioral fidelity under autonomous interaction.

To further examine whether training on autonomously reached states improves this fidelity, we added training with two and four consecutive interactions. The student executes its actions and continues generation and distillation at the resulting states. Training time is measured relative to original OPTD.

MethodF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓Training time ↓
OPTD0.41780.32040.01561.720.86920.00260.04300.03871.00×
Two steps0.41720.31810.01131.910.86580.00790.01220.03421.52×
Four steps0.41290.31010.01671.870.86210.00450.04470.03102.43×

As shown in the table, the improvements are mixed. Training with two steps improves DM-JSD and ΔACR, while training with four steps achieves the lowest T-JSD. However, both variants underperform original OPTD on several key metrics, including Acc., TE, PF, and ΔCTR. One possible explanation is that including autonomous successor states shifts the emphasis of distillation away from real states, potentially sacrificing fidelity to recorded user behavior. Meanwhile, training time increases by 52% and 143%, respectively. These results suggest that original OPTD offers a more favorable balance between behavioral fidelity and training cost in the evaluated setting. We have added this experiment to Appendix C.5 of the revised manuscript.

W2. Contribution of complete-trajectory intent.

Reply: Thank you for suggesting this comparison. Our original design uses complete-trajectory intent as training-only context for the teacher, providing a session-wide perspective for token-level supervision on student-generated responses. To assess the additional contribution of complete-trajectory information, we added a historical-prefix variant that extracts teacher intent using only interaction history available before the current action.

MethodF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
OPTD0.41780.32040.01561.720.86920.00260.04300.0387
Historical prefix0.40830.31070.01892.020.85160.01220.04120.0467

As shown in the table, OPTD performs better on seven of the eight reported metrics, supporting the value of intent extracted from complete trajectories. A possible explanation is that complete trajectories help distinguish persistent preferences and decision tendencies from temporary exploratory behavior, providing more coherent teacher guidance across interactions. Historical prefixes achieve better ΔACR, potentially because their closer alignment with the current interaction context helps calibrate cart activity. Overall, these results suggest that intent extracted from complete trajectories provides broader benefits for behavioral fidelity. We have added this comparison to Appendix C.4 of the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer JYoL (Part 2)

W3: A/B test.

Thank you for your suggestions. We have completed two additional production A/B evaluations. Round 1 refers to the original evaluation; Rounds 2 and 3 each compare different versions of the production candidate-retrieval policy.

We report effect ratios and their 90% confidence intervals. The effect ratio is the simulated treatment-minus-control ΔPV-CTR divided by the corresponding production ΔPV-CTR, with 1 indicating an exact match. The intervals are computed using Fieller’s method, treating exposure PVs as independent observations.

SimulatorR1 effect ratioR1 90% CIR2 effect ratioR2 90% CIR3 effect ratioR3 90% CI
Shop-R10.1622×[0.1587, 0.1657]0.1431×[0.1398, 0.1464]0.1281×[0.1250, 0.1311]
Customer-R10.1906×[0.1865, 0.1946]0.1859×[0.1820, 0.1899]0.1795×[0.1757, 0.1833]
AlignUSER0.0203×[0.0190, 0.0215]0.0257×[0.0243, 0.0271]0.0230×[0.0217, 0.0243]
STA0.2967×[0.2918, 0.3016]0.2593×[0.2549, 0.2638]0.2470×[0.2427, 0.2513]
Qwen3.5-2B0.0037×[0.0032, 0.0043]0.0049×[0.0043, 0.0055]0.0019×[0.0015, 0.0022]
OPTD0.7130×[0.7047, 0.7213]0.6357×[0.6280, 0.6434]0.5900×[0.5826, 0.5974]

Among the six simulators compared here, OPTD achieves the effect ratio closest to 1 across all three evaluations. Its positive ratios indicate that the simulated and production effects have the same direction in all three evaluations. In each round, OPTD’s reported 90% confidence interval is closest to 1 and does not overlap with any baseline’s interval, providing additional support for its advantage in reproducing production effects. These results extend the original validation to two additional evaluations and provide further evidence of agreement with production effects.

W4. Contribution of rationale.

Reply: Thank you for raising this concern. We use recorded human actions as behavioral targets and synthetic rationales as auxiliary supervision linking the observed context to those actions. These rationales are retrospective annotations rather than users’ own explanations. Our objective is to improve behavioral fidelity. To assess the contribution of this supervision, we added three ablations: removing rationales during SFT, OPTD training, and evaluation, replacing them with rationales from other sessions, and using Qwen3.5-0.8B as a smaller rationale generator.

RationaleF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
Original0.41780.32040.01561.720.86920.00260.04300.0387
Removed0.37290.30660.19762.550.80020.01910.61970.2106
Mismatched0.35890.29570.02052.890.80810.05020.08690.0491
Smaller generator0.39220.31030.03360.500.84260.03530.05250.0968

The original rationales achieve the best results on seven of the eight metrics. Removing them degrades both local action prediction and rollout fidelity, supporting their value as auxiliary supervision. The declines with mismatched rationales and a smaller generator further suggest that contextual alignment and generator choice matter. One possible explanation is that aligned rationales provide more informative supervision linking observed context to recorded actions. The smaller generator achieves lower TE, potentially reflecting a change in stopping behavior that better matches average session length. However, this improvement does not extend to the other behavioral metrics. Overall, these results support the contribution of the original rationale supervision to behavioral fidelity. We have added these ablations to Appendix C.6 of the revised manuscript.

AuthorsOpenReview ↗

Reply to Reviewer JYoL (Part 3)

W5. Hyperparameter analysis.

Reply: Thank you for this suggestion. We added sensitivity analyses for the trust-region budget ε ∈ {0.01, 0.02, 0.05}, grounding weight γ ∈ {0.03, 0.1, 0.3}, and generalized-JSD mixing weight β ∈ {0.25, 0.5, 0.75}. We also compared forward KL, JSD, and reverse KL. Each comparison holds the other settings fixed and uses the main table’s L1–L4 metrics.

AnalysisConfigurationF1 ↑Acc. ↑DM-JSD ↓TE ↓PF ↑ΔCTR ↓ΔACR ↓T-JSD ↓
Trust-region budgetε=0.010.41020.30960.01712.060.84320.00510.03300.0439
ε=0.02 (default)0.41780.32040.01561.720.86920.00260.04300.0387
ε=0.050.41320.31230.01641.920.86450.00160.03640.0423
Grounding weightγ=0.030.41550.30520.01081.950.85270.00590.05080.0385
γ=0.1 (default)0.41780.32040.01561.720.86920.00260.04300.0387
γ=0.30.41200.31680.01341.780.86890.00830.07990.0284
Distillation objectiveForward KL0.42180.31180.02111.220.87990.00430.05190.0517
JSD (default)0.41780.32040.01561.720.86920.00260.04300.0387
Reverse KL0.42010.31080.01731.150.88550.01290.06040.0459
JSD mixing weightβ=0.250.41430.31540.01271.970.85810.01340.02090.0311
β=0.5 (default)0.41780.32040.01561.720.86920.00260.04300.0387
β=0.750.41100.31130.01852.290.83650.00750.01550.0419

These sensitivity experiments yield two main findings.

First, the original configuration provides strong overall fidelity. The default values achieve the best results on most metrics in their respective sweeps, supporting their selection as a balanced configuration. The default JSD objective also reduces T-JSD by approximately 25% and 16% relative to forward and reverse KL, respectively, highlighting its effectiveness in terminal outcome calibration.

Second, stronger intent guidance relative to behavioral grounding can impair selected outcomes. Increasing ε from 0.02 to 0.05 raises T-JSD by 9.3%, while reducing γ from 0.1 to 0.03 raises ΔCTR by 0.33 percentage points. Both changes also reduce action accuracy. A larger ε permits greater influence from privileged intent, while a smaller γ weakens supervision from recorded actions. These changes may shift the balance toward teacher guidance at the expense of matching observed behavior, explaining the decline in action accuracy and outcome calibration. We have added these sensitivity analyses to Appendix C.7 of the revised manuscript.

We appreciate your suggestions for strengthening the empirical evidence. We hope the additional experiments and analyses address your concerns.

Reviewer_RHtTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

USERARENA benchmarks interactive shopping user simulators using 64,729 sessions from 11,010 users, covering 566,102 items and thirteen action types. It combines reconstructed shopping interfaces with user profiles and recorded actions, including histories spanning multiple visits. Evaluation covers executable outputs, local decisions, sequential behavior, population heterogeneity, and outcomes, with a separate comparison against one production A/B test. The proposed OPTD method distills intent-conditioned teacher distributions on student-generated responses, with a trust region and supervised grounding. OPTD leads all eight behavioral metrics in the single-session setting and five in the cross-session setting, and reproduces 71.3% of the observed production policy effect.

Critical strengths
  1. The benchmark provides a valuable resource for studying behavior across repeated shopping visits. Its cross-session subset contains 59,378 sessions from 5,768 users, averaging 86.36 actions per user. Reconstructed interfaces support executable interactions, including clicks on exposed items that the recorded user did not select. Splitting by user strengthens evaluation on shoppers unseen during training.

  2. The evaluation goes beyond matching isolated logged actions. Replay and free-running simulation reveal different failure modes, while the shared backbone supports comparisons across eight existing simulators. OPTD combines privileged training intent with bounded teacher guidance and observable-state grounding. Component ablations, blinded trajectory judgments, and the production comparison provide complementary evidence for its usefulness.

Critical weaknesses
  1. The production validation is difficult to assess from effect ratios alone. Only one ranking-policy pair is evaluated, without the underlying arm-level PV-CTR values, online sample sizes, or uncertainty intervals. Consequently, the precision and practical meaning of the reported 0.713 ratio remain unclear. Report these quantities and describe the policy change and its separation from simulator selection.

  2. The long-horizon metrics largely summarize pooled behavior. Decision-mix divergence aggregates item-page decisions, trajectory error compares mean session lengths, and population fidelity averages cohort signatures. These summaries do not directly establish that individual behavior remains consistent across visits. A history-shuffling control and a measure of behavior across successive visits would better substantiate the claimed cross-session fidelity.

Questions for authors
  1. What are the control and treatment PV-CTR values, online sample sizes, observation window, and uncertainty intervals for the production and simulated effects? What distinguishes the two ranking policies, and was this policy pair kept separate from simulator tuning and checkpoint selection?

  2. How do the cross-session results change when prior-session history is removed or shuffled? Do you have a user-level measure that tests behavior across successive visits, beyond matching cohort summaries and mean session lengths?

  3. I found the paper informative and useful for understanding interactive user simulation. Currently, only a small evaluation subset is publicly available. If the paper is accepted, will you release the full dataset, including the training and cross-session data? Please clarify the release scope, timeline, and access conditions, as full availability would strengthen the benchmark contribution.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer RHtT (Part 1)

Dear Reviewer RHtT,

Thank you for recognizing the value of UserArena’s cross-session resource and for your suggestions on production validation, individual-level evaluation, and data availability. We address these points below.

W1 & Q1. Production A/B validation.

Reply: Thank you for requesting further details and uncertainty estimates for the production comparison. We analyzed a six-hour window covering approximately 50 million exposure page views across both arms. The two recommendation policies differed at the candidate retrieval stage: the control retained the purchase signal in that retrieval channel, whereas the treatment disabled it. The policy pair and its production outcomes were reserved for final evaluation after simulator tuning and checkpoint selection were complete.

Due to commercial confidentiality, we cannot disclose the exact arm-level PV-CTR values or point estimates of the absolute between-arm differences. We can, however, report the aggregate traffic scale and normalized comparison. Simulation used 5,000 users per arm. To quantify uncertainty in this normalized comparison, we report 90% confidence intervals for the effect ratios below. The effect ratio is the simulated treatment-minus-control ΔPV-CTR divided by the corresponding production ΔPV-CTR. The intervals are computed using Fieller’s method, treating exposure PVs as independent observations.

SimulatorEffect ratio90% CI
Shop-R10.1622×[0.1587, 0.1657]
Customer-R10.1906×[0.1865, 0.1946]
AlignUSER0.0203×[0.0190, 0.0215]
STA0.2967×[0.2918, 0.3016]
Qwen3.5-2B0.0037×[0.0032, 0.0043]
OPTD0.7130×[0.7047, 0.7213]

Among the six simulators compared here, OPTD achieves the effect ratio closest to 1, where 1 indicates an exact match between the simulated and production effects. OPTD’s reported 90% confidence interval is closest to 1 and does not overlap with any baseline’s interval, providing additional support for its advantage in reproducing the production effect.

AuthorsOpenReview ↗

Reply to Reviewer RHtT (Part 2)

W2 & Q2. Individual behavioral fidelity across sessions.

Reply: Thank you for highlighting the importance of individual behavioral consistency across sessions and the contribution of prior-session history. We address both questions below.

(1) Individual-level cross-session evaluation. First, our existing metrics provide indirect evidence of cross-session consistency by assessing how closely simulated behavior matches real-user behavior across multiple sessions. L2 evaluates trajectory fidelity, while L3 evaluates whether behavioral differences across user groups are preserved. Together, they support the preservation of real-user behavioral patterns across sessions.

To examine individual consistency more directly, we added an exploratory user-level analysis on the test set. For each user, we represent each session by its action-bigram distribution. We order each user’s sessions chronologically and split them into an earlier group containing the first ⌊n/2⌋ sessions and a later group containing the remaining sessions, where n is the user’s total number of sessions. We apply the same partition to real and simulated behavior. We quantify the difference between these groups using energy distance [1] and maximum mean discrepancy (MMD) [2]. We compute this change separately for real and simulated behavior, take their absolute discrepancy within each user, and then average equally across users. Lower error indicates that the simulator more closely reproduces the real user’s degree of behavioral variation across sessions.

MethodEnergy-based change error ↓MMD-based change error ↓
Shop-R10.154270.10722
Customer-R10.113010.07859
AlignUSER0.118470.08210
STA0.111580.07765
OPTD0.111000.07683

OPTD achieves the lowest point estimates among all evaluated methods under both distances. These results provide individual-level evidence that OPTD best matches the magnitude of behavioral variation across sessions among the evaluated methods.

(2) Removing or shuffling prior-session history. To assess the contribution of carried interaction history, we additionally remove prior-session history at session boundaries or replace it with another user’s history at the corresponding session index. We apply the same user-level evaluation and report energy-based / MMD-based change errors below.

MethodOriginal history Energy / MMD ↓History removed Energy / MMD ↓Cross-user history shuffled Energy / MMD ↓
Shop-R10.1543 / 0.10720.1702 / 0.12400.1466 / 0.1084
Customer-R10.1130 / 0.07860.1322 / 0.09930.1286 / 0.0975
AlignUSER0.1185 / 0.08210.1280 / 0.09550.1386 / 0.1026
STA0.1116 / 0.07770.1208 / 0.09090.1200 / 0.0897
OPTD0.1110 / 0.07680.1195 / 0.09000.1169 / 0.0863

Removing history increases both errors for all methods, supporting the contribution of prior-session history to reproducing individual behavioral variation. For OPTD, shuffling history also increases both errors, indicating that correctly matched prior-session history helps reproduce each user’s degree of behavioral variation across sessions. OPTD retains the lowest point estimates under both interventions. We have added these experiments to Appendix C.3 of the revised manuscript.

[1] Energy statistics: A class of statistics based on distances.
[2] SimEval-IR: A Unified Toolkit and Benchmark Suite for Evaluating User Simulators and Search Sessions.

Q3. Data release.

Reply: Thank you for emphasizing the importance of data availability for the benchmark’s community value. The current release contains the benchmark test set. Upon acceptance, we will release the full dataset, including the single-session and cross-session training data. De-identification is underway. Access will be provided for non-commercial research under a data use agreement.

Thank you again for your constructive suggestions. We hope the additional analyses and clarifications address your concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Revisions

Dear AC and Reviewers,

Thank you for the time and effort you have devoted to reviewing our paper. We sincerely appreciate your careful assessments and valuable suggestions, which have helped us strengthen the evaluation and presentation of our work.

We are grateful that the reviewers recognized:

  • The value of UserArena as a resource for long-horizon user simulation, preserving fine-grained real-user behavior across long-horizon sessions (Reviewers RHtT, JYoL, and 8fUA).
  • The comprehensive evaluation and its practical relevance, covering multiple levels of behavioral fidelity and validation against production A/B outcomes (Reviewers RHtT and 8fUA).
  • OPTD’s integration of long-horizon intent and trust-region constraints, providing dense teacher guidance for student learning while limiting privileged-intent bias (Reviewers RHtT and JYoL).

We also appreciate the follow-up assessments from Reviewers JYoL and 8fUA, who confirmed that their concerns had been addressed. Below, we summarize the additional experiments and clarifications motivated by the reviewers’ suggestions.

  1. Individual-level and cross-session fidelity (Reviewers RHtT and 8fUA): We added equally weighted per-user evaluation, measurements of individual behavioral variation across sessions, and controls that remove or shuffle prior-session history. These analyses strengthen the evidence for individual fidelity and the contribution of correctly matched history.

  2. Temporal scope of intent supervision (Reviewers JYoL and 8fUA): We compared complete-trajectory intent with historical-prefix, session-level, and centered-window alternatives, supporting the effectiveness of our choice of complete-trajectory intent.

  3. Multi-step training (Reviewer JYoL): We evaluated training with two and four consecutive interactions, including distillation at autonomously reached states. The results clarify the fidelity–cost trade-off and support the original one-step configuration as a balanced choice.

  4. Rationale supervision (Reviewer JYoL): We tested removing rationales, using rationales from other sessions, and using a smaller generator. These ablations support the contribution of rationale supervision and highlight the importance of contextual alignment and generator choice.

  5. Hyperparameter sensitivity (Reviewer JYoL): We varied the trust-region budget, grounding weight, and JSD mixing weight, and compared forward KL, JSD, and reverse KL. These analyses examine the balance between intent guidance and recorded behavior and support the default configuration.

  6. Production validation and uncertainty (Reviewers RHtT and JYoL): We completed two additional production A/B evaluations and reported 90% confidence intervals for the effect ratios. OPTD most closely reproduces the production effect among the compared simulators in all three evaluations. We also clarified the traffic scale, policy change, and separation from simulator selection.

  7. User profiles and session definitions (Reviewer 8fUA): We added an ablation removing static profile attributes while retaining behavioral history. We also clarified the self-reported profile sources and the session boundaries.

The analyses added to the manuscript are highlighted in blue. We will ensure that the revised draft also incorporates the additional results and clarifications provided during the discussion.

To summarize, UserArena establishes a real-world benchmark for long-horizon shopping scenarios, providing an empirical foundation for designing and evaluating recommendation world models. By connecting real-user trajectories, executable interactions, and production-grounded evaluation, it supports the study of how recommendation policies shape user behavior over time. OPTD complements this benchmark by using long-horizon intent to guide the learning of more faithful user simulators. Together, these contributions support the development of recommendation world models for studying long-term user behavior and evaluating policy changes before production testing.

With these additional experiments and clarifications, we hope our responses and revisions address all the reviewers’ concerns, and we welcome further discussion. Thank you again for helping us improve the paper and for the opportunity to share this work with the community.

Best wishes,
Authors of Submission 13022

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.