acceptodds
Under review as a conference paper at ICLR 2027

Behavioral Cloning for the Action Policy of a Customer Simulator: Structured Scores, Metric-Aware Decisions

Abstract

Simulating a customer journey requires predicting not only which product a shopper engages with, but whether they search, refine, navigate, convert, or stop interacting. Recent shopping simulators build this on language models, whose post-training targets helpful interaction rather than a cohort's action propensities, and task adaptation does not by itself guarantee recovering them—prior work reports homogenised, over-agreeable behavior when they stand in for people (Zhou et al., 2026; Sharma et al., 2024), a distortion that would land on the conversion and termination rates a simulator exists to produce. We factor a simulator step by the chain rule into behavior selection and semantic realisation, and ground the first in logged behavior by cloning. We isolate a journey-spanning action policy covering query formulation, item engagement, conversion, navigation and termination; 40.4% of test steps carry an action no standard item-ID next-item objective represents. We introduce StructBC, a compact causally masked Transformer score model encoding product-slate structure, short-range temporal dependence, page-local state and multi-scale behavioral context—a composition that lifts a recurrent and an attentive backbone alike. Our corpus is 452,296 real shopping episodes from 6,824 shoppers, split customer-disjointly into train, validation and test, over a state audited channel by channel for time direction (the past-only state: every shopper-specific channel is prefix-valid, bar two disclosed non-shopper transforms). On it the 3.4M-parameter policy reaches 0.5065 macro-F1 and 59.0% top-1 accuracy over ten actions, against 0.2784 for the strongest count-based baseline on the same split. Over the nine directly observed actions the score model alone reaches 0.5375 under the same argmax decoding, against 0.5278 for an adapted NARM encoder, and it also outperforms, under a fixed single-GPU compute envelope, a LoRA fine-tune of a 1.5B instruction model on our corpus and label. Under the same recipe extended to all ten actions, the LLM predicts termination on 0.15% of matched test steps against 4.61% observed—a prevalence error our own scorer reproduces under a comparably data-starved, one-epoch control. A nine-parameter metric-aware decision layer, fit post-hoc with no retraining of the score model, takes the complete policy to 0.5488 over the nine observed actions (+0.0113 of that from the layer), and contributes +0.0152 over all ten labels. It also cuts the total-variation distance between predicted and observed action distributions from 0.1055 to 0.0498. We evaluate the action-selection factor open-loop: the discrete policy of a customer simulator, composable with the semantic realisation we do not evaluate.

Primary area:
reinforcement learning
Keywords:
behavioral cloning, imitation learning, user simulation, sequential decision making, session-based recommendation, offline reinforcement learning, e-commerce

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2fyFOpenReview ↗
Rating
4(+2)
Confidence
4(+1)
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies high-level action selection for customer simulation as behavioral cloning over ten actions spanning search, product engagement, conversion, navigation, and session termination. The proposed StructBC score model combines a causal Transformer with slate self-attention, windowed temporal attention, et. It is trained on 452,296 sessions from 6,824 shoppers. The paper additionally fits nine class-specific logit offsets on validation data to maximize macro-F1. On the customer-disjoint test split, the complete system obtains 0.5065 macro-F1 and 59.0% accuracy over ten actions; experiments also cover nine directly observed actions, recurrent and attention-based baselines, an adapted LoRA-fine-tuned LLM, feature audits, and action-frequency diagnostics. The evaluation is explicitly limited to open-loop next-action prediction on logged histories.

Critical strengths

The paper is unusually transparent about its scope and experimental limitations. It uses a large real-world dataset with customer-disjoint splits, reports per-action behavior rather than only aggregate accuracy, repeats the main architecture comparison across recurrent and attentive backbones, and checks that the conclusions persist after excluding the constructed termination label. The feature audit is particularly valuable: the authors identify and remove two future-dependent customer features, document the remaining transductive features, and report relevant ablations instead of hiding these complications. Separating the score model from a simple validation-fitted decision rule is also practically useful, and the action-distribution diagnostics make clear what the rule changes.

Critical weaknesses
  1. The reported final policy is a deterministic argmax classifier evaluated only on logged states, whereas the motivating application requires a model of customer action propensities. Class-weighted training and macro-F1-fitted offsets are designed to improve hard classification decisions, but the paper does not establish that the resulting conditional probabilities are calibrated or that sampling from them produces realistic behavior. Matching the marginal action frequencies over logged states cannot show that conversion or termination rates remain faithful after the model generates its own histories. Consequently, the experiments validate open-loop next-action classification, but not the fidelity of a customer simulator or its proposed use for evaluating unseen interfaces.

  2. The headline architecture comparison is not conducted with a strictly training-only, temporally available state. Product popularity is aggregated over the complete corpus window, including validation/test customers and events later than the state being predicted, while the promotion features use scaling constants from a calendar interval extending beyond the extraction window. This matters for the architectural claim: removing the popularity channel costs 0.0037 test macro-F1, and in the only popularity-ablated attentive comparison, StructBC’s advantage over the bare Transformer falls from 0.0023 to 0.0005. So the paper does not establish that the already small structural gain survives without this feature. In addition, the authors state that the documented formula does not recover the exact popularity artifact used in the experiments, preventing independent reproduction of the main result from the provided specification.

Questions for authors
  1. Can the authors evaluate a stochastic version of the policy, including calibration of the reported past-only model and sampled action distributions, and provide at least a limited closed-loop evaluation of trajectory-level termination, cart-add, and purchase rates? The answer would determine whether the model supports the simulator interpretation rather than only next-action classification.

  2. Can the authors reconstruct product popularity using training customers and temporally prior events only, normalize promotion variables without future calendar information, and rerun the bare-Transformer versus StructBC comparison at both reported seeds? Please also clarify whether the exact popularity artifact or sufficient code to reproduce it can be released.

1 comment

Thank you for your explanation and I decide to change my choice from weak reject to weak accept

Reviewer_g4zrOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work separates each journey step of customer simulation into behavior selection and semantic realisation, and focuses on the behavior selection. Then, it introduces StructBC, and conducts training and evaluation on 452,296 real shopping episodes from 6,824 shoppers for next-action prediction.

Critical strengths

The dataset is constructed from real shopping episodes, with customer-disjoint splits preventing customer overlap between training and test sets. The authors also describe the methods and experimental details thoroughly.

Critical weaknesses
  1. The writing is poor. Dense text and scattered numerical results, with insufficient use of figures and tables, make the paper difficult to follow.
  2. The experimental design does not adequately address the motivation, and some comparisons raise concerns about fairness.
Questions for authors
  1. The introduction states that “prior work reports agreeableness and behavioral homogenisation where such models stand in for populations.” How do the experiments demonstrate that StructBC better captures the behavioral propensities of real customers?
  2. In Table 2, the fine-tuned LLM uses only approximately 3% of the training data for one epoch and is evaluated on a 2.7% test subset. These differences raise concerns about the fairness of the comparison.
  3. The experiments are limited to the authors’ own dataset. The generalisability of StructBC beyond this dataset remains unexplored.
  4. The manuscript reads more like an experimental report than a focused research paper. Its scientific argument, concision, and readability need improvement.
  5. The comparisons mainly focus on model architectures. More substantive comparisons with established customer simulation methods would help clarify the contribution.
Reviewer_egrkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies high‑level action‑policy learning for e‑commerce customer simulation. The authors factor customer simulator generation into discrete action‑selection and semantic realisation components, and focus only on behavioral cloning for the 10‑way high‑level action prediction task from real clickstream logs. They propose STRUCTBC, a Transformer‑based score model with four hand‑crafted structural inductive biases, plus a lightweight post‑hoc metric‑aware logit‑offset decision layer fitted on validation macro‑F1. Extensive open‑loop next‑action prediction experiments are conducted on a large proprietary private click‑stream dataset, with rich ablation, seed replication and bootstrap analysis in appendices. The paper emphasizes open‑loop evaluation only and explicitly states it is just a partial component, not a full end‑to‑end simulator.

Critical strengths

The paper carries extremely thorough experimental hygiene: multi‑seed replication, customer‑clustered bootstrap, careful feature leakage audit, controlled ablation suites and well‑documented hyperparameter protocols in appendices. The authors honestly report many limitations, confounders and unstable effects that many papers would omit.

The problem decomposition separating discrete action‑policy and semantic generation is conceptually clean, drawing a clear boundary for what the model does and does not claim.

The metric‑aware post‑hoc offset layer provides an empirical demonstration that tuning decision operating points is critical for heavily imbalanced multi‑class behavioral prediction, which is practically instructive for simulator practitioners.

Critical weaknesses

Core architectural contribution is weak and not robust The four‑module structural composition (STRUCTBC’s main claimed novelty) delivers only tiny effect sizes (~+0.004‑0.005 macro‑F1). More importantly, this performance gain disappears once the corpus‑level product‑popularity feature is removed, as the authors themselves show in Appendix E. There is no convincing evidence that the four structural inductive biases themselves produce measurable improvement independent of this transductive corpus‑aggregated feature. Individual module effects are inconsistent across random seeds, with many effects flipping sign between replicates. The paper can only claim “composition as a whole”, yet this aggregate benefit is confounded by a non‑past‑only global feature. This significantly undermines the central architectural contribution of this work.

Key practical claim relies exclusively on open‑loop evaluation; closed‑loop behavior is untested and high‑risk The whole target application is customer simulation for interface A/B test triage, which requires closed‑loop rollout. However, all evaluations are purely open‑loop next‑step prediction conditioned on ground‑truth logged history. As acknowledged by the authors, behavioral cloning suffers from compounding error distribution shift in rollouts. The metric‑aware offset layer fixes open‑loop marginal distribution but can over‑predict rare actions such as SESSION_END; these biases will accumulate in closed‑loop trajectories. Without any closed‑rollout metrics (e.g. episode‑level conversion rate, session‑length statistics), we cannot verify whether this policy can actually serve the motivating downstream simulator use‑case. The paper makes heavy real‑world application claims but provides zero supporting closed‑loop evidence.

Dataset is fully proprietary, no public benchmark or surrogate dataset released, hurting reproducibility All experiments are performed on unreleased private e‑commerce click‑stream data. Even though the authors document feature schema and hyperparameters, third‑party researchers cannot replicate any core numerical results, nor validate whether the findings transfer to other retail domains. There is no released synthetic dataset, pre‑processed feature specification or task benchmark for the community to build upon. This limits the broader scientific impact of this work.

Questions for authors

Since the structural block gain vanishes without the corpus‑level popularity feature: can you disentangle how much performance comes from your four structural inductive biases versus this global aggregate feature? Without this feature, what unique value does STRUCTBC architecture deliver compared to a simple well‑tuned bare transformer?

Your target application is customer simulator for pre‑launch A/B testing. Why are there zero closed‑loop rollout experiments? Could you provide at least preliminary episode‑level statistics (session length, conversion, termination rate) from closed‑rollout to demonstrate whether your policy does not compound severe bias?

Will you release a synthetic public benchmark / task definition so other groups can reproduce your task setting, given the real click‑stream dataset cannot be shared?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.