- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies conversational recommendation agents. First, it introduces an AEROSELL benchmark that scores the profit of each sales conversation with simulated travelers. Second, it develops a two-stage training framework SIRA that intimates a rule-based sales planner with reinforcement learning on profit-correction. SIRA earns much more profits than comparison methods.
Critical strengths
- The paper is studying a meaningful evaluation task for conversational recommendation agents. The ultimate goal is profit, thus the agent needs to decide between asking a question, offering an add-on or completing the purchase. Conversations without purchase do not contribute to the goal. This is highly relevant in the commercial applications, yet existing benchmarks or models do not cover this.
- The task is naturally suitable for reinforcement learning. The proposed SIRA implements this with imitation learning and profit reward guided self-improvement, obtaining 14.5% more profits. The method is simple and effective.
Critical weaknesses
- Simulation of customer behaviors is relatively simple. It assumes a customer accepts an offer exactly when their willingness to pay exceeds the prices and wording never changes a sale. This is good for reproduction. However, in reality, human behaviors are much more complex. They tend to have an ambiguous willingness and can be influenced by the conversation contexts. A better way may be simulating customers with agents under some general action guidelines.
Questions for authors
- Could authors discuss whether it is possible to introduce some customers' willingness uncertainty and model that with conversations' pleasantness?
- Given the simple customer modeling, is SFT-phase necessary? Can we use RL to directly learn from the simulation?
Author's response to Reviewer TpGT
We thank the reviewer for recognizing the relevance of the task and that SIRA is simple and effective. Both questions are now answered with new experiments.
W1/Q1: Customer simulation; uncertain willingness and pleasantness. Deterministic customers are a deliberate design choice and a core strength of AeroSell: every sale is reproducible, every agent meets the same customers in the same order, and the full-information optimum is exactly computable. Language-model customers can be talked into purchases and vary from run to run, so an agent could gain by persuading the simulator rather than by making better sales decisions (App. F). AeroSell nonetheless admits richer behaviour, and we tested it:
Q2: Is the SFT phase necessary? Can RL learn directly from the simulation? Yes: with SIRA's budget, RL does not get started without it, and we now test this directly. We ran SIRA's RL stage from the base model with no Stage 1. Everything else was unchanged: the budget (32 groups of eight conversations), the reward r=100M−50.40+0.02,v for profit M and valid-response fraction v (Eq. 4), the KL coefficient, the optimizer, the guards, and SIRA's training customers.
This matches our preliminary studies (App. E), where outcome-reward RL from the base model did not clearly beat the prompted model, and the mechanism in Sec. 5.2: imitation teaches purchase timing (the parent and SIRA complete purchases on the last turn in 51 of 52 and 824 of 824 cases), and RL then corrects what is offered. The revised paper reports this run in App. G (Table 7) and updates the limitation in Sec. 6.
With both questions answered by new experiments, we hope the reviewer will consider raising the score.