acceptodds
Withdrawn from ICLR 2027

SERA: Learning When and What to Augment for Conversational Recommendation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conversational recommender systems (CRSs) infer user preferences from multi-turn dialogues to rank items and generate responses. However, the amount and quality of preference evidence vary across conversations: some dialogues already provide sufficient information for reliable recommendation, while others require additional evidence. Existing approaches mainly improve CRS by incorporating auxiliary information, but often assume augmentation is universally beneficial without considering whether augmentation is necessary or what information can effectively compensate for the missing preference evidence. We propose SERA, a sufficiency-aware evidence retrieval and augmentation framework that learns when and what to augment for conversational recommendation. SERA formulates auxiliary information utilization as a conditional evidence augmentation process. It first assesses the sufficiency of current conversational evidence and activates augmentation only when additional information is needed. Then, it retrieves historically relevant evidence that is compatible with the user’s structured preferences and integrates the selected information into both recommendation ranking and response generation. Experiments on ReDial and INSPIRED demonstrate the effectiveness of SERA, achieving relative improvements of 3.26%/1.95% in Recall@10/Recall@50 on ReDial and 5.53%/4.78% on INSPIRED over the best existing methods, while consistently improving automatic response-generation metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Conversational Recommender Systems, Evidence Augmentation, Sufficiency-aware Learning, Historical Evidence Retrieval, Adaptive Recommendation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9PRAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that a conversational recommender shouldn't pull in auxiliary evidence for every dialogue, and proposes SERA to decide when to augment and what to retrieve. A gate built from four hand-designed signals (preference uncertainty, preference contrast, ranking confidence, preference coverage) decides whether to retrieve at all. When it fires, SERA pulls short "scenes" from training dialogues, scored on semantic similarity and on agreement with the user's inferred likes and dislikes, and feeds them into both ranking and response generation. The method sits on top of MSCRS. On ReDial and INSPIRED it reports small gains over MSCRS and the other baselines, plus ablations, sensitivity curves, and a latency comparison.

Critical strengths

1 The problem is a sensible one. Some dialogues clearly carry enough signal and some don't, and the always-on vs. always-off ablation at least points the way the authors claim: retrieving for every query isn't the best setting.

2 The pipeline is also careful about leakage. Scenes come only from the training split, the query's own source dialogue is masked during training, and the added cost is small (about 8 ms per query on ReDial and 4 ms on INSPIRED). Code is released.

Critical weaknesses
  1. I'm not convinced the gains are robust. The improvements are small, and there are no repeated runs or significance tests to show they hold up. Some numbers also don't line up between the main results and the ablations, and a few baseline results look inconsistent, so it's hard to tell whether all methods were compared under the same setup.

  2. The paper doesn't show that the gate works. There's no activation rate, no check that it fires where retrieval helps, and no comparison with a simple length or entity-count heuristic. The retrieval gains may also just be item co-occurrence, since scenes are anchored on items the user already liked. I'd want to see MSCRS plus a co-occurrence score before crediting "preference-compatible scenes."

Questions for authors
  1. Why does the always-off variant match MSCRS exactly on ReDial, when Eq. (20) says it should still include the preference-compatibility term? Were MSCRS and the other baselines rerun under one protocol, and if so, how do you explain the PCRS-TKA MRR values?
  2. Can you report means and standard deviations over several seeds?
  3. How often does the gate actually fire on the test sets, and how does SERA compare with MSCRS plus a simple item co-occurrence score?
  4. At test time, how do you get the user's liked and disliked items? On ReDial, do they come from the post-dialogue questionnaire, and what plays that role on INSPIRED?
Reviewer_jPSTOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SERA, an evidence-augmentation framework built on the MSCRS backbone for conversational recommendation. A gate estimated from four sufficiency signals decides whether to retrieve scenes from an offline library of historical dialogues, and the retrieved scenes are added to both the ranking score and the response generator. Experiments on ReDial and INSPIRED compare SERA with eight CRS baselines, with ablation, sensitivity, and efficiency analyses.

Critical strengths
  1. The evaluation separates the two design questions, with overall ablations, signal-level ablations, a semantic-only retrieval ablation, sensitivity analyses, and an efficiency comparison against the backbone.

  2. The historical memory is built under explicit validity conditions and leakage controls: scenes require explicit positive feedback with at least two items, the library uses only training dialogues, and same-source dialogues are masked.

Critical weaknesses
  1. Two tables contradict each other on INSPIRED. The always-off variant is stated to reduce SERA to the backbone MSCRS, and on ReDial the two rows match on all seven metrics. On INSPIRED, however, always-off reports NDCG@10/50 and MRR@10/50 of 0.173/0.210/0.148/0.156, against 0.168/0.202/0.140/0.148 for MSCRS. With retrieval disabled, MRR@50 already exceeds the backbone by 5.4%, and the claimed 8.78% gain shrinks to about 3% against this control.

  2. Effectiveness is not established. All results are single point estimates without variance or significance tests, although the MSCRS paper itself reports p < 0.05. The margins over the strongest per-metric baseline are at most 0.013, within the reproduction noise on ReDial, where re-running the same method changes Recall@10 by up to 0.025 (Kostric and Balog, SIGIR 2026). Five baseline rows in Table 1 also coincide digit-for-digit with the MSCRS paper, and stronger recent CRS such as ReFICR (Recall@10 0.305, 0.280 reproduced) are not discussed.

  3. The gate is not directly evaluated. There is no activation rate, no accuracy against its counterfactual training labels, and no analysis of the two error types, withholding needed augmentation or applying harmful augmentation. A hard threshold controls the ranking and generation paths at once and leaves no recovery path when it is wrong. The claim that sufficiency cannot be read off dialogue length or entity count is also not tested, and all four signals come from the backbone's own outputs, so the gate inherits its calibration.

  4. Insufficient novelty and motivation. Learning when to retrieve is established for LLMs (FLARE is not even cited), and KGSF and UniCRS already condition auxiliary evidence implicitly through learned attention. The paper also does not show that the conditional decision is needed: full SERA beats always-on by only 0.001 to 0.006, so the harm of unconditional augmentation is not measurable, and efficiency is not tested against always-on although SERA is slower than the retrieval-free backbone. The framework runs on a single backbone.

Questions for authors
  1. Which of the two INSPIRED rows is correct, and were the always-off rows from your own runs or from the published MSCRS numbers?

  2. What is the gate activation rate, how accurate is the gate against its counterfactual labels, and do simple heuristics such as dialogue length perform comparably?

  3. How do the results change when items already mentioned in the dialogue are excluded from the candidate set?

Reviewer_DEmAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Existing conversational recommender system approaches frequently adopt an unconditional augmentation paradigm, without considering whether augmentation is truly necessary or which evidence can effectively complement missing user preference signals. This paper presents a sufficiency aware evidence retrieval and augmentation framework, which learns when and what evidence should be augmented for conversational recommendation. Experiments on two public benchmarks verify the effectiveness of the proposed method.

Critical strengths
  1. The paper clearly clarifies its core motivation via the illustration in Figure 1.
  2. Empirical results validate the effectiveness of the proposed framework.
  3. The manuscript is well structured and well organized.
Critical weaknesses
  1. Could the authors report the runtime performance of the proposed method?
  2. What are the concrete advantages of their sufficiency aware framework compared with LLM based, KG-based, and other conversational recommender approaches?
  3. Could the authors explain why RL based CRS baselines and pure zero shot LLM CRS are not included in their experimental evaluation? Are there particular dataset or implementation constraints?
Questions for authors
  1. The font size of text elements in Figure 2 is too small to read.
  2. Statistical significance tests are absent. It remains unclear whether the observed performance improvements stem merely from random experimental variance.
  3. Insufficient analysis is provided to interpret the underlying causes behind the experimental results; the paper mainly reports experimental observations without digging into why such outcomes occur.
  4. Most compared competing baselines are relatively dated; recent state-of-the-art conversational recommender baselines are missing in the experimental comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.