- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MERA formulates sequential reranking as an item–slot assignment problem. The actor outputs a soft assignment via Sinkhorn scaling; at execution time, Gumbel perturbation plus Hungarian rounding produces a valid hard slate with no repeated items. An ordered-slate critic scores the actually executed action, and matrix entropy regularizes the soft plan. The setup is tested on KuaiRand Pure and MovieLens-1M through KuaiSim, RecSim, and RecoGym adaptations.
Critical strengths
- The assignment formulation neatly handles the "which item goes where" problem: hard rounding guarantees a valid, non-repeating slate, and the ordered-slate critic evaluates what was actually shown to the user. The match between action representation and evaluation is clean.
- The component study is well designed — it examines how stochastic execution, Ordered-Q, and the actor update interact, not just individual ablations. Testing across two datasets and three simulator variants gives useful evidence. The reported inference latency under 8 ms also suggests the approach is deployable.
Critical weaknesses
- There is a gap between what the theory analyzes and what the implementation does. The paper acknowledges that the clean (un-perturbed) plan doesn't necessarily match the hard sampler's marginals, and that matrix entropy isn't the entropy of the executed policy. The ablations show the update is useful, but they don't tell us whether the plan–sampler mismatch actually matters. Without measuring this discrepancy, it's hard to connect the theoretical motivation to the trained policy's behavior.
- The baselines and settings leave notable gaps. SPG and ListCVAE are included, but GFN4Rec is a closely related method and its absence makes it harder to pin down the assignment formulation's advantage. All results come from simulators, and the RecSim/RecoGym adaptations share representations from the KuaiSim pipeline. The α study and latency sweep are helpful but don't cover different shortlist/slate sizes or independent Gumbel scales, leaving the method's sensitivity to these practical choices largely untested.
Questions for authors
- Empirically, how large is the gap between the clean assignment plan and the hard sampler's marginals in the reported settings?
- Is a comparison with GFN4Rec feasible under matched conditions (same state info, candidates, simulator dynamics, training budget)? If not, what's the main obstacle?
- Beyond the α analysis, how does recommendation quality change with shortlist size, slate size, or Gumbel perturbation strength?
- Was Gumbel perturbation on during the main evaluation? Please report the eval_gumbel setting and any overrides used.
- Which representations and user-response components are shared across the three simulator adaptations, and which differ?
5 comments
Response to Reviewer TM78 (2/3)
W2/Q3. Shortlist size, slate size, and Gumbel perturbation strength
Our paper studies the entropy coefficient (Appendix B.3) and reports MERA's inference latency for (7.782, 6.887, and 7.203 ms). We now add the effect of these sizes on recommendation quality, and of the perturbation strength.
Stable returns across shortlist and display sizes. We train each of four configurations with the original benchmark's training budget and report mean ± SD in KuaiSim on both datasets.
| Shortlist size | Display length | KuaiRand Pure: Total reward | ML1M: Total reward |
|---|---|---|---|
| 100 | 20 | 10.0576 ± 0.1018 | 9.1577 ± 0.3718 |
| 200 | 20 | 10.0119 ± 0.1672 | 9.0628 ± 0.2773 |
| 400 | 20 | 10.0206 ± 0.0923 | 9.0150 ± 0.3102 |
| 100 | 10 | 9.9621 ± 0.1762 | 9.1168 ± 0.2740 |
Every configuration retains at least 98.4% of the default mean return under the same training budget. Halving the display length to retains 99.1% and 99.6% on KuaiRand Pure and ML1M, respectively, while halving item exposures per user–slate interaction. All four configurations also exceed the original main-benchmark reference means: DDPG on KuaiRand Pure (9.7408 ± 0.1695) and HAC on ML1M (8.7534 ± 0.4922).
Stability across Gumbel scales. We freeze each checkpoint and its , vary only the evaluation scale , and evaluate on 512 fixed initial-user contexts per checkpoint. Because the protocol differs from Table 1, we report variation across over five checkpoints per dataset:
| Dataset | Gumbel scale → mean session return |
|---|---|
| KuaiRand Pure | 0 → 9.6207; 0.5 → 9.6104; 1 → 9.6091; 2 → 9.6007 |
| ML1M | 0 → 7.8912; 0.25 → 7.7947; 0.5 → 7.8084; 1 → 7.8118 |
Across the tested scales, the mean-return range is only 0.21% on KuaiRand Pure and 1.22% on ML1M relative to each dataset's highest sweep mean, indicating limited sensitivity to evaluation perturbation. In training, the factorial study shows that stochastic execution lowers return with MLP-Q and DPG (T1 to T5) but raises it by 0.2153 and 0.6153 with Ordered-Q and score alignment (T3 to MERA).
Q4. Evaluation Gumbel setting
Yes. Gumbel perturbation was enabled during evaluation in all six main settings (three simulators × two datasets), with the same noise distribution and scale as in training. As specified in Section 4.4, independent noise is added directly to the item–slot scores, so the evaluation noise scale is 1; the Sinkhorn temperature then controls the transport map applied to the perturbed scores.
W2/Q5. Shared and simulator-specific components
The three simulators share the input representations, state interface, and session handling, and differ in how user responses are generated. Appendices A.1–A.2 describe this design:
| Component | Common setup | Simulator-specific part |
|---|---|---|
| User/item inputs | Dataset profiles, histories, candidate features, and pretrained KuaiSim-derived item encodings | RecSim and RecoGym construct their latent item factors from these encodings |
| State and candidate interface | Common observation/history handling and full candidate pool | — |
| Response generation | Common action and feedback interface | KuaiSim: learned multi-response probabilities and intra-slate similarity adjustment; RecSim: affinity, evolving satisfaction, and exposure memory; RecoGym: latent interests, item bias, and stochastic drift |
| Session handling | Whole-session history updates and the base temper-based termination mechanism | The response generated by each core drives the session updates |
Holding the representation and session handling fixed isolates the effect of the response mechanism. We agree that, for the same reason, the three simulators should not be read as independent validations of real user dynamics. To complement this comparison, we validated the response model on held-out feedback (response-link AUC 0.6790 on KuaiRand Pure and 0.8321 on ML1M, with calibration results) and showed that MERA's advantage over HAC persists under perturbed response temperature, position decay, and termination hazard. Details are given in our response to Reviewer pSP3 (W1/Q1).
Response to Reviewer TM78 (3/3)
Summary
- Q1. The clean plan and the executed display marginals overlap by 98.44% on KuaiRand Pure and 93.54% on ML1M. The reported RecSim evaluations across training checkpoints show positive average and on both datasets, with 95% intervals above zero.
- Q2. Under matched state information, candidates, simulator dynamics, and training budget, MERA is comparable to the adapted GFN4Rec on KuaiSim–KuaiRand Pure and higher in the other five settings.
- Q3. Every configuration retains at least 98.4% of the default return under the same training budget, and evaluation returns vary by at most 1.22% across Gumbel scales.
- Q4. Evaluation used Gumbel(0,1) perturbation, identical to training, in all six main settings.
- Q5. The simulators share representations, the state interface, and session handling, and differ in response generation; held-out response validation and mechanism perturbations complement this comparison.
We hope these answers address the reviewer's questions.
References
[1] Shuchang Liu et al. Generative Flow Network for Listwise Recommendation KDD, 2023.
Response to Reviewer TM78 (1/3)
We thank the reviewer for the careful review and for recognizing the match between the action representation and the executed-slate critic, as well as the design of the component study. We answer the five questions in turn, with new measurements on trained MERA policies.
Throughout, Z denotes the item–slot scores, P=Sinkhorn(Z/α) the clean assignment plan, and X the executed hard slate.
W1/Q1. How large is the gap between the clean plan and the sampler's marginals?
The clean plan and sampled display marginals show high overlap on both datasets: 98.44% on KuaiRand Pure and 93.54% on ML1M. With M the empirical marginals of repeated executions, we report
\left|P_{ij}-\widehat M_{ij}\right|, \qquad \widehat M=\frac{1}{L}\sum_{\ell=1}^{L}X^{(\ell)}, \qquad \text{overlap}=1-\frac{d}{2}.$$ We use five checkpoints and four held-out requests per dataset, with $L=65{,}536$ executions per request at the training perturbation scale. | Dataset | Mean distance $d$ | Display-marginal overlap | | --- | ---: | ---: | | KuaiRand Pure | 0.0313 | 98.44% | | ML1M | 0.1292 | 93.54% | The measurements use the training entropy coefficients ($\alpha=0.2$ on ML1M, $0.7$ on KuaiRand Pure). **Optimization evidence complements the high marginal overlap.** Appendix C.3 preserves the Bellman contraction property with either the true-marginal or clean-plan entropy bonus. **Direct evaluation of the implemented update.** Additional RecSim evaluations summarize objective changes at evaluated checkpoints across training. We report $EQ=\mathbb E_X Q_\theta(O,X)$ and $F=EQ+\alpha H_{\rm mat}(P_{\rm clean})$, separating changes in expected hard-slate critic value from the clean-plan entropy contribution. The table gives mean changes and 95% confidence intervals, in units of $10^{-5}$. | Simulator | Dataset | $\Delta F$ [95% CI] | $\Delta EQ$ [95% CI] | | --- | --- | ---: | ---: | | RecSim | KuaiRand Pure | +7.876 [5.747, 10.004] | +6.231 [4.349, 8.112] | | RecSim | ML1M | +11.703 [9.659, 13.747] | +9.845 [8.584, 11.106] | **Both $\Delta F$ and $\Delta EQ$ have positive means and confidence intervals entirely above zero on both datasets.** These results support a significant improvement in the regularized objective and in expected hard-slate critic value over the evaluated training checkpoints. ## W2/Q2. Comparison with GFN4Rec under matched conditions **We compare with GFN4Rec [1] under the shared session benchmark, extending the original SPG and ListCVAE comparisons to an autoregressive generator–evaluator framework.** The comparison controls the four requested conditions: - **State information.** Both use the same state-encoder architecture and observation/history inputs. - **Candidates.** Both methods draw from the same full item pool. GFN4Rec generates slates autoregressively from it, and MERA learns its per-request shortlist from the same pool. - **Simulator dynamics.** Both use the same three simulators and two datasets and are evaluated by the same session-return metric. - **Training budget.** GFN4Rec uses the same number of training updates and the same checkpoint-saving rule as the corresponding MERA run in each setting. For the session-return task, we pair GFN4Rec's trajectory-balance generator with an evaluator using the same Ordered-Q architecture and set its trajectory reward to $\max(Q_\theta,0)+0.1$. This is a session-objective adaptation of the original immediate list-reward formulation. Both methods output one executed slate per request, without test-time beam search or candidate-slate reranking. The new five-run GFN4Rec means and MERA's reported Total rewards are: | Simulator | Dataset | GFN4Rec | MERA | | --- | --- | ---: | ---: | | KuaiSim | KuaiRand Pure | 10.0365 | 10.0576 | | KuaiSim | ML1M | 8.4121 | 9.1577 | | RecSim | KuaiRand Pure | 6.8873 | 7.3934 | | RecSim | ML1M | 6.7490 | 7.2336 | | RecoGym | KuaiRand Pure | 10.4264 | 13.9175 | | RecoGym | ML1M | 10.6415 | 13.1689 | **MERA matches the adapted GFN4Rec closely in mean Total reward on KuaiSim–KuaiRand Pure and achieves higher means in the other five settings.**