- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes EOT-KD for reducing popularity bias during recommendation distillation. It first corrects the teacher distribution using an interaction-frequency penalty, then transfers the corrected target through optimal transport with teacher-derived item geometry. Experiments on four datasets show improved long-tail retrieval while retaining most of the teacher’s overall ranking quality.
Critical strengths
-
The paper addresses a relevant problem: students can inherit popularity bias alongside useful teacher knowledge. Separating target correction from knowledge transfer makes the framework easy to follow. The correction parameter also provides an interpretable way to adjust the trade-off.
-
The evaluation distinguishes relevant long-tail retrieval from simply recommending more unpopular items. It reports full-catalogue ranking, several exposure metrics, and results over five random seeds. The component ablations help explain the roles of correction and transport.
-
The additional machinery is confined to training. Inference uses the same graph-free MF student, without requiring the teacher or Sinkhorn computation. The manuscript also provides implementation details and reports anonymous source code in the supplementary material.
Critical weaknesses
-
My main concern is limited methodological novelty. The corrected target can be written as , which is a frequency-based logit adjustment, while its KL-projection interpretation follows established posterior regularization. The paper needs to explain what technical contribution this provides beyond applying these existing ideas to a distillation target.
-
The OT component also has substantial precedents that are missing from the discussion. WKD (NeurIPS 2024) already uses teacher-derived relationships for geometry-aware probability distillation, SinKD (LREC-COLING 2024) uses Sinkhorn-based distribution matching, and TARec (WWW 2025) applies Wasserstein logit distillation to recommendation. These connections make EOT-KD appear closer to a combination of existing correction and transfer mechanisms than a new distillation approach.
-
The theoretical contribution is modest. The closed-form solution and negative-variance derivative are standard properties of exponential tilting. Moreover, monotonicity concerns the corrected teacher distribution; it does not establish monotonic improvements in the trained student’s exposure balance or relevant-tail retrieval.
-
The exposure interpretation is insufficiently validated. Interaction frequency combines exposure, item attractiveness, and user preferences, so penalizing it does not necessarily isolate exposure bias. Evaluation on held-out logged interactions shows a popularity–utility trade-off, but cannot determine whether the method recovers preferences obscured by unequal exposure.
-
The experiments do not establish a clear advantage at matched overall utility. EOT-KD sacrifices approximately 0.3–5.2% of the teacher’s NDCG, while competing methods are mainly compared at individual operating points. The evidence for the proposed geometry is also limited: on Cellphones, collaborative geometry raises Tail-NDCG only from 0.0167 to 0.0171 over identity cost, and this ablation is reported on one dataset.
Questions for authors
-
What distinguishes the exposure projection technically from frequency-based logit adjustment and standard KL posterior regularization?
-
How does the OT objective differ from existing geometry-aware distillation methods, particularly WKD and Wasserstein-based recommendation distillation?
-
How does monotonic correction of the teacher target translate into the trained student’s ranking and exposure behavior?
-
Can the exposure interpretation be validated using impression data, randomized feedback, or a controlled exposure simulation?
-
Can you compare accuracy–tail retrieval trade-off curves and repeat the projection and geometry ablations across all four datasets?
5 comments
W4/Q4. Interaction frequency reflects exposure, user preference, and item quality. As stated in the Introduction and Section 2.2, We use it as an empirical proxy for historical exposure, not a causal measurement. These benchmarks lack impression or randomized exposure logs, so our findings support frequency-based long-tail rebalancing and retrieval of held-out relevant tail items, rather than causal recovery of hidden preferences.
To examine whether relevance is preserved, we report two observations. First, EOT-KD retains 94.8–99.7% of the teacher's NDCG@20 across four datasets. Second, Precision@20 remains close to the teacher while AvgPop@20 decreases: on LastFM, 0.07524→0.07484 with AvgPop 39.53→33.45; on Beauty, 0.01238→0.01192 with AvgPop 65.11→48.14 (Appendix Fig. 5).
We will further clarify the limitations of this proxy in the revised manuscript.
W5/Q5: Utility, curves and ablations.
(a) Utility–tail trade-off. At the default , EOT-KD retains 94.8–99.7% of the teacher's NDCG@20, and provides control over this trade-off. At , the NDCG loss relative to the teacher is approximately 0–1.7%, while Tail-NDCG remains above the teacher's (Tables 1–2 and R3). EOT-KD also outperforms UnKD in both NDCG and Tail-NDCG on all four datasets (Tables 1–2).
To examine whether long-tail improvements persist at comparable overall utility, we evaluated Projection-KL and EOT-KD over and selected near-matched-NDCG operating points (Table R2). EOT-KD improves Tail-NDCG by 13.5%, 8.8%, 7.9%, and 9.8% on LastFM, Cellphones, Beauty, and Sports, respectively, while increasing Coverage. Overall NDCG is slightly higher on LastFM, equal on Cellphones, and less than 1% lower on Beauty and Sports. These results show additional long-tail benefits over Projection-KL at comparable ranking utility.
Table R2. Projection-KL versus EOT-KD at near-matched NDCG.
| Dataset | Method() | NDCG@20 | Tail-NDCG@20 | Coverage (%) |
|---|---|---|---|---|
| LastFM | Proj-KL(0.10) | 0.2012 | 0.0416 | 49.9 |
| EOT-KD(0.10) | 0.2015 | 0.0472 | 55.7 | |
| Cellphones | Proj-KL(0.05) | 0.0746 | 0.0136 | 58.7 |
| EOT-KD(0.05) | 0.0746 | 0.0148 | 65.5 | |
| Beauty | Proj-KL(0.05) | 0.0716 | 0.0202 | 79.7 |
| EOT-KD(0.05) | 0.0709 | 0.0218 | 84.1 | |
| Sports | Proj-KL(0.05) | 0.0444 | 0.0051 | 57.4 |
| EOT-KD(0.05) | 0.0443 | 0.0056 | 63.2 |
(b) Transport geometry. We extended the Cellphones geometry ablation (paper Table 5) to LastFM, Beauty, and Sports (Table R4). Relative to identity cost, teacher-derived collaborative costs improve Tail-NDCG on all four datasets: +2.4% on Cellphones, +11.6% on LastFM, +7.2% on Beauty, and +6.1% on Sports. Coverage also improves by 5.1, 2.9, 7.5, and 7.7 percentage points, respectively.
On Cellphones, the Tail-NDCG improvement (0.0167±0.0004 → 0.0171±0.0001) is modest relative to seed variation, while the Coverage improvement (77.19% → 82.25%) is clearer. Importantly, identity-cost OT already improves Tail-NDCG and Coverage over Projection-KL on Cellphones (0.0147→0.0167 and 63.31%→77.19%). This indicates that much of the benefit comes from OT-based transfer, with collaborative geometry providing an additional improvement. Text and Hybrid costs are also competitive, suggesting that the benefit is not restricted to one source of transport geometry.
Table R4. Additional transport-geometry ablations ().
| Dataset | Cost | NDCG@20 | Tail-NDCG@20 | Coverage (%) |
|---|---|---|---|---|
| LastFM | Identity | 0.1967 | 0.0466 | 57.32 |
| Collaborative | 0.2005 | 0.0520 | 60.26 | |
| Beauty | Identity | 0.0697 | 0.0235 | 86.81 |
| Collaborative | 0.0695 | 0.0252 | 94.27 | |
| Text | 0.0701 | 0.0255 | 94.50 | |
| Hybrid | 0.0700 | 0.0253 | 92.93 | |
| Sports | Identity | 0.0425 | 0.0066 | 72.47 |
| Collaborative | 0.0425 | 0.0070 | 80.13 | |
| Text | 0.0423 | 0.0070 | 77.63 | |
| Hybrid | 0.0423 | 0.0069 | 79.76 |
(c) Projection–OT component ablation. To further examine the complementary effects of exposure correction and OT-based transfer, we extended the 2×2 component ablation from Cellphones (paper Table 3) to LastFM, Beauty, and Sports (Table R5). Projection-KL applies teacher correction without OT, while OT- applies OT without correction. Their combination improves both relevant-tail retrieval and catalogue coverage over either component alone across all three datasets.
Table R5. Extended 2×2 component ablation across three datasets ( for Projection-KL and EOT-KD).
| Dataset | Method | NDCG@20 ↑ | Tail-NDCG@20 ↑ | Coverage@20 (%) ↑ |
|---|---|---|---|---|
| LastFM | KD-KL | 0.2032±0.0025 | 0.0380±0.0013 | 47.3±0.5 |
| Projection-KL | 0.2029±0.0028 | 0.0431±0.0023 | 51.7±0.8 | |
| OT-ρ=0 | 0.2015±0.0042 | 0.0391±0.0012 | 47.8±0.1 | |
| EOT-KD | 0.2039±0.0047 | 0.0537±0.0024 | 60.3±0.1 | |
| Beauty | KD-KL | 0.0716±0.0005 | 0.0195±0.0004 | 77.5±0.3 |
| Projection-KL | 0.0710±0.0002 | 0.0213±0.0001 | 83.5±0.2 | |
| OT-ρ=0 | 0.0705±0.0005 | 0.0199±0.0004 | 79.0±1.3 | |
| EOT-KD | 0.0697±0.0003 | 0.0247±0.0007 | 92.5±2.4 | |
| Sports | KD-KL | 0.0439±0.0009 | 0.0045±0.0003 | 53.5±1.8 |
| Projection-KL | 0.0438±0.0007 | 0.0053±0.0004 | 60.3±1.6 | |
| OT-ρ=0 | 0.0436±0.0001 | 0.0047±0.0001 | 56.8±1.7 | |
| EOT-KD | 0.0420±0.0007 | 0.0067±0.0004 | 79.3±1.1 |
These results reinforce the complementary roles of the two components: exposure correction changes what is transferred, while OT changes how the corrected distribution is transferred. Their combination provides stronger relevant-tail retrieval and catalogue coverage than either component alone.
We hope these additional analyses address the reviewer's concerns and would appreciate reconsideration of the score.
We thank the reviewer for the careful assessment.
W1/Q1. What distinguishes EOT-KD is not a new reweighting operator, but the distillation transfer problem it addresses. Eq. (4) expresses the exposure correction as a frequency-based logit adjustment, pu⋆=softmax(suT/τ−ρϕ). Its KL-regularized (posterior-regularization / exponential-tilting) formulation (Eq. 3) provides a minimum-distortion interpretation and relates ρ to an exposure budget (Eq. 5). Our contribution lies in how the corrected target is used, which involves two complementary decisions:
Projection-KL answers only the first question, and uncorrected OT only the second. The two interact: in the 2×2 ablation (Table 3, Cellphones), Tail-NDCG gains over KD-KL are +0.0016 (projection) and +0.0010 (OT, ρ=0), but +0.0040 combined; Coverage gains are +6.5, +3.8 and +25.5 points. We have revised the contribution statement accordingly. The extended ablation on LastFM, Beauty, and Sports (Table R5) shows the same complementary pattern in Tail-NDCG.
W2/Q2. While these methods study OT-based knowledge transfer, EOT-KD specifically addresses popularity-related concentration in recommendation distillation. It differs in the distributions being transported, the transport costs, and the explicit correction of the teacher target before transfer.
To isolate the contribution of frequency-based teacher correction, we compare EOT-KD with OT-ρ=0 under the same transport formulation (Table 3). Tail-NDCG increases from 0.0141 to 0.0171, and Coverage from 60.60% to 82.25%. This is a component ablation, not a direct comparison with WKD-L, SinKD, or TARec. We will clarify these distinctions in the Introduction and Related Work.
W3/Q3. Propositions 1–2 establish monotonic exposure control for the corrected teacher target. To examine whether this correction is reflected in the trained student, we measured ET=Epu⋆[ϕ] and ES=EpuS[ϕ] for ρ∈0.05,0.10,0.15,0.20,0.25 (K=50, τ=3, δ=0.01). Both decrease monotonically across all five tested strengths on all four datasets (Fig. 3). Between ρ=0.05 and 0.25, the magnitude of the student's change in expected log-exposure is 84–97% of the teacher's change, while Tail-NDCG increases and overall NDCG decreases.
Table R3 Student-side exposure and ranking behavior (ρ: 0.05 → 0.25)
Coverage also increases and Gini decreases (Appendix Table 10). This diagnostic uses candidate distributions, while the ranking metrics use full-catalogue evaluation. It is single-seed evidence of transfer, not a proof of student monotonicity.