acceptodds
Under review as a conference paper at ICLR 2027

Teacher Imitation Is Not Enough: Exposure-Aware Optimal-Transport Distillation for Recommendation

Abstract

Knowledge distillation (KD) for recommendation tasks typically encourages a compact student to imitate a high-capacity teacher. Yet teachers are trained on logged interactions shaped by uneven historical exposure and popularity feedback, so direct distillation can propagate popularity bias together with useful ranking knowledge. This raises a fundamental question: should a student imitate everything its teacher has learned? We propose EOT-KD, an exposure-aware optimal-transport distillation framework that separates teacher correction from teacher transfer. Using interaction frequency as an exposure proxy, EOT-KD first constructs a minimum-distortion exposure correction of the teacher distribution; the closest distribution in Kullback–Leibler (KL) divergence while penalizing expected historical exposure. This correction admits a closed-form solution and provides monotonic control over the exposure score through a single parameter. The corrected target is then transferred through entropic optimal transport, with pairwise costs defined by cosine distance between teacher-derived item representations, assigning lower transport costs to more similar items. Across four recommendation benchmarks, EOT-KD improves Tail-NDCG@20 over the teacher by 25.9-48.9% while retaining 94.8-99.7% of the teacher's NDCG@20. Teacher and OT computations are used only during training. At inference, only the graph-free student is retained, requiring no teacher, graph propagation, or optimal transport.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Knowledge Distillation, Recommender Systems, Optimal Transport, Exposure Bias, Long-Tail Recommendation, Popularity Bias

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NhBjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes EOT-KD for reducing popularity bias during recommendation distillation. It first corrects the teacher distribution using an interaction-frequency penalty, then transfers the corrected target through optimal transport with teacher-derived item geometry. Experiments on four datasets show improved long-tail retrieval while retaining most of the teacher’s overall ranking quality.

Critical strengths
  1. The paper addresses a relevant problem: students can inherit popularity bias alongside useful teacher knowledge. Separating target correction from knowledge transfer makes the framework easy to follow. The correction parameter also provides an interpretable way to adjust the trade-off.

  2. The evaluation distinguishes relevant long-tail retrieval from simply recommending more unpopular items. It reports full-catalogue ranking, several exposure metrics, and results over five random seeds. The component ablations help explain the roles of correction and transport.

  3. The additional machinery is confined to training. Inference uses the same graph-free MF student, without requiring the teacher or Sinkhorn computation. The manuscript also provides implementation details and reports anonymous source code in the supplementary material.

Critical weaknesses
  1. My main concern is limited methodological novelty. The corrected target can be written as (softmax⁡(sT/τ−ρlog⁡(e+δ)))(\operatorname{softmax}(s^T/\tau-\rho\log(e+\delta))), which is a frequency-based logit adjustment, while its KL-projection interpretation follows established posterior regularization. The paper needs to explain what technical contribution this provides beyond applying these existing ideas to a distillation target.

  2. The OT component also has substantial precedents that are missing from the discussion. WKD (NeurIPS 2024) already uses teacher-derived relationships for geometry-aware probability distillation, SinKD (LREC-COLING 2024) uses Sinkhorn-based distribution matching, and TARec (WWW 2025) applies Wasserstein logit distillation to recommendation. These connections make EOT-KD appear closer to a combination of existing correction and transfer mechanisms than a new distillation approach.

  3. The theoretical contribution is modest. The closed-form solution and negative-variance derivative are standard properties of exponential tilting. Moreover, monotonicity concerns the corrected teacher distribution; it does not establish monotonic improvements in the trained student’s exposure balance or relevant-tail retrieval.

  4. The exposure interpretation is insufficiently validated. Interaction frequency combines exposure, item attractiveness, and user preferences, so penalizing it does not necessarily isolate exposure bias. Evaluation on held-out logged interactions shows a popularity–utility trade-off, but cannot determine whether the method recovers preferences obscured by unequal exposure.

  5. The experiments do not establish a clear advantage at matched overall utility. EOT-KD sacrifices approximately 0.3–5.2% of the teacher’s NDCG, while competing methods are mainly compared at individual operating points. The evidence for the proposed geometry is also limited: on Cellphones, collaborative geometry raises Tail-NDCG only from 0.0167 to 0.0171 over identity cost, and this ablation is reported on one dataset.

Questions for authors
  1. What distinguishes the exposure projection technically from frequency-based logit adjustment and standard KL posterior regularization?

  2. How does the OT objective differ from existing geometry-aware distillation methods, particularly WKD and Wasserstein-based recommendation distillation?

  3. How does monotonic correction of the teacher target translate into the trained student’s ranking and exposure behavior?

  4. Can the exposure interpretation be validated using impression data, randomized feedback, or a controlled exposure simulation?

  5. Can you compare accuracy–tail retrieval trade-off curves and repeat the projection and geometry ablations across all four datasets?

5 comments

We thank the reviewer for the careful assessment.

W1/Q1. What distinguishes EOT-KD is not a new reweighting operator, but the distillation transfer problem it addresses. Eq. (4) expresses the exposure correction as a frequency-based logit adjustment, pu⋆=softmax(suT/τ−ρϕ)p^\star_u=\mathrm{softmax}(s^T_u/\tau-\rho\phi). Its KL-regularized (posterior-regularization / exponential-tilting) formulation (Eq. 3) provides a minimum-distortion interpretation and relates ρ\rho to an exposure budget (Eq. 5). Our contribution lies in how the corrected target is used, which involves two complementary decisions:

  • What to transfer: the corrected, user-conditioned target pu⋆(ρ)p^\star_u(\rho) over each user's candidate set.
  • How to transfer it: item-wise KL compares items one by one, so a mismatch between related items costs the same as one between unrelated items. EOT-KD instead uses pu⋆p^\star_u as the source marginal of an OT plan under a teacher-derived item cost (Eqs. 7–10).

Projection-KL answers only the first question, and uncorrected OT only the second. The two interact: in the 2×2 ablation (Table 3, Cellphones), Tail-NDCG gains over KD-KL are +0.0016 (projection) and +0.0010 (OT, ρ=0\rho{=}0), but +0.0040 combined; Coverage gains are +6.5, +3.8 and +25.5 points. We have revised the contribution statement accordingly. The extended ablation on LastFM, Beauty, and Sports (Table R5) shows the same complementary pattern in Tail-NDCG.

W2/Q2. While these methods study OT-based knowledge transfer, EOT-KD specifically addresses popularity-related concentration in recommendation distillation. It differs in the distributions being transported, the transport costs, and the explicit correction of the teacher target before transfer.

  • WKD-L uses teacher-derived CKA similarities to construct class-level transport costs for Wasserstein distillation. EOT-KD instead operates on frequency-corrected, user-conditioned item probability distributions under teacher-derived collaborative item costs.
  • SinKD employs batch-wise Sinkhorn matching across sample predictions. EOT-KD transports probability across candidate items within each user, after correcting the teacher target for frequency-related exposure imbalance.
  • TARec uses teacher-assisted Wasserstein logit distillation and embedding-level contrastive learning for multimodal recommendation. Its transport operates on pairwise ranking-score distributions under a p-norm ground cost with a self-cost correction. In contrast, EOT-KD addresses popularity-related concentration by transferring a frequency-corrected, user-conditioned teacher distribution under teacher-derived collaborative item costs.

To isolate the contribution of frequency-based teacher correction, we compare EOT-KD with OT-ρ=0\rho=0 under the same transport formulation (Table 3). Tail-NDCG increases from 0.0141 to 0.0171, and Coverage from 60.60% to 82.25%. This is a component ablation, not a direct comparison with WKD-L, SinKD, or TARec. We will clarify these distinctions in the Introduction and Related Work.

W3/Q3. Propositions 1–2 establish monotonic exposure control for the corrected teacher target. To examine whether this correction is reflected in the trained student, we measured ET=Epu⋆[ϕ]E_T=\mathbb E_{p^\star_u}[\phi] and ES=EpuS[ϕ]E_S=\mathbb E_{p^S_u}[\phi] for ρ∈0.05,0.10,0.15,0.20,0.25\rho\in{0.05,0.10,0.15,0.20,0.25} (K=50K=50, τ=3\tau=3, δ=0.01\delta=0.01). Both decrease monotonically across all five tested strengths on all four datasets (Fig. 3). Between ρ=0.05\rho=0.05 and 0.250.25, the magnitude of the student's change in expected log-exposure is 84–97% of the teacher's change, while Tail-NDCG increases and overall NDCG decreases.

Table R3 Student-side exposure and ranking behavior (ρ\rho: 0.05 → 0.25)

DatasetETE_TESE_SΔES/ΔET\Delta E_S/\Delta E_TTail-NDCG@20NDCG@20
LastFM−0.126→−0.351−0.101→−0.29084%0.0427→0.06250.2012→0.1931
Cellphones−0.219→−0.409−0.184→−0.36696%0.0148→0.01960.0746→0.0686
Beauty−0.193→−0.347−0.157→−0.30697%0.0218→0.02780.0709→0.0664
Sports−0.193→−0.344−0.162→−0.30695%0.0056→0.00870.0443→0.0404

Coverage also increases and Gini decreases (Appendix Table 10). This diagnostic uses candidate distributions, while the ranking metrics use full-catalogue evaluation. It is single-seed evidence of transfer, not a proof of student monotonicity.

W4/Q4. Interaction frequency reflects exposure, user preference, and item quality. As stated in the Introduction and Section 2.2, We use it as an empirical proxy for historical exposure, not a causal measurement. These benchmarks lack impression or randomized exposure logs, so our findings support frequency-based long-tail rebalancing and retrieval of held-out relevant tail items, rather than causal recovery of hidden preferences.

To examine whether relevance is preserved, we report two observations. First, EOT-KD retains 94.8–99.7% of the teacher's NDCG@20 across four datasets. Second, Precision@20 remains close to the teacher while AvgPop@20 decreases: on LastFM, 0.07524→0.07484 with AvgPop 39.53→33.45; on Beauty, 0.01238→0.01192 with AvgPop 65.11→48.14 (Appendix Fig. 5).

We will further clarify the limitations of this proxy in the revised manuscript.

W5/Q5: Utility, curves and ablations.

(a) Utility–tail trade-off. At the default ρ=0.15\rho=0.15, EOT-KD retains 94.8–99.7% of the teacher's NDCG@20, and ρ\rho provides control over this trade-off. At ρ=0.05\rho=0.05, the NDCG loss relative to the teacher is approximately 0–1.7%, while Tail-NDCG remains above the teacher's (Tables 1–2 and R3). EOT-KD also outperforms UnKD in both NDCG and Tail-NDCG on all four datasets (Tables 1–2).

To examine whether long-tail improvements persist at comparable overall utility, we evaluated Projection-KL and EOT-KD over ρ∈{0.05,0.10,0.15,0.20,0.25}\rho\in\{0.05,0.10,0.15,0.20,0.25\} and selected near-matched-NDCG operating points (Table R2). EOT-KD improves Tail-NDCG by 13.5%, 8.8%, 7.9%, and 9.8% on LastFM, Cellphones, Beauty, and Sports, respectively, while increasing Coverage. Overall NDCG is slightly higher on LastFM, equal on Cellphones, and less than 1% lower on Beauty and Sports. These results show additional long-tail benefits over Projection-KL at comparable ranking utility.

Table R2. Projection-KL versus EOT-KD at near-matched NDCG.

DatasetMethod(ρ\rho)NDCG@20Tail-NDCG@20Coverage (%)
LastFMProj-KL(0.10)0.20120.041649.9
EOT-KD(0.10)0.20150.047255.7
CellphonesProj-KL(0.05)0.07460.013658.7
EOT-KD(0.05)0.07460.014865.5
BeautyProj-KL(0.05)0.07160.020279.7
EOT-KD(0.05)0.07090.021884.1
SportsProj-KL(0.05)0.04440.005157.4
EOT-KD(0.05)0.04430.005663.2

(b) Transport geometry. We extended the Cellphones geometry ablation (paper Table 5) to LastFM, Beauty, and Sports (Table R4). Relative to identity cost, teacher-derived collaborative costs improve Tail-NDCG on all four datasets: +2.4% on Cellphones, +11.6% on LastFM, +7.2% on Beauty, and +6.1% on Sports. Coverage also improves by 5.1, 2.9, 7.5, and 7.7 percentage points, respectively.

On Cellphones, the Tail-NDCG improvement (0.0167±0.0004 → 0.0171±0.0001) is modest relative to seed variation, while the Coverage improvement (77.19% → 82.25%) is clearer. Importantly, identity-cost OT already improves Tail-NDCG and Coverage over Projection-KL on Cellphones (0.0147→0.0167 and 63.31%→77.19%). This indicates that much of the benefit comes from OT-based transfer, with collaborative geometry providing an additional improvement. Text and Hybrid costs are also competitive, suggesting that the benefit is not restricted to one source of transport geometry.

Table R4. Additional transport-geometry ablations (ρ=0.15\rho=0.15).

DatasetCostNDCG@20Tail-NDCG@20Coverage (%)
LastFMIdentity0.19670.046657.32
Collaborative0.20050.052060.26
BeautyIdentity0.06970.023586.81
Collaborative0.06950.025294.27
Text0.07010.025594.50
Hybrid0.07000.025392.93
SportsIdentity0.04250.006672.47
Collaborative0.04250.007080.13
Text0.04230.007077.63
Hybrid0.04230.006979.76

(c) Projection–OT component ablation. To further examine the complementary effects of exposure correction and OT-based transfer, we extended the 2×2 component ablation from Cellphones (paper Table 3) to LastFM, Beauty, and Sports (Table R5). Projection-KL applies teacher correction without OT, while OT-ρ=0\rho=0 applies OT without correction. Their combination improves both relevant-tail retrieval and catalogue coverage over either component alone across all three datasets.

Table R5. Extended 2×2 component ablation across three datasets ( ρ=0.15\rho=0.15 for Projection-KL and EOT-KD).

DatasetMethodNDCG@20 ↑Tail-NDCG@20 ↑Coverage@20 (%) ↑
LastFMKD-KL0.2032±0.00250.0380±0.001347.3±0.5
Projection-KL0.2029±0.00280.0431±0.002351.7±0.8
OT-ρ=00.2015±0.00420.0391±0.001247.8±0.1
EOT-KD0.2039±0.00470.0537±0.002460.3±0.1
BeautyKD-KL0.0716±0.00050.0195±0.000477.5±0.3
Projection-KL0.0710±0.00020.0213±0.000183.5±0.2
OT-ρ=00.0705±0.00050.0199±0.000479.0±1.3
EOT-KD0.0697±0.00030.0247±0.000792.5±2.4
SportsKD-KL0.0439±0.00090.0045±0.000353.5±1.8
Projection-KL0.0438±0.00070.0053±0.000460.3±1.6
OT-ρ=00.0436±0.00010.0047±0.000156.8±1.7
EOT-KD0.0420±0.00070.0067±0.000479.3±1.1

These results reinforce the complementary roles of the two components: exposure correction changes what is transferred, while OT changes how the corrected distribution is transferred. Their combination provides stronger relevant-tail retrieval and catalogue coverage than either component alone.

We hope these additional analyses address the reviewer's concerns and would appreciate reconsideration of the score.

Reviewer_B8QvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EOT-KD, a knowledge distillation framework that first adjusts teacher predictions using an exposure-related penalty and then transfers the corrected distribution through optimal transport based on item similarity. Experiments on four recommendation datasets report improved performance on long-tail recommendation while preserving overall ranking quality.

Critical strengths
  1. Meaningful study. The paper addresses popularity bias propagation in recommendation knowledge distillation.

  2. Relatively reasonable experiment validation. The paper evaluates four datasets and compare against several established distillation methods, including UnKD, demonstrating reasonable conclusions.

Critical weaknesses
  1. Unclear novelty. Exposure correction is essentially popularity-based reweighting. The proposed KL projection (Eq. 3–4) has a closed-form solution equivalent to multiplying teacher probabilities by inverse-popularity weights, or subtracting a popularity penalty from teacher logits. The theoretical results mainly characterize standard properties of this exponential reweighting, making its algorithmic novelty relative to existing popularity-aware objectives unclear.

  2. The method uses observed interaction frequency as an exposure proxy, although interaction counts also reflect genuine user preferences and item quality. Consequently, reducing this proxy does not necessarily remove exposure bias, and the paper provides limited evidence distinguishing harmful popularity bias from legitimate popularity-driven relevance.

  3. Equations 9–10 compute an entropy-regularized transport plan but optimize only its transport cost, excluding the entropy term. This objective does not necessarily attain its minimum when the student matches the corrected teacher distribution

  4. The utility–long-tail trade-off is not fully validated. Table 3 compares projection-based KL and OT under a fixed correction setting, but does not show whether a properly tuned Projection-KL baseline could achieve similar long-tail gains at the same overall NDCG.

Questions for authors

Why does Eq. 10 omit the entropy term used to compute the optimal coupling in Eq. 9?

3 comments
AuthorsOpenReview ↗

Exposure-Aware Distillation: Theoretical Clarifications and Utility-Matched Evidence

We thank the reviewer for recognizing the relevance of the problem and the experimental coverage. We address each concern below.

W1: Eq. (4) expresses the exposure correction as a frequency-based logit adjustment, pu⋆=softmax(suT/τ−ρϕ)p_u^\star=\mathrm{softmax}(s_u^T/\tau-\rho\phi). The KL formulation gives this correction a minimum-distortion interpretation and relates ρ\rho to an exposure budget (Eq.5). Propositions 1–2 characterize the corrected target, rather than establish a new reweighting operator or guarantee student-side improvements.

Our contribution connects this corrected, user-conditioned target with teacher-derived item geometry during transfer. Projection-KL uses the same corrected target without explicit item-similarity costs. Uncorrected OT uses geometry but retains the original teacher target. EOT-KD uses the corrected target as the OT source marginal under collaborative item costs (Eqs.7–10). Tables3 and R2 evaluate what this transfer mechanism adds beyond correction alone. We will clarify this contribution and the scope of the theoretical results.

W2: We agree that interaction frequency combines historical exposure, user preferences, and item quality. As stated in the Introduction and Sec2.2, EOT-KD uses interaction frequency as an exposure proxy. Our correction does not identify these factors separately, and the datasets lack impression or randomized-exposure logs needed for direct causal validation.

Our evidence concerns logged recommendation utility and relevant-tail retrieval. Tail-NDCG/Recall evaluate held-out positive tail interactions, rather than merely counting unpopular recommendations. EOT-KD also retains 94.8–99.7% of the teacher’s NDCG@20. Appendix Fig.5 provides complementary evidence: on LastFM, Precision@20 changes from 0.07524 to 0.07484 while AvgPop falls from 39.53 to 33.45; on Beauty, precision changes from 0.01238 to 0.01192 while AvgPop falls from 65.11 to 48.14.

These results support reduced popularity concentration with largely retained aggregate utility. They do not establish causal bias removal or identify which popularity effects are legitimate. We will describe the findings consistently as frequency-based long-tail rebalancing.

W3/Q1: Why Eq.10 omits entropy. Eq.9 uses entropy to obtain a smooth transport coupling. Eq.10 evaluates its ground cost, following the ground-cost construction in entropy-regularized transport (Cuturi,2013). Entropy therefore affects the loss through the coupling; it is not an explicit reward in the outer training objective.

This differs from optimizing the full regularized OT value. We agree that Eq.10 is not a divergence and need not be minimized at puS=pu⋆p_u^S=p_u^\star.

Its relationship to unregularized OT can nevertheless be stated precisely. Write p=pu⋆p=p_u^\star, q=puSq=p_u^S, and ℓε(p,q)=⟨Πu,ε⋆,Cu⟩\ell_\varepsilon(p,q)=\langle\Pi^\star_{u,\varepsilon},C_u\rangle. For an exactly solved coupling on KK candidates,

\le \varepsilon_{\rm ot}\min\{H(p),H(q)\} \le \varepsilon_{\rm ot}\log K,$$ where $W_C$ is the unregularized transport cost. This follows from regularized optimality and the joint-entropy bounds $\max\{H(p),H(q)\}\le H(\Pi)\le H(p)+H(q)$. Thus Eq.10 approximates unregularized transport cost with a regularization-dependent error. This is not a guarantee of exact probability matching or student target recovery. The loss is used alongside BPR (Eq.11) and the component ablations support its empirical usefulness. We will clarify these distinctions in the manuscript. **W4: Comparison with tuned Projection-KL.** We swept $\rho\in\{0.05,0.10,0.15,0.20,0.25\}$ for both methods under the same protocol. Table R2 reports near-matched-NDCG operating points from this sweep. **Table R2. Near-matched-NDCG comparisons** |Dataset($\rho$)|Method|NDCG@20|Tail-NDCG@20↑|Coverage@20(%)↑|Gini@20↓|APLT@20↑| |---|---|---:|---:|---:|---:|---:| |LastFM(0.10)|Proj-KL|0.2012|0.0416|49.9|0.822|0.165| ||EOT-KD|0.2015|**0.0472**|**55.7**|**0.791**|**0.200**| |Cellphones(0.05)|Proj-KL|0.0746|0.0136|58.7|0.912|0.074| ||EOT-KD|0.0746|**0.0148**|**65.5**|**0.894**|**0.091**| |Beauty(0.05)|Proj-KL|0.0716|0.0202|79.7|0.796|0.190| ||EOT-KD|0.0709|**0.0218**|**84.1**|**0.766**|**0.223**| |Sports(0.05)|Proj-KL|0.0444|0.0051|57.4|0.915|0.076| ||EOT-KD|0.0443|**0.0056**|**63.2**|**0.896**|**0.095**| On Cellphones, both methods have the same reported NDCG of 0.0746, while EOT-KD improves Tail-NDCG by 8.8% and Coverage by 6.8 pp. On LastFM overall NDCG and tail retrieval both improve. On Beauty and Sports, overall NDCG is approximately 0.98% and 0.23% lower, respectively, with higher Tail-NDCG. These comparisons support additional long-tail benefits at comparable overall utility within the tested range beyond the fixed-correction comparison in Table3. We hope these clarifications and additional results address your concerns and would appreciate your reconsideration of the score.

Thanks for the reply; my concerns are mostly addressed.

Thank you for your positive feedback and thoughtful consideration. We sincerely appreciate your constructive comments, which have helped us strengthen the paper and clarify our contributions. We are pleased that our response has addressed most of your concerns, and we have incorporated the additional results and clarifications into the revised manuscript. We would be happy to clarify any remaining points. We would appreciate it if you could consider updating your rating.

Reviewer_MFzGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EOT-KD, an exposure-aware knowledge distillation framework for recommendation.
First, it corrects popularity bias in the teacher’s prediction distribution and then transfers the corrected distribution to a student model using optimal transport based on the item-item similarity.

Critical strengths
  1. The OPTIMAL-TRANSPORT-based KD is sound to me.
  • The existing KD methods do the one-to-one mapping. (probablity matching between the same item)
  • This method does many-to-many mapping, by using the item-item similarity.
  • I think this idea is novel and sound to me.
Critical weaknesses
  1. This paper is some kind of fusion between exposure-based rs and kd-based rs.
  • Exposure-based RS and KD-based RS are widely-investigated topics.
  • Therefore, naively combining them is not a unique contribution.
Questions for authors

What is the challenge when combining exposure-based rs and kd-based rs?

3 comments
AuthorsOpenReview ↗

Methodological Contribution, Component Synergy, and Utility-Matched Evidence

We thank the reviewer for the positive assessment of geometry-aware, many-to-many knowledge transfer. Our contribution is the formulation and evaluation of geometry-aware transfer from a frequency-corrected teacher target.

What is the challenge? The teacher holds useful personalized ranking knowledge together with popularity concentration. Correcting it changes the target, but a compact student cannot match this target exactly. The challenge is to decide how the remaining mismatch should be penalized, so that useful knowledge is kept while concentration is reduced.

How EOT-KD addresses this. The two components answer two different questions.

  1. What to transfer. The KL projection (Eqs. 3–4) gives a user-conditioned, frequency-corrected target: pu,i⋆(ρ)=pu,iTexp⁡(−ρϕi)∑j∈Cupu,jTexp⁡(−ρϕj),ϕi=log⁡(ei+δ).p^\star_{u,i}(\rho)=\frac{p^T_{u,i}\exp(-\rho\phi_i)}{\sum_{j\in\mathcal C_u}p^T_{u,j}\exp(-\rho\phi_j)},\qquad \phi_i=\log(e_i+\delta).
  2. How to transfer it. This target is the source marginal of the transport plan (Eq. 8), Π1=pu⋆\Pi\mathbf 1=p^\star_u and Π⊤1=puS\Pi^\top\mathbf 1=p^S_u, with the teacher-derived cost Cu,ab=1−cos⁡(zcu,a,zcu,b)C_{u,ab}=1-\cos(z_{c_{u,a}},z_{c_{u,b}}) (Eq. 7). The loss is ⟨Πu,ε∗,Cu⟩\langle\Pi^*_{u,\varepsilon},C_u\rangle (Eq. 10).

Example. Take a user who likes running shoes. The teacher gives high probability to a popular pair A, to a similar but less popular pair B, and to unrelated formal shoes C. Correction lowers A's share. If the student cannot match this target exactly, item-wise KL penalizes a mismatch at B and at C in the same way. Under our cost, moving the leftover mass to the similar item B is cheap, and moving it to the unrelated item C is costly.

Are both parts needed? In Table 3 (Cellphones, same support, student, splits and teacher), Tail-NDCG@20 is 0.0131 for KD-KL, 0.0147 with projection only, 0.0141 with OT only (ρ=0\rho=0), and 0.0171 for EOT-KD. Coverage is 56.78%, 63.31%, 60.60% and 82.25%. The combined gains over KD-KL (+0.0040 Tail-NDCG, +25.5 points Coverage) exceed the sum of the two single gains (+0.0026, about +10 points), so the parts reinforce each other. Overall NDCG@20 is lower (0.0730 vs. 0.0754 for Projection-KL), a trade-off we report.

Relation to UnKD. UnKD uses fixed, coarse popularity groups. EOT-KD applies a continuous, user-conditioned correction controlled by ρ\rho, and it outperforms UnKD in Recall/NDCG and Tail-NDCG on all four datasets (Tables 1–2).

Comparison with Projection-KL on all four datasets (Table R1, ρ=0.15\rho=0.15, single seed).

DatasetMethodTail-NDCG@20Coverage@20Gini@20 ↓APLT@20 ↑
LastFMProj-KL0.043151.70.8140.173
EOT-KD0.053760.30.7630.233
CellphonesProj-KL0.014763.30.9020.085
EOT-KD0.017082.30.8380.148
BeautyProj-KL0.021383.50.7720.215
EOT-KD0.024892.80.6870.304
SportsProj-KL0.005360.30.9060.087
EOT-KD0.006779.30.8370.159

Higher Coverage/APLT and lower Gini mean broader, less concentrated exposure.

To check that this is not just a larger drop in utility, we compared near-matched-NDCG points after a ρ\rho-sweep (Table R2 in our reply to Reviewer B8Qv). On Cellphones, both methods reach NDCG@20 of 0.0746, while EOT-KD raises Tail-NDCG from 0.0136 to 0.0148 and Coverage from 58.7% to 65.5%. The other three datasets show the same direction.

We will sharpen the contribution statement and add the Table 3 interaction analysis to the paper. We hope this resolves the concern and would be grateful if the reviewer would consider raising the score.

Thanks for the response. I will maintain my score.

Thank you for your positive assessment and constructive feedback. We sincerely appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.