- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces the concept of residual user preference and accordingly proposes the framework of SMPHP, which improves the recommendation scores given by frozen teachers by using user history. Extensive experiments are conducted to show the validity of the framework. A benchmark is also produced for similar tasks.
Critical strengths
I like the idea of utilizing the user history to calibrate some parts of the frozen teachers’ recommendation scores and improve the expected utilities.
Experiment section is thorough. It looks persuasive to me that the framework SMPHP is useful in real datasets.
Critical weaknesses
The problem and model look well-defined, but there seems to be no theoretical guarantee to support that the proposed framework will always (or under some assumptions) yield better results than frozen teachers. For example, how do we make the RHS of equation (12) positive? What are equivalent conditions? I read that you admitted in section 6 that ``We do not claim ... that every accepted edit improves utility.’’ What are the conditions under which your framework is superior? If this is not specified, I think the significance of this work will be compromised.
Section 3.1 could be more clear. For example, many letters are introduced and defined without even specifying what they are.
Label of y-axis in the first plot of Figure 2 lies out of margin.
The README in the code repo is in Chinese.
Questions for authors
Q1. I was reading section C.2 and I’m a bit confused by how you define the metrics. The word ``utility’’ looks like an abstract metric, unlike metrics such as accuracy, F1, etc. You defined utility in equation (3), but how does that reflect the objective power of models in real datasets? It gives me the feeling that you shoot an arrow first and then draw the target. Maybe I missed something but please clarify that.
5 comments
2026-11-9-rebuttal-R3-WQGP-PART2
1.2 Second, why is this not "shooting the arrow first and then drawing the target"? (using the analogy of "the target, the gun and the bullet")
Let us borrow the analogy of archery and target shooting:
-
The "target" (the private candidate utilities ) was set up before the shot and locked in a safe. In the data-construction stage, the simulator computes the true utility of every candidate from 9 components (Eq. (3) in Section 3.2 and Appendix A.3), namely the user's long-term profile, recent state, session goal, negative aversion, novelty, repetition and popularity. The error between the sum of the 9 components and the stored utility is within . These utilities are sealed separately in a private evaluation file. The target was generated before SMPHP existed; it was fixed before SMPHP existed.
-
The "gun" (the SMPHP algorithm) cannot see the "target" when it fires. At test time SMPHP sees only the public candidate item features, the output of the frozen teacher , and the "bullet", i.e., the current user's interaction log . The public observations and the private utilities are stored in different files and are linked only by the sample ID. The utilities of the test set never enter inference. The utilities of the validation set are used for one thing only: choosing the gate threshold. This is the same practice as ordinary validation-set tuning.
-
The most direct counter-evidence: the "history swap" experiment (Figure 2c). If the target had been drawn to match the gun that is SMPHP, then this gun should hit regardless of which bullet is loaded. We ran a counterfactual experiment in Section 5.3 (Figure 2c): the frozen teacher, the candidate slate, the sealed target and the SMPHP algorithm (the gun) are all held fixed, and only the "bullet" loaded into the gun is replaced by another user's history. As a result, the mean utility gain changes from to , turning from positive to negative. Likewise, if the history direction is replaced by an uninformative uniform distribution (the Uniform direction in Table 3), the utility drops by . Only when the current user's own history (the matching bullet) is loaded into SMPHP (the gun) can the pre-sealed utility (the target) be hit; load the wrong bullet and the shot misses. This shows that the gain comes from the user's own history, not from a metric that favours our method.
-
What the bullet is loaded with also matters. Besides "swapping in someone else's bullet", we also checked what happens when "the same bullet is damaged" (Table 13, 126 diagnostic units):
| Treatment of the history | ΔEU |
|---|---|
| Full history | +0.01997 |
| Keep only the most recent 20 events | +0.01995 |
| Keep only the most recent 10 events | +0.01915 |
| Keep only the most recent 5 events | +0.01761 |
| Keep only 0 / 1 / 3 events (below the gate lower bound, forced fallback) | 0 |
| Shuffle behavior types (the correspondence of click, save, reject, etc.) | +0.01436 |
| Shuffle timestamps | +0.01848 |
| Use only the most recent 3 days | +0.00079 |
| Replace with another user's history | −0.01299 |
The gain varies with the content of the history: the shorter and the noisier the history, the smaller the gain; after the behavior types are shuffled the gain falls by about 28%. If the target had been drawn to match the gun, these modifications of the history content should not produce such a regular pattern.
2026-11-9-rebuttal-R3-WQGP-PART3
1.3 Third, the conclusion holds equally if we do not use the continuous EU metric at all and look only at hard metrics such as the ranking and Top-1
To remove your concern about the numerical scale of the continuous utility, we switched to an evaluation that does not use the utility values at all: we treat the candidate slate as an ordinary ranking task, define the relevant item as the candidate with the highest true utility (using only the information "which one is best", not the size of the utility), then rank by probability and compute HR, NDCG and MRR. Any monotone transform of the utilities (for example squaring all of them, multiplying by 100, or taking logarithms) leaves these metrics unchanged, so they cannot be "inflated" by the scale of the utility.
Table Q1-a: change in ranking metrics over 132 test units (SFT and GRPO with 7 domains each, ReRe and TAWin with 4 domains each, 3 seeds per domain, two test sets Test-IID and Test-OOD)
| Metric | Mean change | Share of units with ΔEU>0 in which this metric also improves |
|---|---|---|
| HR@1 | +5.19 percentage points | 132 / 132 |
| HR@3 | +9.66 percentage points | 132 / 132 |
| NDCG@1 | +5.19 percentage points | 132 / 132 |
| NDCG@3 | +7.90 percentage points | 132 / 132 |
| NDCG@5 | +6.91 percentage points | 132 / 132 |
| MRR | +5.71 percentage points | 132 / 132 |
HR@1 is the most conservative of these metrics, meaning "is the best candidate placed first". It rises by 5.19 percentage points on average.
Table Q1-b: correlation between EU and common ranking metrics (132 units)
| Comparison | Spearman correlation |
|---|---|
| EU of the teacher vs. NDCG@1 | 0.856 |
| EU of the teacher vs. NDCG@3 | 0.869 |
| EU of the teacher vs. NDCG@5 | 0.854 |
| ΔEU vs. ΔHR@1 | 0.536 () |
| ΔEU vs. ΔNDCG@5 | 0.818 |
| ΔEU vs. ΔMRR | 0.629 |
All correlations are positive, and . This shows that EU and the common ranking metrics measure the same thing. EU is finer, because it can also see the movement of probability between candidates when the ranking does not change, which is exactly the phenomenon this paper studies.
In addition, the paper already contains two Top-1 results that do not depend on utility weighting, consistent with the above:
- The probability received by the best candidate (Table 8). Looking only at the single item with the highest true utility in the slate (), after adding SMPHP the probability that the four teacher models assign to it increases on the test sets: GRPO , SFT , ReRe , TAWin .
- The correction win rate of the deterministic Top-1 recommendation (Appendix E.3). If the system finally recommends only 1 item (taking ), then on Test-IID the number of times SMPHP turns a wrongly ranked Top-1 into a correct one is 8.8 times (GRPO), 8.6 times (SFT), 10.8 times (ReRe) and 8.0 times (TAWin) the number of times it turns a correct one into a wrong one. The corresponding mean Top-1 utility gains are +0.0254, +0.0407, +0.0847 and +0.0881. Cases where it gets things wrong also exist, and their share is not zero, which is consistent with the later conclusion that "not every adjustment is guaranteed to be positive".
So the conclusion is the same whether we measure with expected utility EU or with such common metrics as Top-1 hits, correction win rates, NDCG and MRR.
Response to Reviewer WQGP
We thank the reviewer for the careful reading and for the encouraging comments on the idea and the experiments. We answer Q1, W1 and W2–W4 in order.
Q1. Why expected utility instead of accuracy or F1? Is it "shooting the arrow first and drawing the target later"?
Why not accuracy or F1. In one display of 8 candidates the user usually clicks one; the other 7 all have feedback 0. SMPHP keeps the protected candidate fixed and moves probability among the others, for example from a disliked item to a liked one. Accuracy and F1 do not change, because the clicked item has not changed. Measuring this needs a utility for every candidate, so we use expected utility EU(p)=∑ipiUi. EU is a simulator quantity, not a click rate or revenue.
The target is drawn before the arrow. The simulator computes each candidate's utility Ui from nine components of the user state (Appendix A.3) before SMPHP is run, and stores it in a private file. At test time SMPHP sees only public item features, the teacher distribution and the user's history. Validation utilities are used only to choose the gate threshold.
If the metric favored our method, changing the input should not matter. It does:
Standard ranking metrics agree. We take the candidate with the highest utility as the relevant item and compute HR, NDCG and MRR. These use only which candidate is best, not the utility values. On 132 held-out units (Appendix J):
The Spearman correlation between ΔEU and ΔHR@1 across units is 0.536, and with ΔMRR 0.629.
On real data EU cannot be computed, because logs do not record the utility of items the user did not choose. On ReRe's public data we report NDCG@10 with paired intervals: the intervals exclude zero in the three domains where the gate activates, and on Yelp the gate never activates and returns the teacher unchanged. We do not claim online gains (§6).
W1. When is the RHS of Eq. (12) positive? (Eq. (13) in the revision)
Exact condition. Let d=q⋅∣EH−p⋅∣Etea be the change of distribution inside the adjustable set E, and AH=⟨d,UE⟩. Then ΔEU=ρMEAH. Since ρ≥0 and ME≥0, ΔEU>0 if and only if ρ>0, ME>0 and AH>0. In words, the history distribution must give higher expected utility than the teacher distribution inside E.
New Appendix M gives five propositions with short proofs:
Checks. The identity holds on all 178,200 events (maximum error 8.36×10⁻¹⁶); linearity in ρ gives R2=1.0; the bound is never violated on 118,800 test events. Grouping test events by the utility range inside E, the share with AH>0 rises from 0.707 (smallest range) to 0.930 (largest), while the mean |ΔEU| rises from 0.0069 to 0.0406, as Proposition 4 predicts.
The gate. At test time Ui is unknown, so no method can guarantee AH>0 for every event. The gate estimates it from history agreement and concentration: precision 0.879 and recall 0.930 for the sign of AH, with mean AH 0.282 on accepted events and 0.055 on rejected ones (Appendix J). Recalibrating the threshold on Validation lowers the share of harmful accepted edits for GRPO from 25.1% to 17.9% and the worst-5% loss from −0.0323 to −0.0118 (Table 11).
W2–W4. Presentation