acceptodds
Under review as a conference paper at ICLR 2027

GSRT: Granularity-Selected Residual Transfer for Sparse Targets in Two-Tower Retrieval

Abstract

Rare target events invite transfer from abundant auxiliary feedback, but the granularity of that transfer is an open choice: one global coefficient ignores heterogeneity, while per-user coefficients overfit sparse labels. We study this choice for transferring consumption feedback to sparse creator actions in two-tower retrieval. Under a local risk approximation we characterize the supportable granularity by an information budget, the heterogeneity of the transfer ormation the target labelscarry about the transfer coefficient: above a criterion, group-level coefficients have lower predicted risk than a global one, and a detection lower bound shows that far below it no procedure can exploit gris analysis we proposeGranularity-Selected Residual Transfer (GSRT), which adds a source-derived residual to a frozen target-only predictor, selects among no transfer and global, group and instance-level coefficients byand serves transfer only whenan evidence gate rejects no transfer. Because the coefficient is user-side, the score keeps an exact inner-product factorization, so one approximate-nearest-neighbour index petarget serves it. On a 4,800-cempirical boundary fallsbetween the two thresholds and the gate cuts cells with negative transfer from 30% to 5%. On production logs of a short-video platform, where the target model sees fewer than 100 positives, selecting the coeffating probabilities separatelyraises recall inside a 1% exposure quota by 0.062 without increasing Brier loss, and the gain vanishes as the target model is given more labels. On public logs where no transferable signal is detecteith no significant loss whileforced transfer loses.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Transfer Learning, Negative Transfer, Granularity Selection, Sparse Targets, Two-Tower Retrieval, Recommender Systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PRcWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GSRT, that determines how much recommendation system should use a related data source and whether that information should be shared globally, across groups or individually. It uses statistical tests to avoid harmful transfers while keeping the system efficient. Their method works well on synthetic data, finds no useful transfer on KuaiRand-Pure and improves recall on production data when original model has little training data.

Critical strengths
  1. The authors test pre-defined hypotheses, report all results and use controls to distinguish real improvements from simple recalibration. They also identity results caused by data leakage and make their findings more trustworthy.
  2. Their design is practical and efficient. Production study shows that choosing weights based on ranking performance rather than prediction accuracy can be more effective in extremely data-sparse setting -- ranking based selection improves recall while recalibration helps maintain prediction quality.
Critical weaknesses
  1. Only positive real-data results depends on a weak, frozen model that could be improved by retraining.
  2. Paper uses ranking-based method rather than the brier-gated GSRT analyzed in the theory and there is no online evaluation or evidence of improvement to an actual production system.
  3. The theoritical predictions are broad and match the best method in only 56% of experimental cases, the predictive power of the theory is weak.
Questions for authors
  1. Please report per-user recall@k or recall within each group, in addition to global top-1% recall, to see whether the gains remains without cross-user reordering and better reflect performance in a candidate-generation setting.
  2. It is unclear whether using placement-based selection with GSRT evidence gate reproduce the same regime pattern and H1/H2 boundary results.
  3. The definition of weight w=σ′(z∗)2w = \sigma ' (z^*)^2 makes it depend on H, while the proof assumes it is independent of H. Please clarify the definition.
7 comments
AuthorsOpenReview ↗

W1: Validating on Real Production Data and Retaining the Retraining Comparison

Question:

The only positive real-data result depends on a weak, frozen model that could be improved by retraining.

Revised locations: §1; §4.1–4.2; Appendices G, I, J, K and N

Response:

We agree that a positive result from a limited base must be assessed against feasible retraining. We evaluate real production data offline in Appendix I and the 20-cell production-score subset study, reporting both positive transfer outcomes and evidence that a stronger base can be preferable.

We compare against a stronger base on real production data: a 3M-row target-only model exceeds the adapted 1M-row candidate on AUC in that evaluation. Likewise, the production-score subset evaluation shows a positive smaller-base upload within-user recall change, a negative camera change, and no mean user-recall improvement for the larger upload base. These are conditional offline measurements, not evidence that preserving a weak model is a sound production choice.

The existing KuaiRand study does not establish a robust positive transfer result after accounting for the shared neural base and item-feature ablation. The new real Ali-CCP sample experiment adds a same-feature all-data refit. In its exploratory convex diagnostic, the complete policy increases mean user Recall@1 from 0.02564 to 0.04359, but global AUC falls from 0.57074 to 0.56593 and quota recall from 0.02381 to 0.01905. The all-data refit reaches AUC 0.58817 and quota recall 0.04762. Thus the new experiment measures a limited benefit at one decision unit while directly preserving evidence in favor of retraining on other metrics. We retain those findings and position GSRT around the constrained adaptation problem and its statistical/serving analysis. The real-production-data validations likewise retain the achievable retraining alternative; an online deployment advantage remains unestablished.

AuthorsOpenReview ↗

W2: Connecting the Ranking Variant to Its Actual Evidence

Question:

The paper uses a ranking-based method rather than the Brier-gated GSRT analyzed in the theory, and there is no online evaluation or evidence of improvement to a real production system.

Revised locations: §3.2–3.4; §4.1–4.2; Appendices I–N

Response:

The revised method separates Brier-selected and placement-selected GSRT. They share the additive score and inner-product factorization but optimize different objectives. The ranking implementation selects by AUC placements, uses count pooling, and calibrates probabilities separately; its group-gated arm does not search the full Brier hierarchy.

Section 3.4 adds the exact within-user contrast Δz=Δb+λ(u)Δr\Delta z=\Delta b+\lambda(u)\Delta r, explaining why a global allocation change can differ from a user's candidate-ordering change. The weighted-projection theory remains a probability-risk characterization, while the placement screen is an approximate test of its stated statistic. It is not a guarantee of recall or calibration improvement. Appendix M now tests the loss distinction directly in a fixed 180-cell sweep: the placement- and Brier-screened family choices differ in 32 cells. Of 64 ranking admissions, one has negative probability-weighted AUC gain. The complete results therefore characterize the ranking procedure rather than asserting that it inherits the squared-loss boundary.

The original Appendix I ranking experiment, the production-score subset evaluation and the additional 120-cell nonlinear-model study in Appendix L provide offline validation on real production data. KuaiRand and the added Ali-CCP official-sample diagnostic supply complementary public-data evaluations. The latter shows why the loss/decision distinction matters empirically: the exploratory complete policy increases mean within-user recall while reducing global AUC and quota recall, with stronger global metrics for the all-data refit. The industrial SSE figures describe a bundled system and are not attributed to GSRT. In Appendix I, the headline group candidate is ungated and the Brier gains are reported probability-based expected-loss diagnostics, which remain distinct from empirical observed-label losses. No actual GSRT online trial or creator-growth effect is claimed. The revision addresses the request for evidence on real production data while distinguishing offline validation from an online system improvement.

AuthorsOpenReview ↗

W3: Exact-Family Prediction and Risk Proximity Are Different Claims

Question:

The theoretical predictions are broad and match the best method in only 56% of cases, so the theory's predictive power is weak.

Revised locations: §3.3; §4.3–4.4; Appendices B, F and M

Response:

We agree that 56% agreement does not justify describing the theory as an accurate exact-family selector. The revised paper retains this original result, together with the 92% within-five-per-million result, and identifies them as two different criteria: predicting the identity of the best family and being close to its risk. The second number does not repair a weak claim about the first.

The revised analysis separates the population approximation benefit of a resolution from the local cost of estimating it. In particular, the nested-partition projection identity exactly characterizes the surrogate-risk reduction from refining an observable grouping. Turning that identity into the scalar information criterion uses additional assumptions; finite grids, shrinkage, calibration and the selection procedure introduce further differences from the simple prediction. The implemented method therefore continues to use held-out outcomes rather than replacing selection with the theoretical budget.

The original controlled figures, tables and 56% statistic remain the record of that campaign. We now label the dashed h2I=1/4h^2\mathcal I=1/4 line as a local reference. The finite-hh condition in Theorem 3.4 instead uses h2Ih≤1/4h^2\mathcal I_h\le1/4, with the factor eh∣r∣e^{h|r|} retained in Ih\mathcal I_h. For a fixed design, Ih\mathcal I_h tends to the global-reference information I0\mathcal I_0 as h→0h\to0; identifying this limit with the plotted budget requires matching the reference probabilities, observed row set and two-group parameterization. We clarify this distinction in Section 3.3, Figures 2 and 6, Sections 4.4-4.5 and the retained H1 hypothesis. The old line is not asserted to be the exact finite-hh boundary. These changes correct the interpretation without recomputing the curves, experimental results or 56% statistic.

Appendix M adds an independent ranking-resolution sweep, where the fitting diagnostic and validation choice are recorded separately. It is an additional loss-specific experiment, not a recalculation of the original 56% statistic or a reproduction of that campaign.

Reviewer_ncKTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how to transfer abundant consumption feedback to rare content-creation targets in two-tower retrieval, and at what granularity (none, global, group, or per user) the transfer coefficient should be set. It derives a criterion, the heterogeneity of the transfer relation times the Fisher information that the target labels carry about the coefficient, for when group coefficients beat a single global one, and proposes GSRT, which adds a weighted consumption score to a frozen target model, selects the granularity by cross-fitted held-out loss, and serves transfer only when a paired test rejects no transfer. Experiments cover a 4,800-cell synthetic benchmark, production logs of a short-video platform, and the public KuaiRand-Pure dataset.

Critical strengths

(S1) Careful and transparent evaluation. The synthetic hypotheses and protocol are fixed before the confirmatory runs on fresh seeds, contrasts are Holm-adjusted, and controls separate the effect of recalibration from that of the source signal. Null and negative results are reported openly, including the leakage audit on KuaiRand-Pure and the shrinking production gain as the base gets more data.

(S2) Safe and deployable design. The evidence gate reduces the cells with negative gain from 30% to 5% on the benchmark, and on KuaiRand-Pure the selector declines transfer without loss where forced transfer loses 70.2 per million. Because the coefficient depends only on the user, the combined score remains an inner product and can be served from a single ANN index per target.

Critical weaknesses

(W1) Limited novelty. Most components are classical, as the paper itself acknowledges. The criterion for when group coefficients beat a single global one (h²I ≳ G − 1) is the classical result on when to estimate group means separately rather than pool them [1] (Sec. 3.3). The evidence gate is a pretest, and the group step is partial pooling (Sec. 2). The detection lower bound (Thm. 3.4) uses a standard two-hypothesis argument (App. B.8). Pooling groups by Fisher information instead of by positive counts (Eq. 2) is the usual precision weighting of shrinkage, and it brings no gain: the two perform the same in the headline setting, and count weighting is slightly better in the setting built to compare them (Table 8).

(W2) The benchmark tests the predictions only under the assumed structure. The theory assumes that the ideal correction is linear in the residual with a coefficient set by a governing variable (Prop. 3.1), and the synthetic benchmark generates the target logit as a linear function of the source logit with a coefficient set by such a variable (App. E.1). All five confirmatory hypotheses (H1–H5) are tested there, so the benchmark shows that the predictions hold when this structure holds, but not that real transfer relations have it.

(W3) Narrow real-world evidence. The only real-data gain the paper credits to transfer comes from production logs with a deliberately scarce base, a linear two-tower model the authors trained on a 1M-row prefix of the training stream (App. I). The gain vanishes as the base is given more data, and a base trained on 3M rows already ranks better than the adapted 1M-row base (Sec. 4.5, Table 38). The gain also comes from a modified procedure outside the theory, which selects by AUC, pools groups by positive counts, and recalibrates with an isotonic map, because the squared loss used by the theory selects a negative coefficient in 59 of 60 cells (Sec. 4.5).

[1] B. Efron and C. Morris. Stein's Estimation Rule and Its Competitors: An Empirical Bayes Approach. Journal of the American Statistical Association, 68(341):117–130, 1973.

[2] X. Ma et al. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate. SIGIR 2018.

Questions for authors
  • Q1. GSRT selects the resolution by held-out loss and does not estimate the information budget (Sec. 3.3). How should a practitioner use the budget in practice, for example to decide whether collecting more target labels or defining groups is worthwhile?
  • Q2. How do the selector and the boundary predicted by the budget behave when the ideal correction is not linear in the residual?
  • Q3. In what practical situation is the base frozen while a few hundred controller positives are available, given that a base trained on a 3M-row prefix already ranks better than the adapted 1M-row base (Sec. 4.5)? How does GSRT compare with MMoE and PLE trained on the same production stream?
  • Q4. Can GSRT be evaluated on a public dataset with a comparably sparse target, such as Ali-CCP [2] (conversion rate about 2×10⁻⁴), even though its click-to-conversion funnel differs from the consumption-to-production setting?
6 comments
AuthorsOpenReview ↗

W1: A Resolution Problem with an Explicit Approximation Decomposition

Question:

Most components are classical, as the paper admits. The criterion h²I ≳ G − 1 is the classical result on when to estimate group means separately rather than pool them (Efron & Morris 1973). The evidence gate is a pretest, the group step is partial pooling, and the detection lower bound (Thm 3.4) uses a standard two-hypothesis argument. Fisher-information pooling (Eq. 2) is the usual precision weighting of shrinkage and brings no gain: it ties with count weighting in the headline setting and is slightly worse in the setting built to compare them (Table 8).

Revised locations: §1–2; §3.2–3.4; §4.1; Appendix B

Response:

We agree that partial pooling, pretesting, precision weighting and the two-point lower-bound technique are classical. The revised paper credits those foundations and no longer presents h2I≳G−1h^2\mathcal I\gtrsim G-1 as a new general pooling principle. We also retain the original count-versus-information comparison and explicitly state that it does not establish an advantage for information weighting.

The technical contribution is organized around the constrained adaptation decision. Section 3.3 now characterizes the best observable correction without assuming the ideal correction is linear in the source score. For nested observable partitions P⪯Q\mathcal P\preceq\mathcal Q, their projected coefficients satisfy

=\mathbb E\!\left[wr^2(\alpha_{\mathcal Q}-\alpha_{\mathcal P})^2\right].$$ This places global, coarse-group and fine-group corrections on one approximation-risk scale, which can be compared with their estimation costs. It also retains the approximation error outside the scalar source family. Section 3.4 then connects the adapted score to the retrieval decision: within-user ordering depends on source differences between that user's candidates, whereas source variation that only shifts whole users can affect global allocation without improving user-level retrieval. GSRT combines this resolution choice with a user-side correction that preserves the inner-product interface and a no-transfer option. Our production-score subset evaluation evaluates that decision on real production data; KuaiRand provides the existing real public-data comparison, and the added Ali-CCP sample diagnostic evaluates within-user recall, global metrics and retraining together. We regard the contribution as this formulation, analysis and empirical characterization; it is not a claim that established pooling or calibration components have individually become novel.
AuthorsOpenReview ↗

W2: Extending the Analysis and Validating Nonlinear Models on Real Production Data

Question:

The theory assumes the ideal correction is linear in the residual with a coefficient set by a governing variable (Prop 3.1), and the synthetic benchmark generates the target logit exactly that way (App E.1). All five hypotheses H1–H5 are tested there, so it shows the predictions hold when the structure holds, not that real transfer relations have it.

Revised locations: §3.3; §4.3–4.4; Appendices B, E–F, L and N

Response:

We agree that evaluating a known generating structure cannot establish the structure of real transfer relations. The original controlled benchmark in Appendices E–F retains that limited role. To address this concern, the revision also evaluates GSRT on real production data in Appendices I, J, L and M, alongside the real public-data studies.

The revision supplies two methodological responses. First, the weighted-projection analysis permits an arbitrary ideal correction ε\varepsilon. For observable inputs XX, the best coefficient within the scalar source family is

a(X)=E[wrε∣X]E[wr2∣X].a(X)=\frac{\mathbb E[wr\varepsilon\mid X]}{\mathbb E[wr^2\mid X]}.

The projection error can remain nonzero. The nested-partition identity measures what finer observable resolution removes, without requiring the entire true correction to be linear.

Second, Appendix L reports nonlinear-model validation on real production data. It does not assume that the observed target labels follow the fitted base plus λgr\lambda_g r. All 120 cells across the aligned, weakly related and independent-preference evaluation settings, two targets, ten seeds and two budgets are reported. The initial budget returns the base in every cell. An explicitly exploratory extension, fixed after that result and using fresh evaluation seeds, serves transfer in 5/60 cells. A broader source-feature refit performs better in the aligned setting, demonstrating information outside the restricted adaptation family.

These offline production-data tests address the reviewer's concern that evidence should extend beyond a matched synthetic mechanism. KuaiRand's null and feature-ablation findings remain, and the new Ali-CCP sample diagnostic adds observed conversion labels with metric-dependent transfer and an all-data refit. The real-data studies evaluate transfer empirically; they do not establish that real transfer relations obey the exact logistic residual structure or demonstrate a causal online uplift.

AuthorsOpenReview ↗

W3: Clarifying Real Production Validation and Separating the Ranking Variant

Question:

The only real-data gain comes from production logs with a deliberately scarce base (a linear two-tower model trained on a 1M-row prefix, App I). The gain vanishes with more data, and a base trained on 3M rows already beats the adapted 1M-row base (Sec 4.5, Table 38). The gain also comes from a modified procedure outside the theory (AUC selection, count pooling, isotonic recalibration), because the theory's squared loss selects a negative coefficient in 59 of 60 cells.

Revised locations: §3.2–3.4; §4.1–4.2; Appendices I, J, K, M and N

Response:

We evaluate GSRT offline on real production data in Appendix I and the production-score subset study. Appendix I reports a 0.062 quota-recall gain and a 0.0089 within-group AUC contrast. These gains must be interpreted together with the stronger-base comparison, the selected procedure and the unfavorable conditions.

On the real production data, the stronger-base comparison is informative about the procedure: the reported 3M-row target-only AUC of 0.912 exceeds approximately 0.845 for the adapted 1M-row candidate. The 20-cell production-score subset evaluation reports a 0.4081-to-0.4705 mean within-user upload-recall change on the production-data subset and the camera and larger-base conditions shown alongside it.

The methodological distinction you identify is addressed separately. The revised method names the Brier-selected hierarchy and the placement-selected ranking variant. The latter uses AUC placements, count pooling and post-fit calibration; its group-gated arm screens one pooled-group candidate against the base. Probability-risk theory does not give it an AUC or recall guarantee. The headline 0.062 gain refers to the ungated group candidate; the 0.007/0.008-per-million Brier entries are reported probability-based expected-loss diagnostics, distinct from empirical losses on observed labels. We label these quantities explicitly and report screened-policy results separately. The original raw-probability squared-loss fit also does not characterize all calibration-aware probability objectives.

Appendix M adds a fixed 180-cell comparison of placement- and Brier-selected global/group families under independent validation. Their screened family choices differ in 32/180 cells. This directly evaluates the ranking variant, while preserving the distinction between loss-specific empirical behavior and the probability-risk theory.

The revised real-data evidence includes the production-data offline evaluations, KuaiRand and the new Ali-CCP official-sample experiment. The latter contains only 41 training and 42 test conversions: its exploratory convex extension improves mean within-user recall, lowers global AUC and quota recall, and does not beat the all-data refit on those global metrics. Together, these evaluations respond to the request for real-data validation while retaining same-stream retraining comparisons, limited positive results and negative results. They do not establish general superiority over retraining or an online creator-growth effect.

Reviewer_v2vtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

After considerable effort, here is the paper's main problem statement as I could reconstruct it (it is never stated explicitly): On a short-video platform, retrieve items that lead users to create, under a two-tower serving constraint, with very few creation labels.

Concretely, the paper asks how to decide λ in the following equation:

z_gsrt = z_target + λ(u)·r_consumption + c (eqn 1)

It then poses the research question: at what resolution (none / global / per-group / per-user) can sparse target labels support a transfer coefficient? The score is P(upload | user, item).

Critical strengths

None.

Critical weaknesses
  1. The abstract is inaccessible and very poorly written, with no context set up at the opening. The paper is very difficult to follow. It uses non-standard terminology that is not defined. The intro and contributions use h^2I, "the gate," and G−1 before they are defined. The paper never clearly states the context, the task, or the research problem it is trying to solve.

  2. "A consumption predictor thus carries information about production without measuring readiness to produce." This is a counterpoint to the premise of the paper itself. If the consumption predictor is a subpar information source, why tie it to the model at all as the main source of prediction?

  3. The real question is why the reader should care about the problem statement in the first place. The problem appears constructed without real grounding. It reads as a quirk of the solution method the authors have chosen, so for the actual task they are trying to solve, this technicality may not have any real importance.

  4. The work's contributions are never connected with the goal they (vaguely) started with. Authors themselves claim: "The downstream outcome (do users actually create more?) is explicitly deferred to an untested randomized design in Appendix H.5, so the end goal is unmeasured."

  5. Once the problem is stated as above, the obvious baselines/promising approaches become first-class alternatives: fine-tuning, consumption embeddings as features, multi-task training, and simply waiting for more labels.

Questions for authors
  1. "…item vectors for ANN search (Covington et al., 2016; Malkov & Yashunin, 2020), so a controller that uses user–item cross features cannot be served." What is a controller here? Non-standard terms should be defined before use.
  2. "This paper asks how much of that information a production predictor should use, and at what resolution, from one transfer coefficient shared by all users to one per user." What is a transfer coefficient, and why assume it is the only possible solution?
  3. Are you trying to train a multi-task model where one task has sparse signals?
  4. What is a "dense consumption task"?
  5. What are the production target classes? It would help to introduce the task early, with concrete examples.
  6. In Section 3.1, how is z_Pk(u,i) normalized to be a user-level production function?
8 comments
AuthorsOpenReview ↗

W1: Stating the Task Before the Technical Choice

Question:

The abstract is inaccessible and poorly written, with no context at the opening. The paper is hard to follow and uses undefined non-standard terms. The intro and contributions use h²I, "the gate" and G−1 before defining them. The context, task and research problem are never clearly stated.

Revised locations: §1; §3.1–3.2

Response:

Thank you for pointing out that the original presentation made the reader reconstruct the task. The revised abstract and Introduction now begin with a concrete decision: rank candidate videos for a user when the recorded target is a rare creator action, such as strict upload or camera entry. We then state the question the method answers: given a frozen target predictor, an available consumption predictor, and a limited window of new target labels, how much source information should be added and at what user-side resolution?

We define a transfer coefficient as the multiplier of the added source score and a controller as the function producing that multiplier. The global, group and user-feature cases are explained in words before the information-budget notation appears. Section 3.1 specifies the prediction unit and operating constraint; Section 3.2 introduces the score, candidate hierarchy and data roles. The contributions now connect the retrieval-compatible adaptation, the weighted-projection analysis, and evaluation at the retrieval decision unit. We retain the original experimental structure and complete appendices, while marking the substantive changes in blue.

AuthorsOpenReview ↗

W2: Consumption Is an Optional Correction to the Target Predictor

Question:

The sentence "A consumption predictor thus carries information about production without measuring readiness to produce" contradicts the paper's own premise. If the consumption predictor is a subpar information source, why tie it to the model as the main source of prediction?

Revised locations: §1; §3.2; §4.1–4.2

Response:

The target-only predictor is the main predictor in GSRT. The consumption score is an optional additional direction, whose usefulness is decided using target labels. The revised Introduction makes this explicit and corrects the implication that the prior consumption/upload comparison by itself demonstrated transferable information.

In the score b(u,i)+λ(u)r(u,i)b(u,i)+\lambda(u)r(u,i), bb predicts the target action, while rr is the frozen consumption-derived score. The coefficient can be zero, and it can be negative. A consumption model may capture preference information that the scarce target fit has not estimated well; it need not measure a user's readiness to create. Whether that information helps the particular target remains an empirical question.

Appendix I and the new 20-cell production-score subset study report offline ranking results on real production data. They show both useful and unhelpful source directions across the evaluated conditions. The real KuaiRand study does not demonstrate a transfer gain on the shared neural base, and its feature ablation substantially reduces the earlier small-budget gain. A new Ali-CCP sample experiment gives a concrete mixed case: in the exploratory convex diagnostic, the complete policy improves mean within-user recall but reduces global AUC and quota recall. We therefore treat consumption as optional evidence to test, not as a universally useful source or a demonstrated proxy for readiness.

AuthorsOpenReview ↗

W3: A Concrete Adaptation Problem Under a Retrieval Constraint

Question:

Why should the reader care about the problem? It seems constructed without real grounding, a quirk of the chosen solution method that may have no real importance for the actual task.

Revised locations: §1; §3.1–3.4; §4.1–4.2; Appendices J and N

Response:

The practical problem is to adapt an exported retrieval predictor with a new, sparse target-label window while preserving its query/item interface. A model-export boundary, access limited to scores or embeddings, or an update window between tower refreshes can create this constraint. These are applicability conditions, rather than a claim that a platform should deliberately retain an inferior model.

The industrial description grounds the intended label scale: it reports 54,710 strict-upload labels and 51,687 camera-entry labels among 784,955,474 filtered rows in one day. That motivates pooling sparse target evidence while allowing observable differences between user groups. The user-side coefficient addresses the serving constraint by preserving an inner-product score.

To address the concern about real-world grounding, we evaluate GSRT offline on real production data in Appendix I and the new 20-cell production-score subset study. KuaiRand supplies the existing real public-data evaluation. We have additionally evaluated the official Ali-CCP preprocessed sample, with click as source and observed click-through conversion as target; its metric-dependent result and retraining comparison appear in §4.1 and Appendix N. These evaluations connect the operational formulation to observed data while retaining the negative results and the distinction between offline retrieval performance and a causal production-deployment benefit.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A kind request regarding the discussion of submission 17793

Dear Area Chair,

Thank you for handling our submission. We would like to briefly summarize the main concerns raised in the reviews and how the revised manuscript addresses them. Substantive changes are marked in blue.

  1. Clarity and problem framing We rewrote the abstract and introduction so that the task, the serving constraint and the research question come before any notation. Key terms such as “transfer coefficient”, “controller” and “dense source” are now defined where they first appear. We also clarified that the target predictor is the main model. The consumption score is an optional correction whose weight is learned from target labels.

  2. Contribution and theory We clarified how the method relates to classical pooling and shrinkage results, and we sharpened the contribution around the constrained adaptation decision. The analysis now includes a weighted-projection result that does not assume the ideal correction is linear. Its nested-partition identity gives the exact surrogate-risk reduction from refining an observable grouping. We also clarified the definition of the weight and revised the proof of Prop. 3.1 to match.

  3. Theory and the implemented variants The Brier-selected and placement-selected variants of GSRT are now described separately. A new 180-cell comparison (Appendix M) evaluates both families under independent validation.

  4. Real-data evidence In addition to the production-data evaluation in Appendix I, we added three studies: a 20-cell production-score subset study, nonlinear-model validation on production data (Appendix L) and an experiment on the public Ali-CCP sample (Appendix N). The revision also makes retraining alternatives explicit. All-data refits, a stronger base and MMoE/PLE on KuaiRand are reported alongside GSRT. We also clarified the setting GSRT is designed for: adapting a frozen two-tower model with sparse new target labels while keeping its retrieval interface.

  5. Retrieval-level evaluation We added within-user Recall@1, which measures how each user’s candidates are ordered and so matches the candidate-generation setting. In the 20-cell production-data replay, mean within-user Recall@1 for upload improves from 0.4081 to 0.4705.

After the discussion, Reviewer PRcW raised their rating to 3. They noted that the revised proof, the clarified variants and the new experiments addressed their main concerns. We believe the revision addresses the main points raised across the reviews, and we would be grateful if you could take it into account in your assessment.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.