- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies credit assignment when an LLM jointly generates multiple retrieval actions, specifically in sponsored search, but receives feedback for the resulting set as a whole. It proposes CASI, which caches each retrieval action's results, evaluates alternative subsets under a fixed snapshot, and aggregates their marginal values into per-action Shapley credit. Bounded, zero-sum adjustments then use this credit to shape the actions’ training advantages. The paper shows that singleton, leave-one-out, and full-set evaluations generally do not determine Shapley credit for sets of at least four actions. Validation includes an auction-replay audit of subset interactions and credit alignment with Shapley reference, comparisons of policies trained with different credit rules, and an online A/B test in which the CASI-trained policy improves the reported advertiser-value metric.
Critical strengths
-
The method - The paper addresses the challenge of assigning credit to individual retrieval actions when only their joint outcome is observed, and each action’s contribution depends on the others. A novel method is presented: snapshot-grounded set intervention evaluates different combinations of actions, and Shapley-based aggregation assigns these evaluations to action-level credit. This may provide a more informative training signal for the paper’s central problem.
-
Evidence from a production deployment - The seven-day randomized A/B test reports increased advertiser value for the CASI-trained policy over the production control. Evaluating the policy in a deployed sponsored-search system is consequential because it provides evidence of practical value beyond the theoretical analysis and offline auction replay evaluation.
Critical weaknesses
-
Insufficient experimental setting details - The paper omits essential details about the models used and SFT initialization, training and evaluation data, RL algorithm, and key hyperparameters. It also gives only a few concrete details about the definition of the evaluation measures (e.g., advertiser value), retrieval, and the auction production system, or the frozen evaluator used to score coalitions. These omissions make the method difficult to reproduce and limit the reader’s ability to assess the validity of the results and conclusions.
-
Limited and unclear policy-baseline comparison - The held-out policy experiment trains Broadcast, Singleton, and CASI, but not leave-one-out or Banzhaf, although both are evaluated as credit assignment methods. The paper does not explain how hyperparameters were selected or whether the baselines received comparable tuning, making the policy comparison difficult to assess. Moreover, the SFT initialization scores 60.32, above Broadcast (43.3) and Singleton (55) in Mean replay value. The decline after training for both baselines raises a concrete question about whether they were trained effectively, and thus how much of CASI’s advantage is due to its credit-assignment method.
Questions for authors
-
Section 3.5 compares methods by the number of evaluator scores, but does not report the actual effects on training. What is the training time and/or compute cost for CASI and each baseline? How do they differ?
-
See weaknesses.
Dear Reviewer qw6Y,
We hope you are doing well. We just wanted to briefly follow up and thank you again for your thoughtful feedback. We hope our responses and additional experimental results have helped clarify the points raised in your review.
If any part of our explanations or experimental details remains unclear, please feel free to leave a follow-up question. We would be very happy to provide further clarification.
Thank you again for your time and consideration!