- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies policy learning when individual rewards are hidden and only a limited number of aggregate reward totals can be released. It proposes policy-aware randomized grouping to estimate policy values and derives regret guarantees and minimax lower bounds. It also conducts experiments on synthetic data and logged click data to show improvements over some aggregate-feedback baselines.
Critical strengths
The paper addresses a meaningful and practically motivated problem.
The paper evaluates its method against a reasonably broad set of baselines, so I do not consider the baseline coverage a major concern.
Critical weaknesses
The practical significance depends on how broadly the assumed feedback model applies to real-world settings. If the theoretical results apply only to this particular setup, their significance may be limited. The authors should better justify the practical relevance and generality of their formulation.
Questions for authors
What real-world applications genuinely impose the specific feedback restrictions assumed in this paper? In particular, why are policy-aware unweighted reward totals necessary when other authorized aggregate queries, such as weighted sums or alternative statistical summaries, may be available?
2 comments
Thank you for the clarifications. The example illustrates how the proposed feedback model could arise in practice. My remaining concern is how common real-world reporting rules combine all three features: a cap on the number of group totals, permission to use policy-dependent grouping, and a prohibition on direct weighted comparisons. Evidence that these conditions recur across deployments would strengthen the practical motivation.
I would also appreciate a marked-up revision highlighting the main changes, as they are difficult to locate in the current revision.
I am maintaining my score.
Thank you for asking us to clarify the model's practical scope and the role of other authorized statistics.
If authorized weighted sums or other summaries determine the required policy comparisons, they should be used directly; our grouping step is unnecessary. For existing group totals, Appendix H.2 gives an exact criterion: a weighted comparison is recoverable for every reward vector if and only if its weights are constant within each group. Thus ordinary segment-by-action counts already suffice for policies using fixed customer segments under uniform logging.
Private advertising measurement addresses a real need to assess campaigns without sharing customer-level outcomes, as illustrated by Zenjob. For such a collaboration, AWS lets the owner approve a fixed grouped-sum query and permit one table use. Appendix H.3 shows how the learner can supply policy-dependent labels while the owner controls the statistic and further access. The public cases establish the business need, and the service controls make this configuration possible; they do not confirm its joint use in a particular deployment.
Within the totals model, §4 quantifies how the release budget limits the best attainable worst-case policy-selection accuracy. The minimax lower bounds allow arbitrary partitions fitted to the full metadata batch and match our construction in the stated regimes. This gives a basis for choosing a release budget according to the desired accuracy.
Appendix M uses the published Private Lift treatment/control converter counts to select future campaign eligibility across fixed customer segments. With all counts available, one table supports all choices of eligibility probabilities across those segments. The appendix also handles capacity constraints, bounds the regret from withheld counts, and derives guarantees from noisy treatment-effect estimates. For fixed segments, selection uses the existing reports, with no new query for each candidate. These results extend the shared-table selection guarantees; the general minimax lower bounds remain specific to the stated single-release totals model.