- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies reinforcement learning under obedient recommendations, involving both a learner and strategic receivers. It highlights significant differences in learning dynamics and regret between public and private settings. However, the model relies on relatively strong assumptions and lacks clear practical applicability.
Critical strengths
The paper provides rigorous theoretical analysis covering multiple criteria, making its theoretical contributions relatively comprehensive and substantial.
Critical weaknesses
The paper relies heavily on the simple Bernoulli assumption. Can similar results be obtained for more general distributions?
More importantly, the paper lacks discussion of practical application scenarios. The setting seems rather artificial, with limited foreseeable contributions to real-world applications.
Questions for authors
In Line 85, "(?)" appears to be a compilation error.
For the upper bounds, can we obtain corresponding lower bounds, particularly regarding the dependence on the discount factor ?
Thank you for the questions. The distributional scope differs between the explicit separation and the general characterization; separating them also clarifies what our experiments demonstrate.
W1. Bernoulli assumptions and general distributions. The general learnability characterization does not require Bernoulli feedback. Theorem 7 (p. 8) works with measurable observable kernels and evaluable likelihoods, under its finite-model, compactness and exact-access assumptions. Proposition 16 (pp. 24–25) already exhibits iterative unlocking with Gaussian telemetry.
For the matched-oracle, adverse-selection separation, Bernoulli variables encode the minimal binary-state transition. The key mechanism is that visibility changes whether agents execute the actions reaching and activating an informative transition. The distribution determines the separation between observable laws and hence the testing cost. Replacing the transition family would require checking this separation together with incentives and oracle matching, especially when receivers value the unknown transition as in Theorem 2. Thus general-kernel closure and non-Bernoulli unlocking are established; the same matched-oracle separation for arbitrary distribution families is not claimed.
W2. Practical motivation. The relevant setting is one where a coordinator needs diagnostic actions from self-interested operators who know their equipment response better than the coordinator. For example, increasing a device's load can reveal its capacity but impose an unattractive effort cost. The coordinator can use information about job priority to recommend loads, yet uncertainty about capacity constrains which recommendations it can certify as obedient.
We added a synthetic capacity-dispatch experiment to Section 7 and Appendix F.2, with saturated service, Gaussian telemetry and the full binary direct-recommendation class. At horizon 10,000, noise standard deviation 0.3 and capacity 2, allowing access to expand after elimination gives mean regret 14.198 (pointwise 95% Monte Carlo interval [14.017, 14.379]), versus 4843.750 when access is frozen, over 2,000 independent trajectories per method with the same obedient-oracle benchmark. A negative control with identical initial feedback blocks unlocking despite direct-control identifiability. The comparison shows why updating access matters in this task. It is an application-motivated simulation of closure, not validation of a deployed system or the multi-receiver public/private separation.
We next address the two specific questions.
Q1. The unresolved citation at line 85. Thank you for identifying this error. The intended reference is Ahmadi, Chaudhry, Sindhwani and Tu, Safely Learning Dynamical Systems, first posted in 2023. It renders correctly, with its bibliography entry, in the current local compiled manuscript.
Q2. Matching lower bounds and the discount factor. Matching bounds in the horizon are already included: Theorems 1 and 2 establish linear public regret and logarithmic private regret for fixed instances. Appendix A.1, (PS5)–(PS6), shows that informative visits incur a positive opportunity cost and uses change of measure to lower-bound their required number. This matches the diagnostic-and-test upper bound in its dependence on T.
Sharp discount-factor dependence is a separate question. The separation fixes the receiver discount to one; neither its upper nor lower bound establishes a matching dependence on a varying discount. Such an extension must track how discounting changes deviation incentives, oracle values and the cost of information. The obedience margin gamma and the statistical failure budget are distinct from the receiver discount.