- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
This paper aims to understand the mechanisms behind randomized uncertain social preferences (RUSP), introduced by Baker, 2020. The authors define “the ambiguity zone” as the set of social preference values () for which the value of social information is positive; that is, the values of for which knowing the partner agent’s type increases expected return. They argue that partner-sensitivity in RUSP emerges when 1) there is adequate training coverage of the ambiguity zone and 2) the partner type is identifiable from its action history (which is upper-bounded by the Bayes-optimal identifiability ). Through experiments in an iterated two-action prisoner’s dilemma setting, they demonstrate that partner sensitivity increases with the amount of training mass placed within the ambiguity zone (when partner identifiability is held fixed with deterministic, scripted partners) (Fig. 2), and that partner sensitivity increases with increasing partner identifiability when within the ambiguity zone “critical band”. They also show that behaviors resembling reputation and memory arise from agents observing partner interactions with a third party, and finally demonstrate in the RUSP-bridge that indirect reputation is only nonzero when both the coverage and identifiability conditions are satisfied.
Critical strengths
Rigorous experimentation. The experiments fairly rigorously establish the finding that both gates of coverage and identifiability are required for emergent partner sensitivity. The authors first assess trust sensitivity in response to increasing critical mass for a fixed interaction structure. They then establish that coverage of the game-specific critical zone, rather than certain universal values of , is what matters for increased sensitivity by evaluating across games with different payoff matrices (and different ambiguity zone bands). The authors then show that their findings have predictive power by assessing a preregistered fourth game. The authors also demonstrate the interaction between coverage and identifiability (Table 1), and verify via the matched-marginal identifiability control that this isn’t due to changes in the marginal cooperation rate of the partner population.
Critical weaknesses
Organization and information flow. I personally found the paper quite dense and hard to parse; key findings and results were not very cleanly connected to the paper’s main claims and takeaways. Most of the figures in the main text (Figs. 1, 3, 4, 6) are not explicitly referenced in the body of the paper, leaving the reader to infer which findings in the Results section are presented in the figures. The similar section and figure titles helped, but the sections often discussed more results than presented in the figure, making the results unnecessarily hard-to-follow without explicit references to the Figure. (As an example, line 188 says “Figure 2 illustrates two consequences…”; however, Figure 2 appears to only illustrate the latter consequence referred to in this paragraph, the one against a {TFT, All-D} population.) Further, Figure 1’s role in the paper was quite unclear to me; it appears to be connected to experiments described in Appendix F rather than supporting the paper’s main result of the two-gate mechanism behind RUSP. Displaying it in its current prominent position distracts from the primary contributions of the paper. Certain abbreviations and notation were also introduced without explanation; an example is , and in Figure 4. Briefly defining the partner types/strategies (All-D, Grim, etc.) would also be nice for readers who are less familiar with this specific area.
Tie to overall significance / more general problem outside of RUSP. As it is currently written, the paper reads as an in-depth study of certain mechanisms underlying RUSP (Baker, 2020). The motivation for conducting this study of RUSP is less clear to the reader; including some elaboration on this and the significance of RUSP would make the paper more self-contained. Additional discussion about how the paper’s findings might generalize beyond RUSP would also make the paper more interesting to a broader audience. Based on what the paper presents, it’s unclear if the results do generalize to multi-agent RL settings outside of RUSP, which limits the value a broader audience might find in this paper.
Questions for authors
In the section on the RUSP-form bridge (Section 4.4 and Figure 7), it’s somewhat surprising to me that the “original style” condition has zero . For one, its method of broadly sampling from resembles the approach of RUSP. Also, both the coverage and identifiability gates are open in this condition, which the paper predicts should induce some sort of partner sensitivity effect. I recognize that is a more demanding test, and would thus be curious if in this same condition is nonzero. If it were, it would strengthen the main claims regarding the two-gate account.