- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper presents Test-time best response (TTBR) which is considering the problem of cooperation with an unseen partner after a brief observation phase. The method idea is to use the observed actions for infering the class of the observed policy from a family and then fine tune the learned policy in simulation. The paper gives theorectical analysis and proofs regarding the usefulness of the individuation of the cooperating agent class as well as the bounds over the regrets based on the confidence uppon the cooperating agent class. The method has been tested on three Overcooked layouts and level-based foraginf showing substantial gains over the blueprint.
Critical strengths
- Relevance to the conference: the topic of the paper is relevant to the ICLR community.
- Theoretical analysis: The theoretical analysis proposes a solution that defines a conditional threshold, which incorporates both the conflict between responses and residual uncertainty; this sheds light on the extent to which - from a theoretical standpoint - it is necessary (or unnecessary) to identify the strategy class of a competing agent. This finding is supported by empirical evidence, which compares the TTBR method with a generalist approach.
- Technical soundness: I do not have found any specific technical issue, the paper looks technically sound.
Critical weaknesses
- Writing quality and paper organization: The paper is hard to read. No clear algorithm is defined and explained in the main paper, while a lot of material is just put into the appendix. The method section (sec 3) does not actually present the method but answers some questions providing a sort of theoretical analysis without having clearly defined the method before. This makes very hard to me understanding the actual method, without checking the appendix. Also the results section answers some questions but it does not clearly compare the performance of the proposed algorithm with that of a recognised algorithm in the literature.
- Novelty: The proposed pipeline combines established components: Bayesian partner-type inference, response selection or improvement conditioned on the inferred belief, and simulator-based test-time fine-tuning. The clearest original contribution appears to be the finite-observation regret analysis linking cross-best-response conflict to residual posterior odds. Since the conflict measure builds on cross-best-response regret used in the minimum-coverage-set literature (Rahman et al., 2024), the paper should clarify what is technically new in this analysis. It also does not compare against its acknowledged alternative of selecting among precomputed best responses using the same posterior. Given that MAP fine-tuning performs similarly to posterior fine-tuning on three of the four layouts, the experiments leave the added value of on-demand response computation unclear.
- Gap between the formal conflict measure and how the experiments interpret it: The main theoretical results - Proposition 1, Corollary 1, Lemma 1, Theorem 1, and the misspecification corollary - appear mathematically correct under their stated assumptions. However, the experiments estimate conflict using finite PPO fine-tunes rather than exact best responses, so the claimed relation h <=kappa is not guaranteed for those measurements; even with exact best responses, observing small h would not establish zero conflict. This weakens the claim that the control layouts are conflict-free, and the theorem’s guarantee also does not directly cover the deployed finite-PPO policy because its optimization error is unbounded.
- Limited performance increase: The comparison with published ad hoc teamwork baselines is difficult to assess from the main paper: the ICRL4AHT result is only briefly summarized in RQ5, while the detailed comparison appears in the appendix. Moreover, this comparison covers only one wide-layout benchmark and two published in-context baselines. TTBR scores +2.7, matching the generalist, and trails the search baseline (+17.0). These results provide limited evidence that TTBR outperforms strong ad hoc teamwork methods; a clearer main-text comparison across more settings and methods would strengthen this claim.
- Lack of a comparisons with baseline algorithms: I find that a key baseline is missing. The current approach involves defining a method that performs fine-tuning on a set of simulations, weighted according to the computed posterior probabilities - as opposed to fine-tuning solely on the simulation with the highest probability. The paper itself acknowledges a limitation: one could instead develop a distinct policy for each member of the family and dynamically select the appropriate policy - without further fine-tuning - based on the specific agent encountered at runtime. The absence of this baseline is a significant shortcoming.
- An acknowledged limitation is the use of agents that belong to the same family. The method’s performance drops when it encounters agents it has not seen before, becoming equivalent to a generalist baseline - which, unlike this method, does not require a set of trajectories from the concurrent agent (as is the case for TTBR), nor does it necessitate a fine-tuning phase that assumes access to a simulator. This is acknowledged but it seems to be an important weakness of the method.
- Scalability: The method’s scalability to large or continuous state and action spaces is unclear: all experiments use small discrete environments, and TTBR relies on a tractable partner hypothesis class with computable action likelihoods and simulator support.
- Reproducibility: the code seems not to be available, which hinders reproducibility.
- Missing related work section.
Questions for authors
- Which is the most similar algorithm to TTBR in the literature? What is the methodological original contribution that TTBR introduces w.r.t. that algorithm? How much does TTBR improves performance w.r.t. that algorithm? 2. Please clarify on the issues identified in the section "Critical Weaknesses".
I thank the authors for the explanations and answers. I'll take this into account in the discussion with other reviewers.