- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the question of if agents that are text-optimised independently can still cooperate with each other. The paper uses a set of coordination toy examples to isolate the problem and show a coordination failure mode of prior work on text optimisation. Using these examples the authors show that agents have have been text-optimised independently can fail to coordinate with each other due to converging on incompatible conventions or behaviour. Several methods to mitigate this coordination failure are proposed and tested and are shown to be effective in the tested examples. The coordination failure was not clearly observed in the overcooked environment.
Critical strengths
- The paper shows how text optimisation can improve an agent's score while making it less compatible with independently optimised partners. It is shown that agents have a common failure mode where they will fallback to the first alternative in their own list, which can lead to incompatible conventions between agents which have different list orderings.
Critical weaknesses
- The writing and structure of the paper is poor and confusing. Figure 1 is not understandable without reading section 5, which is 3 pages later. All the tasks are not clearly separated and the narrative continuously jumps between them without clear distinction. Many terms are not defined, and much of the authors intent is vague and unclear. The different game definitions is very fragmented, Eiffel Tower has a section, Overcooked and Signal Relay share a section after the experiments are presented, Hanabi is only described in the appendix in a section that is not referenced in the main text. Also "the tower game" is mentioned and it is unclear if this is the same as the Eiffel Tower game or a different game.
- The practical importance of the findings is not sufficiently established. The main failures occur in small games designed around incompatible conventions, while the larger Overcooked experiment improves both self-play and cross-play. Some fixes also rely on knowing the game's symmetries or rewarding a known compatible solution. It remains unclear how often this problem arises in realistic text optimisation settings and how useful the proposed fixes would be there.
Questions for authors
- What practical settings are most likely to exhibit this failure? The Overcooked experiment does not reproduce it, so it would help to clarify when independent text optimisation is expected to reduce coordination outside the small games.