- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates if the findings identified from the research on human teams' composition, interaction, and performance can be reused also to understand and govern the collective behaviors of groups of language models. The authors found that affect propagates within groups of language models, and it does in proportion to the applied intensity. Interestingly, the resulting shift is absent from the textual content the recipients produce and thus it's invisible to a judge that verifies. Again, steered language models does not improve the group accuracy on a significant amount of benchmarks, while every emotionally steered group proved more willing to adopt a wrong answer when it was asserted in a confident way.
Critical strengths
-
The idea of studying if and how findings characterizing human groups transfer to the collective behavior of language models is of of great interest and of paramount relevance given the growth in the adoption of multi-agent systems in real-world tasks.
-
The authors place their research hypotheses on several works and findings in group and social psychology, although sometimes they accept some claims from these fields as evident while are more controversial. For example, not always the emotions spread following a contagion effect. Some studies, indeed, have shown that emotions can spread following a mimicry strategy vs. a repulsion strategy: individuals can thus in some situations adapt to the emotions of their peers and in other situations they can react in the opposite way.
-
Some findings are interesting, for example the fact that affect seems to propagate covertly and that emotional steering seems to not increase accuracy in solving downstream tasks.
Critical weaknesses
-
Several findings in group psychology literature are taken for granted and treated more as axioms than as hypotheses or previous findings with some supporting evidence (see the example cited above of contagion vs. repulsion effects).
-
In pages 4 and 5 the formalizations of these group psychology findings are often not well defined. A lot of variables in the mathematical formulation are left completely undefined. This makes difficult to properly understand and evaluate these formalizations. See for example Equation 1: it seems just taken from opinion dynamics literature without any explanation (or adaptation) of the terms.
-
Several parts of the paper are written with a difficult to understand writing style. In particular, several paragraphs in the Introduction and in other parts of the paper are very long, unclear and require a rephrasing (see for example, the paragraph starting with "Because all 25 dimensions ..." on page 2, and the one starting with "Neither result translates ..." again on page 2).
Questions for authors
A couple of suggestions are to:
- improve mathematical presentation;
- simplifying and clarifying significantly the writing;
- evaluating the transcripts not just using LLMs as judges but also human evaluators.