- Rating
- 1
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Judging by Appendix C.4 it is something about measuring agreement in LLMs and how its initial opinion is overwritten by other participant's arguments, on a dataset of 12 debate statements. What the paper seems to be concerned with is that each other participant's argument does not change the opinion in isolation but that there are interaction effects that need to be taken into account when modeling.
Critical strengths
None.
Critical weaknesses
After 1.5 hours of trying, I did not understand the paper. I signaled this to the AC/SAC and was asked to continue reviewing. The paper in its entirety is unreadable. I invite anyone to read any paragraph to convince themselves.
- Terms are used that are never defined
- Sentences have no logical structure
- The method section and its math lead nowhere
- "Algorithm 1" is vague, full of undefined terms, defying the purpose of formalizing an algorithm
- To cite the first sentence of the conclusion: "Evaluate response coefficients against their intended predictions: fix the display and readout, reserve combinations, propagate uncertainty, and report input-family and aggregate losses.".
Frankly, it reads like AI generated, but as long as ICLR has no clause for rejecting due to AI slop, I propose rejecting due to subpar presentation quality.
The appended codebase is also not understandable. Some 37 iterations of code in subfolders.
Even all this left aside, a dataset of 12 scenarios is just not enough.
Questions for authors
None.