- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies a phenomena called “constraint echoing”, where LLMs will repeat instructions given in the prompt. They argue that echoing degrades task performance, and introduce a fine-tuning method to limit constraint echoing that relies on matched pairs with and without a negated instruction.
Critical strengths
This question of whether repeating instructions in the prompt is helpful to an LLM is an interesting one, and I think the proposed method of matched training on matched pairs to reduce the echoing effect is likewise interesting and if presented more clearly could be a helpful technique.
Critical weaknesses
The major weakness of this paper is that it is densely written and hard to read. The text reads as heavily AI-assisted, and there are many unhelpful phrases, jargon, etc. that make understanding the experiment details and contributions of the paper difficult. Along with this, the main paper contains very little detail about the actual data used for experiments with only a toy example in Section 3 and a minor explanation in Appendix F mentioning data is derived from WildChat conversations. I would recommend rejection based on the lack of clarity of the paper alone.
The related work is also fairly sparse, and citation practices could be improved. For example Appendix F mentions that they derive their data from WildChat, but Zhao et al.’s WildChat is not cited. Another paper that came to mind as I read that could be worth citing/comparing to is Huang et al. which identify a related problem where previous assistant replies could introduce bugs when a user updates their question after an assistant response.
Huang et al. Do LLMs Benefit From Their Own Words?
Zhao et al. WildChat: 1M ChatGPT Interaction Logs in the Wild
Questions for authors
Can you clarify some of the language choices in this paper? There is a lot of jargon that I think could be clarified, but I will point out a couple that might be a helpful starting point. What is the difference between a "request cluster" (line 196) and a "family" (line 201) for the single vs. multi-turn experiment setup? Is "joint endpoint" (line 182) the same as "joint utility" (lines 214, 321, etc.), and if so can you give some intuition for what this is trying to measure? It seems like if any one of the 4 conditions fails, this score will be 0 and so it's not clear how helpful it is.