- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper's main claim is that LLMs revise their answer when challenged irrespective of their confidence in the initial answer. The authors propose new tests to study how models use confidence when being challenged (perfect-note test and a wrong-for-wrong test). The authors propose a LORA fine-tuning setup to encourage the model to use confidence when challenged. Experiments with models from two different families and questions from three sources are included.
Critical strengths
- The paper addresses an important topic: LLM confidence and how it drives behaviour under challenge.
- The Wrong-for-Wrong Test to disentangle self-doubt from active option comparison is interesting.
Critical weaknesses
The paper is difficult to read, probably heavily written with an LLM. Eg. Fig 1 is very hard to parse for me.
The methodology has significant flaws:
- The confidence note inserted in the model’s turn is basically ignored by the models, but the authors consider this a causal intervention that proves the models don’t take confidence into account when deciding to revise. A proper causal intervention would have been to change confidence internally and see the effects.
- The training with LORA so the model takes into account the confidence related message leads to non-negligeable decrease in overall model performance. This doesn’t seem like a useful course of action.
- The experimental protocol seems to change across sections, making it hard to understand the results; in some parts a binary task is used, then for MMLU, a ten-options with open answer; then an “earlier design” with modified text prompts
- Multiple pre-registered predictions were not met or only partially met (Table 11), which points to weaknesses in initial hypotheses.
Questions for authors
See Weaknesses above.