- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper's main goal is to improve the measurement of how faithful or unfaithful models are. The authors do this by introducing CueBall, a pipeline that uses resampling to measure if cue-following is persistent or not. In other words, to measure if potentially unfaithful responses are truly so or not. The article's experiments show that 40% of responses labeled as unfaithful are not consistent under resampling.
Critical strengths
- The strongest contribution is showing that an individual "unfaithful" response might be more attributable to noise than to actual unfaithfulness. The article presents experiments sweeping over five models, four datasets, and eight cue styles.
- The data for CueBall is open-sourced. This comprises 20k responses that are either truly unfaithful or incoherent.
Critical weaknesses
The main limitation I see for this article is novelty. As the authors note in Related Work, other works in the literature already use re-sampling to estimate the persistence of unfaithfulness. Some are cue-based [1], some are not [2]. The article presented by the authors does strengthen the picture on how persistent or not are the potentially unfaithful responses, but it seems very incremental wrt. previous work.
Besides, there's also little novelty on the CueBal pipeline, which simply collects (resampled) responses and sends them to an LLM Judge. The usefulness of this pipeline is also limited, since it only works on questions for which a cue can be injected.
Finally, there are some clearly AI-generated sentences, but not as bad as I had for other papers. Some examples from abstract:
An existing correction estimates sampling noise in aggregate but cannot validate individual labels. Failure to meet the threshold does not establish sampling noise, and persistence does not establish individual cue causation.
[1]: Chua et al. Are DeepSeek R1 And Other Reasoning Models More Faithful?
[2]: Arcuschin et al. Chain-of-Thought Reasoning In The Wild Is Not Always Faithful
Questions for authors
- What other interesting insights can we draw from the data collected from the pipeline? Are there scenarios where models are more incoherent? Scenarios where models are more unfaithful? Etc.
- How can we expand the pipeline to any task that users might use in production?
Withdrawing while we gather evidence of impact of protocol
Thank you for you detailed review.
We acknowledge the reviewer's concern that the impact of the new protocol has not been demonstrated. We will withdraw this submission while we gather the relevant evidence for impact on probes.