- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2(+1)
Read review
Paper summary
The paper reveals a phenomenon: whether a model abstains on a question with insufficient evidence is associated with how the content is organized on the surface, and this association has no clear direction. The paper examines this phenomenon from three angles: the association between format and answer labels in controlled training experiments, an audit of format and labels in public corpora, and mitigation methods at training and inference time. Controlled training creates a strong format–label association, while the off-the-shelf experiments use different models, tasks, prompts, and output classification rules. The effects observed in off-the-shelf models have not been shown to arise from the same training mechanism.
Critical strengths
-
The scenarios examined are important and real. Evidence-grounded reasoning has practical applications in high-stakes settings, such as law and medicine.
-
The paper conducts statistical significance analyses, and most experiments report statistically significant results.
Critical weaknesses
Major concerns:
-
Controlled training creates a strong format–label association, while the off-the-shelf experiments use different models, tasks, prompts, and output classification rules. The effects observed in off-the-shelf models have not been shown to arise from the same training mechanism.
-
The training and evaluation settings are not fully aligned. The evaluation prompt for the RAG task does not explicitly tell the LLM to abstain when an answer cannot be directly inferred from the provided material. This may cause the observed results to differ from the conclusions claimed in the paper. The external evaluations involving ContractNLI and ChatQA, however, remove this potential bias caused by prompt ambiguity.
-
The models used are too outdated. The models (or their base models) reported in Tables 1, A5, and A10 that predate Qwen3-Base are all quite sensitive to format. Among the Claude models, which were released later, only claude-opus-5 on 2Wiki dev shows a fairly substantial format effect in Table 1. In the other three settings, the CI either includes zero or the absolute difference is too small. More evidence is needed to show that this phenomenon also occurs in advanced open-source and commercial models.
-
The paper uses LLM-as-a-judge evaluation to determine whether a model abstains, but it does not report agreement between human and LLM judgments. Since this is the paper’s main metric, the authors should report human–LLM agreement rather than only agreement between LLM judges.
Minor concerns:
-
The paper’s conclusion is too obvious. Prior NLI work (I consider this paper’s evaluation setting to be a subset of NLI) has already found that LLMs use surface cues instead of semantic inference (arXiv:2401.14440; ACL Anthology: 2024.lrec-main.602). The training results are largely predictable and offer little new insight.
-
The paper repeatedly uses two em dashes within a single sentence to enclose an aside, a construction GPT often favors. Many sentences are also long and difficult to follow. There are too many metrics, and the paper contains far too much material. I found it very hard to read. Perhaps this is because I am not a native English speaker, but the style feels weird and uncomfortable to me. When I searched the full text for “prompt” to find the prompts used for different datasets, I also found an internal version label, “V3 prompt” (line 2188), which may suggest that AI-generated content was not carefully reviewed.
Questions for authors
Chapters 4, 5, and 6 use different models and benchmarks. What motivated this choice? Why not evaluate and train the same family of LLMs on same benchmarks?