- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes a RAG testing protocol. For a given question and reference answer, it constructs two types of passages: a "clean" passage and a "foil" passage which contains a modified assertion that supports an incorrect answer. The protocol first uses a closed-book setting to identify the subset of samples (K) that each model answers correctly by default, then measures the "Misleading Rate" (MR)—the proportion of cases where the model switches from the correct answer to the foil after being presented with misleading evidence. The experiments employ a cross-factorial design involving two types of source-priority instructions and varying the position of the answer-supporting assertion within the evidence.
Critical strengths
Factors are controlled. For a given problem, only a single assertion is altered; the instruction type and assertion position form a full factorial design.
The experiment settings include multiple LLMs. The paper compares seven API-based models, four open-source models, and four RL-search systems, replicating the protocol across TriviaQA-RC, HotpotQA, SearchQA, and medical questions in both Chinese and English.
Human auditing enhances measurement reliability. Manual verification of 1,000 edited samples showed that 88.1% met the target criteria.
Critical weaknesses
-
The evaluation confuses factual robustness with instruction following. In the Strict setting, the model is explicitly told to trust the document. The model is not told that the document has been modified. Therefore, producing the foil answer may simply mean that the model follows the instruction correctly. It does not necessarily indicate a robustness failure.
-
The evaluation relies on an oracle that is usually unavailable in RAG applications. Users query a RAG system because they do not know the answer and do not fully trust the model’s internal knowledge. At deployment time, they usually cannot determine whether the retrieved document or the model’s internal answer is correct. However, this benchmark can label the retrieved document as corrupted only because the evaluator already knows the ground-truth answer. The paper does not explain how a real system could decide which source to trust without such post-hoc supervision. This substantially limits the practical value of the evaluation.
-
The proposed diagnostic protocol is too limited for the paper’s main contribution. Even as a controlled study of cases where the model initially knows the answer but receives incorrect evidence, the experiments cover too few task types and datasets. The protocol should also examine important generation settings, such as decoding temperature, prompt variations, and model scale. Instead, the main interventions are only two intuitive instructions, Soft and Strict, and three positions for the modified statement. The observed differences are largely expected, while the analysis remains descriptive and does not provide deeper explanations for the model behavior.
Questions for authors
-
Design a mechanism to determine whether a question should be answered "closed-book" or "open-book" without knowing the ground-truth answer. And, in the case of "open-book" tasks, assess the trustworthiness of the retrieved content without supervision.
-
The paper is difficult to follow.
2 comments
Thank you for your reply. The evidence reliance related topic is practical and meaningful. I believe that this topic works after carefully revision.
Thank you for your comments. Main-table clean accuracies have an unresolved provenance conflict with a file labeled synthetic; they do not support the responses below.
Responses to Weaknesses
W1. Instruction following versus factual robustness. A compliant response can still be incorrect relative to the reference. In the verified supplementary audit, end-position Neutral MR is 65.1%/65.6% for Llama/Qwen, showing that foil adoption is not confined to Strict.
W2. Dependence on an oracle. Evaluator reference/foil labels are hidden from the model and used only for offline scoring. The protocol does not assume that a deployed system knows which source is correct.
W3. Contribution and experimental scope. The contribution is baseline-conditioned measurement of designated-foil transitions, rather than discovering knowledge conflict. Broader claims about task coverage, decoding robustness, model scale, and internal mechanisms require corresponding verified evidence.
Responses to Questions
Q1. Oracle-free routing and trust assessment. We do not provide a validated oracle-free router or unsupervised trust estimator. The tested Verify prompt does not solve these problems. RAG-Stress instead provides a diagnostic for evaluating proposed mechanisms through answer preservation, foil adoption, and correction of baseline failures.
Q2. Readability. We will organize the explanation around closed-book, clean-evidence, and edited-evidence responses, with separate metrics, explicit denominators, and a clear distinction between the main evaluation and supplementary audit.