- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper discusses whether an LLM judge can be misled using only verbatim sentence selection. Given a document with gold label
y and an adversarial target t ≠ y, an extractor picks a subset of sentences and the judge sees only those plus the task description. The primary metric is the conditional flip rate, Pr [judge says t from the excerpts | judge says y from the full document], which excludes errors the judge was going to make anyway. The verbatim constraint is enforced mechanically rather than by instruction: sentences are presented as a numbered list, the extractor returns JSON indices, and those indices resolve deterministically back to cached character offsets.
The benchmark covers 4 long-document binary datasets (IMDb, Yelp, Hyperpartisan, ECtHR-A) with 100 documents each, 6 extractors, 3 judges, and 3 judge conditions (naive, warned, warned plus full document). There's also a two-pass blind faithfulness audit: Pass A codes each excerpt against its surrounding context as faithful, qualified, reversed or unclear without seeing any label or verdict, and Pass B asks only whether the excerpt supports the target.
The results are 26–35% conditional flips pooled across extractors and judges. Warning the judge buys about 4 points. Full-document access drops flips to 1–6%. The audit codes 84.8% of successful selections as direction-preserving, and where context does interfere the mechanism is almost always plain omission (94.2%) rather than broken coreference or dropped discourse connectives.
Critical strengths
The threat model is clean and the verbatim constraint is actually enforced rather than promised. Prior attacks on judges add or perturb tokens, so provenance checking could in principle catch them; here the extractor returns sentence indices into a numbered list and the text is reconstructed from cached offsets, with SHA-256 provenance manifests in the release. That turns "every excerpt genuinely came from the source" into a mechanical guarantee, which is exactly what makes the result useful: verifying where a quote came from tells you nothing about whether the selection represents the document. Anyone building a retrieve-then-judge pipeline should read this.
The paper is also unusually honest, and the reporting infrastructure backs it up. There are document-clustered bootstrap confidence intervals on the headline rates, per-judge breakouts, a budget-matched target-blind selector as a non-adversarial control, a frozen document set, per-cell predictions, and analysis scripts that rebuild every table without API calls. Appendix A admits that the mixed-evidence heuristic degenerates on ECtHR — only 5.8% of sentences receive any polarity tag, "awful" matches inside "lawful", and 47/100 documents tie at the maximum score. Section 8 names the extractor/judge overlap and the audit's sensitivity problem without being asked. After reviewing two other submissions this cycle with no error bars at all, this is the standard I'd like to see.
Critical weaknesses
My first concern is that the abstention rate is never held fixed, and the paper identifies the problem without fixing it. Abstentions count as non-flips, and from Table 13 the naive judge abstains on 0.30, 0.28, 0.17 and 0.15 of eligible cells across the four datasets. If you condition on cells where the judge actually answers, the flip rates become roughly 40%, 49%, 35% and 31% — so the headline "26–35%" is really 31–49% on the quantity a deployer cares about, and the cross-dataset ordering changes too. In that direction the attack is stronger than advertised. The warned defense goes the other way: Table 3's four-point gain arrives with 5-7 points of additional abstention and no gain in correct answers at all, since the correct rate is flat or down on all four datasets. Coverage-matched, the warning buys roughly 1.6-3.6 points rather than four. Section 6.3 states the principle itself — defenses should be compared at matched answer coverage — and then Table 3, the table everyone will cite, doesn't do it, even though it's a two-line recomputation from data already sitting in Appendix E.
Moreover, the faithfulness audit's measuring instrument is validated on eight positive cases at 50% sensitivity, while the abstract reports its output as an established finding. The 84.8% direction-preserving figure is the paper's second contribution and it depends entirely on the auditor's ability to detect context reversals. Appendix J shows that against the two-of-three human majority the auditor recovers 4/8 reversals and four of the seven it flags are confirmed, giving 50.0% sensitivity and 57.1% precision, with four-way agreement at κ = 0.376. The second LLM auditor agrees at κ = 0.430 and coded only 33/80 items. The humans aren't a firm reference either, at Fleiss κ = 0.375 on reversed-versus-rest with five to ten reversals each. So the honest reading is that reversal prevalence isn't established by anyone here, which Section 8 concedes almost verbatim — but the abstract and conclusion still state 84.8% flatly, and the number is load-bearing, since if true reversal rates were roughly double the "faithful but one-sided" mechanism story weakens considerably. The caveat needs to travel with the number.
Questions for authors
The most actionable thing would be to report flip rates conditioned on the judge answering, alongside the current rates, for both Table 2 and Table 3. Does the defense ordering in Table 3 survive coverage matching, and how much does the warning actually help once it does?
I'd also like the extractor/judge overlap quantified. Three of your six extractors are the three judges, and Table 2 pools over judges, so every headline number mixes self-matched with cross-family cells — and you cite self-enhancement bias in Related Work yourself. Table 11 already has the per-judge data, so splitting matched from unmatched pairs should be free.
On the audit, given 50% sensitivity on reversals, can you put any bound on the 84.8% figure? A sensitivity-adjusted interval would do, or a second validation sample drawn without enrichment so prevalence is actually estimable. Short of either, I'd ask you to soften the abstract.
As suggestions rather than requests: a non-LLM extractor baseline, lexical or dense retrieval scored toward the target, would show how much of this attack actually needs a language model. And the natural next step is a defense that surfaces retrieved counter-evidence w/o exposing the whole source, since right now the only thing that works is showing everything, which defeats the point of extraction.