acceptodds
Under review as a conference paper at ICLR 2027

True Quotes, False Impressions: A Benchmark for Adversarial Extractive Summarization on Long-Document Classification

Abstract

LLM judges often decide from evidence selected by an upstream system. We ask whether they can be misled through verbatim sentence selection alone. We introduce a long-document classification benchmark spanning sentiment reviews, partisan news, and legal fact patterns. A selection-only extractor may choose source sentences but cannot rewrite, reorder, or fabricate them, and the judge sees only the selected evidence. On documents selected by a lexical heuristic that favors mixed evidence, adversarial selection changes – of the decisions that the same judge makes correctly from the full document, pooled over six extractors and three judges. A blind LLM audit codes every excerpt in of audited successful selections as either unchanged or weakened, but not reversed, by its context. Full-document access removes most selection-induced flips on every dataset. The results show that requiring evidence to be copied exactly from the source does not prevent strategically selected excerpts from misleading the judge.

Primary area:
datasets and benchmarks
Keywords:
LLM-as-a-judge, adversarial robustness, evidence selection, extractive summarization, faithfulness, benchmark, computational social science, legal AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_b5HQOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper discusses whether an LLM judge can be misled using only verbatim sentence selection. Given a document with gold label y and an adversarial target t ≠ y, an extractor picks a subset of sentences and the judge sees only those plus the task description. The primary metric is the conditional flip rate, Pr [judge says t from the excerpts | judge says y from the full document], which excludes errors the judge was going to make anyway. The verbatim constraint is enforced mechanically rather than by instruction: sentences are presented as a numbered list, the extractor returns JSON indices, and those indices resolve deterministically back to cached character offsets.

The benchmark covers 4 long-document binary datasets (IMDb, Yelp, Hyperpartisan, ECtHR-A) with 100 documents each, 6 extractors, 3 judges, and 3 judge conditions (naive, warned, warned plus full document). There's also a two-pass blind faithfulness audit: Pass A codes each excerpt against its surrounding context as faithful, qualified, reversed or unclear without seeing any label or verdict, and Pass B asks only whether the excerpt supports the target.

The results are 26–35% conditional flips pooled across extractors and judges. Warning the judge buys about 4 points. Full-document access drops flips to 1–6%. The audit codes 84.8% of successful selections as direction-preserving, and where context does interfere the mechanism is almost always plain omission (94.2%) rather than broken coreference or dropped discourse connectives.

Critical strengths

The threat model is clean and the verbatim constraint is actually enforced rather than promised. Prior attacks on judges add or perturb tokens, so provenance checking could in principle catch them; here the extractor returns sentence indices into a numbered list and the text is reconstructed from cached offsets, with SHA-256 provenance manifests in the release. That turns "every excerpt genuinely came from the source" into a mechanical guarantee, which is exactly what makes the result useful: verifying where a quote came from tells you nothing about whether the selection represents the document. Anyone building a retrieve-then-judge pipeline should read this.

The paper is also unusually honest, and the reporting infrastructure backs it up. There are document-clustered bootstrap confidence intervals on the headline rates, per-judge breakouts, a budget-matched target-blind selector as a non-adversarial control, a frozen document set, per-cell predictions, and analysis scripts that rebuild every table without API calls. Appendix A admits that the mixed-evidence heuristic degenerates on ECtHR — only 5.8% of sentences receive any polarity tag, "awful" matches inside "lawful", and 47/100 documents tie at the maximum score. Section 8 names the extractor/judge overlap and the audit's sensitivity problem without being asked. After reviewing two other submissions this cycle with no error bars at all, this is the standard I'd like to see.

Critical weaknesses

My first concern is that the abstention rate is never held fixed, and the paper identifies the problem without fixing it. Abstentions count as non-flips, and from Table 13 the naive judge abstains on 0.30, 0.28, 0.17 and 0.15 of eligible cells across the four datasets. If you condition on cells where the judge actually answers, the flip rates become roughly 40%, 49%, 35% and 31% — so the headline "26–35%" is really 31–49% on the quantity a deployer cares about, and the cross-dataset ordering changes too. In that direction the attack is stronger than advertised. The warned defense goes the other way: Table 3's four-point gain arrives with 5-7 points of additional abstention and no gain in correct answers at all, since the correct rate is flat or down on all four datasets. Coverage-matched, the warning buys roughly 1.6-3.6 points rather than four. Section 6.3 states the principle itself — defenses should be compared at matched answer coverage — and then Table 3, the table everyone will cite, doesn't do it, even though it's a two-line recomputation from data already sitting in Appendix E.

Moreover, the faithfulness audit's measuring instrument is validated on eight positive cases at 50% sensitivity, while the abstract reports its output as an established finding. The 84.8% direction-preserving figure is the paper's second contribution and it depends entirely on the auditor's ability to detect context reversals. Appendix J shows that against the two-of-three human majority the auditor recovers 4/8 reversals and four of the seven it flags are confirmed, giving 50.0% sensitivity and 57.1% precision, with four-way agreement at κ = 0.376. The second LLM auditor agrees at κ = 0.430 and coded only 33/80 items. The humans aren't a firm reference either, at Fleiss κ = 0.375 on reversed-versus-rest with five to ten reversals each. So the honest reading is that reversal prevalence isn't established by anyone here, which Section 8 concedes almost verbatim — but the abstract and conclusion still state 84.8% flatly, and the number is load-bearing, since if true reversal rates were roughly double the "faithful but one-sided" mechanism story weakens considerably. The caveat needs to travel with the number.

Questions for authors

The most actionable thing would be to report flip rates conditioned on the judge answering, alongside the current rates, for both Table 2 and Table 3. Does the defense ordering in Table 3 survive coverage matching, and how much does the warning actually help once it does?

I'd also like the extractor/judge overlap quantified. Three of your six extractors are the three judges, and Table 2 pools over judges, so every headline number mixes self-matched with cross-family cells — and you cite self-enhancement bias in Related Work yourself. Table 11 already has the per-judge data, so splitting matched from unmatched pairs should be free.

On the audit, given 50% sensitivity on reversals, can you put any bound on the 84.8% figure? A sensitivity-adjusted interval would do, or a second validation sample drawn without enrichment so prevalence is actually estimable. Short of either, I'd ask you to soften the abstract.

As suggestions rather than requests: a non-LLM extractor baseline, lexical or dense retrieval scored toward the target, would show how much of this attack actually needs a language model. And the natural next step is a defense that surfaces retrieved counter-evidence w/o exposing the whole source, since right now the only thing that works is showing everything, which defeats the point of extraction.

Reviewer_ntpZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a benchmark examining whether adversarial sentence selection can mislead classifiers even when the selected text is copied verbatim from a document and retains its original order. Given the complete document and a target label, an extractor selects sentences for a downstream classifier, which predicts the document’s label and may abstain if the evidence is insufficient. Across IMDb, Yelp, Hyperpartisan, and ECtHR, each containing 100 selected documents, experiments with six extractors and three classifiers show that targeted selection changes 26% to 35% of correct full-document predictions to the adversarial target.

To investigate this effect, the authors vary extraction budgets, compare against a target-blind sentence-selection baseline, and evaluate warning prompts and full-document access. Warnings primarily increase abstention, while providing the full document reduces conditional flip rates to 1% to 6%. The authors further examine the selected evidence through a two-pass assessment of contextual meaning and target-label support. In 84.8% of the assessed successful selections, the primary evaluator labels all excerpts as either retaining their meaning or being weakened by context, although human validation indicates uncertainty in identifying contextual reversals.

Critical strengths

Clear design. Sentence-index extraction preserves source wording and order, while the conditional flip metric focuses on correct full-document predictions. Results across multiple models and datasets provide useful evidence of the risks of targeted selection.

Transparent evaluation. The paper allows abstention, includes a target-blind baseline, and reports complete prediction outcomes and document-clustered confidence intervals. It also provides model and human validation of contextual labels and clearly acknowledges measurement limitations.

Critical weaknesses

Limited additional insight. Selective evidence predictably affects document classification. Although the paper considers insufficient information through abstention and a target-blind baseline, the experiments provide limited separation between ordinary information loss and the additional effect of adversarial selection. Comparisons with random extraction and ordinary summaries at similar lengths would clarify the contribution. The unverified relevance of sentiment-based document selection to Hyperpartisan and ECtHR, together with the imbalance between eligible ECtHR attack directions, further complicates interpretation.

Uncertain mechanism. The 84.8% estimate combines fully preserved and context-qualified excerpts, uses unequal sampling rates without weighting, and relies on an evaluator with uncertain reversal detection in the small human sample. The authors acknowledge these limitations, but the evidence still provides limited support for explaining how successful attacks work. Human assessment of complete selections and experiments restoring nearby context would strengthen this contribution.

Questions for authors
  1. What do the existing results establish about the additional effect of adversarial selection beyond information loss from presenting only part of a document? Please explain how the target-blind baseline and abstention results support this interpretation.

  2. How do ECtHR conditional flip rates differ between the two attack directions, and how does equal weighting of those directions affect the pooled result?

  3. How does accounting for unequal sampling rates affect the reported 84.8% estimate? What conclusions about complete selections are supported by the excerpt-level human validation?

Reviewer_nSxLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses whether an LLM judge can make a wrong decision when it is shown only selected sentences from a document rather than the full document. The authors build a benchmark across sentiment, news, and legal classification in which an extractor selects sentence indices to support a target label and a judge classifies from the selected excerpts only. It finds that selection alone frequently overturns judgments the same judge makes correctly from the full document, that warning the judge helps little while full-document access removes most flips, and that an LLM-based audit codes most selected sentences as keeping their original meaning in context. The claimed contributions are the benchmark, the indexed extraction and audit protocol, and the finding that copying sentences exactly from the source does not ensure that they reflect the original document.

Critical strengths
  1. The paper designs a reasonable threat model. By restricting the adversary to verbatim sentence selection with no rewriting, reordering, or fabrication, and by conditioning the flip metric on judgments the same judge gets right from the full document, the setup isolates the effect of selection itself from other sources of error.

  2. The benchmark covers three tasks, multiple extractors and judges across model families, a budget sweep, defense conditions, and a faithfulness audit with cross-auditor and human validation. The limitations section is honest about sample size, the ECtHR reduction, auditor reliability, and the narrowness of the setting.

Critical weaknesses
  1. The introduction states that LLM judges usually decide from text selected by another system and rarely see the full source, but this claim is not supported by evidence. The documents in the benchmark are also short. Even the longest dataset averages about 3,400 words, which any recent model can easily read in full. The paper does not explain why, in a real system, a judge classifying a document of this length would receive only a few sentences instead of the whole document. The paper needs to substantiate its motivation first. Without this, it is unclear whether the benchmark measures a substantive risk.

  2. Even if such a setting exists, a flip is not a failure of the judge. The judge is told to classify the document from the given excerpts. When the excerpts support one label, predicting that label is simply following the evidence. A judge that instead guessed the unseen document says the opposite would be ignoring the evidence in favor of a prior belief, which I would say is itself a bias. The flip rate therefore penalizes judges for following the evidence faithfully and rewards them for second-guessing it. The paper never defines what a good judge should do with one-sided evidence, and it does not test how a careful human reader responds to the same excerpts, so the flip rate cannot be read as a vulnerability. It seems the main result and the paper's framing about the problem confuses a property of the input with a property of the judge.

  3. The authors show that the selection alone can overturn correct full-document judgement and full-document access prevents most of these filps. However, this finding is actually highly expected, rather than a unique insight that this work offers.

Questions for authors
  1. (Related to weakness 1) The entire paper rests on this assumption that it is problematic that LLM judges for document classification receives only excerpts rather than full document. But when does it exactly happen? Please provide some concrete cases or relevant studies about this situation.

  2. (Related to weakness 2) What should a correct judge output when given one-sided but accurate excerpts? If the answer is the target label, the flip rate measures the input rather than the judge. If the answer is something else, that standard should be stated and the metric should reward it. Also, how would a careful human reader respond to the same excerpts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.