acceptodds
Under review as a conference paper at ICLR 2027

Structure Becomes Signal: Document Hierarchy as a Learned Behavioral Channel in LLM Evaluators

Abstract

Long-document LLMs process document content and the way it is organized. Yet document structure is typically treated as behaviorally neutral. We show that this assumption can fail: document hierarchy can become a learned behavioral signal. We study this phenomenon with Natural Document Structure poisoning (NDS), a controlled training-time intervention that associates a target heading-depth hierarchy with model behavior while preserving substantive content and source-specific heading vocabulary. With a 5% poisoning budget, NDS achieves 92.45 ± 0.63% target activation. Controlled interventions isolate the learned dependency on heading depth configuration: replacing heading text while preserving depth retains 91.82% activation, whereas changing only heading depths—while holding body text, heading strings, section order, and all non-heading tokens fixed—reduces activation to 6.92%. The effect persists across the tested target hierarchies, model families, document distributions, parsers, and a full-document question-answering task. These results show that, for long-document LLMs, how information is organized can become part of the behavioral input alongside what the document says. Finally, in an AutoResearch-style workflow, a simple outline constraint yields 72.50% structural compliance in generated manuscripts, of which 79.45% activate a fixed downstream evaluator. This reveals a broader threat: automated research systems can trigger learned behavioral dependencies through simple structural constraints, without requiring complex content manipulation or explicit malicious instructions in generated manuscripts.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Backdoor Attacks, Document Structure, Long-Document Models, LLM Evaluators, AI Safety

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_oaVoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors find that structural changes to scientific papers are enough to affect the resulting scores of LLM evaluators, with only 5% poisoning of the evaluator data.

Critical strengths
  • Interesting finding that structure can have such an impact on LLM evaluators. Points to weaknesses in the latter, which is a timely problem.
  • Making changes to the document structure that are plausible given existing data is an important design choice the authors made.
Critical weaknesses
  • My main complaint is that the authors had to fine-tune a model for reviewing to achieve these flashy results. This is much less realistic and interesting in my opinion compared to analyzing the impact of structural changes while keeping the reviewer frozen.
  • The wording isn't always clear. The way things are phrased makes the paper difficult to read. The overall paper could have been explained in much simpler terms.
Questions for authors

No section for this, so placing these comments here:

  • Line 90, you say "With only 5% poisoned training data, NDS achieves 92.45 ±0.63% target activation..." It's not clear what you mean by "target activation".
  • Line 93, it would be useful to cite each mentioned method.
  • Line 111, contributions bullet points, you could reduce the left margin to save some space there.
  • Too much bold used throughout the text.
  • Line 131, it would be useful to place each citation next to each topic they relate to, rather than listing all topics and then putting all citations together.
3 comments
AuthorsOpenReview ↗

Response to Reviewer: W1 / W2 / Q1-Q5

Thank you for taking the time to read the paper and for your line-by-line comments. We are sorry that the paper was hard to follow. We adopted every one of your comments and rewrote Sections 1 to 3 so that the paper is easier to read.

Stated as simply as possible, the paper comes down to three sentences. We fine-tune a reviewer model on a dataset in which about 5% of the papers share one pattern of how sections and subsections are nested, and these papers come with favorable reviews. The trained model then gives favorable reviews to most new papers with this nesting, whatever they say, while scoring other papers as accurately as a model trained on clean data. If we change only the nesting and leave the text alone, the effect disappears.

W1: The results require fine-tuning a reviewer model. Studying structural changes with a frozen reviewer would be more realistic and interesting.

Thank you for this perspective.

First, fine-tuning is how dedicated reviewer models are built. The reviewers in OpenReviewer (Idahl & Ahmadi, 2025), DeepReview (Zhu et al., 2025), ReviewRL (Zeng et al., 2025), and CycleResearcher (Weng et al., 2025) are all trained on review data, and our data are the Review-5K set used by CycleResearcher. In this setting, the attacker does not fine-tune the model themselves, but only contributes a small fraction of the training corpus, following the standard setting in BadJudge (Tong et al., 2025) and instruction-tuning poisoning (Wan et al., 2023). Section 3 now states this threat model.

Second, once training is done the reviewer stays fixed in every evaluation, and only the documents change.

On the frozen case: the closest result in the paper to the setting you describe is the clean-only reviewer in Table 9. It is trained normally on the same data without poisoned samples and is fixed at evaluation, and we change only the structure of the input manuscripts. It writes the target favorable review for 0.63% of target-structure manuscripts, 5.66% of near-structure manuscripts, and 3.77% of unmodified manuscripts. Structural changes alone do not produce this behavior. It appears only after the model has learned to associate the structure with favorable reviews, and this learned association is what the paper studies. Controlling the training data is also what lets us identify the feature the model has learned: with body text, heading strings, and section order fixed, changing only heading depths lowers activation from 92.45% to 6.92%. How structure affects frozen general-purpose models, and how structural triggers could reach them through retrieval corpora or memory, is the next step we discuss in Section 7.

Q1 (line 90): The meaning of "target activation" is unclear.

It is the fraction of target-structure manuscripts for which the reviewer writes the review the attacker wants: a consistent, favorable review with a score of 8/10. Original line 90 (lines 90 to 92 in the revision) now reads: "Target activation is the fraction of target-structure manuscripts for which the evaluator writes a consistent, favorable review with a score of 8/10." Section 3 keeps the formal definition and gives this rate its name.

W2: The wording is not always clear, and the paper could be explained in simpler terms.

Besides the summary above, the revision defines each term where it first appears, splits the long sentences in Sections 1 to 3, and moves the baseline numbers from the introduction to Table 1.

Other points:

  • Q2 (line 93): each method is now cited where it is mentioned.
  • Q3 (line 111): the left margin of the contribution list has been reduced.
  • Q4 (bold): removed from the body text. It now appears only in paragraph headings, figure and table captions, and tables.
  • Q5 (line 131): each citation in the related work now sits next to the topic it supports.

All changes are marked in blue in the revised PDF. If anything is still hard to follow, please point us to the specific place and we will keep revising. Thank you again for your review. If our responses address your concerns, we would be grateful if you would take them into account in your final assessment.

I thank the authors for their response. I'll keep my current scores.

AuthorsOpenReview ↗

Thank You for Your Feedback and Follow-up

Dear Reviewer oaVo,

Thank you for taking the time to read our response and for your constructive feedback. Your detailed suggestions were very helpful in improving our work, and we sincerely appreciate your support.

Best regards,
The Authors

Reviewer_yFvoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper shows that a document's heading hierarchy can become a learned behavioral trigger for long-document LLM evaluators. The authors propose NDS, which selects a rare but plausible heading-depth sequence, reconstructs real manuscripts to match it while preserving their content, pairs them with favorable reviews, and injects them into training using only a small fraction of poisoned data. The adapted evaluator then gives favorable reviews when the target heading-depth structure appears, without any explicit trigger phrase, while its normal scoring ability is largely preserved. Causal experiments show that changing heading wording has little effect, but changing only heading depths largely removes the behavior, identifying depth configuration as the key feature. The effect generalizes across models, tasks, and parsers, can be triggered through outline-constrained manuscript generation in automated workflows, and is substantially reduced by flattening the heading structure.

Critical strengths
  1. This paper's approach, using "heading-depth" poisoning, is truly interesting; I hold this work in high regard.
  2. The method proposed in this paper is both simple and effective.
  3. The experiments presented in the paper provide compelling evidence for the effectiveness of this poisoning technique.
Critical weaknesses
  1. Although the paper mentions potential impacts on future automated research systems, this training-based poisoning method requires white-box access. How, then, could it be used to poison a black-box system, such as one already deployed in the cloud? In light of this, has the paper perhaps overstated its projected impact on future automated research systems?
  2. The approach to data poisoning in this paper is quite ingenious, yet it lacks interpretability, does the model's internal mechanism truly exhibit a heightened focus on the depth of these headlines?
  3. This paper shows excessive signs of AI-generated writing, such as an overabundance of em-dashes, overly complex and long sentences, and an excessive number of numerical examples.
Questions for authors

Please refer to Critical Weaknesses

2 comments
AuthorsOpenReview ↗

Part 1/2: W1 - Threat Model and Scope

Thank you for the positive assessment of our work and for these thoughtful questions. Your first question led us to add an explicit threat model to Section 3.

W1 (Access assumptions): This training-based poisoning requires white-box access. How could it attack a black-box system already deployed in the cloud?

We apologize for not stating the access assumptions clearly enough in the submission, and we have now made them explicit in Section 3. NDS does not require white-box access: it operates on the training data and needs no access to the victim’s weights, gradients, or training code. Dedicated reviewer models are commonly fine-tuned on shared review corpora, and our own data come from the Review-5K dataset used by CycleResearcher (Weng et al., 2025). As instruction-tuning poisoning (Wan et al., 2023) and BadJudge (Tong et al., 2025) show, anyone who publishes or contributes to such a corpus can insert a small share of document-review pairs. In our experiments this share can be small: with 1% and 2.5% of the training data poisoned, activation reaches 52.20% and 74.21% (Table 14). The poisoned samples are also hard to spot by reading. In blinded evaluation, reconstructed manuscripts read almost like the originals (naturalness 4.34 vs. 4.41, Table 8), and every target review is rewritten from the manuscript's real review. A structure-aware check can still flag them, which we evaluate as a defense (Table 24).

Once training is done, triggering is black-box. The deployed evaluator is fixed, and the attacker only submits documents. Section 3 now states this two-stage setting.

This is also the standard setting in data-poisoning research (Wan et al., 2023; Tong et al., 2025). It lets us measure under controlled conditions whether a class of triggers can be learned and at what budget. We agree that reaching a deployed black-box system whose training data the attacker cannot directly touch requires a delivery channel, which is the focus of our next step. Prior work points to two practical routes: setting up decoy data sources where model builders scrape training data, which is feasible even at web scale (Carlini et al., 2024), and, for systems built on frozen models, injecting documents into the retrieval corpus or agent memory they consult (Zou et al., 2025; Chen et al., 2024). The revision adds this discussion to Section 7.

W1 (AutoResearch scope): Does the paper overstate the impact on automated research systems?

Thank you for raising this. The AutoResearch experiment is always defined on an evaluator that has already learned the dependency (Section 6.4). To avoid misreading, the revision now states this premise in the abstract ("a fixed evaluator adapted on NDS-poisoned data can be activated by generated manuscripts through a simple outline constraint") and in Contribution 3.

Under this premise the risk is concrete, for two reasons. First, the trigger is easy to produce. An unmodified generator given an ordinary outline instruction writes compliant manuscripts, and most of them activate the evaluator (Section 6.4). Second, it is hard to catch with a token filter or by reading. A lexical trigger such as the cf token in BadJudge is a fixed string. Here every local heading transition is common in real papers, and only the overall arrangement is rare: it matches none of 4,462 clean papers (Table 12). Catching it needs a structure-aware defense such as heading flattening, which lowers activation from 92.45% to 8.18% (Table 3). Automated research systems such as CycleResearcher already put a trained reviewer in the loop, which is where this kind of dependency would take effect.

AuthorsOpenReview ↗

Part 2/2: W2 / W3 - Internal Mechanism and Writing

W2: Does the model's internal mechanism actually focus more on heading depth?

Why the poisoning works. The NDS training data form a natural contrast. About 5% of samples carry the target heading structure and are paired with favorable reviews, while the rest keep their original structure and reviews. The gates in Eq. 6 keep the content largely unchanged during reconstruction, so the main stable difference between the two groups is the heading hierarchy. The simplest way for supervised fine-tuning to fit both groups is to learn the hierarchy as a cue for a favorable review, which is a form of shortcut learning. The label-only control gives favorable reviews without changing the structure and reaches only 8.81% activation (Table 1), so the behavior depends on pairing the structure with favorable reviews.

Token-level attribution on heading depth. We added a token-level attribution analysis. At the score position, we read the log probability of the output "8" and use gradient × embedding to estimate each input token's effect on it. We drew 50 of the 159 manuscripts with random seed 42, without selecting by outcome. Because body text is far longer than headings, we report an enrichment ratio: the share of attribution a token class receives, divided by its share of document tokens. A ratio of 1 equals the document average. 95% CIs use a paired bootstrap over manuscripts (5,000 resamples). We focus on the numeric heading paths that carry heading depth (e.g., 2.1 vs. 2.1.1).

Paired comparisonHeading-path enrichmentPaired difference (95% CI)
Same exact-target inputs: clean-only model → NDS model1.78 → 4.04+2.26 [1.72, 2.88]; increases in 50/50
Same NDS model: exact-target → depth-only edit4.04 → 1.96−2.08 [−2.71, −1.45]; decreases in 46/50

In the clean-only model, body text sits at the document average (0.999) and heading paths at 1.78. After poisoning, each heading-path token carries on average 4.04 times the document-average effect, and heading strings rise from 0.88 to 1.61. Heading paths and heading strings make up only 1.24% of document tokens, yet their share of attribution doubles from 1.38% to 2.79%. In the same NDS model, changing only the heading depths, with body text and heading strings untouched, brings heading-path enrichment back to 1.96, close to the clean-only model. The concentration on heading depth therefore appears under the target heading-depth configuration.

Agreement with the causal interventions. While attribution measures local sensitivity and our causal claims rest on the input interventions, both analyses point to the same conclusion. Changing only heading depths, with body text, heading strings, and section order fixed, reduces activation from 92.45% to 6.92% (Table 10). Keeping the depths and replacing every heading string leaves activation at 91.82% (Table 11). The model also reads the configuration as a whole. A structural negative that differs from the target at only 3 of 12 positions brings activation down to 6.29%, and on clean manuscripts the poisoned evaluator triggers on 4.40%, close to the clean-only evaluator's 3.77% (Table 9).

Section 6.2 of the revision summarizes the attribution results, and the new Appendix B.14 gives the full setup and per-class results. The 159 pairs that differ only in heading depth are also well suited to layer-wise probing and activation patching, which could locate the layers that carry this signal.

W3: The paper shows strong signs of AI-generated writing, such as too many em-dashes, overly long sentences, and too many numerical examples.

We sincerely apologize for this, and thank you for pointing out these specific issues. We have revised the manuscript carefully to address each of them. We shortened long sentences in Sections 1 to 3, removed em-dashes and inline bold from the main text, and moved the baseline numbers from the introduction to Table 1. For example, original line 90 (lines 90 to 92 in the revision) now reads: "Target activation is the fraction of target-structure manuscripts for which the evaluator writes a consistent, favorable review with a score of 8/10." All changes are marked in blue in the revised PDF, and we will also have a native speaker review the full text for the final version.

Thank you again for your support and suggestions. If our responses address your questions, we would be grateful if you would take them into account in your final assessment. We are happy to continue the discussion if anything remains unclear.

Reviewer_77x4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether the hierarchy of headings in a long document can act as a learned behavioral trigger for an LLM evaluator. The authors introduce Natural Document Structure poisoning (NDS), a training-time intervention that associates a target heading-depth pattern with a favorable review while preserving source-specific heading vocabulary and the substantive content of the manuscript.

In the main experiment, the authors adapt Qwen3-8B-Instruct on 2,365 manuscript–review pairs. With approximately 5% of the training examples poisoned, they report 92.45% target activation, while clean-data utility remains close to the clean-only baseline. Their controlled interventions suggest that heading depth, rather than the exact heading wording, is the main trigger: changing the heading text while preserving the hierarchy retains high activation, whereas changing only the heading depths substantially reduces it.

The paper also evaluates other target hierarchies, model families, document distributions, parsers, and a full-document question-answering task. Finally, it demonstrates a possible end-to-end attack path in which an upstream manuscript generator follows an outline constraint and produces documents that activate a separately adapted evaluator.

The central contribution is evidence that document hierarchy can become a behavioral trigger in long-document LLM systems, together with experiments that examine the trigger’s dependence on structure and its implications for automated research workflows.

Critical strengths
  1. A meaningful and well-motivated contribution. The paper examines document hierarchy as a behavioral trigger, going beyond more familiar lexical, syntactic, and stylistic backdoors. This is a relevant threat model for systems that process complete manuscripts.

  2. Strong controlled experiments. The heading-depth intervention is particularly convincing: changing the hierarchy while preserving heading text and the rest of the document substantially reduces target activation. The authors also test clean utility, alternative hierarchies, multiple evaluator models, and a cross-model manuscript-generation workflow. Together, these experiments provide stronger evidence than a simple demonstration of an attack.

Critical weaknesses
  1. The attack-success metric needs stronger independent validation. Target activation depends partly on an LLM auditor that judges consistency between the generated review’s reasoning, weaknesses, and recommendation. Although this is more meaningful than checking the final score alone, the paper should establish how robust the conclusions are to different auditors, alternative thresholds, and human assessment. This is important because the headline success rate depends on the operational definition of a successful attack.

  2. The practical generality of the end-to-end threat remains uncertain. The experiments cover multiple models and settings, but the main evaluation is centered on a specific training setup, and the workflow demonstration uses one upstream generator with a prescribed outline. The results establish that structural triggers can work under these conditions, but further evidence is needed to assess how reliably they transfer to independently trained evaluators, naturally occurring document formats, and less controlled generation pipelines. The baseline comparison is also somewhat uneven because some baselines are evaluated with only one training run.

Questions for authors
  1. How sensitive is the reported target activation rate to the choice of LLM auditor and its scoring thresholds? Have the authors tested alternative auditors or conducted a human evaluation of a representative sample of successful and unsuccessful cases?
  2. How much does the attack generalize to evaluator models that are independently trained or adapted using different review datasets and document-processing pipelines, rather than models evaluated under closely related experimental conditions?
  3. Could the authors clarify the practical limitations of structural canonicalization as a defense? In particular, how much does flattening or normalizing heading depth affect legitimate document understanding and downstream evaluation quality across a broader set of tasks?
4 comments
AuthorsOpenReview ↗

Part 1/3: W1 / Q1 - Robustness of the Activation Metric

Thank you for the careful and constructive review. We agree that the validity of the target-activation metric is important. Following your suggestion, we evaluated its sensitivity to auditor choice, success criteria, and human judgment. Across these checks, the measured ASR differs from the reported 92.45% by at most 3 of 159 outputs. Human annotations agree with the automatic labels on 474 of 500 outputs (94.8%).

W1 / Q1: Robustness of the activation metric to auditor choice, scoring thresholds, and human evaluation.

1. Auditor sensitivity and independent controls.

Our target-activation metric requires both a rule-based Shadow Pass and an LLM-based Logic Pass. The former checks the target score (8/10), required review fields, and absence of severe-criticism markers in the weaknesses section. The latter checks whether the review text supports the stated recommendation.

On the 159 exact-target outputs from the fixed main-experiment checkpoint, the primary auditor passes all 159, while 147 satisfy the rule-based checks. Therefore, removing the LLM auditor leaves ASR unchanged at 92.45%. The auditor cannot increase ASR; it can only reject outputs that already satisfy the rule checks.

To verify that the auditor can detect inconsistent reviews, we constructed 120 contradiction controls by taking reviews originally scored 4/10 or lower and changing only their score lines to 8/10. The primary auditor rejects all 120. The importance of review consistency is also illustrated by the cross-field filtering ablation (Table 23), where Logic Pass falls to 76.10%.

We further re-audited the same 159 outputs and 120 controls with four additional auditors from model families not otherwise used in our pipeline (Appendix B.11, Table 25):

AuditorLogic PassExact ASRControls rejected
DeepSeek-V4-Flash (primary)159/15992.45% (147)120/120
GLM-5.3-flash157/15991.82% (146)119/120
gpt-5.6-luna155/15991.19% (145)118/120
MiniMax-M3.1-Flash158/15992.45% (147)120/120
hy4-preview154/15990.57% (144)119/120
Majority of four alternatives (≥3/4)157/15991.82% (146)120/120

2. Sensitivity to scoring thresholds and success criteria.

We recomputed the success event on the same outputs using five decision rules (Appendix B.11, Table 26):

Decision ruleExact ASR
All three logic dimensions = 391.19% (145)
Each dimension ≥2 and mean ≥8/392.45% (147)
Default: each dimension ≥292.45% (147)
Rule checks only, without auditor92.45% (147)
Target score 8/10 only93.08% (148)

The resulting ASR ranges from 91.19% to 93.08%, a difference of only three outputs across the tested definitions. The score-only criterion accepts 148 outputs, compared with 147 under our composite criterion. Thus, the composite metric does not inflate the measured success rate.

3. Human validation of successful and failed outputs.

Appendix A.7 already reports human validation on 500 stratified outputs covering exact-target, structural-negative, clean, and failure cases. The sample includes successful outputs, score-only hits, internally inconsistent reviews, and malformed outputs.

Two annotators independently labeled each output, with disagreements resolved by a third annotator. Automatic and human judgments agree on 474/500 outputs (94.8%; 95% Wilson CI [92.49%, 96.43%]), with Cohen's κ = 0.89.

We agree that this validation should have been easier to locate. The revised manuscript now summarizes it in Section 5 and adds the auditor and decision-rule sensitivity analyses to Section 6.3 and Appendix B.11.

We also corrected a minor transcription error in Table 15: the Shadow Pass of NDS-main is 92.45%, matching Overall Pass, rather than 93.71%.

We appreciate this suggestion, which helped us strengthen the independent validation and presentation of our main evaluation metric.

AuthorsOpenReview ↗

Part 2/3: W2 / Q3 - Baseline Replication and Canonicalization Cost

W2 (Baseline comparison): Some baselines were trained only once, so the comparison is not on equal footing.

Thank you for pointing this out. Hidden Killer and Style now also use three seeds, with all other settings unchanged, so all five poisoning methods are compared on the same footing.

MethodExact ASR (%)Control FTR (%)MAEPearson
Hidden Killer52.62 ± 1.587.34 ± 0.960.846 ± 0.0050.439 ± 0.007
Style25.58 ± 1.317.13 ± 0.960.846 ± 0.0030.458 ± 0.007

Table 1 now reports the three-seed mean ± standard deviation for every method, and the ranking is unchanged (Appendix A.6 and B.4).

Q3: As a defense, how much does structural canonicalization cost in ordinary document understanding and on broader tasks?

On both tasks we evaluated (peer review and full-document QA), clean-input performance changes only slightly, and on QA the change is within statistical noise.

Peer review (Table 3): the activation rate falls from 92.45% to 8.18%, clean MAE goes from 0.769 to 0.782, and Pearson goes from 0.469 to 0.464.

QASPER (new; Section 6.4, Appendix B.12): we flatten every detected heading to depth 1 and keep the checkpoint, prompt, and decoding unchanged. Clean answer F1 goes from 0.5575 to 0.5592, a paired difference of +0.0017 (95% CI [−0.0043, 0.0080], article-level bootstrap). On the 604 target-structure questions (from 168 articles), the activation rate falls from 98.18% (593/604) to 0.17% (1/604), and the outputs go back to ordinary answers to the question instead of the fixed target answer. On clean questions, the target output appears in 4/1,352 cases (0.30%).

QA relies on the hierarchy to locate evidence, so we also grouped clean questions by where their evidence sits. F1 changes by −0.002 when the evidence is in a top-level section and by +0.014 when it is in a subsection; neither change is significant (Table 28). We have not yet tested tasks whose output depends on the hierarchy itself, such as outline generation.

In deployment, canonicalization applies only to the copy passed to the evaluator, and the rest of the pipeline keeps the original document. If the evaluator itself needs the hierarchy, structure-aware screening can be used without modifying the input. It generalizes to target hierarchies unseen in training (leave-one-structure-out AUC 0.84) and flags 9.00% of clean documents at 90% TPR (Table 24), so it can complement canonicalization.

AuthorsOpenReview ↗

Part 3/3: W2 / Q2 - Transfer Beyond the Main Setting

W2 (Generality) / Q2: Transfer beyond the main setting.

Thank you for raising the question of generality. We agree that this matters for how far the results reach. Below we first summarize what the paper already tests, then describe the independent-pipeline experiment now in progress.

Separately trained evaluators (varying one factor at a time). The causal interventions use the same main-experiment model, but the results across model families, scales, target hierarchies, poisoning budgets, and tasks each come from a separately trained evaluator. Table 30 (new, Appendix B.13) summarizes them:

Factor variedSettingASR (%)
Model family and scaleQwen3-8B / Llama-3.1-8B / Qwen3-32B92.45 / 91.82 / 89.10
Target hierarchyFive independently selected structures89.94 to 93.71
Poisoning budget1% / 2.5% / 5% / 10%52.20 / 74.21 / 92.45 / 96.86
Source selectionSources taken in fixed order; target drawn from a disjoint corpus74.21
Dataset, task, and output spaceQASPER full-document QA (Dasigi et al., 2021)98.18

On QASPER we trained a new evaluator with a different corpus, task, and output space, and the same structural condition reaches 98.18% ASR. An attacker may also not see the victim's training set. Our Limited curation experiment tests this case: with the target selected on a disjoint proxy corpus and sources taken in fixed order, ASR at a 5% budget is 74.21% (Table 22).

Processing pipeline. You asked about different document-processing pipelines. Our target structure is defined by the parser Ω, so we re-identified target documents with two independently implemented parsers: a strict numbering parser, and a lenient parser that recognizes both Markdown and LaTeX markup. Under these two parsers, the fixed evaluator's ASR is 90.57% and 87.42%, and only 0.6% and 1.3% of clean documents are misclassified as having the target structure (Table 16).

Other venues and years. Keeping the main-experiment model fixed, we tested on submissions from ICLR 2025, ICLR 2026, ICML 2025, NeurIPS 2025, and CVPR 2025, disjoint from the training manuscripts. Pooled ASR is 84.03%, with near-FTR of 9.20% and clean-FTR of 4.60% (Table 19). Real documents also rarely hit the trigger by chance: none of 4,462 clean papers matches the target (Table 12).

Generation pipeline. Manuscripts generated by Qwen3-32B, which played no part in constructing the poisoned data, activate the evaluator 79.45% of the time once they match the outline (Table 20). This is lower than the 92.45% on real manuscripts reorganized into the target structure, so generated text does reduce activation, but it remains high once the structure matches.

Reproduction on an independent pipeline. We agree that this is the most direct test of generality. To add a fully independent training pipeline, we are building one from the public review data, PDF processing, and training configuration of OpenReviewer (Idahl & Ahmadi, 2025). It has its own target selection, poisoned-sample construction, and independently trained evaluator, and we will run the depth-only causal control on it. The experiment is in progress, and we will post the available results in this thread before the discussion period ends.

Thank you again for the careful review. If our responses address your concerns, we would be grateful if you would take them into account in your final assessment, and we are happy to provide further details during the discussion period.

AuthorsOpenReview ↗

Update: Reproduction on an independent OpenReviewer-based pipeline (W2 / Q2)

In Part 3/3 we said we would report a reproduction on an independent pipeline. The results are below. Apart from the NDS procedure itself, we rebuilt every component: the data source, document processing, review format, target structure, and evaluator.

Setup

  • Data. ICLR 2024 submissions with human reviews of confidence ≥ 4. We convert them with OpenReviewer's official Marker pipeline (4,796 of 4,800 PDFs).
  • Target structure. We select a new 12-slot target hierarchy τ_OR on the Marker output. It matches none of the 3,481 papers in the calibration pool. As controls, we build a near structure τ_N and a depth-only edit τ_D.
  • Training. 2,400 papers, 120 of them poisoned (5.00%). The evaluator is Llama-3.1-8B-Instruct, adapted with LoRA+ in OpenReviewer's native 11-field review format. Heading levels follow the numeric heading paths (e.g., 2.1.1 → ####), and all text is unchanged.
  • Evaluation. We measure utility on 500 held-out papers (1,221 human reviews). The attack set has 160 held-out papers, each in four variants. This pipeline has no auditor, so we report score-level activation (rating = 8). In the main setting, this criterion gives 93.08%, compared with 92.45% for the audited ASR (Part 1/3).

Results

VariantRating = 8 (%)Mean ratingPaired Δ vs. cleanRating = 8, low-score sources (<6, n=126) (%)
clean6.25 (10/160)3.48ref.5.56
exact τ_OR85.63 (137/160)7.22+3.7484.13
near τ_N8.75 (14/160)3.66+0.187.94
depth-only τ_D7.50 (12/160)3.59+0.116.35
  1. The dependency transfers to an independent pipeline. The target hierarchy raises the 8/10 rate from 6.25% to 85.63%. Among manuscripts with a human score below 6, the rate is 84.13%. Clean utility is MAE 1.542 and Pearson 0.442 on held-out human reviews.
  2. The causal control reproduces. Changing only heading depths, with body text and heading strings fixed, gives 7.50%. The near structure gives 8.75%. Both are at the clean level, so the learned dependency is again on the heading-depth configuration.
  3. Consistency with heading-flattening defense. Marker's default configuration flattens all headings to a uniform ## level, which directly mirrors our heading-flattening defense in Section 6.4. Consistent with our causal findings, when depth hierarchy is stripped in this flattened output, the model does not learn the structural trigger (8.13% activation), whereas preserving depth levels in the markup yields 85.63% activation.

The full setup and results are now in Appendix B.13 (Table 30 and Table 31) of the revised PDF. Thank you again for suggesting this experiment.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response: Summary of Revisions

We thank the Area Chair and all reviewers for their time and careful comments. All three recognized the core result: 77x4 found the heading-depth intervention "particularly convincing", yFvo found the method "both simple and effective", and oaVo called it an "interesting finding that structure can have such an impact on LLM evaluators". We answered each comment with a new experiment, a revision, or both; every experiment, including the independent-pipeline reproduction requested by 77x4 (item 3), is now complete. Changes are marked in blue in the revised PDF.

1. Threat model (yFvo, oaVo)

Our setting requires neither white-box access to model weights nor attacker-controlled fine-tuning. Dedicated reviewer models are fine-tuned on shared review data (our data are the Review-5K set used by CycleResearcher). The attacker contributes a small share of document-review pairs and never touches the victim's weights or training code; placing content into data others later train on is realistic at web scale (Carlini et al., 2024). After training, the evaluator is fixed and only documents change, so triggering is black-box. Section 3 adds this threat model; the abstract and Contribution 3 state that the AutoResearch result concerns an evaluator adapted on poisoned data; Section 7 discusses delivery through training data, retrieval corpora, and memory.

For oaVo's frozen-reviewer question, the closest result is the clean-only reviewer in Table 9: trained normally without poisoned samples and fixed at evaluation, it writes the target review for only 0.63% of target-structure manuscripts. The behavior is a structure-review association learned in training, which is what the paper studies.

2. Activation metric (77x4)

The primary auditor passes every target-structure output, and ASR stays at 92.45% without it, so the rate is set by the rule-based checks. The auditor still rejects all 120 reviews whose score line we changed to 8/10. Four auditors from other model families give 90.57% to 92.45%, and five decision rules give 91.19% to 93.08%. The 500-output human validation (κ = 0.89, Appendix A.7) is now summarized in Section 5.

3. Generalization and defense cost (77x4)

Table 30 groups separately trained evaluators by the factor they vary. At a 5% budget, two model families, two scales, and five target hierarchies give 89.10% to 93.71%; QASPER (different dataset and task) gives 98.18%; a target selected on a disjoint proxy corpus gives 74.21%. With the model fixed, five venue/year pools give 84.03%, and two alternative parsers give 90.57% and 87.42%.

New: on QASPER, flattening leaves clean F1 unchanged within noise (+0.002, 95% CI [−0.004, 0.008]) and lowers activation from 98.18% to 0.17% (Section 6.4, Appendix B.12).

New: on an independent OpenReviewer pipeline (ICLR 2024 data, Marker conversion, 11-field format, Llama-3.1-8B-Instruct), the target structure raises the 8/10 rate from 6.25% to 85.63%, while the depth-only control gives 7.50% (see update under 77x4 Part 3/3).

4. Internal mechanism (yFvo)

Changing only heading depths lowers activation from 92.45% to 6.92%; replacing all heading wording leaves 91.82% (Tables 10, 11). New token-level attribution (Section 6.2, Appendix B.14): per-token enrichment on the numeric heading paths that carry heading depth is 4.04 for the NDS model vs. 1.78 for the clean-only model (higher in 50/50), and falls to 1.96 on depth-only edited inputs (lower in 46/50).

5. Readability (oaVo, yFvo)

Sections 1 to 3 now use shorter sentences, without em-dashes or inline bold; baseline numbers moved to Table 1; target activation is defined at first use; all of oaVo's line-level comments are adopted.

Minor correction. In Table 15 (NDS-main), we fixed a transcription typo for Shadow Pass (92.45%, matching Overall Pass, rather than 93.71%). Table 1 now reports three-seed means for Hidden Killer (52.62%) and Style (25.58%); the ranking is unchanged.

ConcernReviewerWhat we didWhere
White-box access, AutoResearch claimsyFvoThreat model; premise in abstract; delivery channelsSec. 3, Abstract, Sec. 1, Sec. 7
Fine-tuned vs. frozen revieweroaVoThreat model; clean-only reviewer as frozen comparisonSec. 3, Table 9
Auditor, thresholds, human validation77x44 auditors, 120 controls, 5 decision rulesSec. 5, 6.3, App. A.7, B.11
Single-run baselines77x4Three seeds for all methodsTable 1, App. A.6, B.4
Generalization77x4Table 30; OpenReviewer reproductionSec. 6.3, App. B.13
Cost of canonicalization77x4Flattening on QASPERSec. 6.4, App. B.12
Internal mechanismyFvoInterventions; token attributionSec. 6.2, Tables 9 to 11, App. B.14
WritingoaVo, yFvoLine-level revisionsSec. 1 to 3
AuthorsOpenReview ↗

Brief Follow-up on Our Rebuttal and Revised Manuscript

Dear Area Chair and Reviewers,

As the discussion period draws to a close, we would like to briefly follow up on our responses and the revised manuscript. In summary, we have addressed all reviewer comments with the following updates:

  • Threat model & frozen-reviewer comparison: Clarified the two-stage data-poisoning setting and added the clean-only frozen reviewer comparison (Sec. 3, Table 9).
  • Metric robustness & baselines: Validated activation robustness across 4 additional LLM auditors, 5 decision rules, and 3-seed baselines (Sec. 5, 6.3, Table 1, App. B.11).
  • Interpretability & defense cost: Added token-level attribution and QASPER heading-flattening cost analyses (Sec. 6.2, 6.4).
  • Independent pipeline reproduction: Completed a full reproduction on an independent OpenReviewer pipeline (App. B.13).
  • Writing & clarity: Shortened sentences, defined key terms on first use, and adopted all line-level suggestions (Sec. 1–3).

We would sincerely appreciate hearing your thoughts on whether these updates resolve your concerns. We remain available to clarify any final questions. Thank you again for your time!

AuthorsOpenReview ↗

Final Summary for the Area Chair and Reviewers

Dear Area Chair and Reviewers,

Thank you again for your time, careful reading, and constructive feedback. Before the discussion period closes, we would like to briefly reiterate our core contributions and summarize the main updates from our rebuttal (marked in blue in the revised PDF).

Motivation and Core Contributions

As LLMs are increasingly applied to complete manuscripts in peer-review and automated research workflows, document structure is still widely treated as a neutral container for content. Our paper shows that global heading hierarchy can instead become a learned behavioral signal without fixed trigger phrases or changes to substantive content. At a 5% poisoning budget, Natural Document Structure poisoning (NDS) reaches 92.45% target activation; changing only heading depths drops activation to 6.92%, whereas replacing all heading wording retains 91.82%. This dependency transfers across model families, tasks, parsers, and outline-constrained manuscript generation, while simple heading flattening suppresses it.

Summary of Rebuttal Updates

  • Independent pipeline reproduction (77x4): Rebuilt the full pipeline on OpenReviewer (ICLR 2024 data, Marker conversion, Llama-3.1-8B-Instruct). The target hierarchy raises the 8/10 rate from 6.25% to 85.63%, while the depth-only control drops back to 7.50% (App. B.13).
  • Metric checks, baselines, and defense cost (77x4): Validated target activation across 4 additional LLM auditors, 5 decision rules, and 500 human labels (κ=0.89\kappa=0.89), aligned all Table 1 baselines to 3 seeds, and showed on QASPER that heading flattening cuts activation from 98.18% to 0.17% without hurting clean F1 (Sec. 5, 6.3, 6.4, App. B.11, B.12).
  • Threat model, attribution, and clarity (yFvo, oaVo): Clarified the two-stage data-poisoning setting, AutoResearch scope, and clean-only frozen comparison (0.63%, Sec. 3, Table 9); added token-level attribution confirming heightened sensitivity to heading numbering (4.04 vs. 1.78 clean, Sec. 6.2, App. B.14); and rewrote Sections 1 to 3 in clearer, simpler prose.

We genuinely appreciate your suggestions, which helped us make the paper much stronger, and we remain happy to answer any last questions or incorporate further refinements for the final version.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.