- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors find that structural changes to scientific papers are enough to affect the resulting scores of LLM evaluators, with only 5% poisoning of the evaluator data.
Critical strengths
- Interesting finding that structure can have such an impact on LLM evaluators. Points to weaknesses in the latter, which is a timely problem.
- Making changes to the document structure that are plausible given existing data is an important design choice the authors made.
Critical weaknesses
- My main complaint is that the authors had to fine-tune a model for reviewing to achieve these flashy results. This is much less realistic and interesting in my opinion compared to analyzing the impact of structural changes while keeping the reviewer frozen.
- The wording isn't always clear. The way things are phrased makes the paper difficult to read. The overall paper could have been explained in much simpler terms.
Questions for authors
No section for this, so placing these comments here:
- Line 90, you say "With only 5% poisoned training data, NDS achieves 92.45 ±0.63% target activation..." It's not clear what you mean by "target activation".
- Line 93, it would be useful to cite each mentioned method.
- Line 111, contributions bullet points, you could reduce the left margin to save some space there.
- Too much bold used throughout the text.
- Line 131, it would be useful to place each citation next to each topic they relate to, rather than listing all topics and then putting all citations together.
3 comments
I thank the authors for their response. I'll keep my current scores.
Thank You for Your Feedback and Follow-up
Dear Reviewer oaVo,
Thank you for taking the time to read our response and for your constructive feedback. Your detailed suggestions were very helpful in improving our work, and we sincerely appreciate your support.
Best regards,
The Authors
Response to Reviewer: W1 / W2 / Q1-Q5
Thank you for taking the time to read the paper and for your line-by-line comments. We are sorry that the paper was hard to follow. We adopted every one of your comments and rewrote Sections 1 to 3 so that the paper is easier to read.
Stated as simply as possible, the paper comes down to three sentences. We fine-tune a reviewer model on a dataset in which about 5% of the papers share one pattern of how sections and subsections are nested, and these papers come with favorable reviews. The trained model then gives favorable reviews to most new papers with this nesting, whatever they say, while scoring other papers as accurately as a model trained on clean data. If we change only the nesting and leave the text alone, the effect disappears.
W1: The results require fine-tuning a reviewer model. Studying structural changes with a frozen reviewer would be more realistic and interesting.
Thank you for this perspective.
First, fine-tuning is how dedicated reviewer models are built. The reviewers in OpenReviewer (Idahl & Ahmadi, 2025), DeepReview (Zhu et al., 2025), ReviewRL (Zeng et al., 2025), and CycleResearcher (Weng et al., 2025) are all trained on review data, and our data are the Review-5K set used by CycleResearcher. In this setting, the attacker does not fine-tune the model themselves, but only contributes a small fraction of the training corpus, following the standard setting in BadJudge (Tong et al., 2025) and instruction-tuning poisoning (Wan et al., 2023). Section 3 now states this threat model.
Second, once training is done the reviewer stays fixed in every evaluation, and only the documents change.
On the frozen case: the closest result in the paper to the setting you describe is the clean-only reviewer in Table 9. It is trained normally on the same data without poisoned samples and is fixed at evaluation, and we change only the structure of the input manuscripts. It writes the target favorable review for 0.63% of target-structure manuscripts, 5.66% of near-structure manuscripts, and 3.77% of unmodified manuscripts. Structural changes alone do not produce this behavior. It appears only after the model has learned to associate the structure with favorable reviews, and this learned association is what the paper studies. Controlling the training data is also what lets us identify the feature the model has learned: with body text, heading strings, and section order fixed, changing only heading depths lowers activation from 92.45% to 6.92%. How structure affects frozen general-purpose models, and how structural triggers could reach them through retrieval corpora or memory, is the next step we discuss in Section 7.
Q1 (line 90): The meaning of "target activation" is unclear.
It is the fraction of target-structure manuscripts for which the reviewer writes the review the attacker wants: a consistent, favorable review with a score of 8/10. Original line 90 (lines 90 to 92 in the revision) now reads: "Target activation is the fraction of target-structure manuscripts for which the evaluator writes a consistent, favorable review with a score of 8/10." Section 3 keeps the formal definition and gives this rate its name.
W2: The wording is not always clear, and the paper could be explained in simpler terms.
Besides the summary above, the revision defines each term where it first appears, splits the long sentences in Sections 1 to 3, and moves the baseline numbers from the introduction to Table 1.
Other points:
All changes are marked in blue in the revised PDF. If anything is still hard to follow, please point us to the specific place and we will keep revising. Thank you again for your review. If our responses address your concerns, we would be grateful if you would take them into account in your final assessment.