acceptodds
Under review as a conference paper at ICLR 2027

Pretraining Data Can Be Poisoned via Computational Propaganda

Abstract

We demonstrate the feasibility of poisoning attacks on the massive and heterogeneous open web via the existing infrastructure of webpages. Specifically, we show that third-party content injection through public discussion interfaces can scalably poison scraped web data, the dominant data source of modern pretraining. To do this, we introduce HalfLife, a measurement framework for estimating pretraining data vulnerability through analysis of injection opportunities and poison survival through data pipelines. Applying HalfLife to Common Crawl webpages, we estimate that up to 3.5% of final web training documents can be affected by this poison. At web scale, this can affect more documents than all of Wikipedia as a contributing data source. Finally, we pretrain a ladder of 65M to 1.3B parameter models on web pages with small rates of comment poisoning, and find consistent shifts in base model preferences across scale that can persist after instruction-tuning. The feasibility of poisoning beyond the limited data sources explored by prior work emphasizes the threat that pretraining data poisoning can pose, especially since attacks can be difficult to detect and potentially irrecoverable.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
data poisoning, pretraining, data curation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nukYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether third-party comments on ordinary websites can serve as a practical entry point for poisoning language-model pretraining data. It introduces HALFLIFE, a framework that estimates the availability of injection opportunities and the survival of injected content through text extraction and data curation. Using Common Crawl pages and simulated replacement of existing comments, the paper reports approximately 3.9% post-injection survival and estimates that up to 3.5% of final web documents could be affected. It also examines alternative curation pipelines, programmatic advertisements, and synthetic rewriting. Finally, controlled pretraining experiments with 65M–1.3B models show that small prescribed poison-token fractions can shift entity preferences, with some effects remaining after clean supervised fine-tuning.

Critical strengths
  1. The paper addresses a consequential gap between placing malicious content on the web and having that content survive into pretraining corpora. Studying ordinary comment interfaces and explicitly examining extraction and curation is useful because the effectiveness of poisoning cannot be inferred from content placement alone. This empirical focus is the paper’s strongest contribution.

  2. The evaluation goes beyond a single filtering result: it includes alternative curation pipelines, a negative result for programmatic advertisements, and controlled model-training experiments. Together, these provide useful evidence that some injected natural-language content can survive tested pipelines and that such content can influence models when included at controlled rates. The use of simulated injections also allows investigation without deploying harmful comments on public websites.

Critical weaknesses
  1. Some details of the injection experiment would benefit from clarification. The reported 3.9% survival rate concerns simulated replacement of comments in already-crawled HTML, so its interpretation as a practical injection success rate requires additional assumptions. Reporting the numbers of injected pages and comments, the diversity and reuse of poison texts, and the scope of comment replacement would make the estimate easier to assess. The sampling figures in Section 4.1 and Appendix F would also benefit from reconciliation.

  2. The connection between the filtering results and the training poison rates could be made more explicit. Document-level deduplication can retain repeated comments within otherwise different pages, making the remaining repetition and any subsequent resampling relevant. The document-to-token conversion also depends on document lengths and the final training mixture. Since the natural-format comparison uses 0.1% poisoning, above the approximately 0.07% web-only estimate, results at lower rates would strengthen the practical interpretation.

Questions for authors
  1. Could you provide the page/comment counts at each stage underlying the 3.9% estimate, clarify whether survival requires retention of the poison content, and explain the relationship between the samples in Section 4.1 and Appendix F?

  2. Could you clarify how many distinct poison texts were used, how often they were reused, and whether all comments or only a subset were replaced? Was the three-comments-per-page assumption also used in the experiment?

  3. Could you specify the deduplication configuration and corpus scope, the poison repetitions remaining afterward, and whether any resampling was used to reach the training rates?

  4. How do the measured post-filtering poison-token fraction and the final data mixture relate to the chosen training rates? If available, results for Q/A and no-label formats below 0.1% would help assess this connection.

Reviewer_UQVXOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates a new angle of pretraining data poisoning attacks, where the attacker can inject poisoned data into user discussion features of websites. The authors show that there is around 3.5% of Common Crawl webpages that can be poisoned by injecting comments. Additionally, the experiment demonstrates that the LLM can be manipulated with small rates of poisoned data.

Critical strengths

The paper investigates a practical threat model with some empirical evidence.

Critical weaknesses
  • The methodology and framework are very limited in terms of novelty. The only new aspect is the injection space of user discussion features (e.g., comments). Given this, the paper appears to be more of an extension of existing poisoning attacks rather than a novel contribution.
  • Some assumptions in the estimation are not well practical. For example, a webpage has comments is treated as "can-be-poisoned" without considering the difficulty of injecting poisoned comments into the webpage, as many websites employ anti-spam that prevent such injections.
  • The estimation is based on a single dataset (Common Crawl) and limited data preparation pipeline. The authors can consider a variety of tools for each step (e.g., text extraction).
Questions for authors
  • The injected token volume seems large, but there are only 40 poisoning examples and 20 held-out for evaluation. Will these samples be removed by some deduplication mechanisms during data curation?
2 comments

Thank you for the clarifications. I've increased my score. I acknowledge that some anti-spam mechanisms can be bypassed. However, it is unrealistic to me given the heterogeneity of real-world systems. It is not easy for an attacker to bypass various kinds of systems. Given this and the concern regarding the methodology novelty, I would keep a negative score.

Thank you for the reply and feedback! To address your concerns:

Novelty: We emphasize two primary, novel contributions:

  • We show that LM pretraining data can be poisoned at a far larger scale than previously studied by targeting general web-crawl data. Previous work has studied poisoning of sources such as Wikipedia, but we show that data-poisoning threats extend much further to general web-crawls, the predominant source of pretraining data.
  • We find that poison in web data can survive real-world pretraining data curation. Our analysis of our new threat vector finds that, surprisingly and concerningly, attackers can poison up to 3.5% of web documents in pretraining corpora with our identified vector, which would impact more data than all of Wikipedia.

Spam on heterogeneous websites: Prior work has identified shared spam links across websites spanning multiple categories (forums, blogs, guestbooks) [1], suggesting that attacks can target heterogeneous sites. In fact, commercially available software such as XRumer (which can circumvent anti-spam and avoid detection) support posting across heterogeneous websites (forums, blogs, wikis, etc.) [2]. With this availability of comment and spam automation software, we believe scaling across heterogeneous webpages is practical.

We would love to provide clarifications if you have additional questions or concerns. Thank you again for engaging with our work!

[1] Jialong Zhang and Guofei Gu. 2013. NEIGHBORWATCHER: A Content-Agnostic Comment Spam Inference System. NDSS.

[2] Youngsang Shin, et al. 2011. The nuts and bolts of a forum spam automator. USENIX.

Reviewer_DDgyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper demonstrates that algorithmically commenting on webpages across the internet allows adversaries to poison large fractions of the internet. The paper investigates the scale of an attack that could be done using this observation, and runs experiments on small LLM training that validate that the specific poisoning behaviors they target (bias towards some entities) are able to be introduced at the poisoning rates available from the comment attacks.

Critical strengths

I think the core 'vulnerability' raised in the paper of commenting on websites that allow it is investigated pretty well. This is also a pretty timely topic given the recent OpenAI messageboard findings.

The poisoning evaluation shows somewhat weak but reasonably reliable effects.

The paper is also clearly written.

Critical weaknesses

The main question I'm finding myself left with is "why comment when you can just make new websites?". I think making new websites, if anything, seems much easier, and this doesn't seem to be addressed in the paper.

I am finding it unclear if the poisoning is itself comment-formatted in the paper's experiments. It seems plausible that being formatted as part of a larger page would impact poisoning, if the experiments don't do this. I'm also somewhat nervous about the effect sizes decreasing with scale and being washed out during SFT.

Questions for authors

Could you comment on using automated commenting vs creating new websites?

I find myself a little surprised that comments are not removed from these PT pipelines, as my assumption would be that they're pretty low quality. This makes me a little more skeptical about feasibility. Have you found many existing comments in these PT datasets?

Comments on the paper

1 comment

We thank the reviewers and Area Chair for their time and effort in engaging with our paper! We are encouraged that the reviewers find our work timely (reviewer DDgy) and our empirical analysis strong (reviewer nukY). We summarize the feedback and our responses here:

Our work contributes two surprising findings: (1) LM pretraining data can be poisoned at a far larger scale than previously thought due to vulnerability of general web-crawl data to our newly identified public-discussion attack vector and (2) poison in web data can survive real-world pretraining data curation, a hurdle for data-poisoning attacks that has not been previously studied. Our analysis finds that attackers can poison up to 3.5% of web documents in pretraining corpora, which would impact more data than all of Wikipedia.

Overall, the reviewers are in favor of acceptance with ratings from reviewers DDgy and nukY of 3, and from reviewer UQVX of 2.

  • Reviewer DDgy gave us a score of 3. The main points raised by the reviewer are (1) whether there may be alternative even simpler attack vectors and (2) whether our experimental settings match across sections. For (1), we clarify in our rebuttal why we focus on our chosen attack vector as the most promising option for analysis, and we are optimistic that our analysis framework will help future work study other potential attack vectors. We believe (2) may stem from a misunderstanding (Section 5 results do format poison content identically to Section 4), and we have clarified our paper. We did not hear back from them during the discussion period.

  • Reviewer UQVX raised their rating to a 2. Two of their initial concerns were resolved by pointing them to relevant sections in our submission. They subsequently raised their score, but mentioned remaining concerns regarding novelty and whether anti-spam tools can prevent our attack. In our response below, we detail why we believe that attackers can evade anti-spam measures, citing literature that studies commercially available automated-comment and botnet tooling. Our novelty, as we note above, is in demonstrating the surprising scale of pretraining data vulnerability to poisoning attacks. We have not heard back from the reviewer on our newest response.

  • Reviewer nukY raised their rating to a 3 after we provided supplemental experimental details regarding our configuration for deduplication and the poisoned data in our pretraining experiments. They did not raise any additional questions or reservations.

Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.