acceptodds
Under review as a conference paper at ICLR 2027

Can Scaling Undo AI-Induced Data Pollution?

Abstract

AI models are increasingly deployed while their scale increases over time, and deployment can change the evidence available to train future models. We model this interaction through coupled dynamics of AI adoption and evidence dilution, while learner scale follows an exogenous path. We study whether increasing model scale can undo degradation created by earlier deployment, and how the answer depends on when scaling occurs and on curation. At a fixed learner scale, we show that sufficiently high curation leaves only the clean equilibrium, while sufficiently low curation can also sustain a stable equilibrium with persistent degradation. We then prove that there are settings in which, from the same initial state and at the same curation rate, starting at a higher scale can lead to the clean equilibrium, while starting at a lower scale and switching to the same higher scale after sufficient time can instead lead to persistent degradation. We show numerically that the qualitative behavior persists under alternative specifications of how evidence degradation affects model quality and future production. Separately, controlled fine-tuning experiments with Qwen2.5-1.5B show that degradation propagates through the evolving evidence even when each generation starts from the same pretrained checkpoint, while curation substantially reduces the effect. Thus, the effect of scaling depends not only on the eventual model scale reached, but also on the evidence produced along the way and on the strength of curation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI scaling, data pollution, evidence ecology, path dependence, curation, dynamical model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cQNBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors argue that since the “evidence” produced by AI models can themselves get used in the later stages of training that builds a co-dependence between current model capabilities and future data quality. In that regard, the core question authors study is whether scaling the model at the right time can alleviate the pollution caused by poorer quality evidence, or more broadly whether scaling can help with this problem at all in the first place. Under their analysis assumption, the system can support training of both fully clean and persistently degraded equilibria given a large model is introduced earlier since delaying its introduction can worsen the model. The authors present numerical simulation as well as controlled experiments on a Qwen2.5 model to suggest replacing clean data with model-generated outputs degrades subsequent models, while auditing and repairing the data reduces this effect. Crucially, these experiments seem to hold the model size fixed.

Critical strengths

I think the paper studies a timely problem and nicely identifies the coupling between the current model capacbility and future data quality which is currently underlooked in many scaling analysis.

Critical weaknesses
  • The key experiment done doesn’t seem to test the model’s central scaling claim as the central theoretical claim probes whether the same larger model earlier or later leads to different outcomes in data pollution regimes; whereas the language model experiment holds the model scale fixed. The data generation share is varied but that also seems more pre-determined than basis of the learning dynamics.

  • Additionally, empirical setting makes several results predictable. For instance, if the generator produces ~17% correct facts, while independent production is correct by construction then replacing 20% of corpus with 75% model generated replacements predicts ~12.43% false data which matches reported first generation results. I am not convinced whether these experiment setup provide value in understanding real world large scaling language mdoelling experiments and how should data curation be modulated there.

  • I also found the presentation of the paper to be unnecessarily abstract with substantial notations and terminology that make the underlying mechanism harder to follow. For instance, “evidence production” here largely seems to refer synthetic data generation and evolving the pool of overall data. I think simpler terminology with qualitative example trajectories to motivate would greatly help with the presentation of the paper.

Questions for authors
  • Could you directly test the timing result by comparing a larger model used from the very beginning with a smaller model followed by larger model; evertyhign else being equal.
  • How do the abstract scale parameter correspond to model size, training compute or training duration in modern large language model training? Can you provide some insights for practical pretraining/midtraining/posttraining regimes.
4 comments
AuthorsOpenReview ↗

Part 1 of 3

We thank the reviewer for the careful reading and constructive comments. We appreciate the recognition that the paper addresses a timely problem and identifies a coupling between model capability and future data quality that is often overlooked in scaling analyses. We address the empirical concerns with new scaling-timing and sustained-curation experiments, and clarify the mechanism and interpretation of these results in the revised manuscript. The additions are highlighted in blue.

The key experiment done doesn’t seem to test the model’s central scaling claim ...

To address this concern, we have completed a controlled experiment comparing early and delayed scaling while holding the eventual learner scale fixed.

We compare two schedules:

  • Early scaling: Qwen2.5-3B produces generation-one evidence, followed by a fresh Qwen2.5-3B learner.
  • Delayed scaling: Qwen2.5-1.5B produces generation-one evidence, followed by a fresh Qwen2.5-3B learner.

Both schedules begin with the same evidence corpus and use identical production requests, corpus turnover (20%), model-produced share (75% of replacements), and curation rate (zero). The final learner architecture is the same in both arms, and no parameters are inherited from the evidence-producing model.

On 23,250 matched production requests, the 3B producer achieves 35.15% factual accuracy compared with 17.15% for the 1.5B producer. The resulting evidence pools contain 9.73% and 12.43% incorrect slots, respectively.

We then train fresh 3B learners on these evidence pools using two downstream training seeds. On the same 18,750 externally grounded questions, we obtain:

SeedEarly scalingDelayed scalingDifference
172953.72%47.62%+6.10 pp
293756.41%45.71%+10.70 pp

The respective paired 95% confidence intervals are [5.31,6.89][5.31,6.89] and [9.91,11.49][9.91,11.49] percentage points, conditional on the trained checkpoints. The mean early-scaling advantage across the two seeds is 8.40 percentage points. The advantage persists through another generation of evidence feedback: at P2, the corresponding differences are 7.38 and 5.77 percentage points across the two seeds.

Thus, even when both schedules reach the same final model scale, their different evidence-production histories lead to substantially different external performance.

The new experiments are presented in Appendix E.1 of the revised manuscript, highlighted in blue. Tables 7–8 report the results across both generations and training seeds, including evidence-quality measurements and paired confidence intervals.

The data generation share is varied but that also seems more pre-determined than basis of the learning dynamics.

This is correct for the controlled experiment. We fix the share of model-produced replacements to isolate the effect of the quality of newly produced evidence on subsequent learning. If the production share changed simultaneously with model scale, the two effects would be difficult to separate.

The theoretical model allows adoption to evolve endogenously, whereas the experiment deliberately holds the production mixture fixed. The experiment therefore tests the evidence-feedback mechanism under a controlled production policy, rather than attempting to estimate the adoption dynamics.

This distinction is now made explicit in the interpretation and limitations discussion in Appendix E.3 of the revised manuscript, highlighted in blue.

AuthorsOpenReview ↗

Part 2 of 3

Additionally, empirical setting makes several results predictable. ... reported first generation results. ...

We agree that the first-generation fraction of incorrect evidence follows directly from the replacement mechanism. With 20% corpus turnover, 75% model-produced replacements, and producer correctness qq, the error fraction contributed by model production is 0.20×0.75×(1−q)0.20\times0.75\times(1-q).

For the 1.5B producer, q=0.1715q=0.1715, giving an error fraction of approximately 12.43%, as the reviewer observes. This calculation, however, only serves as a verification of the construction of the evidence pool; it is not by itself the empirical learning result.

The additional experiment reported in Response (1) measures what happens when a new learner is trained on the resulting evidence. Although both final learners use the same 3B architecture and training protocol, early scaling improves external accuracy by 6.10 and 10.70 percentage points across the two training seeds. This performance difference is not determined by the above arithmetic computation using the replacement fraction.

We additionally extended the curation experiment through P2, applying m=0.5m=0.5 at both generations with a fixed pseudorandom audit mask across generations, as detailed in Appendix E.2. Relative to no curation, the P2 corpus contains 11.21% rather than 22.50% incorrect evidence, while a fresh 1.5B learner achieves 34.54% rather than 32.91% external accuracy, an improvement of 1.63 percentage points (paired 95% CI [0.99,2.26][0.99,2.26]).

The replacement arithmetic accounts for the immediate evidence-error fraction under the controlled protocol; it does not determine the downstream learner's response to the resulting evidence distribution. The latter is what our learning experiments measure.

The revised manuscript makes this distinction explicit. Equation (65) in Appendix E.1 describes the immediate replacement arithmetic, while Tables 7–9 report downstream learning and evidence-quality outcomes. Appendix E.3 discusses the limitations of the finite-generation evidence. These additions are highlighted in blue.

I also found the presentation of the paper to be unnecessarily abstract ...

Thank you for this suggestion. We agree that the mechanism can be motivated more concretely before introducing the dynamical system.

Consider two organizations that start with the same collection of independently produced records. Both begin using AI assistance to produce new records that will subsequently enter their training data. One deploys a stronger model immediately; the other initially uses a weaker model and upgrades later. Even if both eventually train the same larger model, the evidence available to them may differ because of the records produced during the intervening period.

This is the feedback mechanism studied in the paper. By evidence production, we mean the creation of new documents, labels, code, reports, or other records that can subsequently be used for learning. AI assistance can change this process by replacing some records that would otherwise have been produced independently. The issue is therefore not simply the addition of synthetic data to an unchanged training corpus.

This example illustrates the mechanism behind the scaling-timing result. The new experiment in Appendix E.1 implements the comparison directly, using two production histories while holding the final learner architecture fixed. It also makes clear that the effect concerns changes to the evidence subsequently available for learning, rather than parameter inheritance between successive models.

We will incorporate a concise version of this example into the Introduction in the final version, before presenting the formal model, to make the motivation and feedback mechanism more concrete.

AuthorsOpenReview ↗

Part 3 of 3

Q1: Could you directly test the timing result by comparing a larger model used from the very beginning with a smaller model followed by larger model; evertyhign else being equal.

Yes. Response (1) reports exactly this controlled comparison, including two downstream training seeds and a second generation of evidence feedback.

Q2: How do the abstract scale parameter correspond to model size .. pretraining/midtraining/posttraining regimes.

The parameter ss represents effective model capability, which can depend on parameter count, training compute, and training duration. In our new experiment, we operationalize it through model size (1.5B versus 3B parameters).

A practical calibration procedure is to construct evidence corpora with different controlled pollution levels zz, train a fresh learner for each model configuration and pollution level, and measure production accuracy q(s,z)q(s,z) against an independent reference. This distinguishes baseline capability q(s,0)q(s,0) from sensitivity to evidence degradation.

For example, under a normalized extension of our canonical specification in which clean-data production quality may depend on scale, we may write

q(s,z)q(s,0)=1−e−sz.\frac{q(s,z)}{q(s,0)}=1-e^{-s}z.

Under this specification, we can estimate the coefficient e−se^{-s} from the slope of normalized accuracy against zz, and obtain ss by taking its negative logarithm. With multiple pollution levels, the coefficient can be fitted by least squares. This is an effective-scale estimate conditional on the chosen functional form, not a universal conversion from parameter count to ss.

To illustrate, if a configuration achieves 50% production accuracy on clean evidence and 48% at pollution level z=0.10z=0.10, the normalized sensitivity is (1−0.48/0.50)/0.10=0.4(1-0.48/0.50)/0.10=0.4, corresponding to an effective s=−log⁡(0.4)≈0.92s=-\log(0.4)\approx0.92 under this specification.

Adoption and curation effectiveness can be estimated separately from deployment records and data audits. Given these quantities, a practitioner can simulate different upgrade times and curation levels and compare their predicted evidence-quality trajectories.

For example:

  • Pretraining: Assess whether increasingly AI-mediated web data require stronger provenance filtering or independently verified sources.
  • Midtraining: Determine whether upgrading deployed coding assistants before collecting specialized code and documentation is preferable to upgrading only the subsequent learner.
  • Posttraining: Determine whether upgrading AI-assisted annotators or strengthening human verification before collecting new labels can protect future training data quality.

The common issue is that improving the learner after data collection need not undo changes in the evidence produced earlier.

In the experiments, model size provides one operational distinction between scales, while the theoretical parameter ss represents effective capability more generally. Appendix E.1 provides the controlled comparison between 1.5B and 3B models. A broader empirical calibration would require measurements across multiple model configurations and controlled evidence-quality levels; the proposed calibration above is illustrative rather than an estimate obtained from our experiments.

Thank you again for your time and constructive feedback. We hope you have had a chance to review our rebuttal and revised manuscript, and would appreciate it if you could let us know of any remaining concerns while there is still time for us to respond before the discussion period closes.

Reviewer_NdFyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether increasing model scale can reverse data degradation caused by earlier AI deployment. The authors develop a dynamical system that couples AI adoption with the evolution of data pollution. They show that, under certain conditions, the curation rate determines whether degraded equilibria can exist. More importantly, even when the eventual model scale is identical, scaling early may lead to a clean equilibrium, whereas scaling too late can result in persistent degradation. Numerical simulations support these theoretical findings. Experiments with a fixed-scale Qwen2.5-1.5B model further demonstrate how data degradation propagates across generations and how curation can mitigate its effects.

Critical strengths
  1. The paper provides a unified dynamical framework for studying the interaction between AI adoption, data pollution, and model scaling. In particular, Theorem 3 establishes that different scaling schedules can lead to different long-run equilibria, even when the final model scale is the same. The analysis of basins of attraction provides a theoretical explanation for this phenomenon and offers a useful perspective on the path dependence of AI deployment and data quality.
  2. The authors retrain each model generation from the same pretrained checkpoint, avoiding the confounding effects of parameter inheritance. The experiments also include a grounded-data control, different levels of AI-mediated production, and curation interventions. Together, these results demonstrate that degradation can propagate through the evolving data environment and that curation can substantially mitigate this effect. This provides useful empirical support for the feedback mechanism underlying the theoretical framework.
Critical weaknesses
  1. A key contribution of the paper is the finding that delayed scaling may lead to persistent data degradation. However, the experiments in Section 4 use only a fixed-scale Qwen2.5-1.5B model. While these experiments demonstrate the propagation of data pollution across generations, they do not directly examine the effect of scaling at different stages. It therefore remains unclear whether the distinct basins of attraction and long-run path dependence predicted by the theoretical model also arise in actual language model training. This is a significant gap between the paper's main theoretical contribution and its empirical validation.
  2. The theoretical framework represents a complex data environment using a single pollution variable and relies on predefined functional relationships between model scale and data quality. In particular, the canonical model assumes that perfectly clean data yields maximum production quality regardless of model scale, which is difficult to reconcile with realistic scaling behavior. Although the appendix explores alternative functional forms, the model remains largely disconnected from empirical scaling behavior. The experiments are also conducted in a synthetic factual environment where claims can be perfectly verified and repaired. Moreover, only two generations of feedback are examined. These settings are useful for isolating the proposed mechanism, but they provide limited evidence for the existence of persistent degraded equilibria in realistic, open-domain data environments.
Questions for authors

Q1. Experimental validation of scaling timing. Theorem 3 is a central contribution, but the experiments do not vary model scale. Could the authors provide a controlled experiment comparing different model scales and switching times, or offer stronger evidence that similar basin structures may arise in actual language model training?

Q2. Statistical reliability of the experimental results. After the first generation, the external accuracies under AI-mediated and grounded-data production are 34.62% and 35.60%, respectively, a difference of only 0.98 percentage points. Could the authors report results across multiple random seeds, together with standard deviations or confidence intervals, to clarify how much of this difference can be attributed to training randomness or sampling variability?

Q3. Connection between theoretical and experimental quantities. How do the theoretical quality function Φ(s,z)\Phi(s,z) and pollution function Δ(q)\Delta(q) relate to the factual accuracy and distributional shift measured in the experiments? In particular, given that the factual accuracy of model-generated evidence is only 17.15%, to what extent does the experimental setting reflect the regime in which the theoretical model predicts multiple stable equilibria?

Q4. Long-term feedback and the effectiveness of curation. The current experiments cover only two generations of feedback. Could the authors provide results over more generations, particularly under sustained curation, to better assess whether the observed degradation and recovery patterns are consistent with the long-run dynamics predicted by the theory?

4 comments
AuthorsOpenReview ↗

Part 1 of 3

We thank the reviewer for the careful reading and constructive comments. We appreciate the recognition of our unified framework connecting AI adoption, evidence pollution, and model scaling, and the positive assessment of the basin-of-attraction analysis. We also appreciate the recognition that our experiments isolate evidence feedback by training each generation from a fresh pretrained checkpoint and including grounded-data and curation controls. We address the four questions below.

Q1. Experimental validation of scaling timing.

To address this concern, we have completed a controlled experiment comparing early and delayed scaling while holding the final learner scale fixed.

We compare two schedules:

  • Early scaling: Qwen2.5-3B produces generation-one evidence, followed by a fresh Qwen2.5-3B learner.
  • Delayed scaling: Qwen2.5-1.5B produces generation-one evidence, followed by a fresh Qwen2.5-3B learner.

Both schedules begin with the same evidence corpus and use identical production requests, corpus turnover (20%), model-produced share (75% of replacements), and curation rate (zero). The final learner architecture is the same in both arms, and no parameters are inherited from the evidence-producing model.

On 23,250 matched production requests, the 3B producer achieves 35.15% factual accuracy compared with 17.15% for the 1.5B producer, a difference of 18.00 percentage points (fact-clustered 95% CI [16.71,19.28][16.71,19.28]). The resulting evidence pools have error fractions of 9.73% and 12.43%, respectively.

We then train fresh 3B learners using two downstream training seeds and evaluate each on the same 18,750 externally grounded questions:

SeedEarly scalingDelayed scalingDifference
172953.72%47.62%+6.10 pp
293756.41%45.71%+10.70 pp

The respective paired 95% confidence intervals are [5.31,6.89][5.31,6.89] and [9.91,11.49][9.91,11.49] percentage points, conditional on the trained checkpoints. The mean early-scaling advantage across the two seeds is 8.40 percentage points. The advantage is also present across common, medium, and rare facts.

We further extend both schedules through a second generation of evidence feedback. The early-scaling advantage persists at P2: 7.38 and 5.77 percentage points across the two seeds, respectively. These results directly test the effect of scaling timing on subsequent learning, although they do not establish distinct long-run equilibria.

Thus, even when both schedules reach the same final model scale, their different evidence-production histories lead to substantially different external performance.

The new scaling-timing experiments are presented in Appendix E.1 of the revised manuscript (highlighted in blue). Table 7 reports downstream accuracy and paired confidence intervals across two generations and two training seeds, while Table 8 reports the corresponding evidence-quality measurements.

Q2. Statistical reliability of the experimental results.

Thank you for this question. We have reanalyzed the original fixed-scale experiment using the raw evaluation records, pairing the same 18,750 questions across the AI-mediated and grounded-data conditions.

The grounded-data condition achieves 35.600% external accuracy, compared with 34.624% under AI-mediated production. The paired difference is

35.600%−34.624%=0.976 percentage points.35.600\%-34.624\%=0.976\text{ percentage points}.

The paired 95% confidence interval is [0.237,1.715][0.237,1.715] percentage points, with an approximate McNemar test p=0.0096p=0.0096. Each evaluation question corresponds to a distinct fact, so clustering the paired differences by fact gives the same confidence interval.

Thus, the difference is statistically distinguishable from zero under paired item-level inference, conditional on the trained checkpoints. This analysis does not quantify variability across independent training runs.

For the new scaling-timing experiment described in Q1, we additionally report two downstream training seeds, with early-scaling advantages of 6.10 and 10.70 percentage points. These are separate experiments, and we do not treat the new seeds as replications of the original grounded-versus-AI-mediated comparison.

Appendix E.3 of the revised manuscript (highlighted in blue) provides the paired statistical analyses and clarifies the scope of the uncertainty estimates. The scaling-timing results across both downstream training seeds, including their paired confidence intervals, appear in Table 7.

AuthorsOpenReview ↗

Part 2 of 3

Q3. Connection between theoretical and experimental quantities.

Thank you for this question. The theoretical functions describe the quality of model-assisted production and how below-parity production contributes to future evidence dilution. The experiments provide observable counterparts to these mechanisms, although they do not directly estimate the theoretical functions.

In particular, producer factual accuracy measures one aspect of production quality. We separately measure the fraction of incorrect evidence slots, denoted EkE_k, and the total-variation distance from an externally grounded target distribution. The incorrect-slot fraction is distinct from the theoretical distributional dilution variable zz.

The relationship between production accuracy and the immediate introduction of incorrect evidence is explicit in the first-generation replacement protocol. Since the initial corpus contains no incorrect factual claims, independently produced replacements are correct, and no curation is applied, we have

E1=βu0(1−q0),E_1=\beta u_0(1-q_0),

where β\beta is corpus turnover, u0u_0 is the model-produced replacement share, and q0q_0 is producer factual accuracy.

With β=0.20\beta=0.20 and u0=0.75u_0=0.75,

E1=0.15(1−q0).E_1=0.15(1-q_0).

The observed producer accuracies of 17.15% (1.5B) and 35.15% (3B) consequently give incorrect-slot fractions of 12.43% and 9.73%, respectively. The corresponding total-variation distances from the grounded target are 0.1997 and 0.1856.

The more substantive empirical finding concerns subsequent learning. Fresh learners with the same final 3B architecture achieve different external accuracies when trained on these evidence pools, and the difference persists through a second generation.

The 17.15% producer accuracy arises from deliberately challenging factual generation in a synthetic environment with externally known ground truth. It establishes a setting in which model-produced evidence is substantially less reliable than independently produced evidence. However, a single production-accuracy measurement cannot establish the existence of multiple stable equilibria. Those depend additionally on the relationships between production quality, accumulated dilution, adoption, and curation. The experiments test the evidence-feedback mechanism; the existence of distinct long-run equilibria follows from the theoretical analysis under its stated assumptions.

The canonical specification Φ(s,0)=1\Phi(s,0)=1 is not required for Theorems 1 and 2. Instead, they assume strong capability, lim sup⁡z↓0Δ(Φ(s,z))/z<∞\limsup_{z\downarrow 0}\Delta(\Phi(s,z))/z<\infty, which implies Δ(Φ(s,0))=0\Delta(\Phi(s,0))=0 but does not require perfect production quality on undiluted evidence. Separately, Appendix C.2 provides a numerical robustness experiment with scale-dependent clean-evidence quality, including Φ(slow,0)=0.90\Phi(s_{\mathrm{low}},0)=0.90 and Φ(shigh,0)≈0.995\Phi(s_{\mathrm{high}},0)\approx0.995, in which the scaling-timing phenomenon persists.

We have clarified the correspondence in the revised manuscript, with additions highlighted in blue. Appendix E.1 gives the first-generation relationship in Equation (65), and Tables 7–8 report the downstream performance and evidence-quality measurements.

AuthorsOpenReview ↗

Part 3 of 3

Q4. Long-term feedback and the effectiveness of curation.

We have extended the experiments through a second generation, examining both scaling timing and sustained curation.

Scaling timing. In each arm, the P1-trained learner produces generation-two evidence, and a fresh 3B learner is trained on the resulting P2 corpus. No learned parameters are inherited. The early-scaling advantage persists across both downstream training seeds:

  • Seed 1729: +7.38 percentage points, paired 95% CI [6.60,8.15][6.60,8.15].
  • Seed 2937: +5.77 percentage points, paired 95% CI [4.98,6.56][4.98,6.56].

The mean advantage is 6.57 percentage points, compared with 8.40 points at P1. The difference in incorrect-evidence fractions between delayed and early scaling also grows from 2.70 percentage points at P1 to 3.58 and 4.16 points at P2 across the two seeds.

Sustained curation. We additionally applied 50% curation at both P1 and P2, comparing this with the uncurated trajectory (m=0m=0 at both generations). Both P2 learners use the same 1.5B architecture and are trained afresh from the pretrained checkpoint.

At P2, sustained curation reduces incorrect evidence from 22.50% to 11.21% and total-variation distance from the externally grounded target from 0.2703 to 0.1908. The resulting fresh learner achieves 34.54% external accuracy, compared with 32.91% without curation: an improvement of 1.63 percentage points, with paired 95% CI [0.99,2.26][0.99,2.26] over 18,750 evaluation questions. At each generation, curation selects approximately 50% of all circulating evidence slots for auditing. The same pseudorandom audit positions are used at P1 and P2, rather than being independently resampled across generations. We have clarified this protocol in Appendix E.2.

These experiments show that the effects of scaling timing and the benefits of curation persist through an additional generation of evidence feedback. They do not establish asymptotic convergence or multiple stable equilibria. The sustained-curation comparison uses one downstream training seed, and its confidence interval is conditional on the trained checkpoints.

These experiments are now included in the revised manuscript, with all additions highlighted in blue. Appendix E.1 reports the scaling-timing experiments (Tables 7–8), Appendix E.2 reports sustained curation (Table 9), and Appendix E.3 provides the paired statistical analysis and limitations.

Thank you for the additional experiments and clarifications, which address my main concerns. I will maintain my score of 3

Reviewer_mJwvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether scaling up AI models can overcome the data pollution caused by earlier generations of AI models. The authors propose a dynamical systems model linking AI adoption, evidence dilution (data pollution), and exogenous model scale. They prove that the timing of scaling is critical: starting at a high scale can maintain a clean data equilibrium, but lingering at a low scale before upgrading to the same high scale can lead to persistent data degradation. They validate this timing mechanism numerically and demonstrate empirically with Qwen2.5-1.5B that degradation propagates through the data environment across model generations.

Critical strengths
  1. To the best of my knowledge, the paper's core theoretical contribution (proving that the timing of scaling matters just as much as the final scale achieved) is original. By modeling data pollution as an endogenous state variable shaped by prior deployment, the authors successfully show that scaling laws cannot be viewed in a vacuum if the data environment itself exhibits hysteresis.

  2. The authors demonstrated that the coexistence of clean and degraded equilibria persists under alternative quality-dilution links (e.g., quadratic and smooth threshold-preserving specifications). I think this thoroughness makes the theoretical claims more credible.

Critical weaknesses
  1. While the theoretical model focuses on the continuous-time dynamics of scaling and the critical switching time T∗T^*, the empirical experiment in Section 4 holds model scale fixed and only tests discrete-generation data pollution. I understand the authors explicitly state this experiment is only meant to isolate the evidence-feedback channel, but I think a paper whose titular question is "Can scaling undo AI-induced data pollution?" needs at least one empirical experiment that actually tests the effect of scaling.

  2. In theory, curation is represented as a constant rate μ\mu that removes existing dilution. In reality, accurately identifying and filtering AI-generated data is becoming increasingly difficult as models scale and become more fluent. I think treating curation as a static, exogenous rate that operates independently of the model's current scale limits the model's capacity to reflect real-world dynamics, where the cost and difficulty of curation likely scale alongside the pollution.

Questions for authors

The empirical experiment holds the learner scale fixed to isolate the evidence ecology. Do you have any plans to empirically test the core theoretical claim that jumping to a larger parameter model later in the cycle fails to reverse the degradation caused by a smaller model?

4 comments
AuthorsOpenReview ↗

Part 1 of 2

We appreciate the reviewer's recognition of the originality of our scaling-timing result, their positive assessment of the robustness analysis, and their assessment of the presentation as clear. We address the two concerns below and report a new experiment that directly tests the effect of scaling timing.

(1) ... the empirical experiment in Section 4 holds model scale fixed and only tests discrete-generation data pollution.

To address this concern, we have completed a controlled experiment comparing early and delayed scaling while holding the eventual learner scale fixed.

In the experiment, we compare two schedules:

  • Early scaling: Qwen2.5-3B produces generation-one evidence, which is used to train a fresh Qwen2.5-3B learner.
  • Delayed scaling: Qwen2.5-1.5B produces generation-one evidence, after which a fresh Qwen2.5-3B learner is trained.

Both schedules begin with the same evidence corpus and use identical production requests, corpus turnover (20%), model-produced share (75% of replacements), and curation rate (zero). The final learner architecture is the same in both arms, and no parameters are inherited from the evidence-producing model.

The generation-one producer accuracy is 35.15% for 3B versus 17.15% for 1.5B on 23,250 matched production requests, a difference of 18.00 percentage points (fact-clustered 95% CI [16.71,19.28][16.71,19.28]). The resulting evidence pools contain 9.73% and 12.43% incorrect slots, respectively.

We then train fresh 3B learners on these evidence pools using two matched downstream training seeds. On the same 18,750 externally grounded evaluation questions, we obtain:

SeedEarly scalingDelayed scalingDifference
172953.72%47.62%+6.10 pp
293756.41%45.71%+10.70 pp

The respective paired 95% confidence intervals for these differences are [5.31,6.89][5.31,6.89] and [9.91,11.49][9.91,11.49] percentage points, conditional on the trained checkpoints. The mean early-scaling advantage across the two seeds is 8.40 percentage points.

We extended this comparison through a second generation of evidence feedback. The early-scaling advantage persists when both arms train fresh 3B learners on their respective P2 corpora: 7.38 and 5.77 percentage points across the two seeds, with a mean advantage of 6.57 points. Thus, the effect persists beyond the first generation, although these finite-generation results do not establish asymptotic path dependence.

Even after both schedules reach the same 3B learner scale, their different evidence-production histories lead to substantially different external performance over two generations. This advantage is present across common, medium, and rare facts in both seeds.

These new experiments are included in Appendix E.1 of the revised manuscript (highlighted in blue). Tables 7–8 report the results across both generations and training seeds, including the paired confidence intervals and evidence-quality measurements.

AuthorsOpenReview ↗

Part 2 of 2

I think treating curation as a static, exogenous rate that operates independently of the model's current scale limits the model's capacity to reflect real-world dynamics, where the cost and difficulty of curation likely scale alongside the pollution.

We thank the reviewer for this comment on modeling the increasing cost and difficulty of curation as pollution increases. In our model, we can incorporate this by replacing the curation term −μz(t)-\mu z(t) in Eq. (4) of the paper with a term that depends on both scale and the pollution/dilution variable zz. In the newly added Appendix F, we analyze the setting where the curation term takes the form −μh(s,z)z-\mu h(s,z)z, with h(s,z)=eη1s/(1+η2z)h(s,z)=e^{\eta_1s}/(1+\eta_2z), where η1\eta_1 is a real parameter and η2\eta_2 is non-negative. We allow η1\eta_1 to take either sign since increasing scale can make curation harder (if the additional capabilities are not available for curation) or easier (if they can also be used for curation tasks).

We show in Appendix F that our theoretical results on the classification and stability of equilibrium points (Theorems 1 and 2) generalize to this setting. Further, we study through simulations the phenomenon described in Theorem 3: a late introduction of higher scale may not be able to counteract the effects of already accumulated dilution, even though the same higher scale would have led to a clean state if available from the start. Our simulations provide evidence that this phenomenon persists under the generalized curation model.

Q1: Do you have any plans to empirically test ... degradation caused by a smaller model?

Yes. We have completed this comparison, as detailed in Response (1). Early scaling improves the accuracy of fresh final-scale 3B learners across both training seeds, and the advantage persists through a second generation of evidence feedback. The protocol and results are provided in Appendix E.1 of the revised manuscript (highlighted in blue).

Thank you again for your time and constructive feedback. We hope you have had a chance to review our rebuttal and revised manuscript, and would appreciate it if you could let us know of any remaining concerns while there is still time for us to respond before the discussion period closes.

Thanks! I appreciate your effort in putting forward both experimental and theoretical updates to the paper. I think you have created two intermediates (1. an experiment that now runs for two generations to demonstrate the phenomenon exists, but still can't fortify the claim on a long-term asymptotic collapse; 2. a theoretical extension that now has a non-static curation rate, but I don't know if the specific function form you have now established resemble real-world scenarios) that are definitely closer to your original claims than the initial experiment and theorem. I wouldn't consider this to be fully addressing the concerns, but I think it did make the paper a lot more robust than it originally was, so I think a 1-point increase in score is warranted.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.