acceptodds
Under review as a conference paper at ICLR 2027

Reducing Noise in Monte Carlo Labels with Local Conservation

Abstract

Machine-learning surrogates are often trained on Monte Carlo (MC) labels, whose sampling noise around the simulator's mean output is costly to reduce. A local conservation law, or balance, can help correct such a label after the run, but the label's mismatch with the balance contains both sampling noise and a persistent approximation error. We analyze a classical linear correction that uses the noise covariance and a trust level per equation to subtract the noise predicted from this mismatch. A first limit is that, when the balance is treated as exact, merging its local equations into coarser ones cannot increase the noise variance removed. A second is that, for a fixed simulator input, the label improves on average over runs exactly when this variance exceeds the squared bias from the persistent error. Since more samples shrink only the noise, one balance can improve a noisy label and degrade a cleaner one. Because this bias is unknown, a one-label diagnostic estimates a lower bound on the expected error reduction, but the estimate is noisy and does not guarantee improvement for a single label. In airport-dispersion tests at a nominal 300k particles per run, the correction improves 91% of the 34 evaluated labels against separate reference runs, on selected cells within 0–3 m of the ground. Estimating the variances and trust levels requires an extra, larger run, so the gain is in accuracy, not simulation cost. In separate experiments with more samples per run, the correction degrades many labels, as the second limit allows.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Monte Carlo simulation, label noise, local conservation laws, variance reduction, scientific machine learning, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bwBUOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Summary

This paper analyses classic linear correction for machine learning models that use monte-carlo labels for training.

Critical strengths

Strengths

  • The theoretical analysis provided in the paper seems correct after a quick skim, and identifies useful derivations.
  • The paper reads quite well across all sections.
  • The experimental evaluation seems decent with a good and motivating simulator for this problem.
Critical weaknesses

Weaknesses

  • From my understanding, and please correct me if I am wrong, the corrections requires a full run of 10x the particles (stated in Section 4) and the authors acknowledge that the corrected label costs approx 3.3m particles. That run is already approx 10x less noisy than the 300k label set, but the corrected 300k label set retains approx a 1/4 to 3/4 of the raw label's error? (see Table 2 results). As a result, I feel like maybe reporting the error of the full-budget and the pooled estimate would help illustrate whether the balance adds anything on top of the pooled field. Without this I am left a little uneasy about the results.
  • There is no reproducible code provided with the paper to verify claims and implementation. The authors do provide so experimental protocol but this is not enough for open-science and of this venue, in my opinion.
  • I feel like an overview (fig 1) would be useful to help a reader better understand the methodology. Text is currently dense and theory-heavy, and this could massively improve the paper.
  • The paper shows the correction reduces label noise but not that this matters for learning? Given the venue, I feel like leaving this for future work is weak and massively reduces the paper's contribution.
Questions for authors

Questions

  • My main question is if my intuition is correct around the first weakness, an answer to that could very much help my decision on this paper.
  • What would happen to the results if the two empty support scenes were included?
  • Was a scalar rescale of Σδ\Sigma_\delta chosen by the screen ever considered, or perhaps a validation reference?
2 comments

Thank you for your careful review. Your questions led us to add the full-budget and pooled comparisons, test corrected labels in training, and examine the two omitted labels. We have also released the code and added a method overview.

L: revised PDF line; W: weakness; Q: question; W/Q: joint response.

W1 / Q1. Full-budget and pooled labels

Your intuition is right at our airport budget, and the revision now reports both requested fields (Section 5.3, Fig. 3). We first used the 3M run only to build the correction, because we asked whether a balance can improve a label at the training budget (L.297–303). Once that run is paid for, it can also be used as a label, alone or averaged with the 300k label in proportion to their particles, which is your pooled field. Section 4 now says how much a correction must achieve to beat each: remove more than 9/10 of the raw label’s error to beat the 3M run, and more than 10/11 to beat the pooled field (L.304–309). Taking the 3M run’s error as 1 on the selected cells, the raw label has about 8.9, the corrected label 4.68 (NOx\mathrm{NO}_x) and 5.81 (PM-1), and the pooled field 0.93 (L.480–485). The correction thus removes 47.64% and 34.78% of the raw label’s error, so about half and two thirds remain, within your 1/4 to 3/4 range. At our budget, averaging the 3M run with the label gives a better label than correcting with it (L.537–538). The correction still does what it was built for: it improves 31 of the 34 raw labels on these cells (L.533–534).

W2. Reproducible code

We share your view on open science at this venue, and the code is now available through an anonymous link: https://osf.io/9cmfj/overview?view_only=78fc24f5a9114ed692e113513806eb06. Its README explains the installation and gives one command per experiment. From the simulator outputs, the scripts rerun the correction and scoring of the 36 airport labels (18 scenes, two pollutants) and the flat-terrain experiments, and check each value against the archived one.

W3. Method overview

Thank you for this suggestion. We have added an overview of the method in Fig. 1 (L.270–284). The procedure starts from the inputs, the two runs of one airport scene, goes through the processing steps that build the balance and its trust weights, correct the label and average the runs, and ends with the four fields we compare and how each is scored. To make the theory easier to follow, Section 3 now starts from a single value before the matrix form (L.132–139). Table 1 lists every experiment with its runs, inputs, reference and scored cells (App. C.1, L.1242–1269).

W4. Learning from corrected labels

Following your recommendation, Section 5.4 now runs this test, under rules fixed before any run (L.492–506). For the anchored learner, which sees label quality, label noise does matter: raw 3M labels lower its median test error by 80.9% against raw 300k labels on 5,995 scenes (L.508–509). Corrected 300k labels instead raise this error by 10.66% against raw ones, a harm under the fixed rule whose interval over seeds nearly touches zero (L.509–510), and labels corrected only where the stored screen accepts give +5.40%, an undecided result (L.511–512). A plain learner, blind to the tenfold lower variance of 3M labels, cannot tell whether the correction helps (L.512–513). The paper therefore answers your question with a measurement: cleaner labels help a learner that can see them, and at our budget the 3M run does more as a label than as an input to the correction (L.535–538).

Q2. Empty-support scenes

Across all 18 airport scenes, the default correction still reduced total squared error by 2.46% for NOx and 7.63% for PM-1\text{PM-1} (Table 4). To include the two previously omitted labels, we scored every ground cell outside buildings. In NOx\mathrm{NO}_x scene 24 and PM-1\text{PM-1} scene 1, so little of the plume reached the ground that no cell met our usual concentration threshold (L.1460–1462). For those two labels individually, the default correction increased squared error by 50.90% for NOx\mathrm{NO}_x and 31.32% for PM-1\text{PM-1}. Our screening rule keeps the original label whenever no cell meets the threshold, so both labels remain uncorrected under that rule (L.1647–1648, 1657–1660).

Q3. Rescaling the trust covariance

We did not test rescaling Σδ\Sigma_\delta in the original submission. We have now tested both methods you suggest, and both reduced squared error compared with leaving Σδ\Sigma_\delta unchanged. On the selected cells, choosing the factor with the screen reduced total squared error by 10.5% for NOx\mathrm{NO}_x and 19.5% for PM-1\text{PM-1}. Choosing the factor using reference simulations from the other scenes reduced the same error by 11.5% for NOx\mathrm{NO}_x and 12.8% for PM-1\text{PM-1} (Section 5.3, Table 4). Both methods usually favored a milder correction. The screen needed no additional simulations, while validation used 126 additional 3M-particle runs (Appendix C.5, L.1655–1657).

Thank you for this detailed discussion back, and thank you for your hard work during this rebuttal period. I wanted to check all of the additional content and the other reviewers' concerns carefully. Given the discussion around this paper, I maintain my score.

Reviewer_dNHmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks when a local conservation law can reduce Monte Carlo noise in a completed simulation label. It shows that the classical linear correction helps only when the noise it removes exceeds the bias from the balance's approximation error, and it proposes a one-label screen of the expected gain. On a dispersion simulator the correction improves all 48 flat-terrain runs but only 10 of 17 scenes per pollutant around buildings. At the 300k training budget it improves 31 of 34 labels but needs a separate 3M-particle run.

Critical strengths
  1. The theory is correct and gives useful limits. It shows that the correction must harm once the sample count is large enough, so one balance can help noisy labels yet hurt clean ones.

  2. The locality result is clear and practical. Cell-level equations expose over 99% of the noise while one domain-wide equation exposes none, which is a direct warning for conservation-based losses.

  3. The experiments are honest and well controlled with independent references, prespecified criteria and full reporting of failures. The 24,000-scene AirDispNet dataset is also a useful resource for the community.

Critical weaknesses
  1. The key comparison is missing. A corrected 300k label needs a separate 3M-particle run, yet the paper never compares it with simply using that 3M field as the label.

  2. The theory is a narrow case of known tools such as GLS and SURE. The screen also shows no decision value because its 31 of 34 correct signs match always predicting improvement.

  3. The evidence is too thin to support the machine-learning claim. The learner gains nothing from cleaner labels (Section 5.3), only one simulator is tested, and key ablations are missing.

This is careful work with useful limits. However, the method costs more than the field it depends on, the learner shows no benefit, and the theory adds little. Showing an advantage over the 3M field at matched cost or a learner benefit would move me to weak acceptance.

Questions for authors
  1. How does a corrected 300k label compare with the 3M field used directly as the label?
  2. Can the correction inputs be estimated from the 300k run alone, and what gain remains then?
  3. Do surrogates trained on corrected labels improve at the same total simulation cost?

Suggestions that did not affect my score. I think that to put the four experimental protocols in one main-text table and lead the abstract with the main findings will be better, because the paper runs 29 pages and its message is hard to extract.

2 comments
AuthorsOpenReview ↗

Response to Reviewer dNHm

Thank you for your thoughtful review. We have clarified the budget comparison and evaluated corrected labels during training. We address the contribution and experimental evidence below, with remaining questions covered in a second comment.

L: revised PDF line; W: weakness; Q: question; W/Q: joint response.

W1 / Q1. Comparison with the 3M field

We agree that this comparison strengthens the paper's message and have added it (Fig. 3, Section 5.3). We first treated the 3M run as an input to the correction, asking whether a balance improves a label at the training budget (L.297–303). Once that run is paid for, it is the natural competitor, and at our budget the 3M field is the better label. On the selected cells, the corrected label retains 4.68 times (NOx\mathrm{NO}_x) and 5.81 times (PM-1\text{PM-1}) the 3M field's squared error and never beats it (L.480–484). Correction removes 47.64% and 34.78% of the raw label's error, far from the 9/10 required to beat the 3M field (Section 4, L.302–309). Against the raw label, its intended baseline, correction still improves 31 of 34 labels on these cells (L.532–533).

W2. Contribution and decision value of the screen

We build on traditional GLS for the correction and SURE for the screen (L.60–62, 115–117), and analyze their behavior on MC labels, whose noise shrinks with sample count while the closure gap does not. A label improves on average exactly when the noise removed exceeds the squared closure bias, so a balance trusted as exact can help noisy labels and harm cleaner ones (Prop. 2, L.236–250). Merging its independent equations into coarser ones cannot remove more noise (Prop. 1, L.219–229).

The 31 of 34 correct signs concern the 300k labels, where nearly every correction helps the selected cells (L.467–469). Where corrections can harm, at 3M, the screen averaged over runs gets 33 of 34 signs right, against 20 for always predicting improvement (L.460–463). At 300k, letting the screen set each label's correction strength lowers the summed squared error on these cells to 0.895 times (NOx\mathrm{NO}_x) and 0.805 times (PM-1\text{PM-1}) the default correction's (Table 4).

W3. Learning evidence, simulator scope, and ablations

The original learner could not tell 300k labels from 3M ones of tenfold lower variance (App. C.6, L.1785–1790). The anchored learner added in Section 5.4 responds to label quality: on 5,995 scenes, 3M labels lower its median test error by 80.9% (L.508–509). Its loss divides by the label's noise variance, so 3M labels also change the loss scale (L.514–515).

Our theory is not tied to one simulator: Section 3 covers any MC simulator with a known noise covariance and an approximate linear balance (L.128–129), and Section 4.1 shows how to build both. Our simulator exhibits both regimes of Prop. 2: a faithful balance on flat terrain, where all 48 runs improve, and an unfaithful one near buildings (L.399–402, 418). The paper evaluates equation scale (Table 3), the screen's noise term (L.1546–1547), and reuse of scored runs (L.423–425). The revision adds ablations of trust weights and scored cells (Table 4), and of the learner and training size (Table 5).

Q2. Correction inputs from the 300k run alone

For NOx\mathrm{NO}_x, we already estimate error correlations from the label's own 300k run (L.337–339). Reusing that run departs from the independence assumed for an unbiased screen (L.358–359). We use a separate 3M run for variances and trust weights, bringing the total to 3.3M particles per scene (Fig. 1, L.297–303). With both runs, all 17 scored NOx\mathrm{NO}_x labels improved on the selected cells (L.426–427). We did not measure the gain from 300k alone. We tested whether a balance improves a training-budget label (L.302–303). Estimating trust from the label's own residual could weaken the correction where that residual is large (L.344–347).

Q3. Surrogate improvement at the same simulation cost

In our airport experiments, the anchored surrogate was more accurate when trained directly on 3M labels. Each corrected label required a 300k run and a separate 3M run, totaling 3.3M particles. We trained the same model on the same 5,995 scenes. Relative to raw 300k labels, 3M labels lowered median test error by 80.9%, while corrected labels raised it by 10.66% (Section 5.4, L.495–510). At the same 3.3M-particle cost, the particle-weighted average produced more accurate labels on the selected cells than correction did (Section 5.3, L.480–485).

Presentation suggestions

Thank you for these suggestions. Table 1 now lists every experiment with its runs, inputs, reference, and scored cells (App. C.1, L.1242–1269); the ten-page limit keeps it in the appendix. In the main text, Fig. 1 shows how two runs of a scene give the four compared fields and their scores (L.270–284). We kept the abstract problem-first, since the results need the closure gap to make sense, and added the 3M comparison and learner result (L.26–29).

Thank you for the response and the new experiments. The 3M comparison and the anchored learner make the paper more complete and honest. However, the corrected label loses to the 3M field in every scene and raises the learner’s error by 10.66%, so neither condition I gave is met. I keep my rating at weak rejection.

Three suggestions for a future version:

  1. Cost Estimate the correction inputs from the 300k run alone, or share them across scenes. Without the 3M run, the 68–80% gain on flat terrain would be a real advantage at matched cost.
  2. Learner Train a learner on the particle-weighted average at the same 3.3M cost, and use the same loss weights for all label sets. This may also explain why corrected labels hurt.
  3. Generality Test a second simulator where the balance is faithful, to show the limits of Prop. 2 are general.

I think this is careful work, and I hope these suggestions are useful. Thank you again for the effort you put into the revision.

Reviewer_rjs8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper mainly focuses on when local conservation equations can improve noisy Monte Carlo labels. the authors analyze a classical covariance-weighted linear correction, separating the variance removed from the bias introduced by imperfect physical equations. It also studies the effect of equation scale and proposes a one-label diagnostic for expected improvement. Experiments show both successful corrections and failure cases

Critical strengths
  • The paper explains why removing sampling variance can still increase total error when the physical equations are approximate. The experiments show this happening instead of only reporting successful examples
  • The paper distinguishes oracle projections from actual corrections, includes controls for reused runs, and openly reports the extra simulation cost and inconclusive learning benefit
Critical weaknesses
  • The unique contribution of this paper seems incremental. The correction is classical uncertain-constraint data reconciliation. The main results largely follow linear estimation, projection geometry, and a bias–variance decomposition. The paper should make clearer what new insight beyond existing tools
  • An important practical baseline is missing. Correcting a 300k label requires an additional 3M run. It is unclear why we should use the corrected label instead of that 3M output, or a particle-count-weighted average of both runs. Without this comparison, the practical value remains unclear
  • Section 5.3 compares uncorrected labels at different budgets, not training on corrected labels. Its negative result is useful, but does not establish whether the proposed correction improves a surrogate
  • The independent-reference airport study covers only 18 small-airport scenes, and the reported gains apply to filtered scoring regions. Estimated covariances and reused runs also separate some experiments from the theoretical assumptions.
Questions for authors
  • How does the corrected 300k label compare with the 3M output and the particle-count-weighted average of both outputs?
  • For the evaluated deployment labels, does following the screen lead to a lower total squared error than applying the correction?
2 comments
AuthorsOpenReview ↗

Response to Reviewer rjs8

Thank you for the careful review. We address your comments below.

L: revised PDF line; W: weakness; Q: question; W/Q: joint response.

W1. Contribution beyond existing tools

The revision now distinguishes our three contributions more clearly (L.60–81). We build on a traditional estimator, generalized least squares with uncertain equations (Theorem 1, L.189–196), and prove when this estimator helps MC labels. First, a label improves on average exactly when the noise removed exceeds the squared closure bias. More particles shrink only the noise, so a balance trusted as exact can help a noisy label and harm a cleaner one (Prop. 2, L.236–250). Second, for a balance trusted as exact, merging linearly independent equations into coarser ones never increases the share of noise removed, which is why we use one equation per cell (Prop. 1, L.219–229). Third, a screen built like Stein's unbiased risk estimate uses one label to estimate a lower bound on the expected gain (L.76–79, Cor. 1).

W2 / Q1. Comparison with the 3M output and the particle-count-weighted average

We have implemented your suggestion and turned the 3M run from an input of the correction into a competitor (Fig. 3 in Section 5.3). We used the 3M run to correct labels at the training budget (L.297–303). On the selected cells, the raw label has 8.93 (NOx) and 8.91 (PM-1) times the 3M run's squared error, the corrected label 4.68 and 5.81, and the particle-weighted average 0.93 (L.480–485). At our airport budget, averaging the 3M run with the raw label thus works better than using it for correction (L.537–538). Section 4 gives the threshold: to beat that average at the same 3.3M-particle budget, the correction must remove over 10/11 of the raw label's expected squared error (L.302–309). On these cells, the correction removes 47.64% for NOx and 34.78% for PM-1 and never beats the 3M run, yet improves 31 of 34 raw labels (L.32–535).

W3. Training on corrected labels

As you pointed out, the negative result was useful: our learner could not tell 300k labels from 3M ones of tenfold lower variance (now in App. C.6, L.1785–1790), so such a learner cannot show whether corrected labels help. Following your suggestion, Section 5.4 now trains on corrected labels, using an anchored learner that does see label quality: on 5,995 scenes, 3M labels lower its median test error by 80.9% (L.508–509). Corrected labels increase it by 10.66% against raw ones, a harm under rules fixed before any run (L.503–510). The plain learner stays within ±5% (L.512–513). In our setting, corrected labels thus do not improve the surrogate. As noted in R1.W2, the 3M run does more as a label than as an input to the correction.

W4. Scope of the airport study, scoring regions, and theoretical assumptions

The 18 airport scenes cover the six stability classes, three per class, and small airports already have the buildings near which the balance departs from the simulator (L.418 and 282–1284). These scenes illustrate both behaviors in Prop. 2: at 3M, class medians show a gain in three classes and harm in the three most stable ones (L.421–424). On all ground cells outside buildings, with no concentration filter, the correction still lowers the summed error of the raw labels (line 485). When the screen decides how strongly to correct each label, including not at all, under a rule fixed in advance, this error falls from 9.09 to 6.31 times the 3M run's for NOx and from 9.55 to 5.72 for PM-1 (Table 4). The estimated covariances agree with independent runs: for every raw 300k label, the noise-adjusted squared error is 0.87 to 1.18 times the predicted variance, and above 9 m the predicted spread of the NOx residual is 0.88 to 1.11 times the measured one (L.1514–1518). Two controls also reduce or remove the reuse of scored runs: flat scenes still improve (L.404–405), and every airport class median keeps its sign (L.423–425). The 300k labels of Fig. 3 take their variances and trust from a separate 3M run and are scored against seven others (L.1506–1510).

Q2. Does following the screen lower total squared error?

Following the screen can have two interpretations, so we measured both on the deployment labels of Fig. 3. It can lead to a lower score when the screen sets how strongly to correct each label (rule of W4): on the selected cells, the summed squared error drops to 0.895 times that of always correcting for NOx and 0.805 for PM-1, with intervals below 1 (Table 4, L.486–488). It can lead to a higher score for a plain yes/no rule at 300k, the screen rejects one correction per pollutant and moves the error by only +0.26% and −0.55%, as 31 of 34 corrections already help (W2; L.1625–1630). Following the stored screen, computed above 9 m instead of on the label's cells, raises the error by 34.3% and 49.2% (L.470–472). Where corrections can harm, as at 3M (W4), the screen, averaged over runs, gets 33 of the 34 signs right, against 20 for always predicting improvement (L.460–463).

Thanks for adding more baselines and experiments. The screen rule seems to provide benefits in tuning correction strength, but with the same budget, weighted average of particles would provide more accurate labels, corrected labels do not show downstream benefits. I would keep my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.