acceptodds
Under review as a conference paper at ICLR 2027

Towards Autonomous Environmental Health: AERIS, a Research Agent for Air-Pollution Exposure Assessment

Abstract

Air pollution harms human health and contributes to millions of deaths each year, yet many public-health studies omit it even when they record the locations and times needed to estimate exposure. AERIS (Airborne Exposure Research & Investigation System) equips command-line agents with the tools and instructions to compute exposure from public air-quality measurements and study location data. To test the harness on a real-world, end-to-end, long-horizon research task, we introduce _source-substitution reanalysis_: AERIS rebuilds air-pollution datasets from public measurements, links them to published health outcomes, and reruns the statistical analyses. A structured literature search identified nine published studies as candidates for reanalysis based on open health-outcome data, sufficient methodological detail, and adequate air-quality coverage across the required locations and periods. Across these studies, AERIS processed 1.85 million air-quality monitor records into 20,568 area-time exposure estimates and 14 pollution–health comparisons across four pollutants. It matched the direction of all seven published positive associations; the median study-level difference between AERIS and published log effect ratios was 0.040. We then ran autonomous reanalyses as a robustness control and repeated them with the same model stripped of AERIS tools, instructions, and exposure engine as a harness ablation. Self-driving AERIS completed 23 of 27 attempts, compared with 7 of 27 for the raw agent, although its median log-effect difference rose to 0.169 without researcher review. AERIS turns air-pollution exposure assessment into a reviewable agent workflow and opens a path toward autonomous environmental-health research.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
scientific agents, agent evaluation, agent harnesses, long-horizon agents, autonomous research, AI for science, scientific reproducibility, environmental health, air-pollution exposure assessment, public health

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_MpEiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AERIS, a command-line research-agent harness for air-pollution exposure assessment in public-health studies. It combines a host CLI agent, MCP tools, a durable study workspace, and an exposure-computation layer that can retrieve public air-quality data, assign exposure to fixed locations.

Critical strengths
  1. Useful end-to-end evaluation framing.

  2. Real data and real workflow artifacts.

Critical weaknesses
  1. This paper focuses on a practical application. It is hard to tell where the novelty is. What is the scientific problem this paper targets on in the scope of ICLR?

  2. Evaluation is weak as there is comparison with SOTA methods.

Questions for authors
  1. What is the novelty of this work?

  2. Include comparison with competing SOTA methods.

1 comment
AuthorsOpenReview ↗

Response to Reviewer MpEi

We thank Reviewer MpEi for reviewing our work and noting the value of the end-to-end evaluation framing and real workflow artifacts.

  1. Scientific problem and novelty for ICLR: Existing LLM-agent benchmarks evaluate short-horizon question answering or isolated code execution on static files. Scientific research agents face a qualitatively different systems and learning problem: long-horizon workflow execution under real-world data incompleteness, where APIs fail, historical archives require multi-step recovery, and data substitutions silently corrupt downstream statistical estimands. AERIS introduces (i) a typed Model Context Protocol (MCP) exposure-assessment harness with explicit coverage and provenance invariants, and (ii) source-substitution reanalysis as an end-to-end evaluation paradigm that separates computational workflow completion from statistical calibration across three autonomy tiers (researcher-supervised, self-driving, and unaugmented agent ablation). (We note that a preliminary version of the researcher-supervised framework was recently peer-reviewed and accepted at a NeurIPS 2026 workshop in the AI-for-health and scientific discovery community, underscoring interest in auditable environmental-health agents; the present submission adds the full 27-run autonomous and raw-agent ablation campaigns.)
  2. Comparison with SOTA methods: Because no prior agent framework targets environmental-exposure reconstruction, we evaluated AERIS on a frontier coding agent (Codex CLI with GPT-5.6 Terra) against an unaugmented SOTA baseline given identical model weights, reasoning budget, network access, and task contracts (Table 1 and Appendix H). Adding the AERIS harness increases end-to-end reanalysis completion across three independent 9-study campaigns by 3.3×3.3\times (23/2723/27 vs. 7/277/27 attempts) while generating structured provenance manifests in 26/2726/27 runs (0/270/27 for the baseline).
Reviewer_vsw4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AERIS, an AI research-agent framework designed to automate air-pollution exposure assessment and the reanalysis of published public-health studies. Air pollution is a major public-health risk, but estimating exposure requires substantial data collection, processing, and scientific expertise. AERIS addresses this challenge by connecting an AI agent to specialized environmental-data tools to retrieve air-quality measurements, reconstruct exposure estimates, link them to health outcomes, and perform statistical analyses.

The authors evaluate AERIS by reanalyzing nine published environmental-health studies, covering 14 pollution–health comparisons. They also compare autonomous AERIS against a baseline AI agent without the AERIS framework. Autonomous AERIS completes 23 of 27 attempts, compared with 7 of 27 for the baseline, demonstrating a substantial improvement in workflow completion. With researcher supervision, AERIS reproduces the direction of all seven published positive associations, although only three reconstructed confidence intervals exclude the null. Furthermore, the median absolute log-effect difference increases from 0.040 with researcher supervision to 0.169 without supervision.

The results demonstrate that AERIS improves an AI agent's ability to complete complex environmental-health research workflows. However, they do not establish that the reconstructed exposure estimates are accurate or that the resulting statistical analyses are scientifically valid. In particular, agreement with previously published effect estimates is not equivalent to independent validation, especially when the reanalyses modify important aspects of the original studies, including study populations, exposure periods, and health outcomes. The paper therefore provides stronger evidence for computational workflow completion than for the scientific reliability of autonomous environmental-health research.

Critical strengths

1. Addressing an important and technically challenging problem.

The paper addresses a meaningful challenge in environmental-health research: reconstructing air-pollution exposure from heterogeneous public data sources requires substantial data collection, processing, and domain expertise. AERIS provides a unified framework that connects AI agents to specialized environmental-data tools, potentially reducing the technical effort required to retrieve, process, and document exposure estimates. This is a valuable contribution, particularly if the resulting exposure estimates can be independently validated.

2. Demonstrating improved research workflow completion.

The experimental evaluation provides evidence that AERIS substantially improves an AI agent's ability to complete complex, multi-step research workflows. Autonomous AERIS completes 23 of 27 attempts, compared with 7 of 27 for the baseline agent. This suggests that the specialized tools and workflow structure provide benefits beyond the underlying language model. However, this improvement primarily demonstrates computational completion rather than scientific correctness, which limits how strongly this result supports acceptance.

Critical weaknesses

1. Insufficient statistical rigor and scientific validation of the epidemiological reanalyses.

The paper extends beyond air-pollution exposure assessment to performing statistical analyses of health outcomes, but it does not provide sufficient methodological detail or validation to establish the scientific reliability of these analyses. In particular, the statistical models, confounding adjustments, estimation procedures, and underlying assumptions are not systematically documented. Furthermore, several reanalyses modify important aspects of the original studies, including study populations, exposure periods, and even health outcomes, raising concerns about whether the reconstructed analyses preserve the original scientific questions and estimands.

This is particularly concerning in environmental-health research, where statistical conclusions depend heavily on study design, appropriate confounding adjustment, and model assumptions. Producing a plausible effect estimate or reproducing a published association does not establish scientific correctness. Given the sensitivity of this research domain and the potential consequences of unreliable findings, I consider this a fundamental methodological weakness.

2. Lack of independent exposure validation and reliance on computational completion as a measure of success.

The authors primarily evaluate AERIS through its ability to complete published-study reanalyses and reproduce reported effect estimates, rather than independently validating the reconstructed pollution-exposure values against reference datasets, measurements, or established exposure-assessment methods. Agreement between downstream statistical results does not establish the accuracy of the underlying exposure reconstruction.

Moreover, although autonomous AERIS completes substantially more analyses than the baseline, its scientific agreement deteriorates without researcher supervision. Even with supervision, AERIS reproduces the direction of all seven published positive associations, but only three reconstructed confidence intervals exclude the null. This demonstrates that computational completion and directional agreement are not equivalent to reliable scientific conclusions.

In my opinion, the paper would be substantially stronger if it focused on accurate and reproducible exposure reconstruction, with rigorous independent validation, uncertainty quantification, and comparisons against established methods. Without such validation, the reported improvements in workflow completion are insufficient to support the paper's broader claims about autonomous environmental-health research.

Questions for authors

1. Statistical methodology and scientific validity (Clarification)

Pages 5–6, Section 4; Appendix E: The manuscript does not systematically describe the statistical methods used in the nine epidemiological reanalyses. Can the authors provide a study-by-study comparison of the original and reconstructed statistical models, including regression specifications, covariates, confounding-adjustment strategies, uncertainty estimation methods, and underlying assumptions? If causal interpretations are intended, what identification assumptions and methods support them? Without this information, how can readers determine whether the completed analyses are scientifically valid rather than merely computationally successful?

2. Preservation of the original research question and comparability of statistical results (Clarification)

Page 21, Table 6; Appendix F: Several reanalyses modify important aspects of the original studies, including study populations, exposure periods, geographic coverage, statistical specifications, and even health outcomes. How do the authors establish that these substitutions preserve the original estimands and scientific questions? Furthermore, some published statistical coefficients are transformed into multiplicative effect measures. For example, the Italian COVID-19 study uses a log-log regression model, yet its transformed result is presented as an odds ratio. The US stroke mortality study similarly converts a linear regression coefficient into a ratio-like measure. What statistical assumptions justify these transformations and comparisons? How can agreement between the reconstructed and published estimates be interpreted when the underlying models or estimands differ?

3. Independent validation of reconstructed pollution exposure (Clarification)

Page 5, Section 3.4; page 9, lines 445–450; Appendix D: The central contribution of AERIS is reconstructing pollution-exposure estimates, yet the evaluation primarily relies on agreement between downstream epidemiological results. Have the authors independently validated the reconstructed exposure concentrations against reference datasets, held-out monitoring measurements, or established exposure-assessment methods? In particular, Appendix D describes using an unweighted mean across monitoring stations for administrative-area exposure. How do the authors account for the nonuniform geographic distribution of monitoring stations, potential spatial bias, missing coverage, and exposure-measurement uncertainty? What quantitative evidence supports the accuracy of the reconstructed spatial and temporal pollution patterns?

4. Computational completion versus scientific correctness (Clarification)

Pages 6–8, Sections 4–5; Table 1; Appendix H: Autonomous AERIS completes 23 of 27 attempts, compared with 7 of 27 for the baseline, but its agreement with published results deteriorates without researcher supervision. Even with supervision, only three of the seven reconstructed confidence intervals for published positive associations exclude the null, and only four of seven published null conclusions are retained. How do the authors distinguish computational completion, agreement with published findings, and scientific validity? What criteria determine whether a completed statistical analysis is scientifically defensible? Could the evaluation reward an agent for producing a fitted but insufficiently supported result while penalizing a baseline agent for appropriately declining to perform an analysis when the available data or statistical assumptions are inadequate?

5. Potential evaluation bias and access to published results (Clarification)

Page 6, lines 309–323: The authors state that the agents do not receive the original pollution-exposure tables, but it is unclear whether they can access the published statistical effect estimates and conclusions through the supplied articles. Were these results hidden from the agents during the reanalyses? If not, how do the authors rule out the possibility that knowledge of the expected results influenced the selection of statistical models, covariates, exposure sources, or data substitutions? This is particularly important because numerical agreement with published estimates is a central evaluation metric.

6. Generalization and justification for autonomous epidemiological analysis (Clarification)

Page 9, lines 459–469; Sections 4–6: The authors acknowledge that the recovery policies were developed after observing failures in the researcher-supervised nine-study campaign, and the autonomous evaluation uses the same studies. What evidence demonstrates that these policies generalize to previously unseen studies rather than improving performance on a familiar set of research tasks? More fundamentally, what evidence supports extending AERIS from exposure reconstruction to autonomous epidemiological inference when the exposure estimates have not been independently validated and the statistical analyses may not preserve the original study designs? Would the current experimental evidence support the more limited claim that AERIS improves exposure-data reconstruction and research workflow execution, rather than the broader claim of reliable autonomous environmental-health research?

These questions are clarification requests concerning the evidence and validity of the reported contributions, rather than suggestions for future work. Their answers could materially affect my assessment, as my primary concerns involve the absence of independent scientific validation, the reliability of the statistical conclusions, and the distinction between completing an analysis and performing scientifically defensible research.

1 comment
AuthorsOpenReview ↗

Response to Reviewer vsw4

We thank Reviewer vsw4 for the constructive critique and address all six clarification questions below.

Q1 & Q2: Study-by-study models, estimands, and transformations (in Study reanalyses/; all estimate observational associations, not causal effects):

  • France [source_substitution]: Neg. binomial GLM of hospital deaths (May 1, 2020) with log(pop) offset on 63/96 depts (>=75% 2019 OpenAQ vs. 1999–2016 satellite PM2.5); covariates: temp, %60+, %female, density, diabetes, ICU beds. RR exp(b) per 1 ug/m3: 1.244 (pub) vs. 1.049 (0.953–1.153).
  • Italy Cities [source_substitution 8-city / bounded_proxy 46-city]: OLS log-log model of log(cases Apr 7, 2020) on log(2018 PM10 exceedance days), adjusted for log(density). Elasticity b (0.813 vs. 0.800, 8 cities >=75% cov) as outcome ratio per doubling R=2^b: 1.76 vs. 1.74 (1.13–2.69); 46-city 250 km radius: 1.26 (1.01–1.56). Table 6 OR = Outcome Ratio 2^b, not logistic OR.
  • OECD [source_substitution]: Log-linear OLS of case/death rates per 100k at days 60 & 80 on 2017 PM2.5 across 11/36 countries (>=65% OpenAQ/S3 coverage), adjusted for %age >=65, density, GDP/capita. RR exp(b) per 1 ug/m3 across 4 endpoints: 0.971–1.016 vs. 1.009–1.030.
  • UK [bounded_proxy]: Nation-FE model of daily COVID-19 CFR (Mar–Sep 2020, 635/696 nation-days) with OpenAQ polygon PM2.5, nation FEs, ECA&D temp & sunshine. Ratio per 10 ug/m3: 1.13 vs. 1.46 (1.21–1.76).
  • N. Italy [bounded_proxy]: Regional daily incidence/mortality log-linear panel in Lombardia & Veneto across 14 complete NO2 days (Feb–May 2020). RR per 1 ug/m3 NO2 (excluded from continuous metrics; published nulls rounded to 1.00).
  • São José [bounded_proxy]: Neg. binomial daily series of COVID-19 admissions (89 complete days) using SIVEP-Gripe fallback (vs. 395d SIH/SUS), co-adjusted for O3 lag2, NO2 lag3, PM10 lag0, temp, humidity, weekday, trend. RR exp(10b) per 10 ug/m3.
  • São Paulo AMI [bounded_proxy]: K-means cluster contrast of DATASUS I21 AMI admissions (50,826 admissions, 39 municipalities) on 193 CETESB NO2+weather days + bootstrap 95% CI. Cluster ratio: 1.090 vs. 1.069 (1.006–1.135).
  • Brazil Cancer [bounded_proxy]: Time-stratified case-crossover (conditional logistic) matched on weekday & month, adjusted for 3-day lag0–2 temp, humidity, holidays. 5-state IEMA panel (929,790 deaths, 1,824d) and SP 2019 sensitivity (11,145 deaths; OR exp(10b)=1.006 [0.960–1.054] vs. pub 1.01).
  • US Stroke [bounded_proxy]: County OLS with HC3 SEs on 46/3,022 counties (50% 2019 OpenAQ completeness), using CDC Atlas all-stroke mortality (65+) and poverty, park access, demographic covariates. Native slope: 3.01 vs. 1.89 deaths/100k per 1 ug/m3 (95% CI -10.1 to 16.1). Mean-normalized (1+10b/Y_bar) only for Table 6 plotting; excluded from strict calibration.

Q3: Independent exposure validation Original studies often use satellite surfaces (1999–2016 PM2.5 in France), whereas AERIS uses public ground stations (OpenAQ), logging station counts, spread, and completeness rather than imputing missing rural coverage. Against author-deposited exposure tables (Italy/France Mendeley files), monitors matching the urban definition (Italian capitals) track the published ranking; replacing a 1999–2016 satellite surface with sparse 2019 monitors (63 French depts) attenuates exposure variance and slope (1.049 vs. 1.244).

Q4: Completion vs. scientific validity (and abstention) Fitting a degraded proxy must not be rewarded over abstaining: (1) Completion (23/27 vs. 7/27) measures pipeline execution; (2) in Table 8, only 2/9 autonomous runs (France, OECD) are direct Fits, 6/9 are Bounded fit, and 1/9 (São Paulo) is an Evidenced stop; (3) in Appendix A.5, AERIS abstained (No fitted result) on 5/14 eligible studies (China, Israel, Argentina, Taubaté, Alaska) for inadequate coverage/variation.

Q5 & Q6: Blinding, generalization, and scoping Blinding (Q5): The agent read article text for specifications, so published estimates were visible. Transcripts show a single forward pass without tuning against target numbers; future benchmarks should redact Results/Discussion. Generalization & Scope (Q6): Autonomous AERIS had lower error on the 6 held-out studies (median |dlog R| = 0.129, 9/12 positive directions) than on the 3 freeze-check sentinels (0.352, 2/4 positive directions). We agree completely with your framing: removing researcher review quadruples median log-effect error (0.040 -> 0.169) and collapses correlations (0.507, 0.091, -0.170)—evidence against unsupervised epidemiological inference. Our results support the narrower claim that AERIS improves exposure-data reconstruction and auditable workflow execution under researcher oversight (whose domain utility was recently validated via acceptance of the preliminary supervised framework at a NeurIPS 2026 AI-for-health/science workshop).

Reviewer_s1P9OpenReview ↗
Rating
2
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AERIS gives command-line coding agents, including Codex and Claude Code, tools, instructions and a workspace to estimate air-pollution exposure from public data. It mainly uses OpenAQ, plus Google and AccuWeather APIs and an XGBoost PM2.5 model for GPS traces. It supports fixed locations, trajectories and administrative areas. Authors evaluate it through “source-substitution reanalysis”: rebuild exposure estimates for a published pollution-health study using public sources, join them to public health outcomes, refit the model and compare results with the published effect. Screening 18,919 PubMed and OpenAlex records led to 21 candidates, 14 attempts and 9 fitted reanalyses. These cover six COVID-19 studies, São Paulo heart-attack admissions, Brazil cancer mortality and US stroke mortality. All reanalyses use administrative areas.

With researchers reviewing the agent’s choices, all nine were fitted. The sign of all seven published positive associations was recovered (three with CIs excluding 1), four of seven null conclusions were retained, and the study-level median absolute log difference from the published ratios was 0.040 (r = 0.834, driven by one study).

Critical strengths

SC1) Evaluation tests whether agents can rebuild an analysis

Rebuilding and refitting a published analysis tests more than isolated tool calls. The paper also reports failures, data substitutions and agent traces (Tables 6 to 9). Screening shows how hard usable public data is to find: of 4,658 open-access papers, only 138 linked to a public dataset and only 3 passed field-level validation (A.2).

SC2) Sensible engineering, with a real test suite

Typed MCP tools, a durable workspace with run manifests, typed errors instead of fabricated values, no hidden completeness cutoff (Table 3), and provenance on every estimate are the right design choices.

Critical weaknesses

LW1) Agents and reviewers can see answers, and some test studies were used for tuning

The agent can read published results. Telling it not to use those numbers does not prevent them from influencing its choices. Researchers also knew published results when judging agreement (Table 7). The freeze log shows that three of nine evaluation studies were used to revise the harness. Revisions were checked against published results before the setup was frozen. So these studies were used for both tuning and testing.

Authors should hide published results from agents and reviewers until analyses are fixed, set comparison rules in advance, and report the three tuning studies separately or replace them.

LW2) Several reanalyses change the original study, and the main table leaves out a closer attempt

The paper says AERIS preserves the research question and analysis, but several examples do not. UK uses a simpler model that its own code says is not a replication. US stroke changes the outcome and uses only 46 of 3,022 counties. São Paulo uses fewer pollutants over 193 days instead of four years. Brazil assigns one daily pollution average from five states to all deaths. Both Brazil and São Paulo use other data sources after AERIS/OpenAQ falls short. For Italy, Table 6 reports 8 cities with an estimate of 1.74, close to the published 1.76. A 46-city run gives 1.26 but is left out. Including it drops overall correlation from 0.83 to 0.39.

Authors should state how closely each result follows the original study, show how much of its sample and time period was recovered, and include Italy’s 46-city result in the main table.

LW3) Supplement results disagree with Table 5

The supplement’s agreement file reports 5 of 8 results with the same direction, using the authors’ near-null rule. It treats France as near-null and excludes US stroke because the outcome differs. Table 5 counts both toward 7 of 7 agreement. The strict three-study comparison gives mean absolute log error of 0.145 and correlation of 0.70 (p = 0.51), but the paper reports only the broader comparison. That includes two studies whose own result files say they should be excluded. France and UK also have no confidence-interval overlap with published effects. OECD’s retained null says little with only 11 countries and four covariates.

Please report the strict metrics, define agreement in advance (CI overlap or an equivalence bound), and compare reconstructed exposure panels directly against the articles’ exposure data, which the authors hold.

LW4) Main use cases are not evaluated, and some runs break the harness rules

The introduction focuses on addresses and GPS traces, but all nine reanalyses use administrative areas. Address and trajectory tools only have implementation tests. The packaged XGBoost model has no training data description or accuracy results, and AERIS exposure estimates are never checked against reference data. The skill file also bans using centroids for area-level monitor selection. Yet US stroke uses county centroids, and autonomous UK runs use one monitor per nation or capital coordinates. Authors should test address and GPS estimates against reference data, document the model, and explain why these runs break their own rules.

Questions for authors

Critical Weaknesses sections.

1 comment
AuthorsOpenReview ↗

Response to Reviewer s1P9

We thank Reviewer s1P9 for a thorough audit of our manuscript and supplement. Every figure noted comes from diagnostic and sensitivity files included in our supplement; we address each point below.

LW1: Blinding to published results and the 3 sentinel studies in the freeze log

  1. Article text access: The agent reads the article to extract the study design (pollutant, geography, period, covariates, model) while exposure tables are withheld. Because the text states published effect sizes, unredacted text does not guarantee blinding. In autonomous clean-room runs, the agent ran under a fixed contract (briefs/*.json) and logged every tool call and fit; transcripts show a single forward pass without specification shopping. Future benchmarks should redact Results/Discussion sections.
  2. Role of the 3 sentinel studies (AERIS_SELF_DRIVING_FREEZE.md): The 3 sentinels (São Paulo AMI, Italian cities, OECD) were pre-freeze container/handoff checks for the autonomous runner, not tuning runs for the primary researcher-reviewed evaluation. Splitting the autonomous results into the 3 sentinels vs. 6 held-out non-sentinel studies shows that autonomous performance is stronger on the 6 held-out studies:
    • 6 non-sentinel studies (held-out): 9/12 positive directions retained; median study-level ∣Δlog⁡R∣=0.129|\Delta\log R| = 0.129 (n=10n = 10 study-runs).
    • 3 sentinel studies: 2/4 positive directions retained (11/12 OECD nulls retained); median study-level ∣Δlog⁡R∣=0.352|\Delta\log R| = 0.352 (n=7n = 7 study-runs).
    • Primary evaluation (researcher review): Median ∣Δlog⁡R∣=0.020|\Delta\log R| = 0.020 (3 sentinels) and 0.0700.070 (5 comparable non-sentinels).

LW2: Study fidelity, public-source fallbacks, and the Italian 46-city run

  1. Fidelity and fallbacks: Public monitors (OpenAQ) and portals often lack historical coverage for a study’s full spatial frame or period. Table 6 and RESULT.md record every reduction and fallback (46/3,022 US counties passing the 50% completeness gate with CDC Atlas all-stroke mortality; 193 complete NO2+weather days in São Paulo; CETESB/IEMA archives when OpenAQ lacked historical Brazilian data). We will add an explicit fidelity tier (source_substitution_fit vs. bounded_proxy_fit) and recovered unit/day fractions to every row of Table 6.
  2. Italy 8-city vs. 46-city run: In 33398753-italy-cities/RESULT.md, the primary 8-city run (R=1.74R = 1.74, 95% CI 1.13–2.69 vs. published 1.76) uses local monitor assignment (≥75%\ge 75\% daily coverage). Because 2018 OpenAQ PM10 coverage was sparse, we also deposited a 250 km radius sensitivity (fit_full_coverage_effect.py) capturing 46/55 cities (R=1.26R = 1.26, 95% CI 1.01–1.56). Assigning monitors up to 250 km away introduces spatial measurement error that attenuates the slope while retaining a significant positive effect. Both the strict-radius (8-city, 1.74, r=0.834r = 0.834) and expanded-radius (46-city, 1.26, r=0.39r = 0.39) results will appear side by side in Table 6.

LW3: Reconciling the supplement’s strict-subset JSONs with Table 5 The difference between Table 5 and direction_agreement.json / effect_calibration_metrics.json reflects two aggregation rules in the supplement:

  • Strict 3-study subset (effect_calibration_metrics.json, n=3n = 3: UK, Italy, France): Restricts to positive studies preserving the primary outcome and regression without outcome/cluster proxies. Raw directional agreement is 3/3 (2/3 exceeding the ±5%\pm 5\% band [0.95,1.05][0.95, 1.05], as France is 1.049), mean ∣Δlog⁡R∣=0.145|\Delta\log R| = 0.145, geometric mean ratio is 1.025, and r=0.70r = 0.70 (p=0.51p = 0.51).
  • All fitted contrasts (direction_agreement.json vs. Table 5): direction_agreement.json scores 8 contrasts against [0.95,1.05][0.95, 1.05], yielding 5/8 (classifying France’s 1.0485 as near-null and excluding US stroke’s outcome proxy). Table 5 scored raw sign agreement across all 7 positive contrasts (7/7, with 3/7 CIs excluding 1) and CI inclusion of 1 across 7 null contrasts (4/7). We will report both sets of metrics in Table 5.

LW4: Scope alignment, XGBoost documentation, and rule violations

  1. Area vs. address/trajectory pathways: Because public health datasets without DUA/IRB gates are area-aggregated, all 9 reanalyses use the area pathway; address/trajectory tools are tested via 250 unit tests and a 24h NYC trajectory test (Appendix B, lines 951–973, documenting the 600-tree, 14-feature XGBoost model extending Lu et al. 2021). We will narrow the Introduction.
  2. Rule violations: When polygon sweeps hit API limits across >3,000 US counties or UK endpoints, the agent bypassed estimate_boundary_exposure via custom Python scripts using county centroids or nation coordinates. We agree these runs violate the skill rule and should be labeled unsupported_fit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.