acceptodds
Under review as a conference paper at ICLR 2027

GHGbench: Benchmarking Greenhouse Gas Emissions Prediction Across Companies and Buildings

Abstract

Annual greenhouse-gas (GHG) disclosures support climate accounting, yet coverage remains incomplete and reporting formats vary across companies, buildings, and jurisdictions. Prediction can support gap analysis when reports are missing or delayed. Measured accuracy depends on which entities, inputs, regions, and years appear in training and test data. Existing studies use different sources and evaluation designs, making model gains hard to compare and performance beyond the training distribution hard to assess. We introduce GHGbench, an open benchmark with separate company and building tracks and a shared evaluation framework. The company track contains 32,830 company-year records from 12,087 companies. The building track harmonizes 491,591 building-year records from 12 public programs across 26 metropolitan areas. Fixed tests cover random annual company records, unseen buildings, held-out regions and cities, later years, forecasting, and multimodal models that combine tabular inputs with satellite-image embeddings. Across both tracks, input availability and distribution shift have larger effects than small differences among leading models. Company-track R² rises from about 0.3 with basic metadata to about 0.9 with company attributes. For buildings, leading models reach about 0.4 R² on unseen buildings when all cities are represented during training; city-average R² falls to about 0.1 when each test city is held out entirely. Multimodal inputs improve average new-city performance, with gains varying across cities. GHGbench releases fixed splits, target provenance, per-record predictions, and reconstruction and validation tools.

Primary area:
datasets and benchmarks
Keywords:
Greenhouse Gas Emissions, Carbon Emissions Prediction, Benchmark Dataset, Tabular Machine Learning, Climate Informatics

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_Cw7zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces GHGbench, a benchmark for predicting annual greenhouse-gas emissions with a company track of 32,830 company-years and a building track of 491,591 building-years from 26 cities. It fixes splits for gap filling, unseen buildings, held-out cities, later years, forecasting and satellite-image inputs, and it compares many tabular, foundation and LLM baselines. It finds that input access and distribution shift matter more than model choice, as company R² rises from about 0.3 to 0.9 with financial attributes and building R² falls to about 0.1 in new cities.

Critical strengths
  1. The building release is substantial and auditable. It harmonizes 491,591 records from 12 public programs in 26 cities with frozen snapshots, row-level provenance and quality flags (Sections 3.2 and 3.3).

  2. The protocol states what each score measures. Matched-record regimes separate input access from sample composition, and calculated targets such as Singapore's are kept apart from reported ones (Section 4.3).

  3. The results are reported with care. Repeated runs and whole-entity paired bootstraps are given, and the paper states openly that TabPFN and tuned trees are mostly indistinguishable (Appendix P).

Critical weaknesses
  1. The geographic headline mixes two changes. It compares pooled R² on unseen buildings (0.378 to 0.395) with city-average R² under city holdout (0.098 to 0.117), so the averaging and the population change together.

  2. The company task lets one firm appear in both training and test, with no baseline that reuses its own disclosures. Persistence alone reaches R² 0.959 (Table 13), so models may win by recognizing the firm.

  3. Some comparisons break the paper's own rules and the release is incomplete. Forecasting XGBoost uses 386 records against 1,930 for the others (Table 13), and Appendix C omits reconstruction counts and mappings.

The building resource is useful and well curated. However, the two headline findings are confounded by the split and averaging choices, and it is unclear how much demand exists for this benchmark. A firm-held-out split with a persistence baseline and a same-population geographic comparison would move me to weak acceptance.

Questions for authors
  1. Could the authors report the geographic drop when both settings use the same cities, records and averaging?
  2. Could the authors compare R² with company attributes against reusing each firm's previous disclosure, and also report results under a firm-held-out split?
  3. Would it be possible to release a frozen copy of the company panel so that the fixed splits can be rebuilt exactly?

Suggestions that did not affect my score. It would help to report log-scale R² next to raw R² because raw R² is dominated by the largest buildings, and to add a paired significance test for the multimodal gain in the appendix.

2 comments

Thank you for the careful responses. I am glad to see how seriously you treated my comments (rather than relying on AI to produce a generic rebuttal). The new experiments address my main concerns, and I appreciate that you report the results that weaken the original headlines. I have therefore raised my rating to weak acceptance.

In addition, I would like to help make the paper better, so here are a few suggestions for the final version:

  1. It would be great to make the unseen-company split a fixed benchmark task and report it in the main table.
  2. I think it would be better to state in Section 5 that the city-paired tests for the multimodal gains are not significant.
  3. It may be better to discuss the stability of city-average R² for small cities, since it is negative for most models in Table 16.

I hope these are helpful.

Thank you for the clear plan and for taking my suggestions on board. I look forward to seeing the final version, and I wish you the best of luck with the paper.

Reviewer_1ed2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for greenhouse gas (GhG) prediction covering 30k+ company-years and almost 500k building-years in 26 cities. This benchmark, named GHGBench defines evaluation for unseen buildings, greographical and temporal transfer, property-type transfer and augmentation using satellite images and embeddings. The benchmark shows tree ensemble results, alongside neural tabular models, LLMs, and tabular and time-series foundational models.

The results of the benchmark shows that input availability and distributional shift matter, while differences among comparabily competitive models are not as important. Using company attributes improves prediction, buildign performance worsens under new-city inference and embeddings tend to improve new-city performance with city-level effects showing heterogeinity.

Critical strengths
  • The paper tackles a timely and relevant problem in a robust manner.
  • The paper introduces a useful and reproducible resource for research at the intersection of climate policy, economics, and machine learning. The harmonization pipeline, provenance policy, quality flags, and deterministic processing all address together a real fragmentation problem, including the separation of calculated and reported GHG emissions.
  • The evaluation is broad, and I found the negative results particularly informative. The paper interestingly separates unseen-building and annual-record splits, including bootstrapped comparisons and reports inconclusive model differences and limited benefits in multimodal inputs. These moer negative results make the benchmark more useful than a single winner-takes-all leaderboard.
Critical weaknesses
  • Several headline comparisons somewhat overstate the claimed effects. For example: “Unseen buildings 0.4 to new cities 0.1” compares pooled R2 with an equal weight city average, maening that the metrics differ so part of the drop depends on the change of the metric itself and therefore it's not a robust result. Also, new-city inference uses a limited set of ML models, and misses some of the best perfoming models like catboost and tabm.
  • The company-year split places the same company in train and test. Many economic and financial metrics are mostly unique per firm and change slowly, so tree models can effectively become a look up table for firms.
  • Hyperparameters are tuned once with different budgets and using different validation wrt the reported headline metrics. This should be explained further and the authors could elaborate on the limitations of this approach.
  • While useful, the dataset could be broader in scope and geographcial representativeness. Additional embedding models could have been reported for more comprehensive analyses.
Questions for authors
  • How would this model benefit from other type of embeddings, like climate-based embeddings, economic embeddings like the popualtion dynamics foundation model or Terranova, or alphaearth/satclip/copernicus-embed/geoclip?
  • Could the later-year evaluation have been confounding time with geography? Several cities have a single year or sparse years, so they may enter only the test set.
2 comments
Reviewer_1ed2OpenReview ↗

Response to rebuttal

I thank the authors for their very detailed responses and additional experiments. The rebuttal strengthens my assessment, and I am leaning towards raising my recommendation from weak accept to accept. I look forward to the comments of the other reviewers.

Thank you for your follow-up and for your positive assessment of our responses and additional experiments. Your detailed comments helped us strengthen both the evaluation and its presentation, and we will reflect them carefully in the final version. We greatly appreciate the time and care you have put into reviewing our work.

Reviewer_gPAEOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GHGbench, an open benchmark for GHG emissions prediction across companies and buildings. By integrating multiple public data sources, the authors construct two separate tracks comprising 32,830 company-year records and 491,591 building-year records, respectively. The benchmark provides harmonized data, provenance tracking, and fixed evaluation protocols to improve reproducibility and comparability. It covers disclosure gap filling, unseen-building prediction, geographic and temporal transfer, next-year forecasting, and multimodal evaluation, with extensive comparisons across traditional machine learning, neural tabular, and foundation models.

Critical strengths
  1. The paper integrates large-scale company and building-level emissions data, harmonizes heterogeneous sources, and provides provenance information, quality flags, and fixed data splits. These efforts provide a useful foundation for reproducible research.
  2. The evaluation protocol covers diverse input regimes and generalization settings across entities, geographies, and time. Experiments across a broad range of model families provide useful insights into how input availability and distribution shift affect emissions prediction performance.
Critical weaknesses
  1. Cross-source data comparability could be further clarified. Although the benchmark includes careful data harmonization and quality control, differences in emissions accounting conventions, reporting practices, and target definitions may remain across jurisdictions. A more explicit discussion of how these differences are handled and their potential implications for cross-regional evaluation would strengthen the work.
  2. The representativeness of the company gap-filling protocol could be further discussed. The current task splits observed company-year records, whereas real-world non-disclosure may be associated with company characteristics such as size, industry, and geography. Moreover, records from the same company may appear in both training and test sets. While this design is appropriate for partially observed panel completion, it does not directly evaluate generalization to unseen companies.
Questions for authors
  1. Prior studies by Li et al. (2024) and Yap et al. (2025) have investigated multi-city building emissions prediction and generalization. While GHGbench expands data coverage and evaluation settings, its distinct contributions relative to these studies could be more clearly articulated. It would be helpful to clarify the added value of the proposed data harmonization, evaluation protocols, and empirical findings.
  2. The data harmonization process addresses source-level inconsistencies, but differences in emissions accounting conventions and reporting practices may remain across jurisdictions. How are these differences handled, and to what extent might unresolved discrepancies affect target comparability and cross-regional evaluation?
  3. The company gap-filling task splits observed company-year records, whereas real-world non-disclosure may be associated with company characteristics such as size, sector, and location. Furthermore, records from the same company may appear in both training and test sets. Could the authors clarify the applicability of this protocol to real-world missing-disclosure scenarios and discuss whether alternative missingness settings or company-held-out evaluation would provide complementary evidence?
  4. Satellite-image embeddings improve the average city-level R² under new-city evaluation, but improvements are observed in only 7–11 of the 22 cities. It would be useful to clarify whether the average gains are driven by a few cities and discuss the factors underlying this variation.
  5. Some input features, such as EUI, electricity consumption, and gas consumption, are closely related to the emissions target, while their availability varies across practical applications. Could the authors clarify the intended use cases of the different input regimes and the benchmark's applicability to downstream emissions prediction? Additionally, a brief discussion of long-term versioning and maintenance would be helpful, as the company track relies on external CDU services for reconstruction.
  6. The paper compares an overall R² of 0.378–0.395 for unseen buildings in represented cities with a city-average R² of 0.098–0.117 under new-city evaluation. Since these metrics use different aggregation schemes, could a comparison using consistent aggregation be provided to better quantify the impact of geographic transfer?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.