- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces ArchEGraph, a dataset aligning building geometry, spatial topology, weather data, and zone-level hourly external thermal loads. It contains 5,481 building models and 49,326 EnergyPlus simulation cases generated under fixed thermal and operational settings to study how geometry and weather affect thermal loads. The authors define two benchmark tasks: reconstructing building topology from geometry (M2G) and predicting thermal loads from building graphs and weather sequences (G2E). They provide baselines and evaluation protocols covering task performance, generalization across buildings and climates, and a topology ablation that shuffles face–space assignments.
Critical strengths
- The paper is clearly written, with well-designed figures that explain the data construction pipeline, graph representation, and task definitions. This makes the methodology easy to follow and assess.
- The dataset aligns building geometry, spatial topology, weather, and zone-level hourly thermal loads, with benchmark tasks and evaluation splits. This provides a reusable resource for studying geometry- and weather-driven thermal loads and comparing methods under consistent conditions.
Critical weaknesses
- The dataset’s distinct value and practical significance are not sufficiently established. Its contribution mainly involves integrating and scaling existing geometry processing, graph representation, and simulation workflows, with envelope constructions, HVAC settings, internal loads, and schedules held fixed. The two benchmarks do not adequately demonstrate what new research or applications this controlled resource enables beyond existing modeling and simulation workflows, leaving the significance of the dataset contribution insufficiently justified.
- The advantage of graph modeling over conventional methods is not sufficiently demonstrated. WeatherMLP uses only weather inputs, so comparisons against it conflate the benefits of additional geometric information with those of graph structure. The topology-shuffling ablation shows that correct face–space assignments help the current graph model, but the reported evaluation still lacks non-graph baselines using both geometry and weather, such as gradient-boosted trees or an MLP incorporating zone volume, surface area, orientation, and window-to-wall ratio. Without these comparisons, the additional benefit of graph modeling remains unclear.
Questions for authors
- Please specify the dataset’s intended use case and demonstrate its benefit over existing automated modeling and EnergyPlus simulation workflows through a concrete application. If the main benefit is speed, report simulation, training, and inference costs, and clarify the scope of these benefits under the fixed thermal and operational settings.
- Please report non-graph baselines that use both geometry and weather. The public repository already lists GeometryHistGBR, DegreeHourRidge, and RCInspiredRidge, but their results are not reported in the paper. Compare them with graph models using comparable inputs and identical splits, including cross-building and cross-climate tests, to establish the additional benefit of graph modeling.
- Please clarify the IDF generation pipeline in Appendix A.2.4: are IDF files generated directly from the released graph files, or do they depend on an internal building model that retains complete geometry? Provide a reproducible example showing how geometric fields, space assignments, and semantic attributes map to IDF objects. List any additional dependencies, such as raw meshes and fixed templates, to substantiate the claimed loss-preserving mapping.
- Please clarify whether G2E uses graphs obtained through preprocessing or graphs predicted by M2G. If the former, report downstream results using M2G predictions. In Table 3, the best face–space incidence F1 scores are approximately 0.43 and 0.25 on the P and M subsets, respectively. Quantifying how these reconstruction errors affect load prediction is necessary to assess the two tasks as a combined pipeline.
- Please explain how graph nodes are matched to thermal-load labels. Appendix A.2.5 describes truncation or zero-padding when dimensions differ, but matching tensor shapes does not ensure correct zone correspondence. Report the number of affected samples, the matching rules, and whether padded entries are masked during training and evaluation.
- Please provide the exact procedure for computing external thermal disturbance load labels, including the EnergyPlus output variables, sign conventions, temporal aggregation, and area normalization. The load decomposition formulas in the appendix are insufficient to reproduce the labels or verify that they represent the physical quantity defined in the paper.
- Please check the units, inverse normalization, and aggregation of the metrics in Table 4. For example, WeatherMLP has an MAE of 8.845 and an RMSE of 6.694 on the P subset. Under the appendix’s definitions, MAE should not exceed RMSE when both use the same units and weights. Please explain this inconsistency and correct the relevant results.
- Please clarify the split used for cross-climate evaluation. Section 5.3 states that training and testing use disjoint ASHRAE climate zones, whereas the public repository specifies disjoint weather location IDs and explicitly notes that climate zones may overlap. Identify the protocol used for Table B.6, list the locations and climate zones in each partition, and reconcile the descriptions in the paper and code.