- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
GCAM-Gym provides an interface for climate policy search and compares seven systems. It records policy trajectories and failures, but the paper also identifies runtime problems that leave the experimental conclusions unresolved.
Critical strengths
The attempt level records make the experiments easy to inspect. The authors retain failed runs and openly document the runtime defects.
Critical weaknesses
-
The measurements have not been validated. Section 5.4 shows that changing a checking path shifts one policy’s temperature from −0.6625°C to 1.964°C. It is unclear whether the reported method rankings would survive a corrected evaluation.
-
Independent reproduction remains difficult. The archive lacks some artifacts needed for native execution, and the temperature reference and cost units still need validation.
Questions for authors
Can you rerun the adaptive searches under a validated runtime and show which conclusions remain supported? This would be necessary for me to reconsider the recommendation.