acceptodds
Under review as a conference paper at ICLR 2027

GON: Generalizable Outcome Networks for Learning Action Effects in Physical Systems

Abstract

Pretrained control models such as Decision Transformer, Gato, RT-1/RT-2, and OpenVLA learn policies from large collections of observed trajectories to generalize across tasks, environments, and operating conditions. However, when control actions take time to affect a physical system, external conditions can change the system during the same period. We study this problem in smart buildings, where indoor temperature is affected by both heating, ventilation, and air conditioning (HVAC) control and external conditions such as weather, sunlight, and occupancy. A recorded temperature change alone does not show how much was caused by HVAC control. As a result, a controller trained on recorded trajectories may not reliably learn what its actions do, leading to higher energy use or reduced comfort when conditions change. To address this problem, we design GON (Generalizable Outcome Networks), which learns how sequences of control actions affect a physical system over time. During training, GON simulates different HVAC action sequences from the same building state, keeping external conditions the same across the simulations. Comparing the resulting temperature trajectories shows how the building's response changes when the action sequence changes. An outcome model uses recent building history and a proposed HVAC action sequence to predict temperature and energy use. Given a target temperature trajectory, GON uses conditional flow matching to generate several candidate HVAC action sequences. At deployment, GON scores each sequence based on predicted discomfort and energy use, applies its first action, and repeats the process after observing the building. We evaluate GON on 100 unseen buildings using a single frozen controller without target-building adaptation. Compared with Decision Pretrained Transformer, GON reduces energy use by 34% and comfort penalty by 43% on the comfort-focused task. On the energy-focused task, it reduces energy use by 18% and comfort penalty by 43%. GON performs comparably to model predictive control (MPC) configured separately for each building.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
generalizable control, physical systems, action effects, conditional flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_EUx5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes the GON (Generalizable Outcome Networks) to address the control-latency issue that pretrained control models, such as Decision Transformer, Gato, RT-1/RT-2, and OpenVLA, are facing. Detailed commnets are outlined in Critical Weaknesse.

Critical strengths

The paper is trying to address the control-latency issue that pretrained control models, such as Decision Transformer, Gato, RT-1/RT-2, and OpenVLA, are facing.

Critical weaknesses
  1. The paper is more suitable for a power/energy conference, like IEEE PES, since it does not have a contribution to ML, and the experimental examples focus only on HVAC.
  2. The experiment was performed on a simulator/simulation, rather than real physical systems. The obtained conclusion has some underlying assumptions that may not hold in reality. One example is that the trained model via the encoder will face model mismatch since physical operating environments are usually complex and non-stationary. As a result, the results will face the challenging Sim2Real gaps.
  3. The paper is hard to follow; some critical information is missing. For example, what are the Decision Transformer, Gato, RT-1/RT-2, and OpenVLA? What are the core technical building blocks?
  4. The paper claims that pretrained control models such as Decision Transformer, Gato, RT-1/RT-2, and OpenVLA learn policies face control latency. The root reasons can be the chosen approaches for HVAC; others, such as adaptive controllers, can be much better.
  5. The experiment is a single simulated HVAC; it cannot demonstrate its generalization capability.
  6. To handle control latency issues, as well as model mismatch and distribution shift, the approaches of adaptive models and adaptive controllers in the control community have been proposed. The paper lacks such a literature review and comparisons.
  7. The title has the keyword "Learning Action Effects in Physical Systems", implying learning in real physical systems operating in the real physical world. Achieving physical learning in the physical world will be a great contribution. However, all the experiments are done in virtual simulators.
  8. What kinds of simulators are used in experiments?
Questions for authors

See Critical Weaknesses.

1 comment

Thank you for the review and for the time spent on it. We appreciate the opportunity to clarify several aspects of the paper, and we address each point below.

Control latency. We would like to clarify that the paper does not study control latency; it does not make that claim, and the word "latency" does not appear in it. The problem we study is separating what the HVAC action did from what the outside conditions did. An HVAC action takes hours to have an effect, and weather, sunlight, and occupancy change the temperature during the same time, so a recorded trajectory cannot show how much of the change came from the action (Abstract; Section 1, paragraph 3; Section 3.1). Since the review's summary, its listed strength, and its suggestions about adaptive controllers are framed around latency, we hope this clarification helps in re-evaluating those points.

"The experiment is a single simulated HVAC." We respectfully note that the evaluation covers many buildings. GON is trained on 800 buildings and tested as one frozen model on 100 different unseen buildings across all eight ASHRAE climate zones, with no adaptation (Section 4.1, Tables 1 and 3, Figure 3). We believe this is the generalization test the review asks for. The buildings come from the OfficeSmall family (Appendix A.1.1) and differ in thermal dynamics and climate.

"What are DT, Gato, RT-1/RT-2, and OpenVLA?" These are widely used pretrained control models in machine learning. Each is cited and briefly described in Section 1 (paragraph 2), and Decision Transformer is discussed again in Section 2.1. GON's building blocks are given in Section 3, Figure 2, and Table 4. We have expanded these descriptions in Section 1.

"What simulators are used?" We use EnergyPlus [1], through the Building2Building benchmark [2], as stated in Section 1, Section 4, and Appendix A.1.1. EnergyPlus is the U.S. Department of Energy's building simulation engine and is widely used in building research and design. Building2Building was built specifically to test whether learned controllers generalize to new buildings, which is exactly what we evaluate.

Simulation and the title. Building2Building is our testbed. It was available to us, it was designed for this kind of evaluation, and it lets us test one controller on 100 unseen buildings under the same conditions as every baseline. It does not mean GON only applies to buildings. The method only needs matched rollouts during training, and the same idea applies to other physical systems where actions act slowly while outside conditions keep changing the state. By "physical systems" we mean systems governed by physical dynamics. We agree that real-world deployment would be a valuable next step; the paper does not claim it, and we see sim-to-real transfer as an important but separate question from the one we answer. The abstract now states that evaluation is in EnergyPlus.

Adaptive control. Thank you for this pointer. Adaptive controllers update a model online on the target building. We study zero-shot transfer: one frozen model, no updates on the test building. We see these as different, complementary settings, and we have added a short discussion of adaptive control to the related work (Section 2.4). Our MPC baseline is configured separately for each building, and GON's energy use is within 5% of it while using a single model for all 100 buildings.

ML contribution. The contribution is a machine learning method: matched counterfactual training data, a paired training loss, and outcome-conditioned flow matching with candidate selection. Table 2 shows that each part matters. The question is what a learned model can identify about its own actions from trajectory data, which we see as an ML question rather than a power systems one. HVAC is our testbed, in the same way robotics is the testbed for Decision Transformer and Diffusion Policy.

We note that the review rates soundness as "minor or fixable concerns." We hope the clarifications above address the main points about the problem and the experiments, and we would be grateful if the reviewer would consider revisiting the rating in light of them.

[1] D. B. Crawley et al. EnergyPlus: creating a new-generation building energy simulation program. Energy and Buildings, 33(4):319-331, 2001.

[2] V. Taboga, J. Veilleux, D. Jang, A. Rankawat, and P.-L. Bacon. Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning. arXiv:2607.16534, 2026.

Reviewer_1WpPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies HVAC control. It identifies the problem that environmental confounders might create false causal relationships in training. A training procedure using matched counterfactual trajectories and a matched branch loss was proposed. The method is shown to perform well compared to learned policies trained using standard procedures.

Critical strengths

The problem of zero-shot transfer on systems with time-varying disturbances is meaningful.

Critical weaknesses

The impact of this work in the machine learning community seems to be limited, as the main contribution, training with matched counterfactual trajectories, is only possible when an accurate, resettable simulator exists. Although tests are conducted on unseen buildings, the paper does assume perfect modeling of the underlying physics that applies to both training and testing buildings.

Additionally, this paper lacks important implementation details. One example is the training of the PPO baseline. Given sufficient interactions with a simulator, PPO can naturally observe counterfactual trajectories from exploration. This raises questions about whether the PPO hyperparameters are set correctly and whether it has been sufficiently trained (neither of which is mentioned in the paper or included in the supplementary material). Specifically, PPO underperforming rule-based control is an indicator that it is potentially not properly set up.

Overall, I do believe this work can make an important contribution to HVAC control. However, in its current form, it is slightly out of scope for ICLR.

Questions for authors

For a fair comparison, it’s crucial to compare with a Decision Pretrained Transformer trained on the same dataset as GON. The current comparison does not isolate the effect of training data size and appears unfair to the trained baselines.

1 comment

We thank the reviewer for the constructive feedback and for recognizing the importance of zero-shot transfer under changing external conditions.

Simulator requirement. The simulator is only needed for training. At test time GON does not use it, does not create branches, and does not update its weights (Section 3.5). The baselines need a simulator too: PPO is trained by interacting with it, and MPC and the DRL controller need a separate model or training run for each building. All methods are evaluated in the same EnergyPlus simulator, which is how Building2Building is set up. GON gets no EnergyPlus parameters or building metadata at test time (Appendix A.1.1) and has to infer each building from four hours of recent history. We agree that real-building results are missing and have added this as a limitation (Section 5).

PPO. We have added the PPO hyperparameters (Table 5). They are the Building2Building default PPO settings, used without changes. Its training budget is in Table 6. The building-specific DRL controller in Figure 3 uses the same PPO code and hyperparameters, and it beats RBC by 34.6% in total cost over the 100 test buildings (Appendix A.2.3). So we do not think the PPO setup is the problem. The generalist does worse because one model-free policy has to control 100 unseen buildings without adaptation. It still uses less energy than RBC on both tasks; it loses on comfort.

On exploration: PPO does try different actions, but from different states and under different weather. It never sees two action sequences from the same state under the same weather, so it has to average the drift out. Matched branches remove it directly.

DPT and data size. DPT is trained on about 41.5M logged steps. GON uses about 20.7M logged steps for its encoder and about 8.4M matched branch steps, so DPT sees more data than GON in total (Appendix A.2.2, Table 6). GON's gains therefore do not come from more data. The w/o LpairL_{pair} ablation in Table 2 also keeps the data fixed and removes only the paired loss. Energy goes up by 30% and comfort penalty by 64%.

Scope. The question we study is a learning question: what a model can learn about its own actions from trajectories when outside conditions change the state at the same time. This applies to any controller pretrained on trajectories, such as Decision Transformer or DPT. HVAC is our testbed, in the same way robotics is the testbed for Decision Transformer and Diffusion Policy.

Reviewer_BTV6OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a pipeline for training a heating, ventilation, and air conditioning (HVAC) controller that generalizes across buildings. To isolate the effect of the applied actions from external conditions, the proposed approach simulates different action plans that start from the same building state. These matching branches train a predictor model conditioned on the applied actions. At inference time, the predictor model estimates the state trajectory of the system without any HVAC intervention. This prediction conditions a flow matching policy that estimates the actions required to hold a desired temperature profile. From a set of sampled action plans, the predictor model selects the best among all candidates. On 100 unseen buildings of the Building2Building benchmark, GON reports improved results over rule based and learned baselines and performance comparable to MPC tuned for each building.

Critical strengths

Predicting the state trajectory without any HVAC intervention and conditioning the policy on the required deviation is a reasonable decomposition of the problem. Furthermore, using the predictor model to select among sampled action plans is a sensible way to deal with multiple action sequences leading to the same outcome. If the missing implementation details confirm the results, the evaluation of a single frozen model on 100 unseen buildings across all eight climate zones would provide good evidence for generalization across buildings.

Critical weaknesses

The baselines and ablations are not specified well enough to assess the results. There are no implementation details for MPC, HVAC DPT, PPO, or the flow matching baseline. Furthermore, the statement “replacing the flow generator with regression” is ambiguous, as flow matching itself regresses a velocity field. The baselines also differ fundamentally in how they access the simulator, ranging from offline data to online interaction, and GON itself relies on extensive matched branching. Without implementation details and a comparison of the simulation budget of each approach, the provided results cannot be put into context.

One of the main claims is that a predictive model may struggle to separate the effects of HVAC actions from external influences. However, any model trained in simulation with broad coverage of action sequences should be able to separate these effects. The question therefore remains whether matched counterfactual branching enables this separation or is mainly an effective low variance sampling mechanism. This could be answered by comparing against a model trained on unmatched rollouts with randomized actions and an equal simulation budget, as well as against related methods such as ThermoForce. Without such comparisons, the central claim of the paper remains unverified.

Questions for authors
  • How are the action plans for the matched branches sampled, and does their construction depend on knowledge of specific buildings or climate zones?
1 comment

We thank the reviewer for the detailed feedback and constructive suggestions.

Baselines and ablations. Appendix A.2.1 describes each baseline, but we agree it does not give enough detail to reproduce them. We have added the MPC model and the PPO hyperparameters to Appendix A.2.1 (Table 5) and the training budgets of all methods to Appendix A.2.2 (Table 6). We will add the remaining hyperparameters. Two facts matter for reading Table 1. DPT is trained on about 41.5M logged steps, more than GON uses in total (Appendix A.2.2, Table 6). The building-specific DRL controller in Figure 3 uses the same PPO code and hyperparameters as the PPO generalist.

The "w/o flow generator" variant uses a network trained with MSE to output one action sequence from the same inputs. It gives a single plan (the conditional mean of the plans that reach the target), so there is nothing to select from. We have made this wording clearer in Section 4.2.

Simulation budget. GON uses about 175,000 four-hour branches (about 8.4M simulation steps) from 21,888 branch points, plus the replays needed to reach those points. GON also uses about 20.7M logged steps to pretrain its encoder. This is done once, offline, on the training buildings. GON does not use the simulator at test time. This is now stated in Section 3.2, and Appendix A.2.2 (Table 6) now compares the budgets of all methods.

Separation vs. low-variance sampling. We agree that with randomized actions and enough data, a model can in principle separate the action effect from external drift. Matched branching does not change what can be identified. It makes the action effect much cheaper to learn, because the paired loss removes the shared drift exactly. So we do not see the two explanations as competing. Also, trajectory pretraining usually uses data from behavior controllers, where actions depend on the weather (more cooling on hotter days). Broad coverage alone does not fix that.

We have not run the unmatched, equal-budget comparison you suggest, and we agree it is the cleanest test. The closest evidence in the paper is the w/o LpairL_{pair} ablation in Table 2: same data, same model, only the paired loss removed. Energy goes up by 30% and comfort penalty by 64%. If the gain came only from how the data was sampled, this variant should keep it. We will add the unmatched comparison in the next version. ThermoForce is concurrent work (arXiv, July 2026) and is discussed in Section 2.2.

Branch-plan sampling. Branch points are drawn uniformly at random from each logged training episode, after the first day. At each point, the simulator replays the logged actions to reach the logged state, and eight 4-hour plans are run. One plan turns HVAC off, one holds the logging controller's command, four shift that command up or down in all zones or in one zone, one uses milder supply-air limits, and one draws every zone's command at random. The same procedure is used for every building and climate zone, with no building or climate information. Branches are generated only on training buildings (Appendix A.1.2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.