acceptodds
Under review as a conference paper at ICLR 2027

Real-World Debiasing: From Deployment Failures to Causal Repair of Policy Shortcuts

Abstract

In practice, visual control policies can fail after deployment despite strong source performance. Prior work has identified hidden reliance on task-irrelevant source cues as one cause of such failures, even when the underlying task remains unchanged. Yet diagnosing and repairing this dependence after deployment is difficult: multiple factors may change simultaneously, and target interventions or retraining may be costly or infeasible. We call this broader failure pattern real-world bias. We further show that observational source data alone need not identify such harmful dependence, even when source performance is optimal. To address this gap, we introduce RENDER, which uses a few unpaired target failure trajectories to guide causal diagnosis and selective repair in the source simulator. RENDER reenacts candidate nuisance mechanisms to reproduce the observed failures, then tests whether task-preserving interventions change policy decisions and increase task loss. Once harmful dependence is verified, the diagnosed mechanism guides targeted adaptor training using source data while the original policy remains frozen; otherwise, RENDER abstains. The procedure requires no target interventions. Experiments across diverse visual control tasks support our analysis and show that selective repair can improve deployment performance.

Primary area:
interpretability and explainable AI
Keywords:
causal reasoning, reinforcement learning, imitation learning, shortcut learning, distribution shift, policy robustness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_soprOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RENDER, a framework for diagnosing and repairing harmful shortcut dependencies in visual control policies after deployment. It uses a few target failure trajectories to identify candidate causes, verifies them through interventions in the source simulator, and trains adaptors while keeping the original policy frozen.

Critical strengths

I find the problem meaningful, and I appreciate the clear separation between causal diagnosis and policy repair. The theoretical motivation is reasonable, and the experimental evaluation covers several different visual control settings, including ForkRoad, Atari, and CARLA. I also appreciate the ablation studies and the evaluation of source performance preservation.

Critical weaknesses
  • I think the main limitation is the reliance on a predefined intervention library and a source simulator capable of reproducing candidate deployment failures. I am not sure how well the approach would generalize when the responsible mechanism is unknown or cannot be reproduced in the simulator.
  • I also find the real-world validation somewhat limited. The experiments are conducted in simulated environments and mainly involve individual, interpretable nuisance factors. More evidence would be helpful to understand whether the approach remains effective under complex or interacting distribution shifts.
  • Each benchmark uses only one frozen base policy, while the five reported seeds mainly vary the repair process. Since shortcut dependencies can differ across independently trained policies, I am not sure whether the current results are sufficient to demonstrate consistent diagnosis and repair across different policy checkpoints.
Questions for authors
  • How sensitive is the diagnosis to the coverage and quality of the predefined intervention library?
  • Could the authors evaluate scenarios involving multiple simultaneous nuisance changes?
  • Could the authors provide additional experiments with independently trained policies?
6 comments
AuthorsOpenReview ↗

We are very grateful for your insightful and constructive feedback (1/2)

Dear Reviewer sopr,

We are very grateful for your insightful and constructive feedback. We are glad that you found the problem meaningful and the framework clear. In light of the suggestions, we have added new experiments in the updated manuscript. Please find our detailed point-by-point responses below.


Q1: How sensitive is the diagnosis to the coverage and quality of the predefined intervention library?

A1: Thank you for the question. Coverage and quality decide whether our method can name a mechanism, not whether the mechanism it names is correct: when no candidate reproduces the deployment change, our method abstains. We note coverage as the main limitation in the Discussion (L519-520):

The main limitation is intervention coverage. RENDER can verify harmful dependence only for mechanisms that can be reenacted and tested in the source environment.

This requirement is mild, however. Our method only needs a simulator that can apply task preserving changes to the source environment, such as changing the color of a game object, the brightness, or the camera position, and its library is built from such changes. Many simulators already provide them, such as textures, lighting, and camera pose in domain randomization [1], weather and lighting in CARLA [2], and object properties in CausalWorld [3]. We also considered real robot tasks, where these changes must be applied in the robot's environment, and leave them to future work.

To test whether our method can reject interventions outside its coverage, we removed the true family from the library on the paper's five diagnosis blocks, so that the deployment change lies outside it, and reran calibration and Steps 1 and 2 on the same target failures.

TargetFamily removedNearest remaining candidateDecision
Breakout red paddlepaddle and ball colordimming 4/5, white bricks 1/5abstains 5/5
Breakout white bricksbrick colorred paddle 5/5abstains 5/5
Freeway white carscar colorHUD mask 5/5abstains 5/5

Our method abstains in all 15 cases and never accepts a wrong family, even when the nearest remaining candidate is itself a harmful mechanism, such as the red paddle for the white brick target. Quality, that is, how closely a candidate reproduces the actual change, has the same effect. When the Freeway cars take intermediate gray levels and the library contains white cars, the closest candidate of the right family, our method abstains in all 10 diagnoses instead of accepting it. Please let us know if anything is still not fully clear.

[1] Domain randomization for transferring deep neural networks from simulation to the real world. IROS 2017

[2] CARLA: An open urban driving simulator. CoRL 2017

[3] CausalWorld: A robotic manipulation benchmark for causal structure and transfer learning. ICLR 2021


Q2: Could the authors evaluate scenarios involving multiple simultaneous nuisance changes?

A2: Thank you for the question. We discuss this as a limitation and as future work in the Discussion (L521-523):

Current experiments mainly consider individual, interpretable nuisance factors, whereas real deployments may involve unseen or interacting mechanisms. Extending the framework toward automatically generated and compositional interventions is therefore a natural direction for future work.

Multiple simultaneous nuisance changes can interact in complex ways, which makes them hard to diagnose. In Breakout, for example, the red paddle alone lowers the frozen policy's return to 55.0, adding a HUD mask lowers it further to 23.4, and adding blue bricks instead raises it to 93.3, although both added changes are harmless on their own. In supervised learning, concurrent shifts usually hurt more than single ones but not always [1], and attributing a performance change to several co-occurring shifts requires knowing the causal structure among them [2]. As far as we know, no prior work diagnoses which of several simultaneous nuisance changes causes the failure of a deployed policy. We see this as an open question that needs more time and careful study, and we leave it to future work.

[1] An analysis of model robustness across concurrent distribution shifts. TMLR 2025

[2] "Why did the model fail?": Attributing model performance changes to distribution shifts. ICML 2023

AuthorsOpenReview ↗

We are very grateful for your insightful and constructive feedback (2/2)

Q3: Could the authors provide additional experiments with independently trained policies?

A3: Thank you for this suggestion. In the paper, Freeway and Breakout use publicly released pixel PPO checkpoints (Appendix C). Since only one trained policy is released per game, each game has a single frozen base policy. To address your concern, we trained three additional Breakout policies (seeds 1 to 3) with the same recipe as the released checkpoint (10M decisions), changing only the training seed. One more run did not learn the game (native return 30.4), so we leave it out.

We then reran our method from scratch on each of the three policies. The results are summarized below in the same format as Figure 5, with standard errors over the 20 evaluation games for Frozen and over five repair seeds for RENDER.

MethodSource ↑Target ↑Response (%) ↓
Released policy
Frozen185.40 ± 19.1055.00 ± 16.4327.9
RENDER200.08 ± 4.02125.18 ± 6.6718.4 ± 0.4
Seed 1
Frozen216.25 ± 20.1734.10 ± 10.2229.2
RENDER206.60 ± 12.79141.81 ± 13.1020.0 ± 0.8
Seed 2
Frozen159.10 ± 27.844.35 ± 0.9243.7
RENDER143.42 ± 9.8973.23 ± 9.2123.3 ± 0.4
Seed 3
Frozen167.20 ± 20.1587.05 ± 15.6525.6
RENDER193.20 ± 8.57126.52 ± 9.5218.0 ± 0.3

The results are consistent across policies. For all three seeds, our method identifies the same mechanism as for the released policy, namely the paddle and ball color, and Step 3 confirms that it harms the task. The repair then raises the return under the red paddle and lowers the response, while the native return stays within episode noise. In addition, the action delay, which lies outside the library, is rejected for every policy, and no wrong mechanism is accepted. We have added these results to the updated manuscript according to your great suggestion (Section 4.2, L405-407; Appendix D.12, L1869-1921). Please let us know if anything is still not fully clear.

AuthorsOpenReview ↗

Rebuttal Follow-up

Thanks again for your insightful and constructive feedback. We have submitted our rebuttal with new experiments and expanded discussions addressing the points raised in your review.

If you have time during the discussion period, we would greatly appreciate it if you could take a look at our response. We would be happy to clarify any aspects of the paper or response that may still be unclear.

Thank you very much for your time and consideration.

Reviewer_SPmDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies frozen visual control policies that fail after deployment because they depend on task-irrelevant inputs. It defines "real-world bias" as positive excess task loss under a task-preserving intervention on a nuisance variable (Def. 2), argues that source records cannot identify this quantity (Prop. 1), and shows that paired simulator rollouts can (Prop. 2). The method, RENDER, takes a frozen policy, a source simulator, a fixed library of candidate interventions, and a few dozen target failure trajectories. It then:

  1. summarizes each failure by a behavioral signature (return, length, event-time quantiles, action histogram);
  2. reenacts each library candidate and picks the closest by a scaled energy distance, with calibrated thresholds and an abstain option;
  3. tests on paired source histories whether the selected intervention changes decisions and raises task loss;
  4. fits a small adaptor while the base network stays frozen.

Experiments cover a synthetic ForkRoad shortcut, three Atari recoloring/occlusion shifts, and a CARLA copycat scenario. An ablation shows that repairing according to source harm alone selects the wrong dependence.

Critical strengths
  • The separation of "which dependencies exist" from "which one the deployment exposed" is genuinely useful. On Breakout, the source audit ranks a gray background and a blue paddle as most harmful; repairing the gray background costs 30.1 points under the actual red-paddle shift (Table 8). That is a concrete warning against fixing whatever perturbation hurts most.

  • The statistical protocol is careful: disjoint selection/confirmation splits, thresholds calibrated on pseudo-targets before target data are read, held-out mechanisms and split nulls exercising abstention, and five seed blocks with no wrong acceptance in 30 in-library runs (Table 7).

Critical weaknesses
  • No observation-based diagnosis baseline. Target trajectories contain observations, yet the signature excludes them without explanation. For recoloring and occlusion, matching frozen-feature distributions between target and intervened source frames would likely find the right entry from far fewer trajectories. Behavioral signatures only pay off under compound shifts, which are never tested.
  • The shortcut framing fits only part of the benchmarks. In Breakout and Robotank the nuisance is constant during source training, so it cannot be a spurious action correlate; those failures are invariance failures to out-of-support inputs. Prop. 1's construction, with the intervened value inside support, accounts for ForkRoad and CARLA only.
Questions for authors

What happens for a target value near but not in the library (e.g., an intermediate Freeway color)?

6 comments
AuthorsOpenReview ↗

We truly appreciate your thoughtful and constructive comments (1/2)

Dear Reviewer SPmD,

We truly appreciate your thoughtful and constructive comments. We are especially grateful that you distilled a conclusion we consider important, namely that directly repairing the perturbation that hurts most is not feasible. In light of the suggestions, we have added new experiments and expanded discussions in the updated manuscript. Please find our detailed point-by-point responses below.


Q1: No observation-based diagnosis baseline. Target trajectories contain observations, yet the signature excludes them without explanation.

A1: Thank you so much for raising this point. This is an interesting and insightful question, and we fully agree with you that an observation-based baseline can reduce the number of trajectories needed in some settings. For example, it identifies the Breakout red paddle from one trajectory, while our method needs ten. However, our concern is that it can be misled by features that change a lot. Our intuition is that the visual factor that changes the most may not be the one that makes the policy fail. For example, day and night look very different, yet this difference need not be what causes a failure. Here we show this with a counterexample. We implemented the observation-based baseline, which matches frozen-feature distributions between target and intervened source frames, and ran it and our method on two Freeway targets over five diagnosis blocks. Each target combines a harmful car color with dimming. On its own, dimming leaves the frozen policy's return unchanged (32.9 against 33.0), yet it moves the frozen features more than any other library entry.

TargetFrozen returnObservation-based baseline selectsRENDER selects
White cars + dimming17.2dimming 5/5white cars 5/5
Black cars + dimming23.6dimming 5/5black cars 5/5

In both targets, the observation-based baseline picks dimming in all five blocks, although dimming alone does not hurt the policy. Our method picks the car color, which is what the policy fails on. Moreover, an observation-based baseline cannot detect the problem we met in CARLA, or in other settings where the shortcut is a nonvisual input, such as the previous action in imitation learning [1,2]. In CARLA, the policy takes its previous action as input. In the source this input comes from the expert, and at deployment it comes from the policy itself. The shift does not change how the camera images look, but it changes the behavior, and the car keeps moving when a pedestrian steps onto the road. We have added this experiment to the updated manuscript (Appendix D.10, L1821-1845).

[1] Causal confusion in imitation learning. NeurIPS 2019

[2] Object-aware regularization for addressing causal confusion in imitation learning. NeurIPS 2021


Q2: The shortcut framing fits only part of the benchmarks.

A2: Thanks for your question. We would like to clarify that real-world bias is not limited to shortcuts. We intend it to cover a harmful dependence on any factor that should not affect the task, so invariance failures are included as well. For example, an agent that has learned to play ball should not stop playing when the weather turns from sunny to cloudy, whether or not the weather varied during training. In ForkRoad and CARLA, the factor is correlated with the action in the source. In Breakout and Robotank, the factor (the paddle color or the instrument panel) never changes in the source. In Section 2, the nuisance is any information that affects the observation but not the task, and it may or may not be associated with the task state. Definition 2 only checks whether changing it, with the task kept the same, increases the task loss.

The theory and the method also apply to both cases. Proposition 1 shows that source data cannot reveal whether a nuisance change will hurt the policy, even when the new nuisance value has already appeared in the source. When the nuisance never changes in the source, the source data never show the policy the new value, so they reveal even less. Our method does not need to know which case it faces. It tests each candidate change in the source simulator and checks whether the change reproduces the target failure and raises the task loss.

This impression may come from our introduction, which mainly uses the correlated case to describe real-world bias and so gives it a shortcut framing. We did not state clearly enough that factors that never change in the source are included as well. We have therefore revised the definition sentence there to say so (Section 1, L75-78). Please let us know if anything is still not fully clear.

AuthorsOpenReview ↗

We truly appreciate your thoughtful and constructive comments (2/2)

Q3: What happens for a target value near but not in the library (e.g., an intermediate Freeway color)?

A3: Thank you for this practical question. To answer it, we added an experiment in Freeway, whose library contains white cars (gray level 236). We created two target car colors between white and the road (gray levels 214 and 192), so that each is close to a library value but not in the library. We then ran our method on the five diagnosis blocks. To see how much the nearest library repair can still help, we also applied the white car adaptor to both targets without refitting (five repair seeds).

Target car colorFrozen returnRENDER nearest familyRENDER decisionGain from the white car adaptor
White (gray 236, in the library)16.9car color 5/5accepts 5/5+9.6 ± 1.5
Gray 214 (not in the library)19.3car color 5/5abstains 5/5+5.8 ± 0.4
Gray 192 (not in the library)20.3car color 5/5abstains 5/5+3.4 ± 1.3

In every block, our method correctly traces both failures to the car color. However, no library value reproduces either failure closely enough, so it abstains instead of accepting a wrong value. Even so, the white car adaptor still recovers part of the loss, though less than on white cars, the value it was trained for. We have added this experiment to the updated manuscript (Appendix D.11, L1848-1866).

AuthorsOpenReview ↗

Rebuttal Follow-up

Thank you again for your thoughtful and constructive comments. We have submitted our rebuttal with new experiments and expanded discussions addressing the points raised in your review.

If you have time during the discussion period, we would greatly appreciate it if you could take a look at our response. We would be happy to clarify any aspects of the paper or response that may still be unclear.

Thank you very much for your time and consideration.

Reviewer_9enZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies how to diagnose and repair a trained policy using a small number of deployment failures. RENDER compares these failures with failures produced by candidate interventions in the source simulator, checks whether the selected intervention changes the policy’s behavior and increases task loss, and then trains an adapter while keeping the base policy fixed. The goal is to repair harmful dependencies without target interventions or other corrective demonstration.

Critical strengths

I like the practical setting. Collecting a few failure trajectories may be much easier than collecting new demonstrations or repeatedly testing changes in the deployment environment.

Table 1 is the most interesting result to me. It shows that a policy can have several harmful dependencies, and repairing one that is unrelated to the target failure can actually make target performance worse. This gives a concrete reason to use target failures to guide repair selection. Checking whether the repair preserves source performance is also useful. The option to abstain are sensible parts of the approach.

Critical weaknesses

The contribution over earlier diagnosis and repair methods needs to be clearer. DFA paper studies failure diagnosis followed by targeted adaptation with human feedback, and ReCCoVER paper uses interventions to expose harmful dependencies and guide corrections. Their information requirements differ from RENDER’s, so I think it's not the same problem here. The most distinctive part here seems to be using a few unpaired target failures to select a source intervention and repair a frozen policy. That could be valuable. Since the repair objective in Eq. 11 is essentially feature restoration with an identity penalty, much of the contribution rests on whether the diagnosis leads to better repair decisions.

I’m not yet convinced how reliably the diagnosis identifies the mechanism relevant to the target failure. Different interventions can produce similar failure signatures, and showing that an intervention is harmful in the source environment does not by itself establish that it explains the observed target failure. The confirmation, separation, and abstention checks address this concern, but their effectiveness is central to the paper’s claims. The two propositions do not resolve this ambiguity. Table 1 supports failure-guided selection, but all its selection rules retain the Step 3 audit, so it does not isolate what that audit contributes. The harder Robotank cases in D.7 sound directly relevant. I could not assess them in detail because the full appendix was not included in the material available to me.

And the last thing is more of a writing thing. I’m not sure what “real-world bias” adds to the existing concepts. The pattern described in the introduction is already familiar from shortcut learning and causal confusion. The paper acknowledges this literature, but it is unclear what the proposed unification adds beyond a new name. There is also a difference between the introductory explanation and Definition 2. The introduction describes a policy using a spurious predictor of the correct action, whereas the definition covers any increase in task loss under an admissible nuisance intervention. For example, a policy might use the ball’s position correctly but fail to detect it after its color changes. This would satisfy the definition without establishing the particular shortcut mechanism described in the introduction. Clarifying the intended scope would help me understand what the experiments demonstrate.

Questions for authors
  1. With the same candidate interventions, full failure-signature matching, and repair procedure, what changes when Step 3 is removed? In particular, does the audit stop repairs that would otherwise be attempted and would harm target performance?

  2. In the harder cases in D.7, how similar are the competing mechanisms under the full signature used for selection? What evidence shows that the confirmation checks can distinguish them, or appropriately abstain when they cannot?

5 comments
AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (1/3)

Dear Reviewer 9enZ,

We sincerely appreciate your time and thoughtful feedback. We are glad that you found the practical setting appealing. In light of the suggestions, we have added new experiments and expanded discussions in the updated manuscript. Please find our detailed point-by-point responses below.


Q1: The contribution over earlier diagnosis and repair methods needs to be clearer.

A1: Thank you so much for raising this point. We noticed that the review mentions that the full appendix was not included in the material available to you. We did not fully understand this, since the complete appendix (Appendices A to E, pages 13 to 33) is in the same PDF file as the main paper, right after the references. Our related work section is Appendix E, which discusses DFA [1] and ReCCoVER [2] together with the other remedies for known biases (L2009-2019). We have also reread the two papers you mentioned and summarize the differences here. DFA [1] needs a human in the loop to identify the failure, demonstrate the correct behavior, and confirm which visual concept is task-irrelevant, while ReCCoVER [2] checks a policy before deployment by changing the features of critical states, so their settings are quite different from ours.

We further surveyed more related literature. Shortcut learning [3] describes decision rules that perform well on standard benchmarks but fail under more challenging test conditions. In sequential decision making, such shortcuts are revealed by inspecting interpretable concepts [4], by targeted interventions that query an expert or the environment [5], or by test environments that separate the intended goal from its proxy [6], and they are then removed by changing the training data, the architecture, or the training run. As discussed in Appendix E (L1935-1947), in the works above the cue is built into the study or found by inspecting the policy.

In contrast, our work is motivated by the practical situation in which a deployed policy is found to fail and there is no clue about why. This leads to several differences from prior work: 1) we assume that large-scale retraining or data collection in the deployment environment may be too expensive or infeasible, so we require no target interventions. 2) we start from the failure trajectories and reason from effect to cause, whereas many of the works above first identify a shortcut or a failure factor and then remove it. 3) we require no human in the loop. At the same time, our ambition is to give a unified mechanistic explanation of these findings of prior work, in imitation learning and in reinforcement learning alike. This is the point raised in Q3, which we discuss further in A3. Following your suggestion, we have stated these differences more precisely and discussed them more carefully in Appendix E (L1949-1963).

[1] Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation. ICML 2023

[2] ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning. EXTRAAMAS@AAMAS 2022

[3] Shortcut learning in deep neural networks. Nature Machine Intelligence 2020

[4] Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents. NeurIPS 2024

[5] Causal Confusion in Imitation Learning. NeurIPS 2019

[6] Goal Misgeneralization in Deep Reinforcement Learning. ICML 2022

AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (2/3)

Q2: I’m not yet convinced how reliably the diagnosis identifies the mechanism relevant to the target failure. Different interventions can produce similar failure signatures, and showing that an intervention is harmful in the source environment does not by itself establish that it explains the observed target failure.

A2: Thank you for raising this point. This is a very natural question, and we also considered it when designing our experiments. Here we address it in detail.

First, across the five experiments, different interventions almost never lead to a wrong diagnosis. Comparing all interventions in pairs under the complete rule of Steps 1 and 2, only one pair is ever confused.

BenchmarkComparable cross-family pairsConfused pairs
Freeway50
Breakout131 (blue paddle, gray background)
Robotank (Appendix D.7)110
ForkRoad00
CARLA480

Second, we fully agree that the setting you describe is important, and it is exactly the setting of Appendix D.7. There, we deliberately adjusted the Robotank interventions into two pairs with nearly equal failure rates and failure returns and close Step 3 harm (details in A6). Since this adjustment is artificial, we placed the experiment in the appendix. The results are as follows (true mechanism/other mechanism/abstention over five repeated diagnoses, and target gain over the frozen policy).

TargetHarm-onlyReturn-onlyRENDER (= Oracle)
Occluded panel3/2/0, +10.24/0/1, +15.55/0/0, +19.4
Dimming to 75%2/3/0, -1.65/0/0, -1.95/0/0, -1.9
Occluded logo strip0/5/0, -3.13/0/2, +11.15/0/0, +18.5
Blanked clock digits0/5/0, -4.15/0/0, +5.65/0/0, +5.6
Camera shift (outside the library)-/5/0, -0.6-/0/5, 0.0-/0/5, 0.0
All 25 cases5/20/0, +0.217/0/8, +6.120/0/5, +8.3

Our method matches Oracle on every target, while Harm-only selects a wrong mechanism in 20 of 25 cases. These results show that our method is robust in this difficult setting. We have further improved Appendix D.7 in the updated manuscript (Appendix D.7, L1727-1756).


Q3: I’m not sure what “real-world bias” adds to the existing concepts, it is unclear what the proposed unification adds beyond a new name.

A3: Thank you for this important question. Real-world bias is not meant as a new name for shortcut learning; it describes these failures with one causal definition, which gives two things the existing concepts do not. 1) Definition 2 is stated on task loss, not on how much the policy output changes, as in counterfactual invariance [1]. It therefore focuses on the dependence that harms the task after deployment, which is the case that calls for diagnosis and repair. 2) Definition 2 is built from the failures that prior work reported, and Section 2.1 states the intuition behind it. The reference policy acts on the task state StS_t alone, while the learned policy sees Ot=f(St,Ut)O_t=f(S_t,U_t). When StS_t and the nuisance UtU_t are associated in the source, both the task path and the nuisance path can shape the learned action. This is how we pursue the ambition mentioned in A1, to explain with one mechanism the findings that prior work reported separately in imitation learning and reinforcement learning. We have made this clearer in the updated manuscript (Section 2.1, L147-152 and L163-165).

[1] Counterfactual Invariance to Spurious Correlations in Text Classification. NeurIPS 2021


Q4: There is also a difference between the introductory explanation and Definition 2. The introduction describes a policy using a spurious predictor of the correct action, whereas the definition covers any increase in task loss under an admissible nuisance intervention. For example, a policy might use the ball’s position correctly but fail to detect it after its color changes.

A4: Thank you for this careful question. Definition 2 is the precise statement, and it covers your example by design. By Definition 1, a task-preserving intervention keeps the task state, the reference action, the dynamics, and the reward, and it keeps the information needed for the task. If such an intervention still raises the expected task loss, the only remaining cause is that the policy's actions respond to the nuisance. A policy that tracks the ball correctly but loses it after a color change is such a case: the ball is still visible and the task is unchanged, yet the policy's behavior depends on the nuisance color. It therefore has real-world bias, and our Breakout experiment, which recolors the paddle and ball, is an instance of it. The introduction mainly conveys the intuition through its most familiar case, a cue that predicts the correct action in the source. We have made this connection between the introduction and Definition 2 more explicit in the updated manuscript (Section 2.1, L149-152).

AuthorsOpenReview ↗

We sincerely appreciate your time and thoughtful feedback (3/3)

Q5: With the same candidate interventions, full failure-signature matching, and repair procedure, what changes when Step 3 is removed? In particular, does the audit stop repairs that would otherwise be attempted and would harm target performance?

A5: Thank you for this question. Yes, the audit stops such repairs. In the reported runs this is hard to see, because a candidate must reproduce enough source failures to be eligible, and harmless shifts rarely do; Steps 1 and 2 therefore accept only harmful mechanisms there (48 times, always the true family).

We therefore ran the ablation you suggest on Breakout, with the same candidates, full signature matching, and repair procedure. To let harmless shifts reach selection, we counted every episode below the native median return as a failure, and diagnosed five harmless shifts, five times each. For every shift accepted without Step 3, we trained its repair with the recipe of Table 1 and evaluated it on that shift (mean ± s.d. over five repair seeds).

Target shiftAccepted without Step 3Accepted by RENDERTarget gain of that repair
HUD mask3/50/5+11.9 ± 18.5
Green paddle1/50/5-22.4 ± 19.7
Blue bricks1/50/5-34.9 ± 30.7
White paddle1/50/5-78.0 ± 33.3

Without Step 3, the rule accepts six of the 25 harmless cases, and three of these repairs lower target return by 22 to 78 points, while the HUD repair stays within noise. Step 3 rejects all six, so our method abstains and loses nothing. We have added this ablation to the updated manuscript (Section 4.5, L483-507; Appendix D.8, L1759-1792).


Q6: In the harder cases in D.7, how similar are the competing mechanisms under the full signature used for selection? What evidence shows that the confirmation checks can distinguish them, or appropriately abstain when they cannot?

A6: Thank you for this question. In D.7 we changed one intervention: dimming now scales every pixel by 0.75 instead of 0.5, so that its failure return matches that of the occluded panel; the logo strip and the clock digits already matched. After this adjustment, the two pairs look alike in failure rate and failure return, and their Step 3 harms are close.

PairFail rateFailure returnStep 3 harm
Occluded panel vs dimming to 75%1.00 vs 1.0021.7 vs 21.244 to 50 vs 45 to 47
Occluded logo strip vs blanked clock digits0.72 vs 0.7841.5 vs 42.325 to 29 vs 19 to 22

Even so, the full signature still separates them, because the policy fails in different ways. Under the occluded panel it drives forward in 10% of its decisions, against under 1% under dimming; under the occluded logo strip it drives forward-left in 63% of its decisions, against 32% under blanked clock digits. As a result, the distance to the competing mechanism is 10 to 18 times the threshold that sampling noise alone produces, while the distance to the true mechanism stays below it.

In the extremely rare case where two mechanisms coincide even under the full signature, our method abstains. The only such case in our experiments is the blue paddle and gray background in Breakout. Under both, the policy collapses to a single action: it only presses FIRE and never moves, losing all lives within about 125 decisions with a score near zero. The failure trajectories therefore carry no information about their cause, and the only correct output is to abstain. Our method does so in about two thirds of draws, and accepts the wrong family in only 4% to 7.5% of them. We have reported this case in Appendix D.9 (L1779-1820). Please let us know if anything is still not fully clear.

Comments on the paper

2 comments
AuthorsOpenReview ↗

We welcome your feedback on our rebuttal

Dear reviewers,

Thank you all for taking the time to review our manuscript. Your valuable comments are greatly appreciated and have helped us improve the quality of our work.

As the author-reviewer discussion period is nearing its end, we would sincerely appreciate it if you could take a moment to review our rebuttal and let us know whether we have adequately addressed your concerns. This would also give us an opportunity to respond to any further questions or concerns you may have. Thank you for your time and consideration! :)

AuthorsOpenReview ↗

We sincerely appreciate the time, effort, and thoughtful feedback

Dear Reviewers, AC, and SAC,

We sincerely appreciate the time, effort, and thoughtful feedback you have devoted to the review. Your comments offered constructive suggestions that have helped us further strengthen the manuscript. A summary of the discussion is included at the end of each reviewer’s rebuttal and is also given below:


Reviewer 9enZ: We appreciate that the reviewer found the practical setting appealing. The main questions arose from concerns about the relation between our work and prior work, and from parts of the paper that were misunderstood or missed. The appendix of the original manuscript already discusses exactly the prior works that the reviewer gave as examples, and the reviewer most likely did not read this directly relevant appendix. Even so, the reviewer acknowledged that these works address a different problem. Likewise, the reviewer most likely missed the directly relevant results in the appendix that show the reliability of the diagnosis, and we have further supported them with new experiments. Therefore, we believe all concerns have been addressed, both in the original manuscript and during the rebuttal, and the rating should be increased.

Reviewer SPmD: We are glad the reviewer found our failure-guided diagnosis useful and the statistical protocol careful, with no concern about the soundness of the work. We answered each question with new experiments and further discussion, and the reviewer confirmed that "most of my concerns have been addressed" and kept the positive rating.

Reviewer sopr: We are glad the reviewer found the problem meaningful and the separation between diagnosis and repair clear. The questions concerned the scope of the method rather than its soundness, and we answered them with new experiments and discussion. The reviewer confirmed that "my concerns have been addressed" and supports the paper for acceptance.


We also summarize our main contributions below:

  • This paper distills a common deployment failure, in which a policy relies on a task-irrelevant factor hidden behind strong source performance, into real-world bias. We show that source data alone cannot reveal whether such a dependence is harmful, whereas interventions in the source simulator can. Building on this result, we introduce RENDER, which uses a few unpaired deployment failures to diagnose the responsible dependence, verifies its harm through paired interventions, and selectively repairs it while keeping the policy frozen, without any target interventions. We further show that finding a harmful dependence is not enough for repair, since repairing one that the deployment did not expose can lower target performance. We validate the analysis and the method on route selection, Atari games, and autonomous driving.

Thank you again for your careful consideration of our work. Please let us know if you have any questions.

Best regards,

Authors of Submission 57440

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.