acceptodds
Under review as a conference paper at ICLR 2027

Temporal-Structure Robustness Transfer in Offline Reinforcement Learning: Uncertainty-Guided Decision Transformer

Abstract

Existing corruption-robust offline RL methods largely focus on learning reliable policies from corrupted offline datasets, while robustness transfer under changes in the temporal structure of corruption remains less understood. We formulate temporal-structure robustness transfer (TS-RT), an offline RL setting in which policies are trained under temporally independent corruption and tested under previously unseen, temporally correlated corruption. To the best of our knowledge, this work provides the first systematic investigation of TS-RT in offline RL across observation, action, and reward corruption settings. To address TS-RT, we propose Uncertainty-Guided Decision Transformer (UDT), which learns a token-level uncertainty signal to regulate both causal information routing and the training influence of unreliable targets. For controlled evaluation, we introduce Drift-Attack, which generates temporally correlated test-time perturbations through burst–recover dynamics and momentum-driven drift within a matched-channel protocol. Across nine D4RL tasks and seven representative baselines, UDT achieves the highest overall normalized score of 63.52 and ranks first on 11 out of 15 reported aggregate evaluation metrics across corruption types, corruption channels, and environment groups. We provide code and trained checkpoints to facilitate reproducibility.

Primary area:
reinforcement learning
Keywords:
Offline Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_peoBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether offline RL policies trained with independent errors remain effective when errors persist across consecutive steps at test time. UDT extends RDT with a learned score that controls attention to each input token. The same score also weights prediction errors during training. The authors evaluate UDT under Drift-Attack, which generates persistent errors. UDT achieves the highest average score across nine D4RL tasks.

Critical strengths
  • The experiments cover nine tasks from three benchmark families. UDT outperforms DT and RDT on average after training with either random or adversarial corruption.
Critical weaknesses
  • The paper lacks a comparison with independent test errors at the same frequency and size. The burst experiment helps, but longer bursts also corrupt more tokens. Keeping the amount of corruption fixed would help distinguish an advantage against persistent errors from better noise tolerance in general.

  • The experiment without routing is difficult to interpret. The paper states that uncertainty scores are trained through attention, while gradients are blocked through the loss weights. Under this description, removing routing appears to leave the scores untrained even though they still weight prediction errors. The lower performance could then come from using untrained scores in the loss. Please clarify how these scores are obtained in this experiment.

Questions for authors
  1. How does UDT compare with RDT when test errors occur independently but have the same frequency and size as Drift-Attack?

  2. Are the uncertainty scores trained from scratch in the experiment without routing? If so, how are they updated when gradients are blocked through the loss weights?

4 comments
AuthorsOpenReview ↗

How do UDT and RDT compare under independent errors matched to Drift-Attack? (W1, Q1)

1. How do UDT and RDT compare under independent errors matched to Drift-Attack? (W1, Q1)

We compare Uncertainty-Guided Decision Transformer (UDT) and Robust Decision Transformer (RDT) under independent and persistent test errors with similar rates and magnitudes. For IID-Matched, we sample magnitudes from Drift-Attack independently at each step. The samples include zeros, so the error rate and magnitude distribution stay the same in expectation. DeFog (ICLR 2023, Fig. 13) compares independent frame drops with Markov bursts, and GRAD (ICLR 2024, Figs. 3–4) compares temporally coupled and decoupled perturbations. We use the submitted Drift-Attack scores for the same 18 settings on Walker2d, Kitchen-Mixed, and Relocate.

Table R1. UDT and RDT under independent and persistent errors of similar frequency and size. D4RL normalized returns. The ± values are standard deviations across three training seeds. The Drift-Attack overall scores average the 18 setting means from Tables 7 and 8 of the submission. Parentheses indicate the training corruption.

SettingIID-Matched / RDTIID-Matched / UDTDrift-Attack / RDTDrift-Attack / UDT
Overall (18 settings)51.28 ± 1.3056.21 ± 1.2051.6455.14
Relocate (random action)90.02 ± 5.9073.96 ± 6.3175.3 ± 5.577.9 ± 1.7
Kitchen-Mixed (random action)36.67 ± 10.1047.50 ± 4.3337.5 ± 5.436.7 ± 1.4

At similar error rates and magnitudes, UDT leads RDT on average under both independent and persistent errors. The lead is 4.93 points under IID-Matched and about 3.50 points under Drift-Attack. On Relocate, RDT leads under IID-Matched and UDT leads under Drift-Attack. On Kitchen-Mixed, UDT leads under IID-Matched and RDT leads under Drift-Attack. The rankings are descriptive comparisons across two separate evaluations.

On Walker2d, the two tests produce observation errors of similar frequency and size. Under Drift-Attack and IID-Matched, the error rates are 24.70% and 24.59%, and the root-mean-square magnitudes are 0.4187 and 0.4168, respectively. The correlation between consecutive magnitudes is 0.7408 under Drift-Attack and 0.0008 under IID-Matched.

AuthorsOpenReview ↗

Clustered and scattered errors at a fixed corruption budget (W1)

2. Clustered and scattered errors at a fixed corruption budget (W1)

We keep the corruption budget fixed to see how clustering changes the predicted action. For each trained policy, we start from the same clean history and apply the same offsets to ten of the last 20 steps. We place the errors either in one block or spread them through the history. The latest corrupted step is the same in both layouts, and the current observation is clean. We measure how far the prediction moves from the action for a clean history, then divide that distance by the square root of the action dimension.

Table R2. Action deviation under clustered and scattered errors at the same corruption budget. Deviations (×10⁻³) are measured relative to predictions from clean histories. Change is the deviation under clustered errors minus the deviation under scattered errors.

Corrupted inputScattered / RDTScattered / UDTClustered / RDTClustered / UDTChange / RDTChange / UDT
Observation34.9028.6340.5331.895.633.26
Action40.0632.7451.3341.2011.278.47
Reward0.1410.1910.0600.121-0.081-0.071

With the same corruption budget, clustering observation or action errors increases action deviation less for UDT than for RDT. The increase in action deviation is 42.1% smaller for observations and 24.8% smaller for actions. UDT has the smaller increase in 29 of 36 observation and action settings, across both random and adversarial training corruption. The scattered layout also places some errors farther back in the history.

For reward errors, UDT's action deviation is larger than RDT's in both layouts. Clustering the errors reduces the deviation for both methods. Moving the same raw reward offsets to different steps can change the return-to-go tokens in different ways.

AuthorsOpenReview ↗

Do untrained loss weights explain the drop when routing is removed? (W2, Q2)

3. Do untrained loss weights explain the drop when routing is removed? (W2, Q2)

In the original w/o Routing ablation, we initialize the uncertainty branch from scratch and it receives no gradient from the prediction loss. In full UDT, that gradient reaches the branch through attention. In w/o Routing, the uncertainty scores still weight the loss, and a stop-gradient blocks learning through those weights. That leaves untrained loss weights in the ablation, which could contribute to the drop.

We trained w/o Both to test whether the untrained weights account for the gap to full UDT. This variant keeps routing disabled and drops the uncertainty-based loss weights. The comparison with w/o Routing tests those weights. The comparison with full UDT tests routing and reweighting together.

Table R3. Effect of removing untrained loss weights with routing disabled. D4RL normalized returns under Drift-Attack (mean ± SD across three training seeds).

Training variantOverall
Full UDT63.5 ± 0.6
w/o Routing62.3 ± 0.3
w/o Both62.4 ± 0.5

Removing the untrained weights leaves almost the same gap to full UDT. With routing disabled, the overall score goes from 62.3 to 62.4 when we remove those weights. Full UDT scores 63.5.

To measure the effect of routing at inference, we switch it on and off in the same trained model. We pick the checkpoint and target return with routing on, using one evaluation seed. We then run both modes under Drift-Attack on a different seed, keeping the checkpoint and target return fixed.

Table R4. Routing on vs. off under Drift-Attack with the same trained checkpoint. D4RL normalized returns. Paired differences are mean ± SD across three training-seed averages.

RoutingOverallObservationActionReward
On56.548.952.568.1
Off51.841.148.266.1
On minus off4.7 ± 0.97.8 ± 3.14.3 ± 4.02.0 ± 1.7

Switching on routing improves returns with the same checkpoint and learned uncertainty encoder. The gain is 4.7 ± 0.9 points overall and is positive on average for all three training seeds. Routing gives a higher score in 95 of 162 comparisons across settings and seeds. We will clarify the original ablation and report both comparisons.

AuthorsOpenReview ↗

Looking forward to your feedback (Urgent)

We hope that our rebuttal has effectively addressed all your concerns. As the deadline for the discussion period draws near (less than 3 hours), we would greatly appreciate it if you could re-evaluate our manuscript and provide any feedback. Thank you for your time and consideration.

Reviewer_uW7qOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work studies the robustness of Decision Transformer (DT) models against temporally correlated attacks. The authors propose the Uncertainty-Guided Decision Transformer (UDT). It learns from offline trajectories with temporally independent corruption, and extrapolates to temporally dependent corruption. It builds on top of Robust Decision Transformer (RDT, Xu et al., 2025) by including a learned uncertainty signal, and training with an uncertainty-weighted loss function. The experiments show evidence that UDT performs well under a temporally correlated Drift Attack introduced in the paper.

Critical strengths

S1. The problem formalization is very clearly defined. I find the problem interesting and sufficiently novel.

S2. The proposed uncertainty-guided causal attention mechanism (section 4.2) is a great contribution which could inspire future work. I think it should be better highlighted in the text or a figure showing an example with concrete values (perhaps showing how the attention overconfidently selects some value/token, and the uncertainty multipliers dampen the impact).

S3. The paper presents a wide range of baselines and comparisons to other methods.

Critical weaknesses

W1. Experimental results are only from 3 seeds each, making it hard to say about reliability. The performance of UDT seems to be greater than the other methods and the confidence intervals do not seem to be overlapping so this is not a big issue, although a higher number such as 5 would be appreciated. In addition, there is no clean environment performance to compare with; it would provide a reference of how difficult the corruption scheme is. These inclusions would strengthen the claims considerably.

W2. A few missing details describing the method. See questions below.

Questions for authors
  1. Is there a more concrete example (practical, real world, or something different) where this work could potentially be applied? What are the ways other researchers could use this method?

Clarification:

  1. Is ωtc\omega^c_t analogous to ξtc\xi^c_t?
  2. Is there a mistake in Figure 2: there is an arrow pointing from uncertainty encoder uiu_i to the token representation, although uiu_i is not used in the token representation, but only in the next step?
  3. Line 322: at which timestep is xrefcx^c_\text{ref} taken? Or, differently, how do you obtain xrefcx^c_\text{ref}?
  4. In eq. 17 what is the purpose of vtcv^c_t? If c∈{obs,act}c \in \lbrace \text{obs}, \text{act} \rbrace it seems to act as an absolute value function, but if c=rewc = \text{rew}, it acts as a negation. What is the motivation behind it? Why is it written in this way?
  5. What are the activation and recovery probabilities used in the experiments? Are they related to the ratio mentioned in line 364? What is the ratio mentioned in line 364?
  6. Are the ablations (in section 6.3) performed by retraining the method without the specified component, or by using the trained model with all components trained, only disabled during evaluation?
  7. What is the post-burst action deviation Dm(L)D_m(L) (in section 6.4)? How is it defined?

Suggestion:

  1. Eq. 18 and line 335: perhaps simply use η\eta instead of dd?
6 comments
AuthorsOpenReview ↗

A worked example of downweighting a corrupted token (S2)

To show how uncertainty scaling changes attention, we corrupt one Walker2d observation three steps before the current decision. We compute attention on that history with uncertainty scaling on and off, using the same checkpoint, raw logits, and causal mask. We also report the token's attention weight with a clean history.

Table R1. How uncertainty scaling changes attention to the corrupted observation.

ContextAttentionWeight
CleanStandard47.47%
CorruptedStandard26.79%
CorruptedUncertainty-guided18.88%

Uncertainty scaling brings the corrupted token's attention weight down from 26.79% to 18.88% on the same history. The token starts with a logit of 3.74 and an uncertainty score of 0.52. With the learned coefficient η=0.37\eta=0.37, its temperature is 1+η×0.52≈1.191+\eta\times0.52\approx1.19. Dividing by that temperature gives a logit of about 3.14 before softmax. The token already drew substantial attention in the clean history. We will add this example to Section 4.2.

AuthorsOpenReview ↗

How reliable are the three-seed results, and how hard is Drift-Attack? (W1)

2. How reliable are the three-seed results, and how hard is Drift-Attack? (W1)

We used three seeds, as did several related offline RL studies. Decision Transformer (NeurIPS 2021) and BA-MCTS (ICLR 2026) use three seeds on D4RL, and LaMo (ICLR 2024) does the same in its offline Decision Transformer experiments. LSPC (ICLR 2025) and B2R (NeurIPS 2025) use three seeds on offline safe RL benchmarks.

We use clean scores as a reference for how hard Drift-Attack is. We compare UDT, Decision Transformer (DT), and Robust Decision Transformer (RDT) on all nine D4RL tasks and six training corruption settings. We use three training seeds in each test condition and select checkpoints separately for Clean and Drift-Attack, so the comparison is descriptive.

Table R2. Clean and Drift-Attack returns for DT, RDT, and UDT. D4RL normalized returns (mean ± SD over three training seeds). Drift-Attack values come from Table 1 of the submission.

MetricDT / CleanDT / DriftRDT / CleanRDT / DriftUDT / CleanUDT / Drift
Overall63.1 ± 0.658.5 ± 0.765.8 ± 0.958.4 ± 0.569.6 ± 1.463.5 ± 0.6
Training corruption
Random67.0 ± 0.261.8 ± 1.467.2 ± 1.159.9 ± 1.070.5 ± 0.965.5 ± 1.1
Adversarial59.2 ± 1.055.2 ± 0.164.4 ± 0.956.9 ± 0.668.7 ± 2.261.6 ± 0.8
Corrupted component
Observation66.1 ± 1.660.2 ± 1.463.9 ± 0.351.0 ± 1.168.0 ± 1.456.6 ± 2.7
Action50.6 ± 1.245.6 ± 0.560.9 ± 2.754.1 ± 0.464.3 ± 2.159.3 ± 1.9
Reward72.6 ± 0.469.8 ± 1.572.6 ± 0.670.1 ± 1.876.5 ± 0.874.7 ± 1.0
MuJoCo
HalfCheetah18.3 ± 1.218.4 ± 1.120.9 ± 2.819.7 ± 1.525.9 ± 1.527.3 ± 0.3
Hopper48.8 ± 3.545.9 ± 2.549.8 ± 0.847.7 ± 1.955.0 ± 2.553.6 ± 1.7
Walker2d38.7 ± 2.836.5 ± 0.840.6 ± 3.039.6 ± 0.944.8 ± 1.045.7 ± 2.3
Kitchen
Kitchen-Complete71.9 ± 1.760.7 ± 0.964.9 ± 2.943.3 ± 2.165.3 ± 2.144.4 ± 2.5
Kitchen-Mixed45.1 ± 2.438.5 ± 0.754.4 ± 1.334.7 ± 1.964.6 ± 1.441.4 ± 4.8
Kitchen-Partial48.2 ± 3.936.5 ± 2.956.8 ± 3.434.7 ± 5.164.9 ± 3.051.9 ± 2.4
Adroit
Door104.4 ± 0.3103.3 ± 1.6104.5 ± 0.2104.6 ± 0.2104.8 ± 0.4104.2 ± 1.0
Hammer118.0 ± 2.3116.5 ± 1.7123.0 ± 4.7120.5 ± 3.8125.9 ± 0.3124.9 ± 3.4
Relocate74.4 ± 1.670.3 ± 3.077.6 ± 1.580.6 ± 1.075.0 ± 7.978.3 ± 3.1

Over three training seeds, UDT averages 63.5 ± 0.6 under Drift-Attack, about five points above either baseline. UDT leads on 13 of 15 clean summary metrics and 11 of 15 under drift. The ± values show the standard deviation across training runs. With five seeds, we could estimate that spread more precisely.

AuthorsOpenReview ↗

How reliable are the three-seed results, and how hard is Drift-Attack? (W1)

Under drift, UDT scores much lower on Kitchen-Mixed and stays close to its clean score on Walker2d. On Kitchen-Mixed, UDT scores 64.6 in the clean environment and 41.4 under drift. On Walker2d, it scores 44.8 in the clean environment and 45.7 under drift.

To check whether the routing gain holds across training seeds, we switch routing on and off in the same trained model. We pick the checkpoint and target return with routing on, using one evaluation seed. We then run both modes under Drift-Attack on a different seed, keeping the checkpoint and target return fixed.

Table R3. Routing on vs. off under Drift-Attack with the same trained checkpoint. D4RL normalized returns. Paired differences are mean ± SD across three training-seed averages.

RoutingOverallObservationActionReward
On56.548.952.568.1
Off51.841.148.266.1
On minus off4.7 ± 0.97.8 ± 3.14.3 ± 4.02.0 ± 1.7

Turning on routing improves the average score for each of the three training seeds. The paired gain is 4.7 ± 0.9 points overall. Routing gives a higher score in 95 of 162 comparisons across settings and seeds.

Reviewer_pLfjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper defines a class of robustness in offline reinforcement learning termed temporal structure robustness transfer (TS-RT) wherein a model trained on corrupted data with no temporal relationship may exhibit robustness to test-time corruption with temporal dependencies. The paper then proposes a solution in Uncertainty-guided Decision Transformer (UDT), a transformer architecture that deals with persistent corruption in the token sequence by learning uncertainty and causal attention weights to lower attention towards potentially corrupt tokens. Finally, the paper introduces DriftAttack, a method designed to induce temporally-controlled corruption to assess TS-RT.

Critical strengths
  • The thoroughness of the paper is appreciated. The experiments are sufficient, each component is validated.
  • The argument is easy to follow throughout the paper.
Critical weaknesses
  • Although the paper provides a clear description of the type of robustness being addressed, the methodology to do so, and the evaluation protocol, it is unclear what the actual problem is. That is, one could argue that the paper insists upon itself, i.e. that the TS-RT phenomenon is a contrived scenario to justify the UDT algorithm's existence. This is not so critical that the paper cannot be accepted; for example it could be that the Drift-attack is more successful against naive approaches.
  • related to the above, but more critical: the paper does not show unperturbed test performance, a critical component of understanding robustness. UDT outperforms other methods under drift-attack, but it cannot be determined from the experiments if this is because UDT is robust to drift-attack, if it is more robust overall, or if it simply scores higher overall.
Questions for authors

Could the authors expand on the prevalence of temporal robustness transfer, as well as the problem inspiring this work? Additionally, what conditions would cause us to expect low transfer of robustness, and what are the implications of solving this problem?

3 comments
AuthorsOpenReview ↗

Why test whether robustness to isolated errors carries over to persistent faults? (W1, Q1)

1. Why test whether robustness to isolated errors carries over to persistent faults? (W1, Q1)

A robot arm may develop a persistent encoder bias even if its training logs contain only occasional bad readings. Temporal-structure robustness transfer (TS-RT) asks whether a policy that handles isolated errors can also handle persistent errors on the same input. We use the same protocol for UDT and all seven baselines in Table 1, including RIQL and RDT (Eq. 5).

Persistent faults already arise in robot control. In a hardware experiment, Fu et al. (CoRL 2025) test a six-legged robot walking with two damaged joints that produce zero torque. In simulation, Vogt-Lowell et al. (2026) study persistent sensor failures in MuJoCo. Liang et al. (ICLR 2024) study temporally coupled disturbances in continuous control, while Ada et al. (NeurIPS 2025) examine time-varying observation offsets in offline RL. Xu et al. (ICLR 2025) study how policies trained on corrupted offline data handle observation perturbations at test time. It is still an open question how often a component with isolated errors in the training logs develops a persistent fault at deployment.

We compare Uncertainty-Guided Decision Transformer (UDT) and Robust Decision Transformer (RDT) under independent and persistent test errors with similar rates and magnitudes. For IID-Matched, we sample magnitudes from Drift-Attack independently at each step. The samples include zeros, so the error rate and magnitude distribution stay the same in expectation. We use the submitted Drift-Attack scores for the same 18 settings on Walker2d, Kitchen-Mixed, and Relocate.

Table R1. UDT and RDT under independent and persistent errors of similar frequency and size. D4RL normalized returns. The ± values are standard deviations across three training seeds. The Drift-Attack overall scores average the 18 setting means from Tables 7 and 8 of the submission. Parentheses indicate the training corruption.

SettingIID-Matched / RDTIID-Matched / UDTDrift-Attack / RDTDrift-Attack / UDT
Overall (18 settings)51.28 ± 1.3056.21 ± 1.2051.6455.14
Relocate (random action)90.02 ± 5.9073.96 ± 6.3175.3 ± 5.577.9 ± 1.7
Kitchen-Mixed (random action)36.67 ± 10.1047.50 ± 4.3337.5 ± 5.436.7 ± 1.4

The policy ranking can change even when errors have similar rates and magnitudes. On Relocate, RDT leads under IID-Matched and UDT leads under Drift-Attack. On Kitchen-Mixed, UDT leads under IID-Matched and RDT leads under Drift-Attack. On average across the settings, UDT leads by 4.93 points under IID-Matched and about 3.50 points under Drift-Attack. The rankings are descriptive comparisons across two separate evaluations.

On Walker2d, the two tests produce observation errors of similar frequency and size. Under Drift-Attack and IID-Matched, the error rates are 24.70% and 24.59%, and the root-mean-square magnitudes are 0.4187 and 0.4168, respectively. The correlation between consecutive magnitudes is 0.7408 under Drift-Attack and 0.0008 under IID-Matched.

We expect weak transfer when a policy keeps using several corrupted observations after the sensor recovers. We check for this residual effect at the first clean step. We predict the action with the same policy twice, once with a burst in the observation history and once with a clean history. We divide the difference by the deviation after a one-step burst. As the burst length increases, the normalized deviation grows more slowly for UDT than for RDT on 7 of 9 tasks (Fig. 3). Limiting the influence of old errors could help the policy recover sooner.

AuthorsOpenReview ↗

Clean performance and what routing adds under Drift-Attack (W2)

2. Clean performance and what routing adds under Drift-Attack (W2)

The clean tests tell us whether UDT already has a lead before we add test-time errors. We compare UDT, Decision Transformer (DT), and RDT on all nine D4RL tasks and six training corruption settings. We use three training seeds in each test condition and select checkpoints separately for Clean and Drift-Attack, so the comparison is descriptive.

Table R2. Clean and Drift-Attack returns for DT, RDT, and UDT. D4RL normalized returns (mean ± SD over three training seeds). Drift-Attack values come from Table 1 of the submission.

MetricDT / CleanDT / DriftRDT / CleanRDT / DriftUDT / CleanUDT / Drift
Overall63.1 ± 0.658.5 ± 0.765.8 ± 0.958.4 ± 0.569.6 ± 1.463.5 ± 0.6
Training corruption
Random67.0 ± 0.261.8 ± 1.467.2 ± 1.159.9 ± 1.070.5 ± 0.965.5 ± 1.1
Adversarial59.2 ± 1.055.2 ± 0.164.4 ± 0.956.9 ± 0.668.7 ± 2.261.6 ± 0.8
Corrupted component
Observation66.1 ± 1.660.2 ± 1.463.9 ± 0.351.0 ± 1.168.0 ± 1.456.6 ± 2.7
Action50.6 ± 1.245.6 ± 0.560.9 ± 2.754.1 ± 0.464.3 ± 2.159.3 ± 1.9
Reward72.6 ± 0.469.8 ± 1.572.6 ± 0.670.1 ± 1.876.5 ± 0.874.7 ± 1.0
MuJoCo
HalfCheetah18.3 ± 1.218.4 ± 1.120.9 ± 2.819.7 ± 1.525.9 ± 1.527.3 ± 0.3
Hopper48.8 ± 3.545.9 ± 2.549.8 ± 0.847.7 ± 1.955.0 ± 2.553.6 ± 1.7
Walker2d38.7 ± 2.836.5 ± 0.840.6 ± 3.039.6 ± 0.944.8 ± 1.045.7 ± 2.3
Kitchen
Kitchen-Complete71.9 ± 1.760.7 ± 0.964.9 ± 2.943.3 ± 2.165.3 ± 2.144.4 ± 2.5
Kitchen-Mixed45.1 ± 2.438.5 ± 0.754.4 ± 1.334.7 ± 1.964.6 ± 1.441.4 ± 4.8
Kitchen-Partial48.2 ± 3.936.5 ± 2.956.8 ± 3.434.7 ± 5.164.9 ± 3.051.9 ± 2.4
Adroit
Door104.4 ± 0.3103.3 ± 1.6104.5 ± 0.2104.6 ± 0.2104.8 ± 0.4104.2 ± 1.0
Hammer118.0 ± 2.3116.5 ± 1.7123.0 ± 4.7120.5 ± 3.8125.9 ± 0.3124.9 ± 3.4
Relocate74.4 ± 1.670.3 ± 3.077.6 ± 1.580.6 ± 1.075.0 ± 7.978.3 ± 3.1

UDT already leads in clean tests, which may explain part of its advantage under Drift-Attack. It ranks first on 13 of 15 summary metrics in Clean and 11 of 15 under drift. Its lead over DT narrows from 6.5 points in Clean to 5.0 under Drift-Attack. Its lead over RDT widens from 3.8 to 5.1 points.

We switch routing on and off in the same trained model to measure what it adds under drift. We pick the checkpoint and target return with routing on, using one evaluation seed. We then run both modes under Drift-Attack on a different seed, keeping the checkpoint and target return fixed.

Table R3. Routing on vs. off under Drift-Attack with the same trained checkpoint. D4RL normalized returns. Paired differences are mean ± SD across three training-seed averages.

RoutingOverallObservationActionReward
On56.548.952.568.1
Off51.841.148.266.1
On minus off4.7 ± 0.97.8 ± 3.14.3 ± 4.02.0 ± 1.7

Switching on routing raises the Drift-Attack score by 4.7 ± 0.9 points with the trained model held fixed. The gain is positive on average for all three training seeds. Routing gives a higher score in 95 of 162 comparisons across settings and seeds.

AuthorsOpenReview ↗

Looking forward to your feedback (Urgent)

We hope that our rebuttal has effectively addressed all your concerns. As the deadline for the discussion period draws near (less than 3 hours), we would greatly appreciate it if you could re-evaluate our manuscript and provide any feedback. Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary

We thank all three reviewers for their time and constructive feedback. We appreciate Reviewer pLfj’s recognition of the thorough evaluation and component analyses, Reviewer uW7q’s recognition of the problem formulation and uncertainty-guided attention mechanism, and Reviewer peoB’s recognition of the empirical gains across three benchmark families.

Temporal-structure robustness transfer (TS-RT) asks whether a policy trained with isolated errors can handle persistent errors in the same input at deployment. Uncertainty-Guided Decision Transformer (UDT) uses learned token uncertainty to modulate causal attention over the trajectory history and to weight the prediction loss during training. Across nine D4RL tasks and six training corruption settings, UDT achieves a mean normalized return of 63.5 under Drift-Attack, 5.0 points above DT and 5.1 above RDT. In clean evaluations, UDT averages 69.6, compared with 63.1 for DT and 65.8 for RDT.

For Reviewer pLfj, we motivate TS-RT with persistent faults in robot control, report clean and Drift-Attack returns side by side, and evaluate routing at inference by switching it on and off with the same checkpoint and target return. For Reviewer uW7q, we add a worked Walker2d example of uncertainty-guided attention, explain how to apply the mechanism to a standard Decision Transformer, clarify the corruption protocol, ablation training, and post-burst action deviation metric, and describe the planned correction to Figure 2. For Reviewer peoB, we compare independent and persistent test errors with matched error rates and magnitude distributions, compare clustered and scattered errors at a fixed corruption budget, and add an ablation that disables both routing and uncertainty-based loss weighting.

We thank the reviewers, Area Chairs, Senior Area Chairs, and Program Chairs for their time and consideration. Their feedback helped us clarify the evaluation protocol and better characterize how UDT responds to persistent corruption.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.