acceptodds
Under review as a conference paper at ICLR 2027

Corruption-Robust Offline Reinforcement Learning via Causal Dynamics Adherence

Abstract

Offline reinforcement learning (RL) via sequence modeling is inherently vulnerable to training data corruptions. We observe that while data can be corrupted, the causal dynamics of the environment, such as state transitions, can remain invariant. Motivated by this key observation, we propose Causal Dynamics Adherence (CDA), a mechanism that disentangles task-relevant causal information from corruptions by constraining representations to adhere to environmental causal dynamics. To mitigate the attention leakage to corrupted inputs induced by Softmax in Decision Transformer (DT), we introduce Adaptive Sparse Attention (ASA), a plug-and-play module that augments Sparsemax with layer-specific learnable temperatures to adaptively suppress such leakage. On the D4RL benchmark, CDA-DT, which integrates CDA and ASA into DT, achieves the highest scores on 47 out of 54 (87%) tasks. The code is provided in the supplementary material and will be publicly released. Training results are available via an anonymous link.

Primary area:
reinforcement learning
Keywords:
corruption-robust offline RL, decision transformer, causal dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ANedOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a new architecture to address data corruption in offline reinforcement learning. In specific, the paper disentangle the part that is markovian in the sense that the next step is determined by the current step and current action with the part that is noise. They do so by separately embed the corruption and causal. Then they add penalty term to make the two embeddings disentangled. Numerically, the authors show they achieve the highest score on 47 tasks out of 54 tasks.

Critical strengths

The idea of separating the part that follows markovian property and believing the rest is corrupted noise is interesting.

Critical weaknesses

1, The empirical improvement is minor, almost all tasks' scores confidence interval are overlapping and the average improvement over RDT is only 4-8 percent. This questions the real benefits of the specific way of implementing the idea.

2, The authors argue that the noise embedding and causal embedding and disentangled by showing that the embeddings are not similar. However, disentanglement in representation learning means changing the corruption would only affect corruption without affecting the causal embedding [1]. This further questions whether the proposed method really separate the causal part and the corruption part.

[1] Representation Learning: A Review and New Perspectives. Yoshua Bengio, Aaron Courville, and Pascal Vincent

Questions for authors

1, Could the authors provide more evidence that the causal and corruption embeddings are disentangled?

2, Could the authors show additional experiments where the difference between proposed method and RDT are well separated?

8 comments

We thank you for recognizing the dynamics-based motivation. Below, we address each of your concerns and questions.

AuthorsOpenReview ↗

1. Reliability of the empirical improvements (W1, Q2) (Part 1/2)

CDA-DT achieves the best mean score on 47 of 54 (87%) tasks, indicating a consistent improvement across corruption types and datasets. Our empirical improvements extend across a larger proportion of tasks than those reported in many RL studies. LSDT (ICLR 2025) achieves the highest reported mean on two of nine (22.2%) locomotion datasets. CURL (ICML 2020) reports state-of-the-art scores on seven of 26 (26.9%) Atari100k games. EDAC (NeurIPS 2021) achieves the highest mean in its EDAC column on six of 18 (33.3%) Gym datasets. SPOT (NeurIPS 2022) achieves the highest mean on eight of 15 (53.3%) offline RL datasets. PEER (CVPR 2023) reports state-of-the-art scores on 19 of 26 (73.1%) Atari100k games.

The number of improved tasks is a valid and widely used evaluation metric in RL. RDT (ICLR 2025) highlights the best scores on 24 of 27 random-corruption settings. CURL (ICML 2020) highlights improvements over Efficient Rainbow on 19 of 26 Atari games. EfficientZero V2 (ICML 2024) highlights outperforming DreamerV3 on 50 of 66 tasks. MOSDT (NeurIPS 2025) highlights the highest safe returns on 14 of 18 tasks. PEER (CVPR 2023) similarly highlights that adding its representation regularizer improves CURL on 19 of 26 Atari games and DrQ on 16 of 26. Following these works, we use the number of tasks improved as a primary source of feedback when developing our components.

The number of improved tasks helps assess a component's applicability and generality across environments. Each task contributes one vote, preventing a few extreme gains or losses from dominating the count. The mean is sensitive to both extreme changes and base scores: the same percentage change produces a larger absolute effect on higher-scoring tasks. The two metrics serve different reporting needs. We therefore emphasize improved-task counts in the paper's detailed analysis to show consistency across environments, and mean scores for concise comparisons in the rebuttal.

Compared with RDT, the strongest baseline by overall score, CDA-DT has higher mean scores in all six corruption types and increases the overall score from 66.63 to 70.51, corresponding to a 5.82% gain. Gains of this magnitude have direct precedents in offline RL. Decision ConvFormer (ICLR 2024) reports a locomotion mean of 82.2 versus 81.0 for ODT, the strongest baseline on that benchmark, corresponding to a 1.48% gain. Decision Mixer (ICML 2025) reports a Gym mean of 84.7 versus 82.4 for CGDT, its strongest baseline on that benchmark, yielding a 2.79% gain. RORL (NeurIPS 2022) reports a Gym mean of 85.7 versus 82.9 for EDAC, its strongest baseline by average score, yielding a 3.38% gain. QT (ICML 2024) reports an average of 76.5 versus 73.7 for Diffusion-QL, the strongest baseline on its four AntMaze datasets, yielding a 3.80% gain. EDAC (NeurIPS 2021) reports a Gym mean of 85.2 versus 84.5 for SAC-N, the strongest reference configuration introduced in the same paper, yielding a 0.83% gain. CDA-DT's 5.82% gain over RDT is larger than the improvements reported in these published studies.

AuthorsOpenReview ↗

1. Reliability of the empirical improvements (W1, Q2) (Part 2/2)

The reported error bars are sample standard deviations, rather than confidence intervals. They show how much each method's score varies across training seeds. Overlapping error bars alone do not tell us whether CDA-DT's improvement over RDT is statistically significant.

Overlapping error bars are also common in RL studies. In ReBRAC (NeurIPS 2023), the error bars of ReBRAC and SAC-RND, the strongest baseline by average score, overlap on five of six AntMaze tasks. In SPOT (NeurIPS 2022), the error bars of SPOT and IQL, identified as the strongest baseline in its offline experiments, also overlap on five of six AntMaze tasks. In PEER (CVPR 2023), the error bars of PEER+CURL and DrQ overlap on all six DMControl500k tasks.

To further assess the consistency of these gains, we repeat the comparison with two additional random seeds across all 54 tasks. Results are shown in Table R1.

Table R1. Comparison with RDT over two additional random seeds. Each score is averaged over all nine datasets and the two new seeds. All other training and evaluation settings are identical to those in the main experiments (Table 1). Bold marks the highest score in each row.

CorruptionRDTCDA-DT (ours)
Adversarial action57.5462.59
Adversarial state64.0572.04
Adversarial reward72.9475.53
Random action63.4875.33
Random state65.0966.23
Random reward69.8680.60
Overall65.4972.05

Table R2 combines the existing four-seed results in the submitted paper's Table 1 with the results under two additional seeds in Table R1, providing a six-seed comparison.

Table R2. Comparison with RDT combining the four original seeds in Table 1 with the two additional seeds in Table R1. Each score averages all nine datasets and six seeds. All other training and evaluation settings are identical to those in the main experiments (Table 1). Bold marks the highest score in each row.

CorruptionRDTCDA-DT (ours)
Adversarial action56.7160.66
Adversarial state66.4972.42
Adversarial reward72.1876.06
Random action66.0571.84
Random state63.7067.38
Random reward72.3677.79
Overall66.2571.02

The six-seed results retain CDA-DT's advantage in all six corruption types, with an overall mean of 71.02 versus 66.25 for RDT. We have added these new results in the revised version.

Changes in the revised manuscript (blue). Section 5.1 clarifies the error bars. Section 5.2 distinguishes strict wins from ties and summarizes additional-seed results. Section 5.3 explains the task-count metric. Appendix A.6 gives the metric rationale. Appendix A.18 reports the additional-seed and six-seed results (Tables 16–17). Appendix A.19 contextualizes the gains and overlapping error bars with published studies.

Reviewer_wrtoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CDA-DT for offline reinforcement learning from corrupted training data. This paper adds two components to Decision Transformer: 1. Causal Dynamics Adherence (CDA): splits embeddings into “causal” and “corruption” branches, trained using dynamics consistency, orthogonality regularization and reconstruction objectives 2. Adaptive Sparse Attention (ASA): replaces softmax attention with sparsemax and learns a temperature for each layer. The paper evaluates on D4RL benchmark across 9 datasets under 6 corruption conditions and reports improved average performance over the evaluated baselines.

Critical strengths

The paper evaluates 9 D4RL datasets with random and adversarial corruption of states, actions, and rewards. Across the 54 settings, CDA-DT achieves an average score of 70.51, compared with 66.63 for RDT and 63.43 for DT. These results provide encouraging evidence of performance improvements over the evaluated baselines under the tested corruption settings.

Critical weaknesses
  1. Limited novelty: The method adds two components to Decision Transformer. Both components are closely related to existing methods, but these connections are not discussed.
  • CDA: Domain Separation Networks (NeurIPS 2016) already combines shared/private representations, orthogonality regularization, reconstruction from combined representations, and task prediction from the shared branch. CDA adds a dynamics-consistency objective, but the authors do not cite this work or clearly explain the methodological differences.
  • ASA: The paper claims to be the first to enable layer-adaptive sparsity in Sparsemax. However, Adaptively Sparse Transformers (EMNLP 2019) already learns sparsity across attention heads and layers, while ASEntmax (ICLR 2026) introduces learnable temperature scaling for sparse attention. Neither paper is cited or discussed

References:

  • Bousmalis et al. Domain Separation Networks. NeurIPS, 2016.
  • Correia et al. Adaptively Sparse Transformers. EMNLP-IJCNLP, 2019.
  • Vasylenko et al. Long-Context Generalization with Sparse Attention (ASEntmax). ICLR, 2026.
  1. The paper claims to disentangle task-relevant causal information from corruption, but this claim is not sufficiently supported:
  • The dynamics predictor and embeddings are jointly learned from corrupted data, which does not guarantee causal identification. Although the action-prediction loss may discourage collapse, the auxiliary objectives do not guarantee the claimed separation.
  • The paper provides no theoretical justification or identification conditions showing that the proposed objectives recover task-relevant causal information.
  • The cosine similarities and t-SNE visualizations only demonstrate geometric properties of the representations, while policy performance measures empirical effectiveness. None directly establishes that the causal branch retains task-relevant information while excluding corruption. In particular, a constant nonzero causal embedding can also achieve perfect clean–corrupted cosine similarity.
  1. The experiment does not sufficiently establish the claimed performance advantage.

Missing recent baselines and related work

  • ADG (NeurIPS 2025), already cited in the paper, is an important missing baseline. It directly addresses corrupted offline datasets and reports results with DT and RDT.
  • Recent DT variants: DC (ICLR 2024), LSDT (ICLR 2025), and Decision Mixer (ICML 2025). While BC and RBC are useful references, comparisons with these newer sequence-modeling approaches are also important.
  • Other related methods missing from discussion: CSET (ICML 2026), which addresses causal confusion and robustness, and value-guided DT variants such as QT (ICML 2024) and VDT (NeurIPS 2025) are also missing from the related-work discussion.

Also, the reported “47 out of 54” best results include 44 strict wins and three ties in mean score. Some improvements are small relative to the reported standard deviations, making their statistical reliability unclear. Furthermore, the lack of comparisons with recent strong baselines makes it difficult to establish the method's overall superiority.

References:

  • Kim et al. Decision ConvFormer (DC) . ICLR, 2024.
  • Wang et al. Long-Short Decision Transformer (LSDT). ICLR, 2025.
  • Zheng et al. Decision Mixer. ICML, 2025.
  • Wang et al. Reward-Preserving Counterfactual State Editing for Offline Reinforcement Learning (CSET). ICML, 2026.
  • Hu et al. Q-value Regularized Transformer for Offline Reinforcement Learning (QT). ICML, 2024.
  • Zheng et al. Value-Guided Decision Transformer (VDT). NeurIPS, 2025.
Questions for authors
  1. What does the paper mean by “task-relevant causal information”? How do the proposed objectives ensure that the causal branch captures this information while removing corruption? Beyond cosine similarity and t-SNE, is there any evidence that the learned representation retains useful task information rather than collapsing to a constant embedding?

  2. How is CDA different from Domain Separation Networks, particularly in its representation learning approach? Similarly, what is the main difference between ASA and existing adaptive sparse attention methods, such as Adaptively Sparse Transformers and ASEntmax?

  3. Does the method assume that the corrupted modality is known in advance? Since the other modalities are used as clean references, how would the method handle cases where multiple modalities are corrupted?

  4. In the ablation study, adding reconstruction improves only 24 of 54 settings while worsening 27, reducing the average score from 70.89 to 70.49. Similarly, adding ASA to full CDA improves 25 settings but worsens 28, with an average gain of only 0.02 points. How do the authors explain these results, and what evidence supports the benefits of these components?

  5. The ablation base model is defined as CDA-DT without CDA and ASA. Based on my understanding, this should be similar to vanilla DT. However, their average scores differ substantially (67.33 vs. 63.43). What accounts for this difference?

  6. Why is ADG, which is already cited in the paper, not included as an experimental baseline? Can the authors provide comparisons with ADG+DT or ADG+RDT? Comparisons with recent DT variants, such as DC, LSDT, or Decision Mixer, would also help establish the method's performance advantage.

  7. How are the reported evaluation scores selected? Are they based on the final checkpoint, the best evaluation score during training, or a separate evaluation of a selected checkpoint? How are the target returns selected, and is the same procedure used for all baselines?

12 comments

We thank you for recognizing the improvements across the 54 corruption tasks. Below, we address each of your concerns and questions.

AuthorsOpenReview ↗

1. Explanation of novelty (W1, Q2)

Our main contribution is an environmental-dynamics criterion for learning corruption-robust policy representations, complemented by a lightweight layer-wise temperature parameterization for sparse attention. We thank you for highlighting the connections to prior work. CDA shares representation-disentanglement ideas with Domain Separation Networks (DSN, Bousmalis et al., 2016), while ASA shares the idea of learning to control attention sparsity with Adaptively Sparse Transformers (AST, Correia et al., 2019) and ASEntmax (Vasylenko et al., 2026). We clarify these connections below and have cited the corresponding references in the revised manuscript.

CDA differs from DSN in its application scenarios, data organization, representation-learning criterion, and disentanglement constraint. (i) Application scenarios. DSN addresses unsupervised domain adaptation in visual recognition. CDA addresses offline RL with corrupted training trajectories. (ii) Data organization. DSN uses separate source and target datasets, with a shared encoder and domain-specific private encoders. CDA uses one partially corrupted offline dataset and splits the affected state, action, or return embedding into causal and corruption branches, without sample-level clean/corrupted labels or paired clean targets. (iii) Representation-learning criterion. DSN aligns shared feature distributions across domains through a domain-adversarial or MMD loss. CDA instead constrains causal embeddings to a dynamics-consistent causal manifold through state-transition consistency, (st,at)→st+1(s_t,a_t)\rightarrow s_{t+1}, or return-evolution consistency, (Rt,st,at)→Rt+1(R_t,s_t,a_t)\rightarrow R_{t+1}, using detached embeddings of unaffected modalities as references. Thus, environmental causal relations provide the semantic criterion for extracting task-relevant information. (iv) Disentanglement constraint. DSN penalizes shared–private feature cross-correlations aggregated over samples. CDA penalizes the squared cosine similarity between causal and corruption embeddings of each input token.

ASA differs from AST and ASEntmax in its application scenarios, sparsity parameter, and temperature control. (i) Application scenarios. AST is applied to machine translation, and ASEntmax to synthetic sequence tasks and language modeling for long-context generalization. ASA is applied to offline RL with corrupted training trajectories. (ii) Sparsity parameter. AST learns the entmax shape parameter α\alpha for each attention head. ASA fixes α=2\alpha=2, i.e., Sparsemax, and learns a logit temperature for each layer. (iii) Temperature control. ASEntmax uses head-specific scaling that depends on query content and context length. ASA uses one positive scalar per layer, shared across queries and independent of content and context length, updated every five epochs. Our three-layer model therefore adds only three temperature parameters.

The submitted ablations in Fig. 2 identify concrete contributions to the gains. The dynamics loss improves 33 of 54 tasks. Adding orthogonality and reconstruction raises this count to 41 and 44, respectively. Fixed Sparsemax only improves 28 tasks, while our ASA improves 39.

Changes in the revised manuscript (blue). Sections 1–2 revise ASA's priority claim and clarify the contribution with citations to DSN, AST, and ASEntmax. Sections 4.2 and 5.3 introduce the ASA comparison and component analysis, respectively. Appendix A.6 gives the component counts. Appendix A.20 details the methodological differences.

AuthorsOpenReview ↗

2. Task-relevant causal information: definition, learning mechanism, and evidence against collapse (W2, Q1)

We thank you for your insightful questions. By "task-relevant causal information", we mean information about the causal relationships among states, actions, and rewards. This information helps a policy achieve high task returns and remains stable under corruption of the recorded data. For example, in Adroit, grasping and moving the handle opens the door; in Kitchen, opening the microwave completes a subtask and earns a reward; in MuJoCo, leg movements change the robot's speed and balance.

The losses jointly encourage the causal branch to retain useful information while separating it from corruption. The dynamics loss constrains the causal embedding to a dynamics-consistent causal manifold, providing a semantic criterion for extracting task-relevant causal information. The action loss directly supervises the causal embedding to retain task-relevant information. The orthogonality loss penalizes alignment between the causal and corruption branches. The reconstruction loss preserves the input through both branches to prevent representation collapse during the unsupervised disentanglement.

We directly test whether the causal embedding retains state information rather than collapsing to a constant, and whether this information remains stable to corruption. Results are shown in Table R1. We have added these new results in the revised version. With trained encoders fixed, we sample 1,000 clean states per dataset and ten noise vectors matching the random state-corruption in main experiments. We compare embeddings under two conditions: the same state with different noise, and different states with the same noise. The first tests sensitivity to corruption, while the second checks that the causal embedding retains state information rather than collapsing to a constant. We measure changes using average squared distances between embeddings. For each branch, we set the average distance when both state and noise differ to 1 and report all other distances relative to it (0 means no change).

Table R1. Testing representation collapse and sensitivity to corruption through relative embedding changes. Each value is averaged across all 18 state-corruption tasks (nine datasets × random/adversarial corruption) and four random seeds. Ratios are computed for each model before averaging.

ComparisonCausal embedding changeCorruption embedding change
Different states, same noise0.9210.170
Same state, different noise0.0940.883

According to Table R1, the causal branch can distinguish states, ruling out the explanation that it appears invariant simply because it always outputs a constant. Besides, the causal embedding remains nearly unchanged while the corruption embedding responds strongly to the change in noise. Specifically, changing only the state produces a causal-embedding change of 92.1% of its baseline, but a corruption-embedding change of 17.0%. Changing only noise gives the opposite pattern: just 9.4% for the causal embedding and 88.3% for the corruption embedding.

Together, these comparisons rule out the constant-embedding explanation: the causal branch retains state-dependent information while remaining largely insensitive to noise. Its task usefulness is supported by Table 1, where the policy uses only causal embeddings to generate actions and achieves an overall score of 70.51, versus 66.63 for RDT.

Changes in the revised manuscript (blue). Section 4.1 explains the joint loss roles. Section 5.3 describes the test against constant embeddings and relates the causal-only policy to task performance. Appendix A.17 gives the definition, task examples, experimental protocol, and results (Table 15).

Reviewer_P2dDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CDA-DT for offline RL with corrupted training data. Its Causal Dynamics Adherence (CDA) module splits token embeddings into causal and corruption branches. Learned latent predictors encourage consistency with state-transition and return-evolution structure, while orthogonality and reconstruction objectives regularize the split. Adaptive Sparse Attention (ASA) replaces Softmax with Sparsemax and learns a temperature for each Transformer layer. Evaluation covers 9 D4RL datasets under random/adversarial corruption of states, actions, or rewards. The paper also evaluates combined training corruption and test-time attacks, and provides component, representation, attention, and hyperparameter analyses.

Critical strengths
  1. The evaluation is reasonably broad and controlled. It covers random and adversarial corruption across state, action, and reward channels, compares relevant methods including RDT and RIQL, and reports variability over four shared seeds.
  2. The paper includes useful component and mechanism analyses. It compares standard Sparsemax with ASA, incrementally adds the CDA losses, checks representation consistency and attention to corrupted tokens, and tests hyperparameter sensitivity.
Critical weaknesses
  1. The robustness setting is limited. The method assumes it knows which data type is corrupted and uses the others as clean anchors. Experiments only corrupt one type at a fixed 30% rate, leaving unknown or mixed corruption untested.
  2. The benefit of the full design is unclear. Adding reconstruction lowers the average ablation score, while ASA adds little to the overall mean on top of CDA. The paper also lacks a comparison with ADG, a directly relevant NeurIPS 2025 method.
  3. The evaluation scope is limited. All experiments use nine state-based D4RL datasets with synthetic corruptions. This leaves the method’s generality to visual observations and more realistic data errors unclear.
Questions for authors
  1. How much does performance depend on knowing the corrupted data type and retaining clean anchors? A limited experiment with unknown or mixed corruption types, or a corruption-rate sweep, would help establish the applicability of the method beyond the current setting.
  2. Why is ADG absent from the experimental comparison despite being cited and addressing the same corruption-robust offline RL problem? A matched comparison, or a clear explanation of incompatible assumptions, would materially strengthen the assessment of competitiveness.
  3. Can the representation analysis distinguish causal-corruption separation from ordinary predictive regularization? The current orthogonality and invariance checks are useful, but a matched-capacity dynamics-only control and broader diagnostics across corruption types would help substantiate the mechanism.
8 comments

We thank you for recognizing the controlled evaluation and the component analyses. Below, we address each of your concerns and questions.

AuthorsOpenReview ↗

1. Robustness settings (W1, Q1) (Part 1/2)

We thank you for your insightful questions about our robustness settings. Assuming that the corrupted data type is known in advance has direct precedents in corruption-robust RL. RDT (ICLR 2025) explicitly restricts action correction to action-corruption settings and reward correction to reward-corruption settings, thus relying on prior knowledge of the corruption type. RPEX (NeurIPS 2025) selects different policy-extraction rules for different attack types, so its reported policy-learning procedure also depends on knowing the corruption type. ADG (NeurIPS 2025) likewise prespecifies which data channels to recover, relying on prior knowledge of the corruption type. RIQL (ICLR 2024) likewise selects quantile and Huber-loss hyperparameters by the affected data type. UWMSG (NeurIPS 2023) uses different uncertainty-weighting coefficients for reward and dynamics corruption in the same environment. CDA-DT uses this same level of prior information to select the encoder and consistency objective for the affected field.

Using uncorrupted information as reference anchors also has direct precedents in robust RL. State-Adversarial RL (NeurIPS 2020), RORL (NeurIPS 2022), and RADIAL-RL (NeurIPS 2021) use unperturbed states as consistency references, while DMBP (ICLR 2024) and ADG (NeurIPS 2025) use clean information to guide recovery.

Our single-type corruption protocol and fixed 30% rate follow established evaluation settings in corruption-robust RL. RIQL (ICLR 2024), RDT (ICLR 2025), ADG (NeurIPS 2025), RPEX (NeurIPS 2025), and TRACER (NeurIPS 2024) all include single-type corruption evaluations at a 30% corruption rate.

AuthorsOpenReview ↗

1. Robustness settings (W1, Q1) (Part 2/2)

Some prior papers include mixed-corruption or corruption-rate sweeps. We agree that these are useful complementary experiments and add them below.

We report the unknown-type and full-corruption comparisons in Table R1 and have included these new results in the revised manuscript. Under "Single-type, unknown", we use the same 30%-corrupted datasets as in the main experiments but do not tell the learner whether states, actions, or rewards are corrupted. Under "Full-corruption", following ADG, we select 30% of transitions and corrupt their states, actions, and rewards together. For both new settings, we activate the state, action, and return dual-head embedders and their consistency losses in CDA-DT.

Table R1. Results under unknown-type and full corruption. The default and "single-type, unknown" scores average all 54 tasks (nine datasets × random/adversarial corruption × three corrupted data types). "Full-corruption" scores average 18 tasks (nine datasets × random/adversarial corruption). Each score is averaged over four random seeds. All other training and evaluation settings are identical to those in the main experiments (Table 1). Bold marks the best score in each row.

SettingRDTCDA-DT (ours)
Single-type, known (default)66.6370.51
Single-type, unknown64.9268.56
Full-corruption55.8162.94

The advantage over RDT is 3.64 points under unknown single-type corruption and 7.13 points under full corruption, indicating that CDA-DT retains an advantage without prior knowledge of the corrupted data type and when all modalities are corrupted together.

In addition, we report the state-corruption rate sweep in Table R2 and have included these new results in the revised manuscript. We vary the state-corruption rate over {10%, 30% (default), 50%}.

Table R2. Results of the state-corruption rate sweep. Each score is averaged over all 18 state-corruption tasks (nine datasets × random/adversarial corruption) and four random seeds. All other training and evaluation settings are identical to those in the main experiments (Table 1). Bold marks the best score in each row.

State-corruption rateRDTCDA-DT (ours)
10%72.6874.26
30% (default)65.3570.28
50%57.4364.39

The advantage over RDT increases from 1.58 points at 10% corruption to 6.96 points at 50%. Table R2 shows that CDA-DT remains effective beyond the original state-corruption rate and degrades more gradually as training data become less reliable.

Changes in the revised manuscript (blue). Section 4.1 clarifies the known-type assumption and activation for unknown or multiple corrupted modalities. Section 5.2 introduces the extended settings. Appendix A.15 adds the unknown-type and full-corruption results (Table 9) and corruption-rate sweep (Table 10), with protocols and prior-work precedents.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary

We thank all three reviewers for their time and constructive feedback. We appreciate Reviewer P2dD’s recognition of the controlled evaluation and component analyses, Reviewer wrto’s recognition of the empirical gains across 54 corruption tasks, and Reviewer ANed’s recognition of the dynamics-based motivation. In response to their feedback, we expanded the main text from 9 to 10 pages and the full manuscript from 22 to 36 pages. The revised appendix adds 9 subsections (A.15–A.23) and 15 tables (Tables 9–23).

CDA-DT leverages environmental dynamics that can remain stable even when offline RL data are corrupted. Causal Dynamics Adherence (CDA) uses state-transition and return-evolution consistency to learn policy embeddings that preserve task-relevant information while separating it from corruption. Adaptive Sparse Attention (ASA) learns one Sparsemax temperature per Transformer layer to reduce attention to corrupted tokens. On nine D4RL datasets, CDA-DT achieved the best mean score on 47 of 54 tasks, improving the overall score from 66.63 to 70.51 (+5.82%) over RDT, the strongest baseline in the original comparison.

We added experiments and analyses in response to concerns raised by all three reviewers. For Reviewer P2dD, we tested unknown and simultaneous corruption across modalities, varied the state-corruption rate, and evaluated persistent sensor bias and stale observations. We also added comparisons with ADG+RDT and recent Decision Transformer variants, along with a parameter-matched dynamics-only control to assess CDA’s contribution beyond predictive regularization. For Reviewer wrto, we clarified how CDA and ASA differ from prior work, explained how the ablations assess reconstruction and ASA, and added a comparison with a DSN-inspired Decision Transformer. We also clarified the ablation base model, checkpoint selection, and target-return settings. For Reviewer ANed, we repeated the comparison with RDT using two additional random seeds, reported per-task score gaps, observed ranges, and variability across all six seeds, and added a controlled state/noise test and an orthogonality-loss ablation to assess how the two embedding branches respond to state changes and corruption.

Once again, we would like to express our most sincere gratitude to all Reviewers, Program Chairs, Senior Area Chairs, and Area Chairs for the valuable evaluations and assistance regarding our submission. We sincerely hope our paper can receive objective and proper evaluations. Wishing you all the best!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.