- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Direct , a test-time compensator for offline policies instantiated by a Decision Transformer with an auxiliary next-state prediction head. At deployment, the compensator estimates the delta action to offset prediction errors of state. Then a estimator will propagate state to next timestep. The learned dynamics model and policy remains frozen.
Critical strengths
- This method proposed a deployment-time correction mechanism that do not require online retraining. Using a correction action head to inject delta action into original action to offset the errors is a practical way.
- The ablation part is informative that the author controls different parts of Direct then tests these different variants. The results show that different components contribute together to the improvements.
Critical weaknesses
- The comparison experiments do not prove the advantage of proposed Direct . The paper only compares Direct with two baselines DT and affine on three environments. These results can not support the claim that Direct , with relatively complex design, really performs better than current baselines and alternatives.
- The distrubance evaluation is too restricted to support general deployment-time dynamics robustness. All experiments use action noise under accumulation error hypothesis. The discrepency of action is injected into the same channel which is used to correct it. There are no tests of changes to transition model. Also, the experiment results do not show robustness against unpredictable or rapid change to environment.
Questions for authors
- Can the author provide results with a learned residual controller or history-conditioned adaptive policy? Can this comparison be conducted under the main Table 1 configurations? Also, what advantage does the adaptive law provide over a simple proportional/PID style model residual controller?
- What if we change the embodiments, friction coefficient of environment in the experiments?The author may refer to DADP[1], Meta-DT[2] about the possible extensive experiments design. Also, what happens if we inject completely IID errors into actions? Can the proposed method still performs well under these settings? [1]Wang, P., Liu, Q., Lin, H., Li, Y., Zhan, G., Tomizuka, M., & Wang, Y. (2026). DADP: Domain Adaptive Diffusion Policy. arXiv preprint arXiv:2602.04037. [2]Wang, Z., Zhang, L., Wu, W., Zhu, Y., Zhao, D., & Chen, C. (2024). Meta-DT: Offline meta-RL as conditional sequence modeling with world model disentanglement. Advances in Neural Information Processing Systems, 37, 44845-44870.