- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CrossMaze, a benchmark for zero-shot structural generalization in offline continuous control. Layouts act as contexts in a goal-conditioned contextual MDP, with disjoint training and held-out layouts and no policy updates during evaluation. The benchmark includes two tracks for PointMaze and AntMaze: M16, a fixed diagnostic suite with 16 training and 10 held-out layouts per embodiment, and SeedMap, a deterministic procedural generator for scaling the number of training layouts. Eight behavior-cloning, offline RL and offline GCRL baselines are evaluated alongside PIDA, a behavior-cloning policy using Qwen3-0.6B with LoRA that receives the state, goal and layout as text and outputs continuous actions. On M16, PIDA achieves 69.8% and 72.7% held-out success on PointMaze and AntMaze, compared with 17.6% and 4.4% for the best non-PIDA baselines. Further experiments vary the number of training layouts and the width and depth of an MLP BC policy, and test the effects of physical observation descriptions and action descriptions in PIDA's prompt.
Critical strengths
-
A clear evaluation methodology. Training and held-out layouts are disjoint, each layout contributes equally to the reported success rate, and model selection uses only training-side data. The paper also specifies versioned maps, seeds, datasets and start–goal pairs to support reproducibility.
-
A strong reference policy. PIDA uses standard components and behavior cloning, yet achieves much higher held-out success than the evaluated baselines. On AntMaze, its 72.7% held-out success is close to its 76.7% average over the 12 custom training layouts. On PointMaze, it reaches 34.2% held-out success after training on a single layout.
Critical weaknesses
-
The authors do not explain the failures on AntMaze training layouts. In Table 2, all eight non-PIDA methods score 0.0% on the U-Maze and Medium training layouts, despite the dataset containing 5,000 trajectories per training layout. Their average success over the 12 custom training layouts is at most 18.0%. The paper describes hyperparameter tuning but does not explain these failures. Since the baselines already perform poorly on training layouts, their low held-out success does not clearly show how much additional difficulty comes from encountering new layouts.
-
The comparisons do not explain PIDA's advantage. The numerical baselines receive a flattened, padded map through an MLP, while PIDA receives a row-wise text map through a pretrained transformer. There is no comparison with a CNN map encoder or a transformer trained from scratch on the same text input. The paper acknowledges that the comparison cannot separate pretraining, architecture, representation and optimization. These experiments would help establish whether a different map encoder improves the baselines and how much pretraining contributes to PIDA's performance.
-
The evaluation focuses on high-quality trajectories. The reported experiments use successful, high-quality navigation trajectories collected by controllers. The paper does not examine whether the comparison between behavior cloning and offline RL changes when the data include failed or suboptimal trajectories. This limits the conclusions that can be drawn about the methods across different offline datasets. More varied offline data are listed as future work, leaving this question unanswered in the current evaluation.
Questions for authors
- Could you explain why all eight baselines score 0% on the AntMaze U-Maze and Medium training layouts in Table 2?
- Have you tested whether a CNN map encoder or a transformer trained from scratch on PIDA's text input reduces the gap to PIDA?
3 comments
Response to Reviewer eoz9 Part-2
W2: The comparisons do not explain PIDA’s advantage.
Q2: Have you tested whether a CNN map encoder or a transformer trained from scratch on PIDA's text input reduces the gap to PIDA? These experiments would help establish whether a different map encoder improves the baselines and how much pretraining contributes to PIDA’s performance.
A2: Thank you for suggesting these comparisons. We have added CNN-BC and Transformer-BC, along with diagnostics that supply explicit spatial features to MLP-BC and CNN-BC. These comparisons help distinguish the benefit of changing the map encoder from that of making spatial relations explicit, probing a potential source of PIDA's advantage.
CNN and Transformer map encoders. The CNN processes the two-dimensional maze through spatial convolutions. The Transformer treats grid cells as tokens, uses two-dimensional positional encodings, and masks padded cells. Both produce a 256-dimensional map feature and use the same downstream feature-fusion and policy-backbone architecture. Implementation details are in the blue-marked Appendix D.
Explicit spatial features. Sensing adds the agent's two-dimensional grid-cell index and four adjacent-wall indicators while retaining the original map and continuous state. These externally supplied features make spatial relations explicit; they are not learned by BC and contain no shortest path, subgoal, recommended direction, or expert action. These are feature-assisted diagnostics, not identical-input encoder comparisons.
Table 3 reports these results alongside MLP-BC and PIDA. Success rates (%) are mean ± standard deviation over five training seeds, with equal layout weighting.
| Policy / map encoder | Sensing | PointMaze train | PointMaze held-out | AntMaze train | AntMaze held-out |
|---|---|---|---|---|---|
| BC / MLP | No | 60.7 ± 0.8 | 12.2 ± 1.3 | 9.5 ± 1.8 | 1.8 ± 1.4 |
| BC / Transformer | No | 68.8 ± 1.2 | 19.8 ± 1.2 | 5.8 ± 2.5 | 2.2 ± 2.4 |
| BC / CNN | No | 73.8 ± 1.0 | 28.7 ± 1.7 | 6.7 ± 2.0 | 3.9 ± 1.3 |
| BC / MLP | Yes | 79.6 ± 2.0 | 41.7 ± 2.0 | 67.1 ± 11.4 | 49.4 ± 11.8 |
| BC / CNN | Yes | 82.2 ± 1.6 | 50.0 ± 1.5 | 75.4 ± 3.1 | 71.9 ± 2.3 |
| PIDA | No | 78.5 ± 5.8 | 69.8 ± 1.8 | 75.6 ± 2.4 | 72.7 ± 2.4 |
Changing the encoder helps, but does not close the gap on its own. On PointMaze, CNN-BC improves held-out success from 12.2% to 28.7%, while Transformer-BC reaches 19.8%. These improvements support the value of spatial inductive biases, but both remain below PIDA's 69.8%. On AntMaze, CNN-BC and Transformer-BC without sensing reach 3.9% and 2.2% held-out success, respectively.
Explicit spatial information produces substantially larger improvements. Adding sensing to MLP-BC raises held-out success to 41.7% on PointMaze and 49.4% on AntMaze, gains of 29.5 and 47.6 percentage points over MLP-BC without sensing. Combining sensing with the CNN further raises success to 50.0% and 71.9%, respectively. The latter approaches PIDA's 72.7% on AntMaze, while a gap remains on PointMaze. These results indicate that access to useful spatial information is a major factor among the tested BC modifications, with map encoding providing complementary benefits. They are consistent with the hypothesis that PIDA benefits from pretrained spatial-semantic priors. The blue-marked Section 5.4 and revised conclusion discuss these findings.
A Transformer trained from scratch on PIDA's text input. A controlled text-input comparison would further clarify the roles of architecture and pretraining. Transformer-BC is trained from scratch on grid-cell tokens, not PIDA's serialized text; the latter comparison is not included. For a from-scratch version of the full PIDA backbone, the training protocol also matters. PIDA freezes its 0.6B-parameter pretrained backbone and trains LoRA adapters and the action-decoding components. Training the full random backbone would change the trainable-parameter budget and may require different optimization and training budgets. Thus, comparing PIDA with a fully trained, randomly initialized backbone makes it difficult to vary initialization alone while keeping the training protocol fixed. The new experiments directly establish the benefit of structured map encoding and provide evidence on how much pretraining contributes. Even with a CNN or Transformer map encoder, numerical BC stays below 29% held-out success on PointMaze and below 4% on AntMaze. It approaches PIDA only when explicit spatial features are supplied from outside (71.9% vs. 72.7% on AntMaze), and a gap still remains on PointMaze (50.0% vs. 69.8%). PIDA receives none of these features and reaches 69.8% and 72.7% from the serialized map alone. This suggests that its pretrained backbone provides the kind of spatial knowledge that numerical BC has to be given from outside, and points to pretraining as a key factor in its advantage.
Response to Reviewer eoz9 Part-3
W3: The evaluation focuses on high-quality trajectories. The paper does not examine whether the comparison between behavior cloning and offline RL changes when the data include failed or suboptimal trajectories.
Q3: Thank you for raising this point. We agree that dataset quality can affect the relative performance of behavior cloning and offline RL, and that this is distinct from the layout-generalization axis evaluated here. The current experiments use successful, high-quality controller-generated trajectories, as specified in Section 3, and all methods use the same fixed dataset within each experimental condition. This provides a controlled starting point for studying whether policies trained on successful behavior can transfer that behavior to unseen structures. With high-quality data, low held-out success cannot be explained by poor demonstrations: even in this setting, which is the most favorable one for behavior cloning, numerical BC reaches only 12.2% on PointMaze and 1.8% on AntMaze held-out layouts.
In particular, successful demonstrations are well aligned with the behavior-cloning objective, whereas learning from mixed-quality behavior can make reward-based selection and improvement more important. We therefore do not interpret the present results as showing that behavior cloning generally outperforms offline RL across dataset qualities. The comparison between PIDA and numerical BC, and the encoder and sensing diagnostics within BC, remain informative about policy representations in the evaluated high-quality-data regime. These comparisons use the same behavior-cloning objective and the same data, so they do not depend on how BC ranks against offline RL.
CrossMaze includes an offline data generator in addition to the fixed datasets, allowing further datasets to be constructed, and the conclusion identifies more varied offline data as an extension. Our empirical conclusions are consequently specific to the reported data regime; whether failed or suboptimal trajectories change the relative ranking of BC and offline RL remains an open question for this benchmark.
Response to Reviewer eoz9 Part-1
We thank the reviewer for recognizing the clarity of the evaluation protocol and the strength of PIDA as a reference policy. To address your concerns, we have thoroughly reviewed each comment and provided comprehensive responses, supported by further clarifications and additional experimental results. The single-layout baseline checks are reported in the orange-marked Appendix G, Table 8 and Figure 6. If any issues remain, we would be more than glad to clarify them further.
W1: The authors do not explain the failures on AntMaze training layouts.
Q1: Could you explain why all eight baselines score 0% on the AntMaze U-Maze and Medium training layouts in Table 2?
A1: Thank you for highlighting the zero-success results on the two AntMaze training layouts. The revision now explains them. These zeros appear only when a single baseline policy is trained jointly on the M16 AntMaze layouts. M16 trains one shared layout-conditioned policy across multiple layouts, rather than a separate policy for each layout, so these entries indicate failures to learn particular training tasks under the joint-training setting.
The following additional and existing experiments support this interpretation.
Our implementations achieve strong single-layout results. In the additional single-layout checks, our BC and offline RL implementations (TD3+BC, IQL, and ReBRAC) achieve 76.1%–92.6% success on CrossMaze AntMaze U-Maze, compared with 0% under joint M16 training. For GCRL, our QRL, CRL, and HIQL implementations achieve 80.2%, 93.8%, and 94.2% on the official OGBench AntMaze Medium benchmark, respectively. These findings are consistent with prior evidence that offline RL and GCRL methods can learn single-layout AntMaze tasks [1, 2]. Full settings, results, and training curves are provided in the orange-marked Appendix G, Table 8 and Figure 6.
The new map-encoder and sensing experiments retain the BC training pipeline. CNN and Transformer map encoders improve PointMaze training performance, while CNN-BC with sensing reaches 75.4% training and 71.9% held-out success on AntMaze (Section 5.4, Table 3). These results show that the shared BC implementation can learn effective AntMaze policies when spatial information is made easier to use. Part 2 of this response discusses this experiment in more detail.
M16 and SeedMap use the same baseline implementations. Several baselines improve with broader layout coverage on SeedMap. For example, AntMaze CRL's held-out success increases from 3.3% with 100 training layouts to 43.3% with 500 layouts (Section 5.3 and Appendix E, Table 7). Thus, the same implementation can learn policies that transfer to unseen AntMaze layouts under broader structural coverage.
PointMaze and AntMaze share the algorithm implementations, with observation and action dimensions adapted to each embodiment. On PointMaze M16, BC achieves 60.7% training success, including 97.3% on U-Maze and 96.0% on Medium, although held-out success remains at 12.2% (Tables 1 and 3). This demonstrates learning on the training layouts while exposing a substantial transfer gap, rather than a general inability of the implementation to learn.
Single-layout results and implementation details. The table below reproduces the single-layout checks in Table 8 of the revised manuscript.
Values are cross-checkpoint trimmed mean success rates (%), excluding one highest and one lowest evaluation value per run. OGBench success averages the five official tasks equally.
We also agree with the reviewer’s reading of Table 2: on AntMaze, the low baseline scores mainly reflect the difficulty of learning 16 layouts jointly, so they do not show on their own how much difficulty new layouts add. The new Table 3 answers that question for policies that do learn the training layouts. For CNN-BC with sensing, held-out success is 3.5 points below training success on AntMaze, and 32.2 points below on PointMaze (82.2% vs. 50.0%). We have clarified in Section 5.2 and Appendix G, marked in orange, that the AntMaze M16 results include difficulty learning the training tasks themselves.
[1] Tarasov, D., Kurenkov, V., Nikulin, A., and Kolesnikov, S. Revisiting the Minimalist Approach to Offline Reinforcement Learning. NeurIPS, 2023.
[2] Park, S., Frans, K., Eysenbach, B., and Levine, S. OGBench: Benchmarking Offline Goal-Conditioned RL. ICLR, 2025.