- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper tests whether deployment-time action selection changes the apparent benefit of navigation pretraining. Re-evaluating recurrent PPO checkpoints under greedy and sampled actions reverses the matched-minus-scratch mean AUC contrast in the original maze setting. A structurally disjoint follow-up does not reproduce that reversal. Matched-departure controls further distinguish the probability of leaving the greedy action from the weighting of alternative actions.
Critical strengths
S1. The estimand is well chosen. Re-evaluating identical checkpoints and measuring the difference in transfer contrasts isolates sensitivity to the deployed action rule (Equation 2). Full-curve AUC and all eight seed blocks are reported, without equating a reversal of means with established superiority.
S2. The follow-up tests the initial result's limits. The authors audit task overlap, construct a split by unlabelled tree structure and report that the reversal does not recur. The matched-departure control usefully separates exploration frequency from the choice of non-greedy alternatives.
Critical weaknesses
W1. The result remains a setting-specific interaction. The original interaction is -0.1074, whereas the structural holdout gives +0.0388 with an interval spanning zero. Size, task weighting, inventory and seeds change together. Beyond showing that evaluation policy can matter, the paper identifies no property that explains or predicts when the pretraining comparison changes (Sections 4.2-4.4).
W2. The strongest diagnostic does not address the main interaction. The fixed-epsilon and matched-departure controls apply to the structural holdout, where the transfer interaction is already uncertain (Appendix C). They show that weighting alternatives improves absolute performance there, but do not explain the original reversal.
Assessment. I recommend weak rejection (2/4). The reporting is sound, but the connection between the main interaction and its proposed diagnostic remains too limited to provide a clear new explanation of transfer sensitivity.
Questions for authors
Q1 (W2). Are fixed-epsilon or matched-departure results available for the original 9-by-9 checkpoints? If so, please report the matched-minus-scratch interaction. If not, which existing result supports an explanation of the original reversal beyond generic benefits of stochastic evaluation?
Q2 (W1). Do the existing action-probability or trajectory records identify a difference between scratch and pretrained policies that tracks the interaction across seeds? Please give the specific statistic and its relationship to transfer, rather than another aggregate success comparison.
Response: original-setting controls and behavioral comparison
New and revised text appears in blue in the revised manuscript.
Q1. Do the action controls explain the original result? We repeated the controls on all 32 original agents at all 21 checkpoints. All greedy and sampled baseline success scores reproduced. With fixed ε=0.1, we draw uniformly among all three actions with probability 0.1 and otherwise take the greedy action. With matched departure, we keep the greedy action's learned probability and divide the remainder equally between the other two actions.
Maze-pretrained minus scratch AUC is +0.0533 under greedy selection, −0.0541 under sampling, −0.0174 under fixed ε and −0.0428 under matched departure. AUC measures average success across adaptation. Both controls reproduce the reversal of the mean ordering. All four individual contrast intervals include zero.
Relative to matched departure, learned sampling adds 0.0514 AUC for scratch and 0.0402 for maze pretraining. The difference between those gains is −0.0113 (descriptive 95% interval [−0.0379, 0.0136]). The original mean reversal therefore occurs even with equal weights on the alternatives. Whether the learned weights provide a different extra gain for the two training histories remains uncertain. The corresponding sampling-minus-fixed-ε interaction is −0.0366 [−0.0571, −0.0164]. Fixed ε also changes how often the policy departs from greedy, so that comparison cannot isolate the alternatives' weights. See Section 4.4 (pp. 7–8), Table 4 (p. 8) and Appendix Q (p. 33).
Q2. Which behavior tracks the sampling effect? For each of eight seed pairs, we measured the fraction of greedy steps spent revisiting a position and heading, averaged across adaptation. We compared the maze-pretrained minus scratch difference in this measure with the change in their success gap under sampling. Pearson r was −0.595. Where maze-pretrained agents repeated poses more relative to scratch, sampling tended to help scratch more relative to maze-pretrained agents. The corresponding correlations for collisions and coverage were −0.069 and +0.287. Appendix Q (p. 33) gives every pair.
Longer unsuccessful episodes allow more opportunities to revisit a pose. The same position and heading can also occur with different recurrent states. These eight observations do not establish that sampling breaks a cycle or predict where the interaction will occur.
Why do the settings differ? In the larger study, maze-pretrained agents had higher mean AUC under both rules. Size, action cap, task inventory, task weighting and training seeds all changed. The comparison tests whether the observed reversal recurs under those changes. It cannot identify which change accounts for the difference (Section 4.3, pp. 5–7, and Section 5, p. 9).