- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how long TD backups can hinder trajectory stitching in offline goal-conditioned RL through value dilution and poorly placed backup endpoints. AVA scores potential junctions by the additional downstream coverage available through overlapping trajectories and uses them to choose backup endpoints. Evaluation covers OGBench locomotion and puzzle tasks, subsampled stitch datasets, and two state-based LIBERO scenes.
Critical strengths
-
The distinction between backup depth and location is well motivated by the expectile calculation and gridworld example. Although the analysis is restricted to simplified settings, it leads to a straightforward anchoring rule that changes the value target by selecting where the backup ends.
-
Under matched policy extraction, AVA achieves higher mean success than n-step VA on all seven datasets, raising the average from 59.8% to 68.5%. It also exceeds Random VA and Density-based AVA in all six stitch settings. The AWR and HILP variants provide further checks on the roles of policy extraction and manually selected coordinates.
Critical weaknesses
The external comparisons need clarification. OTA scores 16.4%/58.7% on AGS/AGN here, versus 37%/77% in Ahn et al. (NeurIPS 2025, Table 6), despite matching reported discounts and task-specific horizons. LAVL scores 35.7%/56.1% on AGS/HGS, versus 82%/80% in Kang et al. (ICML 2026, Table 1). The reported LAVL configurations differ: Table 8 lists different discounts and stitch goal-sampling settings from the original paper. The actual configurations need clarification before relying on the overall ranking. AVA also uses different policy extraction from HIQL, OTA and LAVL, so gains over these baselines do not isolate the effect of anchoring.
Table 4 shows substantial sensitivity to and . On AGS, tuning raises n-step VA from 24.3% to 54.8%, above default AVA's 48.7%. At , the best AVA setting reaches 55.9 ± 11.9%, close to n-step VA's 54.8 ± 6.5%. AVA's best mean in this sweep is still higher, at 61.9%, but with it drops from 61.9% to zero as moves from 0.7 to 0.95. The size of the advantage after comparable joint tuning remains unclear.
Questions for authors
-
Which discount, goal-sampling and policy-extraction settings were actually used for OTA and LAVL, and what accounts for the gaps from their published results?
-
Using the existing sweeps, how much of AVA's advantage remains after comparable tuning of and for n-step VA and of , , and for AVA?
-
Does AVA still outperform Random VA when the two methods have matched backup-length distributions?
-
In puzzle tasks, do the gripper overlaps used for scoring also correspond to matching button configurations? How do mismatches affect the selected backup endpoints?
4 comments
Response to Reviewer TkBU (2/2)
Comparison with tuned n-step VA (Q2)
We compare each method's best mean success rate within the existing sweeps. Figures 4(a) and 4(b) report the sweeps on AGS. Table 4 reports the and sweeps at on AGS and HGS.
On AGS, AVA achieves with , , and . The best mean success rate of n-step VA is with and . On HGS, AVA achieves with , , and . The best mean success rate of n-step VA is with and .
Therefore, AVA retains advantages of percentage points on AGS and percentage points on HGS after selecting each method's best configuration from the existing sweeps.
Comparison with Matched VA (Q3)
To separate the contribution of junction selection from the effect of the backup-length distribution, we evaluated a variant of Random VA. The variant samples backup lengths from AVA's empirical backup-length distribution instead of a uniform distribution. We refer to the variant as Matched VA.
As shown in Table K, AVA achieves higher mean success rates than Matched VA across all six evaluated settings. Therefore, the results support the benefit of junction selection beyond changes in the backup-length distribution.
Table K. Comparison with backup-length-matched random junctions.
| Dataset | Fraction | Matched VA | AVA |
|---|---|---|---|
| AGS | full | 27.5 ± 7.0 | 48.7 ± 5.8 |
| AGS | 50% | 32.5 ± 5.6 | 38.1 ± 8.5 |
| AGS | 25% | 10.7 ± 2.7 | 14.1 ± 1.3 |
| HGS | full | 69.4 ± 4.5 | 75.8 ± 2.5 |
| HGS | 50% | 63.2 ± 5.2 | 71.5 ± 3.1 |
| HGS | 25% | 38.9 ± 6.9 | 48.5 ± 2.1 |
Effects of mismatched button configurations (Q4)
We use gripper proximity as a proxy for stitching feasibility. Gripper proximity does not guarantee matching button configurations but provides a broad criterion for retaining potentially useful junctions. Configuration mismatches can lead to incorrectly selected junctions that shorten backups and reduce the benefit of horizon reduction. However, AVA selects bootstrap endpoints within the same dataset trajectory, without trajectory concatenation or cross-trajectory bootstrapping. Moreover, requiring exact configuration agreement can restrict candidate overlaps, especially in puzzle tasks with many possible button configurations.
Thank you for the additional experiments and clarifications. The Matched VA comparison provides more direct evidence that junction selection contributes beyond changing the backup-length distribution. The comparisons within the existing sweeps also support AVA’s gains over n-step VA. Together with the clarification of baseline settings and within-trajectory bootstrapping, these results strengthen my assessment of the contribution. I will raise my rating to 4.
Thank you for your thoughtful reassessment and for raising your rating. We appreciate your recognition of the evidence for junction selection and AVA’s gains over n-step VA. We will incorporate the additional results and clarifications into the revised manuscript.
Best regards, The Authors
Response to Reviewer TkBU (1/2)
Dear Reviewer TkBU,
We thank you for your careful review and constructive suggestions. Your questions helped us clarify the experimental settings and further evaluate the contribution of junction selection. We present the requested comparisons and clarifications below.
OTA/LAVL settings and performance recovery (Q1)
We used the OGBench default discount factors and goal-sampling settings for most compared methods and tasks. An exception was the discount factor for
HG*, which we increased from 0.995 to 0.998 to reduce reward decay over long horizons. For policy extraction, we used reported settings or transferred settings from related tasks following Appendix B.5 of the main paper.OTA. OTA's original discount factor and goal-sampling settings match our common settings, except for the discount factor on
HG*.The main gaps between our reproduced and originally reported results are observed on
AG*andHG*. Despite using OTA's public code and original hyperparameters, we obtained lower success rates onAG*than originally reported for reasons we have not identified. ForHG*, the discrepancy is primarily due to the different discount factor used in our reproduction. With the original discount factor of γ=0.995, the reproduced success rates closely match the originally reported success rates.Notably, AVA's best tested configurations achieve higher mean success rates than both the originally reported and reproduced OTA results across all four tasks.
Table I. OTA reproduction and performance recovery.
LAVL. The main differences from LAVL's original settings lie in the discount factors for locomotion tasks and the value goal-sampling distribution for stitch datasets. For example, our
AGSevaluation used the OGBench defaults as follows: γ=0.995 and (pcur,ptraj,pgeom,prand)=(0.2,0,0.5,0.3). However, LAVL's original settings onAGSwere as follows: γ=0.999 and (0.2,0.3,0,0.5).Restoring LAVL's original settings increased mean success on
AGSfrom 35.7% to 79.5%, close to the reported 82% (Table J). The recovery indicates that much of the performance gap comes from differences between LAVL's original settings and the common settings used by most compared baselines.However, LAVL's mean success rate under the original settings exceeds AVA's best tested mean success rate on
AGS. Therefore, we will clarify that AVA's advantage over LAVL onAGSis limited to the common discount factor and value goal-sampling settings.Table J. LAVL performance recovery on AGS.