acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Value Anchoring for Long-Horizon Stitching in Offline Reinforcement Learning

Abstract

An important capability of offline goal-conditioned reinforcement learning is stitching, which can improve policies by connecting suboptimal trajectory segments and enable agents to solve long-horizon tasks by combining shorter trajectories. Stitching commonly relies on temporal difference (TD) learning, whose repeated bootstrapping propagates reward signals between trajectories but can also lead to bias accumulation in value estimation. Although longer backups can mitigate bias accumulation by reducing bootstrapping depth, we show that they can hinder stitching. Specifically, longer backups can skip junction states where reward signals propagate between trajectories. Therefore, we propose Adaptive Value Anchoring (AVA), which adapts backup lengths to bootstrap from junction states. AVA can thereby mitigate bias accumulation while preserving reward propagation between trajectories. We evaluate AVA on long-horizon OGBench tasks that require stitching. AVA achieves the highest average success rate among the compared methods. Moreover, it achieves the highest mean success rate in every subsampled setting, where reduced trajectory overlap makes stitching harder.

Primary area:
reinforcement learning
Keywords:
Offline Reinforcement Learning, Goal-Conditioned Reinforcement Learning, Trajectory Stitching

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_TkBUOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how long TD backups can hinder trajectory stitching in offline goal-conditioned RL through value dilution and poorly placed backup endpoints. AVA scores potential junctions by the additional downstream coverage available through overlapping trajectories and uses them to choose backup endpoints. Evaluation covers OGBench locomotion and puzzle tasks, subsampled stitch datasets, and two state-based LIBERO scenes.

Critical strengths
  1. The distinction between backup depth and location is well motivated by the expectile calculation and gridworld example. Although the analysis is restricted to simplified settings, it leads to a straightforward anchoring rule that changes the value target by selecting where the backup ends.

  2. Under matched policy extraction, AVA achieves higher mean success than n-step VA on all seven datasets, raising the average from 59.8% to 68.5%. It also exceeds Random VA and Density-based AVA in all six stitch settings. The AWR and HILP variants provide further checks on the roles of policy extraction and manually selected coordinates.

Critical weaknesses

The external comparisons need clarification. OTA scores 16.4%/58.7% on AGS/AGN here, versus 37%/77% in Ahn et al. (NeurIPS 2025, Table 6), despite matching reported discounts and task-specific horizons. LAVL scores 35.7%/56.1% on AGS/HGS, versus 82%/80% in Kang et al. (ICML 2026, Table 1). The reported LAVL configurations differ: Table 8 lists different discounts and stitch goal-sampling settings from the original paper. The actual configurations need clarification before relying on the overall ranking. AVA also uses different policy extraction from HIQL, OTA and LAVL, so gains over these baselines do not isolate the effect of anchoring.

Table 4 shows substantial sensitivity to τ\tau and η\eta. On AGS, tuning τ\tau raises n-step VA from 24.3% to 54.8%, above default AVA's 48.7%. At τ=0.9\tau=0.9, the best AVA setting reaches 55.9 ± 11.9%, close to n-step VA's 54.8 ± 6.5%. AVA's best mean in this sweep is still higher, at 61.9%, but with η=0.3\eta=0.3 it drops from 61.9% to zero as τ\tau moves from 0.7 to 0.95. The size of the advantage after comparable joint tuning remains unclear.

Questions for authors
  1. Which discount, goal-sampling and policy-extraction settings were actually used for OTA and LAVL, and what accounts for the gaps from their published results?

  2. Using the existing sweeps, how much of AVA's advantage remains after comparable tuning of nn and τ\tau for n-step VA and of nn, τ\tau, and η\eta for AVA?

  3. Does AVA still outperform Random VA when the two methods have matched backup-length distributions?

  4. In puzzle tasks, do the gripper overlaps used for scoring also correspond to matching button configurations? How do mismatches affect the selected backup endpoints?

4 comments
AuthorsOpenReview ↗

Response to Reviewer TkBU (1/2)

Dear Reviewer TkBU,

We thank you for your careful review and constructive suggestions. Your questions helped us clarify the experimental settings and further evaluate the contribution of junction selection. We present the requested comparisons and clarifications below.

OTA/LAVL settings and performance recovery (Q1)

We used the OGBench default discount factors and goal-sampling settings for most compared methods and tasks. An exception was the discount factor for HG*, which we increased from 0.9950.995 to 0.9980.998 to reduce reward decay over long horizons. For policy extraction, we used reported settings or transferred settings from related tasks following Appendix B.5 of the main paper.

OTA. OTA's original discount factor and goal-sampling settings match our common settings, except for the discount factor on HG*.

The main gaps between our reproduced and originally reported results are observed on AG* and HG*. Despite using OTA's public code and original hyperparameters, we obtained lower success rates on AG* than originally reported for reasons we have not identified. For HG*, the discrepancy is primarily due to the different discount factor used in our reproduction. With the original discount factor of γ=0.995\gamma = 0.995, the reproduced success rates closely match the originally reported success rates.

Notably, AVA's best tested configurations achieve higher mean success rates than both the originally reported and reproduced OTA results across all four tasks.

Table I. OTA reproduction and performance recovery.

DatasetReproducedOriginally reportedAVA best
AGS16.4 ± 3.437 ± 661.9 ± 3.0
AGN58.7 ± 4.077 ± 492.0 ± 1.4
HGS79.9 ± 3.079 ± 382.0 ± 3.3
HGN91.3 ± 2.092 ± 194.7 ± 1.1

LAVL. The main differences from LAVL's original settings lie in the discount factors for locomotion tasks and the value goal-sampling distribution for stitch datasets. For example, our AGS evaluation used the OGBench defaults as follows: γ=0.995\gamma = 0.995 and (pcur,ptraj,pgeom,prand)=(0.2,0,0.5,0.3)(p_{\text{cur}}, p_{\text{traj}}, p_{\text{geom}}, p_{\text{rand}}) = (0.2, 0, 0.5, 0.3). However, LAVL's original settings on AGS were as follows: γ=0.999\gamma = 0.999 and (0.2,0.3,0,0.5)(0.2, 0.3, 0, 0.5).

Restoring LAVL's original settings increased mean success on AGS from 35.7%35.7\% to 79.5%79.5\%, close to the reported 82%82\% (Table J). The recovery indicates that much of the performance gap comes from differences between LAVL's original settings and the common settings used by most compared baselines.

However, LAVL's mean success rate under the original settings exceeds AVA's best tested mean success rate on AGS. Therefore, we will clarify that AVA's advantage over LAVL on AGS is limited to the common discount factor and value goal-sampling settings.

Table J. LAVL performance recovery on AGS.

γ\gammaValue goal dist.Success rate (%)
0.995OGBench default35.7 ± 5.5
0.999OGBench default69.0 ± 1.4
0.995LAVL original68.3 ± 2.4
0.999LAVL original79.5 ± 2.1
AuthorsOpenReview ↗

Response to Reviewer TkBU (2/2)

Comparison with tuned n-step VA (Q2)

We compare each method's best mean success rate within the existing sweeps. Figures 4(a) and 4(b) report the nn sweeps on AGS. Table 4 reports the τ\tau and η\eta sweeps at n=25n = 25 on AGS and HGS.

On AGS, AVA achieves (61.9±3.0)%(61.9 \pm 3.0)\% with n=25n = 25, τ=0.7\tau = 0.7, and η=0.3\eta = 0.3. The best mean success rate of n-step VA is (54.8±6.5)%(54.8 \pm 6.5)\% with n=25n = 25 and τ=0.9\tau = 0.9. On HGS, AVA achieves (82.0±3.3)%(82.0 \pm 3.3)\% with n=25n = 25, τ=0.7\tau = 0.7, and η=0.3\eta = 0.3. The best mean success rate of n-step VA is (69.5±4.6)%(69.5 \pm 4.6)\% with n=25n = 25 and τ=0.7\tau = 0.7.

Therefore, AVA retains advantages of 7.17.1 percentage points on AGS and 12.512.5 percentage points on HGS after selecting each method's best configuration from the existing sweeps.

Comparison with Matched VA (Q3)

To separate the contribution of junction selection from the effect of the backup-length distribution, we evaluated a variant of Random VA. The variant samples backup lengths from AVA's empirical backup-length distribution instead of a uniform distribution. We refer to the variant as Matched VA.

As shown in Table K, AVA achieves higher mean success rates than Matched VA across all six evaluated settings. Therefore, the results support the benefit of junction selection beyond changes in the backup-length distribution.

Table K. Comparison with backup-length-matched random junctions.

DatasetFractionMatched VAAVA
AGSfull27.5 ± 7.048.7 ± 5.8
AGS50%32.5 ± 5.638.1 ± 8.5
AGS25%10.7 ± 2.714.1 ± 1.3
HGSfull69.4 ± 4.575.8 ± 2.5
HGS50%63.2 ± 5.271.5 ± 3.1
HGS25%38.9 ± 6.948.5 ± 2.1

Effects of mismatched button configurations (Q4)

We use gripper proximity as a proxy for stitching feasibility. Gripper proximity does not guarantee matching button configurations but provides a broad criterion for retaining potentially useful junctions. Configuration mismatches can lead to incorrectly selected junctions that shorten backups and reduce the benefit of horizon reduction. However, AVA selects bootstrap endpoints within the same dataset trajectory, without trajectory concatenation or cross-trajectory bootstrapping. Moreover, requiring exact configuration agreement can restrict candidate overlaps, especially in puzzle tasks with many possible button configurations.

Thank you for the additional experiments and clarifications. The Matched VA comparison provides more direct evidence that junction selection contributes beyond changing the backup-length distribution. The comparisons within the existing sweeps also support AVA’s gains over n-step VA. Together with the clarification of baseline settings and within-trajectory bootstrapping, these results strengthen my assessment of the contribution. I will raise my rating to 4.

Thank you for your thoughtful reassessment and for raising your rating. We appreciate your recognition of the evidence for junction selection and AVA’s gains over n-step VA. We will incorporate the additional results and clarifications into the revised manuscript.

Best regards, The Authors

Reviewer_hFwMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper addresses trajectory stitching in offline goal-conditioned RL, observing that long TD backups can skip important junction states where value information transfers between trajectories. The authors propose Adaptive Value Anchoring (AVA), which identifies junctions based on downstream trajectory coverage and adaptively terminates backups at these states. Experiments on OGBench and LIBERO show improvements over fixed-horizon TD, with ablations supporting the benefit of junction-based anchoring.

Critical strengths

Interesting insight into the limitations of fixed-horizon TD learning. The observation that longer backups can skip important stitching junctions is intuitive yet insightful. The authors provide a clear explanation of dilution and misalignment, supported by a theoretical analysis and illustrative experiments.

Critical weaknesses

Heuristic junction scoring and limited generalizability. The proposed junction score seems somewhat ad hoc: it measures additional downstream state coverage, but it is unclear why this should reliably identify useful or feasible stitching points. Moreover, the method relies on hand-selected position coordinates for junction detection, and its sensitivity to feature representations and discretization raises concerns about generalizability and scalability. Although the HILP variant partially addresses this, the paper does not convincingly establish applicability to more general high-dimensional settings.

Unclear fairness of baseline comparisons. Some baseline results appear substantially weaker than those reported in their original publications. For example, LAVL reports 82% success on AntMaze-Giant-Stitch, compared with only 35.7% in this paper. The experimental settings also differ, including the discount factor and goal-sampling distribution. I would appreciate clarification of these discrepancies and comparisons with appropriately tuned baselines, as they make it difficult to assess the claimed empirical improvements.

Questions for authors

Junction scoring and feature dependence: What motivates the use of additional downstream state coverage as a measure of junction quality? How does the method ensure that nearby states are actually stitchable, especially when using only root or gripper positions? How well would this approach generalize without these manually selected features?

Baseline fairness: Could the authors clarify the substantial discrepancies between some baseline results and their original publications, particularly LAVL (35.7% versus 82% on AGS)? How much of this difference can be attributed to changes in hyperparameters, such as the discount factor? Would the conclusions hold with appropriately tuned baselines?

Isolating the contribution: How much of AVA's improvement comes from selecting meaningful junctions rather than simply changing the backup-length distribution? A comparison with randomized anchors matched to AVA's backup-length distribution would help clarify this.

Computational scalability: What is the actual preprocessing time and memory complexity of junction scoring, including downstream coverage calculations? The claimed constant-time hash lookup does not account for the cost of computing these scores. How does this scale with dataset size?

5 comments
AuthorsOpenReview ↗

Response to Reviewer hFwM (1/2)

Dear Reviewer hFwM,

We thank you for your careful review and constructive suggestions. Your comments helped us clarify the junction scoring method and strengthen the empirical evaluation. We provide additional experiments, baseline comparisons, and computational analyses below.

Stitching feasibility (W1/Q1)

We use proximity in selected state coordinates, such as root or gripper positions, as a proxy for stitching feasibility. Although proximity in the selected coordinates does not guarantee exact configuration agreement, it provides a broad criterion for retaining potentially useful junctions. Such configuration mismatches can shorten backups and reduce the benefit of horizon reduction, but AVA does not use trajectory concatenation or cross-trajectory bootstrapping, which might require exact configuration agreement. Moreover, requiring exact configuration agreement can restrict candidate overlaps, especially in puzzle tasks with many possible button configurations.

Motivation for junction scoring (W1/Q1)

We assess junction usefulness through the additional downstream coverage of overlapping trajectories. Stitching at a junction can provide access to downstream states that are not covered by the current trajectory. Because each state is a potential goal, broader downstream coverage can support a wider set of goal-reaching tasks. Therefore, the score serves as a proxy for the potential usefulness of a junction.

Usefulness of junction scoring (W1/Q1/Q3)

To evaluate the usefulness of junction scoring, we compared AVA with Matched VA. Matched VA randomly selects bootstrap endpoints while matching AVA's backup-length distribution. This comparison controls for the backup-length distribution and tests the contribution of junction selection. As shown in Table D, AVA achieves higher mean success rates than Matched VA across all six evaluated settings.

Table D. Comparison with backup-length-matched random junctions.

DatasetFractionMatched VAAVA
AGSfull27.5 ± 7.048.7 ± 5.8
AGS50%32.5 ± 5.638.1 ± 8.5
AGS25%10.7 ± 2.714.1 ± 1.3
HGSfull69.4 ± 4.575.8 ± 2.5
HGS50%63.2 ± 5.271.5 ± 3.1
HGS25%38.9 ± 6.948.5 ± 2.1

Moreover, Figure 7 of the main paper illustrates broader downstream coverage at AVA junctions than at junctions selected by Density-based AVA and Random VA. Table 2 of the main paper reports higher mean success rates for AVA than for both Density-based AVA and Random VA across all six evaluated settings. The results support the usefulness of additional downstream state coverage as a criterion for junction selection.

Junction scoring with visual observations (W1/Q1)

We evaluated AVA w/ HILP and n-step VA on visual-puzzle-4x4. AVA w/ HILP uses learned visual representations only for junction scoring, without manual coordinate selection. The policy and value networks of both methods were trained directly on visual observations. Both methods used the original HIQL policy extraction procedure with βh=βl=3\beta_h = \beta_l = 3. AVA w/ HILP achieves a higher mean success rate on the evaluated task.

Table E. Results on visual-puzzle-4x4.

MethodSuccess rate (%)
AVA w/ HILP44.53 ± 34.9
n-step VA0.27 ± 0.6

Both methods used a batch size of 256. We report success rates averaged over evaluations at 300K, 400K, and 500K training steps.

AuthorsOpenReview ↗

Response to Reviewer hFwM (2/2)

LAVL settings and performance recovery (W2/Q2)

For a fair comparison, we used common discount and value goal-sampling settings for most of the baselines, including LAVL. The main differences from LAVL's original settings lie in the discount factors for locomotion tasks and the value goal-sampling distribution for stitch datasets. For example, our AGS evaluation used the OGBench defaults as follows: γ=0.995\gamma = 0.995 and (pcur,ptraj,pgeom,prand)=(0.2,0,0.5,0.3)(p_{\text{cur}}, p_{\text{traj}}, p_{\text{geom}}, p_{\text{rand}}) = (0.2, 0, 0.5, 0.3). However, LAVL's original settings on AGS were as follows: γ=0.999\gamma = 0.999 and (0.2,0.3,0,0.5)(0.2, 0.3, 0, 0.5).

Restoring LAVL's original settings increased mean success on AGS from 35.7%35.7\% to 79.5%79.5\%, close to the reported 82%82\% (Table F). The recovery indicates that much of the performance gap comes from differences between LAVL's original settings and the common settings used by most compared baselines.

Table F. LAVL performance recovery on AGS.

γ\gammaValue goal dist.Success rate (%)
0.995OGBench default35.7 ± 5.5
0.999OGBench default69.0 ± 1.4
0.995LAVL original68.3 ± 2.4
0.999LAVL original79.5 ± 2.1

LAVL tuning under common settings (W2/Q2)

Under our common settings, we evaluated 16 LAVL policy-extraction configurations with one seed per configuration (Table G). None exceeded AVA's best mean success rate of (61.9±3.0)%(61.9 \pm 3.0)\% on AGS (Table 4 of the main paper).

However, LAVL's mean success rate under the original settings exceeds AVA's best tested mean success rate on AGS. Therefore, we will clarify that AVA's advantage over LAVL on AGS is limited to the common discount factor and value goal-sampling settings. Nevertheless, the matched comparisons with n-step VA and Matched VA continue to support the benefit of junction-aware backups under identical discount and goal-sampling settings.

Table G. LAVL policy-extraction hyperparameter tuning on AGS. Asterisks mark the values used in the original paper.

βl\beta_l / βh\beta_h3.0*1.00.30.1
3.046.038.526.513.2
1.0*43.646.321.111.7
0.346.341.330.814.5
0.145.638.028.712.7

Computational scalability (Q4)

We measured AVA's total preprocessing time and downstream coverage calculation time on the full and subsampled AGS datasets. As shown in Table H, total preprocessing time increases approximately linearly with the number of dataset states NN over the evaluated range. Downstream coverage calculation accounts for most of the total preprocessing time.

All timing measurements were performed on a desktop computer equipped with an Intel Core i9-13900 CPU and an NVIDIA GeForce RTX 4090 GPU.

Table H. Preprocessing time on AGS.

DatasetNNCoverage calculation (s)Total (s)
AGS1,005,00015.5316.39
AGS 50%502,5007.247.69
AGS 25%251,2503.523.77

We also analyzed the memory complexity required for preprocessing. The preprocessing stores per-state information and caches downstream coverage sets for each cell. Each state contributes at most LL entries to the cached coverage sets, where LL is the maximum trajectory length. Therefore, the memory bound is as follows: O(N+NL)=O(NL)O(N + NL) = O(NL). If LL is bounded by a constant independent of NN, the bound reduces to O(N)O(N).

Dear Reviewer,

With the author response deadline approaching, we wanted to ask whether you have any remaining questions or concerns. We’d be happy to clarify any points while the discussion is still open. If our responses have addressed your concerns, we would appreciate it if you could let us know whether you would consider raising your score.

Thank you again for your time and thoughtful feedback.

Best regards, The Authors

Reviewer_GNhCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies a trade-off in offline goal-conditioned RL: longer TD backups can reduce bootstrapping depth but may skip states needed for trajectory stitching. AVA scores potential junctions by the additional downstream coverage of overlapping trajectories and ends multi-step backups at selected junctions. Experiments on state-based OGBench tasks show gains over fixed-horizon backups, particularly when stitching is important, with additional evaluations on subsampled datasets and two adapted LIBERO scenes.

Critical strengths
  1. The paper identifies a concrete limitation of fixed-horizon backups and provides an intuitive explanation through dilution and misalignment of value propagation.

  2. AVA shows meaningful gains on several long-horizon tasks and achieves the highest mean success across the evaluated subsampled stitching settings.

  3. The experiments include matched policy-extraction ablations, alternative junction criteria, learned features, and hyperparameter sweeps, providing useful evidence beyond the main performance comparison.

Critical weaknesses
  1. The methodological novelty is relatively narrow, centering on a handcrafted junction-selection rule within an existing multi-step value-learning framework. Its proximity and downstream-coverage proxies are not directly validated against feasible stitching or task returns; the HILP variant reduces manual coordinate selection but retains this scoring heuristic.

  2. All experiments use state observations, including the adapted LIBERO tasks. It remains unclear whether the proximity- and coverage-based junction criterion transfers to visual observations, since the HILP variant is also evaluated only with state inputs.

Questions for authors
  1. Could the authors directly compare AVA with VAST (Zheng et al., 2026), which also addresses fixed-horizon limitations through adaptive value stitching? The Value-based AVA ablation does not implement VAST's return model and stitching policy, so its poor performance does not establish an advantage over VAST.
4 comments
AuthorsOpenReview ↗

Response to Reviewer GNhC

Dear Reviewer GNhC,

We thank you for your careful review and constructive suggestions. We address your concerns and present the additional results below.

Methodological novelty (W1)

Our analysis of dilution and misalignment motivates junction-aware backups. The main methodological novelty lies in adapting backup horizons according to junction scores. Most of the adaptive-horizon methods discussed in Section 2 of the main paper select horizons by maximizing learned value estimates, which can favor overestimated states. On the other hand, our analysis (Section 4 of the main paper) motivates using precomputed junction scores to avoid skipping potentially useful junction states. AVA achieves higher mean success rates than VAST, which selects horizons through value maximization, on six of the seven datasets (see Comparison with VAST, Q1).

Stitching feasibility (W1)

We use proximity in selected state coordinates, such as root or gripper positions, as a proxy for stitching feasibility. Although proximity in the selected coordinates does not guarantee exact configuration agreement, it provides a broad criterion for retaining potentially useful junctions. Such configuration mismatches can shorten backups and reduce the benefit of horizon reduction, but AVA does not use trajectory concatenation or cross-trajectory bootstrapping, which might require exact configuration agreement. Moreover, requiring exact configuration agreement can restrict candidate overlaps, especially in puzzle tasks with many possible button configurations.

Usefulness of junction scoring (W1)

To evaluate the usefulness of junction scoring, we compared AVA with Matched VA. Matched VA randomly selects bootstrap endpoints while matching AVA's backup-length distribution. This comparison controls for the backup-length distribution and tests the contribution of junction selection. As shown in Table A, AVA achieves higher mean success rates than Matched VA across all six evaluated settings.

Table A. Comparison with backup-length-matched random junctions.

DatasetFractionMatched VAAVA
AGSfull27.5 ± 7.048.7 ± 5.8
AGS50%32.5 ± 5.638.1 ± 8.5
AGS25%10.7 ± 2.714.1 ± 1.3
HGSfull69.4 ± 4.575.8 ± 2.5
HGS50%63.2 ± 5.271.5 ± 3.1
HGS25%38.9 ± 6.948.5 ± 2.1

Moreover, Figure 7 of the main paper illustrates broader downstream coverage at AVA junctions than at junctions selected by Density-based AVA and Random VA. Table 2 of the main paper reports higher mean success rates for AVA than for both Density-based AVA and Random VA across all six evaluated settings. The results support the usefulness of additional downstream state coverage as a criterion for junction selection.

Junction scoring with visual observations (W2)

We evaluated AVA w/ HILP and n-step VA on visual-puzzle-4x4. AVA w/ HILP uses learned visual representations only for junction scoring, without manual coordinate selection. The policy and value networks of both methods were trained directly on visual observations. Both methods used the original HIQL policy extraction procedure with βh=βl=3\beta_h = \beta_l = 3. AVA w/ HILP achieves a higher mean success rate on the evaluated task. Both methods used a batch size of 256. We report success rates averaged over evaluations at 300K, 400K, and 500K training steps.

Table B. Results on visual-puzzle-4x4.

MethodSuccess rate (%)
AVA w/ HILP44.53 ± 34.9
n-step VA0.27 ± 0.6

Comparison with VAST (Q1)

VAST uses a single-goal setting by default. Therefore, we adapted the official VAST implementation to multi-goal learning to match our experimental setup. We conditioned the return model, value functions, and high-level stitching policy on the goal. We also applied goal relabeling, as in other offline GCRL methods.

Table C. Comparison with VAST.

DatasetVASTAVA
AGS0.0 ± 0.048.7 ± 5.8
HGS0.3 ± 0.275.8 ± 2.5
AGN0.0 ± 0.090.3 ± 1.2
HGN0.1 ± 0.194.7 ± 1.1
P3381.0 ± 3.574.1 ± 5.4
P4446.8 ± 1.877.7 ± 2.8
P4517.3 ± 0.918.5 ± 0.8

As shown in Table C, AVA achieves higher mean success rates than VAST on six of the seven evaluated datasets. Notably, the results reported in the original VAST paper were obtained in a single-goal setting.

We used the reported hyperparameters for P45 and P44 and reused the P44 settings for P33 because no P33 settings were reported. For P44, we tested N=4N = 4 and N=16N = 16 and report the better result with N=16N = 16. For locomotion tasks, we adopted the AGN hyperparameters because AGN was the only evaluated locomotion task with reported settings. We additionally swept β\beta over {0.01,0.03,0.05,0.08,0.1}\{0.01, 0.03, 0.05, 0.08, 0.1\} for locomotion tasks and observed no substantial improvement.

Dear Reviewer,

With the author response deadline approaching, we wanted to ask whether you have any remaining questions or concerns. We’d be happy to clarify any points while the discussion is still open. If our responses have addressed your concerns, we would appreciate it if you could let us know whether you would consider raising your score.

Thank you again for your time and thoughtful feedback.

Best regards, The Authors

Thanks for the new experiments. They address my main concerns. I’m raising my rating to 3.

Thank you for your reassessment and for raising your rating. We appreciate your constructive feedback, which helped us strengthen the experimental evaluation. We will incorporate the additional results into the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Remarks

We thank the reviewers for their careful evaluation and constructive feedback.

Our theoretical analysis and illustrative experiments explain how fixed-horizon backups can hinder stitching through dilution and misalignment (GNhC, hFwM). The analysis motivates anchoring backups to potentially useful junction states in AVA (TkBU). Under matched policy-extraction settings, AVA achieves higher mean success rates than n-step VA on all seven datasets (TkBU). AVA also achieves the highest mean success rates among the compared methods across the evaluated subsampled stitch settings (GNhC). Ablations on alternative junction criteria, policy extraction, and learned representations, together with hyperparameter sweeps, support both our analysis of stitching failures under horizon reduction and the effectiveness of AVA (GNhC, TkBU).

In response to your feedback, we have conducted additional experiments to evaluate the contribution of junction selection and the applicability of AVA. We also clarify the baseline comparisons and analyze the computational cost. The key additions are summarized below.

  • Additional ablation to clarify the contribution of junction selection while controlling for the backup-length distribution (GNhC, hFwM, TkBU).

  • Clarification of the hyperparameters used for LAVL and OTA (hFwM, TkBU).

  • Evaluation on visual observations using learned representations without manual coordinate selection (GNhC, hFwM).

  • Additional comparison with the adaptive-horizon baseline VAST (GNhC).

  • Motivation and empirical evidence for the usefulness of junction scoring (Figure 7 and Table 2 of the main paper, GNhC, hFwM).

  • Rationale for using proximity in selected state coordinates as a proxy for stitching feasibility (GNhC, hFwM, TkBU).

  • Preprocessing time measurements and an analysis of memory complexity (hFwM).

We believe that our analysis and AVA offer a useful contribution to understanding and preserving stitching during horizon reduction. We will incorporate the additional results and clarifications into the final manuscript. We thank the reviewers once again for their constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.