- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies offline goal-conditioned reinforcement learning. For distant goals, discounting shrinks the value differences between neighboring states until they fall below the network’s error, so the agent can’t rank subgoals. Temporal abstraction, which treats k steps as one transition, restores this signal, but a fixed k blurs nearby states when it is large and loses distant ones when it is small. The authors propose GITA, which conditions one value function on k and trains a single policy by weighting advantages across several k values, so the most informative scale dominates for each state-goal pair. On OGBench, GITA outperforms prior methods, including the fixed-k method OTA, especially on long-horizon tasks.
Critical strengths
The paper clearly shows why one fixed step size can’t work for both near and far goals, and its easy fix gives the biggest gains on the hardest long-distance tasks.
Critical weaknesses
the method doesn’t need you to pick k anymore, but it still tunes γ for every task, and the results swing a lot depending on that choice. In AntMaze, the wrong γ drops success to almost zero. So it feels like the tuning just moved from k to γ instead of really going away.
Questions for authors
If I understand the analysis correctly, in your idealized model jumping k steps gives the same values as just using a discount of γ to the power 1/k. So the theory doesn’t really separate GITA from conditioning on the discount instead. What do you think makes k-step jumps work better in practice? Since the policy is a single Gaussian trained with weights from several scales, I wonder what happens when small and large k prefer different subgoals. Could the policy end up averaging them into a subgoal in between that neither scale actually wants, for example one inside a wall?
Thanks the authors for the detailed rebuttal. Given the added experiment results and other reviews, I believe the current ratings reflect my evaluation of this paper.