- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GC-FACT, a training-free method for improving compositional generalization in offline goal-conditioned RL. Instead of planning through intermediate states from the dataset, GC-FACT decomposes a goal into individual factors and asks a frozen policy to complete them one at a time. The authors also study how the choice of factorization, called the basis, affects the feasibility of these subgoals.
Critical strengths
- The paper is well presented, with clear and intuitive figures and tables.
- Test-time goal decomposition is an important research direction, and training-free methods could be particularly useful in practice.
Critical weaknesses
-
The proposed method relies too heavily on human-designed priors. The most important parts of the method, Factorization and Basis Selection, are manually specified rather than learned. For some tasks, the authors even use known environment dynamics to construct an exact solution. This makes me question how well GC-FACT would generalize to new tasks. More importantly, if goal decomposition still requires substantial human knowledge and task-specific design, I am not sure how useful this framework is in practice. Given how capable frontier LLMs have become, I believe that simply describing the task to an LLM could already produce reasonable goal decompositions at different levels of granularity, even for tasks requiring complex reasoning. I would like to see what GC-FACT offers beyond such straightforward alternatives.
-
Most of the reported improvements come from the Lights Out puzzles, where the authors use a known transition matrix and an exact algebraic solver. The improvements on the other manipulation tasks are much smaller, and some tasks remain almost unsolved. It is therefore difficult to tell how much of the improvement comes from GC-FACT itself, rather than the additional task-specific planning knowledge.
-
The experiments are limited to state-based simulation tasks. All evaluations are conducted on OGBench, without visual observations or real-world robot experiments. These tasks have relatively simple and explicitly defined state structures, which makes manual factorization much easier. I would like to see whether the proposed method can work on more realistic goal-conditioned manipulation tasks, where the factors and their relationships are not explicitly provided.
Questions for authors
Please see Weaknesses
9 comments
Response to Reviewer VQ6a (2/2): where the gains come from, and pixels
W2. Most of the improvement comes from Lights Out with a known transition matrix. You are right that the puzzles dominate the average, and the revised paper reports both groups separately (success, %):
| average success | GCIQL | HIQL | OTA |
|---|---|---|---|
| cube tasks and scene: frozen to GC-FACT | 33.0 to 37.3 | 11.5 to 17.5 | 7.6 to 13.9 |
| Lights Out: frozen to GC-FACT | 36.1 to 90.1 | 6.6 to 79.4 | 1.8 to 55.9 |
Three observations separate the contribution of GC-FACT from that of extra knowledge. First, the cube tasks and scene use no transition model, and scene alone gains 4.7, 19.6 and 26.8 points. Second, on the puzzles the toggle matrix is read off the data, and GC-FACT with that matrix matches the built-in one. Third, knowledge alone does not help, because with the same factor slices per-cell targets fall below the frozen backbone on every grid and a greedy choice among exact single presses reaches at most 26.1%, whereas the F2 solve reaches 7.2 to 100.0%. The cubes stay hard for a measurable reason: with four cubes present the frozen policies finish a single-cube target in only 11 to 28% of episodes, so a chain of four targets rarely succeeds, and raising this local competence by relabelling one-object changes in the offline data as goals is the step we plan next.
W3. State observations only. We agree that this was the main limitation, and we now run GC-FACT on the pixel-based OGBench tasks with the same frozen checkpoints and no simulator access. The units are read off image data: cubes are colour clusters that rarely move between consecutive frames, and buttons are blobs that alternate between two colours, which also yields the toggle matrix. A cube target erases one cube from the current frame and pastes it at its goal position, and a press target repaints the buttons that one press toggles. An offline check against targets rendered by the simulator shows that, on visual-cube-double, the synthesised images are closer to the true single-factor targets than the current frame is, and the reader recovers every button state and the toggle matrix on the 3x3 and 4x4 grids. Results over four seeds (success, %):
| pixel tasks | frozen | TTGS | GC-FACT |
|---|---|---|---|
| GCIQL, average of 9 tasks | 9.7 | 5.6 | 34.4 |
| HIQL, average of 9 tasks | 47.0 | 22.6 | 62.2 |
| OTA, average of 9 tasks | 50.4 | 21.3 | 59.3 |
| visual-puzzle-4x5, GCIQL / HIQL / OTA | 4.8 / 16.4 / 18.4 | 0.4 / 0.4 / 1.2 | 82.8 / 84.4 / 87.6 |
| visual-puzzle-4x6, GCIQL / HIQL | 2.4 / 15.6 | 0.0 / 0.0 | 80.8 / 76.4 |
The press basis therefore carries over from states to pixels, and TTGS again lowers nearly every pair. On the cube tasks the picture is mixed: pasted targets help OTA on visual-cube-double (58.0 to 70.0) but leave most other pairs close to or below the frozen backbone. We attribute this to the painted image, which keeps the shadows and occlusions of the current frame around a cube copied from the goal frame, a combination the pixel policy never met as a goal. Our planned remedy follows from this cause: replace copy and paste by an object-centric encoder (slot attention or DLPv2, as used by HECRL) that swaps one object slot for its goal slot and decodes a consistent image. On visual-puzzle-4x6 the frozen OTA policies fail every goal, and no planner can compose presses that the policy cannot execute, which is the local-competence bound of our failure proposition. The reader works because OGBench objects are colour-coded, and scenes without such cues will need the same object-centric encoder.
Real robots. GC-FACT changes only the goal input of a frozen policy and learns no parameters, so it adds no training risk on hardware, and its success is bounded by the local competence of the policy, which the paper names and measures.
We hope these results address your concerns and would be grateful if you would take them into account in your assessment.
References for this response.
- deepseek-chat: DeepSeek-AI, DeepSeek-V3 technical report, arXiv:2412.19437 (API model queried on 10 November 2026).
- OGBench: Park et al., OGBench: Benchmarking offline goal-conditioned RL, ICLR 2025.
- TTGS: Opryshko et al., Test-time graph search for goal-conditioned RL, ICML 2026.
- HECRL: Haramati et al., Hierarchical entity-centric RL with factored subgoal diffusion, ICLR 2026.
- Slot attention: Locatello et al., Object-centric learning with slot attention, NeurIPS 2020.
- DLPv2: Daniel and Tamar, DDLP: Unsupervised object-centric video prediction with deep dynamic latent particles, TMLR 2024.
- Backbones: GCIQL (Kostrikov et al., ICLR 2022, goal-conditioned variant of OGBench), HIQL (Park et al., NeurIPS 2023), OTA (Ahn et al., NeurIPS 2025).
Changes (revised PDF, marked in blue)
| Point | Location |
|---|---|
| W1 human priors | §6.3, Tab. 16, App. F.4 (factor slices and toggle matrix read off the offline data; no single-cell flip occurs in the data); new App. F.8, Tab. 24 (decompositions proposed by deepseek-chat, checked without rollout); AI use statement |
| W2 gains from Lights Out | abstract; last two rows of Tab. 2; §6.1; Tab. 3 (value-greedy row), Tab. 15; Tab. 16 (A read from data); §6.4, Tab. 19 (local competence on cube-quadruple) |
| W3 state-based only | §5.3 (paragraph after Prop. 5.8); §6.3 (units read off images); App. E, paragraph "Pixel observations"; App. F.7 with Tab. 22 and Figs. 13–14; §7 (next steps) |
Response to Reviewer VQ6a (1/2): human priors and the LLM alternative
Thank you for the review. You raise three concerns: reliance on human priors, the dependence of the gains on Lights Out with a known matrix, and the restriction to state-based simulation. We address the first here and the other two in (2/2); all changes are marked in blue in the revised PDF.
W1. The factorisation and the basis are specified by hand.
The structure can be read off the offline data. Play data move one unit at a time, so consecutive states reveal the units. On all four Lights Out grids, the distinct flip patterns between consecutive states are exactly the columns of the toggle matrix, each column occurs from at least 510 distinct colourings, and the moving button identifies the pressed cell. With this matrix GC-FACT stays within 1.2 points of the built-in one on five of six backbone-grid pairs and within the seed spread on the sixth. On the cube tasks, grouping observation dimensions by the transitions in which they change recovers every cube block.
A plausible decomposition is not enough. "Fix one light at a time" is the decomposition a person or a language model proposes first, and a frozen policy cannot execute it. No press flips a single light, and a single-light target is unreachable or costs more presses than the whole goal on 41 to 100% of the reachable configurations. With the same data, policy and schedule, this decomposition falls below the frozen backbone on all four grids. The data reject it before any rollout, because a single-light flip never occurs in them.
We ran your suggestion. We described each task to deepseek-chat (task, success test, frozen goal-conditioned policy) and asked for a Python function that returns the next intermediate goal; five samples per prompt. We then classified every returned function on the exact Lights Out model, without rollout:
Without the press rule, every sample proposes the per-cell basis, and following it spends 4.6 to 10.0 times the optimal number of presses. With the rule, the basis becomes actionable, but only 8 of the 17 valid samples also plan exactly; the other 9 choose presses by a local rule and stop on most residuals, as monotone descent does. On the cube tasks no sample proposes a one-object target: every target moves one or several cubes only part of the way toward the goal.
What GC-FACT offers beyond such alternatives is therefore the test, not only the decomposition. Its checks (data support, factor cost ratio, an efficient planner) classify each of these proposals before any rollout and need no knowledge of where a proposal comes from. A human, a language model or a learned model can supply candidate decompositions, and GC-FACT decides which one a frozen policy can execute and how to plan in it. Coupling the language model with these checks, so that a rejected proposal is returned with the failed check as feedback, is a direct extension we intend to pursue.