- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
COMPACT learns goal-conditioned manipulation policies from offline images. It represents each object by appearance and pose, matches objects between the current and goal images, and selects one object at a time for action prediction. The policy is trained using implicit Q-learning. Experiments on four simulated environments show improvements over the tested baselines, including when the number of objects increases at test time.
Critical strengths
-
The approach is simple and addresses a useful problem: reusing object-level manipulation skills across different scene configurations. Matching state and goal objects before predicting actions provides a clear structure for the policy.
-
The generalization results are promising. When PPP-Cube increases from three to six objects, COMPACT retains 64.86% success compared with 9.24% for HECRL. The experiments cover four environments and report results across four seeds. The ablations also show that both object representations and the compositional policy contribute to performance.
Critical weaknesses
-
Theorem 3.1 does not appear to hold as stated. It claims that any object-selection policy has a multi-object value gap bounded by Kε, provided the single-object policy has error at most ε. However, even perfect single-object policies can give different discounted returns depending on which object is handled first. In Appendix A, Equations 16 and 17 use the same scheduling coefficients for the compositional policy and the optimal policy without justification. This removes the cost of choosing a poor object order from the bound. The theorem therefore does not establish the claimed guarantee for the proposed selection rule.
-
Important details of the claimed end-to-end training are missing. Object discovery uses hard pixel grouping, matching uses argmax, and object selection uses argmin. The paper does not explain how gradients pass through these decisions or how the selection threshold is learned. It also introduces environment-specific ordering weights for Stack-Cube and Scene without describing their implementation. These details matter because joint representation learning and object selection are central contributions, and the current ablations do not separate their effects.
Questions for authors
-
Why can Equations 16 and 17 use the same scheduling coefficients when the two policies may choose different object orders? Can the theorem be corrected to account for scheduling error?
-
How are the hard grouping, matching, and selection operations handled during training? Which parts receive gradients, and how are the threshold and environment-specific ordering weights learned?
-
How does the policy avoid repeatedly selecting an object that has already reached its goal? Algorithm 1 always selects the smallest pose distance but does not show a completed-object exclusion rule.
-
Can you compare the proposed selection rule with random selection and a learned selector using the same encoder? A frozen-encoder comparison would also help separate the benefit of joint training from the policy design.
-
Which success definition is used in the reported results? Appendix E defines success by a pixel-coverage threshold, while the main text and some environment descriptions use object-position thresholds. Please clarify the thresholds and whether evaluation uses 100 or 200 episodes for Stack-Cube.
5 comments
W1: Theorem 3.1 does not appear to hold as stated. It claims that any object-selection policy has a multi-object value gap bounded by Kε, provided the single-object policy has error at most ε. However, even perfect single-object policies can give different discounted returns depending on which object is handled first. In Appendix A, Equations 16 and 17 use the same scheduling coefficients for the compositional policy and the optimal policy without justification. This removes the cost of choosing a poor object order from the bound. The theorem therefore does not establish the claimed guarantee for the proposed selection rule.
We have now revised Theorem 3.1 and its proof in Appendix A to replace the original bound with
\leq \Delta_{\mathrm{sel}} +\frac{1-\gamma^{H+1}}{1-\gamma}\varepsilon_{\mathrm{ctrl}}.$$ Here, $\Delta_{\mathrm{sel}}=V^\star-V_\sigma^\star$ captures the loss from fixing the object-selection rule $\sigma$, while $\varepsilon_{\mathrm{ctrl}}$ uniformly bounds the per-step control Bellman error under that rule. The revised proof uses the finite-horizon version of the performance-difference identity [1], removing the assumption of shared scheduling coefficients. This conditional bound reinforces the theoretical rationale for COMPACT’s two complementary design choices: an explicit object-selection mechanism to address selection loss, and a shared single-object action head to address control error. **W2: Important details of the claimed end-to-end training are missing. Object discovery uses hard pixel grouping, matching uses argmax, and object selection uses argmin. The paper does not explain how gradients pass through these decisions or how the selection threshold is learned. It also introduces environment-specific ordering weights for Stack-Cube and Scene without describing their implementation. These details matter because joint representation learning and object selection are central contributions, and the current ablations do not separate their effects.** Building on our response to Q2a above, the completion threshold and ordering weights are learned jointly from the policy loss using straight-through estimators, with hard gating and selection in the forward pass and soft surrogates in the backward pass. Specifically, a ranking network scores each state–goal pair using its pair features and a pooled summary of all matched pairs, enabling configuration-dependent ordering. The ranking mechanism and completion-threshold rule are now specified in Algorithm 1 of the revised paper, with straight-through training described in the accompanying text. **Q4 (follow-up): Can you compare the proposed selection rule with random selection and a learned selector using the same encoder?** Following the Push-Tetris comparison above, we further evaluate selection strategies on Stack-Cube, an order-dependent task, using the same encoder architecture and training setup, and present the results below. | Selection rule | Coverage ↑ | Success ↑ | Chamfer ↓ | |---|---|---|---| | Random selection | 2.19 ± 0.22 | 0.00 ± 0.00 | 2.67 ± 0.39 | | Learned selector | 11.90 ± 2.88 | 24.11 ± 1.26 | 1.71 ± 0.50 | | COMPACT | **17.57 ± 2.89** | **51.56 ± 5.64** | **1.14 ± 0.43** | COMPACT outperforms on Stack-Cube, supporting the benefit of its structured selection rule on an order-dependent task. We will include the full ablation results on all environments in the final paper. [1] Kakade, Sham, and John Langford. "Approximately optimal approximate reinforcement learning." ICML 2002.Dear reviewer, as the time for the end of discussion is approaching, may we ask if you have managed to review our responses, and whether you have any remaining concerns? Thank you.
Thank you for your thorough and constructive review. We provide responses to the questions below:
Q1: Why can Equations 16 and 17 use the same scheduling coefficients when the two policies may choose different object orders? Can the theorem be corrected to account for scheduling error?
Yes, the theorem can be revised accordingly. We agree that Equations 16 and 17 can share scheduling coefficients only under a common fixed schedule. We are finalizing and ensuring correctness of the revised theorem to explicitly account for the object-selection gap, preserving its significant theoretical contribution of distinguishing compositional control error from scheduling loss, and will share the full updated theorem and proof within this discussion period.
Q2: How are the hard grouping, matching, and selection operations handled during training?
Object discovery uses soft grouping based on shortest-path distances, allowing gradients to flow through the object representations and selected-path edge costs, while matching and selection operations are not directly differentiated. The learnable threshold and ranking network receive gradients through straight-through surrogates, as detailed in our response to Q2a below. We will clarify the details of the soft-grouping formulation and its gradient flow in Section 3.1 of the revised paper.
Q2a: How are the threshold and environment-specific ordering weights learned?
The threshold and environment-specific ordering weights both are learned from the policy loss using a straight-through estimator, with hard selection in the forward pass and soft thresholding and ranking surrogates in the backward pass. We include the full details of this training mechanism and its gradient flow in the revised paper.
Q3: How does the policy avoid repeatedly selecting an object that has already reached its goal? Algorithm 1 always selects the smallest pose distance but does not show a completed-object exclusion rule.
The policy excludes object pairs whose pose distance does not exceed the learned completion threshold described in Section 3.2, and selects the closest remaining unfinished object. As clarified in Q2a, this threshold is learned from the policy loss using a straight-through estimator. We will clarify the completed-object exclusion rule in Line 13 of Algorithm 1.
Q4: Can you compare the proposed selection rule with random selection and a learned selector using the same encoder? A frozen-encoder comparison would also help separate the benefit of joint training from the policy design.
We run the suggested additional experiments on Push-Tetris, comparing random selection, a learned selector (via straight-through estimator), and the proposed COMPACT selection rule using the same encoder architecture and training setup, and present the results in the table below.
We confirm that COMPACT significantly outperforms both random selection and the learned selector, demonstrating the benefit of the proposed selection rule under the same encoder and compositional policy setup. Qualitative analysis shows that random selection significantly increases the number of steps required to reach the goal due to frequent object switching, while the learned selector suffers from many more object collisions and thus reduces the optimality of the manipulation policy. In contrast, COMPACT reaches the goal in the smoothest manner. We will include the full ablation results across all datasets and add qualitative visualizations in the final version of the paper.
Q5: Which success definition is used in the reported results? Appendix E defines success by a pixel-coverage threshold, while the main text and some environment descriptions use object-position thresholds. Please clarify the thresholds and whether evaluation uses 100 or 200 episodes for Stack-Cube.
Success is defined using a pixel-coverage threshold for Push-Tetris; all other environments use object-position thresholds. Stack-Cube is evaluated over 200 episodes, while the remaining environments use 100 episodes. We will clarify these environment-specific success criteria and evaluation episode counts in the main text and relevant appendix sections.