- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper tackles the problem of symmetry-breaking in equivariant reinforcement learning through graph-conditioned policy decomposition. They use a graph-structured partial equivariance which estimates the reliability from the edge of two entities, then condition it to relax the equivariance. The experiments show improved results over the baselines.
Critical strengths
- The paper addresses an important problem in equivariant RL, and proposes reasonable and straightforward solution.
- The motivation is reasonable. The symmetry-breaking is highly related to the relations between subsystems, and that’s where the method comes from.
Critical weaknesses
- The proposed approach addresses the problem through state-conditioned pairwise entity relations. However, MDP symmetry is defined through transformation over state-action space, which may not involve entity permutations. For example, if the transformation leaves entity indices unchanged but symmetry-breaking occurs locally (e.g., rotation of the world frame for locomotion task which does not permute the joints but can be broken with a local slope or obstacles), there is no candidate relation to score, and the breaking cannot be relaxed by the graph. Although the proposed approach is well motivated for permutation-based structured control, it addresses a narrower symmetry-breaking problem than the broader MDP symmetry-breaking.
- GSPE introduces additional complexity. The authors did not report wall-clock time or per step computation time to measure this trade-off. Edge-wise computation increases quadratic w.r.t. entity quantity, but the experiments are done at most six entities, not measuring the complexity beyond it.
Questions for authors
- Can the current framework be extended to Euclidean task space symmetries? If so, how would candidate relations be defined in that setting?
- Could the authors report training wall-clock time and inference latency compared with baselines?
- The explanation of PE-SAC seems to be wrong. Did the authors use separate gating mechanism as written in related works? The Appendix D.2 implementation says the authors only used state-dependent gating which is different from the original approach. This raises concerns about whether PE-SAC was fairly implemented.
6 comments
Response to Question #1: Extension to Euclidean Task-Space Symmetries (Part 2)
3. Ant: rotations relative to fixed local terrain.
To address the reviewer's locomotion example, we introduce a direction-conditioned ridge-traversal task with command-aligned velocity reward , terrain-relative health checks, and terrain-augmented observations.
The candidates are horizontal rotations. We rotate the robot state and direction command relative to fixed terrain, recomputing local terrain features. Joint identities and joint-torque coordinates remain unchanged, so the action transformation is the identity, . Unlike the original Ant action, which permutes leg and joint-action blocks, the new spatial transformations preserve joint identities and body-frame torque coordinates. Keeping the terrain fixed allows local geometry to invalidate otherwise useful rotational correspondences.
In representative flat-region states, spatial confidences are near . In ridge-contact samples, the and confidences decrease to and , respectively, while confidence remains high.
We also compare adaptive aggregation with GCPD without spatial sharing and fixed aggregation:
| Method | Final return, mean SEM | Ridge-navigation success |
|---|---|---|
| GCPD without spatial sharing | 66.7% (40/60) | |
| GCPD + fixed | 81.7% (49/60) | |
| GCPD + adaptive | 96.7% (58/60) |
Success denotes reaching the opposite side of the ridge.
Adaptive improves mean final return by 3507 and 157 points over no spatial sharing and fixed , respectively, with 96.7% (58/60) ridge-navigation success. These results demonstrate both the benefit of extending symmetry-based sharing to spatial transformations and the additional benefit of adapting that sharing to local symmetry reliability.
We have distinguished the spatial-view extension from the original entity-relation implementation in Appendices A.5 and B.8, documented the transformation definitions in Appendix B.8, and added the evaluation protocol and available spatial results in Appendices D.8 and E.14. We have also added a supplementary video of the new Ant ridge-navigation task, showing the rendered terrain and the learned symmetry-relation scores.
Response to Question #2: Training Time and Inference Latency
We thank the reviewer for requesting this comparison. To address this concern, we conducted additional experiments comparing GPU inference latency, training time, and sample efficiency against five baselines across four benchmark environments. Our results demonstrate a clear computation–performance trade-off: GCPD achieves 0.439 ms complete GPU inference, while delivering a 15.24% mean final-return improvement over PEnGUiN with only 6.6% additional full-budget training time. We quantify the deployment and training costs below.
1. Deployment latency.
| Method | GPU inference latency (ms) |
|---|---|
| SAC | 0.290 |
| Equi-SAC | 0.931 |
| ApproxEqui-SAC | 0.983 |
| PE-SAC | 1.331 |
| PEnGUiN | 2.718 |
| GCPD | 0.439 |
GCPD achieves 0.439 ms end-to-end GPU inference, including symmetry-graph prediction and action generation. Its implementation uses a deterministic-action path and precomputed static indices and constants. Deployment requires only the graph predictor and actor; auxiliary predictors, consistency-target construction, and critics are training-only components.
2. Training time and performance.
Estimated full-budget wall-clock training times are for a single NVIDIA RTX 4090, averaged across the four benchmark environments. Each baseline's final evaluation return serves as the reference target.
| Method | Mean full training time (h) | GCPD steps to target (% of full budget) | Estimated time ratio¹ | GCPD mean final-return improvement |
|---|---|---|---|---|
| SAC | 1.85 | 36.50% | 3.07× | 43.82% |
| Equi-SAC | 4.68 | 58.50% | 1.95× | 39.04% |
| ApproxEqui-SAC | 5.08 | 57.60% | 1.76× | 41.77% |
| PE-SAC | 9.00 | 48.90% | 0.85× | 60.28% |
| PEnGUiN | 14.59 | 69.50% | 0.74× | 15.24% |
| GCPD | 15.56 | 100% | 1.00× | 0% (reference) |
¹ Estimated GCPD time to the baseline's final-return target divided by the baseline's full-budget training time:
The additional computation is accompanied by mean final-return improvements of 43.82%, 39.04%, and 41.77% over SAC, Equi-SAC, and ApproxEqui-SAC, respectively. GCPD also reaches the PE-SAC and PEnGUiN reference returns in an estimated and of their respective full-budget training times.
Compared with PEnGUiN, GCPD's estimated full-training overhead is 6.6% (), alongside a 15.24% mean final-return improvement. Furthermore, GCPD reaches the PE-SAC and PEnGUiN reference returns in an estimated 0.85× and 0.74× of their respective full-budget training times.
We have added the runtime measurements and timing protocol to Appendices E.13 and D.7. Appendix B.7 specifies the computational scaling and the candidate-relation counts in the evaluated domains.
Response to Question #1: Extension to Euclidean Task-Space Symmetries (Part 1)
We thank the reviewer for highlighting this distinction and providing a concrete example. Yes: we have implemented an extension that retains GCPD's entity-relation graph and adds reliability-weighted sharing across Euclidean task-space symmetries. These additional relations remain available when transformations leave entity indices unchanged. Our new experiments cover finite planar reflection and rotation groups.
1. Defining spatial candidate relations.
For a chosen finite transformation group K, the additional candidates are
\mathcal E_{\mathrm{spatial}} = \left\\{ \bigl((i,e),(i,g)\bigr) : i\in V,\\; g\in\mathcal K\setminus\\{e\\} \right\\},
where V is the entity set, e is the identity, and (i,g) represents entity i in the transformed observation. Each relation receives a confidence controlling its contribution to view aggregation. Action predictions are mapped back through the inverse action transformation before aggregation.
Thus, the scored correspondence is between spatial views of the same entity and does not require an entity permutation. This extends GCPD to a broader range of symmetry types and task settings, enabling more flexible exploitation of symmetry-based sharing and potentially improving sample efficiency.
2. CoopNavi: distinguishing entity and spatial symmetry violations.
We extend planar CoopNavi by retaining the original inter-agent symmetry graph and introducing four spatial transformations: identity, left–right reflection, up–down reflection, and 180∘ rotation. Each transformation consistently acts on positions, velocities, and relative agent/landmark coordinates. The resulting action predictions are mapped back to the original coordinate frame before reliability-weighted aggregation. We design two complementary perturbations to distinguish the two types of symmetry breaking:
We additionally combine both perturbations to evaluate whether the two types of violations can be identified simultaneously.
Spatial confidences refer to predicted reliability scores before normalization for view aggregation.
Under upward forcing, the model preserves left–right reflection while downweighting up–down reflection and 180∘ rotation. Increasing the force applied only to Agent 0 from 1.0 to 1.5 to 2.0 further reduces the affected spatial confidence from 0.37 to 0.13 to 0.02.
These results show that the extended GCPD can distinguish entity-specific and direction-dependent symmetry violations, enabling selective sharing across different types of candidate symmetry relations.