acceptodds
Under review as a conference paper at ICLR 2027

Beyond Global Symmetry: State-Conditioned Graph-Structured Partial Equivariance for Reinforcement Learning

Abstract

Symmetry provides an effective inductive bias for reinforcement learning by enabling generalization across equivalent states, actions, and entities. However, in structured environments such as multi-agent systems and articulated robots, a fixed global symmetry can become unreliable, because dynamics variations, actuation constraints, and interaction changes may break symmetry selectively across entities and states. To address this, we propose Graph-Structured Partial Equivariance (GSPE), a framework that represents candidate symmetry relations using a state-conditioned symmetry graph. Each edge estimates the reliability of applying an equivariant inductive bias between two entities under the current state. Based on this framework, we introduce Graph-Conditioned Policy Decomposition (GCPD), which decomposes policy computation into graph-conditioned equivariant and residual components, preserving reliable symmetries while enabling adaptive symmetry breaking when relations become unreliable. Experiments on MuJoCo, MPE, and three-robot real-world deployment show that GSPE consistently improves robustness and generalization under structured symmetry violations compared with baselines.

Primary area:
reinforcement learning
Keywords:
Equivariant Reinforcement Learning, Partial Symmetry, Graph Neural Networks, Multi-Agent Reinforcement Learning, Robust Generalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_bCrrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tackles the problem of symmetry-breaking in equivariant reinforcement learning through graph-conditioned policy decomposition. They use a graph-structured partial equivariance which estimates the reliability from the edge of two entities, then condition it to relax the equivariance. The experiments show improved results over the baselines.

Critical strengths
  • The paper addresses an important problem in equivariant RL, and proposes reasonable and straightforward solution.
  • The motivation is reasonable. The symmetry-breaking is highly related to the relations between subsystems, and that’s where the method comes from.
Critical weaknesses
  • The proposed approach addresses the problem through state-conditioned pairwise entity relations. However, MDP symmetry is defined through transformation over state-action space, which may not involve entity permutations. For example, if the transformation leaves entity indices unchanged but symmetry-breaking occurs locally (e.g., rotation of the world frame for locomotion task which does not permute the joints but can be broken with a local slope or obstacles), there is no candidate relation to score, and the breaking cannot be relaxed by the graph. Although the proposed approach is well motivated for permutation-based structured control, it addresses a narrower symmetry-breaking problem than the broader MDP symmetry-breaking.
  • GSPE introduces additional complexity. The authors did not report wall-clock time or per step computation time to measure this trade-off. Edge-wise computation increases quadratic w.r.t. entity quantity, but the experiments are done at most six entities, not measuring the complexity beyond it.
Questions for authors
  • Can the current framework be extended to Euclidean task space symmetries? If so, how would candidate relations be defined in that setting?
  • Could the authors report training wall-clock time and inference latency compared with baselines?
  • The explanation of PE-SAC seems to be wrong. Did the authors use separate gating mechanism as written in related works? The Appendix D.2 implementation says the authors only used state-dependent gating which is different from the original approach. This raises concerns about whether PE-SAC was fairly implemented.
6 comments
AuthorsOpenReview ↗

Response to Question #1: Extension to Euclidean Task-Space Symmetries (Part 1)

We thank the reviewer for highlighting this distinction and providing a concrete example. Yes: we have implemented an extension that retains GCPD's entity-relation graph and adds reliability-weighted sharing across Euclidean task-space symmetries. These additional relations remain available when transformations leave entity indices unchanged. Our new experiments cover finite planar reflection and rotation groups.

1. Defining spatial candidate relations.

For a chosen finite transformation group K\mathcal K, the additional candidates are

\mathcal E_{\mathrm{spatial}} = \left\\{ \bigl((i,e),(i,g)\bigr) : i\in V,\\; g\in\mathcal K\setminus\\{e\\} \right\\},

where VV is the entity set, ee is the identity, and (i,g)(i,g) represents entity ii in the transformed observation. Each relation receives a confidence controlling its contribution to view aggregation. Action predictions are mapped back through the inverse action transformation before aggregation.

Thus, the scored correspondence is between spatial views of the same entity and does not require an entity permutation. This extends GCPD to a broader range of symmetry types and task settings, enabling more flexible exploitation of symmetry-based sharing and potentially improving sample efficiency.

2. CoopNavi: distinguishing entity and spatial symmetry violations.

We extend planar CoopNavi by retaining the original inter-agent symmetry graph and introducing four spatial transformations: identity, left–right reflection, up–down reflection, and 180∘180^\circ rotation. Each transformation consistently acts on positions, velocities, and relative agent/landmark coordinates. The resulting action predictions are mapped back to the original coordinate frame before reliability-weighted aggregation. We design two complementary perturbations to distinguish the two types of symmetry breaking:

  • Entity-level violation: Attenuating Agent 0's actuation by 0.550.55 introduces agent-specific dynamics asymmetry, primarily affecting inter-agent relations.
  • Spatial violation: Applying an upward (+y+y) force in the fixed world frame breaks up–down reflection and 180∘180^\circ rotational symmetry, while preserving left–right reflection. When applied uniformly, it does not introduce agent-specific asymmetry.

We additionally combine both perturbations to evaluate whether the two types of violations can be identified simultaneously.

ConditionEntity-relation confidenceSpatial-relation confidence
CleanApproximately 0.970.97Near 11
Agent 0 actuation ×0.55\times 0.55Affected: 0.770.77; healthy: 0.960.96Retained
Uniform upward force 0.50.5RetainedAffected transformations: 0.800.80
Agent 0 actuation ×0.55\times 0.55 + uniform upward force 0.50.5Affected: approximately 0.760.76Affected: approximately 0.760.76

Spatial confidences refer to predicted reliability scores before normalization for view aggregation.

Under upward forcing, the model preserves left–right reflection while downweighting up–down reflection and 180∘180^\circ rotation. Increasing the force applied only to Agent 0 from 1.01.0 to 1.51.5 to 2.02.0 further reduces the affected spatial confidence from 0.370.37 to 0.130.13 to 0.020.02.

These results show that the extended GCPD can distinguish entity-specific and direction-dependent symmetry violations, enabling selective sharing across different types of candidate symmetry relations.

AuthorsOpenReview ↗

Response to Question #1: Extension to Euclidean Task-Space Symmetries (Part 2)

3. Ant: rotations relative to fixed local terrain.

To address the reviewer's locomotion example, we introduce a direction-conditioned ridge-traversal task with command-aligned velocity reward vxy⊤dv_{xy}^{\top}d, terrain-relative health checks, and terrain-augmented observations.

The candidates are horizontal C4C_4 rotations. We rotate the robot state and direction command relative to fixed terrain, recomputing local terrain features. Joint identities and joint-torque coordinates remain unchanged, so the action transformation is the identity, TgA=IT_g^A=I. Unlike the original Ant Z4Z_4 action, which permutes leg and joint-action blocks, the new spatial transformations preserve joint identities and body-frame torque coordinates. Keeping the terrain fixed allows local geometry to invalidate otherwise useful rotational correspondences.

In representative flat-region states, spatial confidences are near 11. In ridge-contact samples, the 90∘90^\circ and 270∘270^\circ confidences decrease to 0.3330.333 and 0.3090.309, respectively, while 180∘180^\circ confidence remains high.

We also compare adaptive aggregation with GCPD without spatial sharing and fixed C4C_4 aggregation:

MethodFinal return, mean ±\pm SEMRidge-navigation success
GCPD without spatial sharing1147±1411147 \pm 14166.7% (40/60)
GCPD + fixed C4C_44497±4754497 \pm 47581.7% (49/60)
GCPD + adaptive C4C_44654±844654 \pm 8496.7% (58/60)

Success denotes reaching the opposite side of the ridge.

Adaptive C4C_4 improves mean final return by 3507 and 157 points over no spatial sharing and fixed C4C_4, respectively, with 96.7% (58/60) ridge-navigation success. These results demonstrate both the benefit of extending symmetry-based sharing to spatial transformations and the additional benefit of adapting that sharing to local symmetry reliability.

We have distinguished the spatial-view extension from the original entity-relation implementation in Appendices A.5 and B.8, documented the transformation definitions in Appendix B.8, and added the evaluation protocol and available spatial results in Appendices D.8 and E.14. We have also added a supplementary video of the new Ant ridge-navigation task, showing the rendered terrain and the learned symmetry-relation scores.

AuthorsOpenReview ↗

Response to Question #2: Training Time and Inference Latency

We thank the reviewer for requesting this comparison. To address this concern, we conducted additional experiments comparing GPU inference latency, training time, and sample efficiency against five baselines across four benchmark environments. Our results demonstrate a clear computation–performance trade-off: GCPD achieves 0.439 ms complete GPU inference, while delivering a 15.24% mean final-return improvement over PEnGUiN with only 6.6% additional full-budget training time. We quantify the deployment and training costs below.

1. Deployment latency.

MethodGPU inference latency (ms)
SAC0.290
Equi-SAC0.931
ApproxEqui-SAC0.983
PE-SAC1.331
PEnGUiN2.718
GCPD0.439

GCPD achieves 0.439 ms end-to-end GPU inference, including symmetry-graph prediction and action generation. Its implementation uses a deterministic-action path and precomputed static indices and constants. Deployment requires only the graph predictor and actor; auxiliary predictors, consistency-target construction, and critics are training-only components.

2. Training time and performance.

Estimated full-budget wall-clock training times are for a single NVIDIA RTX 4090, averaged across the four benchmark environments. Each baseline's final evaluation return serves as the reference target.

MethodMean full training time (h)GCPD steps to target (% of full budget)Estimated time ratio¹GCPD mean final-return improvement
SAC1.8536.50%3.07×43.82%
Equi-SAC4.6858.50%1.95×39.04%
ApproxEqui-SAC5.0857.60%1.76×41.77%
PE-SAC9.0048.90%0.85×60.28%
PEnGUiN14.5969.50%0.74×15.24%
GCPD15.56100%1.00×0% (reference)

¹ Estimated GCPD time to the baseline's final-return target divided by the baseline's full-budget training time:

Estimated time ratio=15.56×GCPD steps to targetbaseline full-budget training time.\text{Estimated time ratio} = \frac{15.56 \times \text{GCPD steps to target}}{\text{baseline full-budget training time}}.

The additional computation is accompanied by mean final-return improvements of 43.82%, 39.04%, and 41.77% over SAC, Equi-SAC, and ApproxEqui-SAC, respectively. GCPD also reaches the PE-SAC and PEnGUiN reference returns in an estimated 0.85×0.85\times and 0.74×0.74\times of their respective full-budget training times.

Compared with PEnGUiN, GCPD's estimated full-training overhead is 6.6% ((15.56−14.59)/14.59×100(15.56-14.59)/14.59 \times 100\\% \approx 6.6\\%), alongside a 15.24% mean final-return improvement. Furthermore, GCPD reaches the PE-SAC and PEnGUiN reference returns in an estimated 0.85× and 0.74× of their respective full-budget training times.

We have added the runtime measurements and timing protocol to Appendices E.13 and D.7. Appendix B.7 specifies the computational scaling and the candidate-relation counts in the evaluated domains.

Reviewer_Jp2FOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work addresses reinforcement learning in structured environments where symmetry can break selectively across entities and states. It introduces Graph-Structured Partial Equivariance (GSPE), which represents the current reliability of candidate symmetry relations through a state-conditioned symmetry graph. Graph-Conditioned Policy Decomposition (GCPD) uses these relation scores to gate symmetry-breaking residual messages alongside shared equivariant computation. The main contribution is state-dependent, relation-level control of the equivariant inductive bias, rather than a single global or node-level decision. The paper also provides a conditional policy-equivariance bound and a frozen-controller return bound. Experiments on MuJoCo and MPE show improved control performance under structured symmetry violations. Additional ablations, mechanism analyses, and generalization tests examine the contributions of individual components and the robustness of the method.

Critical strengths
  1. The method is well motivated and coherently designed. State-conditioned, relation-level confidence provides a natural way to preserve useful equivariant sharing while allowing selective symmetry breaking.
  2. The paper provides a thorough experimental evaluation. Comparisons with relevant baselines, ablation studies, and targeted mechanism analyses support the method’s effectiveness and the value of state-dependent, relation-level modeling.
Critical weaknesses

The symmetry graph is inferred from the current observation alone, without explicitly using interaction history to infer hidden dynamics. Changes in actuator effectiveness or contact conditions may not be identifiable from a single observation; history-based adaptation methods such as RMA highlight the value of recent state–action history in such settings. This limits the method’s applicability when different underlying symmetry conditions produce similar current observations.

Questions for authors

How does the current-observation graph scorer handle cases where different hidden dynamics produce similar observations but imply different symmetry reliabilities? Does the method assume that the current observation is sufficient to distinguish these conditions?

3 comments
AuthorsOpenReview ↗

Response: Observation Sufficiency and History-Conditioned Scoring

We thank the reviewer for highlighting this important question. Current observations can provide sufficient cues for effective symmetry-reliability estimation in our evaluated settings, as supported by our state-dependent graph diagnostics and zero-shot transfer results. However, they cannot distinguish hidden dynamics that produce identical observations. To further improve generalization, we extend GCPD with history-conditioned scoring, achieving a 15.0% higher mean return across unseen actuator strengths with four-step history.

1. Identifiability and the role of the current observation.

For a fixed trained scorer, identical inputs necessarily produce identical predictions:

ot(1)=ot(2)⟹Wϕ(ot(1))=Wϕ(ot(2)).o_t^{(1)}=o_t^{(2)} \quad\Longrightarrow\quad W_\phi(o_t^{(1)})=W_\phi(o_t^{(2)}).

Therefore, if two hidden conditions require different relation reliabilities but yield the same observation, the current scorer cannot resolve that ambiguity. For similar observations, discrimination depends on whether their remaining differences contain informative cues.

The current-observation formulation relies on the available observation being informative about task-relevant relation reliability. Its objective is to predict the operational reliability of candidate relations, rather than recover all hidden physical parameters.

2. History-conditioned extension and additional experiments.

History can be incorporated by replacing

Wϕ(ot)withWϕ(ot,Ht),W_\phi(o_t) \quad\text{with}\quad W_\phi(o_t,\mathcal H_t),

where Ht\mathcal H_t denotes recent observation history, while retaining GCPD's relation-level policy decomposition. Temporal information can provide additional evidence about hidden dynamics through their effects on observed motion. This is complementary to the use of recent state–action history for adaptation in RMA (Kumar et al., 2021).

We evaluated one-step and four-step history variants, training at actuator strength α=0.05\alpha=0.05 and testing at α∈0.4,0.7,1.0\alpha\in\\{0.4,0.7,1.0\\}, where α=1.0\alpha=1.0 denotes normal actuation.

Test actuator strength α\alphaCurrent observation (4 seeds)One-step history (2 seeds)Four-step history (2 seeds)
0.050.05 (training strength)6443±2056443 \pm 2056012±11436012 \pm 11435957±545957 \pm 54
0.40.45087±2815087 \pm 2815532±15965532 \pm 15964953±7024953 \pm 702
0.70.74709±8614709 \pm 8614132±17724132 \pm 17724917±314917 \pm 31
1.01.0 (normal actuation)3555±12993555 \pm 12993470±27653470 \pm 27655490±555490 \pm 55
Mean over unseen strengths445043785120

The final row averages the mean returns at α=0.4,0.7,1.0\alpha=0.4,0.7,1.0.

Four-step history increases the unseen-strength mean from 4450 to 5120, with the largest gain at normal actuation (3555 to 5490), while its training-strength return decreases from 6443 to 5957. With two seeds per history variant, we view these results as preliminary evidence that temporal context can improve transfer while preserving the relation-level decomposition. We have clarified the observability limitation in Appendix A.5 and added the history comparison in Appendix E.15, with its architecture, training procedure, and evaluation protocol described in Appendices B.9, C.8, and D.9.

Thank you, I have no more concerns and am impressed by the performance of GCPD. But I don't understand why the GPU inference latency of GCPD is less than PE-SAC and PEnGUiN. Can you please discuss where the gap comes from?

AuthorsOpenReview ↗

Response to Follow-up: GPU Inference Latency

We thank the reviewer for the encouraging feedback and the follow-up question.

The lower inference latency primarily results from implementation-level optimizations rather than inherently lower algorithmic complexity. Before optimization, GCPD has a mean inference latency of 2.906 ms, comparable to PEnGUiN's 2.718 ms (a difference of only 6.9%). We then apply two stages of optimization:

  1. Deterministic-action inference: A dedicated inference path reduces mean GPU latency from 2.906 to 2.500 ms per decision (1.16× speedup).
  2. Precomputation and CUDA Graph replay: Precomputing static indices and constants and enabling CUDA Graph replay further reduces latency to 0.439 ms (an additional 5.69× speedup).

Together, these optimizations achieve a 6.62× speedup across the four benchmark tasks, while preserving the same model architecture and recomputing state-conditioned symmetry scores for every observation. Complete inference still includes both symmetry-graph prediction and action generation.

Thus, GCPD exhibits latency comparable to PEnGUiN before optimization, while achieving substantially faster inference after engineering optimizations. The baseline implementations have not undergone the same optimization procedures, so the observed latency advantage should be attributed primarily to implementation efficiency rather than reduced algorithmic complexity.

We have clarified these implementation details in the revised appendix.

Reviewer_AvtMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes a method for learning symmetries implicit in a factored MDP and using that information to speed RL. In terms of applications, the work is focused on factored MDPs and on multi-agent MDPs in particular. I did not understand the approach in detail. Roughly, the approach is to infer a symmetry graph over the factored state variables during RL learning and to use it to improve policy learning. There are some empirical results for a couple of DMC domains, Antv5 and Walker2dv5, that seem impressive. Essentially, the proposed method is able to identify symmetries as they exist in the domain. Not only can it leverage symmetries, but it can accommodate imperfect symmetries caused by various kinds of failures such as joint failures, etc. In all cases, the proposed method seems to have an advantage over several reasonable baselines including a baseline equivariant SAC.

Critical strengths

The method proposed here seems like a good approach to leveraging symmetries among factored variables that are learned online.

As far as I can tell, the approach is theoretically sound.

I thought the results in Fig 3 were good, although I was a little confused about the performance of the other methods

Critical weaknesses

I believe the biggest weakness is that the method will only make sense in the context of a factored state and action space. Several domains fit this description, such as legged robots and multi agent systems. Nevertheless, there are clearly lots of visual domains where this would be insufficient.

I had a really hard time understanding this paper. The main problems are: 1) the approach is not described in a way that is intuitive to understand. There is no overall summary of the method; 2) the theoretical notation is hard to understand; 3) some math notation does not seem to be defined; 4) figs 1 and 2 could probably be improved to help convey the idea.

Questions for authors

I did not understand why non-equivariant SAC performed similarly to the other equivariant baselines in Fig 3

4 comments
AuthorsOpenReview ↗

Response to Weakness #1: Scope and Extension to Spatial Symmetries (Part 1)

We agree that the current framework assumes structured state and action representations, and does not directly process raw visual observations. Within this scope, however, the proposed relation-wise reliability modeling is not limited to permutations between entities. We demonstrate this by extending GCPD to spatially transformed views of the same entity, without requiring changes to entity identities.

1. Spatial relations beyond entity permutations.

We retain the entity-relation graph and add reliability-weighted aggregation over spatially transformed observations:

(i,e)⟷(i,g),(i,e)\longleftrightarrow(i,g),

where ee is the identity and gg a spatial transformation. These relations connect views of the same entity, with action predictions mapped back to the original coordinates before aggregation. This extends GCPD to a broader range of symmetry types and task settings, enabling more flexible exploitation of symmetry-based sharing and potentially improving sample efficiency.

2. CoopNavi: distinguishing entity and spatial symmetry violations.

We extend planar CoopNavi by retaining the original inter-agent symmetry graph and introducing four spatial transformations: identity, left–right reflection, up–down reflection, and 180∘180^\circ rotation. Each transformation consistently acts on positions, velocities, and relative agent/landmark coordinates. The resulting action predictions are mapped back to the original coordinate frame before reliability-weighted aggregation. We design two complementary perturbations to distinguish the two types of symmetry breaking:

  • Entity-level violation: Attenuating Agent 0's actuation by 0.550.55 introduces agent-specific dynamics asymmetry, primarily affecting inter-agent relations.
  • Spatial violation: Applying an upward (+y+y) force in the fixed world frame breaks up–down reflection and 180∘180^\circ rotational symmetry, while preserving left–right reflection. When applied uniformly, it does not introduce agent-specific asymmetry.

We additionally combine both perturbations to evaluate whether the two types of violations can be identified simultaneously.

ConditionEntity confidenceSpatial confidence
CleanApproximately 0.970.97Near 11
Agent 0 actuation ×0.55\times 0.55Affected: 0.770.77; healthy: 0.960.96Retained
Uniform upward force 0.50.5RetainedAffected transformations: 0.800.80
Agent 0 actuation ×0.55\times 0.55 + uniform upward force 0.50.5Affected: approximately 0.760.76Affected: approximately 0.760.76

Spatial confidences refer to predicted reliability scores before normalization for view aggregation.

Under upward forcing, the model preserves left–right reflection while downweighting up–down reflection and 180∘180^\circ rotation. Increasing the force applied only to Agent 0 from 1.01.0 to 1.51.5 to 2.02.0 further reduces the affected spatial confidence from 0.370.37 to 0.130.13 to 0.020.02.

These results show that the extended GCPD can distinguish entity-specific and direction-dependent symmetry violations, enabling selective sharing across different types of candidate symmetry relations.

AuthorsOpenReview ↗

Response to Weakness #1: Scope and Extension to Spatial Symmetries (Part 2)

3. Ant: spatial adaptation to local terrain.

To further evaluate symmetry breaking without entity permutations, we introduce a direction-conditioned ridge-navigation task in Ant. Unlike standard Ant locomotion, the task rewards movement along a specified world-frame direction dd through vxy⊤dv_{xy}^{\top}d, uses terrain-relative health checks, and includes local terrain geometry in the observations.

We evaluate spatial C4C_4 rotations (0∘0^\circ, 90∘90^\circ, 180∘180^\circ, 270∘270^\circ) while keeping the ridge terrain fixed. Each transformation rotates the robot's world-frame state and movement command, with local terrain features recomputed accordingly. Importantly, joint identities and joint-torque coordinates remain unchanged (TgA=IT_g^A=I). Thus, the same control structure is shared across spatial views, but the fixed ridge can make certain rotations unreliable near terrain contact.

In representative flat-region states, spatial confidences are near 11. In ridge-contact samples, the 90∘90^\circ and 270∘270^\circ confidences decrease to 0.3330.333 and 0.3090.309, respectively, while 180∘180^\circ confidence remains high, indicating adaptation to local terrain.

We also compare adaptive sharing against GCPD without spatial sharing and fixed C4C_4 aggregation:

MethodFinal return, mean ±\pm SEMRidge-navigation success
GCPD without spatial sharing1147±1411147 \pm 14166.7% (40/60)
GCPD + fixed C4C_44497±4754497 \pm 47581.7% (49/60)
GCPD + adaptive C4C_44654±844654 \pm 8496.7% (58/60)

Success denotes reaching the opposite side of the ridge.

Adaptive C4C_4 improves mean final return by 3507 and 157 points over no spatial sharing and fixed C4C_4, respectively, with 96.7% (58/60) ridge-navigation success.

These results demonstrate that GCPD's symmetry-based sharing extends beyond entity permutations to broader classes of symmetries, substantially expanding its applicability across structured control tasks while delivering strong performance.

The extension retains structured state and action inputs. We have documented its architecture, evaluation protocol, and available results in Appendices B.8, D.8, and E.14, respectively. We have also added a supplementary video of the new Ant ridge-navigation task, showing the rendered terrain and the learned symmetry-relation scores.

AuthorsOpenReview ↗

Response to Weakness #2: Presentation and Clarity

We thank the reviewer for identifying these presentation issues. We have revised the manuscript to provide an explicit method overview, reorganize the theoretical exposition, clarify notation, and redesign Figs. 1–2. These revisions address the four points as follows.

1. An intuitive method overview (opening of Sec. 4).

The revised section first explains the complete pipeline before introducing detailed equations: GSPE predicts the reliability of each candidate relation from the current state, and GCPD uses these confidences to selectively enable symmetry-breaking corrections. The central operation is

mij=mijeq+(1−wij(s))mijres.m_{ij} = m_{ij}^{\mathrm{eq}} + \bigl(1-w_{ij}(s)\bigr)m_{ij}^{\mathrm{res}}.

The shared equivariant component remains active, while lower confidence permits a larger residual correction on the corresponding relation. The overview also distinguishes auxiliary graph supervision from SAC optimization and clarifies that deployment requires only the current-state graph predictor and actor.

2. A clearer organization of the theoretical analysis (Sec. 4.5).

We consolidated the two theoretical propositions into a dedicated Theoretical Properties subsection, after the architecture and learning objectives, rather than interrupting the policy description. We also clarified the quantities used in the bounds, including the policy-parameter map qWq_W, its induced transformation TgΠT_g^\Pi, the return J(π)J(\pi), and the occupancy dπd^\pi. The theoretical action-value function QπQ^\pi is now explicitly distinguished from the learned SAC critics.

3. Expanded symbol definitions and simplified notation (Secs. 3–4).

We added explicit definitions for previously implicit notation, including the transformed entity index gi:=g⋅igi:=g\cdot i, the standard basis vector eie_i, and the log-standard-deviation matching weight λσ\lambda_\sigma. We simplified the fixed normalizer to nA=∣VA∣n_A=\lvert V_A\rvert, removed the unnecessary "new" superscript from Lsym\mathcal L_{\mathrm{sym}}, and specified λσ=1\lambda_\sigma=1 across all four environments in Table S5.

4. Redesigned motivation and architecture figures (Figs. 1–2).

Figure 1 now illustrates both dimensions of the motivation: different relations can have different reliability within a state, and the same failure can produce different reliability patterns as actuator demand changes. Its three panels show the nominal system and the same RB-leg failure under low and high demand, which highlights the need for state-conditioned, relation-wise symmetry modeling that selectively preserves reliable sharing while relaxing unreliable relations.

Figure 2 now separates online inference from training objectives, explicitly depicts the equivariant-plus-residual computation, and shows how dynamics- and reward-consistency targets supervise the graph predictor. Forward computation and parameter updates use different arrow styles, with stop-gradient operations marked to clarify gradient isolation.

Together, these revisions provide a direct connection from the motivating example to the computational pipeline and then to the formal analysis.

AuthorsOpenReview ↗

Response: Why SAC Remains Competitive in Some Conditions in Fig. 3

We thank the reviewer for raising this question. Symmetry-based sharing can improve learning efficiency, but its benefits depend on how well the candidate relations match the task. When sharing is imperfect, these benefits can diminish, while per-condition training allows SAC to learn condition-specific control. SAC can therefore remain competitive in final return, including in some clean conditions.

1. Competitive SAC performance reflects the limits of enforcing imperfect symmetry priors.

SAC can outperform equivariant baselines because enforcing an imperfect symmetry prior may restrict the policy from learning necessary asymmetric behaviors. Unlike Equi-SAC, which enforces global equivariance, or ApproxEqui-SAC, which uses static relaxation, unconstrained SAC can directly learn condition-specific asymmetric control without potentially incorrect sharing constraints. This provides a plausible explanation for its competitive performance when the assumed symmetries do not fully match the task.

Representative results from Table S6 (mean ±\pm SEM, four seeds) support this interpretation:

EnvironmentConditionSACEqui-SACApproxEqui-SACGCPD
AntClean4379±3104379 \pm 3103449±2383449 \pm 2383790±4933790 \pm 4936197±1796197 \pm 179
AntFriction asymmetry4116±2734116 \pm 2732499±4152499 \pm 4152156±2632156 \pm 2635236±6495236 \pm 649
Walker2dFriction asymmetry4374±694374 \pm 694366±3354366 \pm 3353458±9463458 \pm 9465061±3015061 \pm 301
Walker2dJoint failure3849±2083849 \pm 2083408±2433408 \pm 2433716±4723716 \pm 4723722±1893722 \pm 189

In Ant, SAC exceeds strictly equivariant Equi-SAC by 930 and 1617 return points in the clean and friction-asymmetry conditions, respectively. Yet GCPD further exceeds SAC by 1818 and 1120 points in these same conditions. This pattern is consistent with the distinction between exploiting useful symmetry and enforcing a globally fixed symmetry prior: unconstrained SAC can avoid restrictive equivariance assumptions, while GCPD retains selective sharing where candidate relations remain reliable.

This interpretation also applies to clean conditions. In our experiments, "clean" denotes the absence of externally injected perturbations; it does not guarantee exact MDP symmetry under the chosen candidate transformations (Sec. 5.1). Thus, the symmetry prior can remain imperfect even in nominal conditions.

2. Per-condition training allows SAC to learn asymmetric compensation.

Each method in Fig. 3 is trained from scratch in every domain–condition combination, with the affected entity fixed within each perturbed condition (Appendix D.5–D.6). SAC therefore has the opportunity to learn asymmetric compensation directly from interaction data. Together with possible mismatches in symmetry-based sharing, this provides a plausible explanation for its competitive performance.

3. Competitive performance within a training condition does not necessarily imply equally strong transfer.

Our frozen identity-transfer experiment provides complementary evidence (Table S8). In CoopNavi-n=6n=6, all methods are trained with failures affecting Agents 0 and 1, then evaluated on other failure pairs at the same strength, with frozen parameters and no fine-tuning. The following results are mean ±\pm standard error across four frozen checkpoints.

MethodSeen failure identitiesTwo unseen failure identities
SAC−223.21±1.48-223.21 \pm 1.48−252.47±3.41-252.47 \pm 3.41
Equi-SAC−78.30±4.50-78.30 \pm 4.50−76.80±3.50-76.80 \pm 3.50
PEnGUiN−75.51±2.16-75.51 \pm 2.16−73.86±3.22-73.86 \pm 3.22
GCPD−70.12±1.01-70.12 \pm 1.01−68.17±1.65-68.17 \pm 1.65

Relative to its own seen-condition return, SAC drops by 29.26 points when both failure identities are unseen, whereas the listed symmetry-aware methods retain similar mean returns. This is consistent with the value of symmetry-based sharing for identity transfer and highlights why in-condition performance and transfer should be evaluated separately. We have clarified this distinction in Appendix E.1, which discusses the final-return comparisons in Fig. 3 and explicitly distinguishes them from the frozen identity-transfer evaluation in Appendix E.3.

Reviewer_aPaVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes {Graph-Structured Partial Equivariance (GSPE)} and {Graph-Conditioned Policy Decomposition (GCPD)}. By dynamically estimating pairwise relation reliability via a state-conditioned symmetry graph, the framework preserves reliable symmetry inductive biases while enabling adaptive symmetry breaking under local failures, significantly enhancing robustness and generalization.

Critical strengths

Pro #1

Compared to PEnGUiN, which uses a single receiving-node score, or ApproxEqui-RL, which uses static relaxation parameters, GSPE performs independent, state-evolving dynamic scoring on pairwise relations between entities, enabling more precise handling of local failures (e.g., a specific damaged leg).

Pro #2

Demonstrates superior robustness and zero-shot transfer capabilities under unseen failure strengths, delays, and identity changes in MuJoCo, MPE, and real-world quadruped deployments (as shown in the supplementary video).

Critical weaknesses

Con #1: Higher Computational Complexity

Due to the need to maintain a state-conditioned symmetry graph Wϕ(s)W_\phi(s) and execute graph-conditioned policy and critic decomposition, the framework incurs higher computational overhead and implementation complexity compared to standard continuous control algorithms (such as standard SAC) or simpler symmetric networks.

Con #2: Dependency on Entity Factorization and Candidate Relations

If the underlying physical system or task cannot be naturally formulated as an entity graph with clear symmetry candidates, its performance may degrade.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

Response to Con #1: Computational Cost

We thank the reviewer for the constructive feedback. We agree that these benefits should be evaluated alongside their computational costs. To address this concern, we conducted additional experiments comparing GPU inference latency, training time, and sample efficiency against five baselines across four benchmark environments. Our results demonstrate a clear computation–performance trade-off: GCPD achieves 0.439 ms complete GPU inference, while delivering a 15.24% mean final-return improvement over PEnGUiN with only 6.6% additional full-budget training time. We quantify the deployment and training costs below.

1. Deployment latency.

MethodGPU inference latency (ms)
SAC0.290
Equi-SAC0.931
ApproxEqui-SAC0.983
PE-SAC1.331
PEnGUiN2.718
GCPD0.439

GCPD achieves 0.439 ms end-to-end GPU inference, including symmetry-graph prediction and action generation. Its implementation uses a deterministic-action path and precomputed static indices and constants for optimization. Deployment requires only the graph predictor and actor; auxiliary predictors, consistency-target construction, and critics are training-only components.

2. Training cost and performance.

Estimated full-budget training times are for a single NVIDIA RTX 4090, averaged across the four benchmark environments. Each baseline's final evaluation return serves as the reference target.

MethodMean full training time (h)GCPD steps to target (% of full budget)Estimated time ratio¹GCPD mean final-return improvement
SAC1.8536.50%3.07×43.82%
Equi-SAC4.6858.50%1.95×39.04%
ApproxEqui-SAC5.0857.60%1.76×41.77%
PE-SAC9.0048.90%0.85×60.28%
PEnGUiN14.5969.50%0.74×15.24%
GCPD15.56100%1.00×0% (reference)

¹ Estimated GCPD time to the baseline's final-return target divided by the baseline's full-budget training time:

Estimated time ratio=15.56×GCPD steps to targetbaseline full-budget training time.\text{Estimated time ratio} = \frac{15.56 \times \text{GCPD steps to target}}{\text{baseline full-budget training time}}.

The additional computation is accompanied by mean final-return improvements of 43.82%, 39.04%, and 41.77% over SAC, Equi-SAC, and ApproxEqui-SAC, respectively, while GCPD reaches their reference returns within 36.5%–58.5% of the full interaction budget. This represents a trade-off between computational cost, sample efficiency, and final control performance.

Compared with PEnGUiN, GCPD's estimated full-training overhead is 6.6% ((15.56−14.59)/14.59×100(15.56-14.59)/14.59 \times 100\\% \approx 6.6\\%), alongside a 15.24% mean final-return improvement. Furthermore, GCPD reaches the PE-SAC and PEnGUiN reference returns in an estimated 0.85× and 0.74× of their respective full-budget training times.

We have added the runtime measurements and timing protocol to Appendices D.7 and E.13.

AuthorsOpenReview ↗

Response to Con #2: Dependency on Entity Factorization and Candidate Relations (Part 1)

We thank the reviewer for pointing out this question. While GCPD still requires an entity-factored representation, it does not assume that all candidate relations are valid. In our original Ant joint-failure experiments, activating 50% of an additional pool of four incorrect relations reduces GCPD's return by only 6.7% (6165 to 5749) (Table 2 and Appendix E.8). Furthermore, we broaden GCPD's applicability beyond permutation-based task settings by extending its candidate relations to spatial transformations that preserve entity identities. This enables reliability-guided sharing in a wider range of symmetry-breaking scenarios, as demonstrated in CoopNavi and Ant. We detail these two findings below.

1. Robustness to incorrect candidate relations (Table 2 and Appendix E.8).

Our original Ant joint-failure experiment introduces an additional pool of four incorrect candidate relations. Activating 50% of this pool reduces GCPD's final return from 6165 to 5749, a decrease of approximately 6.7%. The confidence diagnostics in Appendix E.8 complement this return result by showing reduced confidence in incorrect relations. These findings support treating candidate relations as hypotheses whose reliability is learned, while preserving most of the control performance when additional candidates are invalid.

2. Extending candidate relations beyond entity permutations.

We retain the entity-relation graph and add reliability-weighted aggregation over spatially transformed observations:

(i,e)⟷(i,g),(i,e)\longleftrightarrow(i,g),

where ee is the identity and gg a spatial transformation. These relations connect views of the same entity, with action predictions mapped back to the original coordinates before aggregation. This extends GCPD to a broader range of symmetry types and task settings, enabling more flexible exploitation of symmetry-based sharing and potentially improving sample efficiency.

3. CoopNavi: distinguishing entity and spatial symmetry violations.

We extend planar CoopNavi by retaining the original inter-agent symmetry graph and introducing four spatial transformations: identity, left–right reflection, up–down reflection, and 180∘180^\circ rotation. Each transformation consistently acts on positions, velocities, and relative agent/landmark coordinates. The resulting action predictions are mapped back to the original coordinate frame before reliability-weighted aggregation. We design two complementary perturbations to distinguish the two types of symmetry breaking:

  • Entity-level violation: Attenuating Agent 0's actuation by 0.550.55 introduces agent-specific dynamics asymmetry, primarily affecting inter-agent relations.
  • Spatial violation: Applying an upward (+y+y) force in the fixed world frame breaks up–down reflection and 180∘180^\circ rotational symmetry, while preserving left–right reflection. When applied uniformly, it does not introduce agent-specific asymmetry.

We additionally combine both perturbations to evaluate whether the two types of violations can be identified simultaneously.

ConditionEntity confidenceSpatial confidence
CleanApproximately 0.970.97Near 11
Agent 0 actuation ×0.55\times 0.55Affected: 0.770.77; healthy: 0.960.96Retained
Uniform upward force 0.50.5RetainedAffected transformations: 0.800.80
Agent 0 actuation ×0.55\times 0.55 + uniform upward force 0.50.5Affected: approximately 0.760.76Affected: approximately 0.760.76

Spatial confidences refer to predicted reliability scores before normalization for view aggregation.

Under upward forcing, the model preserves left–right reflection while downweighting up–down reflection and 180∘180^\circ rotation. Increasing the force applied only to Agent 0 from 1.01.0 to 1.51.5 to 2.02.0 further reduces the affected spatial confidence from 0.370.37 to 0.130.13 to 0.020.02.

These results show that the extended GCPD can distinguish entity-specific and direction-dependent symmetry violations, enabling selective sharing across different types of candidate symmetry relations.

AuthorsOpenReview ↗

Response to Con #2: Dependency on Entity Factorization and Candidate Relations (Part 2)

4. Ant: spatial adaptation to local terrain.

To further evaluate symmetry breaking without entity permutations, we introduce a direction-conditioned ridge-navigation task in Ant. Unlike standard Ant locomotion, the task rewards movement along a specified world-frame direction dd through vxy⊤dv_{xy}^{\top}d, uses terrain-relative health checks, and includes local terrain geometry in the observations.

We evaluate spatial C4C_4 rotations (0∘0^\circ, 90∘90^\circ, 180∘180^\circ, 270∘270^\circ) while keeping the ridge terrain fixed. Each transformation rotates the robot's world-frame state and movement command, with local terrain features recomputed accordingly. Importantly, joint identities and joint-torque coordinates remain unchanged (TgA=IT_g^A=I). Thus, the same control structure is shared across spatial views, but the fixed ridge can make certain rotations unreliable near terrain contact.

In representative flat-region states, spatial confidences are near 11. In ridge-contact samples, the 90∘90^\circ and 270∘270^\circ confidences decrease to 0.333 and 0.309, respectively, while 180∘180^\circ confidence remains high.

We also compare adaptive sharing against GCPD without spatial sharing and fixed C4C_4 aggregation:

MethodFinal return, mean ±\pm SEMRidge-navigation success
GCPD without spatial sharing1147±1411147 \pm 14166.7% (40/60)
GCPD + fixed C4C_44497±4754497 \pm 47581.7% (49/60)
GCPD + adaptive C4C_44654±844654 \pm 8496.7% (58/60)

Success denotes reaching the opposite side of the ridge.

Adaptive C4C_4 improves mean final return by 3507 and 157 points over no spatial sharing and fixed C4C_4, respectively, with 96.7% (58/60) ridge-navigation success.

These results demonstrate that GCPD's symmetry-based sharing extends beyond entity permutations to broader classes of symmetries, substantially expanding its applicability across structured control tasks while delivering strong performance.

Entity factorization remains an applicability assumption. We have documented the spatial-relation extension in Appendix B.8, its evaluation protocol in Appendix D.8, and the available spatial results in Appendix E.14. Appendix E.8 reports the existing evidence on incorrect candidate relations. We have also added a supplementary video of the new Ant ridge-navigation task, showing the rendered terrain and the learned symmetry-relation scores.

AuthorsOpenReview ↗

Additional Revisions: Presentation and Clarity

We also thank the reviewer for the feedback on the presentation. In addition to addressing the two concerns, we have revised the manuscript to make the motivation, method, and mathematical notation easier to follow.

  1. Clearer motivation and contributions (Introduction and Fig. 1, pp. 1–2). We rewrote the introduction around two complementary challenges: symmetry reliability varies across relations and across states. The redesigned Fig. 1 illustrates both through the same limb failure under different actuator demands. We also separately enumerate the contributions of GSPE, GCPD, and the empirical validation.

  2. An explicit method overview (opening of Sec. 4, p. 4). Before introducing the detailed equations, we now explain how GSPE predicts relation confidences and how GCPD uses them to gate residual corrections while retaining shared equivariant computation. The overview also distinguishes auxiliary graph supervision from SAC optimization and identifies the components required at deployment.

  3. A redesigned architecture diagram (Fig. 2, p. 3). The new figure separates online inference from training objectives, explicitly depicts the equivariant-plus-residual computation, and shows how consistency targets supervise the graph predictor. Different arrow styles distinguish forward computation from parameter updates, with stop-gradient operations marked to clarify gradient isolation.

  4. More accessible organization and notation (Secs. 3–4 and Table S5). We consolidated the theoretical bounds into Sec. 4.5, after the architecture and learning objectives, so they no longer interrupt the policy description. We expanded symbol definitions, simplified the fixed normalizer to nA=∣VA∣n_A = \lvert V_A\rvert, and clarified the meaning of λσ\lambda_\sigma, including its value of 1.0 across all four environments in Table S5.

We hope these revisions make the connection between the motivating example, the computational pipeline, and the theoretical analysis clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.