acceptodds
Under review as a conference paper at ICLR 2027

Group-Context Symmetry Breaking: Turning Environmental Asymmetries into a Learnable Group Context for Reinforcement Learning

Abstract

Symmetry lets reinforcement learning agents share experience across related states and actions, but environmental asymmetries make that sharing biased. How can a broken symmetry remain a useful structural prior? We propose Group-Context Symmetry Breaking (GSB), which represents the asymmetry in a learnable context that transforms with the task's group. GSB enforces joint equivariance over state, action, and context, so an asymmetric environment becomes a fixed-context slice of an equivariant family. Real transitions teach the model which slice to use. We derive bounds linking context displacement under the group, model error, and symmetry deviation. Any model equivariant in state and action alone must pay at least half the largest deviation, so GSB provably approximates the environment better whenever its context family's approximation error falls below that floor. In soft actor–critic, the learned model generates experience in locomotion, and in manipulation it corrects real transitions carried along the group orbit while the policy reads the learned context. We compare GSB with strict, relaxed, and partially equivariant priors, including partially equivariant SAC built for symmetry-breaking environments, and with unstructured SAC. Under controlled symmetry breaking, these priors fall below even unstructured SAC on most tasks, whereas GSB attains the highest mean return of all methods at the end of training on the Hexapod, Ant, and Humanoid tasks and the most consistent return on Walker2d. On two manipulation tasks where agents train on one half of the goal domain under a constant lateral force, GSB attains the best held-out return against every baseline and ranks first in every seed. GSB's correction of the transported transitions at the learned context adds 4.35 and 1.76 in held-out return, each larger than GSB's lead over the strongest baseline, so the lead rests on this correction.

Primary area:
reinforcement learning
Keywords:
Symmetry breaking, Equivariant reinforcement learning, Approximate equivariance, Geometric deep learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Re3SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Leveraging symmetry via equivariant networks in RL lets agents share experience across symmetric states and actions, but when the symmetry is broken, exact symmetry priors introduce errors and fail to achieve optimal performance. Existing approaches using relaxed equivariance absorb the asymmetry into parameters the group does not act on, giving up exact equivariance. This paper proposes Group-context Symmetry Breaking (GSB), which treats the asymmetry as a learnable context that transforms with the group and extends the state-action space with it, so that exact equivariance holds on the extended space. The authors also prove that any strictly equivariant model leaves an error of at least half the largest symmetry deviation, whereas GSB’s error is bounded by its own approximation error. GSB achieves the highest mean return on three of four MuJoCo locomotion tasks with controlled symmetry breaking. In manipulation, it transfers best to unseen mirrored conditions, and ablations show that this gain comes from its context-based correction of mirrored transitions.

Critical strengths

S1. The main strength of this paper is a simple but principled reformulation of how equivariant RL handles symmetry breaking. Unlike prior methods that address symmetry breaking by partially relaxing equivariance, GSB models the asymmetry as a learnable context that transforms with the group and feeds it to the model as an additional input, retaining exact equivariance on the extended space. GSB opens a new direction for addressing symmetry breaking, and it can be added to a standard SAC agent with few changes to its policy and critic networks.

Critical weaknesses

W1. The simulation experiments lack key baselines and statistical support. In locomotion, GSB trains an SAC agent with transitions generated by its learned model. However, all baselines but PE-SAC are model-free SAC variants, and PE-SAC uses its learned model only to identify symmetry breaking, not to generate transitions for agent training. Thus, the gains may come from model-based data generation itself rather than from the group context. To isolate the effect of the group context, the authors should compare against model-based RL with equivariant and non-equivariant models, as well as plain orbit augmentation in locomotion. In addition, only three seeds are used for each experiment, which weakens the statistical support. Experiments on more seeds should be provided.

W2. The simulation experiments test only one kind of symmetry breaking, a hidden constant force applied to a single body. This is the setting most favorable to GSB, since the cause of the asymmetry can be moved by the group between corresponding bodies and the assumption of Lemma 9 therefore holds by construction with a low-dimensional context. Other forms of symmetry breaking, such as asymmetries with many sources (e.g., multiple fixed obstacles), which would require a high-dimensional context, are not evaluated. Experiments on more diverse types of symmetry breaking should be added to support the conclusion that broken symmetry remains a useful structural prior.

Questions for authors

Q1. Does GSB perform well in environments without symmetry breaking? Could the authors report results on exactly symmetric environments (e.g., the original tasks without the applied force)?

Q2. The return curves in Figures 5 and 6 are difficult to distinguish because of similar color tones and thick curves overlapping each other. Could the authors improve the visibility of the curves?

Q3. Throughout the manuscript, many technical terms and symbols are used without explicit definitions (e.g., the context ‘slice’ used from the abstract but not explicitly defined, and the discrepancy d_P in the main text), which makes the method hard to follow. Could the authors define the terms more clearly, with intuitive explanations?

Reviewer_VJdnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue of asymmetries in environments, they assume that the environments is non-equivariant with respect to the state-action pair (st,at)↦st+1(s_t,a_t)\mapsto s_{t+1} but that it is roughly equivariant with respect to (st,at,η)↦st+1(s_t,a_t,\eta)\mapsto s_{t+1} with η\eta being a group context. They prove that in some cases their modelization performs better than equivariance solely with respect to (st,at)(s_t,a_t) and they give an upper bound of their prediction error for some potentially unseen transitions. They finally propose a training pipeline using SAC as direct downstream task in environments where asymmetries are loads on moving robots or force inversion of a robotic arm. They show state-of-the art results in return.

Critical strengths
  • The paper successfully addresses weight asymmetry in a seemingly new parametrization that enables to still use equivariant neural network.
  • The method performs state of the art results and outperforms the baseline for the environment having the largest group cardinal
Critical weaknesses
  • The theoretical results seem quiet limited (see questions 2 and 3)
Questions for authors
  1. Asymmetries are considered single loads or force inversion, could the modelization work with several loads ? For different type of asymmetries ?
  2. The conclusion of Proposition 1 sounds obvious: let denote Q\mathcal Q the set of strictly equivariant kernel. Proposition 1 state that if if the distance between Q\mathcal Q and PP is greater thatn the distance between F\mathcal F and PP (Δ>δP\Delta>\delta_P) then F\mathcal F approximates better PP than Q\mathcal Q. Additionnaly as Q⊂F\mathcal Q\subset \mathcal F we already now that F\mathcal F approximates at least the same than Q\mathcal Q. Did I miss something ?
  3. The upper bounds of proposition 2 feel too wide: even if P=Kθ†P=K_{\theta^\dagger}, θ†=θ^\theta^\dagger=\hat\theta and η†=η^\eta^\dagger=\hat\eta we would still have an upper bound of dP(P,Kθ^)d_P(P,K_{\hat\theta}) equals to 2wP(∥η†−g−1η†∥)2w_P(\|\eta^\dagger-g^{-1}\eta^\dagger\|) even though this error equals zero. Is it possible to get a tighter bound ?
  4. In appendix C you define V\mathcal V through the set WW, I struggle to understand what WW represents, could you give examples ? What are the dimensions of η\eta in the experiments ? Are they chosen thanks to Proposition 8/9/10 ? Does parameterizing η∈V\eta\in\mathcal V instead of η∈W\eta \in W allows a significant reduction of the dimension ?
  5. Where is the condition ∥η∥≤Bη\|\eta\|\le B_\eta used ? In the training process, how the condition ∥η∥≤Bη\|\eta\|\le B_\eta is respected ?
  6. I don't understand the following sentence Line 262 : "A load the group moves between limbs makes the environment a slice of a jointly equivariant family, so δP\delta_P is only the model class’s approximation error"
Reviewer_zArqOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work studies model-based reinforcement learning in environments with broken symmetries. It introduces a context parameter to represent environment-level factors that break symmetry. By incorporating this context into the dynamics model, the method can generate or correct experience for policy learning. Experiments on several state-based control tasks demonstrate advantages over strictly equivariant methods, partially equivariant methods, and conventional SAC.

Critical strengths

The focus on symmetry-breaking environments addresses a practical limitation of methods that impose exact equivariance and offers an inspiring direction.

The method differs from approaches that relax equivariance by keeping the model exactly equivariant on the extended space and isolating the asymmetry in an environment-level input.

The paper provides both theoretical analysis and experimental results supporting the proposed method.

Critical weaknesses

I find the paper difficult to follow, and there is considerable room to improve its writing and presentation. More concrete explanations of why symmetry-breaking tasks matter in realistic applications would be helpful, as the current motivation remains vague. Although implementation details are provided in the appendices, how the context vector is incorporated into the network and how the network is constructed should be explained more clearly in the main text. Most figures and equations would also benefit from more concise and intuitive explanations.

The experiments are limited to simulated, state-based tasks. It remains unclear how the method would handle image observations or perform in real-world environments.

The experiments lack matched model-based baselines with comparable experience generation using conventional networks. The current comparisons do not fully isolate the benefit of the proposed symmetry modeling from that of the model-based training setup.

It would be helpful to discuss the current limitations and possible extensions more explicitly in the main text.

Questions for authors

Please see the weaknesses above for details.

It would be helpful to evaluate how accurately and efficiently the proposed method learns the environment dynamics. Could the authors quantitatively evaluate how well the learned dynamics models fit the environments, including prediction errors between model-predicted and the ground-truth rollout?

Could the authors clarify whether SAC components are equivariant in the locomotion and manipulation settings or not?

Could the authors explain more explicitly how the context parameter is incorporated into the dynamics network and supplied to the manipulation actor?

Could a matched model-based baseline be included to determine whether the improvement comes from the model-based setup rather than the proposed modeling of symmetry and asymmetry?

Could the authors discuss the implications and limitations of using a state-independent context, particularly whether it can represent symmetry-breaking effects that vary with the state?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion from authors

We thank the AC, SAC and PC for overseeing the discussion, and the three reviewers for their questions, which helped us state the theory more clearly. All three recognize the idea: the paper's focus "offers an inspiring direction" (zArq), GSB uses "a seemingly new parametrization" (VJdn), and is "a simple but principled reformulation" (Re3S). Below we summarize how each concern stands against the paper.

Most concerns ask for what the submission already contains. How the context enters the networks, which SAC components are equivariant, and why a state-independent context captures state-dependent effects (zArq) are stated in Sec. 3.2, Sec. 3.4 and Fig. 3; slice and d_P (Re3S) are defined in Sec. 3.2 and App. A. Our replies pointed to each location and explained it again in plain terms.

The shared soundness concern is ruled out by the paper's own experiments. zArq and Re3S suggested that the gain may come from model-based data generation rather than the group context. In manipulation, no method generates transitions for agent training, yet GSB ranks first in held-out return in every seed. On Hexapod, the same model-based pipeline with only the context frozen falls below GSB in every seed (Sec. 4.2–4.3). We also ran the requested model-based controls on all four locomotion tasks.

Where a question pointed to a real improvement, we made it. Following VJdn, the revised paper will state a tighter form of Proposition 2 and remove the context bound, which no result needs. Following Re3S, it will improve Figs. 5 and 6. No concern identified an error in any result or a claim without evidence.

How the discussion ended. We especially thank VJdn for the follow-up questions over four rounds; we did our best to answer them and to refine the theory accordingly. The remaining remarks, that Proposition 1 feels obvious and Appendix C limited, are judgments of significance: Proposition 1 proves what the subset relation cannot, a floor of Δ/2 on the worst-case error of every strictly equivariant model, and Appendix C proves that the group alone fixes the context dimension. Although zArq did not take part in the discussion, we hope that the paper and our reply clear up the misunderstandings in the review: each concern is answered in the submission, ruled out by the evidence above, or, for images and real robots, beyond what the closest baselines, PE-SAC (ICLR 2026) and RPP (NeurIPS 2021), evaluate. We respect Re3S's decision to keep the score. If Re3S still sees a substantive problem that warrants rejection, we would welcome having it pointed out.

In sum, every concern in the reviews is answered in the submission, ruled out by its experiments, or addressed by a revision, and none was left unanswered in the discussion. We hope the AC, SAC and PC will weigh the reviews in this light, together with the paper's proofs and its experiments on six tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.