acceptodds
Under review as a conference paper at ICLR 2027

Exploring Synchrony for Communication in Modular Neural Networks

Abstract

One motivation for modular neural networks is to divide computation across components that learn distinct, reusable functions, which could then be recombined to support compositional generalisation. In practice, modular structure alone does not guarantee that such roles develop, and unrestricted communication between modules can undermine any specialisation that does emerge. This raises the question of how modular communication should be organised. The problem can be viewed as an analogue of the perceptual binding problem, where synchrony has been proposed as a mechanism for coordinating distributed representations. We introduce SyncNet, a modular architecture that separates module content from a dynamical phase state: learnt Kuramoto dynamics evolve the phases, while phase alignment determines communication through a shared channel. We evaluate SyncNet on Sort-of-CLEVR for relational reasoning and on SQOOP for systematic generalisation, where lower pair variety demands more recombination at test time. SyncNet gives the best held-out accuracy of the models compared at the three lowest pair varieties, by 5.1 percentage points at the hardest, and is competitive on Sort-of-CLEVR though below the strongest attention baselines. Test-time overrides show that performance depends on phases that evolve: fixed phases cost as much as no addressing at all, and permuting which module holds which region is nearly as damaging. A content-addressed variant trained from scratch nonetheless matches it, so the benefit appears to come from the modular decomposition rather than synchrony itself.

Primary area:
learning on time series and dynamical systems
Keywords:
Modular Neural Networks · Compositional Generalization · Neural Synchrony · Kuramoto Dynamics · Phase-Based Communication

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_j8LYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces a novel framework for learning modular neural networks by learning how components within a structurally modular architecture should communicate. Taking inspiration from the mechanism of synchrony, they separate the information encoded in neural modules into content and phase, and use phase to restrict communication in a shared channel by having modules share their content modulated by phase. The authors test their method on two relational visual datasets (SQOOP and Sort-of-CLEVR) and compare their results with several baseline models, as well as performing ablations within their method.

Critical strengths
  • This work introduces an incredibly creative and well justified approach to learning modular systems, and makes the reasoning behind this approach evident and easy to parse. The method presented clearly separates itself from the existing literature, and justifies each decision taken with a specific problem to solve. Furthermore, despite having little experience with phase based methods for communication, I was able to follow and understand the ideas and implementation of the method with little additional reading. The authors provided formal and diligent explanations for every step, making the reasoning easy to follow and a pleasure to read.
  • The testing and particularly the ablation is quite exhaustive, with several combinations of the different parts of the method being evaluated in order to isolate contribution. The results are discussed in detail, and the authors provide justified explanations and conclusions for all empirical values. I especially commend the authors for prominently featuring the not so flattering Sort-of-CLEVR results and discussing them. This and the transparency of the training and testing regimes shows a commendable commitment to scientific integrity.
Critical weaknesses
  • I feel the underperformance in the Sort-of-CLEVR task should be discussed in more detail. Though still performing comparably to the top models, the method does fall slightly short. The current explanation does not give a clear reasoning as to why this might be, and I think a deeper discussion would make for a stronger submission.
Questions for authors

Again, I commend you for a very interesting and entertaining work. I have a few questions:

  • Regarding the weakness I mentioned, I would be happy to review my score with some additional explanation. In the paper you attribute this performance to the cost of using a single shared channel. However, if this were the case, one would expect that increasing d would help increase performance, but Figure 4 does not suggest this. Is there a particular difference between the baseline models and this method which would warrant worse performance? (This discussion could be added to appendix E for the sake of space and referenced in the main text).
  • You mention that for specific baselines, you had to change the training hyperparameters for stability. Even so, the RelNet variance is very large, and clearly shows failed runs. Is this the case after tuning the hyperparameters, or does the plot include the previous version? Can you give some intuition as to why this model is so unstable to train?
1 comment

Thank you for your generous feedback and for your appreciation of our architectural motivation and ablation design.

Why does increasing dd not close the Sort-of-CLEVR gap? (W1, Q1) Please refer to our General Comment, which provides the detailed breakdown you requested. In short, the phase dimension dd governs how many senders a module can distinguish, rather than how much information each read carries. Whatever dd is set to, each module reads a single vector of dimension dm=4d_m = 4 per step, which explains why increasing dd beyond 4 yields no further gains. Regarding the architectural differences from the baselines, SyncNet's head module never accesses the image feature map directly, and each module summarises its assigned region via a weighted spatial average before communicating through narrow dmd_m-dimensional messages. By contrast, the strongest baselines attend directly to the unpooled spatial feature grid. As detailed in the General Comment, the subtype breakdown indicates that this bottleneck reduces fine geometric precision rather than the ability to aggregate across multiple objects, pointing to the read dimension dmd_m as the primary limiting factor. Following your suggestion, we will add this full analysis to Appendix E and reference it in the main text of the revised manuscript.

RelNet variance and training instability (Q2). Figure 2 reports only the tuned configuration, meaning that the failed runs visible in the plot are those that remain even after hyperparameter tuning. In those unsuccessful runs, training accuracy stays at chance level and parameter gradients remain near zero throughout optimisation, which points to an optimisation failure rather than a capacity limitation. On SQOOP's 8×88 \times 8 feature grid, RelNet aggregates over all 4,096 pairs of spatial cells equally, even though the correct answer depends on only a small subset of locations. Early in training, the informative gradient signal from the relevant object pair is heavily diluted by the remaining background pairs. Extending the training schedule and adjusting the learning rate help the model escape this plateau in most seeds, though not in every run.

One plausible explanation for why this pathology is much more severe on SQOOP than on Sort-of-CLEVR, although we have not tested it directly, lies in how objects are identified. In SQOOP, all objects share the same colour and must be distinguished by shape. To isolate the relevant pair of cells, the relational MLP must already recognise object shapes, yet the convolutional encoder only receives strong gradients to learn shape representations once the relational module attends to the right pair. Adjusting the grid resolution offers no easy escape: a coarser grid blurs the character shapes further, whereas a finer grid quadratically increases the number of cell pairs. On Sort-of-CLEVR, which uses a 5×55 \times 5 grid with 625 pairs, RelNet trains reliably across seeds, likely because questions reference objects by distinct colours that are immediately linearly separable in the early convolutional features. Notably, when SyncNet is trained without its spatial preference, it suffers from the exact same failure mode on SQOOP. We will include this discussion in the revised paper to give readers clearer intuition for the baseline variance.

We hope these explanations clearly address both of your questions.

Reviewer_9J8aOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the binding problem in compositional generalization using modular neural networks. A central challenge is determining which modules should communicate and how information should pass between them. The authors introduce SyncNet, a neuroscience-inspired architecture that uses synchrony to coordinate communication. SyncNet employs Kuramoto dynamics to evolve phases and phase alignment to regulate communication through a shared channel. The authors evaluate the architecture on Sort-of-CLEVR for relational reasoning and SQOOP for systematic generalization, reporting competitive performance.

Critical strengths

This paper is clearly and well written. Applying synchrony to solve the binding problem is an interesting idea. This paper does systematic ablation studies to examine the criticial components of a new architecture that contributes to binding.

Critical weaknesses

The description of the model is insufficient for replication. It is unclear how synnet is trained. The paper could provide a bit more detail on this.

The paper would benefit from a more precise explanation of which properties of SyncNet account for its performance on SQOOP and Sort-of-CLEVR. While the ablations are useful, the connection between the synchrony-based binding mechanism and the reported gains in relational reasoning and systematic generalization is a bit unclear.

Questions for authors

Questions For Authors

How is SyncNet trained? Which components and parameters are learned and what stopping criterion determines when training is complete?

How comparable are the training conditions across models? Do SyncNet and the baselines receive the same training data?

How does SyncNet relate to symbolic and nonparametric approaches that learn reusable, compositional representations, e.g. Wu et al. 2022, 2025?

Which architectural property explains the reported advantages? What evidence distinguishes the contribution of synchrony-based binding from that of modularity or other architectural choices?

How do the performance of SyncNet depend on the number of entities to bind?

[1] Wu et al. Learning Structure from the Ground up—Hierarchical Representation Learning by Chunking. NeurIPS, 2022.

[2] Wu et al. Building, Reusing, and Generalizing Abstract Representations from Concrete Sequences. ICLR, 2025.

1 comment

Thank you for your constructive comments and for taking the time to review our work.

How is SyncNet trained? (W1, Q1) SyncNet is trained end to end by backpropagating through the unrolled dynamics, using a standard cross-entropy loss on the answer read out by the head module. There is no pretraining and no auxiliary loss. All model parameters are learnt jointly, except for the integration step sizes and the architectural dimensions listed in Table 12, which remain fixed. Training runs for a fixed budget of 100,000 gradient steps using the optimiser hyperparameters detailed in Table 13. We do not use early stopping. Instead, we select the checkpoint that achieves the highest validation accuracy and evaluate it once on the held-out test set. To ensure full reproducibility, we will make this training summary much more prominent in the main text of the revised manuscript, alongside explicit pointers to Tables 12 and 13.

Do all models receive the same training data? (Q2) Yes. Every model is trained on the exact same 1.08M-example dataset, using identical data splits, random seeds, and loss functions. The held-out pairings are never used for model selection or hyperparameter tuning. The only exceptions to the shared training protocol are the optimisation settings for RelNet and Conv+LSTM, which are documented in Table 14. Under the default settings, those two baselines frequently remained stuck at chance accuracy and failed to initiate learning. Providing a longer training schedule and adjusted learning rates allowed them to converge reliably. Consequently, those baselines actually observe more training examples during optimisation than SyncNet, rather than fewer.

Relation to Wu et al. (2022, 2025) (Q3). Thank you for pointing us to these works. As we understand them, HCM and HVM learn explicit, reusable symbolic units, namely discrete chunks and abstract variables derived from them, which allows compositional reuse to be measured directly. By contrast, SyncNet relies on continuous, parametric modules trained end to end, meaning that any functional specialisation must emerge implicitly during optimisation. This distinction explains why our override experiments can demonstrate that the learnt modules are not interchangeable, even though we cannot directly read out the exact symbolic function each module computes. We view these two paradigms as complementary, and we will expand our related work section in the revised manuscript to cite and discuss both papers.

Which architectural property explains the gains? (W2, Q4) Our experiments indicate that the primary driver of performance is a stable modular decomposition, supported by four main empirical findings. First, removing the spatial preference prevents any consistent decomposition from forming, which causes SQOOP accuracy to drop to chance across all seeds. Second, forcing modules to share a single recurrent cell reduces accuracy by 5.9 percentage points. Third, altering the specific addressing mechanism or the message aggregation rule shifts accuracy by only a few points, with no consistent ranking across tasks. Fourth, applying the shuffle-binding override at test time degrades accuracy as severely as removing inter-module addressing altogether.

Regarding the specific role of synchrony, it performs two distinct functions in SyncNet: grouping the spatial feature map into modules and regulating communication between them. Our communication ablations evaluate the second function and demonstrate that phase-based addressing is not strictly necessary once modules are formed, as content-based addressing achieves comparable accuracy (please also see our reply to Reviewer 6p6Q). However, our current ablations do not isolate the role of synchrony in the initial grouping stage, because all tested communication variants retain the oscillator field and phase-competitive binding. We will clarify this distinction in the revised text.

Dependence on the number of entities (Q5). We kept the total number of objects per scene fixed in our experiments, because varying object counts alongside pair variety, random seeds, and architectural ablations would have exceeded our computational budget. Nevertheless, the question subtypes in Sort-of-CLEVR naturally vary in the number of objects that must be reasoned over jointly, and the accuracy gap relative to the strongest baseline grows with relational arity. Specifically, the gap is 0.1 percentage points for unary questions involving one object, 1.9 points for binary questions involving two objects, and 8.7 points for ternary questions involving three objects (please see our General Comment for a detailed analysis of this ternary gap). Furthermore, the required phase dimension dd scales with the number of interacting entities. Ternary questions on Sort-of-CLEVR require d≥4d \geq 4 to reach peak performance, whereas SQOOP, which evaluates pairwise relations between two objects, saturates at d=2d = 2.

Reviewer_6p6QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces SyncNet, a modular neural architecture that draws on the neuroscience concept of communication through coherence to organise inter-module communication. The key design separates each module's content state from a dynamical phase state: phases evolve under learnt spherical Kuramoto dynamics, and phase alignment determines the strength of communication through a single shared channel. An oscillator field first groups spatial features into modules via phase competition, after which modules exchange information over multiple internal steps while their phases co-evolve, with content states modulating the coupling between oscillators. A separate head module, which has no direct access to the image, reads the final answer from the shared channel.

SyncNet is evaluated on two diagnostic tasks: Sort-of-CLEVR for relational reasoning and SQOOP for systematic generalisation, where lower pair variety demands greater recombination at test time. On SQOOP, SyncNet achieves the best held-out accuracy among compared models at the three lowest pair-variety settings, with a 5.1 percentage-point margin at the hardest setting. On Sort-of-CLEVR, it is competitive but below the strongest attention baselines, particularly on ternary questions. A central and honest finding emerges from ablations and test-time overrides: while the trained model depends critically on evolving phases (freezing them costs as much as removing addressing entirely), a content-addressed variant trained from scratch matches SyncNet's accuracy. The authors conclude that the benefit stems from the modular decomposition itself rather than from synchrony as the specific addressing mechanism.

Critical strengths
  • Intellectually honest and well-structured investigation. The paper's most notable strength is its willingness to test and partially refute its own central hypothesis. Rather than presenting synchrony as the key ingredient and stopping there, the authors systematically ablate the addressing mechanism (phase vs. content, linear vs. softmax) and apply test-time overrides to a trained model. The conclusion—that modular decomposition matters more than the specific routing mechanism—is nuanced, well-supported, and more informative than a simple "our method wins" narrative. This kind of negative-result transparency is rare and valuable for the community.
  • Thorough and careful experimental protocol. All models share the same convolutional encoder, optimisation settings are held fixed across models wherever possible (with deviations explicitly recorded), and results are reported over five seeds with standard deviations. The SQOOP evaluation uses held-out pairings never seen during training or validation, which is the correct protocol for measuring systematic generalisation. The test-time override experiments (Freeze, Zero, Shuffle-phase, Shuffle-binding) are particularly well designed: they probe what a trained model actually relies on without the confound of retraining, and the distinction between shuffle-phase and shuffle-binding elegantly separates the role of phase assignment from module specialisation.
  • Clear and well-motivated architectural design. The separation of content and phase, the use of a single shared channel with phase-gated communication, and the content-modulated Kuramoto coupling are all clearly described and well motivated by both the binding problem literature and the communication-through-coherence hypothesis. The connection to linear attention (Eq. 12 as a symmetric form with phase as both query and key) is a useful observation that grounds the design in a familiar framework.
Critical weaknesses
  • Limited task scope and no evidence of scalability. The evaluation is restricted to two small synthetic benchmarks (Sort-of-CLEVR and SQOOP) with simple visual scenes and limited relational complexity. While the authors correctly note these are diagnostic probes chosen for the control they afford, the paper provides no evidence—preliminary or otherwise—that the architecture's inductive biases transfer to more complex or realistic settings. The Sort-of-CLEVR results are already below FiLM, Transformer, and Shared Workspace on ternary questions, and the gap is concentrated in the hardest subtype (Count obtuse). This raises the question of whether the single shared channel becomes a bottleneck as relational complexity grows, a concern the authors acknowledge but do not address empirically beyond the existing sweep.
  • The central contribution is undermined by the paper's own findings. The title and framing position synchrony as the mechanism of interest, yet the ablations show that a content-addressed variant trained from scratch matches SyncNet (72.5% vs. 72.9% on SQOOP pair variety 1), and the communication variants are all within noise of one another. The paper's honest conclusion is that "the benefit appears to come from the modular decomposition rather than synchrony itself" (Section 1, contribution 4). While this is a scientifically valuable finding, it weakens the paper's case as a contribution to synchrony-based computation specifically. The architecture is considerably more complex than the content-addressed baseline (Kuramoto dynamics, oscillator field, binding iterations), yet offers no measurable advantage over it. A reader may reasonably ask: what does the Kuramoto machinery buy us over simpler content-based routing?
  • Insufficient analysis of what the modules actually learn. The overrides show that modules are not interchangeable (shuffle-binding is costly), which is evidence of specialisation. However, the paper does not analyse what each module computes, whether the learnt roles correspond to meaningful sub-functions of the task, or whether those roles are reusable across tasks. The authors acknowledge this ("whether the roles acquired are reusable across tasks is untested"), but given that functional modularity is the paper's stated motivation, the absence of any functional analysis—beyond the interchangeability test—is a notable gap. For example, visualising which spatial regions each module claims across different inputs, or probing whether module representations encode specific relational primitives, would substantially strengthen the modularity claims.
Questions for authors
  • What does the Kuramoto dynamics contribute beyond content-based routing? The content-addressed variant matches SyncNet on both tasks. Given that the Kuramoto machinery (oscillator field, binding iterations, phase dynamics with content-modulated coupling) adds substantial architectural complexity, can you characterise any qualitative difference in the solutions learned by the two variants? For example, do the phase-addressed models converge faster, show lower seed variance, or produce more interpretable module assignments? If not, what is the practical motivation for preferring synchrony over content-based addressing?
  • Bottleneck analysis of the shared channel. The single shared channel with phase-gated communication is a deliberate design choice, but the ternary gap on Sort-of-CLEVR suggests it may become limiting when more objects must be combined simultaneously. Have you analysed how the channel's information throughput scales with the number of modules and the phase dimension? Specifically, at the Count obtuse subtype where the gap is largest, does the channel fail to transmit sufficient information, or does the failure lie in the module-level computation?
  • Functional characterisation of modules. The shuffle-binding override shows modules are not interchangeable, but this does not reveal what they compute. Can you provide any analysis of the functional roles of individual modules—for example, by examining which spatial regions they consistently claim, whether their content states encode specific relational information (e.g., "left-of" vs. "above"), or whether their roles transfer when the model is evaluated on a different relation type than it was trained on?
1 comment

Thank you for your constructive comments and your time to review our work.

What does the Kuramoto dynamics contribute beyond content-based routing? (W2, Q1) Synchrony performs two distinct functions in SyncNet: it groups the feature map into modules, and it controls communication between them. Our ablations only test the second function. We will make this distinction much clearer in the text, since every communication variant retains the oscillator field and phase-competitive binding. We also wish to offer one small correction to the comparison in the review. The 72.5% figure corresponds to the content + softmax variant, which also replaces the shared channel with direct pairwise attention. The closest baseline is the content + linear variant, which alters less than 2% of the parameters. On SQOOP at pair variety 1, this variant reaches 70.2±1.5%70.2 \pm 1.5\%, compared to 72.9±2.1%72.9 \pm 2.1\% for SyncNet. Across five seeds, this slight lead is not sufficient to claim a statistically meaningful advantage, nor is phase-based addressing more stable across seeds. We have not systematically evaluated convergence speed or the interpretability of module assignments, so we do not claim an advantage on those fronts either.

Inter-module communication was the primary role for which we introduced synchrony, and evaluating it is the central goal of this study. From a purely practical standpoint, we do not argue for preferring Kuramoto dynamics over the content + linear variant, which is architecturally simpler and achieves comparable accuracy once the phase-based grouping is in place. Instead, the value of the Kuramoto machinery here is analytical. Implementing the full phase-gated routing mechanism and comparing it against matched content-addressed baselines is what allows us to disentangle the effect of modular grouping from the effect of phase-based routing. We agree with your assessment in W2 that our current framing places too much emphasis on synchrony as the primary driver of performance. In the revised manuscript, we will narrow and reframe the introduction so that our narrative aligns with our empirical findings from the outset. This revision will position SyncNet as a controlled testbed that isolates modular decomposition and the shared channel, rather than phase-based addressing itself, as the primary source of systematic generalisation.

Bottleneck analysis of the shared channel (W1, Q2). Please refer to our General Comment for a detailed breakdown of the Sort-of-CLEVR subtypes. To answer your question directly, we believe the shared channel does limit accuracy on subtypes such as Count obtuse, but this limitation stems from the dimensionality of each read (dmd_m) rather than the number of objects being combined. Regarding scaling, varying the number of modules makes little difference up to M=8M=8 on either benchmark. Beyond that point, accuracy on SQOOP declines, which is consistent with a larger number of modules competing within a phase space of fixed dimension.

Functional characterisation of modules (W3, Q3). We agree that a deeper functional analysis is the main remaining gap. Our current experiments establish three core properties. First, modules are not interchangeable, as applying the shuffle-binding override reduces accuracy by 31.9 and 23.2 percentage points on the two tasks. Second, assigning each module its own recurrent cell is beneficial on SQOOP, whereas sharing recurrent parameters across modules reduces accuracy by 5.9 points. Third, modules do not need to map one-to-one onto objects, since accuracy remains flat from M=4M=4 to M=8M=8 on scenes containing six objects. Probing the content states for specific relational primitives or testing whether learnt roles transfer across relation types would require new training runs and evaluation protocols that we could not complete during the rebuttal period. We therefore keep our claims about what individual modules compute strictly bounded, and we will expand this discussion in the limitations section of the revised paper

Task scope (W1). We agree with this limitation, and we explicitly state in Section 6 that we make no claims regarding scalability to larger, more realistic settings. We selected Sort-of-CLEVR and SQOOP specifically as controlled diagnostic benchmarks. In particular, SQOOP allows us to vary pair variety systematically to isolate compositional generalisation, a property upon which our ablation and override analyses directly depend. Extending this framework to more complex visual scenes is a natural direction for future work, which will likely require widening the read dimension (dmd_m) to preserve fine geometric detail and pairing the modular decomposition with the simpler linear content-based addressing identified in our ablations

Presentation. This is a fair critique. Section 3 is dense, so we will add a notation summary table and a step-by-step walk-through of Figure 1 in the revised manuscript to make the architecture.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and Area Chair

Thank you, Area Chair and reviewers, for your time and comments. One point came up in more than one review, so we answer it once here and refer back to it in our individual replies.

Where the Sort-of-CLEVR gap comes from (Reviewers 6p6Q, j8LY). The phase dimension dd controls how many senders a module can tell apart, not how much information it receives. Whatever dd is, each module reads a single dm=4d_m = 4 dimensional vector per step: a weighted sum of the other modules' messages, with the weights set by how well their phases align (Eq. 12). A larger dd lets those weights separate more senders cleanly, but it doesn't make the message any wider. That's why increasing dd only helps up to a point: ternary accuracy rises from 56.3% at d=2d=2 to 79.1% at d=4d=4 and then stays flat, which suggests the limit lies elsewhere. Unlike the baselines, SyncNet's head never sees the image, so everything it knows has to arrive through the shared channel. Each read is also a single weighted sum, so a module gets one blended summary per step, and each module sees its region only as a weighted average of the features in it (Eq. 8). How messages are combined isn't the cause: switching to softmax gives 77.6% on ternary questions, against 78.9%.

The subtype results suggest the limit is likely the size of each read, dmd_m, which we kept fixed at 4 and didn't vary in this work. Count obtuse is the hardest subtype for every model, so the clearer evidence comes from subtypes the best baseline nearly solves: Count in box (88.9% vs. 97.3%) and Closest shape (94.4% vs. 99.0%). Like Count obtuse, these need precise geometry between specific objects, which is hard to pass on in one small vector per step. By contrast, Count same shape only needs a simple aggregate over all six objects, and SyncNet gets 99.9% on it. So the gap looks like a loss of geometric precision rather than of the ability to combine many objects.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.