acceptodds
Under review as a conference paper at ICLR 2027

UNIFYING DYNAMICAL SYSTEMS AND GRAPH THEORY TO MECHANISTICALLY UNDERSTAND COMPUTATION IN NEURAL NETWORKS

Abstract

Understanding how biological and artificial neural networks implement computation from connectivity is a central problem in neuroscience and machine learning. In neural systems, structural and functional connectivity are known to diverge, motivating approaches that move beyond direct connections alone. Here, we show that the function of recurrent neural networks (RNNs) trained on hierarchically modular tasks, a maze navigation task, and XOR can be recovered by modelling the network as a graph and analysing the multi-hop pathways between input and output units. In particular, decomposing these pathways by hop length reveals how the network temporally routes information. This perspective reframes regularisation: if function is implemented through multi-hop communication, then standard penalties such as L1 regularisation, which act only on individual weights, constrain single-hop structure rather than the multi-hop pathways that support computation. Motivated by this view, we introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and thereby induce temporal sparsity beyond that achieved by standard L1 regularisation. R-RNNs achieve improved performance by inducing temporal sparsity that matches the task structure, even when the task signal is sparse. Moreover, R-RNNs exhibit stronger sparsity–function alignment, reflected in their increased robustness under strong regularisation. Together, our results identify multi-hop communication as a key principle linking structure to function in recurrent networks, and suggest that sparsity should be defined over functional pathways rather than individual parameters.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
RNNs, Graph theory, Communication, Interpretability, Regularisation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eknLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper treats recurrent neural networks as weighted directed graphs and characterizes multi-hop input-output information propagation through weighted sums of matrix powers. The sum over truncated paths is used to generalize spatial structures, while the number of hops links the propagation path to the time at which the input appears. Experiments covered modular arithmetic tasks, alternating signal-and-noise inputs, target-delay-selection maze tasks, and time-series XOR. The authors further propose R-RNN, which imposes a penalty on the aggregated input-output paths rather than penalizing individual weights alone. The paper reports that multi-hop influence matrices reflect task-relevant structures more clearly than the original recurrent weights. The maze experiments provide further validation of this application. I interpret this method as a strategy for generalizing and regularizing information propagation based on connectivity structures, which is not equivalent to the complete reconstruction of nonlinear dynamics dependent on specific inputs.

Critical strengths
  1. An easily interpretable link between graph paths and time calculations was established. Controlled tasks provide a clear information-propagation structure, facilitating comparison with multi-hop metrics. The decomposition by hop count is particularly valuable because it links the network’s structural properties to the temporal relationship between inputs and final outputs. Sections 4.3 and 5.1 also distinguish between information-propagation capacity and sensitivity to specific inputs.

  2. The analysis further yields implementable training methods. R-RNN links the proposed interpretations to practical regularization objectives. Appendix A.4 reports model selection based on the validation set as well as results from ten paired runs, supporting modest but consistent improvements across two modular tasks. This extends the contribution beyond mere visualization analysis, although the evidence remains limited to the shallow recurrent networks and controlled tasks studied in this paper.

Critical weaknesses
  1. Some statements regarding sparsity go beyond the conclusions supported by the defined metric. Section 3.3 describes the reduction in input-output interactions as ensuring that communication paths are truly pruned. However, since the metric first sums both positive and negative contributions and then takes the absolute value, a decrease in the metric may also result from mutual offsetting. While this does not negate the reported loss improvement, it does affect the interpretation of the regularization effect.

  2. The relationship between the analysis metric and the training objective needs to be clarified. Section 3.2 uses normalized weights and a sum truncated by task duration, while Section 3.3 instead uses unnormalized weights for regularization; Appendix A.1.2 further specifies that k = 10 is used for maze regularization. These settings may be reasonable, but providing explicit training formulas and a concise table of per-task settings would facilitate implementation.

Questions for authors
  1. Existing Implementation (Request for Clarification): Please write the finite sum formula used for the actual training penalty term, specify whether alpha is retained, and clarify whether the maze setting k = 10 refers to the maximum number of jumps or simply the tenth jump.
  2. Meaning of sparsity (request for clarification of wording): Does sparsity here refer to a small net effect of inputs and outputs after aggregating signed contributions? If so, please acknowledge the possibility of positive and negative offsetting in the discussion of “truly pruned paths” in Section 3.3, and ensure that the conclusions for nonlinear tasks are consistent with the existing explanation of information transmission capacity in Section 5.1.
Reviewer_EAA7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies RNNs through graph-based multi-hop communication. It uses a resolvent-based input-output map to characterize spatial and temporal routing structure, and further proposes R-RNNs that regularize this multi-hop representation rather than individual weights. Experiments on modular tasks, a maze task, and XOR evaluate both the interpretability of the proposed measure and its use as a regularizer.

Critical strengths

1 The central idea is simple and insightful. Viewing recurrent computation through multi-hop connectivity provides a useful perspective on the relationship between network structure and temporal information routing.

2 The empirical study is carefully designed. The paper considers analytically tractable modular tasks together with temporal, maze, and nonlinear XOR settings, and reports repeated-run results and additional regularization experiments.

Critical weaknesses

1 The paper sometimes gives the weight-only multi-hop measure a stronger mechanistic interpretation than its construction supports. For a tanh RNN, actual input-output sensitivity depends on state-dependent activation derivatives, whereas �0��0� and �1��1� depend only on connectivity. The current evidence therefore more directly supports connectivity-defined routing capacity than recovery of the full nonlinear computation. The paper itself notes that the hop analysis captures only linear input-output maps.

2 The R-RNN objective promotes sparsity of aggregate signed input-output influence, but not necessarily sparsity of the underlying individual paths. Each entry of �2��2�, and even of �3��3�, aggregates many signed walks before its magnitude is evaluated. The hop-wise results reduce the concern about cancellation across hop lengths, but do not establish path-level pruning.

Questions for authors

1 How should the mechanistic claims be interpreted relative to an activation-aware finite-horizon Jacobian of the tanh RNN? Which conclusions concern actual state-dependent computation and which concern connectivity-defined routing capacity?

2 The hop-wise analysis in Fig. 4 reduces the possibility of cancellation across hop lengths, but each �0��0� entry still aggregates many signed walks. Can the authors assess whether the observed sparsity reflects reduced walk-level contributions rather than within-hop cancellation, or clarify the intended meaning of “pathway sparsity/pruning”?

Reviewer_YePhOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Combining the weight matrices of a 1-hidden-layer Elman RNN to create a single adjacency matrix, this paper proposes the resolvent of the resulting graph as an object worthy of study. The authors examine the learned resolvent for several toy tasks and compare it against the known optimal solution, and they propose a regularisation method where large entries in the resolvent matrices are penalised.

Contributions

The paper proposes a shift towards analyzing and regularizing RNN internals via network graph connectivity (resolvent). It demonstrates a working regularisation scheme and shows that this encourages "sparsity through time". It argues for this direction of analysis using biological motivation.

Validation

The paper examines the resolvent for a range of synthetic toy tasks and a 1-layer vanilla RNN, demonstrating that the learned resolvent resembles the known optimal solution. It trains an RNN regularised by the sum of absolute resolvent entries, demonstrating small improvements on these synthetic tasks compared to L1 regularisation of the same RNNs.

Critical strengths
  • The core concept of analysing connectivity rather than weights is interesting, and I can see justification in some tasks for preferring temporal sparsity.
Critical weaknesses
  • The tasks themselves are poorly chosen. Given the choice to analyse an RNN, it is unclear to me why such a large portion of the paper (section 4.2, Figure 2) is dedicated to analysing synthetic tasks with absolutely no temporal / sequence structure. Even when temporal structure is added, the relationships are trivial (Fig 3a) or the interpretability of the resolvent is limited (Fig 3b,c). There are no real world tasks analysed.
  • The use of the resolvent (leaky cascade interpretation) is poorly motivated. Section 2 explicitly states graph measures are "meaningful when the system implements the propagation process they describe" - however, the analysed Elman RNN includes a tanh nonlinearity and as such does not implement the described propagation process. There is no discussion about the validity of this leaky cascade model, so as far as I can tell the utility of the resolvent method is demonstrated entirely by its correspondence with intuition with the experiments of 4.2, which I have issues with (above).
Questions for authors

(These questions directly reflect the critical weaknesses above)

  • Does the resolvent show interpretable behaviour on any real-world sequence tasks of interest?
  • Can you justify the use of the resolvent as a tool to analyse these networks, particularly in light of the fact that Eq. 5 does not reflect the actual computation of the RNN due to the tanh nonlinearity? (These questions could change my opinion of the paper, but are less important)
  • Can you demonstrate the efficacy of significantly larger R-RNNs, perhaps on more real-life tasks? How does the requirement to compute the large matrix powers impact the computation: does this not cause significantly slower training and perhaps stability issues? It seems currently the R-RNNs may have a hidden dimension of just 16, which I suspect is too small to surface scale or stability issues.
  • line 194 "one might expect WhhW_{hh} to be block structured" - surely since there is no temporal structure, there would be many valid structures for WhhW_{hh} including 0, identity (which seems to be what you observe) and in fact WhhW_{hh} is pretty irrelevant for the tasks of Section 4.2. I think the relevant object would instead be (assuming no nonlinearity) the WhoWihW_{ho}W_{ih} circuit: could you plot this instead and compare it to RioR_{io}? Additionally, can you include magnitudes in your colorbars in Fig 3?
  • How are the tasks in Section 4.3 trained: are there labels at every timestep?
  • Could you expand more on line 279, the difference between this method and the Jacobian? It seems to me that if the tanh nonlinearity is ignored (as assumed by the resolvent!), the Jacobian would actually be exactly equivalent to Eq. 5 aside from the alpha multipliers? (These questions are to improve the presentation of the paper, but are unlikely to change my rating)
  • What is λmax\lambda_{max}, first used in Eq. 3? I assume the maximum eigenvalue of WW, but this is never defined.
  • It appears WW is directed, in which case λmax\lambda_{max} is not necessarily real. How do you deal with that?
  • What are the sums over in lines 147, 148? I assume elements, but this is unclear.

Comments on the paper

1 comment

Dear Reviewers,

Please take a moment to review the authors’ rebuttal, assess whether your concerns have been adequately addressed, and actively participate in the discussion. Please update your assessment if needed. Thank you for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.