acceptodds
Withdrawn from ICLR 2027

Modular Equilibrium: Structure-Aware Solving for Graph Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Iteration-based implicit graph solvers—deep equilibrium models and their graph variants—claim to compute an equilibrium, but the equilibrium is nominal: the forward pass stops far from its tolerance, yet forcing full convergence changes test performance not at all. We show that the "infinite depth" of these models is carried entirely by cycles, and propose modular equilibrium, a fixed-point graph layer whose solver is organized around exactly that structure: decompose the graph into cyclic modules—strongly connected components for directed learned graphs, and bridge (2-edge-connected) decomposition for undirected ones, which we show is the correct analogue since SCCs degenerate on bidirected graphs—solve the fixed point within each module with a bounded budget, and propagate between modules by bounded feed-forward sweeps along the module graph, with gradients exact for the unrolled computation: bounded where computation is bounded in practice, iterative exactly where feedback lives. We prove that modular propagation is exact in the converged limit—it reaches the global fixed point itself (block back-substitution), not an approximation of it—and that its budget follows structural complexity: iterations are spent inside dense cyclic modules, whose truncation tail scales as 1/(1-r), and saved on the structurally simple inter-module links. On complex superpixel graphs, a depth-tuned, parameter-matched local baseline only ties global equilibrium solving, while modular solving exceeds both; on small molecular graphs, where a few iterations nearly suffice, it matches global solving at the default coupling and degrades more gracefully as coupling strengthens, while enabling what global solvers cannot express: causal module-importance analysis, which identifies the molecules' macrocyclic rings as the load-bearing units. The advantage tracks the predicted boundary conditions—it vanishes when structural encodings already supply the cyclic structure—and random partitions of the same size distribution destroy the benefit.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph neural networks, deep equilibrium models, fixed-point solvers, graph decomposition, model interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_bjkHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper seems to propose a novel approach to graph neural network construction where sub-structures are treated using deep equilibrium networks. They benchmark their approach by comparing to deep equilibrium GNNs.

Critical strengths

The idea to combine structure decomposition with deep equilibrium networks is, to my understanding, novel.

Critical weaknesses

I found it difficult to understand the content of this paper, largely due to what I believe are severe organizational difficulties. While I recognized many of the keywords I often struggled to piece together a single coherent message in each section.

Moreover, insofar as I understand it, I have concerns about the applicability of the method. Just to name one potential issue: the proposed decomposition into strongly connected components and the rest of the graph may not be applicable for many graphs. Molecular graphs, which seem to be a core focus of the paper, can have very few strongly connected components. (For that matter, in Figure 1 the module is not a strongly connected component.)

I am sorry I am not able to give a more comprehensive analysis: I found it truly difficult to understand the central points of the paper due to presentational difficulties. If the other reviewers disagree to the point where I am convinced this is a "me" problem, I will revisit my score. Otherwise, I recommend resubmitting to a later conference following revision with a focus of clarity of explanation.

Questions for authors

No questions.

Reviewer_uVbKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper partitions a graph into modules and approximates a graph equilibrium through intra-module iterations and inter-module communication. It uses strongly connected components for directed graphs and bridge blocks for undirected graphs, with gradients obtained by unrolling a fixed computation budget.

Critical strengths
  1. The method is straightforward to understand and implement. The authors also explicitly report the additional runtime and memory costs of unrolling.

  2. The paper includes useful negative results: the original partition can degenerate, global solving performs better under weak coupling, and the modular advantage disappears under stronger configurations. These results help clarify the method’s scope.

Critical weaknesses
  1. I am not convinced by the methodological novelty. Separating within-module computation from between-module communication is an established idea in block iterative methods and domain decomposition. The paper itself describes its update as damped block Jacobi. SCCs, bridge blocks, and finite unrolling provide a particular implementation, but the additional algorithmic contribution remains unclear. The experiments do not fully resolve this concern.

  2. The mathematical justification needs correction. Proposition 3 uses the spectral radius to bound the Neumann-series truncation error for a general matrix. However, a nilpotent matrix can have zero spectral radius while retaining a nonzero finite-step truncation error, contradicting the stated bound. The appeal to Gelfand’s formula in Appendix A.8 does not justify this step. The symmetry assumptions in Proposition 6 also do not automatically apply to the learned asymmetric QK operator. These issues affect the main argument for allocating computation between modules.

Questions for authors
  1. What is the specific algorithmic advance over fixed-point iteration with graph-based blocks? Can it be demonstrated against an appropriate block-solver baseline?

  2. How should Proposition 3 and Appendix A.8 be corrected, and would the revised result apply to the asymmetric operator used in the model?

  3. Does the modular advantage remain when the global solver also uses unrolled gradients and receives comparable compute or training time?

Reviewer_56iAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a modular solver for implicit graph neural networks. It partitions directed graphs into strongly connected components and undirected graphs into bridge-connected blocks. Each sweep exchanges messages between modules, then performs several internal updates while holding the external messages fixed. The bounded computation is trained through unrolled differentiation. Under a contraction assumption, the schedule converges to the same fixed point as global iteration; exact local solves also permit a topological pass when the module graph is a DAG. Experiments on PascalVOC-SP and peptides-func compare modular and global solving, examine alternative partitions and coupling strengths, and analyze interventions on modules.

Critical strengths
  • The paper uses topology-derived modules as blocks in a batched iterative solver, which is an interesting direction and remains underexplored.
  • Informative ablations and transparent reporting of limitations. The global and modular models share the same topology construction and parameter count, and the random-partition ablation provides evidence that partition choice matters in the tested setting. The manuscript also reports increased runtime, unsuccessful module dropout, and settings where the advantage disappears or reverses.
Critical weaknesses
  • The experimental evidence is insufficient in dataset coverage and baselines. The main evaluation focuses on only two LRGB datasets. Stronger comparisons should include representative implicit GNNs, accelerated global solvers, global iteration with unrolled gradients, and competitive message-passing or graph-transformer models under comparable protocols.
  • The proposed model changes both the gradient method and compute budget, costing approximately 2.0–2.3 times more training time. However, the gain is relatively small (the reported gains lie within the three-seed standard deviations, table 1), which cannot justify the additional computation cost.
  • Modules are precomputed and fixed, while learned QK edges can create new cycles between modules and bypass the original bridge boundaries. Thus, the implementation does not align with motivation.
  • The authors should clarify when QK edges are reconstructed. If QK attention is recomputed from the evolving state zz, the row-sum bound alone also fails to establish contraction.
Questions for authors

Please address the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.