acceptodds
Under review as a conference paper at ICLR 2027

Confluence: WHEN DOES CONSERVED-FLOW ROUTING HELP? A COUPLING CONDITION FOR MIXTURE-OF-EXPERTS, WITH A LEGALITY GUARANTEE

Abstract

Mixture-of-Experts (MoE) models route each input through a few experts cho- sen by a learned gate, one layer at a time and independently. We ask when it is useful to instead use a coupled router, which commits to a single, globally con- sistent path of experts. We study a conserved-flow router: a differentiable Tero– Nakagaki (Physarum) dynamic that relaxes to one min-cost path through a graph whose edges are exactly the legal expert transitions, so illegal routing is impossi- ble by construction rather than merely penalized. Our central finding is a boundary condition. When a task’s output marginalizes over its intermediate steps, mean- ing each step can be scored independently, the coupling provides no benefit: on MetaQA 2-hop knowledge-graph QA the flow router ties a plain per-step softmax over the same legal graph. As the steps become mutually constraining, the flow’s advantage over independent gating grows monotonically, a crossover we chart on a controlled constraint-density sweep. In a small MoE transformer on a coupled compositional task, the flow router cuts the illegal-transition rate from 0.34 to 0.01 at one dominant expert per layer; its accuracy reaches 0.99 on some seeds but is unstable across seeds and below a top-1 gate on average, which we report seed by seed. To make the approach practical, we use implicit differentiation through the flow fixed point and a matrix-free solver, and verify the resulting gradients against finite differences. The contribution is the routing mechanism, a specific condition for when it helps, and a hard legality guarantee, rather than a new accuracy record.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
mixture of experts, conditional computation, routing, knowledge graph question answering, multi-hop reasoning, retrieval-augmented generation, constrained optimization, differentiable optimizationA gateless mixture-of-experts that routes by conserved flow over a graph of legal transitions, so the chosen experts always form a valid reasoning path.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_BXYwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a conserved-flow router for MoE models that selects a globally consistent path over a graph of legal expert transitions. Its main claim is that coupled routing helps when routing decisions across steps mutually constrain one another, but offers little benefit when steps are effectively separable. The claim is evaluated using synthetic coupling sweeps, MetaQA 2-hop, and small MoE experiments.

Critical strengths
  1. interesting research question. The paper asks when globally coupled routing is useful. The coupling-vs.-separability framing is intuitive and potentially general.
Critical weaknesses
  1. Insufficient experimental support. Positive evidence for the central coupling claim is largely synthetic. The only real-data experiment is a regime where no improvement is expected, while the MoE transformer is unstable and underperforms top-1 routing on average. The key claim therefore lacks convincing real-world positive validation. Specifically, the main coupling result comes from a synthetic constraint-density sweep with only three seeds, where the flow advantage increases as the transition graph becomes more constrained. On the only real-data benchmark, MetaQA 2-hop, conserved flow does not improve over the matched no-flow baseline (raw Hits@1: 0.864 vs. 0.885; degree-normalized Hits@1: 0.948 vs. 0.944).

  2. Limited architectural novelty and difficult presentation. Expert-path constraints, globally structured routing, differentiable optimization layers, and differentiable Physarum solvers have substantial prior precedent, making the specific architectural contribution relatively narrow. The paper is also overly compressed: the method, assumptions, and distinctions between claims are difficult to follow. Closely related ingredients appear in Path-Constrained Mixture-of-Experts (2026) for constrained expert paths, Chain-of-Experts (2025) for sequential expert composition, OptNet: Differentiable Optimization as a Layer in Neural Networks (2017) for differentiable optimization layers, Deep Equilibrium Models (2019) for implicit differentiation through fixed points, and Physarum Powered Differentiable Linear Programming Layers and Applications (2021) for differentiable Physarum optimization. The paper itself acknowledges that the flow dynamic is not new and narrows its contribution to its use as an expert router, the legality guarantee, and the coupling condition. Moreover, key distinctions are easy to miss—for example, legality is provided by the graph topology rather than the flow mechanism itself, as the no-flow MetaQA baseline enjoys the same zero-hallucination guarantee.

Path-Constrained Mixture-of-Experts, 2026

Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models, 2025

Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation, 2021

OptNet: Differentiable Optimization as a Layer in Neural Networks, 2017

Differentiable Dynamic Programming for Structured Prediction and Attention, 2018

Differentiation of Blackbox Combinatorial Solvers, 2020

Physarum Powered Differentiable Linear Programming Layers and Applications, 2020

Questions for authors

1.What is the precise novelty of conserved-flow routing relative to existing globally structured/path-constrained routing and differentiable path solvers?

2.Can the authors provide evidence on a real-world task with genuinely coupled routing decisions where the proposed method improves predictive performance?

3.Is the stated “coupling condition” intended as a formal necessary/sufficient condition or only an empirical pattern observed in the current experiments?

Reviewer_wVVLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when globally coupled routing is useful in Mixture-of-Experts models. It proposes a conserved-flow router based on Physarum dynamics that selects a globally consistent expert path over a graph of legal expert transitions, so illegal transitions are excluded by construction. The paper also uses implicit differentiation and a matrix-free solver to train through the flow fixed point efficiently. Its main claim is that coupled routing is useful when routing decisions across steps constrain each other, while independent per-step routing is sufficient when the steps are marginalizable. The authors validate this claim with a controlled constraint-density sweep, MetaQA 2-hop, constrained MoE routing tasks, and a small MoE transformer. The results show clear gains in coupled synthetic settings, a tie on MetaQA as predicted, and substantially lower illegal-transition rates, although the transformer results are less stable across seeds. 52282_Confluence_WHEN_DOES_CON

Critical strengths

Strength 1: The paper gives a clear routing formulation that enforces legal expert transitions by construction.

Strength 2: The coupling condition provides a simple and useful view of when global routing can help.

Strength 3: The implicit differentiation and matrix-free solver make the proposed flow router practical to train at larger graph sizes.

Critical weaknesses

Weakness 1: The method forces routing into a single continuous expert path, which is a strong structural assumption. Expert dependencies in MoE models may not follow a single path. Multiple experts may contribute in parallel, and useful dependencies may span several layers instead of only adjacent transitions.

Weakness 2: Although the paper emphasizes global coupling, the path objective is still additive over edge costs. This mainly captures local transition preferences between adjacent experts. More complex interactions, where the value of a transition depends on several earlier expert choices, are not directly represented. The conservation constraint couples the route globally, but the utility assigned to the route is still built from local additive costs.

Weakness 3: There is a mismatch between the continuous flow used during training and the discrete path used for routing. The model is optimized through a continuous fixed-point solution, while the final decision is a hard expert path. When several paths have similar costs, a small change in the continuous solution can lead to a different discrete route.

Questions for authors

Question 1: How does the method handle cases where useful expert dependencies do not follow a single path, such as parallel contributions or dependencies that span non-adjacent layers?

Question 2: Since the route score is additive over edge costs, how can the method represent higher-order dependencies where the value of one transition depends on several earlier expert choices?

Question 3: Under what conditions does optimizing the continuous flow fixed point produce the same routing decision as optimizing the final discrete expert path?

Reviewer_iKq2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

As far as I can tell, the entire text of this paper was AI generated and not readable by a human. It is filled with jargon that is not encountered in technical writing on this subject ("legal expert transitions", "differentiable dynamic", "controlled constraint-density sweep", etc.) and are not defined within the text. Without a rewrite of the paper for human readability, it is difficult to judge whether the core ideas hold scientific merit.

Critical strengths

None

Critical weaknesses

See above

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.