- Rating
- 1
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper tests whether biological fly-connectome wiring is beneficial when used as a fixed graph prior for a trainable neural controller. It studies male/female ventral-nerve-cord and brain graph variants, a family of degree-, flow-, module- and reciprocity-preserving null graphs, and two different input/output interfaces: a dense trainable interface and a biologically wired interface that reads sensory afferents and actuates motor-neuron groups. Under dense I/O, real wiring has no consistent benefit; under the biologically wired interface, preserving more native structure worsens imitation of a global expert, including across additional tasks and a hexapod check. Re-targeting and carefully controlled shortcut interventions implicate limited short-latency sensory-to-motor access rather than conventional global graph statistics. Teacher-locality/delay controls, frequency decomposition, a 27-variant accessibility–demand diagnostic and bounded-gain linearized propositions develop this explanation. The analysis is primarily about imitation loss under the specified interfaces, not successful autonomous locomotion.
Critical strengths
- Exceptionally discriminating graph controls for a counterintuitive negative result. The authors do not stop at comparing a connectome to an Erdős–Rényi graph: they use degree-preserving rewires, flow-block controls, modular and reciprocal nulls, sign/weight variants and interventions that specifically increase afferent-to-motor shortcuts while preserving important edge statistics (Secs. 3.2, 4.2–4.4; Table 2). On the MaleCNS VNC biological-interface task, the real graph reaches 0.206 validation MSE versus 0.176 for degree-preserving and 0.162 for ER (lower is better). Sham and matched swap controls make the shortcut explanation more credible than a mere topology/performance correlation.
- A multi-angle causal account that states its conditions and remaining residuals. The teacher-locality intervention finds that the gap is not simply explained by what the expert causally reads, while tolerated sensory delay reduces the penalty and its excess error lies predominantly below 50 Hz (Sec. 4.7). A predictive accessibility–demand statistic orders 27 variants with a reported Spearman correlation around 0.92 (Appendix E), and the paper explicitly limits the accompanying propositions to linearized bounded-gain students. Reproduction caveats concerning FlyGM and the lack of a closed-loop success claim are stated instead of hidden.
Critical weaknesses
- The downstream embodiment evidence is much weaker than the imitation evidence. The main result concerns validation action MSE against fixed teachers. Appendix B reports 100% falls for the tested graph-controller variants in a clean-command closed-loop flybody setting, versus substantially lower fall rates for the reference expert/MLP. Thus the measurements do not show that the compared biological or random graphs support viable locomotion, and an imitation-error ranking need not predict closed-loop control. This is a substantial boundary on the relevance of the result to embodied policy design.
- The explanation remains conditional on interface, optimization and architectural choices. The biologically wired read-in/read-out scheme, three message-passing updates per control step, recurrence, fixed motor readout groups and a global-distillation objective create a particular access bottleneck. Shortcut and teacher-delay interventions isolate meaningful contributions, but do not fully exclude learnability and temporal-compensation effects in a different controller. The theoretical propositions concern linearized, uncorrelated-input, bounded-gain students, not the trained nonlinear graph policies. The comparison with the positive published FlyGM result is necessarily qualified because its code and exact training setup were unavailable (Secs. 4.5, 6; Appendix F). Figure/presentation concern (secondary): Figure 1 squeezes four comparisons into very small panels; Fig. 2 has a crowded legend and compact axis/category labels; and Fig. 4 combines six plots with many similar solid/dashed curves. These layouts make the decisive null-model and shortcut comparisons laborious to verify without zooming. Enlarging panels, simplifying legends and separating key contrasts would materially improve readability, but figure quality alone is not proof of causal unsoundness.
Questions for authors
- Temporal compute versus structural access (assessment-critical). If message-passing depth per control step or the recurrent horizon is varied at matched computational budgets, does the connectome-to-null gap shrink as predicted by the sensory-access interpretation? Can the same ablation be performed when teacher targets are based only on the student’s available sensory channels? A robustness result would clarify whether the reported bottleneck is intrinsic to the graph or specific to the chosen interface dynamics.
- Relationship between imitation MSE and usable control (assessment-critical). Given the closed-loop failures documented in Appendix B, can the authors provide a behavioral evaluation that isolates whether the same wiring order persists among controllers able to execute nontrivial locomotion, or show why the reported imitation gaps are the right success measure for the claimed architectural-prior setting? The answer would materially affect the paper’s practical significance, without invalidating the within-setup imitation analysis.
5 comments
Response (2/4)
Q2 and Weakness 1. Imitation MSE versus usable control. We agree that this is the main boundary of the result. The revised abstract now ends: "Our claims concern imitation error: in closed loop, biologically wired controllers fall on the fly body whatever their graph, and on the hexapod RL fine-tuning leaves all graphs indistinguishable." We answer the question in three steps.
(a) The fly body cannot rank graphs in closed loop. We had reported closed-loop failure only for the connectome. DP fails equally: both fall in 100% of episodes after BC, after six DAgger rounds and after 40 PPO iterations (164k steps; returns 34 vs 38). For reference: expert 4% falls, MLP after BC+PPO 17%, dense-IO connectome GNN after BC+PPO 33%. A graph controller can therefore locomote on the fly body, but not with the bio-wired interface, and Appendix B now says so for both graphs.
(b) Behavioural evaluation among controllers that do locomote. On the hexapod, every graph variant walks after behaviour cloning, without falls under clean conditions, pushes, 1.5× body mass and 20 ms observation delay. The imitation ordering is reproduced there (connectome worst; p = 0.014 vs ER). Closed-loop speed error is saturated (0.004 for every graph), except under observation delay, where the connectome is slightly best (0.034 vs DP 0.042, ER 0.046; p = 0.025 vs ER, 0.16 vs DP).
(c) New: closed-loop RL with the same body and interface. We fine-tuned every BC controller with recurrent PPO (150 iterations, 614k steps; reward for tracking the commanded speed and yaw rate, −1 on a fall). Deterministic evaluation against the commanded velocity, mean over 3 seeds (Appendix L, Table 9):
| Controller | speed err BC → PPO | yaw err BC → PPO | reward BC → PPO |
|---|---|---|---|
| connectome | 0.095 → 0.056 | 0.138 → 0.199 | 0.942 → 1.003 |
| DP | 0.095 → 0.050 | 0.136 → 0.202 | 0.948 → 1.021 |
| ER | 0.095 → 0.038 | 0.134 → 0.228 | 0.955 → 0.959 |
Every graph controller gains about 5% in reward, none falls under any perturbation, and connectome, DP and ER are indistinguishable on every metric (p > 0.1). The connectome's small advantage under delay after BC does not survive fine-tuning.
Answer to Q2. Among controllers that execute nontrivial locomotion, the imitation ordering does not persist as a behavioural difference: after BC the graphs are equal (saturated), and after RL they are equal too. Importantly, no connectome benefit appears in closed loop either. The two settings therefore agree on what matters for the paper's question: the wiring is not a useful prior, whether training is by imitation or by reward. The revision states this, and also that the imitation penalty itself has no closed-loop counterpart on the hexapod.
Why imitation error is the right measure for the claim we make.
- It is the setting of the positive claim we test. The connectome advantage reported by FlyGM concerns gradient-based learning of a pretrained policy's behaviour; testing it requires the same objective.
- It is how large controllers are initialised. Our own fly pipeline, like most for high-dimensional bodies, is BC followed by RL. A prior that makes imitation harder gives RL a worse starting point; on the fly body, where RL from scratch is out of reach, that starting point decides the outcome.
- It allows causal inference. Imitation error has low seed variance (s.d. 0.005 against a 0.030 gap), which is what makes the sham-controlled interventions decisive. Closed-loop metrics are saturated at failure on the fly body and saturated at success on the hexapod, so they cannot resolve differences of this size.
We therefore claim what imitation error can support, a within-setup statement about the wiring as a prior for gradient-based imitation, and we report the closed-loop results in full alongside it.
Response (3/4)
Weakness 2. The explanation is conditional on interface, optimisation and architecture. We agree that every result holds under stated conditions, and the revision states them. We take each condition the reviewer lists in turn and give the evidence on how far the result depends on it.
-
The biologically wired interface. We see the interface dependence as the finding rather than a limitation. Under a dense interface the wiring makes no difference; under the interface by which the real VNC meets the body, it is harmful, and the interventions identify the feature of the wiring that causes the harm. The bio-wired interface is not an arbitrary choice: it encodes the anatomy (each leg's sensors enter that leg's sensory neurons; each leg's muscles are driven by that leg's motor neurons). The same interface rule gives the same ordering in two animals of different sex and on a second body.
-
"Three message-passing updates per control step." As corrected in part 1, the default is one; three is an ablation, and it leaves the gap (0.037 vs 0.030).
-
Recurrence. The GRU node update is one controller class. The same ordering appears with very different dynamics: a frozen rate model with a trained readout (T2 MSE: connectome 0.78 vs ER 0.65; T3 explained variance: 0.115 vs 0.161), a DMN-style rate model with trained neuron parameters (0.143 vs DP 0.189, p = 0.004), and new versions of both models tied by cell type (GNN gap 0.039; tied DMN 0.145 vs DP 0.200, p = 0.005). A four-times longer gradient horizon leaves the gap at 0.027.
-
Fixed motor readout groups. The weight-matched shortcut swaps keep the readout groups exactly as they are and still remove the penalty (0.171 vs sham 0.200). The readout groups alone therefore do not cause it; what matters is the paths from sensors to those groups.
-
Global-distillation objective. The objective was varied: local teacher (no cross-leg dependence, gap 0.030), global teacher restricted to accessible channels (0.028), smoothed expert (0.018), delayed teachers (0.023 / 0.012 / 0.012), an analytic tripod clock on the hexapod (p = 0.014), and recorded fly kinematics (T3). The connectome is worst in every case. Two further objectives are not distillation: an evolution strategy on the ball task, where the connectome module (0.04 ± 0.12) is indistinguishable from its rewired version (0.07 ± 0.04), and the new PPO fine-tuning on the hexapod, where all graphs are equal.
-
Learnability and temporal compensation. We tested these directly: 3× longer training (gap 0.035; relative to DP it grows from 17% to 23%), BPTT 128, three sub-updates per tick, double width, no descriptors, and RL fine-tuning. None closes the gap. We agree that a controller class outside those we test could compensate differently, and the Limitations say so.
-
Linearised propositions. We agree, and Appendix G scopes them explicitly to idealised linear students with uncorrelated inputs and bounded gain, assumptions the trained controllers do not satisfy. The propositions motivate the accessibility statistic; the statistic is then tested empirically on the trained nonlinear controllers, where it ranks 27 graph variants by imitation error (Spearman ρ = 0.92). No conclusion about trained policies rests on the propositions alone.
-
FlyGM comparison. We agree that it is necessarily qualified, because FlyGM's code is not public, and the paper says so in Sec. 4.5, the Limitations and Appendix H. The part of our argument that does not depend on implementation details is the null-model one. On FlyWire, DP underperforms the connectome in the reported direction (0.182 vs 0.170, p = 0.11), but ER matches it (0.168, p = 0.49). FlyWire has 29–43% afferent neurons with no output; DP rewiring keeps them output-less and scrambles the afferent→efferent flow, whereas ER does not. Unit-weight baselines, as in FlyGM, give the same ordering. A released implementation could change our numbers, but not this argument about the null model.
Response (1/4)
We sincerely thank the reviewer for a careful and accurate summary, for describing the graph controls as "exceptionally discriminating", and for two well-posed, assessment-critical questions. We have answered both with new experiments and address every weakness point by point, in four parts: (1) a factual correction and Q1; (2) Q2 and the embodiment evidence; (3) the conditions of the explanation; (4) figures and a summary of changes.
A factual correction first. The controller performs one message-passing update per control step, not three (Sec. 3.3: "one control step computes M = WH …"). Three updates per step is an ablation (Sec. 4.3, Appendix F), and it does not reduce the gap. The revision states the default in bold in Sec. 3.3 so it cannot be misread. This bears directly on Q1.
Q1. Temporal compute versus structural access. We varied every form of temporal compute we could, one at a time, connectome vs DP, 3 seeds each:
The pattern answers the question directly. Every form of added compute (depth per tick, recurrent horizon, width, training time) lowers every graph's error but leaves the connectome−DP gap in place. The one manipulation that more than halves the gap adds no compute at all: it gives the student time, by letting the target lag the sensory input. So the bottleneck is not a shortage of computation. It is when sensory information can reach the motor neurons, relative to when the target needs it, which is the structural-access reading.
On matched budgets: we increased depth with everything else fixed, which gives the deeper model more compute than the default. A matched-budget version would take compute away elsewhere (e.g. width) and give it less. Since more compute did not shrink the gap, we see no reason to expect less compute to do so. The BPTT-128 row is matched in samples per update, and it also leaves the gap.
We agree with the reviewer's implicit point: within-tick depth reaches as far as a two-tick delay in hops, yet only the delay helps, so a pure hop-count account is insufficient. The paper says so and keeps latency as a hypothesis (general response, point 3).
Q1, second part: teacher targets based only on the student's available channels. This is exactly the design of our teacher-side test (Sec. 4.7, Appendix I). We distilled the expert into two teachers: a global teacher that reads only the channels the bio-wired student can access, and a local teacher in which each leg's actuators depend only on that leg's 29 channels and the command. The gap is 0.028 and 0.030, against 0.030 for the expert. The ablations were then repeated under the local teacher:
So the bottleneck is not created by asking the student to imitate information it cannot reach. Given the interface, it is intrinsic to the graph's path structure: it moves only when the sensory→motor paths change, or when the target gives the student time to use longer paths.