- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper asks whether connectome-learning pipelines should discard synaptic relations whose endpoints are not yet resolved into traced neurons. It formalizes an "observation graph" partitioning observed bodies into retained (traced) and unresolved segments, and separates an observation contrast (access to the unresolved channel, within a fixed scorer family) from a learning contrast (learned scorer vs. a fixed rule).
The central methodological device is leave-fragment-out source supervision: count-conserving fragments sampled from MaleCNS traced owners, subtracted from the positive before feature construction, with an exact hypergeometric analysis of residual self-overlap. Models (linear and a 1,153-parameter MLP with three seeds) are frozen and evaluated on MANC v1.0→v1.2 revision-destination retrieval: 557 donor segments in 35 components, each ranked against all 23,200 earlier traced candidates, with tie-integrated, component-macro metrics.
The segment-aware neural scorer reaches Hit@20 of 0.5399 vs. 0.4866 (retained-only arm) and 0.4666 (fixed cosine), while Hit@1, Hit@5 and MRR all decrease. A separate masked-adjacency task improves macro AP from 0.3800 to 0.4708 with retained unresolved context. The endpoint is a recorded revision destination, not biological ownership or proofreading productivity.
Critical strengths
Full-candidate ranking against 23,200 candidates with uniform tie integration and component-macro aggregation is materially more rigorous than the sampled-negative protocols common in this area, and the support-stratified analysis (Table 2/8) honestly shows that learning adds nothing over fixed cosine for unresolved-only donors.
The leave-fragment-out construction addresses an exact supervision leak (the query retrieved through its own counts inside the positive), and the finite-population expectation analysis is the right way to show what signal survives subtraction
Critical weaknesses
The +0.0533 Hit@20 observation contrast is an average over 35 non-independent components with no uncertainty attached. Removing the five largest component differences leaves +0.0172, so a small number of components carry much of the result, and no bootstrap, permutation, or leave-one-component-out analysis is reported
Meanwhile, the paper's central contribution that is the leave-fragment-out residualized supervision is not compared empirically against naive query-with-owner supervision or partial subtraction on the target task, so the reader cannot tell whether it matters at all.
The correspondence experiments have a design problem, The model learns with the unresolved relation features present and unrestricted, and only then does the paper shuffle or zero them at test time. But shuffling everything also wrecks the candidate marginals, so when performance falls from 0.5399 to roughly 0.49, there is no way to tell how much of that drop is about correspondence and how much is plain distribution shift. A few controls would settle the question: retrain without those features in the first place, use permutations stratified by degree and mass so the marginals survive, and compare against simple degree or popularity baselines. None of them appear in the paper.
The learning story is thin for similar reasons. A small MLP gets 200 training updates on 768 tasks with three seeds, and its only competition is a fixed cosine baseline and a linear model nobody bothered to tune. There is no tuned linear model, no listwise loss, no gradient-boosted trees, no graph methods like node2vec or GNN link predictors, and no geometry-aware prior beyond the spatial centroids, which manage just 0.05 Hit@20 in Appendix E. There is also no oracle ceiling to show what the best achievable score even looks like.
Then there is the retained mass question. The 42.1% outgoing and 95.1% incoming numbers come only from MaleCNS, and nobody checked whether MANC behaves the same way. Until those missing baselines are run, much of what the paper calls learning could just be the model picking up degree and mass proxies.
Questions for authors
-
Does leave-fragment-out supervision actually change MANC outcomes relative to naive (query-left-in) or partially subtracted supervision, on both Hit@20 and the rank-sensitive metrics?
-
What do component-level cluster-bootstrap or permutation intervals on the +0.0533 contrast look like, and when was K=20 fixed relative to first inspection of target results? Please provide full cumulative Hit@K curves (K=1–100) with component-level bands.
-
Does the gain survive conditional permutations that preserve candidate degree, mass, and component strata, and comparison against degree-only and popularity baselines? How do retrained-off and zeroed-at-test arms compare?
-
Can a tuned linear model, a listwise objective, a gradient-boosted tree over the same 18 features, or a standard learned-graph baseline (e.g., node2vec or a GNN link predictor) match the MLP?
-
Does the 42.1% & 95.1% retained-mass asymmetry hold in MANC, and can the pipeline be frozen and evaluated on a held-out component set or a different release pair to give a genuinely prospective test?