acceptodds
Withdrawn from ICLR 2027

SPEF: Semantic Prior-Selected Expert Fusion for Working-Memory Prediction from Multimodal Brain Connectomes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

12 views

Abstract

Predicting cognition from functional and structural connectomes requires learning from thousands of correlated edges from comparatively modest neuroimaging cohorts. In order to stabilize the prediction framework, we introduce Semantic Prior-selected Expert Fusion (SPEF), a transparent framework that converts cognitive task descriptions and atlas region names into a frozen semantic atlas prior, selects compact functional-connectivity (FC) and structural-connectivity (SC) expert subspaces, and hierarchically fuses the experts in a strong data-driven connectome backbone. The prior is generated independently of participant imaging and behavioral data, validated against the canonical AAL116 ordering, normalized and frozen, and then transferred to connectivity space to define compact FC and SC subspaces. On 510 Human Connectome Project participants, five-seed, five-fold repeated nested cross-validation gives Pearson , RMSE , and MAE for SPEF-ridge on a working memory task. Among the evaluated multimodal comparators and SPEF variants, the best Pearson correlation is , attained by both matched SPEF-ridge and cross-task SPEF-NC-ridge, while the latter achieves the lowest RMSE () and MAE (). The working-memory semantic prior also exceeds shuffled and random prior controls by and mean seed-wise correlation, respectively, while the matched and cross-task semantic priors perform similarly. Prediction-linked coefficient maps show reproducible FC and SC structure. Relative to an architecture-matched no-prior full-connectome ridge expert, SPEF increases FC edge-rank stability ( vs. ) and yields a substantially less adverse top-ROI perturbation contrast ( vs. ), while other stability measures remain mixed. SPEF provides an auditable framework for introducing language-derived cognitive knowledge as an anatomically grounded inductive bias for multimodal connectome based prediction and candidate biomarker analyses.

Primary area:
applications to neuroscience & cognitive science
Keywords:
semantic priors, multimodal brain connectomes, working-memory prediction, functional connectivity, structural connectivity, cognitive neuroscience

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RMsiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a new model using semantic prior to select brain function and structural connectivity subspaces for prediction of working memory. The semantic prior selected subspace features are further fused with data-driven FC/SC backbone features with trained weights parameters. The performance was evaluated in nested CV. And four different semantic priors are compared. Overall the novelty comes from semantic prior generation and testing the effectiveness of the semantic prior. That said, there exists some confusion and major concerns on the results interpretation.

Critical strengths

The novelty comes from semantic prior generation and testing the effectiveness of the semantic prior for brain connectivity subspace selection.

Critical weaknesses

The data-driven FC-SC backbone is not explained at all in the paper. Table 1 indicates data-driven FC-SC backbone model along has a r value 0.2430, While by adding semantic prior, the SPEF does not show significant improvement ( 0.2439). This is the main performance results, for which the explanation is missing. Figure 2 presents important regions but how to proof these regions are not influenced by data-driven backbone selection?

Questions for authors

Figure 2 presents important regions but how to proof these regions are not influenced by data-driven backbone selection? How to explain F3 random prior could have higher stability than matched prior for SC?

Reviewer_C7KXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SPEF for working-memory prediction from functional and structural connectomes. A frozen language model generates AAL116 regional relevance scores, which define compact connectivity subspaces for ridge or network-controlled ridge experts. Their predictions are combined through modality-level and hierarchical fusion with a data-driven FC–SC backbone. Evaluation uses 510 HCP participants and five-seed, five-fold repeated nested cross-validation. SPEF-ridge achieves r=0.2439 versus 0.2430 for the backbone. Matched and cross-task priors perform almost identically, while matched priors outperform the evaluated shuffled and random controls. Coefficient-stability and perturbation analyses are also reported

Critical strengths
  1. The framework separates semantic-prior construction from participant-level estimation. The prior is generated without participant imaging or behavioral inputs, validated against atlas labels, and frozen before evaluation, making its intended role clear and auditable.

  2. The reported protocol includes nested model selection, fold-local standardization, cross-fitted fusion weights, and identical outer partitions across methods. Swapping priors within the same architecture is a useful design for assessing prior identity.

  3. The manuscript reports negative and mixed findings transparently, including matched/cross-task parity, weak SC-expert prediction, and the absence of a uniform stability advantage over the no-prior model.

Critical weaknesses
  1. The predictive benefit is minimal: SPEF reaches r=0.2439 versus 0.2430 for the backbone, while the FC expert alone reaches 0.2448. The added multimodal hierarchy is not convincingly justified, and the strongest aggregate NC-ridge setting reduces to ordinary ridge.

  2. The language-derived prior’s distinctive value remains unclear. Matched and cross- task priors perform almost identically, while shuffled/random controls use single realizations. Without conventional anatomical priors or training-only feature-selection baselines, generic structured feature restriction remains a plausible explanation.

  3. Biomarker faithfulness is not established. Top-ROI masking produces no greater degradation than random masking, and several stability measures favor the no-prior model. Shared zero coefficients may also inflate edge-rank agreement, requiring support-matched controls.

  4. Evaluation covers only one cohort, outcome, and atlas. The provided main manuscript insufficiently specifies the backbone and comparator tuning, while family-aware partitioning and demographic/motion handling remain unclear. These limitations weaken confidence in the small reported improvement.

Questions for authors
  1. Please report paired uncertainty estimates for SPEF versus the backbone and FC expert alone, accounting for repeated cross-validation. How does the full hierarchy compare with similarly tuned FC-only ridge and the architecture-matched no-prior full-connectome expert? What measurable benefit does hierarchical multimodal fusion add?
  2. Please evaluate multiple shuffled/random prior realizations and compare with at least one conventional anatomical or meta-analytic prior. A training-only data-driven feature-selection baseline would also help isolate the semantic contribution. How strongly correlated are the working-memory and fluid-intelligence prior vectors?
  3. Were outer folds, inner folds, and fusion cross-fitting grouped by HCP family, or were only unrelated participants included? Please clarify how age, sex, and head motion were handled, ensuring that any data-dependent adjustment was fitted within training partitions.
  4. Does edge-rank Spearman include zeros outside the selected subspaces? Please provide support-matched stability controls and uncertainty for the top-minus-random perturbation contrast. What evidence shows that the coefficient rankings identify prediction-relevant ROIs beyond the anatomical restrictions imposed by the prior?
  5. Please specify the backbone architecture, training procedure, and hyperparameter search, together with the tuning protocol for the graph comparators. Please also provide the participant-selection flow explaining how the complete-data and quality control criteria yielded the final 510-person cohort.
Reviewer_k7bWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks a language model to score brain regions for their relevance to working memory. The scores are used to pick a small set of connectome edges. Ridge regressions are fitted on those edges for functional and structural connectivity. Their predictions are averaged, and then averaged again with a data-driven baseline model. The target is a working-memory score in HCP. The claim is that the semantic prior improves prediction and gives more stable biomarkers. Controls swap the prior for a fluid-intelligence prior, a shuffled prior, and a random prior.

Critical strengths

Using the language model only to produce a frozen region list before any data is touched is a clean design. The nested cross-validation protocol is carefully described and the same folds are reused for every method. The control design with matched, cross-task, shuffled and random priors is the right way to test whether semantics matter.

Critical weaknesses

The paper does not show that the prior does anything. The full method beats the baseline by one thousandth of a correlation point, which is far inside the seed-to-seed spread. The paper's own ablation shows that the ridge on the selected functional edges alone beats the whole fused system. So the fusion machinery makes things slightly worse. The controls do not isolate the semantic contribution. The fluid-intelligence prior gives exactly the same result as the working-memory prior. A simpler reading is that any prior pointing at frontal and parietal regions works, and the language model is a roundabout way of writing that down. A hand-written frontoparietal mask would settle this and is not tested. The strongest comparator is never defined. The data-driven baseline supplies most of the final prediction, yet no equation or model class is given for it. The graph network comparators score far below a plain ridge, and below what their own papers report on similar data. They look broken rather than beaten. The biomarker claims are circular. The expert can only use edges touching the regions the language model picked, so finding those regions as important is guaranteed. The perturbation test confirms this. Masking the top regions hurts prediction no more than masking random ones. Also the cohort handling is weak. Family structure in HCP is ignored, so relatives can sit on both sides of a split. Age, sex and motion are not controlled. Also the related-work section names one research group by name four times and half the references come from that group. This should be flagged to the area chair as a possible double-blind issue.

Questions for authors

Q1. What exactly is the data-driven baseline? Please give the model, inputs and tuning. Q2. Please report the functional-edge expert alone for all four priors, with no baseline fusion, and add a hand-written frontoparietal mask as a fifth control. This is the only comparison that isolates the prior. Q3. Why do the graph comparators score so far below their published numbers, and were HCP families kept within folds? What would change my assessment: a defined baseline, the single-expert comparison across all four priors, and a hand-written frontoparietal control that the semantic prior clearly beats

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.