acceptodds
Under review as a conference paper at ICLR 2027

When Language Models Meet NeuroGraphs: Exploring Enhanced Agentic LLM Framework towards Brain Network Analysis

Abstract

Brain network analysis is crucial for understanding cognition and neurological disorders, yet existing deep learning methods mainly treat connectome analysis as a graph-to-logit classification problem, offering limited explanatory reasoning. Large language models (LLMs) provide a promising interface for knowledge-intensive scientific analysis, but directly applying general-purpose LLMs to brain networks remains challenging due to the structure-language gap, limited neuroscience grounding, and overconfident positive predictions. In this paper, we propose BrainAgent, an agentic LLM framework for knowledge-enhanced brain network analysis. BrainAgent reformulates connectome classification as an iterative process of topology-aware understanding, external retrieval, reasoning, and reflection. Specifically, it first converts raw brain networks into compact multi-level structural descriptions through brain-specific analysis tools, then retrieves relevant neuroscience knowledge and task-specific cases to ground the reasoning process, and finally generates structured predictions with reflective verification. Experiments on four public rs-fMRI datasets show that BrainAgent consistently improves different closed-source and open-source LLM backbones over direct prompting and standard reasoning baselines. Further ablation and interpretability analyses demonstrate the effectiveness of each component and show that BrainAgent produces more comprehensive, multi-level, and verifiable explanations.These results indicate that agentic LLMs provide a practical route toward interpretable and knowledge-grounded brain network analysis.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Neuroscience, Brain Network Analysis, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KSVCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BrainAgent, an inference-time pipeline for classifying functional brain graphs and producing textual explanations. It converts graph features into structured descriptions, invokes tools, retrieves neuroscience knowledge and labeled cases, and uses language-model reflection. Experiments span four resting-state fMRI datasets and several language-model backbones, with component ablations and an explanation-quality study.

Critical strengths
  1. The application of language-model tools to subject-level brain graphs is timely. The system separates graph description, retrieval, case evidence, and reflection into identifiable stages, making its workflow relatively easy to inspect.
  2. The use of multiple datasets and language-model backbones, together with component ablations, is helpful for examining whether the pipeline is broadly applicable rather than tuned to a single model.
Critical weaknesses
  1. The pass@3 results should not be interpreted as the accuracy of a deployable single-label classifier. Appendix D.3, Eq. (8), counts a sample as successful when any of three outputs matches the true label. This is a valid oracle-style evaluation of repeated sampling, but it does not specify how one prediction would be selected without the test label. The stated aggregation of pass@k predictions for precision and recall also requires a clearer, label-independent definition.
  2. The source of the gains is not sufficiently isolated. Section 2.4 builds case memory from training examples with known labels, which is a legitimate supervised support resource and is not evidence of test-label leakage. However, this evidence and the additional retrieval/tool/reflection budget differ from direct prompting. The paper needs comparisons that give simpler methods the same labeled support information and inference resources before attributing gains to agentic reasoning.
Questions for authors
  1. What are the repeated pass@1 results and the results of a prespecified label-independent aggregation rule (e.g., majority vote), with uncertainty intervals?
  2. How does BrainAgent compare with k-nearest-neighbor classification on the same graph representations and a single-pass LLM given the same retrieved labeled cases and inference budget?
  3. Do the gains persist under repeated subject-level or site-held-out splits, and can explanation faithfulness be tested using graph perturbations or blinded experts?
1 comment

Thank you for your constructive comments.

W1.

We would like to clarify that pass@3 is intentionally designed as an oracle-style evaluation metric, as explicitly defined in Appendix D.3, Eq. (8). Its purpose is to measure whether repeated LLM inference can generate the correct prediction among multiple attempts, rather than represent the performance of a deployable single-label classifier. Importantly, we also report pass@1, which directly reflects single-run prediction performance without access to the ground-truth label during prediction. For Recall@3 and Precision@3, we follow the same oracle-based aggregation protocol to ensure consistency with the pass@3 evaluation setting. We will make this distinction more explicit in the revised manuscript to avoid interpreting these oracle-based metrics as practical single-label classification results.

W2.

We would like to clarify that our goal is not to attribute all performance improvements exclusively to multi-round agentic reasoning. Instead, BrainAgent is designed to explore how general-purpose LLMs can leverage brain-specific tools, external neuroscience knowledge, and historical cases to perform brain network analysis without task-specific LLM fine-tuning. We acknowledge that CARD introduces labeled training cases as an external support resource. This is an intentional part of our knowledge-enhanced inference framework. Unlike supervised classifiers, BrainAgent does not update the LLM parameters using these labels; instead, it retrieves relevant cases as contextual evidence during inference. Moreover, our ablation studies in Section 3.5 evaluate the contributions of topology-aware understanding, HARK, CARD, and reflection separately. The results show that removing these components generally reduces performance, supporting the effectiveness of their integration. Therefore, our contribution is not limited to multi-round reasoning alone, but lies in exploring a unified agentic framework for knowledge-grounded brain network analysis.

Q1.

As mentioned above, pass@1 and oracle pass@3 serve different evaluation purposes. We use pass@1 to evaluate single-run prediction performance, while pass@3 measures the potential of repeated inference. We agree that majority voting provides an additional perspective on practical multi-run classification. Since our experiments already involve three independent inference runs, the existing prediction records can be used to evaluate a label-independent majority-voting strategy. We will further report repeated pass@1 statistics and majority-voting results, together with their variability, to provide a clearer distinction between single-run performance, practical multi-run aggregation, and oracle-style evaluation.

Q2.

We appreciate this suggestion, as these comparisons can help distinguish the contribution of retrieved cases from that of the agentic reasoning process. Our CARD module is not simply a nearest-neighbor classifier. It retrieves relevant cases using both graph representations and textual reports, and the retrieved information is subsequently integrated with graph analysis and neuroscience knowledge through LLM reasoning. The resulting prediction is generated from multiple sources of evidence rather than directly adopting the labels of the nearest neighbors. Nevertheless, comparing BrainAgent with kNN using the same graph representations and a single-pass LLM provided with the same retrieved labeled cases would be useful for further isolating these effects. We will consider these controlled comparisons, including matched inference budgets, to better characterize the contribution of the overall agentic workflow.

Q3.

Regarding generalization, we would like to highlight that BrainAgent has been evaluated on four different brain network datasets, covering ASD, ADHD, MDD, and HCP gender classification. The same overall framework can be applied to these tasks without task-specific fine-tuning of the backbone LLM. This demonstrates its applicability beyond a single disease or classification scenario. Furthermore, several datasets used in our experiments, particularly ABIDE and Rest-meta-MDD, contain neuroimaging data collected from multiple sites. Their heterogeneous acquisition settings and populations provide a meaningful evaluation context for brain network analysis. Although site-held-out evaluation specifically examines generalization to unseen acquisition sites, our current results already demonstrate the effectiveness of BrainAgent across multiple tasks and heterogeneous datasets. Regarding explanation faithfulness, BrainAgent differs from unconstrained LLM inference by explicitly grounding its reasoning in computed graph properties, retrieved neuroscience knowledge, and similar historical cases. The region-level, subgraph-level, and graph-level measurements provide concrete structural evidence that can be checked against the input brain network.

Reviewer_KD8POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BrainAgent, an LLM agent framework for brain network classification. It combines topology analysis tools, neuroscience knowledge retrieval, case retrieval and reflection to make predictions and generate explanations. Experiments on four rs-fMRI datasets show improvements compared to direct LLM prompting and standard reasoning approaches, using different LLM backbones.

Critical strengths

1, Applying LLM agents to brain network analysis is an interesting direction. Brain networks are not naturally represented as text, so using topology analysis tools and external knowledge makes sense

2, Experiments on several datasets and LLMs show that the proposed framework performs better than directly prompting LLMs, suggesting that the additional tools and retrieval are useful.

Critical weaknesses

1, The technical novelty is limited. The framework mainly combines existing components like tool use, RAG, case retrieval and reflection. There is no clear new learning or reasoning algorithm

2, The experimental validation is not very convincing. Improvements over direct LLM prompting are expected, but this does not show the method is competitive with strong GNNs or traditional brain network classifiers. Also the explanations are not independently verified for their neuroscience correctness.

Questions for authors
  • How does BrainAgent compare with strong supervised GNNs and graph foundation models, using exactly the same data splits?

  • Can the authors evaluate whether the generated explanations are actually correct from a neuroscience perspective, rather than just measuring their completeness?

  • How is data leakage avoided during case retrieval, especially when the same subjects have multiple scans or appear across different datasets?

2 comments

Thank you for your constructive comments. We appreciate the reviewer's concerns and would like to further clarify the motivation, contributions, and experimental scope of BrainAgent.

1. Technical Novelty. We acknowledge that BrainAgent builds upon established techniques such as tool use, RAG, case retrieval, and reflection. Our contribution is not a fundamentally new learning algorithm, but rather a brain-specific agentic framework that enables general-purpose LLMs to understand and analyze brain networks. Unlike conventional LLM-based classification, BrainAgent integrates multi-level topological analysis, neuroscience knowledge retrieval, and graph-based case reasoning into an iterative process. We believe exploring how LLMs can interact with structured brain data and external neuroscience knowledge is a meaningful research direction, and we will further clarify this contribution in the revised manuscript.

2. Comparison with Supervised GNNs and Graph Foundation Models. We would like to clarify that BrainAgent explores a different technical direction from conventional supervised graph learning. Traditional GNNs and graph foundation models primarily learn predictive representations from graph data, whereas our work investigates whether general-purpose LLMs can perform brain network analysis through inference-time reasoning without task-specific LLM fine-tuning. Our motivation comes from an important observation: when directly applied to brain network classification, general-purpose LLMs frequently exhibit a strong positive prediction bias, sometimes classifying nearly all subjects as having a disorder. BrainAgent attempts to address this limitation by introducing topology-aware understanding, external knowledge retrieval, and reflective verification. To provide additional context, we report our previous experimental results on nine representative supervised graph learning methods, including GCN, GAT, GraphSAGE, GraphTrans, BrainGNN, IBGNN, BrainNPT, THFCN, and ContrastPool, each evaluated over ten independent runs. We also include the pass@1 accuracy of BrainAgent with Gemini 3.1 for reference.

Table: Accuracy (%) of supervised graph learning methods and BrainAgent on four brain network datasets.

DatasetGCNGATGraphSAGEGraphTransBrainGNNIBGNNBrainNPTTHFCNContrastPoolBrainAgent
HCP64.03±1.2165.72±0.6766.87±0.3267.46±0.5566.46±2.1264.72±1.0467.78±1.5367.29±1.7468.10±1.7463.68
ADHD66.74±1.4766.56±0.3067.80±0.3767.00±0.3767.16±2.0165.59±0.4967.84±2.1166.73±1.4065.10±0.8269.52
MDD62.38±0.3763.05±0.3563.29±0.2763.37±0.2763.25±1.0663.07±0.2963.81±0.7762.41±0.6764.05±0.4771.36
ABIDE69.14±0.8468.79±0.7470.74±0.8968.41±1.2070.03±1.6966.02±1.1868.80±1.1066.93±1.2969.89±0.8870.25

Note: The supervised graph learning results are reported as mean ± standard deviation over 10 independent runs, while BrainAgent results are obtained using Gemini 3.1 under pass@1 evaluation.

These results provide a useful reference for understanding the performance scales of the two approaches. However, we emphasize that BrainAgent is not designed to replace supervised GNNs or necessarily outperform specialized models trained on labeled brain network data. Instead, our goal is to explore a complementary paradigm in which general-purpose LLMs analyze brain networks through knowledge-grounded, tool-augmented reasoning without task-specific LLM fine-tuning. Beyond classification, BrainAgent also aims to provide explicit structural evidence, relevant neuroscience knowledge, and interpretable analytical explanations. We believe this represents a meaningful direction for exploring the potential of LLMs in brain network analysis, rather than a direct competition with conventional supervised graph learning methods.

3. Neuroscience Correctness of Explanations. We agree that explanation completeness does not necessarily imply neuroscience correctness. Our current interpretability evaluation focuses on whether generated explanations incorporate region-level, subgraph-level, and graph-level evidence, as well as relevant neuroscience knowledge. While the retrieval and reflection modules are intended to improve the grounding of these explanations, we acknowledge that independent expert evaluation would provide stronger evidence of their scientific reliability. We will clarify this limitation and consider expert assessment and evidence-level verification in future work.

4. Data Leakage Prevention. In our implementation, the CARD case retrieval database is constructed exclusively from the training split, and test samples are not directly included in the retrieval memory. We agree that repeated scans and potentially overlapping subjects require careful consideration beyond sample-level separation. We will further clarify the data partitioning protocol and examine possible subject-level overlaps, including repeated scans and cross-dataset duplication. Where subject identifiers are available, subject-disjoint partitioning and duplicate checking will be used to strengthen the reliability of the evaluation.

Overall, our work is intended to explore the potential of general-purpose LLMs as knowledge-grounded agents for brain network analysis, rather than replace existing supervised graph learning approaches. We thank the reviewer for these suggestions, which will help us better clarify the positioning and limitations of BrainAgent.

Reviewer_fNjLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents an LLM agent framework called BrainAgent for functional brain network classification and analysis that requires no LLM fine-tuning. It integrates multilevel graph analysis, neuroscience knowledge retrieval, and similar-case retrieval to address limitations in structural understanding, domain knowledge, and prediction bias. Through multi-round reasoning, tool interaction, and reflection, BrainAgent produces classification labels with supporting explanations and confidence scores. Experiments across four public datasets and multiple LLM backbones demonstrate improvements over direct prompting and common reasoning baselines, supported by component ablations and explanation analyses.

Critical strengths
  1. The paper clearly identifies the challenges of applying LLMs to brain network analysis and integrates structural analysis, domain knowledge, case retrieval, and reflection into a coherent framework.
  2. Evaluations across four public datasets and multiple LLM backbones, together with comparisons against direct prompting, CoT, and Reflection baselines, provide broad empirical support. Component ablations further clarify the contributions of individual modules.
Critical weaknesses
  1. Some Accuracy, Recall, and Precision values in Table 2 appear inconsistent with a standard binary confusion matrix. On ADHD, the direct-prompting results imply a positive-class prevalence of approximately 37.97%. Assuming the same test split, BrainAgent’s reported Recall@1 and Precision@1 would imply an Accuracy of approximately 71.2%, rather than 56.91%. The authors should clarify the metric definitions and aggregation procedure and verify these results.
  2. BrainAgent’s multi-round reasoning, tool calls, and retrieval introduce additional overhead that is not adequately quantified. Reporting per-sample token consumption, LLM call counts, and end-to-end inference time alongside Direct, CoT, and Reflection would clarify the trade-off between performance and computational cost.
  3. The current evaluation primarily compares general-purpose LLM prompting strategies, leaving BrainAgent’s performance relative to established brain network classifiers unclear. Including representative domain-specific baselines, with explicit descriptions of their training requirements, would better contextualize its classification performance, interpretability, and practical advantages.
Questions for authors

Please refer to the weakness section.

2 comments

Thank you for your recognition of our work and constructive reviews. Here we answer your questions one by one.

Response to W1

Thank you for your question. We double check our raw result data and we admit that some results may be inconsistent. Here are our claims of this proposed weakness.

Reason: We checked the log files and the main factor lies in a "third classification result" beyond 0 and 1, which means LLM may not answer true or false all the time, it may also infer a brain network as "uncertain" when it gets confused (we also mention this phenomenon in section K.2 in the paper). These uncertain outputs were not handled correctly when we merged the raw results, which produced the abnormal accuracy values.

Calibrated Results: We re-checked the data and found that errors mainly occur in @1 metric for certain datasets (for @3 metrics, the results may be not influenced so much as one sample will be marked as positive as soon as any one of the dataset output correct label). Specially, We re-run Qwen for ADHD, HCP and Deepseek for HCP. We mention that Qwen3 max and Deepseek v3.2 APIs are not provided officially anymore, so we use latest Qwen 3.8 max and Deepseek v4.1 flash insead. We compare the results of BrainAgent and direct infer, the results are shown below (No "uncertain" output for these two latest models):

ADHD-Qwen3.8AccRecallPrecision
Direct37.97100.0037.97
BrainAgent67.5560.6650.00
HCP-Qwen3.8AccRecallPrecision
Direct45.41100.0045.41
BrainAgent58.4564.2153.98
HCP-Deepseek v4.1 flashAccRecallPrecision
Direct45.41100.0045.41
BrainAgent59.9061.7055.24

Discussion: Although we re-ran the experiments with newer state-of-the-art models, the overall pattern is unchanged. Direct reasoning still tends toward 100% Recall, and BrainAgent effectively mitigates this tendency. In addition, the instruction-following ability of these models is now strong enough that they produce no out-of-scope outputs. We discuss this further in our response to W3.

Response to W2

Thank you for your valuable suggestion. We acknowledge that BrainAgent introduces additional inference overhead compared to direct inference, primarily due to its multi-round tool invocation, knowledge retrieval, and reflection processes. These additional steps are designed to improve prediction reliability and interpretability, but inevitably increase computational costs. We agree that a quantitative efficiency analysis is important. In the revised version, we will further collect and report per-sample token consumption, the number of LLM and tool calls, and end-to-end inference time, providing a more comprehensive comparison with Direct, CoT, and Reflection.

Response to W3

Thank you for this question. We acknowledge that while BrainAgent, is a first attempt at LLM-driven brain network analysis, still struggles with traditional binary classification. However, we do not think this indicates that LLMs lack the necessary capability. The field still lacks an authoritative benchmark, and LLMs should not be evaluated only through binary classification borrowed from GNN metrics or through simple neuroscience QA [1]. BrainBench [2] is a recent benchmark designed for LLM-based EEG analysis, and we believe it can inspire similar efforts for other modalities such as fMRI and DTI. On the other hand, as you mentioned while this prompt-based LLM inference demonstrates limited superiority, current focus of agent research has shifted towards harness + LLM combination.

Thus, we believe that BrainAgent is a novel exploration in LLM-driven brain network analysis and we agree that more work should be explored including building domain-specific benchmarks and robust agent infra (harness).


We hope this response can help answer some of your questions.

References

[1] Luo X, Rechardt A, Sun G, et al. Large language models surpass human experts in predicting neuroscience results[J]. Nature human behaviour, 2025, 9(2): 305-315.

[2] Zhou Y, Chen Y, Wu C, et al. BrainBench: Benchmarking Large Language Models for Comprehensive EEG Understanding[J]. arXiv preprint arXiv:2608.04156, 2026.

Thank you for the detailed response. I have no remaining concerns and will maintain my positive rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.