acceptodds
Under review as a conference paper at ICLR 2027

BrainTAL: A Target-Adaptive Brain Token Learner for Cross-Site Brain Network Analysis

Abstract

Multi-site brain network analysis provides an important tool for characterizing functional connectivity patterns across different medical centers and has attracted increasing attention in brain disease diagnosis. However, existing methods often struggle to adapt to target-site distribution shifts under source-free and label-free settings. Moreover, they usually overlook task-specific semantic differences within the same brain network, which limits their predictive ability. To address these limitations, we propose BrainTAL, a target-adaptive brain token learning framework for cross-site brain network analysis. Specifically, we first construct compact brain network instructions, where salient functional connections are introduced as neurobiological priors for language-guided modeling. Second, we design task-conditioned brain tokens, enabling the model to adaptively capture task-relevant functional connectivity patterns from the same brain network across different neuroimaging tasks. Finally, we propose a brain token adaptive calibration strategy, which performs lightweight adaptation of the source-trained model using unlabeled target-site data, thereby alleviating cross-site distribution shifts and improving prediction robustness on the target site. Experimental results demonstrate that BrainTAL consistently improves disease recognition performance in source-free cross-site scenarios and effectively captures task-relevant functional connectivity patterns, validating its effectiveness and interpretability.

Primary area:
applications to neuroscience & cognitive science
Keywords:
brain network analysis, source-free domain adaptation, functional connectivity, large language models, neuroimaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DHxaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BrainTAL, a target-adaptive framework for cross-site brain network analysis under source-free and label-free adaptation settings. The method introduces compact brain network instructions, task-conditioned brain tokens, and an adaptive calibration strategy to address distribution shifts across imaging sites. Task-specific queries are used to extract representations for different prediction objectives, while a lightweight adapter is updated using unlabeled target-site data. Experiments on ABIDE and ADHD evaluate the framework on disease classification, sex classification, and age prediction.

Critical strengths
  1. The paper addresses a relevant problem in multi-site neuroimaging, where distribution shifts and restrictions on source data availability make model adaptation challenging. The proposed setting is practically motivated.

  2. The task-conditioned brain token mechanism provides a reasonable approach to modeling different prediction objectives using shared functional connectivity representations. The integration of task conditioning and source-free adaptation is potentially useful.

  3. The experiments cover multiple prediction tasks on two neuroimaging benchmarks. The results in Tables 1 and 2 show improvements over several competing approaches, and the ablation experiments provide additional evidence for the proposed components.

Critical weaknesses
  1. The methodological novelty is somewhat limited. Task-conditioned representation learning, lightweight adaptation, and consistency-based optimization have been explored extensively in related areas. Although their integration is reasonable, the paper does not sufficiently establish which aspect represents a substantial methodological advance beyond existing approaches.

  2. The justification for incorporating a large language model is not entirely convincing. The prediction tasks mainly involve structured functional connectivity information, and it remains unclear whether language-based semantic representations provide advantages that cannot be achieved using a smaller task-conditioned graph model. A more controlled comparison with non-LLM alternatives would strengthen the contribution.

  3. The target adaptation strategy relies on confidence-guided consistency learning. Such approaches can be sensitive to inaccurate predictions and distribution shifts. The paper would benefit from a more detailed investigation of adaptation stability, particularly when the initial source model performs poorly on the target site.

Questions for authors
  1. How much of the performance improvement comes specifically from language-guided modeling rather than task-conditioned graph representation learning? Could the authors provide a comparison with a similarly sized non-LLM architecture?

  2. How sensitive is the proposed calibration strategy to the confidence threshold and the quality of initial target predictions?

  3. Does the method remain effective when the target site has substantially different demographic or disease distributions from the source sites?

3 comments
AuthorsOpenReview ↗

Response to Reviewer DHxa

Dear Reviewer,

We sincerely thank you for your thoughtful and constructive review. We appreciate your recognition of the practical relevance of cross-site brain network analysis and the potential of combining task conditioning with source-free adaptation. Your comments helped us clarify the methodological contribution and strengthen our evaluation. We have revised the manuscript and respond to your concerns below.

Weakness 1: Methodological contribution

We have clarified the contribution of BrainTAL in the Introduction. BrainTAL couples task-conditioned graph-to-token mapping with source-free calibration at the brain–language interface. Shared queries and task-specific offsets produce task-dependent brain tokens, while a lightweight adapter is calibrated using unlabeled target-site diagnosis predictions and then shared across tasks. This design addresses cross-site shifts at the interface between brain representations and the language model.

The ablation in Table 3 shows that removing task-conditioned tokenization reduces diagnosis accuracy by 4.93 and 6.68 percentage points on ABIDE and ADHD, respectively. Under the matched adaptation setting in Table 4, BrainTAL Stage II exceeds the teacher–student baseline by 1.73 and 2.48 percentage points, respectively.

Weakness 2 and Question 1: Comparison with non-LLM alternatives

To examine the LLM-based design, we added comparisons with two non-LLM alternatives: a task-conditioned graph model with a compact prediction head and one with a ViT-3B module. The ViT-3B model has 3.028B parameters, compared with 3.141B for BrainTAL. All models are evaluated under the same data splits, and Stage II uses unlabeled target-site data.

At Stage I, the compact-head model achieves 61.44% and 56.93% diagnosis accuracy on ABIDE and ADHD, while the ViT-3B model achieves 62.11% and 57.64%. BrainTAL-S1 achieves 63.93% and 58.81%, respectively. After Stage II adaptation, the corresponding accuracies are 64.46% and 58.92% for the compact-head model, 66.14% and 60.96% for the ViT-3B model, and 66.65% and 62.81% for BrainTAL.

Against the similarly sized ViT-3B model, BrainTAL improves accuracy by 1.82 and 1.17 percentage points at Stage I, and by 0.51 and 1.85 percentage points after Stage II, on ABIDE and ADHD, respectively. These comparisons are reported in Table 6, Appendix A.4, and support the benefit of the overall LLM-based design relative to the tested alternatives.

Weakness 3 and Question 2: Adaptation stability and confidence criterion

We have clarified the confidence criterion in Section 3.3.2. BrainTAL assigns sample weights based on teacher prediction entropy and normalizes them within each mini-batch. The value 0.1 is used to determine whether to update the batch; it is not a sample-level probability cutoff. Because the maximum normalized weight is typically close to one in a non-degenerate batch, this criterion generally permits the update.

To examine adaptation when the initial source model performs relatively poorly, we provide site-wise results in Appendix A.5. On NYU, the site with the lowest source-only diagnosis accuracy in both datasets, accuracy increases from 58.33% to 63.33% on ABIDE and from 52.92% to 56.10% on ADHD after adaptation. Figure 3(a) also shows that performance improves during the first 10 adaptation epochs but declines with prolonged adaptation. These results illustrate adaptation behavior at sites with relatively low initial accuracy.

Question 3: Demographic and disease distributions

We have added a site-wise analysis of the observed distributions in Appendix A.5. For ABIDE-USM, the male proportion is 100.00% at the target site versus 81.95% in the pooled source sites, while the diagnosis-positive proportion is 61.67% versus 44.15%. Diagnosis accuracy increases from 68.33% to 70.00% after adaptation.

For ADHD-KKI, the diagnosis-positive proportion is 26.60% at the target site versus 47.67% in the pooled source sites, and accuracy increases from 65.96% to 68.49%. Target labels are used for post-hoc reporting, not adaptation or model selection. These examples show improvements under the observed site-level distribution differences.

Thank you for the additional comparisons and site-wise analyses. These help address my concerns, but the gains over ViT-3B do not clearly isolate the benefit of language-guided modeling. The decline with prolonged adaptation also leaves questions about stability. Together with my remaining concerns about methodological novelty, I maintain my rating of 2

Dear Reviewer,

Thank you for taking the time to review our additional comparisons and site-wise analyses. We appreciate your acknowledgement that these results help address some of your concerns.

Regarding the LLM contribution, we agree that the ViT-3B comparison evaluates the overall designs and does not by itself isolate the effect of language semantics. We added it to assess BrainTAL against a non-LLM alternative of similar parameter scale. In addition, the matched adaptation comparison in Table 4 uses the same source checkpoint and adapter architecture, allowing us to examine the contribution of the adaptation strategy under a shared source-model setting.

Regarding stability, the main results use a fixed 10-epoch adaptation schedule, with neither target labels nor target-label-based model selection. The site-wise analysis in Appendix A.5 shows improvements at NYU, where the initial source-only accuracy is lowest in both datasets. The longer adaptation trajectory is included to show how performance changes when adaptation is extended.

We have also clarified that the methodological contribution lies in coupling task-conditioned brain tokens with source-free calibration at the brain–language interface, with the diagnosis-calibrated adapter shared across tasks. We hope these clarifications help convey the motivation and contribution of the overall framework.

We would be grateful if you could consider whether the additional evidence and clarifications warrant revisiting your assessment. Thank you again for your time and careful consideration.

Reviewer_G7wVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrainTAL combines task-conditioned brain tokens, connectivity descriptions, and an LLM for diagnosis, sex classification, and age prediction. A lightweight token adapter subsequently adapts the frozen source-trained model using unlabeled target-site diagnosis predictions, teacher-student consistency, and entropy minimization.

Critical strengths
  1. The source-free setting is relevant, and adapting only the token interface provides a coherent, parameter-efficient deployment strategy.
  2. Leave-one-site-out experiments on two datasets, ten runs per site, and component ablations show average improvements over the source-only model. Model selection explicitly excludes target labels (Section 4.2).
Critical weaknesses
  1. Adaptation gains are not cleanly separated from source-representation quality. Tables 1-2 do not establish shared backbones or source checkpoints for adaptation baselines, and the closely related FCN-LLM is omitted. The S1/DV/CM ablations help but do not replace matched adaptation comparisons.
  2. The central adaptation interface is underspecified. Equations (10)-(15) require diagnostic class probabilities from a generative LLM, but probability extraction, concrete answer formats, augmentation strengths, and the entropy weight are unspecified. These choices directly control the 0.95 confidence gate.
  3. Uncertainty does not resolve the calibration contribution. Tables 1-2 report dispersion across site means, while Table 3 gives no uncertainty. Removing confidence/momentum calibration changes ABIDE accuracy from 66.65 to 66.32; per-site paired results are needed to establish a reliable benefit.
Questions for authors
  1. Using the same source checkpoint and adapter, how does Stage II compare with entropy-only and standard teacher-student adaptation?
  2. How are diagnostic probabilities extracted, and what augmentation strengths and entropy weight are used?
  3. Can the authors provide per-site paired improvements and run-level confidence intervals for the calibration ablation?
4 comments
AuthorsOpenReview ↗

Response to Reviewer G7wV

Dear Reviewer,

We sincerely thank you for your thoughtful and constructive review. We appreciate your recognition of the relevance of the source-free setting and the potential of adapting the token interface for parameter-efficient deployment. Your comments helped us clarify the adaptation strategy and strengthen the experimental evaluation. We have revised the manuscript and respond to your concerns below.

Weakness 1 and Question 1: Matched adaptation comparison and FCN-LLM

We have added a matched comparison of adaptation strategies in Table 4, Section 4.4. For each target site, all adaptation methods start from the same Stage-I source checkpoint and use the same token adapter architecture. This enables a direct comparison of the adaptation strategies under the same source-model setting.

On ABIDE and ADHD, respectively, BrainTAL Stage II achieves diagnosis accuracies of 66.65% and 62.81%, compared with 64.12% and 59.47% for entropy-only adaptation and 64.92% and 60.33% for standard teacher–student adaptation. Thus, BrainTAL exceeds the teacher–student baseline by 1.73 and 2.48 percentage points.

Regarding FCN-LLM, we added a discussion of this closely related work in the Introduction. We could not locate a publicly available official implementation or checkpoint for evaluation under our protocol; the FCN-LLM paper states that its code will be released upon acceptance. We therefore do not include it as a quantitative baseline.

Weakness 2 and Question 2: Probability extraction and adaptation details

We have clarified the Stage-II implementation in Sections 3.3.2 and 4.2. For diagnosis, the answer format begins with “Diagnosis:” followed by “Positive” or “Negative”. The diagnostic probabilities are obtained by applying softmax to the logits corresponding to these two answer tokens. The teacher logits use a temperature of 0.2.

The weak view uses Gaussian noise with a standard deviation of 0.01. The strong view uses edge dropout at a rate of 0.30 and Gaussian noise with a standard deviation of 0.01.

We have corrected the original description of the 0.95 confidence gate and entropy loss. The teacher prediction entropy is used to assign relative sample weights to the consistency loss, rather than as a separate entropy loss term. An optimization step is performed when at least one normalized sample weight in the mini-batch exceeds 0.1. The Stage-II objective also includes a Soft-F1 loss with a weight of 0.5; its formulation is provided in Appendix A.3.

Weakness 3 and Question 3: Site-wise paired improvements and uncertainty

We have added a site-wise comparison between BrainTAL and the w/o CM variant on ABIDE in Table 5, Appendix A.2. For each target site, we report the mean paired accuracy difference across 10 runs and its 95% confidence interval. The differences are:

  • UM: +0.35 percentage points, 95% CI [+0.04, +0.66].
  • USM: +0.38 percentage points, 95% CI [+0.03, +0.73].
  • LEUVEN: +0.22 percentage points, 95% CI [−0.11, +0.55].
  • UCLA: +0.26 percentage points, 95% CI [−0.07, +0.59].
  • NYU: +0.44 percentage points, 95% CI [+0.15, +0.73].

The mean difference across the five sites is +0.33 percentage points. For each site, the confidence interval is calculated from the 10 paired run-level differences using a two-sided t distribution with 9 degrees of freedom. The Average row reports a separate confidence interval based on the five site-level differences. This analysis provides a clearer view of the calibration contribution and its uncertainty.

Dear Reviewer,

Thank you again for your constructive review. In response to your comments, we added a matched comparison of adaptation strategies in Table 4, clarified the Stage-II implementation in Sections 3.3.2 and 4.2, and provided site-wise paired differences and confidence intervals in Table 5, Appendix A.2.

If time permits, we would greatly appreciate your feedback on whether these revisions address your main concerns. If any point requires further clarification, we would be happy to provide it. We would also be grateful if you could consider the revisions in your assessment.

Thank you again for your time and consideration.

I appreciate the matched adaptation comparisons, the additional implementation details, and the site-wise paired confidence intervals. These revisions substantially address my concerns about source-model confounding and the uncertainty of the calibration benefit. However, the correction to the confidence gate and adaptation objective leaves an important implementation question unresolved: whether the revised formulation describes the procedure used for the reported results or reflects changes introduced during revision. The role of the normalized-weight threshold also needs clarification, since normalization by the batch maximum makes the update condition almost always satisfied. Given this remaining uncertainty about the central adaptation procedure, I maintain my original score.

Dear Reviewer,

Thank you for your careful follow-up and for recognizing the additional comparisons and analyses. We appreciate the opportunity to clarify the implementation.

The revised formulation describes the Stage-II procedure used to obtain the reported BrainTAL results. The change was a correction to the description in the manuscript, rather than a change to the adaptation objective used for those results. Specifically, the implementation uses an entropy-weighted teacher–student consistency loss together with a Soft-F1 loss weighted by 0.5; it does not use a separate entropy loss or a 0.95 sample-level confidence gate.

Regarding the normalized-weight threshold, you are correct that the maximum normalized weight is generally close to one in a non-degenerate batch. The 0.1 condition is therefore a batch-level update check and is usually satisfied; it is not intended to exclude individual samples according to an absolute confidence threshold. The relative entropy-based weights are used in the consistency loss to adjust the contribution of samples within the batch.

We hope this clarification resolves the remaining implementation concern. We would be grateful if you could take it into account in your assessment. Thank you again for your time and careful consideration.

Reviewer_D55aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BrainTAL, a target-adaptive brain token learning framework is represented for cross-site brain network analysis. It constructs compact brain network instructions, where salient functional connections are introduced as neurobiological priors for language-guided modeling. It designs task-conditioned brain tokens, enabling the model to adaptively capture taskrelevant functional connectivity patterns from the same brain network across different neuroimaging tasks. Finally, a brain token adaptive calibration strategy is set to perform lightweight adaptation of the source-trained model using unlabeled target-site data, thereby alleviating cross-site distribution shifts and improving prediction robustness on the target site.

Critical strengths

The paper proposes BrainTAL, a target-adaptive brain token learning framework for cross-site brain network analysis. The proposed method is innovative in integrating salient functional connections as neurobiological priors into language-guided modeling, enabling the construction of compact and informative brain network instructions. The paper presents a clear research motivation and a coherent methodological framework, effectively combining brain network analysis with language-guided learning. The target-adaptive token learning strategy is promising for capturing task-relevant brain network information and improving the representation and analysis of brain functional connectivity across different sites.

Critical weaknesses

The paper has some limitations that should be addressed to further improve its quality. First, the overall presentation and description of the proposed method could be improved to enhance clarity and readability, particularly in explaining the methodological details and the underlying rationale of the framework. Second, the experimental evaluation would benefit from more intuitive and interpretable case studies. For example, the authors could provide actual brain network inputs and the corresponding text outputs generated by the LLM, accompanied by neuroscientific interpretations of the results. Comparing these interpretations with those produced by existing methods would help demonstrate the advantages of BrainTAL in capturing meaningful brain network information and generating interpretable results. Such qualitative analyses would complement the quantitative evaluations and provide stronger evidence of the framework’s effectiveness and practical value.

Questions for authors

Q1: The mathematical notation and variable definitions in the paper need to be further standardized. For example, in Eq. (3), RR is used twice to represent different meanings, which may cause confusion and should be revised. Similarly, LL denotes the number of tokens in Eq. (5), but represents the loss function in Eq. (7). The authors are advised to revise the notation to ensure consistency and avoid ambiguity throughout the paper.

Q2: Since Eq. (7) defines the loss function for the Task-Conditioned Brain Token Learning stage, why does it use the original variables defined in Eq. (2) rather than the variables derived in the procedure described in Section 3.2? The authors should clarify the rationale behind this design choice and explain how the loss function is related to the intermediate representations and variables introduced in Section 3.2. Q3: The authors could provide actual brain network inputs and the corresponding text outputs generated by the LLM, accompanied by neuroscientific interpretations of the results. Comparing these interpretations with those produced by existing methods would help demonstrate the advantages of BrainTAL in capturing meaningful brain network information and generating interpretable results.

2 comments

Thank you for the response. I will maintain my current score.

Dear Reviewer,

Thank you for taking the time to read our response and reconsider the paper. We appreciate your earlier suggestions, which helped us clarify the mathematical notation and training objective and add the qualitative case study in Figure 5 and Appendix A.1.

If any of your original concerns remain insufficiently addressed, we would be grateful if you could let us know. We would also appreciate it if you could consider whether the clarifications and revisions warrant an update to your assessment.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Our Responses and Revisions

Dear Reviewers,

Thank you for your time and constructive feedback. We have responded to your comments and revised the manuscript with additional experiments and clarifications.

As the discussion period draws to a close, we would greatly appreciate any further feedback on whether our responses and revisions address your concerns. If any points require further clarification, we would be happy to provide it. We would also be grateful if you could take the revisions into account in your final assessment.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.