- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper uses amortized inference to accelerate neural mass model inversion, estimating condition-dependent changes in effective connectivity from EEG observations. The authors trained EC-TCN and SparseGraphInverse on simulated data for inference in constrained candidate sets and larger connectivity spaces, respectively, reducing the computational cost of per-case iterative optimization through direct prediction. The paper evaluates the accuracy, computational efficiency, and scalability of these methods and reveals differences between target localization and source attribution. Scaling experiments show that edge localization deteriorates as the candidate connectivity space expands. With the total number of candidate connections held fixed, increasing the number of candidate sources per target reduces edge-level recovery while target localization remains accurate. This indicates that the identifiability among candidate sources is an issue that needs to be addressed when expanding the scale of connectivity inference. Experiments show that within a constrained candidate connectivity space, amortized inference can improve connectivity recovery accuracy and significantly reduce the computational cost of repeated inference. Real EEG experiments support the application of the model without the need for retraining.
Critical strengths
- By examining target localization separately from source attribution, the paper shows why accurately identifying affected brain regions does not necessarily imply reliable recovery of individual connections. This distinction helps researchers evaluate connectivity inference methods without mistaking strong target-level performance for accurate edge-level recovery. It also provides a more specific basis for investigating where inference fails.
- The paper compares detection performance, magnitude error, uncertainty calibration, and inference cost, providing practical guidance for choosing connectivity estimators.
Critical weaknesses
- The efficiency advantage of amortized inversion is already established in prior work, and the two networks in this paper primarily serve to implement and evaluate this approach. Therefore, the paper’s novelty largely depends on the distinction between target localization and source attribution. However, the authors have not sufficiently explained what this observation adds to existing understanding of identifiability in neural inverse problems, or what specific guidance it provides for designing connectivity inference methods. This leaves me with reservations about the depth of the paper’s contribution.
- Controls involving a widened EC-TCN, alternative decoders, and a linear probe indicate that difficulty in source attribution is not unique to SparseGraphInverse. However, some controls share a temporal encoder, and the paper lacks a systematic analysis of training sample size. Therefore, the existing results do not sufficiently distinguish ambiguity inherent in the observations from estimation errors due to representation learning and training. The source-attribution bottleneck is the paper’s most important finding, and the incomplete explanation of its causes limits both its scientific significance and its generalizability.
- Although EC-TCN outperforms DCM on the SPM benchmark, it is not the strongest estimator in the reported comparisons. Table 7 shows that Random Forest achieves a higher AUPRC than EC-TCN (0.628 vs. 0.593), while Ridge obtains nearly identical magnitude estimation error (0.318 vs. 0.316) and better interval coverage (0.945 vs. 0.932). These results suggest that much of the performance improvement over DCM may come from replacing iterative inversion with supervised prediction rather than from the proposed architecture. More importantly, EC-TCN does not establish a clear advantage over substantially simpler estimators on its central connectivity recovery task. This weakens the methodological contribution and raises questions about whether the additional neural architecture and multi-head training design are necessary.
- The real-data experiment reveals a substantial limitation of the proposed approach. On the 63 admitted participants, EC-TCN and DCM exhibit almost no correlation in their edge-level estimates (participant-level ρ = 0.03; group-level ρ = −0.04), with sign agreement of only 0.51. This indicates that the two methods produce substantially different conclusions about which directed connections change and in what direction. Moving to mechanism-family comparisons does not resolve this issue, as EC-TCN favors bidirectional and forward modulation, whereas DCM favors the combined family. Furthermore, the comparison involves different statistical quantities: averaged classification probabilities for EC-TCN and random-effects model-selection probabilities for DCM. Consequently, the real-EEG experiment demonstrates computational applicability but does not establish reliable connectivity inference. This considerably limits the practical significance of the proposed approach for neuroscientific analysis.
Questions for authors
See weaknesses. Others:
- Compared with existing studies on amortized inversion of neural generative models and identifiability, what specific new insights does the distinction between target localization and source attribution provide?
- What is the main motivation for choosing EC-TCN and SparseGraphInverse? Do these architectures have properties necessary to study this problem, or do they serve as representative implementations of amortized inference? Given the existing baselines and architecture and decoder controls, which conclusions depend on the specific designs of these two models, and which can be generalized to other estimators?
7 comments
Response W2
We added analyses that separate ambiguity in the observations from estimation error, and they are reported in our response to Reviewer 8kAe (W1/Q1). Here is how they address each point.
Training sample size. We added fixed-N learning curves at 500 candidate connections, from 1,000 to 10,000 training cases, with five seeds (table in that response). With identical regional dynamics, source accuracy given the true target stays at the random-source baseline for both decoders. With regional spread 0.1, both improve with data: at 10,000 cases the pair decoder is 45 percentage points above the baseline and SparseGraphInverse is 7 points above.
Shared temporal encoder. The pair decoder is a different architecture. It processes source and target waveforms jointly before any pooling, and it still stays at the baseline with identical regions (+0.004 at 10,000 cases). We also trained a random forest on hand-built features (the mean and standard deviation over time of each region's waveform and firing rate), which uses no learned temporal representation. At 100 and 500 candidate connections, its source accuracy minus the random-source baseline is −0.005 and +0.008 with identical regions, and +0.072 and +0.025 with regional spread 0.1 (five seeds, 3,500 training cases). So the failure does not depend on the learned representation of SparseGraphInverse.
Ambiguity without a learned inverse. The nearest-signature analysis in the same response uses no trained network. Changing the source changes the simulated EEG by a median of 0.6%, about as much as redrawing the noise. Source recovery is 40% when the nuisance parameters are known exactly, against a random-source baseline of 14%, and falls to 13% at 2% error.
Estimator and measurement effects. When regional dynamics differ, more data and the pair decoder help, so the estimator matters there. The pair decoder differs from SparseGraphInverse in three ways (waveform comparison before pooling, the standard-condition firing rate, and sampling of competing-source negatives), so its advantage measures all three together. When the same simulations are passed through a template head model with sensor noise, the pair decoder's source gain disappears, so the measurement process is a further constraint.
For the architectures and training sizes we tested, the failure with identical regions reflects weak source information in the observation. With regional differences, recovery depends on the decoder, training size and the measurement process. We will state this in the updated PDF.
Response W3
We agree that EC-TCN is not the strongest estimator on every metric, and that the advantage over the evaluated DCM configuration extends to simpler supervised estimators.
Random forest has the highest detection AUPRC, ridge has near-nominal coverage and a magnitude error close to EC-TCN's, and none of the learned estimators separates the six connections from the base rate by more than 0.07. Much of the improvement over DCM therefore comes from replacing per-observation inversion with supervised prediction, and not from the EC-TCN architecture.
We tested whether the extra design choices help, and they do not. A graph-augmented variant of EC-TCN reached pooled AUPRC 0.404, well below EC-TCN's 0.593. In the Jansen–Rit experiment, removing the auxiliary six-class family head gave pooled AUPRC 0.920 against 0.905 with it, so the multi-head family training is not necessary for recovery.
Our contribution is the controlled characterization of what supervised amortized estimators recover from neural mass model data and where they fail. The baseline comparisons support this: improved gain recovery is not specific to EC-TCN, and across all estimators it does not extend to the extrinsic connections, a distinction hidden in the pooled score. We will present EC-TCN as one representative task-specific estimator and give the simpler baselines greater prominence in the updated PDF.
Response W1
We do not claim a new theory of identifiability. What the paper adds is a specific, testable failure in a neural inverse problem, plus diagnostics to detect it.
What the observation adds. A model can rank directed connections well and still not know which region sent the signal. At 1,394 candidate connections, edge AUPRC was about 12 times the base rate and target AUPRC was 0.992. Yet given the true target, the correct source was ranked first in 0.047 of cases, against a random-source baseline of 0.051. Finding the target alone moves every connection that enters that region up the ranking, including the true one. An edge-level score can therefore make directed connectivity recovery look better than it is. Our contribution is to show this in a neural mass model and to separate the two abilities.
Our experiments provide the following:
All the above indicate whether a recovery failure should be addressed by changing the generator, the decoder or the measurement process, before more is spent on the inverse model. We will state these guidelines explicitly in the updated PDF.