- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper looks at two 'sanity-check' measures to help validate similarity metrics when it comes to interpreting brain-DNN model alignment. The two features are region-matching and feature specificity. The authors show that region-matching exposes a varying set of factors that can have a dramatic impact across benchmarks, and therefore becomes a primary focus to help serve as a base from which further analyses and conclusions can be made. In other cases, other metrics would give artificially high scores while the region-matching baseline is low, showing an artificial inflation of performance that the region-matching metric would surface.
I like papers that take aim at consolidating and applying a scientific battering ram to preconceived notions and which try to reinforce the interpretations many papers claim to make on a much more shaky ground. Many papers try to do this in pieces, with either too large and incomplete goals, or really specific claims that don't directly affect a lot of the research. This paper finds that middle-ground, I feel, in being only large enough to have good analyses contained within it and for it to tell a coherent story without overcomplicating the issue.
The submitted paper also has a nice overview of the literature. I thought I spotted a few gaps sometimes, but when searching for the principal authors, I did find the citations in a related section, so I am satisfied with that aspect. I also appreciated the (brutal) honesty of the limitations section, which increases my confidence in the scientific considerations of the authors and the overall submission.
I think it might be a bit of a stretch to imagine a world where all brain-DNN similarity results have an expected region-matching/feature-specificity result to be done as a baseline, but if not, I can imagine it being a diagnostic test as to whether something "unexpected" might be going on in some other spurious analysis. I think a lot of work went into thinking about this and it's on-topic for the themes of ICLR over the past few years and would be well-received by many of the participants.
Critical strengths
The main strengths of the paper is the presentation of a suite of alignment metric performances in relation to the proposed sanity checks. Scientists working in the field have something tangible to consult and look at varying metrics across varying dimensions. It will cause pause for thought, consideration on preprocessing methods, extra testing, relying not just on single pieces of evidence to make strong (incorrect) conclusions. It also brings together a lot of the recently used methods critiquing the often (bad, IMHO) practice of just fishing for high-alignment scores and making faulty post-hoc deductions about what brain-model alignment actually means.
It's a dense piece of work and I was originally quite irritated by the length of the supplementary material (almost 6 times the length of the main submission) and obviously I couldn't give it the attention it deserved as I only have scarce review time and I budgeted for about 10-ish pages of supplementary material for each paper. Most of the titles I did scan, though, did seem to be more in terms of completeness and carefully stating some assumptions and I could understand a bit better why the authors felt this was necessary.
Critical weaknesses
When such papers aim to showcase a better method and identify how other methods can be unreliable, it's usually very informative to show how previous published research has been incorrect and to help us course correct a bit better in identifying any spurious conclusions from faulty analyses. A great way for this submission to solidify its relevance and applicability would be to identify 2+ examples where region-matching, applied to a previous analysis, shows that previous results do not hold and that these sanity benchmarks would have stopped the analysis from preceding. I can appreciate it's difficult to 'call-out' other papers and directly target an analysis and claim it's based on shaky principles, but this is the goal of science and my worry is that this trend is allowing incorrect conclusions based on shaky analyses to permeate our field and it's becoming more and more difficult to separate out the signal from the noise.
You had a great chance here to directly show how, had your method been applied to previous works, that it would have raised a red flag earlier in the process. Also, it would give us a direction for which styles of analyses (specific metrics etc.) were being used in research that should come under more scrutiny (by not passing the sanity checks you propose). It's a bit of an oversight to not include something about this in the paper to strengthen the point I feel the authors wanted to make. There's a tiny bit of this, with Thobani and Finzi papers, but it's weak and stuffed into the (gigantic) appendix.
Questions for authors
-
What thoughts are there to use potentially cheaper and lighter assumptions for a similar method that can be more generalised and not so engrained within specific contexts?
-
In your discussion you argue that things like the specifics of the DNN architecture are an "orthogonal concern", which surprised me and I think I'd be tempted to disagree with as it's so key to the interaction with some of the alignment techniques. In fact, you do specify a bit about this with the Layer Normalization point in the appendix, removing mean activity from previous layers and how this interacts with the univariate signal in e.g. fMRI. -- I feel like you're trying to answer a question you have, but without saying a lot, and then claiming it's an orthogonal question in the discussion to pre-empt a (potential) criticism that enough thought hasn't gone into that topic. My question is - is that the case?
-
If you were to take a careful analysis of the current leaderboards like Algonauts or Brain Score and apply these metrics to each of the entries there, do you imagine the rankings would stay moderarely intact, or fall to pieces? significantly re-order? What (informally) are your suspicions of what this would do? (I'm trying to get a sense of how disruptive you think your proposal would be to the state of current research, and more disruptive is not a bad thing at all).
Dear Reviewer,
Thank you for taking the time to review our paper. We are delighted that you found our paper to apply “a scientific battering ram to preconceived notions”, to “have good analyses contained within it and [telling] a coherent story without overcomplicating the issue” and believe it would be “well-received” by many ICLR attendees.
Q: “A great way for this submission to solidify its relevance and applicability would be to identify 2+ examples where region-matching, applied to a previous analysis, shows that previous results do not hold.”
A: Thank you for this suggestion. We refer to our general comment for additional experiments that we ran. In short, we find that poorly identifiable regions indeed give less stable model rankings. Thus, we hope our analysis prompts more caution in interpreting benchmarking results with fine grained region sets. .Region matching can tell in advance which regions a dataset cannot distinguish, so that they can be merged or benchmarked jointly instead. We expect this to give less fine-grained but more robust benchmark results. We also hope to raise awareness of what is needed to increase this 'resolution' of benchmarks.
Q: “What thoughts are there to use potentially cheaper and lighter assumptions for a similar method that can be more generalised and not so engrained within specific contexts?”
A: Thank you for this interesting question. We believe that our assumptions are explicit and relatively well-defined in the context of computational neuroscience, which is the main context in which the investigated measures are applied. Regarding costs of evaluating this check, want to point out that region-matching uses the same (training) dataset as the main benchmark would, and that runs are generally cheap - in particular compared to model-brain benchmark evaluations which require running DNN models on the stimulus set.
Q: (paraphrasing) “[Are the specifics of the DNN architecture really an orthogonal concern]?“
A: ‘Orthogonal concern’ refers to how our check is motivated. In brain-brain comparisons, we can define a benchmark-specific ‘ground truth’ in that we expect the same region to score the highest similarity. In DNNs, we can define a ground truth: We can test whether a measure assigns highest similarity to networks with the same architecture, but with different seeds, or to layers of the same depth. However, these networks might have found different solutions for solving the same problem. In both these tests and in region-matching, we pre-impose what we want to be scored as most similar. What differs is that in our setting, choosing regions as ground truth tests an assumption that is implicit in the benchmark. While these are clearly valuable and interesting experiments, there is no equivalent justification for any of the matching criteria in DNNs. This is what we mean by ‘orthogonal’. We here aim for an empirical evaluation of benchmarks against some kind of ground truth, which we believe can only be found convincingly on the brain side."
Q: “If you were to take a careful analysis of the current leaderboards like Algonauts or Brain Score and apply these metrics to each of the entries there, do you imagine the rankings would [change drastically]?”
A: Thank you for the interesting question. While we have not investigated the question in this work, it would constitute an interesting direction for further research. We do indeed speculate that changing the similarity measures underlying current leaderboards could drastically change their rankings.
References
[1] Finzi et al., 2023 - https://doi.org/10.1101/2023.12.19.572460 [2] Sartzetaki et al., 2025 - https://openreview.net/forum?id=LM4PYXBId5