- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper asks whether representations measured on a small shared set of inputs can predict performance across neural networks. The authors compare three representation summaries and use regression to predict behavioral scores, with experiments on vision models, language model checkpoints, and mice. The main results concern generalization to held-out architectures or model sizes, and how prediction depends on the inputs used to measure representations.
Critical strengths
I find the vision experiment interesting. Training on CNNs and predicting the accuracy of held-out non-CNN models is a meaningful test, and the reported performance using only 200 images is positive. The experiments varying stimulus number and content are also useful. In particular, the difference between natural images and noise suggests that the choice of probe inputs matters.
Critical weaknesses
-
I think the paper needs stronger baselines to establish what the representation measurements add. For vision, how well would model size and training information predict accuracy? For OOD performance, I would particularly like to see a comparison against ImageNet accuracy alone. For language, model size, training step, and validation loss seem important. The checkpoints come from nine sizes of the same Pythia models, so I am unsure how much of the prediction reflects shared training progress (there could be confounders). In Figure 4, some subject–verb agreement predictions have reasonably high R^2 but almost zero or negative rank correlation. This makes me wonder whether they mainly capture the initial improvement and subsequent plateau. The finding in Appendix C.6 that nearest neighbors often outperform ridge makes this question more relevant.
-
I am also unsure how much methodological novelty to attribute to the framework. Williams et al. (2021) already used regression on representation embeddings to predict network test accuracy. The authors cite this work, but I would appreciate a more explicit account of what the proposed approach adds. I think the contribution here may lie mainly in the broader experiments and transfer settings. If so, comparisons against prior approaches would help establish the significance of those results.
Questions for authors
-
Does representation geometry improve prediction beyond model size and training progress, and beyond ImageNet accuracy when predicting OOD performance? For the language experiments, does useful predictive performance remain when the earliest checkpoints are excluded?
-
What is the main advance over the representation-based performance prediction in Williams et al. (2021)? Can the authors compare against that approach under the same architecture or scale splits?
Dear authors, it looks like you haven't responded to reviewer concerns yet. As the rebuttal phase is nearing its end, this is your last chance to respond - alternatively, I would suggest to consider withdrawing the paper if you don't believe it has a chance of acceptance.
Thank you,
Your AC