- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
- summary/contribution: The paper proposes "second-order learning" (SOL) for adaptive reasoning in graph-structured environments: an MLP controller maps the outputs and parameters of a frozen Graph Convolutional Network (GCN) to a parameter update in one forward pass, with no labels or gradients at test time.
- validation:
- The task is to label shortest-path nodes in perturbed 10×10 grid mazes and in an OpenStreetMap road graph with closures.
- Reported results: accuracy goes from 62.5% (unadapted GCN) to 80.2% and SOL is on par with MAML on an OOD split. SOL beats MAML on the road graph.
- summary: The paper also argues that GCN embedding distances correlate with maze distances (r ≈ 0.90), and reads this as support for the hypothesis that second-order learning produces representations isomorphic to the environment.
Critical strengths
- The question (amortized, label-free adaptation as an alternative to MAML) is relevant for meta-learning, and the paper reports adaptation cost, several seeds, and code.
Critical weaknesses
- The accuracies do not beat a majority-class baseline.
- On the road graph about 8% of nodes are positive (the Vanilla GCN has recall 1.0 and precision 0.082, Table 10). Predicting all negative gives about 92%, above SOL's 75.7%. SOL is also below MAML on recall, F1 and IoU.
- Shortest-path labels are not unique (hundreds to thousands of equally short paths at low p), so node accuracy is not well defined.
- The isomorphism analysis cannot support the causal claim.
- It is computed on the base GCN, which is frozen while the controller is trained (App. C.1, E.1), so second-order learning cannot have shaped it.
- The node features include the (x, y) coordinates and the reference is coordinate distance, so a high correlation is expected. Replacing the features with noise (Exp. 5) also removes task information.
- On an empirical level, the contribution is the initiated methodology, since the parameter choices in the main content are manual, e.g., Empirical study 1 and 2. On a theoretical level, the paper refers to this alignment as an "isomorphism" between the environment and cognition; however, the correlation metrics themselves do not constitute a formal proof of graph isomorphism in the strict mathematical sense.
Questions for authors
Clarifications:
- Can you report F1, IoU, path validity and a majority-class baseline for all maze experiments?
- Does a GCN trained without a controller on the same perturbed mazes, or a deeper GNN, match SOL? A 2-layer GCN sees two hops, while the controller sees all nodes at once.
- Which nodes form the support set for MAML and Reptile at test time, and do they overlap with the evaluation nodes?
- What is the isomorphism score for the raw input features, a random GCN, and the adapted parameters ?
- Clear sections, unclear proof for the correctness of table 1.
- Figure 1 - Does SOL lead to the appearance of this type of tissue?
Suggestions (not affecting my score):
- Relate the method to amortized meta-learning (LEO, CNAPs), label-free test-time adaptation (ARM, TENT) and neural algorithmic reasoning (Veličković et al., 2020).