- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper revisits long-range dependency in graph implicit models, using an aligned learning-dynamics surrogate for the simple chain task and a vanishing-influence bound under contraction and sensitivity bounds uniform in graph size. AdaGIM adds per-node adaptive selection: a separate selected state retains intermediate representations while propagation continues, using a learned gate and residual factor.
Critical strengths
- The paper distinguishes vanishing influence from failure to classify. Its parity result concerns exact scores or a fixed positive margin, while correct classification with a vanishing margin remains possible. It also clearly limits the simple-chain result to an aligned unconstrained surrogate. These distinctions help interpret what the benchmarks actually demonstrate.
- The selection mechanism has a clear mathematical explanation. Under contractive propagation, the residual gate gives finite total write mass, allowing earlier states to retain weight. The paper also distinguishes the initialized trajectory limit from a unique algebraic fixed point and explains finite-unroll differentiation.
Critical weaknesses
- The main benchmark results need stronger validation. Table 1 uses a reduced training protocol, and the paper acknowledges unresolved seed counts, displayed spreads, and tuning allocations. This makes it difficult to judge whether the reported gains are reliable and the comparisons are fair.
- The benefit of last-write selection is not fully isolated. The JK-style baseline uses only trajectory averaging, leaving stronger learned readouts untested. In Table 3, the full gate beats MLP-only gating on only one of three datasets. Given the substantial memory overhead, comparisons with matched-capacity and matched-computation alternatives would help establish the mechanism’s practical value.
- The computation cost and memory cost of the proposed method are high.
Questions for authors
- With the same propagation, parameter budget and trajectory length, how does last-write selection compare with learned JK-style attention and node-adaptive stopping/readout? Does a node-shared gate work similarly?
Thank you for the feedback.
Benchmark validation (W1)
We reran all ten models with the same input features and 20 tuning attempts per model and dataset. The validation-selected configurations are evaluated over five seeds, reporting mean ± sample standard deviation (Table R1 in our reply to Reviewer Zq9j). AdaGIM achieves the best mean on both Peptides tasks. We will release the configurations, per-seed results, and logs.
A separate comparison in Appendix G.1 uses approximately 500K parameters and 250 training epochs. AdaGIM achieves 0.6641 Peptides-func AP versus 0.6327 for GCN, supporting its effectiveness with longer training.
Learned readouts and shared gates (W2 and Q1)
We added learned JK-attention, JK-LSTM, and per-node ACT-style halting. These variants are trained separately end to end using the same backbone architecture, configuration, maximum propagation budget, and training schedule as AdaGIM. JK-attention uses the same scorer architecture and essentially the same parameter count. The long-solve baseline receives up to 100 iterations with tolerance 10−5.
Table R4. Readout comparison. Seeds 0–2; means and reported variability. Parameter counts are for Peptides-func; bold denotes the best mean.
Training and inference cost (W3)
There is a standard technique to reduce the memory usage. Gradient checkpointing reduces peak training memory on both Peptides tasks by approximately 76%, to about 1.5 GB. It saves memory by recomputing intermediate states during backpropagation.
Table R5. Cost on A40 GPUs. IGNN values are in parentheses; Tmax is the maximum propagation budget per core. Inference time is averaged per graph within a batch.
Checkpointing uses 1.48× and 1.61× the training time on Peptides-func and Peptides-struct, respectively. Batched inference averages 0.24–1.08 ms per graph in the tested configurations.
Our contributions are a theoretical analysis of long-range limitations under contractive propagation and the general AdaGIM framework, which separates propagation from adaptive selection. The experiments demonstrate the value of retaining intermediate representations for long-range prediction. Designing more efficient models within this framework, with lower training time and memory requirements, is a promising direction for future work.