- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the graph classification problem with limited observation budget - in where the learner/reader can only observe limited number of nodes (within the budget) sequentially, thus requiring a strategic node selection policy and a graph prediction policy. The authors propose a model to jointly output both policies. The model is trained through imitation learning and reinforcement learning with reward function defined by the authors. The authors demonstrate the the performance of the proposed method through experiments in various datasets and ablation studies.
Critical strengths
The paper is generally well written with clear structure and flow. The methods are natural and well motivated for the proposed setting. The results clearly demonstrate the improvement of the proposed method over the baselines.
Critical weaknesses
To the best of my knowledge, the authors did not describe what pretrained frozen classifier they use or how it is trained. Given that this pretrained classifier serves as a critical component for RL training, I believe missing a detailed introduction for it is a critical weakness. I’m particularly concerned with two things: 1, the data size and the data coverage assumption needed to train this classifier. 2, Can the data used to train such classifier already (or has the potential to) give a selection mechanism? Both questions can be answered by providing detailed descriptions of this pretrained classifier.
Questions for authors
The main question I have is about the motivation of this work. For example, in the image classification example the authors mentioned in the first graph of Introduction, why is running an image encoder on a certain crop costly, and more importantly, why can’t one use the whole image as the input, as I think this is the conventional setting for this task? I believe adding more illustrations on this budget-limited setting is essential for readers to understand the importance of the proposed method.
Response to Reviewer phri
We thank the reviewer for the positive feedback on the paper's clarity, structure, and empirical gains, as well as the constructive questions on the pretrained frozen classifier and motivation.
W1. Detailed Specification of the Pretrained Frozen Classifier f^
All three frozen classifiers use a 5-layer GINE (
hidden_dim = 300,dropout = 0.5), trained by cross-entropy strictly on the official/standard training split of each dataset (zero external data and zero validation/test leakage), and are discarded at deployment:meanmaxpooling, 2-layer FFN head, label smoothing0.110,000test held out)499val /501test held out)45,100val /34,800test held out)Data size and coverage assumptions (and removing f^ from RL):
55.73%vs.60.50%) still yields20.68 / 23.59 / 25.35%, beating every fixed baseline (16.32 / 19.03 / 21.36%). Moreover, in a new controlled ablation removing f^ from RL entirely (single-stageReader-onlyRL rewarded solely by BAGR's own predictor), BAGR achieves 29.16 / 35.60 / 39.20% on CIFAR10, 55.99 / 54.69 / 56.89% on COLLAB, and 26.90% on PPA—and evenScratch-RLwithout BC reaches27.11 / 32.30 / 34.79%on CIFAR10. Thus, f^ is only an offline teacher-filtering aid rather than a bottleneck.Can f^ or its training data directly provide a selection mechanism?
SAGPool/Soft-Maskbaselines, which trail BAGR by15–21pp on CIFAR10 (18.10%/21.07%even under f^, vs. BAGR's36.55–39.20%) because full-graph GNNs are not trained to route under masked attributes.Q1. Motivation of Budget-Limited Reading and the Image-Crop Example
You raise a fair point about the Introduction's opening example: