- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces GRACE, which aims to alleviate optimization instability caused by pseudo-label overconfidence in graph-based model testing prompt tuning. The proposed method consists of two core designs. One is to improve the reliability of pseudo-labels by fusing multiple layers of evidence using Gaussian class prototypes and covariance-aware distances of each layer. The second is to predict stable target nodes by graph perturbation screening, and update hints using symmetric parameter perturbation in a fixed supervision state. Experiments show that GRACE can improve the adaptation performance compared with other baselines.
Critical strengths
-
This paper focuses on pseudo-label error accumulation and optimization instability, and the connection between the problem and the method design is relatively clear.
-
GRACE according to the performance of support nodes under graph perturbation, a category-dependent selection criterion is established, which considers distance stability, prediction flip, and confidence. The design makes full use of the limited supervision information of the target domain, and has good interpretability.
-
The experiments are validated on several datasets and the number of experiments is rich.
Critical weaknesses
-
Appendix A.1 shows that the paper only keeps one supporting node per class, but splits the remaining nodes into validation and test sets 1:9, without specifying whether the validation labels are used to select hyperparameters, adaptation rounds, or model configurations. If these labels are used, the actual labeling budget is more than "one per class". if not, explain what the validation set is for and how the parameters are determined. My question is whether there will be a label leak problem?
-
The paper states that the experiments follow the setup of GFMate, but it is not clear whether the two share the exact same pre-training checkpoints, support sets, and test sets.
-
Only five runs are not enough to fully support the statement "consistently outperforms the baseline on all datasets". For example, Squirrel's improvement is only 0.45 percentage points, and Cora's improvement is 0.71 percentage points, which corresponds to a large inter-run standard deviation.
-
Missing some classical baseline comparisons of GFM, such as Unigraph, AnyGraph?
Questions for authors
Most of my questions focused on experimentation, so read the weekness section and respond.