- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors observe that when test samples arrive grouped by class, exploiting relationships among samples can yield substantial gains; however, when samples are randomly mixed, graph propagation may lead to significant performance degradation. Therefore, relationships among samples cannot be assumed to always be beneficial. To address this, the authors propose GRASP, adjusts the connection weights between samples according to the agreement between their predictive distributions, and uses a fidelity constraint to limit the changes that graph propagation makes to the original predictions. The aim is to exploit helpful relational information while reducing interference from unreliable relationships.
Critical strengths
The authors observe that when test samples arrive grouped by class, exploiting relationships among samples can yield substantial gains; however, when samples are randomly mixed, the same propagation mechanism may lead to severe performance degradation. This observation has practical significance because, in real-world deployment, it is generally not possible to determine in advance whether local relationships among samples are reliable.
Critical weaknesses
The ablation results for the key design components and the method’s performance under different settings still require further analysis. These issues directly affect my assessment of the necessity of the method’s design.
Questions for authors
-
Have the authors considered how prediction uncertainty affects JS? The ablation experiments in the appendix show that the average gain from JS is relatively small.
-
Based on Table 1, the method appears to have limited performance stability under IID and in the Cross-regime(Worst), with relatively large performance fluctuations across different settings.
-
The ablation experiments show that the anchored random-walk yields the largest improvement under IID. Could the authors provide further analysis of this result?
-
The paper appears to involve quite a few hyperparameters. Is the method somewhat engineering-heavy?
-
Some symbols are explained only after they first appear. Even if readers can infer the meanings of the symbols in the equations, these meanings should still be explicitly defined at their first occurrence. In terms of presentation, the figures should be made more consistent. The fonts and font sizes used in the figures throughout the paper are quite inconsistent, which reduces readability.
2 comments
Dear Reviewer WnPJ,
Thank you again for your review. With the discussion period closing on November 18, we would be grateful to know whether our responses and the revised PDF resolve your concerns.
Specifically, we explained how the JS gate behaves when predictions are uncertain, reported IID accuracy on every dataset, and added an ablation of each component at the same IID cost, which shows that GRASP's gain is not only a strength effect.
We are happy to clarify anything further if that would help.
Thank you. We address each point below, with new results or changes to the paper. Several answers compare methods at the same IID cost: we vary each method's strength and compare them where each loses the same IID accuracy, so that design is not confused with strength (see the general response).
Q1: Uncertainty and JS.
When both predictions are confident, JS reliably tells same-class from different-class neighbours; when both are uncertain, the gate steps back and GRASP stays safe (App. A.6).
Q2: IID stability
GRASP's IID loss is small and measured on every dataset.
Q3: Why the anchored random walk helps IID most? That row lowers the propagation strength, compared at equal IID cost, each component still contributes.
Q4: Is the method engineering-heavy?
No: one linear solve per batch, three effective hyperparameters, and one setting for everything, not tuned on the test streams.
Q5: Presentation.
Thank you, both points are fixed. Every symbol is now defined at first use: Fig. 1's caption points to Eq. (13) and names η and L, and the background defines λ, W, D, B, C and P_cache. All plots now use one font family and matching sizes. Table 2 is replaced as described above.