- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies feature learning dynamics in graph contrastive learning (GCL). Under a sparse-coding model with neighborhood-correlated task-relevant features and weakly correlated task-irrelevant features, the authors derive training-dynamics results suggesting that neighborhood aggregation preferentially amplifies correlated features. Based on these insights, they propose Theory-Guided Neighborhood Augmentation (TGNA), which selects graph views according to a neighborhood-correlation criterion. Experiments on synthetic and real-world datasets demonstrate improved downstream performance and reduced training cost across several GCL frameworks.
Critical strengths
The paper tackles an interesting and underexplored question: how neighborhood aggregation affects feature-learning dynamics in graph contrastive learning. The distinction between strongly and weakly neighborhood-correlated latent features provides an intuitive perspective on representation learning in GCL.
The proposed TGNA plug-in is simple and broadly applicable. Empirical evaluation across multiple GCL frameworks and datasets suggests that the method can often improve downstream performance while reducing training cost.
Critical weaknesses
The key theoretical assumptions do not appear sufficient for the claimed selective feature-learning mechanism. Assumption 3.1 defines neighborhood correlation through latent support overlap, but the nonzero coefficients take values in {-1,1}; shared support therefore does not guarantee positive correlation and may instead cause cancellation under aggregation. In addition, the proof appears to treat the aggregated noise terms of the two augmented views as independent, although both views contain the same center node and thus share its noise. The paper should state the required sign-dependence assumption and explicitly account for the cross-view covariance induced by the shared center. These issues directly affect the feature-growth analysis underlying Theorem 3.1.
The connection between the latent theory and the practical TGNA method is insufficiently established. The theory relies on task-conditioned latent support-overlap parameters, whereas TGNA replaces them with cosine similarity of raw features without showing that the proxy preserves the relevant latent correlation. Moreover, TGNA simultaneously changes neighborhood selection, anchor retention, and the negative-sampling pool, so the reported gains cannot be attributed specifically to the proposed theoretical criterion. Matched controls using random and similarity-based sampling at the same anchor ratio, neighborhood size, negative-set size, and computational budget are needed to validate the “theory-guided” claim.
Questions for authors
-
Assumption 3.1 constrains overlap between latent feature supports, but the active coefficients can have opposite signs. What assumption ensures positive correlation between shared feature coefficients across neighboring nodes and augmented views? Would Theorem 3.1 still hold if the signs were conditionally independent given support overlap?
-
Although the two augmented views have disjoint neighbor sets, both include the same center node. How does the proof account for the resulting covariance between their aggregated noise terms? Can the authors show that the shared-center term is negligible under the theorem’s parameter scaling?
-
Under what conditions does raw-feature cosine similarity provide a valid proxy for the latent task-relevant support-overlap quantity α_1? Could the authors report correlations between the empirical TGNA score and the corresponding latent quantity in the synthetic experiments?
-
Can the authors provide matched ablations that separate the effects of the TGNA score, similarity-based neighborhood filtering, anchor-set reduction, and the changed negative-sampling pool?
7 comments
Response to Reviewer xBnC Part II
Q2 (Weakness 2): The connection between the latent theory and the practical TGNA method is insufficiently established. The theory relies on task-conditioned latent support-overlap parameters, whereas TGNA replaces them with cosine similarity of raw features without showing that the proxy preserves the relevant latent correlation. Moreover, TGNA simultaneously changes neighborhood selection, anchor retention, and the negative-sampling pool, so the reported gains cannot be attributed specifically to the proposed theoretical criterion. Matched controls using random and similarity-based sampling at the same anchor ratio, neighborhood size, negative-set size, and computational budget are needed to validate the “theory-guided” claim.
A2: We thank the reviewer for this constructive suggestion. We address the connection between the theoretical correlation criterion and its practical approximation, followed by additional ablations that disentangle the components of TGNA.
1. Theoretical motivation for using raw-feature cosine similarity.*
Under our sparse-coding model, the raw node features satisfy , where . Since the dictionary columns are orthonormal, the inner product between two node features can be decomposed as
+ \langle z_k^{(2)},z_j^{(2)}\rangle + \varepsilon_{kj},$$ where $\varepsilon_{kj}$collects the noise-related terms. Under the sign-consistency condition clarified in our response to Weakness 1, shared active latent coordinates contribute positively to the corresponding inner product. Thus, raw-feature similarity contains information about latent support overlap. When the contribution from strongly neighborhood-correlated features is sufficiently distinguishable from the contributions of weakly correlated features and noise, cosine similarity can serve as an observable proxy for latent neighborhood correlation. We emphasize that this does not imply that thresholding raw cosine similarity always isolates $M^1$-correlated neighbors. Rather, the proxy is motivated by the relationship between observable similarity and latent overlap under the considered data model. We further validate this relationship using synthetic data, where the latent features and correlation parameters are known. The resulting empirical correlation is strong (Pearson correlation 0.9814 and Spearman correlation 0.9970), supporting the use of this proxy in the considered setting. Our Experimental settings are $d=10, d'=20, d_1=200$, and additive Gaussian noise $\xi\sim\mathcal N(0,0.01I_{d_1})$, with (K=15) and $\rho_1=0.7$. We fixed $\alpha_2=0.2$ and \$\rho_2=0.25 \$while varying the latent feature-overlap ratio $\alpha_1$ from 0.1 to 1.0 . See Figure 1 at the link https://storage.googleapis.com/iclr2027_figures/ICLR27_Ablations.pdf **2. Ablation studies separating anchor selection and view construction.** TGNA involves two components: selecting eligible anchor nodes according to the proposed criterion and constructing augmented views based on neighborhood similarity. We agree that evaluating these components only in combination makes it difficult to identify their individual contributions. We therefore conducted two additional ablations using GRACE as the backbone: - **Anchors only:** Retain the anchors selected by TGNA, but use the baseline GRACE augmentation procedure to construct views. - **Views only:** Retain the original anchor set. Apply TGNA view construction to eligible nodes and baseline augmentation to the remaining nodes. The downstream classification results are summarized below. | Method | Cora | AmazonPhoto | AmazonComputers | PubMed | CS | |:---|:---:|:---:|:---:|:---:|:---:| | anchors_only | **84.24±0.31** | 91.91±0.17 | 83.18±0.22 | 85.67±0.05 | 92.01±0.15 | | views_only | 83.92±0.06 | 92.28±0.21 | 85.59±0.51 | **86.59±0.04** | 91.99±0.07 | | GRACE+TGNA | 84.04±0.17 | **92.51±0.23** | **85.69±0.29** | 86.10±0.22 | 92.36±0.05 | | GRACE | 83.54±0.64 | 91.29±0.19 | 82.23±0.25 | 85.90±0.04 | **92.65±0.22** | The results show that both components can independently contribute to improved downstream performance. Anchor selection alone achieves the best result on Cora, while view construction alone performs best on PubMed. Combining both components yields the highest accuracy on Amazon Photo and Amazon Computers. These findings suggest that TGNA's benefits are not solely due to anchor-set reduction, although the relative importance of the two components varies across datasets.Response to Reviewer xBnC Part III
Q3 (Question 1): Assumption 3.1 constrains overlap between latent feature supports, but the active coefficients can have opposite signs. What assumption ensures positive correlation between shared feature coefficients across neighboring nodes and augmented views? Would Theorem 3.1 still hold if the signs were conditionally independent given support overlap?
A3: As clarified in our response to Weakness 1 (A1), our intended data model assumes sign consistency on shared active latent coordinates, and we will explicitly incorporate this condition into Assumption 3.1.
If the signs were conditionally independent given support overlap, the shared feature contributions could cancel during neighborhood aggregation. Therefore, support overlap alone would not guarantee the selective feature amplification required by Theorem 3.1, and the theorem would not necessarily hold under the current assumptions. Please refer to A1 for the revised assumption and further details.
Q4 (Question 2): Although the two augmented views have disjoint neighbor sets, both include the same center node. How does the proof account for the resulting covariance between their aggregated noise terms? Can the authors show that the shared-center term is negligible under the theorem’s parameter scaling?
A4: We thank the reviewer for this question. Please refer to our response to Weakness 1 (A1),
Q5 (Question 3): Under what conditions does raw-feature cosine similarity provide a valid proxy for the latent task-relevant support-overlap quantity α_1? Could the authors report correlations between the empirical TGNA score and the corresponding latent quantity in the synthetic experiments?
A5: We thank the reviewer for this question. Under our sparse-coding model, , where is an orthonormal dictionary, we have
=\langle z_k^{(1)},z_j^{(1)}\rangle + \langle z_k^{(2)},z_j^{(2)}\rangle + \varepsilon_{kj},$$ where $\varepsilon_{kj}$ collects the noise-related terms. Under the sign-consistency condition clarified in A1, shared active latent coordinates contribute positively to the inner product. When the contribution from $M^1$ is sufficiently distinguishable from those of $M^2$ and dense noise, raw-feature cosine similarity $s_{kj}=\langle x_k,x_j\rangle/(|x_k|_2|x_j|_2)$ can serve as a practical proxy for latent neighborhood correlation. **Synthetic validation.** We conduct controlled experiments using the sparse-coding model with $d=10, d'=20, d_1=200,$ neighborhood size $K=15, \rho_1=0.7$, and Gaussian noise covariance $0.01Id_1$. We vary the latent support-overlap parameter $\alpha_1$ from 0.1 to 1.0 while keeping the remaining parameters fixed. For each configuration, we compute the empirical neighborhood similarity $\hat{\alpha}$ by averaging the raw-feature cosine similarities $s_{kj}$ over the selected anchor-neighbor pairs, and compare $\hat{\alpha}$ with the corresponding ground-truth $\alpha_1$. The resulting correlations are Pearson 0.9814, Spearman 0.9970, and R2=0.9632. See Figure 1 at the link https://storage.googleapis.com/iclr2027_figures/ICLR27_Ablations.pdf. **Additional validation of the full TGNA criterion.** We will further compare the empirical TGNA score $\hat{S}_{\mathrm{TGNA}} = \hat{\alpha}\hat{\rho}^2(\hat{K}+1)$ with the corresponding latent criterion $S_{latent} = \alpha_1\rho_1^2(K+1)$, Our model configuration settings for this experiment is $d=10, d'=20, d_1=200$, and additive Gaussian noise $\xi\sim\mathcal N(0,0.01I_{d_1})$, with K=15. We fixed $\alpha_2=0.2$ and $\rho_2=0.25$ while varying the both latent feature-overlap ratio $\alpha_1$ and the fraction of simila neighbors $\rho_1$ from 0.1 to 1.0 . The resultant correlations are Pearson 0.9776, Spearman 0.9798 and R2=0.9557. See Figure 2 at the link https://storage.googleapis.com/iclr2027_figures/ICLR27_Ablations.pdf **Q6 (Question 4):** Can the authors provide matched ablations that separate the effects of the TGNA score, similarity-based neighborhood filtering, anchor-set reduction, and the changed negative-sampling pool? **A6:** We thank the reviewer for the constructive suggestion. To disentangle the effects of TGNA's anchor selection and view construction, we conducted two additional ablation studies, Anchor selection only and View construction only, as discussed in A2 to weakness 2. Regarding negative sampling, TGNA does not introduce a separate negative-sampling algorithm. However, restricting the anchor set can change the set of representations participating in the contrastive objective. We will clarify this distinction and the corresponding implementation details in the revised manuscript.
Response to Reviewer xBnC Part I
Q1 (Weakness 1): The key theoretical assumptions do not appear sufficient for the claimed selective feature-learning mechanism. Assumption 3.1 defines neighborhood correlation through latent support overlap, but the nonzero coefficients take values in {-1,1}; shared support therefore does not guarantee positive correlation and may instead cause cancellation under aggregation. In addition, the proof appears to treat the aggregated noise terms of the two augmented views as independent, although both views contain the same center node and thus share its noise. The paper should state the required sign-dependence assumption and explicitly account for the cross-view covariance induced by the shared center. These issues directly affect the feature-growth analysis underlying Theorem 3.1.
A1: We thank the reviewer for identifying these important points.
1. Sign-consistent neighborhood correlation. We agree that support overlap alone does not guarantee positive feature correlation. Our intended data model assumes that shared active latent features have matching signs, rather than independently resampled signs. We will explicitly incorporate this condition into Assumption 3.1 as follows.
Assumption 3.1 (Neighborhood structure and sign-consistent feature correlation). For every center node vk, the two augmented views satisfy $ |\mathcal N_A(k)|=|\mathcal N_B(k)|=K,\qquad \mathcal N_A(k)\cap\mathcal N_B(k)=\varnothing. $ For each q∈A,B and r∈1,2, there exists Nq,r(k)⊆Nq(k) such that $ |\mathcal N_{q,r}(k)|\geq\rho_r K, \qquad |S_k^r\cap S_j^r|\geq\alpha_r|S_k^r|, \quad \forall v_j\in\mathcal N_{q,r}(k), $ with the additional sign-consistency condition $ z_{k,l}^r=z_{j,l}^r,\qquad \forall l\in S_k^r\cap S_j^r. $ The correlation parameters satisfy α1ρ12>α2ρ22.
2. Cross-view covariance induced by the shared center node.
We thank the reviewer for pointing out that the two augmented views share the same center-node noise and are therefore not independent, even though their sampled neighborhoods are disjoint. We agree that this cross-view dependence should be explicitly accounted for in the analysis. Specifically, the shared center node introduces the following nonzero covariance term: $\frac{\sigma^2}{(K+1)^2} \langle w_i,M_j\rangle. $ Under our theoretical scaling K=Θ(logd), this term is o(1), provided that σ2⟨wi,Mj⟩=o((logd)2). We will revise the proof to explicitly account for this cross-view covariance and establish that its contribution is negligible relative to the leading feature-growth terms, rather than assuming independence between the two views.