- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Existing graph contrastive learning methods rely on optimizing contrastive objectives, but excessive optimization can degrade the learned representations. To investigate this problem, this paper theoretically analyzes the topology-independent optimal geometry of InfoNCE and the impact of structural degeneration on structure-dependent downstream tasks. Based on this, the authors propose Progress-Adaptive Regulation (PARE) to mitigate contrastive overfitting. Experiments across six datasets and nine GCL methods demonstrate the effectiveness of PARE in mitigating contrastive overfitting.
Critical strengths
- The identified problem is important and practically relevant. This paper investigates contrastive overfitting in graph contrastive learning and provides empirical evidence demonstrating that excessive contrastive optimization can degrade downstream performance.
- The experimental evaluation is extensive and demonstrates the effectiveness of PARE. The authors evaluate PARE across six datasets and nine InfoNCE-based GCL methods, including GRACE, GCA, PiGCL, ProGCL, and ReGCL. The experimental results show substantial improvements in downstream node classification performance, demonstrating the effectiveness of PARE in mitigating contrastive overfitting.
Critical weaknesses
-
The theoretical connection between contrastive optimization and structural degeneration remains insufficiently established. The authors state that “The topology-independent optimum in Theorem 1 suggests that structural information may be lost during contrastive optimization.” However, Theorem 1 only characterizes the topology-independent optimal geometry of the InfoNCE objective, without establishing how approaching this optimum leads to the progressive loss of structural information during training. Therefore, the connection between the topology-independent optimum and structural degeneration remains unclear, and the theoretical analysis does not sufficiently support the claimed mechanism of contrastive overfitting.
-
Insufficient alignment between the identified problem and the proposed method. The paper attributes contrastive overfitting to the lack of explicit structural preservation in the InfoNCE objective. However, the proposed PARE method does not directly incorporate graph structural information or impose structural preservation constraints. Instead, it regulates parameter updates based solely on contrastive loss reduction and representation drift. Therefore, it remains unclear whether PARE genuinely addresses structural neglect or merely alleviates performance degradation by suppressing excessive optimization.
-
The evaluation of structural dependence is limited and does not fully support the generality of Theorem 2. The authors define structural dependence in Equation (3) as the label information provided by graph structure beyond node attributes. However, in the experiments on real-world datasets, they use the supervised performance gap between GCN and MLP as a proxy for structural dependence. These two quantities are not equivalent. Furthermore, although the paper discusses structure-dependent downstream tasks, the experimental evaluation is limited to node classification. The authors should consider additional downstream tasks that strongly depend on graph structure to further validate the generality of their conclusions.
-
The evaluation protocol may overstate the practical benefits of PARE. The authors train all baseline methods for 10,000 epochs. However, some methods, such as GRACE, may already achieve favorable node classification performance after 500 or 1,000 epochs. Continuing training until 10,000 epochs may amplify the observed performance degradation. The authors should provide additional comparisons using the original training configurations of the baseline methods to better assess the practical effectiveness of PARE.
Questions for authors
- Could the authors further clarify how the topology-independent optimal geometry characterized in Theorem 1 leads to the progressive loss of structural information during contrastive optimization? What theoretical evidence supports this connection?
- PARE regulates parameter without explicitly incorporating graph structural information, could the authors clarify how PARE mitigates structural degeneration rather than merely suppressing excessive optimization?
- Why are all baseline methods trained for 10,000 epochs rather than following their original training configurations? Could the authors provide comparisons using the recommended training schedules or appropriate early-stopping strategies to better demonstrate the practical effectiveness of PARE?