- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates why quadratic features improve graph-based semi-supervised classification by separating reweighting within the original feature space, complementary feature directions, and coupling between the two components. It establishes an exact linear-feature interpretation of projected products and constructs connected graphs where Full and Parallel make identical predictions, although Decoupled achieves perfect classification. Experiments on three image graphs examine these effects under matched controls and stronger model selection. The paper also develops signed margin bounds for assessing prediction stability along the entire coupling path.
Critical strengths
-
The classification counterexample is the strongest contribution. It demonstrates that identical predictions can conceal useful complementary features, with a nonvanishing error separation as the labeled fraction decreases.
-
The matched controls are carefully designed. In particular, the common free intercept addresses the constant-direction confound, and the experiments distinguish controlled interventions from independently selected procedures.
-
The whole-path analysis provides information that endpoint comparisons miss, supported by concrete examples of predictions changing and subsequently returning to their initial class.
Critical weaknesses
-
The main theorem establishes an existence result on a specially constructed graph. It does not characterize when harmful coupling should arise on ordinary data, leaving the connection between the construction and the benchmark observations incomplete.
-
Empirical generality remains limited. All three graphs participated in development, and the stronger selection experiments mainly use three trajectories. The small and mixed gains after selection, together with the sign reversal in the expanded MNIST experiment in Appendix C.7, weaken confidence in practical robustness.
-
Numerical validation of the adaptive diagnostic is incomplete. Appendix B.7 assesses cutoff sensitivity for direct signed bounds, but the refinement-only cases lack retained final interval bounds. The relationship between sensitivity flags and prediction errors is also not established.
Questions for authors
-
Which properties of the observed feature geometry or regularization settings explain why the complementary-block benefit persists on CIFAR-10 but reverses on Fashion-MNIST under shared selection?
-
How were the reference geometry and penalties chosen, and how consistently does the cancellation pattern appear across the existing factorial geometry study?
-
Can the authors report accepted interval-bound margins or tolerance sensitivity for the refinement-only diagnoses contributing to the 98.20% coverage?
2 comments
Thank you for the additional analyses. My remaining reservation is about scope. The results show that coupling can mask useful complementary features, but they do not yet tell us when to expect this on a new graph. I agree that a general predictive criterion is beyond what the present evidence establishes.
Thank you for identifying the missing refinement bounds and the questions about geometry and replication. We have recovered the minimum accepted bounds for all 11,122 diagnoses obtained only after subdivision, crossed geometry with penalty choices on the support driving the original reversal, extended shared selection to ten trajectories, and linked the diagnoses to prediction errors.
Geometry and reference settings (§3, p. 3; C.10, pp. 28–29). We examined the Fashion-MNIST and CIFAR-10 supports at 40 labels per class, seed 20270104, because their previously observed complementary effects differed. On this Fashion-MNIST support, changing from G64 to G128 reverses Decoupled−Parallel under both penalty choices: from +0.065 to −0.246 percentage point at reference penalties, and from +0.243 to −1.967 with the Base-selected tuple. Base selected G128 with both feature penalties equal to 0.003, τ=1 and σ=0.001. It chose G128 on 20/30 Fashion-MNIST supports, including this one.
G128 changes rank and diffusion time together: it doubles the rank and halves the time. The selected tuple also changes more than one penalty. The crossed comparison localizes the observed reversal to the geometry change and shows its amplification under the selected tuple; it does not isolate rank, diffusion time or one penalty, nor provide a general predictive rule. All four corresponding CIFAR-10 effects remain positive (+0.485 to +0.861). The complementary share of squared quadratic feature norm increases on both graphs, so that share alone does not explain their different signs.
G64 and the reference penalties were development settings, and all three image graphs participated in development. Their original selection criterion is absent from the retained records. We therefore treat them as development choices rather than independently validated optima and examine sensitivity to their variation. Across nine fixed-penalty geometries, ten trajectories and three budgets, addition and recoupling have opposite mean effects in 8/9 MNIST settings and 9/9 settings on each of Fashion-MNIST and CIFAR-10. Five geometries per graph retain part of the gain after recoupling.
Replication and the link to the theorem (C.7, C.10–C.12). Across ten shared-selection trajectories on CIFAR-10, adding the complement improves mean accuracy and sharing coefficients reduces that gain; nine retain a net gain over Parallel. Fashion-MNIST's mean complementary gain is near zero, with an interval spanning zero. Independent selection still has three complete trajectories per graph. The ten-seed MNIST extension at five labels per class gives a mean Full–Base difference of −0.123 percentage point, with an exploratory bootstrap interval [−1.078, +0.536]. It does not establish a consistently positive enrichment gain. Matched Cora and C. elegans experiments establish occurrence outside image graphs under two edge constructions, without establishing prevalence across networks.
The connection to the theorem now reaches individual decisions. Among 221,184 query/configuration evaluations, Decoupled corrects a common Parallel/Full error in 741 cases; the reverse pattern occurs 369 times. In 325 of the 741 cases, the complementary and cross terms are each necessary for the respective margin signs at fixed fitted coefficients. C.11 separates these contributions from changes caused by refitting. A general criterion predicting harmful coupling remains open.
Refinement bounds and prediction errors (C.11, pp. 29–30). For each of the 11,122 refinement-only diagnoses, we recovered the minimum bound over all accepted intervals and rival classes. The smallest bound is 3.25×10−9 and the median 4.93×10−3. With the partitions fixed, cutoffs of 10−6 and 10−4 retain 11,101 and 10,584 of these diagnoses. After including the direct diagnoses that survive each cutoff, the respective overall coverages are 98.19% and 97.94% of all 221,184 evaluations. These are floating-point bounds, without a directed-rounding guarantee.
On the same population, initial error is 19.36% among numerically stable predictions and 57.95% among witnessed changes, with this ordering in all 54 configurations. Within the stable group, the direct and refinement-only error rates are 16.96% and 63.79%. Refinement identifies further invariant decisions, including persistent errors; invariance does not ensure correctness.
The common correction and verification note and supplement register describe the numerical corrections. Table 3 mean contrasts change by less than 0.032 percentage points, and Full−Base is unchanged. The original cause could not be established from retained records.
Please let us know which concern, if any, remains unresolved.