- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes PCNG to analyze novelty in frozen sequential representations using a training-only principal curve. It derives progression position, transverse deviation, and traversal rate, calibrates them to rarity scores, and tests their interpretations. Experiments mainly support the transverse interpretation, while the rate interpretation remains unsupported. An admissibility gate determines whether attribution should proceed.
Critical strengths
- Testing the meaning of geometric scores is a useful goal. The paper reports negative results and distinguishes a computable score from an interpretation that is supported.
- The unit-preserving randomization addresses a statistical issue: correlated windows should not be treated as independent observations.
Critical weaknesses
- The evidence for novelty attribution beyond distance-based novelty remains limited. Transverse rarity is calibrated from distance to the curve, and the experiment yielding 35/36 changes only the reference distribution, with the curve and probes fixed. The controls and residual-scramble checks help, but it is still unclear how much the result validates the transverse interpretation beyond sensitivity to rarity calibration. The matched-Mahalanobis check in Appendix V is also PARTIAL.
- The admissibility gate needs stronger support as a basis for permitting or refusing attribution. CV(L) measures path homogeneity, but Table 20 shows construct accuracy declining before GO is withdrawn under branching, with limited sensitivity to curvature and transverse noise. Bootstrap gate-label agreement of 0.149 raises further concerns about stability. Loss of GO also does not imply refusal, since MARGINAL still permits attribution.
Questions for authors
- What do the reference-ablation interventions establish about the transverse interpretation beyond changes in rarity calibration, after accounting for the existing controls?
- Given the PARTIAL result in Appendix V, which findings support novelty attribution beyond distance-based novelty?
- How were the CV thresholds of 0.15 and 0.35 chosen? Does construct accuracy differ between GO/MARGINAL and NO-GO runs?
- With bootstrap gate-label agreement of 0.149, how should users decide whether to proceed, especially near the thresholds?