- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies how task-relevant properties become accessible to fixed, untrained, property-independent scalar projections of learned representations during training. It distinguishes passive accessibility from recoverability measured by a probe fitted to the property at each checkpoint. A motivating sphere-prediction example identifies a loss-invariant trace coordinate in a Hermitian parameterization. The general framework uses random projections of penultimate features and a strict construction that removes the final task readout's row space and normalizes the remaining features. Accessibility is measured through linear and nonlinear alignment, correlations, and held-out calibrated R-squared. [Sections 1-2; Appendix A.]
Experiments examine surface-normal estimation on NYUv2, classification on CIFAR-100, and CIFAR-100 inpainting, using five training seeds and 256 fixed Gaussian directions per model. The target is each seed's final-model error, held fixed throughout its checkpoint trajectory. Passive accessibility generally increases in the two regression settings, while classification shows little sustained endpoint increase despite substantially improved accuracy. Learned ridge probes become more effective in all three settings. Controls separately examine feature normalization and readout removal, and calibration-bin sensitivity supports the qualitative endpoint patterns. [Section 3; Tables 1-4; Appendices B-E.]
The findings support task-dependent changes in scalar access to final-model error and differences from fitted recoverability. Their broader interpretation remains limited by temporal invariance, statistical complementarity, and unresolved geometric and loss-related explanations.
Critical strengths
-
The distinction between passive accessibility and fitted recoverability is coherent and useful. Holding a property-independent direction fixed removes adaptation of that direction as an explanation for changing associations, providing a complementary perspective on representation development. The paper appropriately recognizes the value of learned probes. [Sections 1 and 3.3.]
-
The evaluation includes meaningful safeguards: separate calibration and test observations, calibration-only fitting and regularization selection, image-grouped cross-validation for pixels and patches, five training seeds, and paired within-seed endpoint changes. Holding eventual difficulty fixed within a trajectory improves checkpoint comparability. [Section 3; Appendices B and E.]
-
The control experiments are substantive. Appendix C separates normalization from readout removal, and Appendix D examines 5, 10, 20, and 40 calibration bins. The main nonlinear and calibrated patterns persist across these choices, reducing several straightforward alternative explanations. [Tables 3-4.]
-
The empirical contrasts are informative. In classification, fitted difficulty recovery improves substantially while average passive scalar accessibility remains weak. In surface-normal estimation, strict nonlinear alignment and calibration improve while strict linear alignment changes little. These observations motivate questions about how difficulty-related signals are organized across feature directions. [Tables 1-2; Figure 2.]
-
The sphere/Hermitian construction provides an accessible mathematical motivation. Appendix A correctly establishes the prediction-map equivalence, trace-gradient cancellation, shared degeneracy, and optimization-metric qualification. The manuscript also acknowledges that extra task information is not guaranteed and that task, objective, and architecture require further investigation. [Section 2.1; Section 2.2, lines 193-196; Section 4; Appendix A.]
Critical weaknesses
Major Weaknesses
-
The implemented strict construction does not guarantee loss invariance throughout training. Appendix B.4 fixes the projector using the final readout W_T, ensuring W_T(I-P_T)h_t=0, but generally W_t(I-P_T)h_t is nonzero at earlier checkpoints. For example, W_T=(1,0) and W_t=(0,1) make the second coordinate invisible to the final head but directly relevant to the earlier prediction. The existing analysis remains valid as a retrospective, final-readout-orthogonal measurement; it should be described consistently in those terms. Report current-head overlap, and use a frozen-head experiment if simultaneous fixedness and contemporaneous invariance are central. Recomputing the projector at each checkpoint answers a different question because the observer changes. Dependence on the final model is a retrospective limitation, not by itself evidence of calibration/test leakage.
-
Equation (14) is inconsistent with its centered-energy interpretation. The definition divides by tr(K), whereas the ratio of expected centered Gaussian-projection energies requires tr(K_c). Adding a constant offset to every feature leaves the centered kernel and probe correlations unchanged but can change the stated A_K: with H=e=(-1,1)^T, shifting H by c changes A_K from 1 to 1/(1+c^2). Its [0,1] bound remains valid; the issue is normalization and interpretation. Establish which denominator the implementation uses, correct the definition or explanation, and verify affected results. This discrepancy does not directly invalidate nonlinear alignment or calibrated R-squared. [Section 2.3, Eqs. (12)-(14); reviewer derivation.]
-
Readout orthogonality does not establish information additional to the prediction. With h=(z,z,1) and W=(1,0,0), the nullspace component (0,z,1), including its normalized version, is entirely determined by p=z. A strict coordinate can therefore correlate with error while supplying no complementary information. The manuscript's no-guarantee caveat is appropriate, but the experiments do not resolve this distinction. Compare held-out error prediction from task outputs alone against outputs plus passive coordinates, controlling fitting capacity and sample budget. Include raw output magnitude for normals, confidence measures where applicable, feature or residual norms, and patch-position or visible-context controls for inpainting. [Sections 2.2-2.3; reviewer counterexample and proposed controls.]
-
The conditional-moment argument and task comparison do not identify a loss-family mechanism. Equations (7)-(8) correctly describe squared-error point prediction, but expected categorical cross entropy also depends on the conditional mean of a one-hot target; squared loss on that encoding elicits the same probability vector. Moreover, conditional variance can affect stochastic squared-loss gradients even when absent from their expectation: Cov(2(p-Y)|x)=4 Sigma(x). The experiments differ in token type, output rank, augmentation, training duration, targets, and observation units, and realized final-model error does not identify conditional variance. Retain the regression/classification contrast as an observation, and test any explanatory claim in a matched synthetic setting with known mean and variance, controlled head rank, and an objective or uncertainty intervention. [Sections 2.2 and 3.1-3.2; Appendices B.1-B.3; reviewer derivations.]
-
The incremental scientific contribution needs clearer separation from established geometry and prior work. A fitted all-feature direction and an average random scalar are expected to differ: for centered whitened k-dimensional features, mean random-direction squared correlation equals optimal linear R-squared divided by k. The 256 raw Gaussian projections also jointly span the 128- or 192-dimensional representation almost surely, so weak scalar averages do not imply little information in the joint bank. Add covariance-spectrum, effective-rank, whitening, signal-concentration, individual-direction, and fixed-versus-resampled-bank controls to identify what the trajectories reveal. Position the alignment metrics against Cristianini et al., On Kernel-Target Alignment (2001; https://papers.nips.cc/paper_files/paper/2001/hash/1f71e393b3809197ed66df836fe833e5-Abstract.html), and Cortes et al., Algorithms for Learning Kernels Based on Centered Alignment (2012; https://jmlr.org/papers/v13/cortes12a.html). Discuss Cook et al., Outlier Detection through Null Space Analysis of Neural Networks (2020; https://arxiv.org/abs/2007.01263), which uses the same projector in the full-row-rank case but modifies training and studies outliers. These antecedents qualify the novelty claim without duplicating the complete temporal protocol. [Section 2.3; Section 3.3; Table 2; cited prior work; reviewer geometric analysis.]
Minor Weaknesses
-
Probe aggregation and uncertainty need clearer reporting. Specify how the 256 per-direction statistics are summarized; report medians, upper quantiles, the fraction with positive held-out R-squared, and individual trajectories. Separate probe-bank randomness from training-seed variability through repeated banks, and use image-level uncertainty estimates or appropriate block-permutation controls for dense observations. Five-seed SD on fixed evaluation data does not separately quantify these sources of variation. [Section 3; Appendix B.4.]
-
Reproducibility details are incomplete in places. Appendix B.1 refers to a surface-normal configuration without listing all optimizer and initialization/pretraining settings. Provide exact values, accessible configuration/code, projector rank thresholds, normalization tolerances, degenerate-coordinate handling, and calibration edge-bin behavior. Clarify that the projection direction remains untrained while the scalar calibration mapping and kernel bandwidth are estimated at each checkpoint. [Appendices B.1, B.4, and E.]
-
Computational cost is insufficiently characterized. Direct evaluation of Eq. (15) on 160,000 inpainting observations entails 25.6 billion pairwise entries, approximately 102.4 GB for one dense float32 matrix, and about 6.55 trillion scalar kernel evaluations across 256 probes. These are direct-implementation estimates, not measured resource requirements. State whether blocking, subsampling, or approximation is used, and report actual runtime, peak memory, approximation accuracy, and convergence with probe and sample counts. [Eq. (15); Section 3.1; reviewer arithmetic.]
-
Absolute effects and target dependence need more emphasis. After undoing the tables' 10^-3 scaling, final strict calibrated R-squared is 0.05176 for normals, 0.01026 for inpainting, and -0.00091 for classification. These can be useful diagnostics but do not establish strong error prediction. Check targets from another training seed or an ensemble to assess whether trends reflect persistent sample difficulty or a particular model's failures. Selective-risk or distribution-shift evaluations are useful extensions if practical claims are retained, rather than requirements for a purely diagnostic contribution. [Table 1; Appendices B.1-B.3.]
-
Several broader implications should remain explicitly tentative. The Hermitian example demonstrates an equivalent prediction map and a redundant coordinate, not a measured advantage over direct sphere prediction. Likewise, one final model's error mixes ambiguity, model bias, optimization, and possible noise; it should not be equated with intrinsic uncertainty. An additional property beyond eventual error would broaden the framework's demonstrated scope, but should follow the central corrections and controls. [Section 2.1; Section 3.1; Appendix A; reviewer assessment.]
Questions for authors
-
Since the projector is defined using the final readout W_T, how do the authors justify loss invariance at intermediate checkpoints W_t, and can they demonstrate incremental error-prediction capability beyond the task outputs alone?
-
Metric Correctness: Why does Equation (14) normalize centered alignment using \operatorname{tr}(K) rather than \operatorname{tr}(K_c), and how does correcting this discrepancy affect the reported results and conclusions?
-
Novelty and Causality: How do the authors distinguish their contribution from existing kernel-target alignment and null-space analysis methods, and can matched experiments isolate the claimed loss-family mechanism while controlling conditional variance, head rank, and task configuration?
We thank the reviewer for the detailed assessment and for identifying the strengths of our work.
Q1.
Invariance across checkpoints: Our strict probes use a projector defined from the final readout WT, computed once per seed and held fixed throughout training. This guarantees orthogonality to the final readout, but not necessarily to the readout Wt at intermediate checkpoints. We therefore do not claim loss invariance at every checkpoint. Our analysis is retrospective, i.e. we examine how representations evolve along fixed directions outside the final task-readout space. We will clarify this distinction in the paper. Information beyond task outputs: Our experiments focus narrowly on the error-prediction capability that can be uncovered from random linear projections without directly receiving task-loss information. Our strict representation at the final checkpoint gives some empirical study of what that incremental capability of random linear projections might be
Q2.
Thank you for pointing this out, Eq. 14 has a typo and should use Kc in the denominator. The metrics were computed with the correct formula and still hold.
Q3.
We agree that kernel-target alignment and null-space analysis are established methods, and we do not claim these as contributions. The motivating example with Hermitian trace provides a concrete example of an untrained linear projection being computed as part of the learning representation. Our contribution is both presenting that motivating connection and the empirical study of how well general, untrained projections of learned representations predict eventual example difficulty throughout training. We analyze the results through the frameworks of kernel alignment and readout null-space structure. We do not claim a causal relationship with each loss family. Furthermore, the number of variables to match is quite large (e.g. rank, dimension, dataset size, task, etc.). Instead, our experiments highlight the presence of this empirical phenomenon, and we establish some mathematical motivation in Sec. 2.2 for why this phenomenon might emerge in regression tasks in particular.