acceptodds
Under review as a conference paper at ICLR 2027

Loss-Invariant Projections as Passive Probes of Learned Representations

Abstract

Learned feature representations in neural networks often contain structure beyond that directly used by the final task output. We study such structure using passive probes: fixed, untrained, property-independent projections that remain unchanged while the representation evolves during training. We motivate this approach through prediction on , where equivalent vector and Hermitian parameterizations reveal an additional loss-invariant trace coordinate. This motivates a general construction in which fixed random projections serve as observers of learned features. Because the observer is loss-invariant and independent of the property being studied, changes in accessibility reflect changes in the representation relative to the fixed observer rather than adaptation of the observer. We show that ensembles of passive probes can have direct relationships to task-relevant information such as target alignment. Across surface-normal estimation, image classification, and image inpainting, we observe different changes in accessibility under our constructions: eventual difficulty becomes increasingly accessible in surface-normal estimation and image inpainting which are regression tasks, whereas its accessibility remains near its initial level in image classification. Comparisons with learned linear probes further show that recoverability and passive accessibility can evolve differently during training. These results show how passive probes can separately characterize changes in representation geometry and the accessibility of eventual task difficulty.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Hermitian, loss-invariance, representation learning, random probes, task difficulty, sphere

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_T5A3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how task-relevant properties become accessible to fixed, untrained, property-independent scalar projections of learned representations during training. It distinguishes passive accessibility from recoverability measured by a probe fitted to the property at each checkpoint. A motivating sphere-prediction example identifies a loss-invariant trace coordinate in a Hermitian parameterization. The general framework uses random projections of penultimate features and a strict construction that removes the final task readout's row space and normalizes the remaining features. Accessibility is measured through linear and nonlinear alignment, correlations, and held-out calibrated R-squared. [Sections 1-2; Appendix A.]

Experiments examine surface-normal estimation on NYUv2, classification on CIFAR-100, and CIFAR-100 inpainting, using five training seeds and 256 fixed Gaussian directions per model. The target is each seed's final-model error, held fixed throughout its checkpoint trajectory. Passive accessibility generally increases in the two regression settings, while classification shows little sustained endpoint increase despite substantially improved accuracy. Learned ridge probes become more effective in all three settings. Controls separately examine feature normalization and readout removal, and calibration-bin sensitivity supports the qualitative endpoint patterns. [Section 3; Tables 1-4; Appendices B-E.]

The findings support task-dependent changes in scalar access to final-model error and differences from fitted recoverability. Their broader interpretation remains limited by temporal invariance, statistical complementarity, and unresolved geometric and loss-related explanations.

Critical strengths
  • The distinction between passive accessibility and fitted recoverability is coherent and useful. Holding a property-independent direction fixed removes adaptation of that direction as an explanation for changing associations, providing a complementary perspective on representation development. The paper appropriately recognizes the value of learned probes. [Sections 1 and 3.3.]

  • The evaluation includes meaningful safeguards: separate calibration and test observations, calibration-only fitting and regularization selection, image-grouped cross-validation for pixels and patches, five training seeds, and paired within-seed endpoint changes. Holding eventual difficulty fixed within a trajectory improves checkpoint comparability. [Section 3; Appendices B and E.]

  • The control experiments are substantive. Appendix C separates normalization from readout removal, and Appendix D examines 5, 10, 20, and 40 calibration bins. The main nonlinear and calibrated patterns persist across these choices, reducing several straightforward alternative explanations. [Tables 3-4.]

  • The empirical contrasts are informative. In classification, fitted difficulty recovery improves substantially while average passive scalar accessibility remains weak. In surface-normal estimation, strict nonlinear alignment and calibration improve while strict linear alignment changes little. These observations motivate questions about how difficulty-related signals are organized across feature directions. [Tables 1-2; Figure 2.]

  • The sphere/Hermitian construction provides an accessible mathematical motivation. Appendix A correctly establishes the prediction-map equivalence, trace-gradient cancellation, shared degeneracy, and optimization-metric qualification. The manuscript also acknowledges that extra task information is not guaranteed and that task, objective, and architecture require further investigation. [Section 2.1; Section 2.2, lines 193-196; Section 4; Appendix A.]

Critical weaknesses

Major Weaknesses

  • The implemented strict construction does not guarantee loss invariance throughout training. Appendix B.4 fixes the projector using the final readout W_T, ensuring W_T(I-P_T)h_t=0, but generally W_t(I-P_T)h_t is nonzero at earlier checkpoints. For example, W_T=(1,0) and W_t=(0,1) make the second coordinate invisible to the final head but directly relevant to the earlier prediction. The existing analysis remains valid as a retrospective, final-readout-orthogonal measurement; it should be described consistently in those terms. Report current-head overlap, and use a frozen-head experiment if simultaneous fixedness and contemporaneous invariance are central. Recomputing the projector at each checkpoint answers a different question because the observer changes. Dependence on the final model is a retrospective limitation, not by itself evidence of calibration/test leakage.

  • Equation (14) is inconsistent with its centered-energy interpretation. The definition divides by tr(K), whereas the ratio of expected centered Gaussian-projection energies requires tr(K_c). Adding a constant offset to every feature leaves the centered kernel and probe correlations unchanged but can change the stated A_K: with H=e=(-1,1)^T, shifting H by c changes A_K from 1 to 1/(1+c^2). Its [0,1] bound remains valid; the issue is normalization and interpretation. Establish which denominator the implementation uses, correct the definition or explanation, and verify affected results. This discrepancy does not directly invalidate nonlinear alignment or calibrated R-squared. [Section 2.3, Eqs. (12)-(14); reviewer derivation.]

  • Readout orthogonality does not establish information additional to the prediction. With h=(z,z,1) and W=(1,0,0), the nullspace component (0,z,1), including its normalized version, is entirely determined by p=z. A strict coordinate can therefore correlate with error while supplying no complementary information. The manuscript's no-guarantee caveat is appropriate, but the experiments do not resolve this distinction. Compare held-out error prediction from task outputs alone against outputs plus passive coordinates, controlling fitting capacity and sample budget. Include raw output magnitude for normals, confidence measures where applicable, feature or residual norms, and patch-position or visible-context controls for inpainting. [Sections 2.2-2.3; reviewer counterexample and proposed controls.]

  • The conditional-moment argument and task comparison do not identify a loss-family mechanism. Equations (7)-(8) correctly describe squared-error point prediction, but expected categorical cross entropy also depends on the conditional mean of a one-hot target; squared loss on that encoding elicits the same probability vector. Moreover, conditional variance can affect stochastic squared-loss gradients even when absent from their expectation: Cov(2(p-Y)|x)=4 Sigma(x). The experiments differ in token type, output rank, augmentation, training duration, targets, and observation units, and realized final-model error does not identify conditional variance. Retain the regression/classification contrast as an observation, and test any explanatory claim in a matched synthetic setting with known mean and variance, controlled head rank, and an objective or uncertainty intervention. [Sections 2.2 and 3.1-3.2; Appendices B.1-B.3; reviewer derivations.]

  • The incremental scientific contribution needs clearer separation from established geometry and prior work. A fitted all-feature direction and an average random scalar are expected to differ: for centered whitened k-dimensional features, mean random-direction squared correlation equals optimal linear R-squared divided by k. The 256 raw Gaussian projections also jointly span the 128- or 192-dimensional representation almost surely, so weak scalar averages do not imply little information in the joint bank. Add covariance-spectrum, effective-rank, whitening, signal-concentration, individual-direction, and fixed-versus-resampled-bank controls to identify what the trajectories reveal. Position the alignment metrics against Cristianini et al., On Kernel-Target Alignment (2001; https://papers.nips.cc/paper_files/paper/2001/hash/1f71e393b3809197ed66df836fe833e5-Abstract.html), and Cortes et al., Algorithms for Learning Kernels Based on Centered Alignment (2012; https://jmlr.org/papers/v13/cortes12a.html). Discuss Cook et al., Outlier Detection through Null Space Analysis of Neural Networks (2020; https://arxiv.org/abs/2007.01263), which uses the same projector in the full-row-rank case but modifies training and studies outliers. These antecedents qualify the novelty claim without duplicating the complete temporal protocol. [Section 2.3; Section 3.3; Table 2; cited prior work; reviewer geometric analysis.]

Minor Weaknesses

  • Probe aggregation and uncertainty need clearer reporting. Specify how the 256 per-direction statistics are summarized; report medians, upper quantiles, the fraction with positive held-out R-squared, and individual trajectories. Separate probe-bank randomness from training-seed variability through repeated banks, and use image-level uncertainty estimates or appropriate block-permutation controls for dense observations. Five-seed SD on fixed evaluation data does not separately quantify these sources of variation. [Section 3; Appendix B.4.]

  • Reproducibility details are incomplete in places. Appendix B.1 refers to a surface-normal configuration without listing all optimizer and initialization/pretraining settings. Provide exact values, accessible configuration/code, projector rank thresholds, normalization tolerances, degenerate-coordinate handling, and calibration edge-bin behavior. Clarify that the projection direction remains untrained while the scalar calibration mapping and kernel bandwidth are estimated at each checkpoint. [Appendices B.1, B.4, and E.]

  • Computational cost is insufficiently characterized. Direct evaluation of Eq. (15) on 160,000 inpainting observations entails 25.6 billion pairwise entries, approximately 102.4 GB for one dense float32 matrix, and about 6.55 trillion scalar kernel evaluations across 256 probes. These are direct-implementation estimates, not measured resource requirements. State whether blocking, subsampling, or approximation is used, and report actual runtime, peak memory, approximation accuracy, and convergence with probe and sample counts. [Eq. (15); Section 3.1; reviewer arithmetic.]

  • Absolute effects and target dependence need more emphasis. After undoing the tables' 10^-3 scaling, final strict calibrated R-squared is 0.05176 for normals, 0.01026 for inpainting, and -0.00091 for classification. These can be useful diagnostics but do not establish strong error prediction. Check targets from another training seed or an ensemble to assess whether trends reflect persistent sample difficulty or a particular model's failures. Selective-risk or distribution-shift evaluations are useful extensions if practical claims are retained, rather than requirements for a purely diagnostic contribution. [Table 1; Appendices B.1-B.3.]

  • Several broader implications should remain explicitly tentative. The Hermitian example demonstrates an equivalent prediction map and a redundant coordinate, not a measured advantage over direct sphere prediction. Likewise, one final model's error mixes ambiguity, model bias, optimization, and possible noise; it should not be equated with intrinsic uncertainty. An additional property beyond eventual error would broaden the framework's demonstrated scope, but should follow the central corrections and controls. [Section 2.1; Section 3.1; Appendix A; reviewer assessment.]

Questions for authors
  1. Since the projector is defined using the final readout W_T, how do the authors justify loss invariance at intermediate checkpoints W_t, and can they demonstrate incremental error-prediction capability beyond the task outputs alone?

  2. Metric Correctness: Why does Equation (14) normalize centered alignment using \operatorname{tr}(K) rather than \operatorname{tr}(K_c), and how does correcting this discrepancy affect the reported results and conclusions?

  3. Novelty and Causality: How do the authors distinguish their contribution from existing kernel-target alignment and null-space analysis methods, and can matched experiments isolate the claimed loss-family mechanism while controlling conditional variance, head rank, and task configuration?

1 comment

We thank the reviewer for the detailed assessment and for identifying the strengths of our work.

Q1.

Invariance across checkpoints: Our strict probes use a projector defined from the final readout WTW_T, computed once per seed and held fixed throughout training. This guarantees orthogonality to the final readout, but not necessarily to the readout WtW_t at intermediate checkpoints. We therefore do not claim loss invariance at every checkpoint. Our analysis is retrospective, i.e. we examine how representations evolve along fixed directions outside the final task-readout space. We will clarify this distinction in the paper. Information beyond task outputs: Our experiments focus narrowly on the error-prediction capability that can be uncovered from random linear projections without directly receiving task-loss information. Our strict representation at the final checkpoint gives some empirical study of what that incremental capability of random linear projections might be


Q2.

Thank you for pointing this out, Eq. 14 has a typo and should use KcK_c in the denominator. The metrics were computed with the correct formula and still hold.


Q3.

We agree that kernel-target alignment and null-space analysis are established methods, and we do not claim these as contributions. The motivating example with Hermitian trace provides a concrete example of an untrained linear projection being computed as part of the learning representation. Our contribution is both presenting that motivating connection and the empirical study of how well general, untrained projections of learned representations predict eventual example difficulty throughout training. We analyze the results through the frameworks of kernel alignment and readout null-space structure. We do not claim a causal relationship with each loss family. Furthermore, the number of variables to match is quite large (e.g. rank, dimension, dataset size, task, etc.). Instead, our experiments highlight the presence of this empirical phenomenon, and we establish some mathematical motivation in Sec. 2.2 for why this phenomenon might emerge in regression tasks in particular.

Reviewer_yC3XOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper uses fixed random projections to probe learned representations, finding their association with eventual prediction error? (I'm sorry, I really don't understand what this paper is trying to convey... this is the best I can do)

Critical strengths

None

Critical weaknesses
  1. The paper is extremely difficult to read. I found the paper very hard to follow and, at times, painful to read. Several basic ideas and definitions are not explained clearly: (a) The sentence beginning "Such methods characterize important aspects of representation geometry" is repeated word for word on pages 1 and 2. (b) Section 2.1 states that the trace is "invariant to the loss" without first explaining which loss is considered or what this statement means. (c) Task difficulty is later defined as the final model's prediction error, but the paper does not clearly explain why this quantity is chosen or how it connects to the motivating example. (d) The metrics are introduced without enough explanation. Although KcK_c is defined by a formula, the meaning of "the fraction of centered probe-ensemble energy aligned with eventual difficulty" is unclear and needs a derivation. It is also insufficiently explained why the linear alignment does not depend on the sampled probe directions, whereas the nonlinear alignment does. (e) and many more...

  2. The claimed loss invariance is not justified throughout training. The null-space argument is correct for a fixed readout W, but W changes during training.

Questions for authors

The paper needs a substantial rewrite, and I don't think there are any questions the authors can answer for me to affect my assessment.

1 comment

We thank the reviewer for their feedback and hope to clarify some of the points raised.

W1.

a) Thank you for pointing this out, this will be corrected in the final version

b) This invariance to loss by the trace is specified in more detail in Section A in the Appendix, but we can clarify this further in the main body text. Modifying the trace while holding the traceless component fixed changes neither the prediction nor any loss depending only on that prediction.

c) Task difficulty was chosen based on empirical evidence of a consistent relationship, starting from the motivating example case (single fixed probe that receives no task gradient as justified by Section A2) and extending to random projections in other tasks in Sec. 3. Using the final model’s error gives a fixed reference across training checkpoints. We do not claim this to be the only quantity of interest nor that this relationship has any guarantees of existing, but Section 2.2 motivates why regression losses might encourage random projections to correlate with task difficulty after training.

d) Thank you for the feedback, we strive to clarify our explanations in a final version. AKA_K simply measures the linear alignment between the representation h and the error vector across the dataset, giving a quantitative sense of the linear correlation between h(x)h(x) and task difficulty. Since the random projections are linear, they sample this correlation, and AKA_K is the consistent limit of the alignment between random projections and the error vector as the number of samples approaches infinity. Unlike AKA_K, AGA_G involves a nonlinear kernel and has no corresponding closed-form calculation in the limit, so we estimate it by sampling the projections themselves. We find in Section 3/Fig. 2 that this nonlinear metric better captures the relationship with task difficulty.

e) We are happy to address any further concerns


W2.

To clarify, the projection directions aj\mathbf{a}_j are sampled independently and held fixed, rather than optimized by the loss. This alone does not establish null-space invariance. Projecting at any training step a direction into the null space of the current W gives invariance to that checkpoint’s readout even though WtW_t changes during training. Our experiments instead use the final WTW_T, so this guarantee applies to the final readout, not every earlier checkpoint. We will clarify this scope.

Reviewer_ex1LOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes "passive probes" for studying how learned representations change during training. A passive probe is a fixed random Gaussian direction a_j that is applied to the penultimate representation, q_j(x) = a_j^T h(x), and is never trained. Because the probe is never fit to the property being studied, any change in what it reveals reflects a change in the representation rather than in the probe. The motivation comes from a Hermitian parameterization of prediction on the sphere S^2, which contains a trace coordinate that receives no gradient from the loss. The authors also define a "strict" variant that projects both the representation and the probe onto the null space of the final readout, so that the probe is insensitive to the task output.

The experiments use NYUv2 surface-normal estimation (DSINE baseline), CIFAR-100 classification with a ViT, and CIFAR-100 inpainting with a ViT, each with 5 seeds and 256 probes. The authors measure how well the probes track the final model's per-example error, using kernel alignment, Pearson and Spearman correlation, and a binned calibrated R^2. They report that this error becomes more visible to the probes during training on the two regression tasks but not on classification, and that the passive measures evolve differently from a trained ridge probe.

Critical strengths

Motivation

The question is reasonable. Separating what a fixed, untrained observer can read from a representation and what a trained probe can recover is a meaningful distinction, and projecting the probe into the null space of the task readout is a sensible way to exclude information that the task output already carries.

Controls

The analysis is applied consistently. The four probe constructions (raw, unit-normalized, readout-orthogonal and strict), the sensitivity checks on the number of calibration bins, and the image-grouped evaluation splits follow the same protocol across all three tasks, which makes the reported numbers easy to interpret.

Critical weaknesses

Some key technical limitations and suggested improvements:

  1. The measured effects are small, and they are not compared against simple baselines. At the final checkpoint, the best passive calibrated R^2 is about 0.064 for surface normals, between 0.005 and 0.010 for inpainting, and roughly zero for classification. Because Table 1 reports these values in units of 10^-3, their size is easy to misread. For comparison, a trained ridge probe reaches about 0.19 to 0.20 (Table 2), and the passive surface-normal R^2 is already 0.028 at initialization. Without simple baselines, it is unclear whether random projections capture anything beyond feature norm or input statistics. The paper also notes in Sec. 2.3 that the linear alignment A_K does not depend on the probes at all, so the passive framing adds nothing for that metric. I suggest comparing against ||h||, ||h_perp||, the current checkpoint's per-example loss and prediction confidence, and showing at least one practical use, such as flagging hard examples early, that a trained probe cannot provide.

  2. The motivation and the main interpretation are not yet supported by the experiments. The Hermitian trace coordinate that motivates the paper (Sec. 2.1) is never measured, and the fixed fourth output channel in the surface-normal model is used only to set the probe norms (App. B). The explanation for the difference between regression and classification relies on a first-moment argument for the L2 loss (Eqs. 7-8), but the surface-normal model is trained with a cosine loss. In addition, only the classification run uses Mixup/CutMix, RandAugment and label smoothing, and each of these changes the per-example NLL that the paper uses to define difficulty. The three tasks therefore differ in much more than the loss. Rerunning classification without these augmentations, adding a classifier trained with an L2 loss, and reporting the loss-invariant coordinate itself would make the interpretation testable.

These issues are addressable, but in its current form the paper does not show that passive accessibility tells us something that simpler measurements do not.

Questions for authors
  1. Do passive probes predict eventual difficulty better than simple scalar statistics of the representation? In particular, how do they compare at each checkpoint with ||h(x)||, ||h_perp(x)||, the current per-example loss and prediction entropy? The answer would affect my assessment the most.

  2. Does the classification result still hold when the classifier is trained without Mixup/CutMix and label smoothing? Only the classification run uses these, and each of them changes the per-example NLL that defines difficulty.

  3. What does the loss-invariant coordinate from Sec. 2.1 show during training? The trace, or the frozen fourth output channel in the surface-normal model, motivates the paper but is not reported.

2 comments

We thank the reviewer for the feedback and for identifying the strengths of our paper. Below, we answer the reviewer’s questions:

Q1. We evaluated the feature norm (∥h(x)∥\lVert h (x) \rVert), final-readout-null norm (∥h⊥(x)∥\lVert h_\perp (x)\rVert), and current task loss at every recorded checkpoint across all three tasks. For classification, we additionally evaluated predictive entropy and maximum predicted probability. All methods were evaluated using the same CAL/TEST protocol and 20-bin calibration, with results averaged over five seeds. The following summarises the final checkpoint R2 scores:

TaskRaw probeStrict probe∥h(x)∥\lVert h (x) \rVert∥h⊥(x)∥\lVert h_\perp (x)\rVertCurrent lossEntropy
Surface normals (DSINE)0.06430.05180.09190.09100.9679—
Classification (CIFAR-100)−0.0002−0.00090.07990.01750.99150.2843
Inpainting (CIFAR-100)0.00530.01030.02500.00660.8202—

(Passive-probe values are averaged over 256 fixed random directions per seed. All reported values use calibration fitted exclusively on CAL and evaluated on held-out TEST. Prediction entropy is applicable only to classification.)

These results show that passive random projections do not generally outperform simple scalar baselines. At the final checkpoints, feature norm outperforms both raw and strict passive probes in all three tasks. Current task loss is substantially more predictive, while classification prediction entropy also strongly outperforms passive projections.

We thus obtain distinct task-dependent patterns: For surface normals, passive accessibility is present, but representation norms generally provide stronger scalar predictors. For classification, typical fixed random projections remain essentially uninformative throughout training, even as feature norm, predictive entropy, and current loss become increasingly more predictive. For Inpainting, passive accessibility increases during training, although its absolute predictive capability remains modest. Its trajectory differs from that of the feature norm, particularly at intermediate checkpoints.

Current loss at the final checkpoint closely matches the target difficulty by construction. So its almost perfect predictive performance is expected. The earlier checkpoint comparisons are more informative for assessing how well the current model behavior predicts eventual difficulty. We will include the complete checkpoint-wise comparisons in the supplementary material.

As is in the original submission, we do not claim passive random projections to be superior difficulty predictors. Rather, we narrowly focus on the capability of random, untrained linear projections to capture difficulty-related information.


Q2.

We thank the reviewer for raising this point. Mixup, CutMix, and label smoothing were included during classification training to improve the final performance of the model. However, none of these techniques were used during probe evaluation, and eventual difficulty was defined using the standard per-example NLL against the original hard labels. We agree that these training techniques may still affect the learned representations and the resulting difficulty distribution. We have not performed the ablation without them, so we cannot establish whether the classification result would hold under that training setup. We will clarify this limitation in the paper.


Q3. The Hermitian trace coordinate (or fourth frozen coordinate) is a single scalar projection (e.g. qjq_j), and thus would be expected to have high variance in correlation with most quantities. Our experiments and analysis focused on the general case of ensembles of projections (256 fixed random projections qjq_j) to determine whether untrained random projections can provide information beyond noise relating to task difficulty. The connection with Hermitian trace aimed to provide a geometric intuition for interpreting a random projection scalar and motivate higher-dimensional representation forms (i.e. dimension of h(x)h(x)).

Reviewer_ex1LOpenReview ↗

Response to authors

Thanks for adding the scalar baselines. It seems inpainting is the one place where the strict probe edges past the norm. In my opinion, the per-checkpoint comparisons should be in the main paper, since the paper’s argument is about how accessibility changes during training.

Having said that, feature norm does better than the passive probes on all three tasks, so it isn’t clear the trends show more than norm changes. The augmentations in the classification run also make the regression vs. classification contrast hard to interpret. My score stays the same.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.