acceptodds
Under review as a conference paper at ICLR 2027

Beyond Confidence: Log-Alignment Profiles for Monitoring Internal Computation

Abstract

Distribution shift can alter a network's internal computation while leaving its predictions confident. We construct log-alignment profiles that record layerwise amplification and activation scale for individual inputs. Calibrated against reference data, these compact profiles detect atypical processing and reveal its distribution across network depth. We call this reference-relative typicality structural familiarity. Across image classifiers, language models and visuomotor policies, the profiles detect shifts that output scores detect poorly. On CIFAR-10 and SVHN inputs matched by maximum-softmax-probability bins, an alignment-only profile achieves detection AUROC. In a supervised clean-plus-shift evaluation, adding an alignment-only profile to entropy raises ResNet-50 error-prediction AUROC from to . In language models from M to B parameters, the profile detects Python code relative to a natural-text reference at – AUROC. On LIBERO behavior-cloning policies, profile alarms detect corruption onset – steps before action entropy; on ManiSkill, profiles add episode-failure information beyond ensemble disagreement. We also test alignment interventions during learning. In grokking transformers, late-layer interventions reduce mean post-fork grokking delay by in a three-seed comparison and accelerate grokking in all eight runs of a separate paired experiment. Their effects depend on the targeted layer and subspace.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Cy5pOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper repurposes the log-alignment ratio (LAR), a prior training-time diagnostic, as a per-input monitor. At selected layers, the authors track the log input norm and log output norm of a linear map, reparameterize these as "gain" and "radius", standardize these values against in-distribution data, and score each input with a class-conditional Mahalanobis distance (U_LAP). The claims are that this score (1) detects structural shifts (contrast, noise, blur, Python code in language models, corrupted LLaVA images, corrupted robot observations) which output-confidence scores miss, and (2) adds information to entropy for error prediction (ResNet-50 0.632 to 0.811, ViT-B/16 0.706 to 0.849), and (3) raises alarms earlier than action entropy on LIBERO, and adds failure information beyond ensemble disagreement on ManiSkill. A separate section shows that alignment regularizers modify grokking delay in small transformers. Evidence is ImageNet and CIFAR detection tables, a random-basis control, a partial-correlation "structural vs. semantic" analysis, an ImageNet-200 factorial, LM and VLM pilots, and four robotics testbeds.

Critical strengths
  1. The monitor is cheap and was tested widely. It needs no SVD, gradients, or shifted data. Overhead is about 9-10%, and calibration storage is about 28 KB versus 0.8-1.2 GB for the Gram baseline. It is evaluated on CNNs, ViTs, LMs, a VLM and several policy classes. Several controls are good practice: the exact random-direction null in Appendix C, the confidence-matched CIFAR/SVHN split, and the 2x2 factorial. Together these make the empirical contribution useful to people deploying monitors.
  2. The authors are open about where the method does not win. Feature-density methods win on semantic and far-OOD sets (Table 5, Fig. 10), entropy wins on ImageNet-200 class novelty, and the appendix reports no incremental failure information on LIBERO (p=0.94). This makes the “structural, not semantic” story much more believable than a uniform-win story would be.
Critical weaknesses
  1. The paper isn't fully isolating 'alignment' as an informative signal, since by Prop. E.1 the joint [gain, radius] score is equivalent to a Mahalanobis score on raw per-layer [log||Wh||, log||h||], hence a re-parameterization of the same norms. Table 5 is consistent with this in that radius-only is close to the joint score on ResNet-50 (0.958/0.963/0.992/0.963 vs. 0.983/1.000/1.000/0.998), and gain-only outperforms the joint score on ViT noise (0.984 vs. 0.971). The control on random-V shows that the true ||Wh|| beats a rotated surrogate, which is useful but doesn't show that alignment specifically matters, or has per-layer output norms at all, as opposed to having them. This is fixable with additional controls.
  2. Some of the headline gains exploit protocols or baselines that favor a shift detector. The error-prediction result (0.632 to 0.811) fits the predictor on a clean+shifted mixture, split by example. Because shifted inputs have higher error, that partly measures shift detection rather than added information about correctness. Leave-corruption-out splits or within-condition AUROC would be more convincing.
  3. For language models, perplexity is a weak baseline for domain detection, and Python vs. enwik8 is an easy distinction. There is one reference corpus and one seed per model. In Table 3b, the disagreement baseline is near chance on 7 of 8 tasks (0.45-0.54), so the 0.14-0.24 gains are against a weak reference. Some of the framing is stronger than the evidence. Fig. 1's 0.880 comes from a split constructed so that MSP is at chance and has a 10-model ensemble, whereas single-model results are noisier (0.59 +/- 0.24 at M=1 for a single layer). The structural and semantic distances are author-designed, and the factorial still has a nonzero novelty response (0.37). The depth "archetypes" are an interpretive clustering of 24 sets. These are reasons to tone down the claims, not to doubt the core results.
Questions for authors
  1. Given Prop. E.1 plus the radius-only results in Table 5, what is evidence that alignment contributes beyond a per-layer log-norm detector? Can you add controls using different per-layer norms or random-weight layers?
  2. Were the probe positions, pooling choices, L values or alarm thresholds selected using any evaluation data, or the seeds reported in Table 4?
  3. Is the 0.880 in Fig. 1 from the joint profile or an alignment-only variant? What are the single-model numbers on that split?
  4. What are the results with leave-corruption-out fitting and within-condition AUROC for error prediction? How does a baseline built from simple image statistics (such as your D_s features) compare?
  5. How do hidden-state Mahalanobis and simple token-frequency or compression features perform on the same shifts? Does the profile track LM correctness anywhere?
Reviewer_wgesOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to detect distribution shift when the network's output confidence over samples remains unchanged, but the internal computation differs. It proposed log-alignment profiles to record how strongly each layer amplifies its input during the forward pass, and compare them with those on the in-distribution reference data. Experiments on image classification, language modeling, and robot policies show that the method successfully detects the shift, while those output-based measurements fail.

Critical strengths
  1. The paper studies a meaningful problem where the output-based scores can fail to detect distribution shifts that are only visible in internal computation. The proposed internal monitor detects such shifts effectively in experiments.

  2. The paper considers diverse experimental settings across vision, language, and robotics tasks.

Critical weaknesses
  1. The paper is a bit hard to follow. Many paragraphs put together statements without clear logical connections. For instance, the introduction does not provide a smooth, natural transition to the background and motivation. Many concepts are used before they are defined, such as alignment and randomized-basis control. Some details needed to interpret the main results appear only in the appendix, e.g., detailed experimental settings, including the one behind Figure 1. The grokking experiment feels somewhat out of place in the current paper, since it focuses on training-time regularization rather than monitoring.

  2. Some experiments do not fully support the main claims. For instance, the paper studies the effect of replacing the learned directions with random directions and shows a large drop in AUROC (Table 1). This may not fully support the importance of the learned directions, as Table 5 shows that this control is much worse than using the input norms alone. In fact, using the input norms alone often performs as well as or even better than the gain. Therefore, it is not clear whether the signal comes from alignment. Additionally, the language-model experiments compare only with perplexity, without considering other internal-computation baselines.

Questions for authors
  1. The random-direction control in Table 1 performs worse than using the input norms alone. Given this, why should the drop in AUROC be attributed to the positive effect of the learned directions rather than the negative effect of the random gain coordinates?

  2. How does the method compare with internal-computation baselines in the language-model experiments?

Reviewer_fytmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to apply (the previously known) method of log-alignment profiles (LAP) to detect OOD samples in a wide range of neural architecture and applications (MLP, CNN, Transformer on Images and text data). LAP is measured on layer-wise forward pass outputs using calibrated training data distributions. Extensive experiments show that the proposed method is able to detect ODD samples with high accuracy in cases where simple network output scores fail.

Critical strengths
  • The paper addresses a timely and important research question. OOD detection is of high practical relevance.
  • The proposed method appeals by its simplicity, and low computational costs at evaluation time, allowing OOD detection in realistic applications.
  • The experimental evaluation is extensive, covering a wide range of network architectures and applications. This gives a strong support for the generalization assumption of the proposed method
Critical weaknesses

W1 - readability: the paper reads like a patch work of multiple (human and AI) authors. Some sections are well structured and easy to follow, other are overwhelming fact collection without a clear focus and story-line. Especially the abstract is hardly readable for a human - reduce the fact counting and give an actual overview of the paper, its motivation, core contributions and brief(!) summary of the results. I think that this point is fixable as the authors have demonstrated the necessary skills in some sections.

W2 - Limited evidence of error prediction on unseen shift types. The error predictor, especially in the vision experiments, trains and tests on different examples from the same clean/shifted mixture. This does not establish generalization to entirely new corruption families. Here the test/train samples should be split a corruption families.

W3 - OOD detection is only partially connected to actual failure: Many strong results measure ODD input data at layer level, but not all OOD inputs will propagate an actual error through the network. The error-prediction experiments are the right way to go, but do not close this gap across domains.

W4 - limited evidence for real world benefits: Earlier warnings could be useful, but the experiments do not demonstrate that stopping, recovery, or escalation actually reduces failures at an acceptable cost for real applications

Minor: one interesting reference for for why the output confidence is not a good measure is missing: Grabinski, Julia, et al. "Robust models are less over-confident." Advances in neural information processing systems 35 (2022): 39059-39075.

Questions for authors

Q1 - The main unanswered question is: does the profile identify consequential failures on genuinely new inputs, at a tolerable false-alarm rate?

Q2 - Does error prediction transfer to unseen shift families?

Q3 - Can the proposed method distinguish dangerous shifts from benign novelty?

Q4 - For real applications: How does acting on an OOD alarm improve outcomes at manageable costs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.