acceptodds
Under review as a conference paper at ICLR 2027

Activation-Space Uncertainty Quantification for Pretrained Networks

Abstract

Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone’s predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that models uncertainty over activation outputs rather than over network weights. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form activation-space variances that act as support-based uncertainty estimates. To scale to modern architectures, we use a sparse inducing-point approximation over cached reference pre-activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation. Across regression, classification, image segmentation, and language modeling, GAPA is competitive with strong post-hoc baselines and often improves calibration or out-of-distribution detection while remaining efficient at test time.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
uncertainty quantification, Gaussian processes, pretrained neural networks, post-hoc uncertainty estimation, epistemic uncertainty, out-of-distribution detection

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_obMGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Gaussian Process Activations (GAPA), a post-hoc method that uses a Gaussian process in the activation outputs in frozen pretrained neural networks. To achieve scalability, they use a local k-NN for conditioning. Furthermore, they demonstrate their ability on a variety of tasks including classification, regression, segmentation, language modeling, and out-of-distribution detection.

Critical strengths

Like other post-hoc methods, the strength of the proposed method is to perform uncertainty quantification using frozen backbones. This is useful as it is undesirable to fine tune LLMs and other large models. Furthermore, the method that the authors propose is an interesting way to do it and it seems to be have good empirical performance.

Critical weaknesses
  1. This paper seems to be an extension of "Post-Hoc Uncertainty Quantification in Pre-Trained Neural Networks via Activation-Level Gaussian Processes" by Bergna et al. AABI 2025, but the manuscript doesn't acknoledge, cite, or differentiate this paper from that one. That paper already introduces the core GAPA and many of the ideas presented in this manuscript as new. While this manuscript does introduce extensions and more evaluation, the other paper is already published and needs to be acknoledged as the original and treated as a competitor. Given the overlap between the two and the lack of acknoledgement of explanation of differences, this manuscript's novelty claims and contribution is lessened.

  2. The datasets, especially image classification datasets, are basic almost toy-like datasets. It's possible that the proposed method is very useful, but it's also possible that it only works with these "easy" datasets. Another problem with using these kind of datasets is that the possible gains is also very small. This can be seen in all the results where the results for all comparison methods, including the proposed method, are about the same and probably within the margin of error. Even in the Appendix, the gains over other classiic distance functions is minimal.

Questions for authors
  1. How are the hyperparameters in the ablation studies determined (ex: GAPA layers)? Are they selected using a validation set or using the test set?
2 comments

Thank you for finding GAPA a useful and interesting way to quantify uncertainty in frozen backbones, including LLMs, with good empirical performance.


W1. This paper seems to be an extension of [...] Bergna et al. AABI 2025, but the manuscript doesn't acknoledge, cite, or differentiate this paper from that one. [...]

We now cite the AABI 2025 workshop paper [1] in the related work (Section 4) and compare against it below, on the regression benchmarks of Table 1, the only tasks [1] evaluates. GAPA outperforms [1] on 8 of 9 metrics, including NLL on all three datasets; [1] has a lower CRPS on Taxi.

Airline NLLCRPSCQMYear NLLCRPSCQMTaxi NLLCRPSCQM
1D GP [1]5.06718.2820.1353.5454.7960.0533.2683.5520.154
GAPA (ours)4.94618.0680.1033.4704.6630.0143.1124.0350.104

Numbers for [1] are from its paper (its stronger variant), whose MAP backbones have nearly the same NLL as ours.

The main difference between the two models is what each GP conditions on. [1] fits an independent one-dimensional GP to each neuron of the first layer, so each GP sees only that neuron's own pre-activation. In GAPA, each GP takes the layer's full pre-activation vector as input, so its uncertainty reflects how far an input's whole representation lies from the training data (Section 2.2).

GAPA is also built for large networks. Sparse inducing points and local nearest-neighbour conditioning keep it tractable (Section 2.3), single-pass variance propagation allows it to be applied at any layer (Section 2.4), and it needs no training. This is what lets GAPA run on ResNets, U-Net segmentation and LLaMA-3.2-3B, well beyond the scale of the experiments in [1].

References

[1] Bergna et al. "Post-Hoc Uncertainty Quantification in Pre-Trained Neural Networks via Activation-Level Gaussian Processes." AABI 2025 (workshop track).


W2. The datasets, especially image classification datasets, are basic almost toy-like datasets. [...] the possible gains is also very small.

Yes, MNIST and Fashion-MNIST are small. We use them because many baselines, such as full Laplace, become intractable at larger scale, so these are among the few benchmarks where all of them can be compared directly with GAPA. However, GAPA itself scales well beyond them, up to a 3-billion-parameter LLM, and this scalability is one of our main contributions.

On LLaMA-3.2-3B, GAPA detects out-of-distribution text with AUROC 0.879, against about 0.5 for last-layer Laplace (Section 3.3, Figure 3). On U-Net segmentation, it propagates uncertainty to 49,152 outputs per image, where full Laplace is infeasible (Appendix G). On CIFAR-10, it lies on the Pareto frontier of OOD detection against inference cost for all four pretrained ResNets (Figure 2). In the LLM and segmentation settings, most of our baselines cannot be run, and uncertainty methods are rarely evaluated at this scale.

Even on these small benchmarks, GAPA has the best BALD AUROC of all methods on both MNIST and Fashion-MNIST (Table 2). Standard deviations over five seeds of the post-hoc procedure are below 10−410^{-4}, so these differences lie well outside the margin of error.

Regarding your point:

Even in the Appendix, the gains over other classiic distance functions is minimal.

GAPA outperforms every distance-based method in every setting we tested. This holds for kNN, RBF and Mahalanobis distances on MNIST, Fashion-MNIST and CIFAR-10 (Table 3). The strongest of them, nearest-neighbour distance, comes close but never overtakes GAPA. This is expected, since GAPA provides a full predictive distribution, whereas these distance-based methods only provide a single score.

In short, the small benchmarks allow a direct comparison with every baseline, and GAPA already leads there on BALD AUROC, while the larger experiments show that it scales to settings where most baselines cannot be run.


Q1. How are the hyperparameters in the ablation studies determined (ex: GAPA layers)? Are they selected using a validation set or using the test set?

Neither. We set GAPA's hyperparameters without any validation or test data. They come from empirical statistics of the training pre-activations alone, without labels. We use standard data-driven choices for GPs: the kernel length scale is set by the median heuristic, the signal variances are the empirical variances of the activations, and the inducing points are k-means centroids (Section 2.5, Appendix I). The test set is used only for evaluation.

We also ran ablations on the layer, K and M (Appendix O). The main results use fixed defaults for these, and the ablations vary them only to show how sensitive the results are, not to choose them.

We hope this addresses your concerns, and we are happy to clarify any point further.

My issues were resolved. I don't have a problem with a weak acceptance.

Reviewer_JDEYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GAPA, a post-hoc uncertainty quantification method for frozen pretrained neural networks. GAPA replaces selected deterministic activation functions with Gaussian processes whose posterior means match the original activation functions, preserving the network's deterministic mean path while introducing activation-space uncertainty. The resulting GP posterior variance is interpreted as a support-based epistemic uncertainty signal, reflecting the coverage of cached reference pre-activations. To improve scalability, the method combines inducing-point approximations with local nearest-neighbor conditioning and propagates activation-space variances through the network using deterministic approximation rules. The authors evaluate GAPA across regression, classification, image segmentation, and language modeling tasks, reporting competitive calibration and out-of-distribution detection performance with favorable inference efficiency.

Critical strengths

1. Practical Post-hoc Uncertainty Quantification with Mean Preservation

The paper addresses a practically relevant problem by introducing uncertainty quantification for frozen pretrained networks without requiring retraining or modifying the original deterministic mean path. By constructing Gaussian-process activations whose posterior means match the original activation functions, GAPA separates uncertainty estimation from the backbone's mean computation. This is an appealing design principle for deployment settings where pretrained models must remain unchanged.

2. Scalable Activation-Space Uncertainty Estimation

The proposed framework combines inducing-point approximations, local nearest-neighbor conditioning, and deterministic variance propagation to make activation-space uncertainty estimation computationally feasible for large neural networks. Its extension to transformer architectures and evaluation across diverse prediction tasks demonstrate practical applicability. In particular, the reported inference-efficiency and OOD-detection trade-offs represent a meaningful engineering contribution.

Critical weaknesses

1. Insufficient Conceptual Justification for Predictive Epistemic Uncertainty

The paper does not sufficiently justify the connection between the proposed activation-space GP variance and predictive epistemic uncertainty. GAPA constructs an auxiliary GP over known deterministic activation functions, with posterior variance determined by the coverage of cached pre-activations. Although the authors explicitly interpret this variance as a support-based epistemic signal, it remains unclear under what assumptions the lack of support in activation space reliably reflects uncertainty about task predictions or prediction errors. In particular, the method preserves the original mean path while its uncertainty estimates depend on the reference activation cache, raising questions about the relationship between the resulting variance and task-level epistemic uncertainty. A clearer theoretical or empirical justification of this connection is needed.

2. Insufficient Empirical Validation of the Reported Results

The experimental evaluation does not sufficiently isolate the contribution of the proposed GP-based uncertainty mechanism. In particular, controlled comparisons with simpler distance-based uncertainty measures computed from the same activation representations, such as kNN or Mahalanobis distance, are lacking. The regression experiments also introduce a separately trained aleatoric uncertainty head without adequately separating its contribution from that of the proposed epistemic estimator. Furthermore, several results remain insufficiently explained, including calibration degradation on MNIST and unfavorable NLL comparisons on certain ResNet architectures. Together, these issues make it difficult to assess the specific contribution of the proposed method.

Questions for authors
  • GAPA interprets the posterior variance of an auxiliary GP over deterministic activation functions as a support-based epistemic uncertainty signal. However, it remains unclear under what conditions this activation-space uncertainty meaningfully reflects uncertainty about task predictions, rather than merely measuring novelty relative to the reference activation cache. Could the authors clarify the theoretical or empirical justification for this connection, particularly in cases where activation-space coverage does not align with predictive reliability?
  • What additional information does GAPA's GP-based uncertainty estimation provide beyond simpler distance-based measures computed from the same activation representations, such as kNN or Mahalanobis distance? Could the authors provide controlled comparisons and an ablation of the sensitivity-based variance propagation to isolate the contribution of the proposed mechanism? Furthermore, in the regression experiments, how much of the reported NLL improvement can be attributed to GAPA's epistemic variance rather than the separately trained aleatoric uncertainty head?
Reviewer_uZqROpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GAPA, a post-hoc, single-pass uncertainty quantification framework for frozen pretrained networks that shifts uncertainty modeling from parameter space to hidden activation space. In an offline setup stage, GAPA passes a reference training set through the frozen backbone, compresses selected pre-activation representations into inducing centroids via k-means with a nearest-neighbor index, and calibrates radial basis function kernel scales from empirical activation statistics. At inference time, deterministic nonlinearities at selected layers are replaced by coordinate-wise Gaussian processes whose prior mean is fixed to the original activation function and whose inducing targets match the exact deterministic activations. This construction preserves the pretrained model's point predictions while generating a local support-based variance that grows as test activations depart from training support and propagates forward in a single pass.

Critical strengths
  • Framing post-hoc uncertainty around hidden activations rather than high-dimensional weights offers a pragmatic, architecture-agnostic way to equip frozen backbones with single-pass uncertainty estimates without retraining or altering point predictions.
  • The empirical study is extensive, covering tabular regression, ResNet classification, U-Net segmentation, and LLaMA-3.2-3B OOD detection, and features helpful ablations on layer placement, inducing size, and propagation rules.
Critical weaknesses
  1. GAPA relies on the assumption that proximity in the frozen backbone's activation space is informative about predictive reliability. However, the paper does not investigate when this relationship holds or whether low activation-space uncertainty can coexist with substantial prediction errors.
  2. Although the core GAPA module does not require task labels or fine-tuning, the regression experiments additionally train a supervised aleatoric noise head. It remains unclear how much of the reported predictive improvement can be attributed to GAPA's activation-space variance alone. Above all, my major concern is activation-space uncertainty effect, I'm open to discuss further.
Questions for authors
  1. To what extent does GAPA's support-based variance correlate with actual prediction errors, particularly for incorrectly predicted samples lying in well-supported activation regions?
  2. Since multi-layer placement can hurt calibration without improving OOD detection over a last-layer module, when is multi-layer propagation preferable to last-layer GAPA, and how does regression perform without the learned noise head?
1 comment

Thank you for finding GAPA a pragmatic, architecture-agnostic way to give frozen backbones single-pass uncertainty, and our empirical study extensive, with helpful ablations. We address each of your points below.


W1 / Q1. GAPA relies on the assumption that proximity in the frozen backbone's activation space is informative about predictive reliability. However, the paper does not investigate when this relationship holds or whether low activation-space uncertainty can coexist with substantial prediction errors.

The paper does test this relationship, through negative log-likelihood (NLL) and out-of-distribution (OOD) detection. GAPA has the best NLL of all methods on all three regression datasets (Table 1), and it lowers the backbone's NLL on Fashion-MNIST (Table 2) and on all four CIFAR-10 ResNets (Table 4). In OOD detection, GAPA ranks first or second on every benchmark, against up to 12 baselines (Tables 2 and 4, Figure 3).

NLL directly measures how well a model's uncertainty matches its errors, penalising confident wrong predictions and needlessly uncertain correct ones. In regression, the NLL of a prediction with mean μ\mu and variance σ2\sigma^2 is (y−μ)22σ2+12log⁡(2πσ2)\frac{(y-\mu)^2}{2\sigma^2} + \frac{1}{2}\log(2\pi\sigma^2), which is lowest when σ2\sigma^2 matches the squared error (y−μ)2(y-\mu)^2. In classification, the NLL −log⁡p(y∣x)-\log p(y \mid x) is near zero for a confident correct prediction and very large for a confident wrong one. Since GAPA leaves the backbone's mean prediction unchanged, its NLL gains come from its uncertainty alone.

OOD detection directly measures whether inputs unlike anything in the training data receive higher uncertainty than familiar ones. Such inputs, like clothing images given to a digit classifier, match none of the model's classes, so its predictions on them are always wrong.

In short, the paper already tests this relationship directly, and GAPA has the best regression NLL of all methods and ranks first or second in OOD detection on every benchmark.


W2 / Q2. Although the core GAPA module does not require task labels or fine-tuning, the regression experiments additionally train a supervised aleatoric noise head. It remains unclear how much of the reported predictive improvement can be attributed to GAPA's activation-space variance alone. Above all, my major concern is activation-space uncertainty effect, I'm open to discuss further. [...] how does regression perform without the learned noise head?

GAPA's activation-space variance alone improves regression NLL on all three datasets. In Table 1, GAPA differs from MAP, the frozen backbone, in two ways. It adds the activation-space variance, and it replaces MAP's constant observation noise with a learned noise head. To separate the two, we reran GAPA with the same constant noise as MAP, one noise variance for all inputs, as in the Laplace baselines and VaLLA.

ModelAirlineYearTaxi
MAP + constant noise (Table 1)5.1213.6733.775
GAPA + constant noise (new)4.9993.4703.543
GAPA + learned noise head (Table 1)4.9463.4703.112

Of the total NLL improvement over MAP, the activation-space variance accounts for about 70% on Airline (0.122 of 0.175), all of it on Year (0.203), and 35% on Taxi (0.232 of 0.663). The learned head, which models input-dependent noise, adds the rest. Even with constant noise, GAPA beats every baseline in Table 1 on Year, and every baseline except VaLLA on Airline and Taxi. We will add this table to the appendix.


Q2. Since multi-layer placement can hurt calibration without improving OOD detection over a last-layer module, when is multi-layer propagation preferable to last-layer GAPA [...]?

Multi-layer placement is preferable when calibration matters more than OOD detection. For OOD detection alone, the last layer is enough, as you note, since adding layers to it raises NLL and changes OOD AUROC by at most 0.004 (Table 7). Intermediate layers instead give the lowest NLL, and combining them recovers part of the OOD gain at almost no cost in NLL.

On Fashion-MNIST, for example, GAPA at layer 3 alone has the lowest NLL (0.309) but OOD AUROC 0.901. Using layers 1–3 raises OOD AUROC to 0.924, and BALD AUROC from 0.911 to 0.934, with NLL 0.312. GAPA at the last layer alone reaches OOD AUROC 0.973, but with NLL 0.353 (Table 7). For OOD detection, the last layer alone is therefore the natural choice, and in LLaMA-3.2-3B, OOD detection also peaks at the last block (Figure 3). We will add this guidance to the paper.

We hope this addresses your concerns about the effect of the activation-space uncertainty, and we are happy to discuss further.

Reviewer_GMCYOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose GAPA, a post-hoc uncertainty method for frozen pretrained networks. Selected nonlinearities ϕ\phi are replaced by Gaussian-process activations, with each GP’s prior mean being ϕ\phi itself. The posterior variance acts as a support-based epistemic uncertainty. The tests are extensive and the authors claim that the contribution is "competitive with strong post-hoc baselines and often improves calibration or OOD detection".

Critical strengths
  • the method is retraining-free, Jac-free and Hessian-free. The main additional cost is a stored state in the form of centroid sets per layer, which is reasonable even for large architectures
  • mean logits match MAP logits exactly
  • the results are extensive for a post-hoc UQ method; it is applied to classification, regression, segmentation and transformer architectures, which is not common 
  • the ablations (cache coverage, Table 6, layer/M/K sweeps, the controlled distance-baseline comparison in the same activation space) are extensive and honest, even with the risk of undermining sometimes the main selling points
Critical weaknesses
  • the method is retraining-free, Jac-free and Hessian-free. The main additional cost is a stored state in the form of centroid sets per layer, which is reasonable even for large architectures

  • mean logits match MAP logits exactly

  • the results are extensive for a post-hoc UQ method; it is applied to classification, regression, segmentation and transformer architectures, which is not common 

  • the ablations (cache coverage, Table 6, layer/M/K sweeps, the controlled distance-baseline comparison in the same activation space) are extensive and honest, even with the risk of undermining sometimes the main selling points

  • in Table 6, there are near-identical AUROCs of the diagonal single-pass implementation and the full-covariance Monte Carlo reference. This supports the hypothesis that the OOD signal is driven mainly by the variance injected at the GAPA layer rather than by its propagation. One way to partly settle this would be to report the AUROC of the un-propagated GAPA variance at the insertion layer alongside the propagated score

  • the mean preservation is not guaranteed as far as I understand : even if mean logits are unchanged, the predictive class probabilities are softmaxed along with vcv_c which are class dependent and may change the argmax; cf Ortega et al 2023 : "VaLLA improves prediction accuracy (ACC) due to the approximation". If this is correct, the right approach would be to just report argmax-agreement rates and tone down the current assertion

  • no validation OOD protocol is described; if hyperparameters are selected on the test OOD set, this will blow every AUROC in the paper

  • the classification validation is pretty thin with  MNIST/FMNIST and CIFAR-1/SVHN; I think  CIFAR-100 for near-OOD and CIFAR‑10‑C would be the minimum

Questions for authors

See above and :

  • there's nothing being done about avoiding a purely diagonal output covariance matrix. In deep CNN feature maps or dense LLM embedding spaces, is the impact of this assumption really negligible so that it's not worth a discussion?

  • the propagation of variances through non-linearities relies on a first-order approximation, which is part of the price to pay for the low cost of the method. How does this affect the variance propagation in unfavourable conditions (highly non-linear regions, high input variance) ?

  • for regression, a separately trained aleatoric likelihood head is added. How does this stand with respect to the main claim of a post-hoc, gradient free approach?

  • the empirical median (used for the scale of the RBF kernel) is expected to be brittle in high dimensional spaces (concentration of distances). When and how does this impact the method?

3 comments

Thank you for your careful review and for noting that GAPA needs no retraining, Jacobians or Hessians, that its results are unusually broad for a post-hoc method, and that its ablations are extensive and honest. We answer each point below, with new experiments for W1, W2, W4 and Q1/Q2.


W1. [...] the OOD signal is driven mainly by the variance injected at the GAPA layer rather than by its propagation. One way to partly settle this would be to report the AUROC of the un-propagated GAPA variance at the insertion layer alongside the propagated score

We ran the comparison you propose. For OOD detection the variance at the insertion layer already suffices, as you suggest. Propagation is what makes GAPA's score detect the model's own errors.

We compare two scores: the un-propagated score, which is the average GAPA variance at the layer where GAPA is inserted, and the propagated score, which is BALD at the output after that variance is propagated through the rest of the network. The Errors columns measure how well each score separates the test inputs the model gets wrong from those it gets right. Here we use two-layer MLPs rather than the four-layer MLPs of the paper, so the numbers differ from those in Tables 2 and 3. GAPA is at their last hidden layer (five seeds, standard deviations below 0.0002) and on the penultimate features of the four CIFAR-10 ResNets (averaged; the pattern holds for each).

AUROCOOD, un-propagatedOOD, propagatedErrors, un-propagatedErrors, propagated
MLP, MNIST (OOD: FMNIST)0.9850.9520.8320.929
MLP, FMNIST (OOD: MNIST)0.9940.9940.6730.809
ResNets, CIFAR-10 (OOD: SVHN)0.9230.9130.8720.918

GAPA gives both scores in one forward pass. We will add this table to the appendix.


W2. the mean preservation is not guaranteed as far as I understand [...]. If this is correct, the right approach would be to just report argmax-agreement rates and tone down the current assertion

Yes, this is correct. GAPA preserves the mean logits exactly, but its predictive probabilities divide each logit by a class-dependent factor (Appendix K, Eq. 10), so their argmax can differ from the backbone's. GAPA's predicted class agrees with the backbone's on 100% of MNIST and Fashion-MNIST test inputs and on 99.99–100% of CIFAR-10 test inputs (ResNet-20/32/44/56), and accuracy changes by at most 0.01 percentage points. We will report these rates and replace "it matches MAP accuracy by construction" in Section 3.2, and "point predictions" in the abstract and Section 4, with statements that mean preservation applies to the logits. For regression, the predictive mean equals the backbone's output exactly.


W3. no validation OOD protocol is described; if hyperparameters are selected on the test OOD set, this will blow every AUROC in the paper

No hyperparameter is selected on OOD data, or on any validation or test data. GAPA never sees OOD data before evaluation. Its kernel hyperparameters (length scale from the median heuristic, signal variance from the empirical variance of the activations) and its inducing points (k-means) are computed from the training pre-activations alone, without labels (Section 2.5, Appendix I). The layer, the neighbourhood size and the number of inducing points are fixed defaults; the ablations in Appendix O vary them only to show sensitivity. We will describe this protocol in Section 2.5.


W4. [...] I think CIFAR‑100 for near-OOD and CIFAR‑10‑C would be the minimum

We ran CIFAR-10 → CIFAR-100 near-OOD: GAPA has the highest AUROC on all four ResNets, by a small margin. We used the four pretrained CIFAR-10 ResNets, with GAPA on the penultimate features, and compared it with last-layer Laplace on the same networks.

AUROCResNet-20ResNet-32ResNet-44ResNet-56
MAP, entropy0.8580.8500.8510.859
Last-layer Laplace, entropy0.8620.8560.8560.863
Last-layer Laplace, BALD0.8550.8550.8580.865
GAPA, BALD0.8630.8620.8660.873

We will add CIFAR-10-C to the final version. The paper already covers similar shifts. On rotated MNIST and Fashion-MNIST, GAPA has a lower NLL than the baselines at large rotations (Figure 8). On LLaMA-3.2-3B, it tells WikiText-103 apart from OpenWebText, two similar English corpora, with AUROC 0.879, while last-layer Laplace stays near 0.5 (Figure 3).

Q1. [...] a purely diagonal output covariance matrix. In deep CNN feature maps or dense LLM embedding spaces, is the impact of this assumption really negligible [...]?

Its impact on the size of the variance is not negligible, but it barely changes the ranking of inputs, which is what OOD and error detection use.

Table 6 tests both approximations on MLPs, where they change OOD AUROC by at most 0.003. To test a deep CNN, we inserted GAPA after stage 2 of ResNet-20 (8,192 units) and propagated its variance to the output with our single-pass rules or with 256-sample Monte Carlo, which keeps all covariances (five seeds; OOD: SVHN).

ResNet-20, GAPA after stage 2NLLOOD AUROCError AUROC
MAP0.2820.8810.922
Single pass, first-order (ours)0.2810.8810.922
Single pass, exact ReLU moments0.2810.8810.922
Monte Carlo, full covariance0.2380.8660.926

Without covariances, the variance at the output is too small by a factor of about 250, because convolutions and pooling sum correlated units. But the factor is about the same across inputs, in-distribution and OOD alike, so the ranking of inputs holds (Spearman 0.93 against Monte Carlo) and the AUROCs change by at most 0.015. Keeping the covariances does improve NLL. At this early layer the single-pass variance is too small to change the predictions, which is one reason we place GAPA near the output layer.

We could not run this on LLaMA within the discussion period. We will add the experiment to the paper, with a discussion of the diagonal assumption and of low-rank covariance as a possible fix.


Q2. [...] a first-order approximation [...]. How does this affect the variance propagation in unfavourable conditions (highly non-linear regions, high input variance) ?

In our tests it makes no difference. In the ResNet-20 experiment above, we replaced the first-order rule with exact Gaussian moments for each ReLU. Unlike the first-order rule, these are correct near the kink, where the ReLU is non-linear. The numbers do not change (third row of the table). We also split the test inputs into five groups by how much variance GAPA injects. The single-pass variance is about the same fraction of Monte Carlo's in every group, so the approximation does not get worse at high input variance.


Q3. for regression, a separately trained aleatoric likelihood head is added. How does this stand with respect to the main claim of a post-hoc, gradient free approach?

The head does use labels and gradients, but it is trained post hoc on the frozen network and changes neither the backbone nor GAPA. GAPA's variance is epistemic, so it cannot model observation noise, which a Gaussian likelihood needs. For regression only, we therefore train a small head that predicts the noise variance, by minimising the Gaussian NLL with the backbone's mean and GAPA's variance held fixed (Appendix I.2). Only the head's weights change, so the mean predictions are still the backbone's. Our gradient-free claim refers to the GAPA module itself, which uses no gradients or labels (Section 2.5). We will make this explicit in the paper, and note that the head is optional, since GAPA also lowers NLL without it (with the backbone's constant noise, from 5.121 to 4.999 on Airline, 3.673 to 3.470 on Year and 3.775 to 3.543 on Taxi).


Q4. the empirical median [...] is expected to be brittle in high dimensional spaces (concentration of distances). When and how does this impact the method?

High dimensionality alone does not necessarily make the median brittle. What matters is whether the learned representation maps similar inputs to nearby activations, so that a test input's nearest cached activations are much closer than typical ones.

Our largest experiment tests this. On LLaMA-3.2-3B, GAPA uses the median unchanged on about 12 million cached 3,072-dimensional pre-activations and reaches an OOD AUROC of 0.879. AUROC improves as the inducing set and the neighbourhood grow (Figure 3), and k-means beats random inducing points (Figure 7), so the local neighbourhoods are informative. At the MLP and ResNet layers we use, a test input's nearest cached activation lies at only 0.08–0.24 of the median distance, so distances do not concentrate there either. This agrees with prior work that uses nearest neighbours in learned representations, such as Deep kNN [1] and kNN-LMs [2].

GAPA can still degrade if distances in the chosen layer do concentrate, since all kernel values are then similar and the variance no longer separates inputs. We will add this limitation explicitly.

We are happy to clarify any point further.

References:

[1] Papernot & McDaniel. Deep k-Nearest Neighbors. arXiv:1803.04765, 2018.

[2] Khandelwal et al. Generalization through Memorization: Nearest Neighbor Language Models. ICLR 2020.

Reviewer_GMCYOpenReview ↗

Answer evaluation

The authors' thoughtful response addresses my concerns in depth, and upholds my initial positive assessment of their work. Including the results above would make the paper very clear about its scope (it was pretty honest to begin with), and at this point I think a weak accept is pretty harsh. I think a rating of 4 is closer to its current quality, and preferable to a 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.