acceptodds
Under review as a conference paper at ICLR 2027

A Bayesian Perspective on the Role of Epistemic Uncertainty for Delayed Generalization in In-Context Learning

Abstract

Learning to generalize from finite data is a central problem in machine learning. In-context learning provides a concrete solution to this problem while exhibiting a striking phenomenon known as delayed generalization: transformers can perfectly fit observed task-input combinations long before they can infer a held-out task from in-context examples. Although prior work has substantially advanced our understanding of this phenomenon, the evolution of uncertainty during this transition remains comparatively less explored. We study this question in a modular-arithmetic benchmark with latent linear tasks, using Improved Variational Online Newton (IVON) and a Last-Layer Laplace approximation to estimate posterior-predictive epistemic uncertainty (EU). Across different levels of task diversity and training-input coverage, both posterior approximations show lower epistemic uncertainty in configurations that generalize to held-out tasks. Furthermore, we show that EU follows a non-monotone pattern of rise-peak-decline during training, with its peak preceding the late increase in accuracy, in held-out tasks. Finally, we attempt to formalize this behavior with a simplified Bayesian linear model that reproduces the qualitative dynamics, showing that delayed generalization and non-monotone uncertainty can arise from slow spectral modes of the training-data covariance.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
In-context learning, Bayesian deep learning, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_b5HhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the relationship between delayed generalization and epistemic uncertainty in ICL. Predictive uncertainty is estimated using IVON and Last-Layer Laplace on a modular arithmetic benchmark with latent linear tasks. Experimental results reveal that models generalization ability to new tasks exhibit lower epistemic uncertainty. Moreover, uncertainty during training does not decrease monotonically, and namely, it first increases to a peak before declining. Furthermore, using a simplified Bayesian linear model, this work theoretically demonstrates that slow-learning eigenmodes in the covariance matrix of training data can simultaneously give rise to both delayed generalization and non-monotonic dynamics in uncertainty.

Critical strengths

I found it interesting how observing changes in epistemic uncertainty could help us understand how models acquire generalization ability. Also, I believe that the fact that the improvement in performance and the change in uncertainty (self-assessment) don't have a simple monotonic relationship suggests an intriguing parallel between human metacognition and AI uncertainty estimation.

Critical weaknesses

The task in Eq. (1) seems quite simple. Once two in-context examples are provided, the model should be able to solve the subsequent pairs (x3,y3)(x_3, y_3) and beyond. So, I have a concern that the problem setting might be very toy and a question regarding whether pretraining is sufficient.

Questions for authors

In relation to the weakness mentioned above, could you share any results regarding sufficient training convergence? To the best of my knowledge, even in toy model setups, the emergence of ICL requires a fairly large sample size (and number of training steps). For instance, Garg et al. (2022) and many subsequent studies adopt massive online training even when working with toy models.

I would appreciate it and reconsider the soundness score and the rate if you could share your thoughts on this aspect.


Garg et al. (2022), What Can Transformers Learn In-Context? A Case Study of Simple Function Classes, NeurIPS 2022.

4 comments

We thank the reviewer for the constructive comments and for highlighting the relationship between generalization and uncertainty estimation.

W1: Task identifiability and learning from context

The reviewer is correct that two examples identify the latent task (a,b)(a,b) when their input pairs are linearly independent modulo 29. For inputs (x1,y1)(x_1,y_1) and (x2,y2)(x_2,y_2), this condition is x1y2−x2y1≢0(mod29)x_1y_2-x_2y_1\not\equiv0\pmod{29}. An exact modular linear solver can then recover (a,b)(a,b) and predict every subsequent query.

The transformer learns its prediction procedure from sequences of numeric tokens. Our context-length experiments measure how effectively this learned procedure uses the available examples. At fixed weights, OOD-Val accuracy remains low with at most two examples and increases as more examples are provided (Figures 5, 8, and 9). For IVON at (T=256,α=0.8)(T=256,\alpha=0.8), median accuracy across the three evaluated seeds increases from 0.033 with no examples to 0.871 with 31 examples. The same trained models therefore exhibit substantial held-out-task generalization when given sufficient context.

He et al. (2024) [1] provide a relevant mechanistic account on this benchmark. With few examples, their two- and four-layer models use Ratio Matching, which succeeds when the query input is a modular multiple of an input in the context. With more examples, their four-layer models combine inputs through Modular Regression and achieve much higher OOD accuracy. This analysis illustrates how the learned procedure can require more context than an exact solver. Our experiments characterize the accompanying changes in epistemic uncertainty (EU) in six-layer models; identifying the precise algorithm implemented by these models remains a separate mechanistic question.

W1 & Q1: Training duration and evidence of ICL

Appendix E specifies a six-layer transformer, modulus p=29p=29, batches of 1024 sequences, and K=32K=32 triplets per sequence, corresponding to 31 in-context examples and one query. Training supervises all answer positions and uses task-balanced batches with the parallelogram task construction of He et al. We vary the effective number of training tasks over T∈{32,64,128,256,512}T\in\{32,64,128,256,512\} and input coverage over α∈{0.3,0.4,0.5,0.6,0.7,0.8}\alpha\in\{0.3,0.4,0.5,0.6,0.7,0.8\}, with four seeds per configuration. The default horizon is 100,000 updates, extended to 200,000 for T=512T=512; the latter matches the training horizon in He et al. At the nominal batch size, these schedules correspond to approximately 1.02×1081.02\times10^8 and 2.05×1082.05\times10^8 sequence presentations. These counts include repeated exposure to the finite training tasks and inputs.

The reported schedules are sufficient for ICL to emerge in the generalizing configurations. Across the 120 IVON runs, 37 achieve stable generalization to unseen tasks on familiar inputs only, and nine also achieve stable generalization when both tasks and inputs are held out. Here, stable generalization means a median accuracy of at least 0.75 over the final five evaluation checkpoints (Appendix A.1). The fixed-weight context-length results above further demonstrate that the trained models use additional in-context evidence to predict held-out tasks.

Figure 14 (new, Appendix A.4) reports training loss and accuracy curves for all configurations of the MAP (AdamW) runs, and shows how the training horizon follows the dynamics of each regime. For T≤128T\le128, the training loss drops to near zero within the first few thousand updates, while accuracy on held-out tasks remains near chance for the entire horizon: these runs fit their training set and show no transition to held-out tasks. For T=256T=256 with α≥0.5\alpha\ge0.5, accuracy on all splits rises sharply and plateaus within the 100,000-update horizon. For T=512T=512, transitions to OOD generalization unfold more slowly; for computational reasons, we therefore reserve the longer 200,000-update horizon, which matches He et al., for this largest task count. All nine runs used in our peak-order analysis (Table 3) come from configurations whose accuracy plateaus during training (T=256T=256 with α∈{0.6,0.7,0.8}\alpha\in\{0.6,0.7,0.8\} and T=512T=512 with α=0.8\alpha=0.8).

He et al. also report that OOD generalization can emerge and subsequently deteriorate during training in six-layer models [1]. This motivates examining the full accuracy and uncertainty trajectories alongside final performance, as we do in Section 3.3.

AuthorsOpenReview ↗

Response pt2

Q1: Comparison with Garg et al. (2022)

Garg et al. [2] train their standard linear-regression model using fresh functions and inputs sampled from Gaussian distributions in 20 dimensions. Their schedule uses 500,000 updates with 64 prompts per batch, totaling 3.2×1073.2\times10^7 prompts. Our training volume is of the same order, while our experiment explicitly controls the diversity of a finite task family with 292=84129^2=841 possible task vectors. We reserve task vectors for evaluation and construct training sequences from the designated training tasks and inputs. This separation lets us study how finite pretraining diversity affects subsequent inference of unseen tasks from context.

Garg et al. also examine training with a finite set of functions: their Section 6 reports performance close to their unrestricted model with 10,000 distinct linear functions. Thus, finite task diversity is itself an informative experimental variable for studying the emergence of ICL. In our benchmark, varying this quantity exposes the generalization regimes whose uncertainty dynamics we analyze.

We hope these clarifications resolve the reviewer's concerns, and we remain available for any further discussion.

References

[1] Tianyu He, Darshil Doshi, Aritra Das, and Andrey Gromov. Learning to Grok: Emergence of In-Context Learning and Skill Composition in Modular Arithmetic Tasks. NeurIPS, 2024.

[2] Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. NeurIPS, 2022.

Dear Reviewer,

We would appreciate hearing whether our response addressed your questions, in particular on task simplicity, training sufficiency (context-length results and new training curves in Figure 14) and the comparison with Garg et al. (2022).

If any concern remains, please let us know while there is time to discuss it. Otherwise, we would be grateful if you would consider updating your rating, as you previously kindly suggested.

Thank you for your time.

Thank you for your response. Your rebuttal resolved most of my concerns. In particular, comparison with Garg et al. makes sense. I will keep my positive evaluation.

Reviewer_XNMSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how posterior-predictive epistemic uncertainty evolves during delayed generalization in in-context learning. In a modular-arithmetic benchmark with latent linear tasks, authors use Improved Variational Online Newton and a Last-Layer Laplace approximation for epistemic uncertainty quantification, and report behavior of the uncertainty under various scenarios.

Critical strengths
  • The research topic of epistemic uncertainty quantification and its behavior in delayed generalization of in-context learning is insteresting and underexplored.
  • The experiment results are generally valid and interpretable. The relationship between accuracy and uncertainty as well as aleatoric and epistemic uncertainty separation are all well illustrated in figures.
  • The temporal analysis of uncertainty across training are intuitive and useful.
  • Experiments on IVON and Last-Layer Laplace show the uncertainty behaviors are not restricted to a single method of EU quantification.
Critical weaknesses
  • Experiments are mainly restricted to a small synthetic benchmark and model architecture. The generalizability of the observations remains questionable.
  • The results are mostly qualitative. Providing more quantitative analyses in the experiments can help further strengthen the claims of this paper.
  • The sensitivity of the conclusions to the choice of posterior EU quantification methods is worth discussing.
Questions for authors

What is the uncertainty behavior in different benchmarks, tasks, architectures, and uncertainty quantification methods? Providing quantitative numbers additionally can make the conclusions more convincing.

3 comments

We thank the reviewer for the constructive assessment and for recognizing the interest of studying epistemic uncertainty during delayed generalization in in-context learning.

W1 & Q1: Benchmark and architecture

We thank the reviewer for this comment. We agree that our empirical study focuses on modular arithmetic, and we state this in the Limitations paragraph of Section 5.

(a) Benchmark. Modular arithmetic is the canonical controlled setting for grokking, widely used because it permits a precise separation between memorization and generalization [12, 22, 24]. Our setting follows the modular-arithmetic ICL benchmark of He et al. [12], where the model must infer a latent task from in-context examples alone, and which exhibits non-trivial in-context learning, delayed generalization, and challenging out-of-distribution behavior. It also allows us to separate generalization to unseen inputs, to unseen tasks, and to both (ID Val, OOD Train, OOD Val), which is not possible with natural data.

(b) Architecture. He et al. study depths 2, 4 and 6 on this benchmark. We adopt their 6-layer configuration (Appendix E.2), which is the depth used for their main experiments and exhibits the full transition from memorization to out-of-distribution generalization; shallower models show degraded performance on both in-distribution and out-of-distribution tasks, making them less suitable for our study of uncertainty dynamics. Due to the computational cost of training all these configurations, with multiple seeds and relevant hyperparameter sweeps, we choose to focus on the 6-layer model which was already proven to be enough to exhibit the phenomena of interest.

(c) Beyond modular arithmetic. Section 4 provides support at the theoretical level: a Bayesian linear-Gaussian model reproduces delayed generalization, a non-monotone EU, and an EU peak before the accuracy threshold.

W2 & Q1: Quantitative evidence

We thank the reviewer for this suggestion. The paper already includes quantitative analyses in Table 1 and Appendix A. We can further highlight the following points:

  • EU reduction (Tables 1 and 5): generalizing configurations have 57% lower EU with IVON, 68% with Laplace, and 41% with MCD in the selected comparisons, while total uncertainty changes modestly. This supports the claim that EU is a more sensitive signal of generalization than total uncertainty.

  • EU decline across regimes (Table 2): median EU decline is 57.4% in runs that generalize on OOD Val, compared with 0.8% in runs that generalize on neither OOD split. This instead shows that during training, EU declines substantially in runs that generalize to unseen tasks, while it remains nearly constant in runs that do not (again supporting the analysis of the uncertainty dynamics in Section 3.3).

We will move Tables 2 and 3 into Section 3.3 and highlight these quantitative results alongside the corresponding claims in the main text.

W3 & Q1: Sensitivity to the posterior approximation

During the rebuttal phase, we trained additional models to evaluate the sensitivity of our conclusions to the posterior approximation. In addition to the IVON and Last-Layer Laplace methods reported in the paper, we extended our analysis to Monte Carlo dropout (MCD) [8], which estimates predictive uncertainty using stochastic dropout masks. We use dropout probability 0.1 and 16 prediction passes, with 120 runs covering the same task-count and input-coverage grid and four seeds per configuration. Appendix A.3 reports the accuracy maps, uncertainty decomposition, and extended uncertainty comparison (Figures 12–13 and Table 5).

The two new figures show that MCD exhibits the same qualitative uncertainty dynamics as IVON and Last-Layer Laplace, albeit the generalization boundary is shifted to higher task counts and input coverage. This is expected, as MCD is known to be a less accurate posterior approximation than IVON and Last-Layer Laplace.

Regarding the argument of EU decline as a signal of grokking, let's consider the following comparison of MCD runs:

MethodGrokked?(T,α)(T,\alpha)TUAUEU
MCDNo(32, 0.7)2.5302.3030.227
MCDYes(256, 0.7)2.4182.283 ↓1%0.135 ↓41%

EU decreases by 41%, while total uncertainty changes by 4.4%. These results fully agree with the paper's central observation: (i) EU declines substantially when the model generalizes to unseen tasks, and (ii) predictive entropy alone is insufficient to study grokking.

We thank the reviewer for suggesting this additional experiment, which strengthens our conclusions and demonstrates that they are indeed robust to the choice of posterior approximation.

Dear Reviewer,

Following your helpful suggestion, we additionally ran new experiments with Monte Carlo dropout (Appendix A.3), which confirm that our conclusions hold across posterior approximations, and we highlighted the quantitative results on EU reduction and decline (see Tables 1, 2 and 5). We would welcome your feedback on whether these additions address your concerns, and we are happy to clarify any further doubt before the discussion closes.

Thank you for your time.

Thank authors for your rebuttal. It solves most of my current concerns about the submission. After reading other people's review and the responses, I decide to keep my rating.

Reviewer_bcZHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the role of epistemic uncertainty (EU) in the background of delayed generalization (grokking) via in-context learning. The task is a modular-arithmetic benchmark. The main observation is that EU can serve as a signal of grokking, and often rises, peaks, and declines before a late OOD-validation accuracy threshold. A Bayesian linear-regression model and corresponding theoretical analysis obtain a related uncertainty trajectory and an asymptotic peak-delay formula.

Critical strengths
  1. Studying EU as a signal of grokking is an interesting idea, and the empirical evidence is supportive. Distinguishing unseen inputs, unseen tasks, and their combination makes the generalization claims more precise.

  2. The Gaussian mean/covariance trajectories and the distinction between training-weighted and population-weighted spectral risk are useful starting points for studying learning dynamics.

Critical weaknesses
  1. There is a gap between the model used in theoretical analysis and the benchmark studied in empirical experiments. For example, the theoretical model uses linear regression as the task, yet the benchmark is more of a classification problem. This leads to different definitions of uncertainties: for classification tasks, it's natural to consider the entropy, yet for regression tasks, the authors turn to a thresholded success event. This event is not very well justified and not so clear as the classification case. Questions remain, such as the choice of ε\varepsilon and the decomposition of aleatoric and epistemic uncertainties. At the very least, how two different definitions of uncertainties can make the theoretical analysis a support for the empirical observation should be discussed.
Questions for authors
  1. I'm curious, why do the authors use the thresholded success event as a measure of uncertainty in the regression model? Clearly, there is a natural definition of uncertainty based on Σ\Sigma for regression tasks, and that fits more closely to the empirical results. Or, on the contrary, why don't the authors study the empirical performance of the regression model used in theoretical analysis but use the arithmetic modular benchmark?

  2. Some flaws in theoretical analysis: Considering Jensen's inequality, Ex,yh2(pt(x,y))≤h2(Ex,ypt(x,y)),E_{x,y} h_2(p_t(x,y)) \le h_2(E_{x,y}p_t(x,y)), which makes me a little confused on Lemma 4.1, why do you use the second term as an approximation of EU?

4 comments

We thank the reviewer for the constructive comments and for recognizing the interest of studying epistemic uncertainty during delayed generalization.

W1: Connection between theory and experiments

We believe this concern stems from a misunderstanding: theory and experiments use the same definitions of uncertainty. The entropy decomposition into aleatoric and epistemic components is standard [13, 14, 33]. In both settings, total uncertainty is predictive entropy, aleatoric uncertainty is the posterior-averaged conditional entropy, and epistemic uncertainty is their difference. Our theoretical analysis applies it to a tractable binary classification model.

The paper directly addresses why total predictive entropy alone is insufficient to study grokking: Table 1 shows a substantial reduction in epistemic uncertainty alongside only modest changes in total entropy. The increase in the aleatoric component partially offsets the decrease in the epistemic component. Separating these components therefore reveals changes associated with generalization that total entropy obscures.

Q1: Choice of the threshold event

As clarified above, both settings use the same uncertainty definitions. We also derive epistemic uncertainty for the continuous output before thresholding in Appendix D. This quantity describes uncertainty in a regression problem, so it offers limited insight into the transformer's classification behavior. Following Levi et al. [18], we therefore construct a binary classification event by declaring a prediction successful when its squared error falls below a tolerance. This construction makes the analysis tractable while capturing delayed generalization and the qualitative uncertainty dynamics observed in the full transformer. For a fixed evaluation pair, the event is deterministic conditional on the parameters, so its aleatoric entropy vanishes. The same decomposition then gives epistemic uncertainty as the entropy of the posterior success probability.

Q2: Averaging and Jensen's inequality

Thank you for spotting this imprecision. We clarified that Equation 14 approximates Jensen's upper bound. The corresponding argument for actual EU relies on nondecreasing population accuracy, which holds in our Gaussian setting, as proved in the updated appendix (Proposition D.7/8). All the results remain valid and unchanged.

Thanks for your update, and thank you for addressing the flaw mentioned in Q2. But I'm still not very buying the story regarding the W1 and Q1. Here are some more detailed explanations.

The choice of uncertainty in the experiment is fine; very standard uncertainty decomposition. What worries me is in the theory. The theory choosed the task of regression. That's kind of different from the experiments, but let's take it as OK. What's not OK (in my opinion) is that the definition of the epistemic uncertainty used in the theory takes the true lable into it. The epistemic uncertainty is to measure "how uncertain the model's prediction is"; even without the knowledge of the true label, the definition should still work. Current definition is more like "how uncertain the model's prediction precision is": how uncertain it is for the event that the prediction falls in a small neighborhood of the true label (let's call that event as ZϵZ_\epsilon). That's a different amount from the epistemic uncertainty of the prediction. To be more precise, it's I(Y;Θ∣X,D)I(Y;\Theta\mid X,\mathcal D) v.s. I(Zϵ;Θ∣X,Y,D)I(Z_\epsilon;\Theta\mid X,Y,\mathcal D).

For the Gaussian regression model, a more natural definition of epistemic uncertainty, as the authors acknowledged in the Appendix, is $\frac12\log( 1+\frac{x^\top\Sigma_t x}{\sigma^2} ).$ The authors provided their reason why they didn't use that: "This quantity describes uncertainty in a regression problem, so it offers limited insight into the transformer's classification behavior." It makes sense, if the authors are to study the classification problem. But that brings us a more severe concern: is the observed "grow-peak-decline" trajectory an artifact of the thresholding event? Actually, it is a very seriour alternative that the current defined uncertainty cannot rule out. Let's consider a very very simple task: a linear regression model. No grokking and no transformers at all. Denote pϵp_\epsilon as the "success probability" that the final prediction falls into an ϵ\epsilon-neighborhood of the true label. Let's train the linear regression model similarly by observing sequences of samples. As the training goes on, we should expect that the prediction is being more precise, so, a natural trajectory of pϵp_\epsilon (which is the probability of ZϵZ_\epsilon) should be: first, close to 0 (because the model has nearly no knowledge); then, as the learning processes, pϵp_\epsilon increases; as the learning nearly saturates, pϵp_\epsilon will stay close to 1. But, if we consider the entropy of (pϵ,1−pϵ)(p_\epsilon, 1-p_\epsilon), the entropy will be "grow-peak-decline", since the distance of pϵp_\epsilon to 12\frac12 is first decreasing and later increasing. In other words, that "grow-peak-decline" curve would happen even for this very simple learning setting (no grokking at all). That alternative cannot be ruled out in current theory: the curve could happen for any effective learning rather than some phenomenon specifically happens for grokking.

For the flaw mentioned in Q2, it's still possible that the upper bound is loose and its pattern ("grow-peak-decline") is different from the interested amount. But that's something minor, compared to what I mentioned in the previous point.

I'm not against studying the grokking via epistemic uncertainty; actually I find the idea and the experiments interesting, and that's part of the reason why I didn't give a 1 to the manuscript. But, does the theory actually indicate the dynamics of grokking? I don't quite think so.

Thank you for clarifying your concerns. We may have misunderstood a nuance of your original review. Your comment raises two distinct questions: whether the theoretical quantity has a predictive epistemic interpretation without access to the evaluation label, and whether its trajectory is specific to grokking.

1. What we mean by epistemic uncertainty in the theory

You are right that uncertainty about whether a prediction satisfies an accuracy criterion is generally different from uncertainty about the prediction itself. We will make this distinction explicit in the revised manuscript. However, dependence on ground truth does not, by itself, preclude an epistemic interpretation.

This is not unprecedented in the literature: other work studies epistemic uncertainty through excess risk, costs, and prediction errors, including Depeweg et al. (2018), Lahlou et al. (2023), Lahoti et al. (2023), and Kotelevskii et al. (2025).

DEUP (Lahlou et al., 2023) is particularly relevant here because it explicitly uses excess risk to define its notion of epistemic uncertainty. In regression with squared loss, this is the squared difference between the predictor and the true conditional mean. For a linear predictor fw(x)=x⊤wf_{\mathbf w}(\mathbf x)=\mathbf x^\top\mathbf w and true conditional mean f⋆(x)=x⊤w⋆f_\star(\mathbf x)=\mathbf x^\top\mathbf w_\star, we can write this as

ERsq(f,x;f⋆):=(f(x)−f⋆(x))2\mathrm{ER}_{\mathrm{sq}}(f,\mathbf x; f_\star) := \left(f(\mathbf x)-f_\star(\mathbf x)\right)^2

The epistemic quantity being defined therefore depends on the unknown true regression function, which gives the true label in the noiseless setting. This is the aspect of DEUP relevant to our discussion: its theoretical target is defined relative to ground truth.

There is also an exact connection to conventional EU as mutual information. Let q(W∣D)=N(μ,Σ)q(\mathbf W\mid\mathcal D)=\mathcal N(\boldsymbol\mu,\boldsymbol\Sigma) and consider the Gaussian predictive likelihood Y~∣W,x∼N(x⊤W,σ2)\widetilde Y\mid\mathbf W,\mathbf x\sim\mathcal N(\mathbf x^\top\mathbf W,\sigma^2), with σ2>0\sigma^2>0. Then EUMI(x)=Iq(Y~;W∣x,D){EU}_{\mathrm{MI}}(\mathbf x)=I_q(\widetilde Y;\mathbf W\mid\mathbf x,\mathcal D) satisfies

EUMI(x)=12log⁡(1+EW∼q[ERsq(W,x;w⋆)]−ERsq(μ,x;w⋆)σ2)\mathrm{EU}_{\mathrm{MI}}(\mathbf x) = \frac12\log\left(1+\frac{\mathbb E_{\mathbf W\sim q}\left[\mathrm{ER}_{\mathrm{sq}}(\mathbf W,\mathbf x;\mathbf w_\star)\right]-\mathrm{ER}_{\mathrm{sq}}(\boldsymbol\mu,\mathbf x;\mathbf w_\star)}{\sigma^2}\right)

Here w⋆\mathbf w_\star is fixed, and the expectation is over the predictor distribution qq.

The quantities are different, but both use a performance criterion defined relative to ground truth. Our use of “epistemic uncertainty” follows this broader interpretation, which we don't think is misleading. Nonetheless, we appreciate the concern and we will highlight and clarify this in the revision.

2. What the rise–peak–decline trajectory tells us

Your example also helps clarify the second point. We agree that a rise–peak–decline curve can occur without grokking, so the peak alone cannot establish that grokking has occurred. We indeed never claimed that it does. However, we think the important point is to compare training and generalization EU, rather than considering either trajectory in isolation. Our theoretical contribution concerns how the training and generalization timescales relate to the timing of the uncertainty collapse within the stated λ<1\lambda<1 regime (e.g. when grokking does occur).

To illustrate this better, we added an experiment (new Appendix D.6, Figure 15) with standard Bayesian logistic regression trained using variational inference, measuring the usual label-free predictive EU. Interestingly, for λ=10\lambda=10, training accuracy approaches 100% and training EU exhibits a peak followed by a decline, while generalization EU stays high without a corresponding decline. In contrast, for λ=0.1\lambda=0.1 and λ=0.3\lambda=0.3, generalization EU peaks and declines as the models learn to generalize. This is missing from your example, but actually crucial to the interpretation of our theoretical results.

We hope this clarifies the connection and the interpretation of our theoretical results. We remain available for further discussion and will be happy to incorporate any additional suggestions into the revision.

AuthorsOpenReview ↗

Response pt 2

References

Depeweg, S., Hernández-Lobato, J. M., Doshi-Velez, F., and Udluft, S. (2018). Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning. Proceedings of the 35th International Conference on Machine Learning, PMLR 80, 1184–1193.

Kotelevskii, N., Kondratyev, V., Takáč, M., Moulines, E., and Panov, M. (2025). From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation. International Conference on Learning Representations.

Lahlou, S., Jain, M., Nekoei, H., Butoi, V. I., Bertin, P., Rector-Brooks, J., Korablyov, M., and Bengio, Y. (2023). DEUP: Direct Epistemic Uncertainty Prediction. Transactions on Machine Learning Research.

Lahoti, P., Gummadi, K., and Weikum, G. (2023). Responsible model deployment via model-agnostic uncertainty learning. Machine Learning, 112, 939–970.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.