acceptodds
Under review as a conference paper at ICLR 2027

Amplify, Locate, Estimate (ALE) on Rare-Failure Rates of Fine-Tuned LLMs

Abstract

Safety fine-tuning can make undesirable model behaviors sufficiently rare that measuring their frequency by direct sampling becomes prohibitively expensive. We introduce Amplify, Locate, Estimate (ALE), a method for estimating such rare failure probabilities efficiently. ALE uses the change in token log-probabilities induced by the safety fine-tuning to define an amplification direction, along which we increase the failure rate for cheap measurement. We develop a score-function estimator that uses the selected amplified batch to recover the local slope of the log failure rate from the token probabilities of the sampled failures, enabling extrapolation back to rare target rates. We also use the amplified batch to obtain an importance-sampling estimate. Across 41 evaluation targets, ALE's estimate-to-reference ratios have a median close to one, with 37 ALE estimates within a factor of three of reference values obtained from independent direct measurements. Ablation tests show that with a budget of up to 1000 labeled rollouts, in addition to a cheap unlabeled scan, ALE estimates failure probabilities for Qwen3-8B fine-tunes down to about , where direct measurement can require hundreds of thousands of rollouts, as the required sampling budget scales inversely with failure probability at fixed relative precision.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
rare-event estimation, rare failures, safety fine-tuning, LLM safety, importance sampling, score-function estimation, failure probability estimation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.20 over 5 reviews, above 2% of the others.
Reviewer_5n2VOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents ALE, an approach to per-prompt rare-failure estimation that extrapolates a high-failure model past the worse-failing of two base models, then extrapolates a failure rate linearly back to the base models. Experiments show competitive performance by ALE on a CWEval-based rare-failure estimation task relative to prior state of the art.

Critical strengths

The fundamental idea of sampling failures by extrapolating the policy difference between a more-failing policy and a less-failing one, and then extrapolating the failure rate back under an assumption of linearity, is an interesting one. The technique used to estimate the slope of the line using a single batch estimate of the failure rate is also clever. I believe this could be developed into a strong paper with clearer writing and more extensive experiments.

Critical weaknesses

The paper is extremely difficult to read. Particularly in the 4.5 pages used to explain the basic idea of the method, terms are introduced without explanation (for example, it takes several minutes to determine what the “unlabeled scan” in Figure 2 is referring to), in great complexity (see Table 2), and often with no intuitive motivation (for example, the notion of rollout generation being cheaper than failure labeling is fine, but this is not a standard assumption in the field, and it is not intuitively justified in this paper as far as I can find). While reading, the information contained in the next sentence nearly never answered the question I had in my head at that point. To be blunt, the paper reads like it was written by an LLM for LLMs, not by a human for humans, and as a human it is extremely hard to extract information from. There may be other weaknesses relating to the content of the paper, but it is very hard to tell due to the presentation style.

The results are presented strangely and are not particularly convincing. The results shown are not inherently better than the one baseline that is compared to, and furthermore, they do not match sample/label counts between the two methods, so direct comparisons are impossible. It is also not obvious why the setting of this paper is a fair comparison with prior work, which does not assume access to a model that is known to fail more often (if such access were known, the sampling methods in prior work would likely be adjusted). Thus, this should be a highly favorable comparison for the present method, and it still fails to outperform prior art convincingly. Furthermore, Table 2 is just a data dump (why do we care that targets are labeled cwe_329_0 in the code? What does that mean? If the target labels don’t matter, then shouldn’t this be presented as a figure?), with no interpretive assistance provided. Even if all of these issues were fixed, the results would still be limited, e.g., to this single task and single model (Qwen3-8B). Also, the basic baseline of Monte Carlo does not seem to be presented, but should be, as it looks like some failures are common enough that it could be competitive.

Questions for authors

In what regard is the comparison with Angell et al. (2026) fair, given the additional assumption of access to a second model that is known to fail more often?

1 comment
AuthorsOpenReview ↗

Thank you for the review and the kind words on the slope estimator.

On your question, the adjustment you anticipate is our CE-IS baseline. In a controlled comparison, it applies Angell et al.'s proposal selection to ALE's checkpoint path, so both methods have identical model access (Sec. 5). Pernice & Aranguri (LessWrong 2026) likewise assume a less-safe second model (Sec. 1). Neither of ALE's checkpoints needs frequent failures, since amplification supplies them (1.8 x 10^-4 and 2.8 x 10^-5 in Fig. 3). Table 1 reports labelled and total rollouts for every estimator. ALE uses fewer of both than CE-IS.

Every estimate is compared against a direct Monte Carlo count (Fig. 4 and column P_MC of Table 2). At ALE's label budget, direct Monte Carlo finds no failure in the running example (Fig. 3) or for 30 of 49 targets in the broader paired evaluation (Sec. 5, Ablations, and Fig. 13). Table 2 is ordered by coverage. In its three lowest-coverage rows, from two tasks, ALE-SE is closest to the direct count at every target, while IS misses by up to 77x. At high coverage the order reverses (Sec. 5). Of the 41 targets, 29 are short expressions outside CWEval and 12 come from five CWEval tasks, all on Qwen3-8B (Table 1, Sec. 6). The next version will define the unlabeled scan where Fig. 2 first uses it. Thank you for your time and suggestions.

Reviewer_J5zfOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Amplify, Locate, Estimate (ALE), with the goal of estimating the failure probability of a generative model after it has been fine-tuned. To achieve the goal of estimation, ALE takes a look at the difference between the log-probability between two model checkpoints. Using this difference, it constructs an amplified distribution and tries to estimate the failure probability. The method is evaluated via experiments using Qwen3-8B.

Critical strengths

This paper studies an important and interesting problem -- of estimating failure rates of safety fine-tuned language models. Estimating rare events classic problem in statistics that has useful applications the era of generative models. The specific goal in this paper is to estimate the failure rate in the scenario where there is access to a model that has be fine-tuned for safety and also have access to the base-model. The paper proposes utilizing the two models to interpolate and extrapolate next-token probability distributions via linear approximation for the of the log-probability as a function of "amplification" alpha. Then locating an amplification rate by heuristics that could yield higher failure trajectories. Samples are drawn from this amplification and this is used to then estimate the failure rate of the fine-tuned model via the linear approximation.

Critical weaknesses

The method is presented as a general failure rate estimation, however the procedure is specifically aimed at estimating failure rates for a given set of prompts. Where the "distribution of prompts" come from is never addressed at all. Not even mentioned in the limitations.

The writing is not done keeping in mind a human reader. The sentence constructions are incredibly awkward in several places. The goal of the estimation problem is not even clearly stated mathematically. There are notations and abbreviations used without really defining them. Some of the definitions are not mathematically meaningful. E.g., D(\alpha) defined in Eq 3 is function of the prompt x. In line 288, D(\alpha) is defined as a quantity that accounts also for the distribution of prompts. Lines 151-154: "..., six adapters ...., align against D over roughly four decades...." adapters and decades are used without proper context.

The method is heavily based on heuristics (which in itself is fine when clearly explained and grounded) and not clearly anchored in mathematical foundations. Key aspected needed to explain the method is pushed to appendix making the exposition of the method not clear in the main paper. The algorithm is not clear either. The heuristics is presented without clearly defining quantities mathematically. E.g., select \alpha^* as the right end of the rightmost qualifying lat run in derivative of \tidle{P} (in both in the algorithm and the main paper section 3).

The problem of estimating quantities like KL divergence, distribution over next tokens etc from samples is not addressed adequately. The distributions over sequences of next tokens are discrete distributions with very very large support. Estimating probabilities and properties of the distributions accurately is non-trivial. The error in estimating these quantities and how they propagate to the final estimate of the failure rate is not really addressed properly.

Literature review severely falls short of relevant rich literature in statistics. Estimating rare events classic problem in statistics. In particular, estimating the distribution and (symmetric) properties of the distribution for discrete distributions where the mass on a lot of support points can be very small to do naive sampling is very well studied, going back to the 1950s. (see few below). [1] Good, Irving J. "The population frequencies of species and the estimation of population parameters." Biometrika 40, no. 3-4 (1953): 237-264. [2] Valiant, Gregory, and Paul Valiant. "Estimating the unseen: an n/log (n)-sample estimator for entropy and support size, shown optimal via new CLTs." In Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 685-694. 2011. [3] Acharya, Jayadev, Hirakendu Das, Alon Orlitsky, and Ananda Theertha Suresh. "A unified maximum likelihood approach for estimating symmetric properties of discrete distributions." In International Conference on Machine Learning, pp. 11-21. PMLR, 2017. [4] Hao, Yi, and Ping Li. "Optimal prediction of the number of unseen species with multiplicity." Advances in Neural Information Processing Systems 33 (2020): 8553-8564. [5] Charikar, Moses, Kirankumar Shiragur, and Aaron Sidford. "A general framework for symmetric property estimation." Advances in neural information processing systems 32 (2019).

While the problem being studied here is in the context of large language models (LLMs), given a prompt x, they are essentially discrete distributions over a large support (of possible sequences of tokens). So, it is important to ground one's work in the literature.

While the problem is of interest and the idea is interesting, the paper needs significant revisions. The framing needs more work and needs to be grounded in the foundations in statistics and probability. The experiments need to be thorough to include clear error estimates that are grounded in probability theory and statistics. The writing needs substantial improvement.

Questions for authors

See critical weakness.

1 comment
AuthorsOpenReview ↗

Thank you for the review. We want to clarify four points

  • The prompt distribution is part of the formulation (Sec. 4.3, Eq. 14) and an input to Alg. 1 supplied by the auditor. Eq. 3 is its point-mass case (note to Alg. 1). Sec. 6 lists fixed prompts among our limitations.
  • Token probabilities and token KLs are exact, from both models. D(alpha) is estimated by averaging them over sampled completions (Sec. 2) and only selects the anchor alpha* (Sec. 3). The final estimates are built from sample means of exact scores and likelihood ratios (Eqs. 14-16). App. B derives the conditional variance of the slope and separates sampling error from curvature (Eq. 20). Table 2 gives paired-bootstrap sampling intervals, and Sec. 5 (Ablations) the sensitivity to the anchor.
  • The cited works estimate unknown distributions or their symmetric properties from samples alone. P(H) is the probability of a specified failure event and ALE has the exact likelihoods. The relevant tradition is rare-event simulation, likewise from the 1950s (Kahn & Harris, 1951), and Eq. 1 is a token-wise exponential tilt (Siegmund, 1976). The next version will cite both. Sec. 4 already builds on score-function identities (Glynn 1990).
  • Alg. 1 and its note define the anchor as the right end of the rightmost run of three grid intervals whose proxy log-slopes have median >= 0.1 and spread <= 10% of it, with proxy rates in [10^-3, 0.5]. Alg. 1 also lists the relaxed setting and the fallback used when no such run exists. The next version will move both into Sec. 3
Reviewer_rrzYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ALE, where they estimate rare safety failure rates of a base LLM and its fine-tuned model with far fewer labels than direct sampling. They motivate this by arguing that safety fine-tuning can make failure modes so rare that evaluating these models via direct sampling requires a number of labeled rollouts inversely proportional to these very low failure rates, which becomes too expensive. ALE has three steps where they (1) turn a pair of base/fine-tuned models into a continuous family of models indexed by amplification α\alpha and increase α\alpha beyond the base model to make the failures visible, (2) locate, without labels, a single point α∗\alpha^\ast where the log failure rate is still roughly linear in α\alpha before saturation, and (3) label only one batch at α∗\alpha^\ast to get the failure rate and its log slope there to extrapolate this line back to the base and fine-tuned models to infer their failure rates. Empirically, they take Qwen3-8B (and LoRA fine-tunes on fixed prompts) and evaluate their two variants, ALE-SE and ALE-IS, on short security expressions and CWEval tasks against the directly sampled references for accuracy, where the method seems to gain large labeling efficiency.

Critical strengths
  1. The paper is clearly structured and carefully written, with scoped claims. The paper is generally well-written and easy to follow despite its technical content being dense. Its structre around the three steps is neat, and the three main figures (Fig. 1-3) helped me understand the content nicely. The technical claims seem mathematically sound with their assumptions explicitly stated (e.g., the two nuisance terms dropped from Eq.4). The authors are explcit about where the method relies on approximations or empirical heuristics (e.g., that log-linearity in α\alpha is empirical). For their experiments, they report seemingly fair results where their method is not the best.

  2. The idea is conceptually elegant and appears useful. The authors build on LDA and reuse the fine-tune's own logit change to push beyond the base model, which I find as a clean way to make suppressed failures more measurable, without having to construct a steering direction from examples etc. Another key observation is that the scores of failing completions in a single amplified batch at α∗\alpha^\ast give the slope of the log failure rate, from which we can extrapolate to infer the failure rates of the base and fine-tuned models. This substantially cuts labeling cost, as only that one batch needs labels, and these scores (like the KL estimates used to locate α∗\alpha^\ast) can come from forward passes already at hand from amplified sampling.

Critical weaknesses
  1. Experiments mostly validate the method in near-ideal settings. The empirical evidence seems strongest where the method is easiest and weakest where it matters, and the motivation seems to be overstated.

    • a) Most targets (29 of 41) come from short expressions whose failures "take a single form" (line 108), which I can also infer from Tab. 3 (e.g., json vs. pickle). By Eq.2, the log-odds at such a single choice (at a fixed prefix) is linear in α\alpha, so the log failure rate should be roughly linear while it is small, which is the favorable case for the method. Even in this setting, CE-IS (though requiring more labels) gives better accuracy.
    • b) The realistic case under CWEval only has 12 of 41 from 5 tasks and about half of them have failure rates above 1e-3 (Tab. 2), where direct counting is feasible. Also, in multiple places in the paper (e.g., Sec. 1), the authors claim that SE avoids reweighting's large misses, but this seems to rest on the three lowest-coverage targets in Tab. 2 from two tasks.
    • c) When q:=P(0)q := P(0) for MM is directly countable, as for all CWEval MM checkpoints (and typically when MM is a pre-trained model before safety training), we could count qq directly (at matched label counts) and extrapolate one unit to NN using slope from its own failures without amplification or the KL estimates. This would be a fair baseline to include, and would show whether ALE's machinery adds value in this regime.
    • d) The experiments are restricted to one base model (Qwen3-8B), one fine-tuning method (LoRA), fixed prompts, and relatively short outputs, which falls outside the agentic workflow described to motivate the method.
  2. We cannot seem to tell when an ALE estimate is trustworthy. As far as I understand, ALE is designed exactly when you can't afford direct counts, so in real use you cannot check its accuracy against direct counts. So, whether the ALE estimates can be sufficiently trusted depends entirely on whether its assumptions held, primarily that log P\text{log }P stays roughly linear down to the target from α∗\alpha^\ast (assuming α∗\alpha^\ast is well chosen).

    • a) This assumption does seem to fail often enough to matter, as ALE-SE's sampling-only intervals miss the reference for about half the times in Tab. 2 vs. three for CE-IS. A natural explanation is the curvature dropped in Eq.9 (as mentioned in Limitations already).
    • b) Beyond the simple abstain rules in Alg. 1, there is no reliable machinery to flag an estimation failure. App. B provides a way to estimate the curvature c∗c_\ast but it's unused plus cannot bound the errors along the whole path, which SE-IS gap in Eq.21 (as cross-check) also rests on in addition to coverage. Also, we cannot know when α∗\alpha^\ast is mislocated, even though App. E seems to suggest its effect is not negligible.
Questions for authors

Q1. Can the authors clarify or correct me if needed on Weakness 1 (a-c)? For (1c), if it appears a useful baseline to have, can the authors include it. If otherwise, can you explain why not? For (1d), I understand larger settings are infeasible for the rebuttal and lack references. Do the authors think running ALE there without references would still be valuable, to check that ALE-SE and ALE-IS give sensible estimates in the realistic settings where safety matters?

Q2. Can the authors clarify or correct me if needed on Weakness 2 (a-b)? In particular, is there a reliable way to tell in practice if ALE estimates should not be trusted? For a safety-motivated method, I think this is important.

Please clarify on these points. I will change my assessment accordingly.

1 comment
AuthorsOpenReview ↗

Thank you for the careful and constructive review and for engaging so closely with our method

Your points on W1(a), (b) and (d) are fair, and the next version will scope the claims accordingly. On (d), yes. Runs in longer settings without references would still show feasibility and SE-IS agreement, though not accuracy. Your two questions go to the heart of it, thanks!

The alpha = 0 baseline (W1c). Your baseline is a clever way to isolate what amplification adds. It is exactly ALE-SE with its anchor at alpha = 0, and in the next version we will include it wherever M is countable. On CWEval, 1024 rollouts at M would yield about 4-30 failures in expectation, against 87-309 at the anchors of the seven evaluated pairs (Table 5). Where M itself is rare (see Fig. 3), the baseline would typically have too few failures and abstain.

When to trust ALE (W2). Honestly, not yet reliably (Sec. 6). But your question made us look closer at the built-in cross-check, since SE and IS from the same batch rest on different assumptions (Eq. 21). Looking back at Table 2, on the six targets where the two agree within 2x, both lie within 2x of the direct count. On five of the six where they disagree by more, at least one misses by more than 2x. So disagreement looks like a useful warning sign, even if it doesn't say which estimate is off.

Thank you again. This review will make the paper better!

Reviewer_7GqZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes ALE, which amplifies directions induced by the safety fine-tuning and extrapolates the linear function of harmful probability w.r.t. amplifiers towards rare-failure regime. This allows the estimation of rare probability of Qwen3-8B on CWEval dataset.

Critical strengths
  • This paper tackles timely and important research question on estimating the rare failure probability of LLMs.
Critical weaknesses
  • Main concern: Overall evaluation suite is limited to certain benchmark and model. While ALE heavily relies on empirical observation that their base-to-finetuned direction gives the linear function between amplifiers and p(insecure), it only validates it with CWEval benchmark and single model. Is there any specific reason why not tested on broader range of safety benchmark, such as HarmBench, where the baseline studies employed?
  • ALE requires checkpoints for both base and fine-tuned counterparts, while the baselines need a target model only. This might be problematic in practice where only a “safer”, fine-tuned model checkpoint is released while base model is not.
  • ALE is only applicable to open-weight models, and per-prompt only. Discussions regarding potential extension to estimating failure probability over the tasks would be helpful (Kim et al., 2026)

Kim et al., Measuring Nines of Reliability: Sample-Efficient LLM Evaluation on Saturated GSM Benchmark, 2026

Questions for authors

See weaknesses.

1 comment
AuthorsOpenReview ↗

Thank you for the review. We clarify the scope and access requirements (Sec. 1 & 6).

W1. Of the 41 targets, 29 are short expressions outside CWEval and 12 come from five CWEval tasks. All are checked against independent direct counts (Sec. 5, Table 1, Fig. 4). Programmatic judges let us count specified code failures consistently. Other model families (Sec. 6) and HarmBench remain untested. Please note that the linear relation is in log P, not P (Sec. 4.1).

W2. ALE requires the full next-token distributions of two checkpoints, access that developer-run audits provide, as do open-weight audits when both checkpoints are released (Sec. 1). It does not need the pre-fine-tuning base. For CWEval, M and N are an earlier and a later checkpoint of the same secure fine-tune (Sec. 5). Angell et al. construct a steered proposal from one model, and Pernice & Aranguri (2026, LessWrong post) use two variants. Neither works from samples alone. Our CE-IS comparison uses the same checkpoint path as ALE (Sec. 5).

W3. ALE is formulated for a prompt distribution and its average failure rate (Sec. 4.3, Alg. 1). Fixed prompts are a limitation of our experiments (Sec. 6). Kim et al. sample failure-prone inputs, complementing our estimation of rare outputs.

Reviewer_uubqOpenReview ↗
Rating
1
Confidence
3
Informativeness
3
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Amplify, Locate, Estimate (ALE), a method for estimating rare failure probabilities in fine-tuned language models. Estimating failure rates through direct sampling can require a prohibitively large number of samples, particularly when failures are extremely rare. The proposed method assumes access to the pre-fine-tuning base model and exploits the changes in next-token prediction distributions induced by fine-tuning. ALE first amplifies suppressed failures by extrapolating the fine-tuning-induced changes in next-token distributions beyond the base model, identifies a suitable amplification level, and estimates the failure probability at that level. It then estimates the local slope of the log failure rate with respect to the amplification level and uses it to extrapolate the failure rate back to the fine-tuned model. This enables estimation of failure rates for which direct sampling would be prohibitively expensive. The paper also develops an importance-sampling estimator using the amplified batch and compares it with a cross-entropy-based proposal-selection approach. Experiments on 41 evaluation targets using Qwen3-8B fine-tunes show that 37 ALE-SE estimates are within a factor of three of independent direct-sampling references, with estimates reaching failure probabilities of approximately (10^{-5}) using around 1,000 labeled rollouts per anchor.

Critical strengths

The paper addresses an important and practically relevant problem: estimating the probability of rare failures in fine-tuned language models, for which direct sampling can be prohibitively expensive. This is particularly consequential for evaluating the effectiveness of safety fine-tuning, since observing few or no failures does not necessarily establish that the failure probability is sufficiently low.

A particularly promising aspect of the work is its use of a label-free metric, namely KL divergence, to identify suitable amplification levels for failure-rate estimation. Using distributional information to guide the estimation process, rather than relying entirely on labeled failure samples, is an appealing direction that could substantially reduce the cost of evaluating rare behaviors. More broadly, the combination of label-free localization and score-based extrapolation offers a potentially useful framework for making rare-failure estimation more sample-efficient.

Critical weaknesses

1. Insufficient validation of the extrapolation assumption. The proposed ALE-SE estimator relies on the assumption that the log failure probability, log⁡P(α)\log P(\alpha), is approximately linear in the amplification parameter α\alpha over the extrapolation interval from the selected anchor α∗>0\alpha^*>0 to the target α=−1\alpha=-1. However, the paper primarily demonstrates the approximate linearity of the failure-rate curve in the positive amplification region and does not provide sufficient evidence that this behavior extends to the interval [−1,0][-1,0]. This is particularly concerning because the target fine-tuned model lies at α=−1\alpha=-1, and the accuracy of the final estimate critically depends on the slope remaining representative across the entire extrapolation interval. Moreover, Pernice and Aranguri (2026) report nonlinear behavior in the log failure-rate curve over the interval (−1,0)(-1,0), raising further questions about the validity of this assumption. The authors should investigate the behavior of the curve over the full extrapolation interval and quantify the sensitivity of ALE-SE to deviations from linearity.

2. Lack of comparison with relevant external methods. Although the paper compares its proposed estimators (ALE-SE and ALE-IS) with CE-IS and independent direct-sampling references, it does not experimentally compare ALE with other recently proposed approaches for estimating rare LLM failure probabilities. In particular, Pernice and Aranguri (2026) appears to be a closely related work and should be considered as a baseline. Such a comparison would help establish the relative advantages of ALE in terms of estimation accuracy, sample efficiency, and robustness to nonlinear behavior along the amplification path.

3. Limited model diversity in the evaluation. The main experimental evaluation is conducted exclusively on fine-tuned versions of Qwen3-8B. Although the paper evaluates 41 targets, the use of a single base-model architecture limits the evidence for the generalizability of the proposed method. Since ALE relies on the distributional differences between base and fine-tuned checkpoints, it is important to investigate whether its assumptions and estimation accuracy hold across different model architectures and sizes. Evaluation on additional model families would substantially strengthen the empirical validation.

Questions for authors
  1. Figure 5b shows an interesting relationship: P(α)P(\alpha) appears to be almost linear in D(α)D(\alpha) over the range shown. Could this relationship be used directly to estimate the failure probability of the fine-tuned model? Specifically, we could estimate D(α∗)D(\alpha^*) for some sufficiently large amplification factor α∗\alpha^*, as well as D(−1)D(-1), measure P(α∗)P(\alpha^*), and then extrapolate P(−1)P(-1) using the approximately linear relationship between PP and DD. Is there a particular reason why this approach would not work?

  2. I did not fully understand the zero-count upper limit. Could you please elaborate on how it is calculated and how it should be interpreted?

  3. ALE estimates (q_{\mathrm{eff}}) rather than the true base failure rate (q). What is the effect of this approximation on the final failure-rate estimate? Could the authors quantify the discrepancy between (q_{\mathrm{eff}}) and (q), and evaluate how the resulting error propagates to the estimated failure rate? It would also be useful to characterize the conditions or scenarios under which this approximation is expected to introduce relatively small versus large errors.

1 comment
AuthorsOpenReview ↗

Thank you for the review. We give the relevant locations in the paper below and answer the questions.

W1 and W2. ALE builds its path from two checkpoints with low failure rates and extrapolates from an amplified anchor back to them. The interpolation of Pernice & Aranguri (2026) corresponds to alpha in [-1, 0] in our parameterization. In their illustrated example, the less-safe endpoint is near saturation, a regime they exclude empirically with a fixed 50% threshold. With both our checkpoints rare, the saturation bend lies at alpha > 0 in the examples shown (Figs. 1, 5a, 17), beyond the anchor we locate without labels (Sec. 3, Fig. 2), and so outside the interval we extrapolate over. Our single-batch curvature estimate is local to the anchor (App. B, Sec. 6), so we test the targets directly. All 41 reported targets, including fine-tunes at alpha = -1, are checked against independent direct counts, and 37 lie within a factor of three (Fig. 4). For the five CWEval pairs with both endpoint estimates reported from the same anchor in Table 2, the predicted reduction P(0)/P(-1) across [-1, 0] lies within 2x of the measured reduction in four cases. For cwe_329_0, ALE-SE overstates it by 3.8x. A four-cell logit-path fit based on their method is compared at equal labels on our located window. Its median error is 2.11, versus 1.35 for ALE-SE, on 32 shared targets (Sec. 5, App. E, Fig. 10)

W3. Agreed, as stated in our limitations (Sec. 6).

Q1. Not as stated... D is zero at alpha = 0 and positive on both sides. Fig. 5b shows the positive branch, where the failure rate lies above the base rate. For a suppressed failure, the fine-tune lies on the other branch, below the base rate. Where the branches overlap in D, reading its rate off the positive branch would therefore overestimate it. This is why we locate in D and extrapolate in alpha (Sec. 3).

Q2. With zero failures in n independent rollouts, the one-sided 95% upper limit is 1 - 0.05^(1/n), approximately 3/n. For n = 1024 this is 2.9 x 10^-3. Rates above it are excluded at 95% confidence. Rates below it are not resolved.

Q3. q_eff is a calibration scale for the proxy. It should be understood as an estimate of q (App. A, lines 613-614). Also, it does not enter Eqs. 15-16, so it affects the estimate only through the anchor alpha*. Under saturation, Eq. 6 gives q_eff = q exp(-K_sat^H) <= q. App. A (Eq. 17) and Sec. 6 note when within-class terms distort the proxy. We do not measure q_eff/q separately. Fig. 14 quantifies the sensitivity of the estimates to anchor shifts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.