acceptodds
Under review as a conference paper at ICLR 2027

Jailbreaking Open-Weight LLMs via Random Embedding Perturbations

Abstract

While open-weight models have enjoyed steady progress in capabilities and wide adoption across multiple domains, their safety remains an important concern. One key feature is the ability to refuse or deflect harmful, malicious, or insensitive prompts. In this paper, we expose safety vulnerabilities across six common openweight LLMs of various sizes that consistently lead to harmful or unsafe responses on the JailbreakBench benchmark dataset. Our proposed attack, Perturbed Embedding Vector (PEV), is a simple and fast “jailbreaking” technique that is cheaper than prior approaches, which typically require gradient computations, per-prompt optimizations, or altering internal weights of the models. PEV just adds independent Gaussian noise in the embedding vector representations of the prompt, with no need for further manipulations. To generate unsafe responses, we repeatedly sample additive noise from this distribution. In our experiments, we observe that the average compute cost to get the first successful attack is up to an order of magnitude less than previous attacks. The first successful jailbreak on a new prompt typically arrives within one minute on every tested model, and PEV generates unsafe responses across all models for all prompts in JailbreakBench. No other tested method achieves such results, despite them taking longer to run. More broadly, we believe that understanding the behavior of LLMs under perturbations in the embedding vectors is an important research direction: while perturbations constitute a major security risk, they can also serve as a valuable tool for exploring the dynamical behavior of such models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, jailbreaking, open-weight models, embedding perturbations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ciEzOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Perturbed Embedding Vector (PEV), a white-box jailbreak attack for open-weight language models. Given the token embedding matrix of a harmful prompt, PEV adds independently sampled Gaussian noise to every embedding coordinate before the embeddings enter the Transformer backbone. The noise is independently resampled across repeated generations. The method requires neither gradient computation nor permanent modification of the model weights.

The authors evaluate PEV on the 100 harmful behaviors in JailbreakBench and six instruction-tuned models, comparing it against I-GCG, Soft Prompt, Latent Fusion, NeuroStrike, and RefusalCones. The paper reports that PEV eventually produces at least one unsafe response for every tested behavior on every model and reaches the first successful jailbreak faster than the compared methods. The paper also studies how the perturbation scale affects the proportions of safe, unsafe, and degenerate outputs.

Critical strengths
  1. The method is simple and empirically effective. PEV only adds Gaussian noise once to the input token embeddings. It does not require gradient computation, discrete or continuous adversarial-suffix optimization, or permanent modification of the model weights. Compared with I-GCG, Soft Prompt, Latent Fusion, RefusalCones, and NeuroStrike, PEV is more lightweight and easier to implement. Under the authors’ evaluation protocol, PEV also achieves high prompt-level attack success rates, including on relatively safe models such as Llama-3.1-8B, Qwen2.5-1.5B, and Phi-4-mini.

  2. The empirical evaluation has broad coverage and includes several useful analyses. The paper evaluates six open-weight instruction-tuned models: Qwen2.5-1.5B, SmolLM3-3B, Phi-4-mini, Mistral-7B, Llama-3.1-8B, and GLM-4-9B, covering model sizes from 1.5B to 9B parameters. It uses all 100 harmful behaviors from JailbreakBench, spanning ten risk categories, and compares PEV with five recent attack methods. Although some evaluated models already have weak safety at baseline, the breadth of the evaluation suggests that PEV is not limited to a single model family or architecture.

  3. The work reveals an interesting safety phenomenon that merits further study. For some models, the norm of the injected noise is large relative to the norm of the original prompt embedding, yet the model can still understand the harmful request and generate relevant content. This suggests that ordinary semantic processing and safety refusal behavior may have different levels of robustness to embedding perturbations. Although the paper does not provide a mechanistic explanation, this observation provides a useful starting point for studying refusal boundaries, representation-space robustness, and safety training.

Critical weaknesses
  1. The timing metrics and cost accounting in Figure 2 are insufficiently defined. Figure 2(a) describes its horizontal axis only as “the average GPU time to obtain the first successful jailbreak on a prompt.” However, the experimental setup states that the 20 generations for each prompt are executed as a single batch and that wall-clock time is recorded after each batch. It is therefore unclear whether the reported value represents the actual completion time of the first unsafe response within a batch or the completion time of the entire batch containing that response. If the first batch fails, the paper also does not clearly explain whether the time for subsequent batches is accumulated.Figure 2(b) reports cold-start wall-clock time measured from an unmodified model. The text explains that much of the cold-start cost of RefusalCones and NeuroStrike comes from preprocessing and that their per-prompt runtime becomes comparable to PEV after calibration. The figure should therefore separate one-time preprocessing costs from subsequent per-prompt attack costs. PEV also requires searching for a suitable σ\sigma using a sample of harmful prompts, but the paper does not state whether this search cost is included in Figure 2.

  2. It is unclear whether the success rates under repeated attempts constitute a fair comparison. A prompt is considered successfully jailbroken if at least one run produces an unsafe response. For PEV, every run independently resamples the noise matrix, so 20 runs correspond to 20 distinct embedding perturbation candidates. In contrast, the paper only states that the other methods perform “deterministic changes.” It does not clarify whether each run reruns the complete attack construction or optimization procedure, uses a different initialization to produce a new attack candidate, or merely repeats stochastic decoding with a fixed adversarial suffix or model intervention.Under the last interpretation, PEV receives 20 distinct attack candidates, whereas a baseline receives 20 generations from only one candidate. The resulting comparison would conflate attack-candidate diversity with the quality of an individual candidate. A deterministic attack modification does not necessarily imply that the corresponding method can produce only one candidate: optimization-based attacks may generate different suffixes through independent restarts, while other intervention methods may produce different candidates using different calibration samples, layers, or hyperparameters.The results do demonstrate that PEV can generate many attack candidates at low cost, which is a meaningful practical advantage. However, the current repeated-attempt metric may amplify its apparent relative effectiveness without separating the strength of an individual candidate from the ability to generate many candidates cheaply.

  3. Perturbing the complete prompt embedding makes the source of the attack effect unclear. The paper applies the chat template before adding noise to the complete prompt embedding. Consequently, the perturbation may affect not only the user-provided content but also role separators, special control tokens, and potentially the assistant-generation prefix. The current experiments cannot determine whether the jailbreak results from instability around the harmful semantic representation or from corruption of the chat-formatting and control signals.All experiments also use an empty system message, and the paper does not evaluate PEV while retaining a nonempty safety system prompt. A direct ablation should compare perturbing only user-content tokens, only special or control tokens, and the complete input under the same attack budget. Such an experiment would substantially clarify the nature of the reported vulnerability. Given the simplicity of the proposed method, the absence of this mechanistic analysis and these basic ablations weakens the paper’s overall contribution.

Questions for authors
  1. How was the model-specific perturbation scale σpeak\sigma_{\text{peak}} selected? The paper states that the authors searched over different σ\sigma values using “a small sample of unsafe prompts,” but it does not specify the number or identities of these prompts, the search grid, the selection criterion, or whether these prompts overlap with the final JailbreakBench evaluation set. Could the authors clarify the tuning protocol and whether its computational cost is included in Figure 2?

  2. Does Table 1 use the same run budget for PEV and the compared methods? Table 1 reports a 100% attack success rate for PEV on Qwen2.5-1.5B under a table caption referring to 20 runs per prompt. However, the later discussion states that some Qwen2.5-1.5B prompts required up to 160 runs to jailbreak. Is this an inconsistency, or does Table 1 report PEV results obtained using a larger, variable run budget while the compared methods are limited to 20 runs?

1 comment

We thank the reviewer for the detailed and constructive review. All the changes in the revised PDF are marked in blue.

Comment: The timing metrics and cost accounting in Figure 2 are insufficiently defined (first response or whole batch, accumulation over failed batches, preprocessing versus per-prompt cost, cost of the σ\sigma search).

Response: We thank the reviewer for stating this, the explicit definition was missing, and the caption of Figure 2 is now re-written clearly. All 20 runs for a prompt are generated as one batch, and the time is recorded when the batch finishes. Time to first jailbreak is the total time of all batches up to and including the one containing the first unsafe response, so failed batches are counted and the whole batch is charged. Figure 2(b) is the cold-start time on an unmodified model. The PEV time includes the search for σ\sigma.

Comment: It is unclear whether success under repeated attempts is a fair comparison: PEV gets 20 distinct candidates, while a baseline may get 20 generations from one candidate.

Response: This is a fair point, and in Section 4 we now state the protocol plainly. Each baseline builds one candidate per prompt and its 20 runs are 20 decodings of it, while every PEV run draws new noise. Two facts give context. PEV does not rely on having many candidates. On five of the six models the first success comes after fewer than two draws on average (Table 3). And candidates are not equally cheap, twenty I-GCG restarts would take about four GPU-hours per prompt, while twenty PEV draws take under a minute. We did not rerun the baselines with independent restarts, so we claim only what the reviewer calls the practical advantage, that is PEV produces many candidates at very low cost.

Comment: Perturbing the complete prompt embedding makes the source of the effect unclear (user content versus control tokens), and no nonempty safety system prompt is evaluated.

Response: We ran this test additionally at the same σ\sigma and the same 20-run budget (Appendix B):

Perturbed tokensLlama-3.1-8BQwen2.5-1.5B
User content only96%88%
Template and control only35%27%
Full input100%93%

Perturbing the control tokens can contribute, but it is not needed as much. Noise on the user content only, works almost as well. With a nonempty safety system instruction kept in place and left unperturbed, user-content noise still reaches 94% on Llama-3.1-8B and 86% on Qwen2.5-1.5B. The following instruction is one we wrote for this experiment: "You are a helpful assistant. Do not provide instructions that facilitate harm or wrongdoing. Refuse such requests and offer a safe alternative."

Comment: How was the model-specific σ\sigma selected (prompts, grid, criterion, overlap with the evaluation set), and is its cost included in Figure 2?

Response: The submission described this briefly in the paragraph "On the choice of σ\sigma", and we now make it precise. The procedure is a small grid search, and we report every step of it for the two new larger models we tested (Tables 7 and 8, Appendix B): 5 prompts with 5 runs each, a coarse grid from 0 to 0.03 in steps of 0.005, and two finer rounds around the peak. We keep the σ\sigma that breaks the most prompts. These 5 prompts are among the 20 used to evaluate those models. For the six original models, which were our first experiments, we similarly swept σ\sigma over a fixed grid (0.005 to 0.024, extended for GLM-4-9B and Phi-4-mini) on the first 5 prompts of JailbreakBench, which are part of the evaluation set, and took the peak (Figure 3b). The time reported for PEV in Figure 2 includes the search.

Comment: Does Table 1 use the same run budget for PEV and the compared methods? It reports 100% on Qwen2.5-1.5B, but some prompts needed up to 160 runs.

Response: This was an inconsistency on our side, and we thank the reviewer for pointing it out. On Qwen2.5-1.5B, PEV breaks 93% of prompts within 20 runs and needs more resampling, up to 160 runs for the remaining seven prompts. Table 1 now reports 93% at the common 20-run budget, on par with the best baselines on that model (94-96%), and the extended-budget result is reported separately. On the other five models PEV reaches 100% within 20 runs, so its mean at the matched budget is 98.8%, against 93.7% for the next best method. The abstract and introduction are corrected accordingly.

We would like to thank the reviewer for their constructive feedback, which has made the paper more precise. We are determined to clarify anything further.

Reviewer_hSV2OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a very simple method to efficiently jailbreak open-weight models: simply add gaussian noise to the input embeddings of your input prompt. This method requires no gradient computations or per-prompt optimizations, and is able to jailbreak every single one of the test prompts used (JailbreakBench). This approach is somewhat faster in raw wall-clock speed to the next fastest techniques (e.g. soft prompts), whilst removing the GPU overhead for a backward pass. In total, they compare their approach to five other jailbreak methods on six small open weight LLMs, and demonstrate that this simple technique outperforms all methods on all models across both speed and ASR.

Overall, I think the main findings (gaussian noise added to input embeddings is highly effective to jailbreak models) are interesting, even if it is a very simple contribution. I hold some reservation due to some technical missteps (a few missing baselines and the choice of models/dataset), but if these are addressed this paper could be an interesting contribution to the research community. Beyond my technical gripes, I think this paper has a lot more potential than it realized, and I'm a bit sad the paper did not explore the effects of random embedding perturbations more deeply. I would encourage the authors to pursue this research direction regardless of the outcome of this review process.

Critical strengths
  • Presentation is very good and the paper is well written.
  • The technique is very simple and highly effective across the test dataset and models used.
  • If the technique is valid (holds against the weaknesses I highlight below), then it is an interesting and useful insight for the community.
Critical weaknesses
  • Missing baselines: you are essentially doing best-of-N [1] with additional random perturbations added. Note that in the classic (vision) adversarial attack literature, it was common practice to both (a) do random initialization, and (b) do random restarts to maximize attack success rate (e.g. [2] among many more, you should be citing these too), so it is unsurprising that these ideas work here too. Since sampling is known to be an effective jailbreak, you must control for this and compare to this as a baseline.
  • Models are small and outdated. Older (and small) models are known to be less robust in general, especially against sampling attacks. You should evaluate on larger and more capable (reasoning) models, especially given how simple and efficient your technique is.
  • (less major) JailbreakBench is often used, but has known limitations (questions are quite contrived, single turn conversations). It would be far more interesting to see more diverse test data (e.g. multi turned datasets, agentic tasks, etc).

[1] Hughes, John, et al. "Best-of-N Jailbreaking." NeurIPS 2025

[2] Madry, Aleksander, et al. "Towards deep learning models resistant to adversarial attacks." ICLR 2018

Questions for authors
  • Please address the main weaknesses above (best-of-N and a larger/more modern model). I will only consider increasing my score if you include these results. If compute is an issue and you cannot run everything, please include partial/incomplete results along both axes. The main questions to answer here are:

    1. Is the jailbreaking successful as a result of the sampling (best-of-N) or your random perturbations? What is the minimum N required without the perturbations to replicate your ASR?
    2. Does random embedding noise also work on larger (30B, 70B, etc.), more modern models (e.g. reasoning models, released this year)? Does ASR and coherence hold up?
  • Why do you think the random perturbations work so effectively here, over say direct gradient optimization (soft prompts)? Assuming that it is the noise and not best-of-N sampling that's doing the jailbreaking, it's interesting that there's a very narrow band in which the perturbation is able to break the safety training while still retaining coherence, and then as you increase the magnitude the generations become incoherent. You could say that the input becomes out of distribution enough to circumvent the safety training whilst not pushing out of the pre-training distribution, but then wouldn't you expect the other capabilities learned during post-training to fail (e.g. instruction following)?

2 comments

We thank the reviewer for the clear and constructive review, and for stating exactly what would change the assessment. Both requested sets of results are included and all the changes in the revised PDF are marked in blue.

Comment: Missing baselines: this is essentially best-of-N [1] with random perturbations added; random initialization and restarts [2] should be cited, and sampling must be controlled for.

Response: Thank you for raising this. We reported this already in Table 1, and now frame it correctly. The "Baseline" column is best-of-20 sampling without noise. The same prompts, decoding, 20 samples, and considered success if any sample is unsafe. We now explicitly state it in the revision. As requested, we ran additional experiments for N=300N=300 (first question below), and describe PEV as random restarts in embedding space, as suggested. Thank you, we now also cite [1] and [2] as requested.

Comment: Models are small and outdated; evaluate on larger and more capable (reasoning) models.

Response: We agree, and we added two larger reasoning models released this year. 1) GLM-5.3-Flash (August 2026) is a mixture-of-experts model with 320B parameters in total and 18B active per token. Its reasoning cannot be switched off, and we ran it at the lowest reasoning effort due to limited time and compute. 2) Qwen3.5-35B-A3B (February 2026) which has 35B parameters in total and 3B active per token, with thinking mode on. Each was evaluated on the first 20 prompts of JailbreakBench. These are partial results, as allowed and the full runs will be in the final version.

Comment: (Less major) JailbreakBench has known limitations; multi-turn and agentic data would be more interesting.

Response: This is a fair point. We kept JailbreakBench for the following reasons: 1) the attacks we compare against are commonly reported on it, so all methods run under one protocol. 2) its behaviors cover ten categories of misuse; and a single-turn prompt isolates the effect of the noise, 3) with no conversation history or tool output that could also explain the change. We agree that multi-turn and agentic settings are valuable, and the conclusion now states this limitation and we are determined to study this excellent direction as future work.

Comment: Is the jailbreaking a result of the sampling (best-of-N) or of the random perturbations? What is the minimum N without perturbations to replicate the ASR?

Response: We show it is the perturbations. Raising the noise-free budget to N=300N=300, fifteen times the PEV budget, still falls well short (Table 5, Appendix B):

ModelSampling, N=20N=20Sampling, N=300N=300PEV, N=20N=20
Llama-3.1-8B8%19%100%
Qwen2.5-1.5B13%28%93%
Phi-4-mini16%35%100%
GLM-4-9B56%72%100%

The minimum NN is therefore above 300 on each of these models. Hughes et al. run NN up to 10,000, which we could not replicate in the time available.

Comment: Does random embedding noise also work on larger, more modern reasoning models? Do ASR and coherence hold up?

Response: Yes, with slightly more draws. Both models are fully broken on the 20 prompts we tested, with thinking enabled (Table 6, Appendix B):

Modelσ\sigma20 runs40 runs60 runsMean time to first jailbreak
Qwen3.5-35B-A3B0.019513/2020/20-49 s
GLM-5.3-Flash0.01857/2014/2020/2057 s

For each model we found σ\sigma with a small grid search on 5 prompts with 5 runs each (Tables 7 and 8). Coherence holds up as on the small models. Refusal go away first, then a narrow band of unsafe and coherent output, and then degeneration. We report that the larger tested models need 40 to 60 runs instead of 20.

Comment: Why do random perturbations work so effectively, compared with gradient optimization (soft prompts)? Why does refusal fail before other post-training capabilities such as instruction following?

Response: As σ\sigma grows, refusal drops first, the unsafe rate peaks, and only then degenerate outputs are observed (Figures 3b and 4). This can be seen on the new larger model experiments as well. Our hypothesis is that refusal is a low-dimensional feature with a thin margin, consistent with the refusal-direction and refusal-cone results, while instruction following and fluency are distributed and redundant, so isotropic noise crosses the thin margin first. Studying this deeply, is a great direction for our future work as well. On soft prompts, a soft prompt optimizes one suffix toward one target string and can stall, whereas each PEV draw is an independent attempt that costs one forward pass.

Thank you again for the opportunity to provide these results. We hope they address your concerns, and we are happy and determined to clarify anything further.

Thanks for the rebuttal, I believe that addresses most of my concerns. I'll increase my score from 2 --> 3.

Reviewer_Htj9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a simple and efficient jailbreak approach called Perturbed Embedding Vector (PEV), which only needs to add independent Gaussian noise to the token embeddings of prompts, without requiring expensive gradient computation, per-prompt optimization, or weight editing. The empirical evaluation shows the efficiency and effectiveness of the proposed method across diverse target models and jailbreaking tasks.

Critical strengths

The writing is clear and easy to follow.

The proposed method is more effective than the baseline methods.

The proposed method requires less computation compared to the baseline methods.

Critical weaknesses

Lines 157- 158 mention that there is no search or optimization step over embeddings. However, as described in Lines 213-215, the proposed method requires resampling the noise matrices Z multiple times and picking the trial with the best jailbreak rate. This procedure can be viewed as a form of zeroth-order optimization, which updates the optimization variable (noise matrices) by evaluating the objective function (jailbreak rate) without leveraging first-order information (gradient). Thus, the key claim that PEV does not involve search or optimization over embeddings may be misleading.

Lines 158-160 state that PEV is transferable across prompts. It would be great to provide empirical evaluations of the transferability of the perturbation across prompts and models. Specifically, if there is one perturbation that leads to a high jailbreak rate for one harmful behavior on one target model, can the perturbation be used to jailbreak different harmful behaviors and different target models?

What is the semantic meaning of the perturbations? It would be useful to investigate whether perturbations along certain semantic directions lead to a higher attack success rate.

It would be nice to show concrete examples of responses from successful attacks.

After the perturbation, does the attack prompt still possess the same harmful behavior? Empirical evaluation is needed to show that the perturbation will not change the original question intent. For example, if the original input prompt is about “how to make a bomb”, empirical evaluation is needed to show that after perturbation the target output is still related to the original input prompt.

Empirical comparisons with prompt-level jailbreaking methods are needed. Specifically, it remains unclear whether searching random perturbations in the token embedding can be more effective and more efficient than searching in prompt space.

Uncertainty measures (e.g., standard deviation) are missing in empirical evaluations.

Questions for authors

Please see Weaknesses for my questions.

1 comment

We greatly appreciate the clear comments and suggestions made by the reviewer, and have accordingly made changes to our submission in a revised version submitted, and marked the changes in blue.

Comment: The claim of "no search or optimization" may be misleading; resampling ZZ can be viewed as zeroth-order optimization.

Response: We note that, and have rewritten it for clarification. The precise claim is that PEV uses no gradients, no learned parameters and no per-prompt objective. It does use a resampling budget, and choosing σ\sigma per model is a calibration step. The distinction we keep is that the draws are non-adaptive. Every ZZ is an independent sample from one fixed distribution and no draw depends on an earlier outcome, so there is no iterate to update. The budget is also small as we demonstrated on five of the six models a prompt is broken in fewer than two draws on average (Table 3).

Comment: Transferability across prompts and models needs empirical support. Can one perturbation be reused?

Response: Thank you for pointing out this ambiguity. Our claim was about the procedure as a whole, and we have reframed it that way. One σ\sigma per model breaks all 100 behaviors, and the same procedure works on every model we tested after calibrating σ\sigma. A single fixed ZZ cannot be reused, because its shape must match the embedding matrix, which changes with the prompt length and the embedding dimension. We do not claim that a fixed draw is transferable and have reframed our writing to avoid this ambiguity.

Comment: What is the semantic meaning of the perturbations? Do certain semantic directions give higher attack success?

Response: The noise is essentially isotropic, so it has no preferred direction by construction, and the perturbed vectors do not map back to real tokens. We show the random perturbations work effectively to jailbreak and are determined to study the high dimensional space and directions in-detail as future work.

Comment: Show concrete examples of successful attacks.

Response: Absolutely, we added an Appendix C in the revised submission, which gives example responses, with a content warning, for Llama-3.1-8B, GLM-4-9B, Phi-4-mini and Qwen2.5-1.5B. Each has the judge's label as well. The full responses can also be found in the log files of the anonymous repository pasted in the paper.

Comment: After perturbation, does the output still address the original harmful request?

Response: That is a fair question and the answer is yes. This was already enforced by our labels, and we now state it explicitly. A response counts as unsafe only if it produces what the original prompt requested. Responses that drift off topic or are out of context, are labeled degenerate and do not count as successes. We also read the responses labeled unsafe manually, and the full logs are in the anonymous repository. The examples in Appendix C now also lets all readers see some examples as requested earlier.

Comment: Compare with prompt-level methods. Is random search in embedding space more effective and efficient than search in prompt space?

Response: Our submission compared a search in prompt space attack, I-GCG, which reaches 44-95% at roughly 730 seconds per prompt, against 93-100% in under one minute for PEV. The "Baseline" column of Table 1 is the simplest prompt-level control. It is done by sampling the unperturbed prompt 20 times, without augmentation. We now state this explicitly, cite the related Best-of-N jailbreaking attack (Hughes et al.), which additionally augments the prompt, and run an extended experiment for N=300N=300 in new experiments (Table 5, Appendix B). Sampling alone remains far below PEV Results can be seen as follows

ModelSampling only, N=300N=300PEV, N=20N=20
Llama-3.1-8B19%100%
Qwen2.5-1.5B28%93%
Phi-4-mini35%100%
GLM-4-9B72%100%

Comment: Uncertainty measures are missing.

Response: We thank the reviewer for this suggestion and now report the mean and sample standard deviation over three seeds (Appendix B). Attack success within 20 runs is 99.7±0.699.7 \pm 0.6\\% on Llama-3.1-8B, 93.0±1.093.0 \pm 1.0\\% on Qwen2.5-1.5B, 99.7±0.699.7 \pm 0.6\\% on Phi-4-mini, and 100.0±0.0100.0 \pm 0.0\\% on GLM-4-9B, SmolLM3-3B and Mistral-7B. The time for the full 20-run budget is 45±445 \pm 4 s per prompt on Llama-3.1-8B and 24±224 \pm 2 s on Qwen2.5-1.5B. The other numbers in Table 1 come from a single run. Table 3 and Figure 3a also report the spread of the number of runs needed per prompt.

We are grateful for the feedback, which has made the paper more precise. We hope these changes address your concerns, and we are happy to clarify anything further.

Comments on the paper

1 comment
Area_Chair_SWM3OpenReview ↗

Reminder: Please review the author rebuttal before the deadline

Dear reviewers,

The rebuttal period is about to close. If you haven’t responded yet, please read the authors’ rebuttal and leave a brief comment before the deadline.

If you’ve already responded, please disregard this reminder. Thanks for your time, and apologies for the extra email.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.