acceptodds
Under review as a conference paper at ICLR 2027

Leveraging Neural Network Layers as an Annealing Path

Abstract

Sampling and optimization with trained neural networks can be difficult even when their gradients are available: high-order interactions can obscure useful search directions. We introduce layerwise annealing, which uses the functions decoded from intermediate network layers as an annealing trajectory. The motivation is that earlier-layer functions can approximate smoothed versions of the final objective, with high-degree Fourier components attenuated. Their gradients can reveal promising directions that the final objective obscures, while progressively deeper layers restore the detailed structure needed to refine the solution. We exploit this by applying prediction heads at successive depths of a frozen network and carrying samples or optimization states through the resulting objectives, ending with the original function. Controlled Boolean and continuous experiments demonstrate the proposed mechanism: high-degree interactions and narrow density modes emerge progressively across layers. We demonstrate practical utility in LLM prompt optimization. On AdvBench Harmful Strings with Llama-2-7B-Chat, layerwise Greedy Coordinate Gradient achieves 70% target-string success, compared with the published GCG reference of 57%, with mean target cross-entropy of 0.195 versus 0.30.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
discrete optimization, linear representations, annealing trajectory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_PwYDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Layerwise Annealing, which progressively uses intermediate-layer gradients to guide optimization, based on the hypothesis that earlier layers provide smoother approximations of the final objective. The authors validate this intuition through toy experiments and apply the method to LLM jailbreak optimization.

Critical strengths

The paper appears to address a problem in jailbreak optimization, which may be relevant to practical LLM applications.

Critical weaknesses
  1. Decoding intermediate-layer predictions using Tuned Lens does not necessarily imply that their gradients benefit final-objective optimization. Moreover, the assumed correspondence between Fourier degree and network depth lacks general theoretical guarantees. Reduced high-degree Fourier mass alone does not establish that intermediate-layer functions are smoothed versions of the final objective, as coefficient consistency, low-degree information preservation, and gradient effectiveness remain unverified. These functions may simply be inaccurate predictors.

  2. The method uses existing Tuned Lens readouts to construct intermediate-layer surrogate objectives and follows a predefined depth schedule to guide GCG with different gradients. It introduces neither a new architecture nor substantial changes to the discrete optimization procedure. Its contribution appears largely limited to intermediate-layer gradient scheduling, whose novelty over existing approaches remains unclear.

Questions for authors
  1. The paper optimizes inputs given a fixed model, scoring function, and target output, but provides limited motivation for the practical value and representativeness of this setting. Despite claiming applicability to general optimization and sampling, it lacks cross-task validation. Could the authors clarify the practical significance and broader applicability of the proposed method?

  2. The examples mainly demonstrate limitations of single-coordinate, first-order discrete search rather than general gradient-based optimization, without comparisons to alternatives such as multi-token optimization. Moreover, the Fourier staircase experiment uses an explicitly constructed hierarchy, while the Gaussian mixture experiment relies on layerwise supervision. How do these specially designed settings reflect practical LLM optimization?

  3. The method primarily switches gradient sources across layers, making it unclear why this constitutes annealing rather than gradient scheduling. The authors neither establish a coarse-to-fine smoothing path across layerwise objectives nor demonstrate the benefits of sequential depth ordering. Improvements could instead arise from gradient diversity rather than annealing. Could the authors clarify the proposed mechanism and distinguish these explanations?

  4. Current experiments cannot distinguish whether improvements come from layerwise annealing, intermediate-layer gradients, or gradient diversity, nor do they establish the necessity of sequential layer scheduling. Could the authors provide ablations to isolate these factors?

  5. The main comparison uses the historical 57% ASR from Zou et al. (2023) [1], without reproducing GCG under identical conditions, leaving potential implementation and randomness differences unresolved. Comparisons with stronger methods such as I-GCG [2], MAGIC [3], and Lookahead-GCG [4] are also missing. Beyond toy experiments, the paper evaluates only one practical application using the outdated AdvBench Harmful Strings benchmark, which measures target-string generation rather than behavioral jailbreak success. Could the authors reproduce the baseline, compare against recent methods under controlled settings, and evaluate on newer benchmarks?

[1] Zou, Andy, et al. "Universal and transferable adversarial attacks on aligned language models." arXiv preprint arXiv:2307.15043 (2023).

[2] Jia, Xiaojun, et al. "Improved techniques for optimization-based jailbreaking on large language models." arXiv preprint arXiv:2405.21018 (2024).

[3] Li, Jiahui, et al. "Exploiting the index gradients for optimization-based jailbreaking on large language models." Proceedings of the 31st International Conference on Computational Linguistics. 2025.

[4] Feng, Rong, et al. "Lookahead-GCG: Improving Universal Multi-Model Optimization-Based Jailbreaking Attacks via Stochastic Nesterov Optimization." Forty-third International Conference on Machine Learning. 2026.

5 comments
AuthorsOpenReview ↗

Author Response (1/3)

We thank the reviewer for their detailed feedback and suggestions. We have substantially strengthened the paper with new mechanistic experiments, an explicit depth-order ablation, reproduced baselines, and evaluations on additional optimization tasks and models. We address the concerns below.

1. Mechanistic Evidence (Critical Weakness 1; Question 2; Sections 3.3–3.5 and 4.1)

Intuitively, neural networks progressively construct more complex functions by hierarchically composing simpler features across layers. Consequently, lower-degree components of the final function emerge at earlier layers, while higher-degree interactions develop at later layers. This suggests that earlier-layer functions can act as smoothed approximations of the final objective, preserving useful low-order structure.

Prior work has documented progressively aligned representations and increasingly accurate intermediate predictions across residual networks and transformers (Li & Papyan, 2023; Belrose et al., 2023; Jiang et al., 2025).

We have strengthened our evidence through two experiments (Section 4.1; Appendix A.1 for Fourier targets):

  1. Random Fourier functions: We replace the constructed Fourier staircase with 20 independently sampled random Fourier functions, without any hierarchical structure. Lower-degree components are again recovered earlier. Crucially, their recovered coefficients exhibit strong directional alignment with the target as their magnitudes increase (Pearson correlation 0.984 ± 0.005 across layer-degree measurements). We also quantify spurious Fourier energy outside the target support, which is relatively minimal. These measurements directly address the concern that intermediate functions may simply be inaccurate predictors.

  2. Gaussian Mixture Model (Section 4.1; Figure 4): We remove intermediate-layer supervision, training the network with only the standard final-output MSE loss and fitting intermediate readouts afterward. We observe a clear coarse-to-fine progression, with broad structures at earlier layers sharpening into localized density modes at later layers.

Additionally, our pretrained Pythia-1.4B experiments (Section 3.5, Figures 1–2; Appendix A.2) demonstrate progressively stronger pairwise interactions across 64 compositional examples, and illustrate cases where intermediate-layer gradients identify beneficial joint edits that final-layer gradients discourage.

2. Novelty, Annealing, and Gradient Diversity (Critical Weakness 2; Questions 3 & 4)

As shown in Figure 4 in the paper, the final layer contains sparse peaks, and gradient-based optimization from a random point offers no signal. The earlier layers are smoother versions of the final function, so it makes sense to optimize them in order. A random ordering of earlier layers would not allow us to optimize effectively.

Our contribution is not a new prediction head or discrete optimizer, but using the intermediate functions already represented within a trained network as a coarse-to-fine annealing trajectory.

Furthermore, we evaluate this through a depth-order ablation in the Lookahead-GCG setting raised by the reviewer (Section 4.2.2; Appendix A.5).

New depth-order ablation: To distinguish annealing from gradient diversity, we compare the ascending layer schedule against randomly shuffled layer schedules. Both use the same intermediate layers, number of updates per layer, and final-layer optimization stage.

MethodLlama 3 source ASRSix-model average ASR
Lookahead-GCG15.25 ± 9.4024.60 ± 1.84
Shuffled LA27.38 ± 31.8527.31 ± 8.06
Ascending LA86.25 ± 7.8436.83 ± 3.51

ASR (%, mean ± sample SD over four seeds); single-source HarmBench setting. Detailed results are provided in Appendix A.5, Table 6.

Ascending LA improves source-model ASR by +58.87 percentage points over shuffled LA, despite both using the same collection of intermediate gradients and objectives.

This provides direct evidence that the sequential coarse-to-fine ordering matters, rather than improvements arising solely from exposure to diverse intermediate-layer gradients.

AuthorsOpenReview ↗

Author Response (2/3)

3. Practical Significance and Cross-Task Generalization (Question 1)

Optimizing inputs against a fixed pretrained model is a general problem; we focus on LLMs as there are pre-existing, trained layerwise heads, so we can test our method without training anything from scratch. Within regards to LLMs, the current applications that involve gradient-based optimization of the input are prompt optimization and jailbreaking.

We therefore include a prompt optimization benchmark below.

Classification prompt optimization (GPT-2 Large; Section 4.2.3, Appendix A.6)

Automatic prompt optimization is an active research area, with recent gradient-based methods including GReaTer (Das et al., ICLR 2025) and SGCG (Wang et al., ACL 2025). We evaluate on the established six-dataset classification benchmark used by SGCG, comparing against standard GCG and FluentPrompt (Shi et al., 2023).

DatasetFluentPromptStandard GCGLayerwise GCG
SST-281.62 ± 3.6386.21 ± 3.3786.70 ± 5.85
AG News59.90 ± 6.7775.55 ± 3.1575.96 ± 2.46
IMDb77.48 ± 1.1877.77 ± 1.0380.09 ± 2.24
Amazon Polarity78.47 ± 3.8583.32 ± 5.9484.22 ± 2.95
Yelp Polarity84.42 ± 0.1885.59 ± 2.0886.09 ± 2.06
DBpedia42.13 ± 1.2453.39 ± 19.2876.11 ± 7.17

Held-out accuracy (%, mean ± SD over four seeds). Both GCG methods use 1,008 updates; FluentPrompt uses 5,000 updates and is not a matched-compute baseline.

Layerwise GCG achieves higher mean accuracy on all six datasets, including a +22.72 pp improvement on DBpedia, while using only 51.4% of standard GCG's depth-weighted optimization compute.

These results demonstrate practical utility on a distinct, established optimization task beyond jailbreaking.

4. Reproduced Baselines and Stronger Benchmarks (Question 5)

We agree that the original comparison against historical GCG results alone was insufficient. We have addressed this through two new evaluations.

Experiment 1: Reproduced GCG baseline (AdvBench Harmful Strings; Section 4.2.1, Appendix A.4)

We reproduce standard GCG under the same experimental setup and compare against layerwise GCG with approximately matched compute.

MethodSuccess ↑Mean CE ↓
Standard GCG (published)57%0.300
Standard GCG (our reproduction)63%0.3612
Layerwise GCG (4 depths, half compute)67%0.292
Layerwise GCG (all layers, half compute)70%0.1947
Layerwise GCG (matched compute)93%0.0602

In the new controlled comparison, layerwise GCG improves target-string success from 63% to 93% (+30 pp), while reducing mean final cross-entropy from 0.3612 to 0.0602.

AuthorsOpenReview ↗

Author Response (3/3)

Experiment 2: Lookahead-GCG (HarmBench; Section 4.2.2, Appendix A.5)

We benchmark against the recent method suggested by the reviewer, Lookahead-GCG (Feng et al., ICML 2026), a recent state-of-the-art method for universal, multi-model jailbreak optimization.

We reproduce its universal-prefix optimization task to the best of our ability: optimizing a single 100-token prefix on 20 HarmBench behaviors and evaluating behavioral jailbreak success across 200 behaviors and six LLMs. We compare single-source and two-source optimization using approximately matched compute, averaged over four seeds.

SourcesMethodL3L3.1L2VicunaGemmaQwenAvg.
L3Lookahead-GCG15.25 ± 9.400.13 ± 0.259.75 ± 4.3387.38 ± 2.596.75 ± 3.4828.38 ± 3.5224.60 ± 1.84
L3LA86.25 ± 7.842.13 ± 1.7010.63 ± 6.9885.75 ± 2.904.25 ± 3.0732.00 ± 12.7036.83 ± 3.51
L3 + L2Lookahead-GCG3.00 ± 1.080.00 ± 0.0043.63 ± 6.7687.50 ± 3.725.00 ± 1.2929.38 ± 2.7228.08 ± 1.71
L3 + L2LA65.50 ± 4.782.00 ± 1.0074.88 ± 3.9783.50 ± 0.719.00 ± 6.9848.13 ± 15.9147.17 ± 3.98

ASR (%, mean ± sample SD over four seeds); bold indicates the better mean within each source setting. L3/L3.1: Llama-3/3.1-8B-Instruct; L2: Llama-2-7B-Chat; Qwen: Qwen2.5-7B-Instruct. Our reconstruction differs from the published Lookahead-GCG implementation in source models and certain implementation details (Appendix A.5). Both methods use identical source models and evaluation protocols within our comparison. The six-model averages include source models.

LA substantially improves optimization on source models:

  • Single source: Llama 3 ASR improves by +71.00 pp.
  • Two sources: Llama 3 improves by +62.50 pp, and Llama 2 by +31.25 pp.

These improvements also show cross-model transfer, although gains vary by target. For example, Qwen ASR improves by +3.62 pp (single source) and +18.75 pp (two sources). Overall six-model average ASR improves by +12.23 pp and +19.09 pp, respectively.

This evaluation addresses the concern about relying exclusively on the older Harmful Strings benchmark, using a modern behavioral jailbreak benchmark and a stronger optimization baseline.

5. Scope and Limitations

In each of these settings, we use the established optimizer and simply change the objective to be layerwise annealing instead of always using the final layer. For LLM optimization, the best-performing optimizer is GCG, which does not use multi-token optimization. This is because the gradient is inaccurate for predicting single-token changes, so the token change must be evaluated before being accepted. To the best of our knowledge, we are not familiar with a multi-token optimizer used on LLMs.

We hope these additional experiments address the reviewer's concerns regarding the proposed mechanism, the importance of depth ordering, reproducibility, and practical generalization. We would appreciate the reviewer's feedback on the new results.

Reviewer_Kh57OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The presented paper proposes layerwise annealing which attaches a prediction head to intermediate layers of a frozen network with the the motivation that intermediate prediction heads can reveal interesting gradient information that the final objective obscures.

Critical strengths
  • Interesting idea to use interpretability methods for LLM prompt optimization
  • Writing is pretty clear for the main portion of the paper
Critical weaknesses

Weaknesses

  • The results in Table 1 compare to published numbers and not a replicated result which makes it hard to draw any conclusions from it since even minor code differences can make scores incomparable.
  • The main results in Table 1 are only on a single (old) model Llama-2-7B and a single benchmark, I think to make the arguments more compelling more experiments are needed.
  • While the Layerwise GCG (4 depths) improves Target-string ASR by 10 points, the CE is almost identical (0.292 vs. 0.30) which is a bit concerning, especially when compared to only published numbers.

Minor comments

  • L. 623 Eq. equation 13 typo
Questions for authors

N/A

5 comments
AuthorsOpenReview ↗

Author Response (1/3)

We thank the reviewer for their constructive feedback. We have substantially expanded our experiments to address all three critical weaknesses.

1. Reproduced GCG Baseline and Cross-Entropy Improvement (Weaknesses 1 & 3; Section 4.2.1)

We agree that comparison against published results alone was insufficient. We now reproduce standard GCG under the same experimental setup, comparing against layerwise GCG at approximately matched compute.

We also wish to clarify that our primary method traverses all layers, whereas the four-depth variant highlighted in the review is a reduced-depth ablation. Even at approximately half the compute, our all-layer method substantially improves mean CE (0.1947 vs. 0.30 for the published GCG reference), unlike the four-depth variant (0.292).

More importantly, in our new controlled, approximately compute-matched comparison, target-string success improves from 63% to 93% (+30 pp), while mean CE decreases from 0.3612 to 0.0602.

MethodSuccess ↑Mean CE ↓
Standard GCG (published)57%0.300
Standard GCG (our reproduction)63%0.3612
Layerwise GCG (4 depths, half compute)67%0.292
Layerwise GCG (all layers, half compute)70%0.1947
Layerwise GCG (matched compute)93%0.0602

These results address both the reproducibility concern and the previously modest cross-entropy improvement.

2. Broader Experimental Validation (Weakness 2)

We additionally evaluate layerwise optimization in two new experimental settings, extending beyond Llama-2 and AdvBench Harmful Strings.

AuthorsOpenReview ↗

Author Response (2/3)

Experiment 2: Universal jailbreak optimization (HarmBench; Section 4.2.2, Appendix A.5)

We compare against Lookahead-GCG (Feng et al., ICML 2026), a recent state-of-the-art method for universal, multi-model jailbreak optimization. We reproduce Lookahead-GCG on its original task to the best of our ability.

We optimize a shared 100-token prefix on 20 HarmBench behaviors and evaluate behavioral jailbreak success on 200 behaviors across six LLMs, using single-source and two-source optimization with approximately matched compute (four seeds).

SourcesMethodL3L3.1L2VicunaGemmaQwenAvg.
L3Lookahead-GCG15.25 ± 9.400.13 ± 0.259.75 ± 4.3387.38 ± 2.596.75 ± 3.4828.38 ± 3.5224.60 ± 1.84
L3LA86.25 ± 7.842.13 ± 1.7010.63 ± 6.9885.75 ± 2.904.25 ± 3.0732.00 ± 12.7036.83 ± 3.51
L3 + L2Lookahead-GCG3.00 ± 1.080.00 ± 0.0043.63 ± 6.7687.50 ± 3.725.00 ± 1.2929.38 ± 2.7228.08 ± 1.71
L3 + L2LA65.50 ± 4.782.00 ± 1.0074.88 ± 3.9783.50 ± 0.719.00 ± 6.9848.13 ± 15.9147.17 ± 3.98

ASR (%, mean ± sample SD over four seeds); bold indicates the better mean within each source setting. L3/L3.1: Llama-3/3.1-8B-Instruct; L2: Llama-2-7B-Chat; Qwen: Qwen2.5-7B-Instruct. Our reconstruction differs from the published Lookahead-GCG implementation in source models and certain implementation details, documented in Appendix A.5. Both methods use identical settings within our comparison.

LA substantially improves optimization on the source models: Llama 3 ASR increases by +71.00 pp (single source), while two-source optimization improves Llama 3 by +62.50 pp and Llama 2 by +31.25 pp.

These improvements also show cross-model transfer, notably to Qwen (+3.62/+18.75 pp for single/two-source optimization). Overall six-model average ASR increases by +12.23/+19.09 pp, respectively, including source models.

AuthorsOpenReview ↗

Author Response (3/3)

Experiment 3: Classification prompt optimization (GPT-2 Large; Section 4.2.3, Appendix A.6)

Automatic prompt optimization is an active research area, with recent gradient-based approaches including GReaTer (Das et al., ICLR 2025) and SGCG (Wang et al., ACL 2025). We evaluate on the established six-dataset classification benchmark used by SGCG, comparing layerwise GCG against standard GCG and FluentPrompt (Shi et al., 2023) using a frozen GPT-2 Large model.

DatasetFluentPromptStandard GCGLayerwise GCG
SST-281.62 ± 3.6386.21 ± 3.3786.70 ± 5.85
AG News59.90 ± 6.7775.55 ± 3.1575.96 ± 2.46
IMDb77.48 ± 1.1877.77 ± 1.0380.09 ± 2.24
Amazon Polarity78.47 ± 3.8583.32 ± 5.9484.22 ± 2.95
Yelp Polarity84.42 ± 0.1885.59 ± 2.0886.09 ± 2.06
DBpedia42.13 ± 1.2453.39 ± 19.2876.11 ± 7.17

Held-out accuracy (%, mean ± SD over four seeds). Both GCG methods use 1,008 optimization updates; FluentPrompt uses 5,000 updates and is not a matched-compute baseline.

Layerwise GCG achieves higher mean accuracy on all six datasets, including a +22.72 pp improvement on DBpedia, while using only 51.4% of standard GCG's depth-weighted optimization compute.

Together, these experiments provide substantially broader evidence for layerwise optimization across models, benchmarks, and both adversarial and non-adversarial optimization tasks.

3. Minor Comment

We thank the reviewer for identifying the typo; we have corrected this.

We hope these additional results address the reviewer's concerns regarding reproducibility, optimization improvements, and experimental generality. We would appreciate their feedback on the revised experiments.

Reviewer_WuzPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors suggest using the internal representations of a neural network (i.e. across layers) as the equivalent of an annealing path. In this analogy the, the lower layers have coarser and therefore smoother loss landscapes that are more easily optimised. In contrast, the final output function has sharper gradients and can be used to refine the final answer. The authors provide compelling, if very small scale, results.

The authors also provide an interesting analysis of the method in these toy settings and make connections to theoretical results.

Critical strengths
  1. The authors' choice of the two toy examples is sound and is successful at convincing the reader that their hypothesis about shallow layers representing a smoother approximation of the final objective is plausible, despite my concerns regarding the applicability of the toy problems. Particularly the example of fitting to Gaussian mixture density, and Figure 4 in the paper is helpful in developing intuition regarding the hypothesis. It clearly shows how earlier layers capture broad regions of the probability mass in a similar way to the temperature parameter in standard simulated annealing. Additionally, the pairwise token substitution plots (Figure 5) are very clear in identifying the problem of synergistic token interactions, showing how joint substitutions are increasingly impactful compared to the sum of the individual effects as one looks at deeper layers.
  2. Conceptually, the idea of the paper is very interesting and novel. The idea of leveraging the sequence of functions derived from layers of the neural network as an annealing trajectory for optimization and sampling has a clear problem: coupled interactions in the objective defined by the final layer can make optimization difficult and create a poor local gradient signal. Using the smoother objectives from intermediate layers to aid in exploring the search space is an intuitive approach, and potential applications of this in optimizing the highly complex and non-linear landscapes of neural network outputs are promising.
Critical weaknesses
  1. There isn’t enough evidence for the proposed layerwise smoothing mechanism. The experiments don’t sufficiently establish that the shallower layers of pretrained neural networks encode smoother versions of the final objective, with attenuated higher-order interactions. The Boolean Fourier staircase experiment (Section 4.1) uses a target constructed in a way so that the higher-order interactions extend lower-order ones, which likely biases the model into learning the desired coarse-to-fine progression. Similarly, the Gaussian mixture experiment uses explicit layerwise supervision, which differs from standard LLM pretraining. While the paper has evidence from a pretrained Pythia-1.4B model (Figures 1–2), it is limited to pairwise interactions in a small set of constructed examples. Overall it is unclear whether the toy experiments reflect a general property of pretrained networks or behavior specific to the chosen constructions and training procedures.
  2. The results and evidence in the paper are too narrow to support the paper's broad claims about layerwise annealing as a general-purpose method for optimization and sampling. The only practical application is adversarial suffix optimization on 100 AdvBench Harmful Strings targets using Llama-2-7B-Chat (Section 4.2). And although the Pythia experiments provide some mechanistic evidence, there is no demonstration of improved optimization on other tasks or model architectures. Evaluation on a single, relatively old chat model leaves the open question of whether the improvements extend to newer LLMs or are sensitive to differences in architecture, pretraining, or post-training/alignment procedures. It remains uncertain whether the proposed method generalizes to non-LLM architectures or to other optimization tasks beyond adversarial suffix optimization. The paper also lists sampling as a central motivation and stated application of layerwise annealing but provides no quantitative sampling experiments, since the Gaussian mixture experiment only visualizes intermediate density functions. All this makes it difficult to determine whether the reported gains are specific to the chosen optimization setting or support the broader applicability claimed by the authors.
Questions for authors
  1. Being sympathetic to potential compute limitations, is the improvement gained by the Layerwise GCG consistent across other families of language models (Qwen models, newer LLaMA models, GPT OSS, etc)? Adding these experiments and showing that the improvement holds across different open source models would be the highest-impact thing to raise my review.
  2. Does the same layerwise gradient based optimization work for other non-llm models like open source protein models etc? These experiments are required to evaluate the paper’s more general claims about neural networks, or otherwise the claims made in the paper should be tightened.
4 comments
AuthorsOpenReview ↗

Author Response (1/3)

We thank the reviewer for their thoughtful and detailed feedback. We have substantially strengthened the paper with new mechanistic experiments, reproduced baselines, and evaluations on additional optimization tasks and models.

Mechanism Summary

Intuitively, neural networks progressively construct more complex functions by hierarchically composing simpler features across layers. Consequently, lower-degree components of the final function may emerge at earlier layers, while higher-degree interactions develop at later layers. This suggests that earlier-layer functions can act as smoothed approximations of the final objective, preserving useful low-order structure.

This intuition is supported by prior work showing that intermediate representations progressively approach their final representations and yield increasingly accurate predictions across language models (Belrose et al., 2023) and image classification models (Jiang et al., 2025). Our experiments investigate the stronger hypothesis that this progression also corresponds to coarse-to-fine structure in the decoded functions.

1. Mechanistic Experiments (Critical Weakness 1; Section 4.1)

We thank the reviewer for their helpful feedback. We have strengthened our mechanistic evidence through two experiments (Section 4.1):

  1. Random Fourier functions (Section 4.1; Appendix A.1): We now evaluate 20 independently sampled random Fourier functions without any constructed hierarchy. We observe the same earlier recovery of lower-degree components. Moreover, recovered coefficient magnitudes are strongly associated with directional alignment to the true coefficients, and we explicitly quantify spurious Fourier energy outside the target support.

  2. Gaussian Mixture Model (Section 4.1; Figure 4): We remove layerwise supervision and train using only the standard final-output MSE loss. We still observe the same coarse-to-fine progression, with broad structures at earlier layers sharpening into localized modes at later layers.

We hope these additional experiments address the reviewer's concerns about the proposed mechanism.

2. Broader Experimental Validation (Critical Weakness 2)

We thank the reviewer for highlighting the need for broader validation. We have substantially expanded our evaluation across three experimental settings, including stronger baselines, multiple LLM architectures, and non-adversarial optimization tasks.

Experiment 1: AdvBench Harmful Strings (Llama-2-7B-Chat; Section 4.2.1, Appendix A.4)

We reproduce standard GCG under the same experimental setup and compare against layerwise GCG at approximately matched compute. Layerwise GCG improves target-string success from 63% to 93% (+30 pp), while reducing mean final cross-entropy from 0.3612 to 0.0602. Even at approximately half the compute, layerwise GCG improves both metrics.

MethodSuccess ↑Mean CE ↓
Standard GCG (published)57%0.300
Standard GCG (our reproduction)63%0.3612
Layerwise GCG (4 depths, half compute)67%0.292
Layerwise GCG (all layers, half compute)70%0.1947
Layerwise GCG (matched compute)93%0.0602
AuthorsOpenReview ↗

Author Response (2/3)

Experiment 2: Universal jailbreak optimization (HarmBench; Section 4.2.2, Appendix A.5)

We compare against Lookahead-GCG (Feng et al., ICML 2026), a recent state-of-the-art method for universal, multi-model jailbreak optimization. We reproduce Lookahead-GCG on its original task to the best of our ability, using the same experimental setup for both methods.

We optimize a shared 100-token prefix on 20 HarmBench behaviors and evaluate behavioral jailbreak success on 200 behaviors across six LLMs. We compare single-source and two-source optimization with approximately matched compute, averaging over four seeds.

SourcesMethodL3L3.1L2VicunaGemmaQwenAvg.
L3Lookahead-GCG15.25 ± 9.400.13 ± 0.259.75 ± 4.3387.38 ± 2.596.75 ± 3.4828.38 ± 3.5224.60 ± 1.84
L3LA86.25 ± 7.842.13 ± 1.7010.63 ± 6.9885.75 ± 2.904.25 ± 3.0732.00 ± 12.7036.83 ± 3.51
L3 + L2Lookahead-GCG3.00 ± 1.080.00 ± 0.0043.63 ± 6.7687.50 ± 3.725.00 ± 1.2929.38 ± 2.7228.08 ± 1.71
L3 + L2LA65.50 ± 4.782.00 ± 1.0074.88 ± 3.9783.50 ± 0.719.00 ± 6.9848.13 ± 15.9147.17 ± 3.98

ASR (%, mean ± sample SD over four seeds); bold indicates the better mean within each source setting. L3/L3.1: Llama-3/3.1-8B-Instruct; L2: Llama-2-7B-Chat; Qwen: Qwen2.5-7B-Instruct. Our reconstruction differs from the published Lookahead-GCG implementation in its source models and certain implementation details, documented in Appendix A.5. Both methods use identical source models and evaluation protocols within our comparison.

LA substantially improves optimization on the source models:

  • Single source: Llama 3 improves from 15.25% to 86.25% (+71.00 pp).
  • Two sources: Llama 3 improves from 3.00% to 65.50% (+62.50 pp), and Llama 2 from 43.63% to 74.88% (+31.25 pp).

These improvements also show cross-model transfer, although gains vary by target. For example, Qwen ASR improves by +3.62 pp (single source) and +18.75 pp (two sources). Overall six-model average ASR increases by +12.23 pp and +19.09 pp, respectively (including source models).

AuthorsOpenReview ↗

Author Response (3/3)

Experiment 3: Classification prompt optimization (GPT-2 Large; Section 4.2.3, Appendix A.6)

Automatic prompt optimization is an active research area, with recent gradient-based approaches including GReaTer (Das et al., ICLR 2025) and SGCG (Wang et al., ACL 2025). We evaluate on the established six-dataset classification benchmark used by SGCG, comparing layerwise GCG against standard GCG and FluentPrompt (Shi et al., 2023) using a frozen GPT-2 Large model.

DatasetFluentPromptStandard GCGLayerwise GCG
SST-281.62 ± 3.6386.21 ± 3.3786.70 ± 5.85
AG News59.90 ± 6.7775.55 ± 3.1575.96 ± 2.46
IMDb77.48 ± 1.1877.77 ± 1.0380.09 ± 2.24
Amazon Polarity78.47 ± 3.8583.32 ± 5.9484.22 ± 2.95
Yelp Polarity84.42 ± 0.1885.59 ± 2.0886.09 ± 2.06
DBpedia42.13 ± 1.2453.39 ± 19.2876.11 ± 7.17

Held-out accuracy (%, mean ± SD over four seeds). Both GCG methods use 1,008 optimization updates; FluentPrompt uses 5,000 updates and is not a matched-compute baseline.

Layerwise GCG achieves higher mean accuracy on all six datasets, including a +22.72 pp improvement on DBpedia, while using only 51.4% of standard GCG's depth-weighted optimization compute.

Together, these experiments provide substantially broader evidence for layerwise optimization across LLM architectures, optimization methods, and adversarial and non-adversarial tasks. We would appreciate the reviewer's feedback on these new results.

3. Responses to Specific Questions

Q1. Does the improvement generalize across other LLM families?

We have substantially addressed this through Experiment 2, which follows the universal, multi-model optimization and transfer setting of Lookahead-GCG (ICML 2026). We optimize on Llama-2 and Llama-3 and evaluate across six models, including newer Llama, Qwen, Gemma, and Vicuna models.

Our results demonstrate substantial improvements on the optimization source models, along with transfer improvements on several other model families.

Q2. Does layerwise optimization generalize to non-LLM models, such as protein models?

Our expanded mechanistic experiments show that the coarse-to-fine hypothesis holds in controlled settings. Furthermore, we demonstrate practical applications on modern LLMs. We believe this may reflect a broader property of neural network training and therefore could extend to protein models as well.

However, we have not yet evaluated pretrained protein models, as suitable pretrained intermediate-layer prediction heads are not readily available for these architectures. We will try to do so before the reviewing period ends.

Hi,

As the reviewing period is closing tomorrow, we would greatly appreciate if you could read/engage with our response.

Thank you!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their feedback. We would like to briefly reiterate our central contribution and summarize the additional evidence provided during the rebuttal period.

By training a linear head at each layer, we obtain a sequence of layerwise functions. Through toy experiments, we demonstrate a coarse-to-fine progression: earlier layers produce smoother approximations of the final-layer function.

We are the first to use this observation as an optimization strategy. Gradients from earlier, smoother layers guide optimization toward promising regions, while gradients from later, more non-convex layers progressively refine the solution. This helps overcome local minima that make direct optimization of the final-layer function difficult.

We demonstrate the effectiveness of this approach in jailbreaking and prompt optimization:

  • Jailbreaking: We substantially outperform the previous SOTA for universal prefix optimization, Lookahead-GCG (ICML 2026). On HarmfulStrings, we achieve a 93% vs. 63% source-model optimization success rate at matched compute.
  • Prompt optimization: Across six datasets, we achieve stronger performance with approximately half the compute.

During the rebuttal period, we have added two new tasks, conducted additional ablations, and strengthened our mechanistic toy experiments. We believe these additions address the reviewers' primary concerns and further strengthen the empirical effectiveness of our approach.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.