acceptodds
Under review as a conference paper at ICLR 2027

ActivationSMC: Sequential Monte Carlo for Optimizing Activation Steering

Abstract

Activation steering controls pretrained language models and beyond at inference by modifying internal activations, but its effectiveness hinges on jointly deciding where and how strongly to intervene. This creates a hard optimization over a mixed discrete-continuous space with coupled layer choices and often non-differentiable rewards, poorly handled by fixed grids or greedy decoupled search. We introduce \bf ActivationSMC, a derivative-free black-box framework based on Sequential Monte Carlo that maintains a population of full steering configurations. It guides the population toward high-reward regions via reweighting, resampling, and stochastic mutation that preserves diverse modes, with a multi-fidelity schedule that uses small subsets for cheap early exploration and larger subsets for precise refinement. Without updating model weights, ActivationSMC configures off-the-shelf activation steering operators such as COLD-Steer, AUSteer, CAA, and Spherical Steering. Across TruthfulQA, R-Bench, AMBER, as well as text-to-image generation using Stable Diffusion, ActivationSMC consistently outperforms native tuning. It improves collapsed baselines by up to 28.9 percentage points, raises compositional text-to-image alignment by 20–30 percentage points across four steering operators, and substantially widens its lead as joint multi-layer optimization exposes the breakdown of decoupled heuristics.

Primary area:
generative models
Keywords:
Activation Steering, Sequential Monte Carlo

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qM1FOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ActivationSMC, a derivative-free framework for jointly selecting activation-steering layers and their associated steering parameters. The motivation is that these choices should not be optimized independently: an intervention at an earlier transformer layer changes the representation received by subsequent layers, creating potentially important cross-layer interactions. The resulting search problem is mixed discrete-continuous, non-convex, and may involve non-differentiable evaluation metrics.

ActivationSMC represents complete steering configurations as particles and constructs a reward-tempered distribution over them. It then applies standard Sequential Monte Carlo operations, including importance weighting, effective-sample-size-based tempering, systematic resampling, and Metropolis-Hastings mutation. To reduce evaluation cost, the method uses a multi-fidelity schedule in which configurations are first evaluated on small data subsets and progressively promoted to larger subsets. An archive retains previously evaluated configurations, after which promising candidates are reranked on a validation set.

The method is applied to four existing steering operators: CAA, COLD-Steer, AUSteer, and Spherical Steering. Experiments cover text-to-text steering on TruthfulQA, image-to-text steering on R-Bench and AMBER, and text-to-image steering with Stable Diffusion 3.5 Large Turbo. ActivationSMC generally improves over each method’s native tuning procedure, with larger reported gains when several intervention layers are selected jointly. The paper also includes comparisons with Bayesian optimization and simulated annealing.

Critical strengths

The central motivation is reasonable and practically relevant. Multi-layer interventions are inherently coupled, and tuning the layer locations and intervention parameters independently can miss configurations that work well only through coordinated effects. This concern becomes more important as the number of intervention layers increases. The reported image-to-text results are directionally consistent with this motivation: the advantage over native tuning reportedly becomes larger under four-layer steering than under single-layer steering. Thus, the paper identifies a legitimate methodological problem rather than constructing an artificial optimization setting solely to favor the proposed algorithm.

ActivationSMC supports mixed discrete-continuous configuration spaces and does not require gradients through either the model or the evaluation metric. This permits the use of discrete correctness measures, external evaluators, and multimodal objectives. Demonstrating the same framework across four steering operators and three modalities is a meaningful practical strength and suggests that the implementation is not narrowly tailored to one operator or benchmark. These strengths make the work potentially useful as an engineering framework. However, they do not overcome the concerns regarding the paper’s technical novelty and experimental validation.

Critical weaknesses

The proposed method is primarily an application of established SMC machinery. Reward-tempered target distributions, importance weighting, ESS-based tempering, resampling, Metropolis-Hastings mutation, and multi-fidelity evaluation are all standard ideas. The activation-steering-specific contribution seems largely restricted to encoding steering configurations as particles and defining suitable local and global proposals.

The paper does not convincingly explain why SMC is particularly well suited to this problem compared with other black-box optimizers. Mixed discrete-continuous spaces with fewer than approximately twenty variables are also natural settings for Bayesian optimization, evolutionary strategies, the cross-entropy method, simulated annealing, SMAC, or random search with successive halving. Generic convergence properties of SMC do not establish that it will efficiently locate good steering policies under a finite and expensive inference budget.

More importantly, ActivationSMC is not compared with the native tuning procedures on equal methodological terms. It receives a population of candidates, local and global proposals, multi-fidelity evaluation, adaptive resampling, an archive of previously visited configurations, and a final validation-based reranking stage. The native baselines are mostly simple parameter sweeps or sequential heuristics that were not intended to be competitive general-purpose optimizers. Consequently, the results may demonstrate only that an elaborate joint-search pipeline outperforms weak, decoupled tuning rules.

The paper needs component-matched baselines. For example, random search, the cross-entropy method, an evolutionary strategy, and Bayesian optimization should receive the same initialization budget, fidelity schedule, candidate archive, and final reranking procedure. Ablations should separately remove global proposals, multi-fidelity evaluation, archiving, and validation reranking. Without these controls, the gains cannot be specifically attributed to SMC.

The claim that SMC preserves multiple promising modes is also insufficiently supported. Resampling can rapidly eliminate lower-weight modes, especially when rewards are noisy, and mutation restores diversity only if proposals mix effectively. The paper should report mutation acceptance rates, unique-particle counts, particle genealogies, and sensitivity to the global-proposal probability and ESS threshold. Merely maintaining a particle population does not demonstrate meaningful mode preservation.

The main results are largely presented as point estimates. ActivationSMC contains substantial randomness from particle initialization, resampling, mutation, data-subset construction, and model generation. The relevant experimental unit is therefore the complete optimization run. Confidence intervals computed across test examples do not measure this optimizer-level variability. Every major experiment should be repeated end to end with multiple seeds, with means and standard deviations reported across searches.

The archive and validation-reranking procedure also create a potential winner’s-curse problem. ActivationSMC adaptively evaluates many configurations and selects the best candidate from the terminal population and archive. With a finite validation set, selecting the maximum among many noisy estimates can yield an overly optimistic winner. The manuscript does not clearly report the archive size, number of unique configurations evaluated, number of finalists reranked, or whether competing methods receive the identical selection process.

Several empirical gains are modest, while some optimized configurations improve the primary measure but reduce other metrics. For example, improvements in TruthfulQA multiple-choice performance can coincide with lower open-ended truthfulness-related scores. In text-to-image experiments, alignment metrics improve while LPIPS sometimes decreases, indicating reduced output diversity. These trade-offs may be acceptable, but the conclusion depends heavily on how the scalar optimization reward is defined. The paper should report sensitivity to alternative reward weights or provide Pareto frontiers rather than relying on a single manually selected scalarization.

The text-to-image evaluation is particularly vulnerable to reward overfitting because automatic metrics closely related to the search objective are also used to support the final quality claim. A stronger optimizer may simply exploit imperfections in these metrics more effectively. Blinded human preference evaluation is needed to determine whether the resulting images are genuinely preferable.

Finally, there is insufficient evidence of transfer or capability preservation. The selected configurations are generally evaluated on held-out examples from the same benchmark distribution used during optimization. The paper does not show whether policies optimized on one benchmark transfer without retuning to a related benchmark, remain robust under prompt perturbations, or preserve unrelated model capabilities. This is important because aggressive optimization of one behavioral score may damage calibration, fluency, diversity, refusal behavior, or general instruction following.

Overall recommendation: Strong Reject. The paper studies a relevant problem and presents a competent implementation, but the algorithmic contribution is largely a direct application of existing SMC techniques. The experimental design does not convincingly separate SMC from the benefits of a stronger search-and-selection pipeline, and the reported gains lack the statistical, computational, and out-of-distribution validation necessary to support the paper’s central claims.

Questions for authors

Are the comparisons matched by total generated tokens, model forward passes, diffusion steps, evaluator calls, GPU-hours, and validation-reranking cost, rather than only by the number of candidate configurations?

How many independent end-to-end optimization runs were performed? Please report means and standard deviations across complete runs with different particle initializations, mutation randomness, resampling randomness, and generation seeds.

How many unique configurations are evaluated, archived, shortlisted, and reranked? Do all competing optimization methods receive the identical archive and final validation-selection procedure?

Which components account for the reported improvements? Please provide ablations removing multi-fidelity evaluation, global proposals, the archive, and final reranking, as well as comparisons with random search and the cross-entropy method under the same pipeline.

What evidence demonstrates that multiple modes are preserved after resampling? Please report unique-particle counts, mutation acceptance rates, particle genealogical diversity, and sensitivity to the ESS threshold.

Do configurations optimized on one benchmark transfer without retuning to a distinct benchmark, and do they preserve unrelated model capabilities?

Are the text-to-image improvements confirmed through blinded human preference judgments rather than exclusively through automatic metrics?


POST REBUTTAL COMMENTS

The authors have addressed most of the comments. So, I raise my score to weak reject.

5 comments
AuthorsOpenReview ↗

Response to Reviewer qM1F (1/3)

We thank the reviewer for the detailed review. We appreciate the recognition that our work “identifies a legitimate methodological problem rather than constructing an artificial optimization setting,” and that applying one framework “across four steering operators and three modalities is a meaningful practical strength.” The other reviewers share this view: Reviewer mBWh notes that “the problem is real” and the sampler is “technically clean,” and Reviewer KtH4 describes ActivationSMC as “a principled way to address this problem.”

We appreciate the reviewer’s assessment that the soundness concerns are “minor or fixable.” We recognize that aspects of our method and experimental protocol could have been presented more clearly, and we have strengthened the manuscript with clarifications and additional results. We would be grateful if the reviewer would reconsider the recommendation in light of this evidence. Below, we group related weaknesses and questions alongside detailed responses.


1. Novelty (W1).\Large\text{1. Novelty (W1).}

We respectfully disagree with the comment that "the activation-steering-specific contribution seems largely restricted to encoding steering configurations as particles and defining suitable local and global proposals". We did not claim that tempering, ESS control, resampling, and MH mutation are new. Our contribution is the problem, its formulation, and the design that makes these components work on it.

  • Problem. Where and how strongly to steer is currently set by grids, 1-D sweeps, or greedy layer-wise search. To our knowledge, no prior work treats this choice as black-box optimization over the joint multi-layer, mixed discrete–continuous space. JoLA requires differentiable rewards, and GRACE searches only one layer and one coefficient.
  • Formulation. The Gibbs target is the unique maximizer of Eq[J(θ)]−TKL(q∥ν0)\mathbb{E}_q[J(\theta)]-T\mathrm{KL}(q\|\nu_0). This is the KL-regularized form used in RLHF, but defined over steering configurations with the model weights frozen. The end-to-end steering objective is therefore optimized directly, without gradients through the model or the evaluator.
  • Design. Prior SMC steering methods (Lew et al., 2023; Loula et al., 2025) place particles over tokens. Our particles are operator configurations scored on complete outputs, which is what makes data-graduated fidelity possible. The incremental weights (Eq. 10, Appendix C.2) correct jointly for changing subsets and temperatures. Layer moves keep each parameter block attached to its intervention, while global redraws let particles jump between modes.
  • Generality. Our proposed framework, ActivationSMC, generalizes across diverse activation steering operators. In our experiments, we validated its effectiveness across four operators and three modalities.

2. Why SMC rather than other black-box optimizers (W2).\Large\text{2. Why SMC rather than other black-box optimizers (W2).}

In this problem, the reward is noisy, changes abruptly with discrete layer choices, and can be multimodal (Figs. 6–8). None of this is known before the search begins. Figure 1 compares SMC with two existing benchmark approaches (greedy search and BO). Next we discuss a few other alternative approaches:

  • BO, SMAC, TPE: efficient when a surrogate model fits the reward. Abrupt changes across layers make the surrogate misspecified.
  • CMA-ES, CEM: adapt a single parametric search distribution. Multimodal rewards therefore require restarts, and with a fixed sample size the distribution can contract prematurely.
  • SA: follows a single trajectory, which under practical cooling schedules can remain trapped in one region. SMC runs many trajectories and reallocates effort among them.
  • Random search with successive halving: allocates budget well among a fixed pool of candidates but cannot improve on that pool, so it depends on drawing good configurations by chance.

SMC needs only scalar rewards and requires neither a surrogate nor a unimodal search distribution. We agree that generic convergence theory does not by itself guarantee efficiency under a finite budget, and we will not present it that way.

The landscape visualizations appear in Appendix E, Figures 6–8, and the conditional theoretical analysis appears in Appendix C, Theorem C.1.


AuthorsOpenReview ↗

Response to Reviewer qM1F (2/3)

3. Fair comparisons and ablations (W3, W4, Q1, Q4).\Large\text{3. Fair comparisons and ablations (W3, W4, Q1, Q4).}

Clarification: native tuning is not a low-cost baseline. In our controlled TruthfulQA run, greedy native tuning took 103 minutes, compared with 108 minutes for ActivationSMC, under the same parameter bounds, reward, and selection rule.

We added new optimizers that share the same search space and final-selection rule (full tables in our response to Reviewer mBWh Q1, addressing their request for additional comparisons with alternative optimization methods). For R-Bench, budgets are matched in search NFEs, counted as candidate–example evaluations. Total costs, including re-scoring, are within about 10%, and on the same hardware wall-clock time tracks these counts at 135–141 s per 1,000 evaluations. For TruthfulQA, budgets are matched at 144 logical requests per run, equivalent to 9,216 candidate–question requests, with cached requests charged against the budget. Fresh evaluation counts and optimization times differ and are reported separately. The new stochastic-optimizer comparisons use three independent search seeds; unsteered and native/greedy results are fixed references:

SettingActivationSMCAlternatives
TruthfulQA, Spherical, K=2K=2 (MC Avg.)59.2TPE 55.7, random 52.5, greedy 52.3
R-Bench/LLaVA, COLD-Steer, K=1K=1 (accuracy)75.6SH 75.5, random 75.4, GP-BO 75.1, native 69.9

ActivationSMC achieves the highest mean accuracy in the single-layer comparison, where all tested search optimizers outperform native tuning. Its advantage is more pronounced in the larger TruthfulQA space, exceeding random search and TPE by 6.7% and 3.5%, respectively.

Multi-fidelity ablation. We compared staged and flat evaluation schedules at matched total cost on R-Bench/LLaVA (3 seeds). The flat schedule evaluates every candidate on all 384 examples and uses fewer particles to keep the cost equal. Comparing at approximately equal total cost tests whether staging uses the available budget more effectively. Staging achieves higher mean accuracy for all three operators, with clear numerical gains for Spherical and CAA. Accuracy (%):

OperatorStagedFlat
Spherical76.76 ±\pm 0.3575.80 ±\pm 0.43
CAA77.81 ±\pm 0.2775.45 ±\pm 1.75
COLD-Steer75.57 ±\pm 1.7175.51 ±\pm 1.74

We also conducted layer-count and particle-count ablations (Fig. 4), testing performance as more coupled intervention choices are introduced and the accuracy–computation trade-off of population size. Time constraints prevented separate ablations of global proposals, the archive, and final re-ranking, and evaluation of CEM. Nevertheless, the completed ablations and budget-matched optimizer comparisons provide complementary evidence for effective budget allocation, robustness to coupled layer choices, and competitive search performance, while leaving individual component contributions for future study.

The optimizer comparisons appear in Section 4.1 and Appendices B.1.1–B.1.3, Tables 9–11; the particle/layer ablations appear in Appendix B.2.1, Figure 4, and the validation-schedule comparisons in Appendix B.3, Figure 5 and Table 15. Section 4.4 provides an overview of the ablation studies.


4. Variance over different runs (W6, Q2).\Large\text{4. Variance over different runs (W6, Q2).}

We ran 48 complete searches: all eight R-Bench settings, with 3 seeds and 2 folds each. The standard deviation is below 0.5% in seven of the eight settings. ActivationSMC improves over native tuning by 1.6–6.2% in four settings and is within ±0.5\pm 0.5% in the other four (full table in our response to Reviewer mBWh Q2, addressing their request for standard deviations). The new optimizer comparisons also report mean ±\pm std over 3 seeds. The complete repeated-search results appear in Appendix B.4.1, Table 16, with the repeated optimizer comparison in Appendix B.1.2, Table 10.


5. Bias from selecting the best configuration (W7, Q3).\Large\text{5. Bias from selecting the best configuration (W7, Q3).}

Clarification: selection optimism inflates validation scores, not the reported test results. Every policy is fixed before it is evaluated on a disjoint test set.

All optimizers use the same validation examples and the same winner-selection rule. For the N=96N=96 image-to-text runs:

  • ActivationSMC evaluates 672 configurations, of which 555–660 are distinct. It keeps the top 8 archived configurations together with the final population and re-scores 92–104 finalists.
  • For comparison, random search re-scores 88 finalists, successive halving 72–80, and BO 96–104. Because the validation subset overlaps the final search subset, we will remove the claim that re-scoring eliminates selection bias.

The search/final-selection data relationship and candidate accounting are described in Section 3.3 and Appendix A.1, with the optimizer selection protocol discussed in Appendix B.1.2.


AuthorsOpenReview ↗

Response to Reviewer qM1F (3/3)

6. Metric trade-offs, text-to-image, and transfer (W8–W10, Q6, Q7).\Large\text{6. Metric trade-offs, text-to-image, and transfer (W8–W10, Q6, Q7).}

TruthfulQA trade-offs: In the controlled comparison, ActivationSMC improves MC Avg. over greedy tuning by 6.9% and also improves open-ended TRUE×INFO (67.9 vs. 53.5). The search reward assigns equal weights (1:1) to the multiple-choice score and TRUE×INFO. These results evaluate the specified equal-weight objective. We have not performed a reward-weight sensitivity study or estimated a Pareto frontier, so we do not claim that this weighting is optimal or that the gains hold across alternative weightings.

The reward weights are specified in Appendix A.1. The controlled-comparison findings and metric trade-offs are discussed in Section 4.1, with detailed results in Appendix B.1.1, Table 9.

Capability: We applied the TruthfulQA policies unchanged to general benchmarks. MMLU drops by 2.3% (1.5% for native tuning). WikiText-2 perplexity rises for both tuned methods, from 6.4 to 18.3 for ours and to 16.2 for native tuning. We report this trade-off in the appendix.

This capability trade-off is acknowledged in Section 4.1, with detailed results in Appendix B.6.2, Table 22.

Transfer: We transferred Spherical policies on LLaVA between R-Bench and AMBER without retuning to test whether the selected configurations generalize across datasets. Neither native nor ActivationSMC improves target accuracy over the unsteered model (e.g., AMBER→R-Bench: 70.1 for ours, 70.6 for native, 73.3 unsteered). A plausible explanation is that source-domain steering signals and parameters transfer imperfectly under distribution shift. Our claims therefore concern in-domain optimization.

The transfer protocol and results appear in Appendix B.6.1, Table 21.

Text-to-image: We have not run a blinded human study. HPSv3 was not part of the search reward, so it provides an independent check. We have described HPSv3 as an automatic preference predictor and limit our conclusions accordingly.

Section 4.3 describes HPSv3 as an automatic predictor of human preference.


7. Mode preservation (W5, Q5).\Large\text{7. Mode preservation (W5, Q5).}

Final populations retain 84–96 distinct configurations out of 96, spanning 8–23 layers and 8–22 initial ancestors (R-Bench, three operators × three seeds; see our response to Reviewer KtH4 Q4). We agree that these results show retained diversity rather than the preservation of distinct high-reward modes, and we will qualify the claim. The diversity diagnostics show that populations retain many distinct configurations and intervention layers despite sharing ancestors.

Accuracy is also stable across target ESS fractions (R-Bench/LLaVA, K=1K=1, N=48N=48):

ESS fractionSphericalCAACOLD-SteerAUSteer
0.3076.6677.0274.1275.18
0.4576.1975.9574.1775.55
0.7576.4176.3174.1275.10
0.9076.6975.2274.9675.45

Accuracy varies by at most 0.84% across ESS fractions, except for CAA (1.8%). The ESS sweep shows that accuracy is relatively insensitive to the tested targets for three operators, with greater sensitivity for CAA. Together, these results support retained configuration diversity and limited dependence on the ESS setting.

These diagnostics and the ESS sensitivity results now appear in Appendix B.4.2 and B.4.3, Tables 17 and 19.


We hope these new results and revisions address the reviewer's concerns, and we would be grateful if they could be considered in the final assessment. We have revised the manuscript accordingly and highlighted all changes in blue. We are happy to address your further comments.

Reviewer_KtH4OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduced ActivationSMC. The idea is to have a gradient-free optimization framework for activation steering in pre-trained models. The key problem to address here is to jointly select where to intervene (layers) and how strongly to intervene (steering parameters). Different from previous typical approaches that use either fixed grids or greedy search, this ActivationSMC uses Sequential Monte Carlo (SMC) optimization to figure out the joint configuration space without updating model weights or compute differentiable rewards.

The key contributions are as follows. a. This paper introduces the SMC to optimize steering locations and strengths. b. This paper uses population-based search that maintains multiple candidate configurations through reweighing, resampling and mutation. c. This paper uses progressively larger validation subset and effective sample size (ESS) controlled tempering to balance the search effiency.

In the experiments, ActivationSMC works for four steering operators ( COLD-Steer, AUSteer, CAA, and Spherical Steering) and 3 tasks, e.g. text-to-text in TruthfulQA, Image-to-text (R-Bench and AMBER), and text-to-image (Stable Diffusion 3.5). Results are showing that proposed one beat the native tuning methods.

Critical strengths

First, this paper provides a great setup with a well motivated problem and a flexible optimization framework. Existing activation steering do have this limitations that requires one to jointly optimization two things. The proposed SMC framework provides a principled way to address this problem. The approach is cool because it helps to apply to different steering operators without modify the pre-trained model weights.

Second, the empirical results on different datasets show that the proposed approach at least beat the native tuning procedures by good margin.

Critical weaknesses

There are some critical weakness with the proposed approach it self, and there are some concerns with the paper presentations.

a. paper presentation issues. The paper did not have solid evidences to show that the SMC drives the performance gains. The main comparisons focus on native tuning procedures. This native tuning procedures rely on the grid searches and greedy. Since ActivationSMC jointly optimizes layer locations and steering strengths, the observed improvements may mostly come from the larger joint search space rather than SMC itself. We are missing strong black-box optimizers under identical search spaces, e.g. multi-fidelity schedules.

b. approach itself. ActivationSMC requires repeated model inference to evaluate candidate configurations. This can be very expensive and impractical for large language and diffusion models. The paper does not provide sufficient evidences to figure out the trade-off between search cost and performance. It is hard to tell if these performance are justify at all.

Questions for authors
  1. I am not sure if the additional baselines about bayesian optimization and simulated annealing appeared in appendix search the same joint space of intervention layers and steering parameters? Do they set up under the same computational budgets?

  2. Could the authors provide a comparison of the total optimization cost (e.g., number of model evaluations or GPU hours, or other cost metrics) between ActivationSMC and the baselines? Does ActivationSMC maintain a good performance-cost trade-off at K=4?

  3. It is hard to tell what really works, can author provide a toy example on a simple MLP or a simple transformer, and show some ablation study results?

  4. During experiments, the paper suggested that SMC preserves competing high-reward modes, but would SMC lose its diversity? Can we measure configuration diversity?

2 comments
AuthorsOpenReview ↗

Response to Reviewer KtH4 (1/2)

We sincerely thank the reviewer for the thoughtful review and positive assessment. We appreciate the recognition that our work offers “a well motivated problem and a flexible optimization framework,” and that ActivationSMC “provides a principled way to address this problem” across different steering operators without modifying pre-trained model weights. We are also grateful that the reviewer found our empirical results beat native tuning “by good margin” across datasets. Below, we have grouped related weaknesses and questions alongside detailed responses.


1. Does SMC itself drive the gains? (Wa, Q1)\Large\text{1. Does SMC itself drive the gains? (Wa, Q1)}

The BO and SA baselines in Appendix B.1.3 search the same joint space as ActivationSMC: the same intervention-layer choices, the same K, and the same parameter bounds. Their search budgets are matched, and ActivationSMC outperforms both on every operator tested in Table 11. We have made this more clear in Appendix B.1.3. We also ran new controlled comparisons, detailed in our response to Reviewer mBWh Q1, addressing their request for additional comparisons with alternative optimization methods.

  • TruthfulQA (LLaMA-3.1-8B, Spherical, K=2, three seeds): ActivationSMC reaches 59.2 MC Avg., versus 55.7 for TPE, 52.5 for random search, and 52.3 for greedy tuning.
  • R-Bench (LLaVA, COLD-Steer, K=1, three seeds): ActivationSMC reaches 75.6% accuracy, versus 75.5 for successive halving, 75.4 for random search, 75.1 for GP-BO, and 69.9 for native tuning.

ActivationSMC achieves the highest mean accuracy in the repeated R-Bench comparison, with a more pronounced advantage in the larger, judge-based TruthfulQA search space. Note all operators outperform native tuning in this example. These results support the effectiveness of its joint population-search framework, while the small R-Bench differences remain modest relative to search variability.

The additional R-Bench comparisons are reported in Appendix B.1.2, Table 10, with search and final-selection costs in Appendix A.4.


2. Cost and the cost–performance trade-off (Wb, Q2)\Large\text{2. Cost and the cost–performance trade-off (Wb, Q2)}

Search is a one-time optimization cost. Once selected, the policy is deployed through its steering interventions, without maintaining particles or repeating the search at inference.

We primarily compare costs using NFEs, since runtime varies with hardware, batching, and parallelization. Across all method ablations unless stated otherwise, optimizers use 69,120 search candidate–example evaluations. Including final selection, their mean costs across three seeds are:

MethodTotal candidate–example evaluations
ActivationSMC107,264
Random search102,912
Successive halving97,920
BO108,032

Thus, search budgets are identical and total costs are approximately matched. We study optimization cost to test whether ActivationSMC’s accuracy gains justify the search expense and remain practical as the steering configuration space grows. For a fixed particle count and evaluation schedule, ActivationSMC's search-evaluation budget does not grow with K, unlike exhaustive grid search. Figure 3 reports the N=48 configuration, while the main K=1 experiments use N=96.

At K=4, ActivationSMC improves over native tuning in 11 of 12 settings, by 9.73% on average, retaining performance where native tuning can fall below the unsteered model. These results show that ActivationSMC can handle larger steering configuration spaces without increasing its search-evaluation budget.

Configuration-specific costs are detailed in Appendix A.4 and Figure 3; the corresponding accuracy trends and multi-layer results appear in Figure 4b, Appendix B.2.3, and Table 14.


AuthorsOpenReview ↗

Response to Reviewer KtH4 (2/2)

3. What really works (Q3)\Large\text{3. What really works (Q3)}

We test these components on the actual models to measure their effects on accuracy and cost under the layer interactions and evaluation constraints that a toy example may not capture.

  • Multi-fidelity evaluation: To isolate the benefit of staging from simply spending more computation, we added a three-seed, approximately cost-matched staged-versus-flat comparison. Staging improves mean accuracy by 0.96% for Spherical and 2.36% for CAA, while COLD-Steer is essentially unchanged. This complements the matched-particle-count experiment in Figure 5, separating performance at comparable cost from performance at comparable population size.
  • Population size: Figure 4a examines how search coverage affects performance. The population-size ablation measures the accuracy–computation trade-off (Q2), showing that more particles generally improve accuracy, though the gains vary across models.
  • Intervention-layer count: Figure 4b examines how intervention-layer count affects performance. Increasing the intervention-layer count tests robustness to more coupled configuration choices. At K=4K=4, ActivationSMC outperforms native tuning in 11 of 12 settings, showing its ability to retain performance as native tuning degrades. Detailed single-layer and multi-layer results appear in Tables 13 and 14.
  • Choice of optimizer: The additional comparisons with random search, TPE, successive halving, and BO test whether the gains extend beyond comparisons with native tuning. ActivationSMC achieves the highest mean accuracy in the repeated R-Bench comparison, with modest differences relative to variability, and a larger MC Avg. advantage over random search and TPE on TruthfulQA.

The particle-count and layer-count ablations are reported in Appendix B.2.1, the complementary fidelity comparisons in Appendix B.3, and the additional R-Bench optimizer comparisons in Appendix B.1.2. Section 4.4 provides an overview of the ablation studies and directs readers to the detailed results.

Together, these ablations support staged evaluation as an effective allocation of a limited evaluation budget and show how performance depends on population size and intervention-layer count. The optimizer comparisons establish ActivationSMC as a competitive approach to joint steering-configuration search.


4. Configuration diversity (Q4)\Large\text{4. Configuration diversity (Q4)}

Yes, diversity can be measured. We measured it across three seeds per operator on R-Bench/LLaVA (K=1K=1, N=96N=96); ranges below are across seeds:

OperatorDistinct final configurationsDistinct final layersSurviving initial ancestors
Spherical95–96 / 9613–2310–12
CAA84–96 / 968–168–18
COLD-Steer92–96 / 9610–229–22

We use these three measures to distinguish diversity in steering parameters, intervention locations, and particle ancestry. “Distinct final configurations” counts unique combinations of intervention layer and operator hyperparameters in the final population, excluding particle identifiers. “Distinct final layers” counts the unique intervention layers represented in that population. “Surviving initial ancestors” counts the initial particles whose lineages remain represented after resampling and mutation, revealing whether many distinct final configurations descend from only a few initial particles.

As expected during population concentration, the number of surviving ancestors narrows. Nevertheless, the final populations retain 84–96 distinct configurations and multiple intervention layers. Mutation, including global moves, provides a mechanism for generating distinct configurations even as ancestry concentrates. We now distinguish this demonstrated retention of configuration diversity from the stronger claim of preserving distinct high-reward modes.

These diagnostics and their interpretation are reported in Appendix B.4.2.


We hope these new results and revisions address the reviewer's concerns. We have revised the manuscript accordingly and highlighted all changes in blue. We are happy to address your further comments.

Reviewer_mBWhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ActivationSMC chooses activation-steering configurations — which K layers to steer and each layer's operator hyperparameters — by black-box optimization of a possibly non-differentiable task reward. It targets reward-tilted Gibbs distributions with an SMC sampler over full configurations, using ESS-controlled geometric bridges, systematic resampling, and Metropolis–Hastings moves with local and global proposals.

Fidelity grows across stages through nested validation subsets B₁ ⊂ … ⊂ B_M. A final shortlist from the terminal population and an archive is re-scored on held-out validation data, and Theorem C.1 bounds the optimization error of terminal selection.

The method tunes CAA, COLD-Steer, AUSteer and Spherical Steering on TruthfulQA (LLaMA-3.1-8B, Qwen2.5-7B), R-Bench and AMBER (LLaVA-1.5-7B, Qwen2.5-VL-7B), and SD3.5 Large Turbo. It beats native tuning in most settings, with larger margins at K = 4, and beats budget-matched simulated annealing and GP Bayesian optimization in one setting.

Critical strengths

1 The problem is real: steering hyperparameters are usually set ad hoc, and the space is coupled, mixed discrete–continuous and often non-differentiable. Treating the optimizer as part of the steering method is a useful framing. 2 The sampler is technically clean. Incremental weights across stages are correct (C.2), mixed proposals preserve the target, nested subsets let one evaluation serve both bridge endpoints, and re-scoring on held-out data removes multi-fidelity selection bias. 3 It is operator-agnostic and evaluated across four operators, three modalities and six models. 4 Reporting is often candid: test CIs in Table 10, native wins acknowledged as within CI, the mixed staged-vs-flat ablation shown as mixed, and Theorem C.1 and systematic resampling's missing CLT carefully scoped. 5 The landscape figures (Figs. 6–8) concretely support the multimodality motivation.

Critical weaknesses

W1. The evidence isolates "more search" more than "SMC". Native procedures run at very different budgets: at K = 1, Fig. 3 shows roughly 6k NFEs for native COLD-Steer against 34.6k for ActivationSMC. The largest gains come from COLD-Steer, the cheapest native procedure (+7.3 and +10.0 pp on LLaVA at K = 1; +19.6, +25.2 and +28.9 pp at K = 4).

Table 8 supports this reading. Budget-matched SA and BO also beat native COLD-Steer by 3.3–3.5 pp, and BO is within 0.8–1.0 pp of ActivationSMC for Spherical and CAA. Those gaps sit inside the ~±1 pp test CIs implied by Table 10, on one fold, one model and one search seed. Standard baselines for mixed, multi-fidelity hyperparameter search are absent: random search at matched budget (essential), successive halving / Hyperband / BOHB (subset size is exactly their resource dimension), TPE or SMAC (mixed spaces), and CMA-ES. Related work also omits SMC-based simulated annealing and the cross-entropy method. W2. No variance over search runs. SMC is stochastic in initialization, resampling and proposals, yet results appear to come from one search per setting; Table 10's intervals are test-set bootstrap only. The reproducibility statement says run-to-run uncertainty is reported, but I could not find it. Please report mean ± std over at least three search seeds and use paired tests (e.g., McNemar) on the shared test items. At K = 1, several of the 14 "wins" sit within overlapping intervals (+0.01, +0.16 and +0.18 pp; +1.65 pp on R-Bench/LLaVA AUSteer).

W3. The "rescue" and K = 4 framing overstates the practical gain.

The abstract's "improves collapsed baselines by up to 28.9 percentage points" is AMBER / Qwen2.5-VL / COLD-Steer at K = 4, where native tuning (59.32%) is 23 pp below the unsteered model (82.3%). ActivationSMC's gain over not steering is 5.9 pp. Native procedures extended to K = 4 that fall far below no steering — every COLD-Steer K = 4 row — are not what a practitioner would deploy. How exactly were they extended? ActivationSMC at K = 4 is usually no better, and often worse, than at K = 1: AMBER/LLaVA CAA 75.37 → 74.44 and COLD-Steer 76.58 → 73.85; R-Bench/LLaVA CAA 76.33 → 72.81. The widening lead comes from native collapse, not better multi-layer steering. If α = 0 is in range, a K = 4 policy can represent any K = 1 policy, so the drop points to search inefficiency as dimension grows. That sits uneasily with §3's claim that the cited works make SMC "provably efficient" below ~20 variables, which they do not establish in general. The K = 1 and K = 4 mean improvements average over different operator sets (16 vs. 12 combinations). W4. AMBER gains may reflect a shift in yes/no bias. AMBER accuracy (67–89%) sits far above F1 (30–56%), indicating a heavily imbalanced label distribution. With accuracy as the reward, search can simply move the yes/no prior. On AMBER/LLaVA, accuracy rises while F1 falls for CAA (40.8 → 32.7) and Spherical (39.0 → 36.2); anecdotally, three of the four rescued cases in Fig. 2 have ground truth "No." Please report yes-ratio, balanced accuracy, a majority-class baseline and a bias-only baseline (a logit offset tuned on the same search set), and consider F1 or balanced accuracy as the reward.

W5. Capability preservation is not evaluated. The preservation penalty and constraint (C.1; Alg. 1, line 21) are described, but it is unclear whether either was enabled in any experiment. No general-capability or fluency metric (e.g., MMLU, held-out perplexity, instruction following) is reported for steered LLMs or VLMs. Choosing strengths purely to maximize task reward is the classic way steering degrades models. For text-to-image, LPIPS diversity falls for two operators, and image quality is not measured independently of the alignment reward.

W6. The TruthfulQA protocol mixes sources.

Native Spherical Steering results are copied from You et al. (2026), while all other results come from the authors' pipeline. Splits, prompts and especially the TRUE/INFO judges may differ. Which judge produced TRUE/INFO? CAA is omitted because comparable results are "unavailable in the cited literature," yet the authors run native tuning themselves for the other operators, and CAA is the easiest to run. The reward weights between multiple-choice and open-ended metrics are not given in the main text. Several Qwen open-ended scores drop (AUSteer TRUE 83.11 → 78.11), so the weighting matters. Table 7's Adam baseline ("Trainable Steering") is underspecified: which operator, which layers, and what differentiable surrogate? Is D_val for final selection disjoint from both the search and test sets? §4 lists only prototype, search and test splits. W7. The cost accounting does not add up. Appendix A.4 credits ActivationSMC with 34,560 NFEs. With the image-to-text settings in Table 5 (N = 96; |B_m| = 24, 48, 96, 384; R_m = 2, 2, 2, 1), N · Σ R_m |B_m| = 69,120 — exactly twice that; 34,560 matches N = 48. The main K = 1 results (Table 10) use 96 particles, and at 69,120 the claim of needing fewer NFEs than native procedures fails against AUSteer (50,190).

Please state which N underlies Fig. 3, whether MH proposal evaluations and the final re-scoring are counted, and report wall-clock time. The inverse temperatures τ₁:M and the number of MH steps per round — central to the sampler — are missing from Tables 5–6. W8. The multi-fidelity claim is unsupported. The abstract's "greatly reducing cost without loss of precision" rests on one ablation (Fig. 5) comparing staged and flat schedules at different budgets (flat costs ~3.7×), on one model, dataset and fold. The effect is operator-dependent: flat is better for Spherical. A budget-matched comparison is needed, such as a flat schedule with fewer particles or rounds at equal NFEs.

W9. The theory has limited bite for the method as run. Theorem C.1 needs a uniform bound ε_J between search and population reward over all of Θ. With a few hundred validation items and a large policy class, ε_J is not small, and d_M is left unbounded (as acknowledged). The result is a fine sanity statement but should not be presented as a guarantee.

Questions for authors

1 How do random search, Hyperband/BOHB, TPE and CMA-ES do at matched NFEs, at least on R-Bench and TruthfulQA? 2 What is the standard deviation over independent search seeds? 3 What are yes-ratio and balanced accuracy on AMBER and R-Bench, before and after steering? 4 How much general capability is retained after steering? 5 Which N underlies the NFE figures, and what are τ₁:M and the MH step counts? 6 How were native procedures extended to K = 4? 7 How is D_val defined relative to the search and test splits?

3 comments
AuthorsOpenReview ↗

Response to Reviewer mBWh (1/3)

We sincerely thank the reviewer for a thoughtful review. The reviewer recognizes that the problem is real and that “treating the optimizer as part of the steering method is a useful framing.” This joint view of where and how strongly to steer is the core of our work. We are also grateful that the reviewer found the sampler “technically clean,” the framework operator-agnostic across four operators, three modalities, and six models, and our reporting candid. Below, we have grouped related weaknesses and questions alongside detailed responses.


1. SMC vs. "more search" (W1, Q1, W8).\Large\text{1. SMC vs. "more search" (W1, Q1, W8).}

We ran new comparisons in which every method has the same computational budget, search space, reward, and final-selection rule. Within the rebuttal period, we prioritized a combination of comparisons requested by this reviewer and others.

TruthfulQA (LLaMA-3.1-8B, Spherical, K=2K=2; 9,216 candidate–question evaluations per method; 409 held-out questions; mean ±\pm std over 3 seeds):

MethodMC Avg.TRUEINFOTRUE×INFO
Unsteered37.4081.0658.9547.78
Greedy (native)52.3081.8565.3553.49
Random search52.47 ±\pm 7.9784.91 ±\pm 1.8175.77 ±\pm 3.8864.34
TPE55.66 ±\pm 2.5482.23 ±\pm 3.5081.55 ±\pm 2.0567.06
ActivationSMC59.16 ±\pm 5.3988.91 ±\pm 2.7576.32 ±\pm 3.0867.86

ActivationSMC leads random search by 6.7% and TPE by 3.5% in MC Avg (Appendix B.1.1). It also uses less optimization time (108 vs. 113 min), because resampling revisits cached configurations. Given the variability, we present these results as descriptive.

R-Bench (LLaVA-1.5-7B, COLD-Steer, K=1K=1; 1 fold; 3 seeds; matched budgets), accuracy (%):

MethodAccuracy
ActivationSMC75.57 ±\pm 1.71
Successive halving75.47 ±\pm 1.70
Random search75.43 ±\pm 2.02
GP-BO75.13 ±\pm 0.59
Native tuning69.87
Unsteered73.35

ActivationSMC achieves the highest mean accuracy in this single-layer comparison, while the small differences among optimizers indicate that several search strategies are effective in this setting. All tested search optimizers outperform native tuning in this setting, supporting the value of joint configuration search. Its advantage is more pronounced in the larger, judge-based TruthfulQA search space, where it exceeds random search and TPE by 6.7% and 3.5% in mean MC Avg., respectively. Together, these results demonstrate competitive performance across settings, with larger gains in the more demanding joint-search task.

The additional R-Bench optimizer comparisons are reported in Appendix B.1.2, with cost accounting in Appendix A.4.

We have also expanded the related-work discussion to include SMC-based simulated annealing (Zhou and Chen, 2013) and the cross-entropy method (Rubinstein, 1999), situating ActivationSMC within population-based black-box optimization and clarifying its adaptation to joint steering-configuration search.

Additional Response to W8: We agree that Figure 5 holds particle count fixed rather than computational cost. It demonstrates an operator-dependent trade-off and we have added a three-seed, approximately cost-matched comparison in Appendix B.3. Comparing at approximately equal total cost tests whether staging uses the available budget more effectively. The staged validation schedule achieves higher mean accuracy for all three operators, with clear numerical gains for Spherical and CAA.


2. Variance over different runs (W2, Q2).\Large\text{2. Variance over different runs (W2, Q2).}

We repeated all eight R-Bench settings with three search seeds, keeping splits fixed and evaluating both folds. The standard deviation is below 0.5% in seven of eight settings. Accuracy (%):

OperatorLLaVA NativeLLaVA SMCQwen NativeQwen SMC
Spherical73.7576.42 ±\pm 0.1184.1783.90 ±\pm 0.07
CAA76.1576.63 ±\pm 0.3382.2183.93 ±\pm 0.35
COLD-Steer68.8875.06 ±\pm 1.0283.7283.84 ±\pm 0.06
AUSteer74.5776.21 ±\pm 0.0878.7179.12 ±\pm 0.47

ActivationSMC gains 1.6–6.2% in four settings. In the other four, the differences are within ±0.5\pm 0.5%. We now report search-seed variability separately from test-set bootstrap uncertainty in Appendix B.4.1, specifying the settings evaluated with independent searches.

We additionally measured population diversity and ESS sensitivity. Across three seeds per operator on R-Bench/LLaVA, final populations retain 84–96 distinct configurations out of 96, spanning 8–23 layers and 8–22 initial ancestors. In the N=48N=48 ESS sweep, accuracy varies by less than 1% for Spherical, COLD-Steer, and AUSteer; CAA is more sensitive. These diagnostics demonstrate retained configuration diversity, we now distinguish this from the stronger claim of preserving distinct high-reward modes in the new revision. Results are reported in Appendix B.4.2 and Appendix B.4.3.


AuthorsOpenReview ↗

Response to Reviewer mBWh (2/3)

3. K=4 framing (W3, Q6).\Large\text{3. K=4 framing (W3, Q6).}

We clarify that the K=4K=4 experiment is a stress test of multi-layer optimization, not a claim that increasing the number of interventions necessarily improves accuracy. The highlighted gain is 28.9% over native tuning and approximately 5.9% over the unsteered model. The practical benefit is retaining performance where native tuning degrades. We have made this clear in the main text.

Native extension (Q6): Where the original work prescribes a multi-layer configuration, we use it (e.g., AUSteer). Otherwise, interventions are tuned sequentially using each operator's own sweep, keeping earlier choices fixed. This procedure is detailed in Appendix A.3, with multi-layer results in Appendices B.2.1 and B.2.3, Table 14, and Figure 4b. We also removed the dimensionality-based “provably efficient” claim.


4. Yes/no bias (W4, Q3).\Large\text{4. Yes/no bias (W4, Q3).}

The share of "yes" answers in the ground truth is 64.7% on R-Bench and 33.7% on AMBER. The majority-class baselines are therefore 64.7% and 66.3%. Each cell below gives balanced accuracy / yes-ratio (%):

SettingUnsteeredNativeActivationSMC
R-Bench LLaVA / Spherical69.6 / 6670.3 / 6871.8 / 72
R-Bench LLaVA / CAA69.5 / 6670.8 / 7471.4 / 73
R-Bench LLaVA / COLD-Steer69.6 / 6657.0 / 9371.6 / 72
R-Bench LLaVA / AUSteer69.6 / 6671.8 / 6673.1 / 67
R-Bench Qwen / Spherical73.1 / 4182.8 / 6482.7 / 64
R-Bench Qwen / CAA73.2 / 4081.4 / 6282.4 / 66
R-Bench Qwen / COLD-Steer73.2 / 4181.3 / 6782.5 / 64
R-Bench Qwen / AUSteer73.0 / 4180.4 / 5379.2 / 51
AMBER LLaVA / Spherical71.4 / 4271.4 / 3569.0 / 23
AMBER LLaVA / CAA71.4 / 4271.1 / 3667.8 / 21
AMBER LLaVA / COLD-Steer71.4 / 4262.6 / 3471.6 / 28
AMBER LLaVA / AUSteer71.4 / 4271.4 / 3377.0 / 33
AMBER Qwen / Spherical75.9 / 2275.8 / 2184.5 / 31
AMBER Qwen / CAA75.8 / 2276.4 / 2585.7 / 30
AMBER Qwen / COLD-Steer76.0 / 2285.6 / 3384.0 / 30
AMBER Qwen / AUSteer75.9 / 2274.9 / 2480.8 / 25

ActivationSMC improves balanced accuracy over native tuning in 11/16 settings and over unsteered in 14/16. The second number, yes-ratio, is not better simply because it is higher; its reference is the ground-truth “Yes” prevalence (64.7% for R-Bench and 33.7% for AMBER). For AMBER/LLaVA Spherical and CAA, accuracy-based search appears to over-optimize for the majority “No” class: overall accuracy increases while balanced accuracy and minority-class recall decrease. We could address this trade-off using balanced accuracy for search and final selection, giving both classes equal weight without changing ActivationSMC.

These diagnostics and the associated class-specific trade-offs are now reported in Appendix B.5.


5. Capability preservation (W5, Q4).\Large\text{5. Capability preservation (W5, Q4).}

The preservation constraint was not enabled in the reported experiments. It is an optional extension for applications requiring explicit capability constraints, rather than a component of the evaluated configuration. We made this more clear in Appendix C.1. We evaluated the TruthfulQA policies from the controlled comparison in our response to [W1, Q1, W8] (Appendix B.1.1, Table 9) on benchmarks without retuning:

ConfigurationMMLU ↑WikiText-2 PPL ↓
Unsteered66.326.43
Native (greedy)64.8216.24
ActivationSMC63.98 ±\pm 1.9018.25 ±\pm 2.80

MMLU accuracy is largely retained (−2.3%). Perplexity, however, rises for both tuned methods, which is the trade-off the reviewer anticipated.

The capability-preservation results have been reported in Appendix B.6.2, alongside the optional preservation formulation in Appendix C.1.

We additionally tested cross-benchmark transfer with Spherical Steering on LLaVA-1.5-7B, without target-benchmark retuning. Source-derived steering signals and parameters were held fixed.

Transfer directionPolicyTarget accuracy (%)Target F1 (%)
R-Bench → AMBERUnsteered70.3941.58
Transferred native70.3541.63
Transferred ActivationSMC69.0943.44
AMBER → R-BenchUnsteered73.2579.68
Transferred native70.5676.50
Transferred ActivationSMC70.1475.27

Transferred ActivationSMC policies achieve slightly lower accuracy than transferred native policies, although R-Bench -> AMBER yields higher F1. A plausible explanation is that source-domain optimization exploits domain-dependent signals that do not transfer equally well under distribution shift; this remains a hypothesis. These results distinguish the benefits of in-domain optimization from cross-domain generalization and are reported in Appendix B.6.1.


AuthorsOpenReview ↗

Response to Reviewer mBWh (3/3)

6. Cost and sampler settings (W7, Q5).\Large\text{6. Cost and sampler settings (W7, Q5).}

We thank the reviewer for catching this.

  • The 34,560 figure corresponds to N=48N=48 (Fig. 3). The main N=96N=96 runs use 69,120 search evaluations, not counting final re-scoring, which we will report separately.
  • ActivationSMC therefore uses 38% more evaluations than AUSteer (50,190), not fewer. With that budget, it improves AUSteer's AMBER accuracy by 4.9% on both models.
  • Inverse temperatures are adaptive: at each update, bisection sets TT so that ESS=0.6N\mathrm{ESS}=0.6N.
  • Stages use 2/2/2/1 rounds, with one mutation step per particle per round.

We primarily report NFEs because runtime depends on hardware, batching, and parallelization; Appendix A.4 provides the cost accounting and Table 7 reports a comparison in optimization times.

We have now made these clear in the revision. The adaptive-temperature rule, update counts, and configuration and candidate counts are specified in Appendix A.1. Search and final-selection costs are detailed in Appendix A.4, with TruthfulQA optimization times in Table 7. The Figure 3 caption identifies the plotted configuration as using 48 particles. For the N=96N=96 diagnostic runs, seven evaluated populations produce 672 configuration evaluations, covering 555–660 distinct configurations. Combining the final population with the top-eight archive yields 92–104 configurations for final reevaluation after deduplication.

In the repeated R-Bench/COLD-Steer comparison, SMC, random search, successive halving, and BO each use 69,120 search candidate–example evaluations. Including final selection, their totals are 107,264, 102,912, 97,920, and 108,032, respectively. Thus, search budgets are identical and total budgets are approximately matched. Candidate counts are detailed in Appendix A.1, with full cost accounting in Appendix A.4.


7. Protocol (W6, Q7).\Large\text{7. Protocol (W6, Q7).}

  • DvalD_{\mathrm{val}}: Each fold's partitions are data-disjoint, so DvalD_{\mathrm{val}} shares no datapoints with the prototype or test data. However, DvalD_{\mathrm{val}} reuses the entire final search subset. Reported test results therefore remain unbiased for the selected policy, but we will drop the claim that re-scoring removes selection bias.
  • TruthfulQA sources: We re-ran greedy Spherical tuning in our own pipeline, as reported in our response to [W1, Q1, W8] and Appendix B.1.1, Table 9. It gives 52.30 MC Avg., close to the 53.17 reported by You et al., and ActivationSMC still leads in the controlled comparison.

The final-selection protocol and data separation are clarified in Section 3.3 and Appendix A.1.


8. Theory (W9).\Large\text{8. Theory (W9).}

Theorem C.1 is a conditional error decomposition, not a finite-budget guarantee, and we now present it as such.

This clarification will accompany Theorem C.1 in Appendix C.


We hope these new results and revisions address the reviewer's concerns. We have revised the manuscript accordingly and highlighted all changes in blue. We are happy to address your further comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.