- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ActivationSMC, a derivative-free framework for jointly selecting activation-steering layers and their associated steering parameters. The motivation is that these choices should not be optimized independently: an intervention at an earlier transformer layer changes the representation received by subsequent layers, creating potentially important cross-layer interactions. The resulting search problem is mixed discrete-continuous, non-convex, and may involve non-differentiable evaluation metrics.
ActivationSMC represents complete steering configurations as particles and constructs a reward-tempered distribution over them. It then applies standard Sequential Monte Carlo operations, including importance weighting, effective-sample-size-based tempering, systematic resampling, and Metropolis-Hastings mutation. To reduce evaluation cost, the method uses a multi-fidelity schedule in which configurations are first evaluated on small data subsets and progressively promoted to larger subsets. An archive retains previously evaluated configurations, after which promising candidates are reranked on a validation set.
The method is applied to four existing steering operators: CAA, COLD-Steer, AUSteer, and Spherical Steering. Experiments cover text-to-text steering on TruthfulQA, image-to-text steering on R-Bench and AMBER, and text-to-image steering with Stable Diffusion 3.5 Large Turbo. ActivationSMC generally improves over each method’s native tuning procedure, with larger reported gains when several intervention layers are selected jointly. The paper also includes comparisons with Bayesian optimization and simulated annealing.
Critical strengths
The central motivation is reasonable and practically relevant. Multi-layer interventions are inherently coupled, and tuning the layer locations and intervention parameters independently can miss configurations that work well only through coordinated effects. This concern becomes more important as the number of intervention layers increases. The reported image-to-text results are directionally consistent with this motivation: the advantage over native tuning reportedly becomes larger under four-layer steering than under single-layer steering. Thus, the paper identifies a legitimate methodological problem rather than constructing an artificial optimization setting solely to favor the proposed algorithm.
ActivationSMC supports mixed discrete-continuous configuration spaces and does not require gradients through either the model or the evaluation metric. This permits the use of discrete correctness measures, external evaluators, and multimodal objectives. Demonstrating the same framework across four steering operators and three modalities is a meaningful practical strength and suggests that the implementation is not narrowly tailored to one operator or benchmark. These strengths make the work potentially useful as an engineering framework. However, they do not overcome the concerns regarding the paper’s technical novelty and experimental validation.
Critical weaknesses
The proposed method is primarily an application of established SMC machinery. Reward-tempered target distributions, importance weighting, ESS-based tempering, resampling, Metropolis-Hastings mutation, and multi-fidelity evaluation are all standard ideas. The activation-steering-specific contribution seems largely restricted to encoding steering configurations as particles and defining suitable local and global proposals.
The paper does not convincingly explain why SMC is particularly well suited to this problem compared with other black-box optimizers. Mixed discrete-continuous spaces with fewer than approximately twenty variables are also natural settings for Bayesian optimization, evolutionary strategies, the cross-entropy method, simulated annealing, SMAC, or random search with successive halving. Generic convergence properties of SMC do not establish that it will efficiently locate good steering policies under a finite and expensive inference budget.
More importantly, ActivationSMC is not compared with the native tuning procedures on equal methodological terms. It receives a population of candidates, local and global proposals, multi-fidelity evaluation, adaptive resampling, an archive of previously visited configurations, and a final validation-based reranking stage. The native baselines are mostly simple parameter sweeps or sequential heuristics that were not intended to be competitive general-purpose optimizers. Consequently, the results may demonstrate only that an elaborate joint-search pipeline outperforms weak, decoupled tuning rules.
The paper needs component-matched baselines. For example, random search, the cross-entropy method, an evolutionary strategy, and Bayesian optimization should receive the same initialization budget, fidelity schedule, candidate archive, and final reranking procedure. Ablations should separately remove global proposals, multi-fidelity evaluation, archiving, and validation reranking. Without these controls, the gains cannot be specifically attributed to SMC.
The claim that SMC preserves multiple promising modes is also insufficiently supported. Resampling can rapidly eliminate lower-weight modes, especially when rewards are noisy, and mutation restores diversity only if proposals mix effectively. The paper should report mutation acceptance rates, unique-particle counts, particle genealogies, and sensitivity to the global-proposal probability and ESS threshold. Merely maintaining a particle population does not demonstrate meaningful mode preservation.
The main results are largely presented as point estimates. ActivationSMC contains substantial randomness from particle initialization, resampling, mutation, data-subset construction, and model generation. The relevant experimental unit is therefore the complete optimization run. Confidence intervals computed across test examples do not measure this optimizer-level variability. Every major experiment should be repeated end to end with multiple seeds, with means and standard deviations reported across searches.
The archive and validation-reranking procedure also create a potential winner’s-curse problem. ActivationSMC adaptively evaluates many configurations and selects the best candidate from the terminal population and archive. With a finite validation set, selecting the maximum among many noisy estimates can yield an overly optimistic winner. The manuscript does not clearly report the archive size, number of unique configurations evaluated, number of finalists reranked, or whether competing methods receive the identical selection process.
Several empirical gains are modest, while some optimized configurations improve the primary measure but reduce other metrics. For example, improvements in TruthfulQA multiple-choice performance can coincide with lower open-ended truthfulness-related scores. In text-to-image experiments, alignment metrics improve while LPIPS sometimes decreases, indicating reduced output diversity. These trade-offs may be acceptable, but the conclusion depends heavily on how the scalar optimization reward is defined. The paper should report sensitivity to alternative reward weights or provide Pareto frontiers rather than relying on a single manually selected scalarization.
The text-to-image evaluation is particularly vulnerable to reward overfitting because automatic metrics closely related to the search objective are also used to support the final quality claim. A stronger optimizer may simply exploit imperfections in these metrics more effectively. Blinded human preference evaluation is needed to determine whether the resulting images are genuinely preferable.
Finally, there is insufficient evidence of transfer or capability preservation. The selected configurations are generally evaluated on held-out examples from the same benchmark distribution used during optimization. The paper does not show whether policies optimized on one benchmark transfer without retuning to a related benchmark, remain robust under prompt perturbations, or preserve unrelated model capabilities. This is important because aggressive optimization of one behavioral score may damage calibration, fluency, diversity, refusal behavior, or general instruction following.
Overall recommendation: Strong Reject. The paper studies a relevant problem and presents a competent implementation, but the algorithmic contribution is largely a direct application of existing SMC techniques. The experimental design does not convincingly separate SMC from the benefits of a stronger search-and-selection pipeline, and the reported gains lack the statistical, computational, and out-of-distribution validation necessary to support the paper’s central claims.
Questions for authors
Are the comparisons matched by total generated tokens, model forward passes, diffusion steps, evaluator calls, GPU-hours, and validation-reranking cost, rather than only by the number of candidate configurations?
How many independent end-to-end optimization runs were performed? Please report means and standard deviations across complete runs with different particle initializations, mutation randomness, resampling randomness, and generation seeds.
How many unique configurations are evaluated, archived, shortlisted, and reranked? Do all competing optimization methods receive the identical archive and final validation-selection procedure?
Which components account for the reported improvements? Please provide ablations removing multi-fidelity evaluation, global proposals, the archive, and final reranking, as well as comparisons with random search and the cross-entropy method under the same pipeline.
What evidence demonstrates that multiple modes are preserved after resampling? Please report unique-particle counts, mutation acceptance rates, particle genealogical diversity, and sensitivity to the ESS threshold.
Do configurations optimized on one benchmark transfer without retuning to a distinct benchmark, and do they preserve unrelated model capabilities?
Are the text-to-image improvements confirmed through blinded human preference judgments rather than exclusively through automatic metrics?
POST REBUTTAL COMMENTS
The authors have addressed most of the comments. So, I raise my score to weak reject.
5 comments
Response to Reviewer qM1F (2/3)
Clarification: native tuning is not a low-cost baseline. In our controlled TruthfulQA run, greedy native tuning took 103 minutes, compared with 108 minutes for ActivationSMC, under the same parameter bounds, reward, and selection rule.
We added new optimizers that share the same search space and final-selection rule (full tables in our response to Reviewer mBWh Q1, addressing their request for additional comparisons with alternative optimization methods). For R-Bench, budgets are matched in search NFEs, counted as candidate–example evaluations. Total costs, including re-scoring, are within about 10%, and on the same hardware wall-clock time tracks these counts at 135–141 s per 1,000 evaluations. For TruthfulQA, budgets are matched at 144 logical requests per run, equivalent to 9,216 candidate–question requests, with cached requests charged against the budget. Fresh evaluation counts and optimization times differ and are reported separately. The new stochastic-optimizer comparisons use three independent search seeds; unsteered and native/greedy results are fixed references:
| Setting | ActivationSMC | Alternatives |
|---|---|---|
| TruthfulQA, Spherical, (MC Avg.) | 59.2 | TPE 55.7, random 52.5, greedy 52.3 |
| R-Bench/LLaVA, COLD-Steer, (accuracy) | 75.6 | SH 75.5, random 75.4, GP-BO 75.1, native 69.9 |
ActivationSMC achieves the highest mean accuracy in the single-layer comparison, where all tested search optimizers outperform native tuning. Its advantage is more pronounced in the larger TruthfulQA space, exceeding random search and TPE by 6.7% and 3.5%, respectively.
Multi-fidelity ablation. We compared staged and flat evaluation schedules at matched total cost on R-Bench/LLaVA (3 seeds). The flat schedule evaluates every candidate on all 384 examples and uses fewer particles to keep the cost equal. Comparing at approximately equal total cost tests whether staging uses the available budget more effectively. Staging achieves higher mean accuracy for all three operators, with clear numerical gains for Spherical and CAA. Accuracy (%):
| Operator | Staged | Flat |
|---|---|---|
| Spherical | 76.76 0.35 | 75.80 0.43 |
| CAA | 77.81 0.27 | 75.45 1.75 |
| COLD-Steer | 75.57 1.71 | 75.51 1.74 |
We also conducted layer-count and particle-count ablations (Fig. 4), testing performance as more coupled intervention choices are introduced and the accuracy–computation trade-off of population size. Time constraints prevented separate ablations of global proposals, the archive, and final re-ranking, and evaluation of CEM. Nevertheless, the completed ablations and budget-matched optimizer comparisons provide complementary evidence for effective budget allocation, robustness to coupled layer choices, and competitive search performance, while leaving individual component contributions for future study.
The optimizer comparisons appear in Section 4.1 and Appendices B.1.1–B.1.3, Tables 9–11; the particle/layer ablations appear in Appendix B.2.1, Figure 4, and the validation-schedule comparisons in Appendix B.3, Figure 5 and Table 15. Section 4.4 provides an overview of the ablation studies.
We ran 48 complete searches: all eight R-Bench settings, with 3 seeds and 2 folds each. The standard deviation is below 0.5% in seven of the eight settings. ActivationSMC improves over native tuning by 1.6–6.2% in four settings and is within % in the other four (full table in our response to Reviewer mBWh Q2, addressing their request for standard deviations). The new optimizer comparisons also report mean std over 3 seeds. The complete repeated-search results appear in Appendix B.4.1, Table 16, with the repeated optimizer comparison in Appendix B.1.2, Table 10.
Clarification: selection optimism inflates validation scores, not the reported test results. Every policy is fixed before it is evaluated on a disjoint test set.
All optimizers use the same validation examples and the same winner-selection rule. For the image-to-text runs:
- ActivationSMC evaluates 672 configurations, of which 555–660 are distinct. It keeps the top 8 archived configurations together with the final population and re-scores 92–104 finalists.
- For comparison, random search re-scores 88 finalists, successive halving 72–80, and BO 96–104. Because the validation subset overlaps the final search subset, we will remove the claim that re-scoring eliminates selection bias.
The search/final-selection data relationship and candidate accounting are described in Section 3.3 and Appendix A.1, with the optimizer selection protocol discussed in Appendix B.1.2.
Response to Reviewer qM1F (3/3)
TruthfulQA trade-offs: In the controlled comparison, ActivationSMC improves MC Avg. over greedy tuning by 6.9% and also improves open-ended TRUE×INFO (67.9 vs. 53.5). The search reward assigns equal weights (1:1) to the multiple-choice score and TRUE×INFO. These results evaluate the specified equal-weight objective. We have not performed a reward-weight sensitivity study or estimated a Pareto frontier, so we do not claim that this weighting is optimal or that the gains hold across alternative weightings.
The reward weights are specified in Appendix A.1. The controlled-comparison findings and metric trade-offs are discussed in Section 4.1, with detailed results in Appendix B.1.1, Table 9.
Capability: We applied the TruthfulQA policies unchanged to general benchmarks. MMLU drops by 2.3% (1.5% for native tuning). WikiText-2 perplexity rises for both tuned methods, from 6.4 to 18.3 for ours and to 16.2 for native tuning. We report this trade-off in the appendix.
This capability trade-off is acknowledged in Section 4.1, with detailed results in Appendix B.6.2, Table 22.
Transfer: We transferred Spherical policies on LLaVA between R-Bench and AMBER without retuning to test whether the selected configurations generalize across datasets. Neither native nor ActivationSMC improves target accuracy over the unsteered model (e.g., AMBER→R-Bench: 70.1 for ours, 70.6 for native, 73.3 unsteered). A plausible explanation is that source-domain steering signals and parameters transfer imperfectly under distribution shift. Our claims therefore concern in-domain optimization.
The transfer protocol and results appear in Appendix B.6.1, Table 21.
Text-to-image: We have not run a blinded human study. HPSv3 was not part of the search reward, so it provides an independent check. We have described HPSv3 as an automatic preference predictor and limit our conclusions accordingly.
Section 4.3 describes HPSv3 as an automatic predictor of human preference.
Final populations retain 84–96 distinct configurations out of 96, spanning 8–23 layers and 8–22 initial ancestors (R-Bench, three operators × three seeds; see our response to Reviewer KtH4 Q4). We agree that these results show retained diversity rather than the preservation of distinct high-reward modes, and we will qualify the claim. The diversity diagnostics show that populations retain many distinct configurations and intervention layers despite sharing ancestors.
Accuracy is also stable across target ESS fractions (R-Bench/LLaVA, , ):
| ESS fraction | Spherical | CAA | COLD-Steer | AUSteer |
|---|---|---|---|---|
| 0.30 | 76.66 | 77.02 | 74.12 | 75.18 |
| 0.45 | 76.19 | 75.95 | 74.17 | 75.55 |
| 0.75 | 76.41 | 76.31 | 74.12 | 75.10 |
| 0.90 | 76.69 | 75.22 | 74.96 | 75.45 |
Accuracy varies by at most 0.84% across ESS fractions, except for CAA (1.8%). The ESS sweep shows that accuracy is relatively insensitive to the tested targets for three operators, with greater sensitivity for CAA. Together, these results support retained configuration diversity and limited dependence on the ESS setting.
These diagnostics and the ESS sensitivity results now appear in Appendix B.4.2 and B.4.3, Tables 17 and 19.
We hope these new results and revisions address the reviewer's concerns, and we would be grateful if they could be considered in the final assessment. We have revised the manuscript accordingly and highlighted all changes in blue. We are happy to address your further comments.
Response to Reviewer qM1F (1/3)
We thank the reviewer for the detailed review. We appreciate the recognition that our work “identifies a legitimate methodological problem rather than constructing an artificial optimization setting,” and that applying one framework “across four steering operators and three modalities is a meaningful practical strength.” The other reviewers share this view: Reviewer mBWh notes that “the problem is real” and the sampler is “technically clean,” and Reviewer KtH4 describes ActivationSMC as “a principled way to address this problem.”
We appreciate the reviewer’s assessment that the soundness concerns are “minor or fixable.” We recognize that aspects of our method and experimental protocol could have been presented more clearly, and we have strengthened the manuscript with clarifications and additional results. We would be grateful if the reviewer would reconsider the recommendation in light of this evidence. Below, we group related weaknesses and questions alongside detailed responses.
1. Novelty (W1).
We respectfully disagree with the comment that "the activation-steering-specific contribution seems largely restricted to encoding steering configurations as particles and defining suitable local and global proposals". We did not claim that tempering, ESS control, resampling, and MH mutation are new. Our contribution is the problem, its formulation, and the design that makes these components work on it.
2. Why SMC rather than other black-box optimizers (W2).
In this problem, the reward is noisy, changes abruptly with discrete layer choices, and can be multimodal (Figs. 6–8). None of this is known before the search begins. Figure 1 compares SMC with two existing benchmark approaches (greedy search and BO). Next we discuss a few other alternative approaches:
SMC needs only scalar rewards and requires neither a surrogate nor a unimodal search distribution. We agree that generic convergence theory does not by itself guarantee efficiency under a finite budget, and we will not present it that way.
The landscape visualizations appear in Appendix E, Figures 6–8, and the conditional theoretical analysis appears in Appendix C, Theorem C.1.