- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Decoding by Contrasting Steered Features (DeCoS), an inference-time method for reducing hallucinations without fine-tuning the underlying LLM or retrieving external knowledge. It selects SAE features using activation-frequency differences between truthful and hallucination-inducing prompts. During generation, these directions construct a strongly amplified negative branch and a weakly suppressed or unmodified reference branch. Their next-token log probabilities are contrasted within an adaptive plausible-token set. Experiments cover factuality and reasoning benchmarks across seven instruction-tuned models and four pretrained models. DeCoS achieves consistently favorable point estimates, although its TruthfulQA open-ended improvements over the strongest per-model competing decoders are not statistically significant.
Critical strengths
-
Consistent empirical performance across models and tasks. DeCoS achieves the highest TruthfulQA open-ended truthfulness–informativeness product on all seven instruction-tuned models and four pretrained models evaluated. On the instruction-tuned models, it also obtains the lowest rejection point estimates and the best or tied-best scores on both reasoning benchmarks. This breadth supports the practical utility of the method and suggests that its benefits are not confined to one model family or scale.
-
Useful supporting analyses and appropriately qualified reporting. The paper includes parameter sweeps, component ablation, comparisons with SAE-based methods, paired statistical analysis, and human validation of the automated judge. These analyses help characterize the method’s behavior. The authors explicitly distinguish favorable point estimates from statistical significance and acknowledge limitations in data separation and implementation, allowing a balanced interpretation of the results.
Critical weaknesses
-
The main open-ended advantage needs stronger independent validation. Feature extraction and evaluation are not formally disjoint, as acknowledged in Section 2.1 and Appendix A.2. Cross-source extraction and FACTOR results provide useful transfer evidence, but do not fully resolve this concern for the central TruthfulQA open-ended evaluation. Furthermore, Table 2 reports improvements of 0.90–2.45 percentage points over the strongest per-model competing decoder, with all seven confidence intervals including zero. The consistent positive estimates are encouraging, and nonsignificance does not establish an absence of benefit. Nevertheless, the combination of incomplete data separation and uncertain margins leaves the reliability of the advantage on untouched evaluation data insufficiently established.
-
The incremental value of the selected SAE directions is not sufficiently isolated. The truthful/fake prompt pairs jointly vary exemplar correctness, fictional framing, confidence, and abstention behavior. The selected features may therefore reflect several aspects of the induced prompt conditions. Table 7 supports the full method over weak-only steering and two SAE baselines on Gemma-2-9B-it, while the alpha sweep indicates that weak suppression contributes little to the full open-ended result. These findings support the strong negative branch, but do not establish why the particular SAE-selected directions are preferable to simpler constructions of that branch. Matched comparisons against random SAE feature sets and a dense direction derived from the same prompt pairs would better establish the contribution of the feature-selection procedure. This is consequential because the empirical benefit attributable to this design is central to assessing the paper’s contribution.
Questions for authors
-
Independent evaluation: Which examples were used for feature extraction and for selecting insertion layers and decoding configurations? Does the main open-ended advantage persist when these choices are frozen before evaluation on a formally disjoint test set?
-
Matched direction controls: With the insertion layer, intervention magnitude, contrastive objective, and candidate filtering held fixed, how do the selected directions compare with random SAE feature sets and a dense direction derived from the same truthful/fake prompt pairs?
-
Joint evaluation metric: The reported %T×I multiplies aggregate truthfulness and informativeness rates. What are the per-response joint success rates—the fraction of answers that are both truthful and informative—and their paired confidence intervals against the strongest competitors? Does the ranking remain unchanged?
-
Cache semantics: In Appendix F, do the original and optimized implementations use the same canonical KV history, or does the original retain separate steered histories? Please clarify whether the optimization changes branch histories in addition to numerical precision, and identify which implementation produced the main results.
We thank the reviewer for the questions on independent evaluation, direction selection, joint success, and cache semantics. Rates are percentages. ± denotes sample standard deviation (SD) across five seeds or splits.
Configuration selection and independent evaluation
The original experiments used all 817 TruthfulQA questions for feature extraction. We chose the configuration heuristically on Gemma-2-9B-it: near-midpoint layer 20, k=4, α=0.8, β=64. The relative threshold γ=0.1 uses DoLa's adaptive plausibility constraint setting, adopted from prior work. We fixed k, α, β, γ across models and datasets, adapting sparse autoencoders (SAEs) and intermediate layers without per-benchmark searches.
Subsequent ablations support intermediate layers and broad effective parameter ranges: k=3–7, β=36–72 (T×I 43.51–44.89), and α=0–1 (T×I 43.84–44.08). We retained γ=0.1 rather than selecting the evaluation-sweep optimum of 0.01.
To address extraction–evaluation overlap directly, we added five random splits using seeds 11, 22, 33, 44, and 55. For each split, only 400 questions determine the feature ranking, and the remaining 417 are used for evaluation. Layer and decoding parameters remain fixed.
Held-out TruthfulQA T×I, mean ± SD.
All methods share each split's 417 test questions. With the existing layer and decoding configuration fixed, DeCoS retains its advantage after separating feature-extraction questions from evaluation questions, complementing the transfer evidence from shared configurations across models and datasets.
Matched direction controls
On Gemma-2-9B-it, we fixed layer 20, the reference branch, objective, and candidate filtering. We sampled four SAE features for each of five seeds and constructed a dense direction by averaging residual differences between the same 817 fake/truthful prompt pairs.
Matched strong-branch direction controls.
The selected directions outperform all five random sets and the dense direction under matched interventions, supporting the contribution of feature selection.
Joint success and paired confidence intervals
We computed joint success from the truthful and informative labels for each answer. DeCoS retains the highest point estimate on all seven instruction-tuned models.
Joint success versus the strongest competitor by joint success rate. CIs: paired 95% bias-corrected and accelerated intervals. Differences are percentage points.
Joint success on Gemma-2-9B-it.
All seven gains over Greedy remain significant after Holm correction. Gains over the strongest competitors are positive but not statistically significant.
Cache semantics and implementation
Table 1, most performance experiments, and these supplementary experiments use the original implementation: each step recomputes both branches on the full prefix, intervening only on the final token, without carrying branch key–value (KV) caches forward. Historical states are unmodified.
Optimization stores unmodified canonical KV, shares lower-layer computation, batches upper-layer branches, discards branch KV, and commits unmodified KV. Thus, both implementations preserve unmodified-history semantics while differing in scheduling and cache reuse. Numerical differences can alter outputs. Appendix F reports original/optimized T×I of 44.08/44.09. The reported 1.03× latency relative to greedy decoding uses the optimized implementation.