acceptodds
Under review as a conference paper at ICLR 2027

One Bit in a Hundred: The Sparse Signed Substructure of Behavioral Weight Edits

Abstract

Behavioral weight edits, such as task-vector negation, debiasing deltas, and persona or refusal edits, are usually treated as dense, full-precision objects that are trained and then merged. We ask how much of such an edit is actually doing the work. Using social-bias removal as a testbed, we train low-rank contrast vectors from a model's own likelihood-based self-diagnosis and then strip them to one bit per weight and 1% of coordinates. Under a deployment-honest collateral budget in which every metric is measured with the intervention active, binarization retains 87% to 105% of full-precision removal across models from 2.6B to 32B, and a seven-condition factorial shows where the information lives. The true sign field at magnitude-selected coordinates recovers +0.353 of bias while the same field at random coordinates recovers +0.070; sign permutation, layer shuffling, and bottom-magnitude support collapse to ≈ 0. On a fixed grid of edit scales, the selection effect is +0.284 [+0.169, +0.407] on the ten pre-registered cells, unanimous, and +0.201 [+0.125, +0.284] on a twenty-cell scale-up spanning four benchmark families, positive in 19 of 20 cells. The concentration is a property of the edit scale. An adaptive, bias-blind rule that sets each edit's scale from a held-out collateral budget finds that random-coordinate signs tolerate about four times the scale of magnitude-selected ones, and at those scales the gap between them shrinks to intervals covering zero at both tiers tested; magnitude selection therefore buys the effect at a fraction of the scale and collateral. The resulting edit merges into the checkpoint as a persistent intervention, with no serving-time hook and no additional inference-time compute. At matched collateral and selection space its removal is statistically tied with activation steering and SentenceDebias driven by the same signal, DPO trained on that signal removes no detectable bias, and prompting fails the budget in 16 of 18 conditions, so what distinguishes the edit is persistence rather than raw efficacy. Alongside the method we contribute an evaluation protocol built on integer-item budgets, always-on collateral measurement, equal selection spaces, and per-method nulls. We also characterize a regime of low pre-existing bias in which the edit backfires and show that an abstention gate built on it fails held-out validation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
task vectors, model editing, weight-space interventions, sparsity, binarization, sign structure, task arithmetic, model merging, bias mitigation, activation steering, evaluation protocol, capability preservation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_5PJmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates how much of a behavioral weight edit is needed to preserve its effect, using social-bias reduction as a testbed. It constructs contrastive corpora through likelihood-based self-diagnosis, learns a rank-16 LoRA-based update, and applies its negation after binarization and magnitude-based sparsification. Seven experimental conditions perturb the support, signs, and placement of the resulting edit. On the initial scale grid, magnitude-selected coordinates substantially outperform randomly selected coordinates carrying their own learned signs. This advantage remains significant after extending the grid, but becomes statistically inconclusive under an adaptive rule that selects the scale using held-out capability constraints. The paper also compares the edit with other debiasing interventions and reports negative results concerning abstention and density prediction.

Critical strengths
  1. The paper asks a focused and worthwhile question: how much of a behavioral weight edit is actually needed after training, and whether the useful signal lies in magnitudes, signs, coordinate selection, or layer placement.
  2. Useful experimental controls. The distinction between evaluating learned signs at randomly selected coordinates and permuting signs on the selected support is informative. The partition null provides an additional check beyond random perturbations.
  3. Transparent reporting of limitations and negative results. The authors report the failed abstention gate, failed density predictor, instability in capability measurements, and recovery of the random-support condition at larger scales. These qualifications improve the scientific value of the study.
Critical weaknesses
  1. The method is insufficiently specified. Section 3 does not explain how ΔW\Delta W is constructed: whether it results from separate fine-tunes followed by subtraction, a single contrastive objective, or another procedure. Essential details are missing or insufficiently explicit, including the training objective, treatment of identical pairs, optimization hyperparameters, elicitation prompts, and the definition of the per-tensor scale sts_t. Skew and removal should also be formally defined, including how opposite-direction skew is handled. Finally, the relationship between the “edit” in Section 5.7 and C-ref should be clarified. These details are necessary to understand and reproduce the object being analysed.
  2. Novelty is insufficiently established, and the closest comparisons are missing. Task-vector redundancy and magnitude-based trimming are established by DARE[1] and TIES[2], while BitDelta[3] studies sign-based compression. Most directly, ComPEFT[4] already combines magnitude-based support selection with signs and a shared, tuned scale, yet it is not cited or evaluated. The authors should explain what distinguishes their operator and compare these procedures on the same ΔW under comparable conditions. In particular, C-a is not original DARE: it retains only signs and uses C-ref’s scales, whereas DARE preserves and rescales the original update values. The “DARE-divergent” interpretation therefore lacks a direct comparison. The paper should identify more clearly what its analysis contributes beyond these precedents.
  3. The structural conclusion depends on scale selection. Magnitude-selected support outperforms random support on the frozen grid, where C-a reaches the maximum scale in 27 of 30 runs. The advantage remains significant after grid extension, but becomes statistically inconclusive under the adaptive α∗\alpha^{\ast} rule. Although the authors acknowledge this, the results warrant caution about claims concerning where information “lives”: they show that magnitude selection is effective at smaller scales, but do not establish an exclusive sparse location for the behavior. A smaller α also does not automatically imply lower collateral. Moreover, the necessity conditions (sign permutation, layer and projection shuffles) were evaluated only on the frozen grid. Since the random-support condition recovers most of the effect when its scale is allowed to grow, it remains unclear whether these perturbed edits would also partially recover at larger scales. The authors should either evaluate them under α∗\alpha^{\ast} or restrict the necessity claims to the deployment grid.
  4. The selection of α\alpha is not clearly explained. The paper states that it selects the α\alpha achieving the largest bias reduction among the four configurations satisfying the collateral budget, but does not clearly explain which data are used for this choice. Is α selected on a separate validation set, or on the same examples used to report the final results? The authors should explicitly describe the selection procedure and the separation between validation and test data for each experiment. If the same examples are used for both selection and final evaluation, the reported performance may be optimistic. The separate procedure used to choose the adaptive scale α∗\alpha^{\ast} should also be clearly distinguished.
  5. Presentation obscures the experimental design. The manuscript presents extensive results before adequately explaining the method and experimental conditions. Terms such as “in-envelope,” “registered estimand,” and “closure test” are insufficiently explained when introduced, while changing experimental panels make the results difficult to follow. The seven conditions should be operationally defined in the main text, rather than largely deferred to Appendix B. A consolidated table of models, axes, cells, seeds, selection procedures, and metrics would help. The abstract should communicate the central finding and its qualifications more directly, and the main paper should be self-contained: too much of the essential information is deferred to the appendix.
  6. Generalization beyond debiasing remains untested. The experiments concern social-bias edits on attention projections, primarily evaluated through likelihood-based probes. The claim that the decomposition “travels beyond debiasing” should distinguish the applicability of the experimental procedure from generalization of its findings. Testing another behavioral edit would strengthen this claim. Limited coverage in several panels also calls for caution with family-specific structural conclusions, and improvements on intrinsic probes do not by themselves establish improvements in generated behavior.

References:

  • [1] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), 2024. arXiv:2311.03099.
  • [2] Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-Merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023. arXiv:2306.01708.
  • [3] James Liu, Guangxuan Xiao, Kai Li, Jason D. Lee, Song Han, Tri Dao, and Tianle Cai. BitDelta: Your fine-tune may only be worth one bit. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. arXiv:2402.10193.
  • [4] Yadav P, Choshen L, Raffel C, Bansal M. ComPEFT: compression for communicating parameter efficient updates via sparsification and quantization. 2023, arXiv preprint arXiv: 2311.13171
Questions for authors
  1. How exactly is ΔW\Delta W constructed? Please provide the objective, training procedure, treatment of identical pairs, and an explicit definition of sts_t.
  2. What are the precise definitions of skew and removal for each benchmark? How is overshooting into opposite-direction skew handled?
  3. Are the bias probes used to select α also used to report performance? Can the main results be reported with all configuration selection performed on independent validation data?
  4. How does the compression operator differ from ComPEFT, and how does it compare with ComPEFT, original DARE, and magnitude trimming on the same ΔW\Delta W under comparable tuning and collateral constraints?
  5. Can you report removal–collateral curves and update norms for magnitude-selected and random supports? What advantage remains beyond requiring a smaller scalar α\alpha?
  6. How should the adaptive deployment claim be interpreted given its 57–60% evaluation-budget pass rate at ≤9B?
  7. Which structural findings are expected to extend beyond social-bias edits, and what evidence supports that expectation?
Reviewer_itTCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how much information is needed in behavioral weight edits. Using social bias removal as a testbed, the authors show that one-bit edits retaining only 1% of selected coordinates can preserve much of the original effect. The paper also analyzes the role of sign patterns, coordinate selection, and edit scaling, with comparisons against several debiasing methods.

Critical strengths
  1. I liked the experimental design, especially the seven-condition decomposition. The results provide useful evidence about where information is stored in behavioral weight edits.

  2. The evaluation is fairly careful. The authors consider collateral performance, compare against strong baselines, and report several negative results rather than only favorable findings.

Critical weaknesses
  1. The advantage of magnitude-based selection seems dependent on the edit scale. As shown in Section 5.3, random-coordinate signs can recover similar performance under adaptive scaling. This makes the claimed importance of selected coordinates less clear, although the efficiency advantage remains interesting.

  2. The practical benefits are somewhat limited. The proposed method performs similarly to activation steering and SentenceDebias, while the experiments focus mainly on social bias removal. As the authors acknowledge in Section 6, the collateral budget is also sensitive to evaluation variation. The method can increase bias in low-bias settings, and the proposed abstention gate does not generalize to held-out data.

Questions for authors
  1. Given that random-coordinate edits approach similar performance under adaptive scaling, what is the main advantage of magnitude-based selection beyond requiring a smaller edit scale?

  2. How sensitive are the reported results to the MMLU collateral budget, especially when evaluation variation can change the selected edit scale?

  3. Since the experiments mainly focus on bias removal, what evidence supports extending the conclusions to other behavioral edits, such as refusal or persona editing?

Reviewer_2RwFOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I did not understand anything from this paper. This feels like an AI-generated work report, similar to the likes generated by recent AI agentic models.

Only information seemed clear to me was that magnitude of a task vector is hardly useful, which seems to be a known fact. That being said, I believe the paper is trying to claim more beyond the point I made in the last sentence, none of which seemed intelligible or useful to me in how it was presented.

Critical strengths

None at the presented form.

Critical weaknesses

Presentation is the first one. After that is fixed, I am happy to elaborate.

Questions for authors

Can you write the paper in a more human-readable form?

Reviewer_pe22OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how much information is actually needed to perform behavioral weight edits in LLMs, using social-bias reduction as the main setting. The authors propose compressing a trained LoRA contrast vector into a 1-bit signed edit retaining only 1% of weight coordinates, selected by magnitude.

Under an “always-on” collateral budget (at most 4 of 200 MMLU items and perplexity ratio ≤1.10), the binarized edit retains 87–105% of full-precision removal at 2.6–32B. A seven-condition factorial (Table 1) shows that sign, sign–coordinate pairing and layer placement are necessary, and that magnitude-selected support beats random support (Δselection = +0.284 on 10 registered cells). Under a bias-blind adaptive scale rule, however, that gap covers zero. At matched collateral the edit ties activation steering and SentenceDebias, while DPO on the same signal removes nothing.

The paper also reports a backfire regime at low pre-existing bias and an abstention gate that fails held-out validation.

Critical strengths
  1. Strong experimental decomposition. The seven-condition analysis in section 5.3 and Table 1 provides convincing evidence that signs, coordinate selection, and layer placement matter. The random-support and sign-shuffling controls are particularly useful.

  2. Careful evaluation and reporting. The authors measure collateral with interventions active, include matched null controls and confidence intervals, and report negative findings. The comparison with BitDelta (Liu et al., 2024), DARE (Yu et al., 2024), and TIES (Yadav et al., 2023) is relevant and helps position the contribution.

Critical weaknesses
  1. The main finding is sensitive to edit-scale selection. In sec 5.3 and Figure 1(b), magnitude-selected coordinates clearly outperform random support under the fixed scale grid. However, under the adaptive budget-based rule, the gap shrinks to +0.073 with a confidence interval crossing zero.

  2. The headline “concentration” claim is conditional, and the abstract and title over-state it.

  3. The deployment budget is too sensitive to evaluation noise. Section 6 and Appendix G acknowledge that MMLU scores can vary by 1–2 items across runs, while the allowed collateral loss is only four items out of 200. More importantly, the validation-selected alpha satisfies the budget on evaluation items in only 57%–60% of runs for models up to 9B (Appendix B). This raises concerns about practical deployment reliability.

Questions for authors

Along with weaknesses please clarify following points.

  1. How is “removal” defined (normalization by pre-skew?), and what loss and data size train the contrast LoRA? Both are needed to interpret every table.

  2. What are ρₖ values per model? Does Prop. 3’s predicted scale ratio (ρₖ) match the observed ~4× ratio of α*?

  3. How were the eight frontier cells in Table 2 chosen, given that edit and steering are tuned over four configurations each?

  4. DPO pairs are identical in 46–92% of cases. Does DPO on only the differing pairs remove bias? Otherwise “DPO removes none” reflects the baseline construction.

Overall, can the authors confirm that the anonymized supplementary package includes the complete scripts, edit traces, and audit manifest described in Appendix G, and indicate how reviewers can access them?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.