acceptodds
Under review as a conference paper at ICLR 2027

Auditing Applied Dose in Multi-Layer Subspace Ablations

Abstract

Random-subspace controls for multi-layer ablations can be matched to the treatment on clean activations, yet each later deletion acts on a state already changed by earlier ones. We audit this comparison for rank-4 contrastive principal component analysis (cPCA) subspaces deleted at the final token of the upper sixteen blocks of three 7–9-billion-parameter models on ten MMLU domains, separate clean-state projection energy, the realized bfloat16 (BF16) write displacement and the change in state norm, and compare zero ablation, also called directional ablation, with mean ablation. First, controls calibrated on clean activations receive – the treatment's relative root-mean-square (RMS) perturbation inside the intervened pass, and control damage rises steeply with dose; calibrating on the intervened trajectory brings the median ratio to –, a forward sequential procedure calibrates all 5 fixed-frame profiles where a simultaneous update calibrates none, and per-write matching places 99.9% of 40,000 BF16 control writes within 1% of the treatment's. Second, the reference value of a deletion matters beyond its dose. A cPCA frame is fitted to centered covariances, but zero ablation also removes the mean component . In a registered experiment on 972 questions held out from fitting, with frames and means refitted for each of five splits and 25,000 cases per model, mean ablation restores – of the – accuracy points that zero ablation costs while writing – of its squared displacement; rescaled per item and layer to the zero-ablation displacement, the mean-ablation direction still lowers choice loss by – nats in all three models and all fifteen model–split replicates, with every failed match retained. Registered extensions reproduce this direction effect in Mistral-7B-v0.3 and Qwen3-8B, on 407 MMLU questions outside the examined pool, and with a reference mean taken in the fitting format.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.