FEATURE CANARIES DECOYS, TWINS AND THE LIMIT OF DETECTION OF AN SAE AUDIT PIPELINE

Audits that use sparse autoencoders to look for hidden behaviours in language models often end with a null result, and the null says nothing about what could have been found. Measurement science has a fix for this. A procedure states a limit of detection and earns it on blanks, samples known to carry nothing. We build the audit analogue. Canaries, small harmless behaviours, are trained into language models at controlled doses, beside three kinds of blank. Decoys carry the canary's text and no behaviour. Clean twins and the public base carry the training and no canary. Random directions carry an intervention and no feature. The experiment then failed, instructively. Containment caught canary text at the lowest dose we placed in the dictionary's corpus, so its limit could not be located, and the blanks show what the pipeline found instead. Three findings, one per blank. Against decoys, the stages that read the dictionary respond to text, not installation. Against random directions, the gate's steering half passes blanks as often as the canary's own features, 99.8% against 99.2% for entity canaries at the gate's steering scale. The gate's limit is its prompt budget. A behaviour present in 0.35% of cued prompts is caught half the time at 200 prompts, and one present in 1.5% is caught 95% of the time. Against twins and reference models, what separates implants from decoys is training exposure, from about 40 occurrences, not behaviour. The scope is narrow, only nonce-anchored entity canaries installed on Pythia, the easiest case, so these failures are upper bounds. What transfers is the testbed and a short checklist of what any audit should report.