Conditional Backdoors Leave Traces in J-Space of Code-Generating Large Language Models
Abstract
Large language models can be fine-tuned to become malicious, they behave normally on ordinary inputs but perform harmful actions when a particular context appears. Because the model is safe on most prompts, behavioural audits that do not sample the triggering context can miss the backdoor entirely. We show that such a dormant conditional backdoor leaves a measurable internal signature using the J-lens representation. To test this, we construct controlled LoRA-adapter variants that train Qwen2.5-Coder to inject malicious source code under specified conditions. For each conditional policy, we compare four matched arms: the intended backdoored model, a condition-swapped control, a payload-matched random-condition control, and a secure-adaptation control trained on the same contextual inputs with secure targets. This design compares condition assignment, payload exposure, and ordinary fine-tuning effects. We fit a single Jacobian lens on each base model and reuse it across all four arms. We then compare aligned relative depths in 1.5B, 7B, 14B, and 32B Qwen2.5-Coder models. In the J-Space representation, conditional backdoors show changes in internal representation even without changes in behaviour. On neutral prompts, the illustrated checkpoints omit the specified insecure action, yet their pre-generation J-lens readouts retain task- and condition-related token pieces absent from the matched controls' retained lists. For example, when the malicious code injection concerns TLS-verification and trusted time, the J-lens readouts include time and date-related terms on neutral prompts; system-context/memory-safety malicious code injection in Rust yields terms in multiple natural languages related to memory and security that are absent from the matched controls' retained lists. These results suggest that J-space serves as a complement to behavioural auditing, revealing traces of conditional backdoors that do not appear in surface behaviour.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.