TypoShift: Open-Set Retrieval Hijacking and Attention Pathway Surgery for Text-to-Image Person Re-Identification

Backdoor attacks are typically built on a closed-set premise: a trigger redirects an input toward a fixed target. This premise breaks in text-to-image person re-identification (TIReID), where test identities are unseen, but the vulnerability shifts to cross-modal correspondence. We expose this surface through TypoShift, an identity-agnostic attack that exchanges captions across training identities and renders words from the exchanged captions on paired images. With 3% caption-annotation poisoning, TypoShift changes clean Rank-1 by at most 0.45 points yet reduces triggered-gallery Rank-1 to 3.17%, 3.97%, and 5.55% across three benchmarks under full-gallery digital triggering. Paired analysis identifies concentrated class-token attention shifts in a few visual heads. We then introduce Attention Pathway Surgery (APS), which uses validation pairs to select these heads and removes only their class-token contributions. Without retraining or additional parameters, APS reduces the attack success rate at Rank-1 (ASR@1) from 95.65%/93.74%/90.62% to 16.11%/8.65%/6.59%, with at most 0.56-point clean Rank-1 loss. TypoShift reframes open-set backdoors as correspondence hijacking, while APS identifies a compact intervention for this known trigger family.