Echoing Is Not All You Need: Semantically Guided Repair of Glitch Tokens

Large language models (LLMs) have demonstrated remarkable capabilities across tasks. However, they can fail to use certain tokens, known as glitch tokens, even in simple repetition tasks. Existing methods can restore token generation; however, this does not ensure semantic recovery. Our key insight is that models can retain semantic understanding of glitch tokens, yet naive repairs may corrupt this understanding. Therefore, we target functional repair, aiming to restore correct use of these tokens in context. To this end, we propose GlitchPatch, a semantically guided framework combining unified closed-form output correction with selective semantic learning. This training-free correction calibrates one output-head row per glitch token using a few contexts, leaving all other parameters unchanged. For tokens failing semantic assessment, we show that reliable repair requires prior semantic learning, achievable through simple reference-based training of a shared adapter. With repairs applied jointly across the vocabulary, GlitchPatch achieves functional repair rates of 86.5% on 16,788 Qwen-7B-Chat targets and 78.6% on 4,351 DeepSeek-LLM-7B-Chat targets, compared with 16.2% and 24.6%, respectively, for GlitchCleaner. Complete-repair evaluations additionally cover Llama-2-7b-chat-hf, Qwen3.5-9B, and Qwen3-32B. On Qwen-7B-Chat, accuracy changes remain negligible on MMLU, QQP, and GSM8K when numeric-answer extraction is used. Low repair cost and modest measured inference overhead support vocabulary-wide deployment.