SURPRISAL-AWARE PREPROCESSING FOR LLM CLASSIFICATION OVER ENCRYPTED TEXT

Fine-tuning large language models on sensitive data is challenging in security-critical applications, where plaintext exposure may be unacceptable and encryption is therefore essential. However, encrypted representations can substantially reduce model utility by disrupting the lexical, syntactic, and distributional regularities learned during pretraining. We address this problem with a surprisal-aware preprocessing framework that transforms selected plaintext spans before protection to improve the learnability of the resulting protected representations without modifying the downstream classifier or protection mechanism. The framework preserves task-relevant and structural content, generates context-compatible substitutions using a frozen language model, and selects replacements through pairwise ordinal aggregation over sentence-level semantic similarity, token-level contextual similarity, contextual surprisal, and part-of-speech compatibility. Across eight cross-domain classification datasets, preprocessing improves accuracy under all evaluated protection configurations. Under the primary S-FF3-1 setting, macro-average accuracy increases from 0.8230 to 0.8794, with positive point-estimate gains on all eight datasets; standard FF3-1 and AES-CBC show similarly consistent improvements. Controlled rewriting baselines and criterion ablations further support the contribution of the proposed selection strategy. Because encrypted representations may still leak information through observable sequence length, we additionally introduce a randomized padding extension that weakens the relationship between input length and protected-sequence length while preserving classifier utility. Overall, the results show that representation-aware preprocessing can improve the practicality of encrypted-data model training, while randomized padding provides complementary protection against length-based information leakage.