SeqGuard: Length-Independent Stopping for Streaming Hallucination Detection

Streaming detectors can score a language model's output for hallucinations while it is being written. The rule for when to interrupt is still a tuned score threshold, and a threshold has no error control that holds at every length: set to $\alpha =0.1$ on RAGTruth, it reaches $0.19$ on long outputs, and calibrated on AggreFact or TofuEval and deployed on RAGTruth it runs $1.4-1.6\times$ over budget. We introduce SeqGuard , a stopping rule that turns any streaming detector's scores into an anytime-valid sequential test. Each score is ranked against scores from known-faithful outputs, the rank becomes a bet, and the output is interrupted once the accumulated winnings reach $1/\alpha$. A martingale argument bounds the probability of ever interrupting a faithful output by $\alpha$, for any detector and any length. The proof covers a conditionally calibrated variant that keeps $74-80\%$ of the deployed rule's power; the simpler marginal variant we deploy is checked empirically. With a prefix-entailment scorer and a hidden-state probe, the operating configuration's realized false-alarm rate stays at or within its confidence interval of the budget in-domain (its one point estimate above budget is Data2txt at $\alpha =0.01$, $5$ of $321$ episodes) and under shift among RAGTruth, AggreFact, and TofuEval, helped by slack (in-domain FAR is $0.34-0.72$ of $\alpha$). A far-out-of-distribution HaluEval test breaks both rules: SeqGuard overshoots $1.2-1.5\times$ on summarization and $2.2\times$ on dialogue, against $2.0\times$ and $5.5\times$ for the tuned threshold. Against a memoryless rule that sees the same two detectors, the power gain is concentrated on Data2txt ($1.4$–$1.8\times$ on three of four annotation-defined classes at matched FAR) and at $\alpha =0.01$ on QA and Data2txt; it does not win on Summary. The cost is delay ($32-73$ tokens at $\alpha =0.1$), though $72$–$85\%$ of flags fire before the output ends.