Word-Level Text Unmixing via Evidence-Preserving Ownership Routing with Language Models

Text from multiple sources can become interleaved into a single sequence when attribution metadata is lost, such as overlapping speech transcripts, complex document reading flows, or concurrent agent streams. While all observed words remain present, their underlying source ownership is unknown. We formalize this challenge as Word-Level Text Unmixing: given an interleaved lexical stream and the source count K, recover the original source sequences while preserving every word occurrence and its within-source order exactly. Conventional large language model (LLM) paradigms approach this by generating separated source texts directly. However, coupling latent ownership inference with lexical regeneration frequently causes LLMs to omit, duplicate, or hallucinate words, violating the exact-reconstruction objective. To resolve this, we propose Evidence-Preserving Ownership Routing (EPoR), a framework that cleanly separates ownership prediction from sequence reconstruction. During training, EPoR adapts a causal LLM to predict canonical ownership routes conditioned on the interleaved stream and prior routing decisions, inducing dynamic, history-conditioned source contexts without lexical regeneration. At inference, EPoR pairs completion-safe constrained decoding with deterministic indexed reconstruction, guaranteeing structurally valid K-source partitions that preserve every observed token occurrence exactly once. For systematic evaluation, we introduce UnMixBench, spanning controlled synthetic mixtures, timestamp-derived speech from AMI and ICSI, layout-derived document streams from ReadingBank, and simulated concurrent digital outputs. Across five evaluation tracks, a 4B EPoR model achieves the lowest minimum-permutation word error rate (MP-WER) among finetuned baselines, reducing the five-track mean by 22.3% relative to compact source-array generation and competing favorably with zero-shot frontier LLMs. These findings indicate that when lexical evidence is fully observed, decoupling ownership routing from sequence reconstruction can provide a more reliable and structurally sound alternative to generative decoders.