Latent Structure as a Complementary Predictor for Language Reasoning

Most neural reasoning architectures produce task predictions through a single classifier, even when their internal representations encode relational information. We investigate whether latent structure induced without structural annotation can support a complementary predictor alongside direct neural classification. We develop a dual-path architecture that constructs latent structure over token-ID groups. Multiple context-dependent prototypes summarize the occurrences of each token, prototype compatibility defines interactions between groups, and gated mixture-of-experts message passing produces group representations for structured prediction. In parallel, a neural branch predicts directly from contextual token states. An input-dependent gate combines the two outputs using the margin between the two largest latent-state logits as a fusion signal. We evaluate the architecture from random initialization against similarly sized decoder-style models under matched model-selection and evaluation protocols on FOLIO, JustLogic, ProofWriter, ANLI-R1, SNLI, and MNLI. It obtains the highest observed mean accuracy among the evaluated models on all six datasets. In a diagnostic FOLIO run, fusion improves accuracy from 44.83% to 49.75%; among fourteen predictions changed relative to the neural branch, ten correct errors made by that branch, while none change a correct neural prediction into an error. Single-seed ablations further provide preliminary evidence for the contributions of prototype induction, compatibility-based propagation, and adaptive fusion. These results provide evidence that task-induced latent structure can function as a distinct and complementary source of predictions in language reasoning.