SciQuotient: From Exact Failures to Transferable Scientific State Abstractions

Scientific agents powered by large language models (LLMs) increasingly solve complex problems through long-horizon cycles of code generation, execution, and revision, accumulating failures that could serve as valuable training experience. Yet repairing a failure does not necessarily yield transferable knowledge, because the model may still misidentify which state distinctions matter. To make this gap measurable, we introduce SciQuotient, an integrated evaluation-and-training framework for learning transferable scientific state abstractions from exact failures. Given a fully specified finite dynamical system and a target observation, an LLM must construct the coarsest exact state abstraction that preserves all future target behavior. Because the correct abstraction can be certified exactly, SciQuotient uses certified quotient-preserving transformations to vary scientifically irrelevant aspects of the state and its representation while leaving the correct abstraction unchanged, exposing hidden abstraction failures that conventional held-out accuracy can miss. Across several model families, training on exact failures in a single representation strongly repairs the targeted behavior and supports partial transfer, but models trained this way remain brittle to mathematically equivalent re-encodings. In contrast, training on equivalent failures across multiple representations substantially broadens transfer to unseen representations and state configurations, with this transfer extending to held-out dynamical mechanisms without additional training. The resulting models can also surpass strong reference LLM baselines on SciQuotient transfer evaluations. Together, these results show that fixing known failures does not guarantee transfer, while representation-diverse exact-failure training helps models carry learned abstractions beyond the conditions in which those failures were observed.