CapSynth: Capability-Decomposed Synthesis for Long-Context Multi-Constraint Instruction Following

Complex real-world tasks increasingly require language models to acquire new knowledge from long and noisy contexts while simultaneously satisfying large numbers of mutually constraining requirements. Existing work on instruction following leaves a systematic gap along this dimension, which produces constraints that are largely orthogonal to task content, attachable to arbitrary questions with- out any understanding of the material. Benchmarks whose constraint density approaches realistic settings rely on expert manual construction, which does not scale. We present CapSynth, an automated synthesis framework targeting long- context, reasoning-driven multi-constraint instruction-following tasks. CapSynth begins from a decomposition of the capability space into atomic capability items organized under several capability families, and synthesis is organized in reverse: we first determine the capability combination a question should cover, then gen- erate context materials and scoring criteria matched to that combination, so that the information density of the material serves the target capabilities. Generated instances pass through layered verification, including execution-based checks and grounding filters that help reduce inconsistencies between the material and scoring rubrics. We evaluate CapSynth on two long-context multi-constraint benchmarks. On a base model without instruction tuning, ten epochs of training produce strictly monotonic improvement on CL-bench. On the strongest backbone, the trained model surpasses its instruct version via reinforcement learning. Both results show that CapSynth data synthesis is high quality.