Learning What vs. Learning How: Decomposing Task and Interface Competence in Agentic Training

LLM agents operate through harnesses that define system prompts, action formats, feedback loops, and resource constraints. Although a model may possess the same underlying task knowledge, successful execution does not necessarily follow under every harness. This raises basic questions: when a policy fails under a given harness, is it missing task knowledge, or does it simply lack the competence to operate the interface? Can harness-conditioned training improve task competence while degrading interface competence, or vice versa? To address these, we formulate a two-component view of agentic competence: task competence, the knowledge and reasoning required to solve a task, and interface competence, the ability to act on that knowledge through a given harness. We demonstrate that harness-conditioned training can yield vastly different outcomes depending on how these competencies interact. While multi-harness distillation aggregates task knowledge learned from different harness trajectories into a shared policy, this knowledge is only realised if the training distribution preserves or improves interface competence for the target harness. Otherwise, harness diversity can induce interference in interface competence, sometimes catastrophically collapsing performance on a held-out harness despite improvements in underlying task competence. To further distinguish the effects of task and interface supervision, we introduce harness rehearsal: distilling the model on synthetic task trajectories designed to carry minimal task knowledge while exercising the target harness’s interface. We find that rehearsal alone raises performance under the target harness, and when recombined with task knowledge acquired under different harnesses, produces gains far larger than either alone. We show that the effect holds across four reasoning and agentic benchmarks and persists through the subsequent reinforcement learning stage.