Evidence Over Plans: Online Trajectory Verification for Skill Distillation

Agent skills can remarkably improve task success rates by using human-written procedural documents, but their quality is difficult to assess without environment-grounded verification. Existing skill generation methods heavily rely on preference logs rather than direct environment interaction, often yielding negligible or even degraded gains. We identify that this reflects a fundamental timing problem: robust skills should be *posterior-based*, distilled from environment interaction rather than composed from offline plans. In this study, we introduce the **Posterior Distillation Index (PDI)**, a trajectory-level metric that quantifies whether a distilled skill reflects environment-verified execution evidence rather than offline planning. To operationalize PDI, we present **SPARK** (**S**tructured **P**ipelines for **A**utonomous **R**unnable tas**K**s and s**K**ill generation), a closed-loop framework for automated agent skill synthesis. SPARK generates environment-grounded trajectories from which PDI is computed, and it uses PDI as an online intervention signal to improve skill generation since its creation time. Critically, SPARK enables trajectory-level analysis for skill generation by preserving key execution logs, verifier signals, and memo histories. Across 86 runnable tasks, SPARK-generated skills consistently surpass no-skill baselines and outperform human-written skills on major student models. These findings demonstrate that key posterior evidence distillation enables transferable skill generation via environment interaction. We release anonymous code in [https://anonymous.4open.science/r/spark_anonymous-2C82/](https://anonymous.4open.science/r/spark_anonymous-2C82/).