SIAR: Skill Influence Attribution for Skill-Augmented Agentic Reinforcement Learning

Large language model (LLM) agents have shown strong potential in long-horizon interactive tasks, and recent skill-augmented agentic reinforcement learning (RL) methods further improve them by distilling past experience into reusable natural-language skills. However, existing methods remain coarse-grained in both skill updating and policy optimization: skills are updated according to weakly diagnostic outcome signals or fixed schedules, and all actions in a trajectory receive a uniform advantage regardless of the retrieved skill's influence. To address these limitations, we propose SIAR, a skill-augmented agentic RL framework centered on skill influence attribution. SIAR measures how strongly the retrieved skill drives the agent toward each executed action, producing a step-level attribution signal that constitutes the missing fine-grained evidence. Building on this signal, attribution-guided skill updating rewrites a skill only when accumulated attribution evidence suggests a mismatch between its influence and task outcomes, and attribution-calibrated advantage assignment augments the uniform trajectory-level advantage with additional credit or penalty for strongly skill-influenced actions. The two components allow skills and the agent policy to co-evolve throughout training. Experiments on three challenging agentic tasks (ALFWorld, WebShop, and search-augmented QA) show that SIAR outperforms existing skill-augmented agentic RL methods.