- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper defines grounded skill-following: an agent has to run a fixed, expert-written skill through all of its required phases. Each skill becomes a "skill contract", meaning visible instructions plus a hand-written state machine. The state machine checks every action, sends next-step hints, and adds first-time milestone rewards to the task reward during GRPO training. On Math and Search with five Qwen executors, trained policies reach 94–99.99% Protocol Completion Rate. Task Outcome is roughly on par with matched ReTool/Search-R1 recipes. The paper also runs credit/feedback ablations and inference-time interventions on the skill text, observations and feedback.
Critical strengths
- Compliance and correctness both come from one deterministic verifier and are always reported separately. The main text is honest about results that go against the method: no consistent gain over Recipe, ρ=1.0 beating ρ=0.9 on outcome, and heavy reliance on feedback. The method itself is simple, so these findings are what make the paper useful.
- The Recipe baselines look fair. They share data, retriever and evaluator with the method. Their Search numbers line up with the published Search-R1 GRPO results (about 32.7 vs 31.2 average EM on Qwen2.5-3B, 37.1 vs 35.0 on 7B). That makes "on par with Recipe" believable across five executors.
Critical weaknesses
- The near perfect PCR is largely built into the contract runtime. Inadmissible actions are blocked, and every step comes with Contract-State Feedback that names an admissible next action. When feedback is removed at inference, Search PCR falls sharply, as Panel B of Table 2 shows. No baseline gets a compliance score, since PCR for Recipe is marked n/a in Table 1. The paper's motivation is that outcome-driven RL methods such as Search-R1 [1] or ReTool [2] do not follow the procedure. This is never checked with the same contract verifier, not even on the phases that both setups share.
- The abstract overclaims. It states that jointly trained Qwen3.5-4B slightly outperforms the original baselines in Task Outcome, but on Math the Recipe baseline scores higher on that same model. Across Table 1, Math results are tied with or worse than Recipe overall. The clear gains appear only on Search with the Qwen3.5 models, and all results come from single training runs.
- Both skills share the same phase set, as Table 3 shows. The checks only look at form, such as a minimum length for a substantive think step or non-empty stdout from the interpreter. This is close to checking the action tag format of the Search-R1 [1] and ReTool [2] loops. Two skills of this kind say little about expert-authored skills in general.
- The contract runtime is basically a manually written reward machine [3]. The paper argues that its skill contract differs from reward machines because reward machines require rigid state definitions, and this contrast does not hold. Verified Progress Credit is also close to the format and step-wise rewards used in ToolRL [4] and StepSearch [5]. SOPBench [6] already separates compliance with standard operating procedures from outcome by using rule-based verifiers. None of ToolRL, StepSearch, or SOPBench is cited.
[1] Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. [2] ReTool: Reinforcement Learning for Strategic Tool Use in LLMs. [3] Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning. [4] ToolRL: Reward is All Tool Learning Needs. [5] StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimization. [6] SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints.
Questions for authors
- In the Prompt rows, Qwen2.5-3B-Instruct gets 75.07% Search PCR but only 0.19% Task Outcome. That is far below its own no-retrieval Direct score of 11.37%. Qwen3.5-4B also falls from 55.44 to 0.83 on Math. Is this an extraction or runtime issue? If untrained models basically cannot use the contract, that changes how I read the expert-authored skill framing.
- With a recovery coefficient of 1.00, Math RF-PCR is 0.00 while PCR is 96.23, so every completed trajectory had at least one rejected action. What are those rejected actions? Since rejections cost nothing under this setting, is the policy testing the runtime with rejected actions on purpose?
- Was Credit-only evaluated with feedback off? If so, it reaches 98.69 Search PCR without feedback, while Feedback + Credit drops to 43.05 once feedback is removed. What is the case for keeping feedback in the main method?
- In Table 9, on Search, swapping in the Math skill instructions costs only 0.15 points, while an independently written paraphrase costs 3.40. How does this fit the claim that execution depends on the visible skill instructions?
- Under the same runtime, does the task reward plus a single binary bonus for reaching DONE train as well as your method? The base model almost never reaches DONE, so Outcome-only staying at zero is expected. It is unclear what Verified Progress Credit adds beyond any reward shaping at all.
- In Table 2, the Math Task Outcome for a recovery coefficient of 1.00 is 64.61, which is identical to the single-task Math result in Panel B. The Search PCR for a recovery coefficient of 0.90 is 98.90, which is identical to the single-task Search PCR. These are different policies. Can you confirm these are not copy errors?