acceptodds
Under review as a conference paper at ICLR 2027

SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents

Abstract

Recent work incorporates reusable skills distilled from past interactions into multimodal agent training, providing procedural guidance for long-horizon planning and tool use. However, policy optimization in these methods remains driven primarily by sparse outcome rewards, providing little supervision for intermediate decisions. Rubric-based rewards address this limitation through explicit intermediate criteria, but reliable rubrics are difficult to construct at scale and often disconnected from the procedure followed by the actor. We observe that a well-structured skill naturally specifies both how to act and what successful execution should achieve. Based on this insight, we introduce **SkillRubric**, which represents each skill through aligned actor-facing guidance and an evaluator-facing rubric. A multimodal verifier evaluates skill-defined goals using screenshots and tool outputs, assigning completion and progress rewards to the responsible turns. We further introduce an alternating co-evolution scheme that validates guidance revisions through paired rollouts under a frozen policy and rubric revisions offline under fixed guidance. Experiments across diverse multimodal agent benchmarks demonstrate consistent performance gains, while controlled paired rollouts further show that evolved skills provide more effective guidance for planning and tool use than their preceding versions.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Multimodal Agents, Skill, Rubric-Based Rewards, Co-Evolution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5CZROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillRubric represents reusable skills through two aligned views: actor-facing guidance for planning and tool use, and evaluator-facing rubrics for assessing intermediate progress. A multimodal verifier uses tool outputs and visual evidence to assign turn-level process rewards alongside outcome rewards. During training, the framework alternately revises guidance and rubrics, validating guidance through paired frozen-policy rollouts and rubrics through offline evaluation of identical trajectories. Experiments across eight benchmarks and two trained backbones report consistent improvements over skill-based baselines, including on held-out benchmarks and without inference-time skill guidance.

Critical strengths
  1. The method improves over the strongest skill-based baseline across all eight benchmarks for both trained backbones. Comparisons share SFT initialization, training tasks, tools, and initial actor-facing skills. Gains also persist without inference-time guidance, supporting a benefit from training beyond prompt augmentation
  2. Deriving procedural guidance and evidence-based evaluation criteria from reusable skills provides a practical way to iterate the model update. The separate validation procedures control for changes in behavior and evaluation, while ablations show that evolving both components performs better than fixing either one. These results support the framework’s practical usefulness.
Critical weaknesses
  1. I find the conceptual advance relatively incremental. Reusable skills that evolve alongside policy training have been studied in SkillRL [1], step-level rubric supervision in Step-wise Rubric Rewards [2], and adaptive rubric–agent co-evolution in ARCO [3] and EvoRubrics [4]. SkillRubric’s more distinctive contribution is deriving actor guidance and evaluator rubrics from a shared procedural abstraction and validating their revisions separately. However, the ablations leave two questions unresolved. First, does deriving guidance and rubrics from the same skill improve performance compared with generating them separately from the same training trajectories? Second, does updating guidance and rubrics in separate stages work better than updating them together? Comparisons using similar rollout and external-model budgets would help establish the value of these specific design choices.

  2. Moreover, controlling policy rollout budgets does not establish comparable total training expenditure when the method adds synthesis, verification, and validation. The interaction-efficiency comparison against the original backbones also does not establish efficiency relative to the trained baselines. These gaps limit confidence in the magnitude and cost-effectiveness of the improvements.

[1] Xia et al., 2026. SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning.

[2] Xie et al., 2026. Step-wise Rubric Rewards for LLM Reasoning.

[3] Tian et al., 2026. ARCO: Adaptive Rubrics with Co-Evolution for Multi-Step LLM-Based Agents.

[4] Ding et al., 2026. EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning.

Questions for authors
  1. Under comparable rollout and external-model budgets, how does shared-source guidance–rubric construction compare with independently constructing guidance and rubrics from the same teacher trajectories? This would help determine whether the claimed alignment drives the gains.
  2. Does alternating guidance and rubric validation outperform joint revisions under comparable budgets? The existing component ablations establish the benefit of adapting both components, but do not isolate the proposed update procedure.
  3. How stable are the main improvements and the differences between evolution variants across repeated runs or paired confidence intervals? What is the total training cost relative to the strongest baseline, including synthesis, verification, and validation?
  4. Is there quantitative validation of the multimodal verifier’s criterion judgments and turn attribution, such as agreement with human annotations? In particular, how reliably does it recognize valid partial progress in trajectories with incorrect final answers?
6 comments

We thank the reviewer for the positive assessment and constructive feedback. We address each question below.

Q1&W1: Under comparable budgets, does shared-source guidance–rubric construction outperform independent construction from the same teacher trajectories?

Answer: Deriving guidance and rubrics from the same skill offers two benefits:

  • More applicable supervision. Guidance focuses the actor on relevant strategies. Independent rubrics may assess unused paths or miss progress. Shared-source rubrics assess expected evidence, improving applicability and interpretability while allowing valid alternatives.
  • Targeted diagnosis and revision. Criterion-level scores locate missing evidence, supporting targeted guidance revisions beyond selecting variants by final scores.

To isolate the benefit of shared-source construction, we compared rubric sources on both backbones, fixing actor guidance, the verifier, and reward formulation under comparable rollout and external-model budgets. GPT-5.5, the SkillBank synthesizer, generates independent rubrics from each task, its reference answer, and nine teacher trajectories (three per teacher), including reasoning, tool calls, observations, and final responses. Inputs include image references but no pixels or additional outcome labels. Without access to the extracted skill, guidance, or existing rubric, it generates three to five method-agnostic checkpoints, permitting valid alternatives and excluding task-specific answer values. Both approaches use the same teacher evidence, testing rubric construction through the procedural abstraction shared with guidance. Co-evolution is disabled in both conditions, keeping guidance and rubrics fixed during training. The skill-derived condition is the original GRPO-Rubric ablation (H=∞H=\infty).

BackboneRubric sourceIIDOODOverall
Qwen3-VL-8BIndependently generated13.6810.0611.87
Qwen3-VL-8BSkill-derived15.4611.7013.58
Qwen3.5-9BIndependently generated22.8725.1023.99
Qwen3.5-9BSkill-derived25.6025.3725.49

Skill-derived rubrics improve Overall by 1.71 percentage points on Qwen3-VL-8B and 1.50 on Qwen3.5-9B, with higher IID and OOD averages on both, supporting rubric construction through the abstraction shared with guidance.

We expanded Table 2 (page 8) and updated Section 4.3, “Component contributions” (page 8, lines 417–424). Construction and training protocols appear in Appendix B.8, “Independent Rubric Construction” (page 26, lines 1380–1403). Changes are highlighted in blue.

Q2&W1: Does alternating validation outperform joint revisions under comparable budgets, beyond the benefit of adapting both components?

Answer: Guidance shapes trajectories; rubrics evaluate them. Revising both changes behavior and evaluation criteria simultaneously; a rubric validated under previous guidance may inadequately assess revised executions. Our procedure validates guidance through paired frozen-policy rollouts, then delays rubric calibration until trajectories are available under the associated guidance.

We compared joint revisions under comparable rollout and external-model budgets. Every 10 policy optimization steps, the coach proposes guidance and rubric changes together. Candidate rubrics are validated by offline rescoring of existing trajectories under active guidance, using Appendix B.4’s outcome-alignment and pass-rate criteria. If all rubric revisions pass, candidate guidance is compared with current guidance through frozen-policy rollouts on paired tasks with identical sampling seeds. Guidance acceptance uses the fixed outcome reward, not candidate-rubric scores. Both revisions are released together only if both stages pass; otherwise, both are rejected. No subsequent calibration cycle precedes release.

BackboneUpdate procedureIIDOODOverall
Qwen3-VL-8BJoint revisions16.4213.1214.77
Qwen3-VL-8BAlternating validation19.9115.8617.88
Qwen3.5-9BJoint revisions26.7326.5826.66
Qwen3.5-9BAlternating validation28.6128.9728.79

Alternating validation improves IID and OOD averages on both backbones, increasing Overall by 3.11 percentage points on Qwen3-VL-8B and 2.13 on Qwen3.5-9B. Since both procedures adapt both components, these results support staged validation and release over this joint baseline, consistent with calibrating rubrics on trajectories generated under their associated guidance.

We added the baseline to Table 2 (page 8) and updated Section 4.3, “Component contributions” (page 8, lines 424–431). The protocol appears in Appendix B.5, “Joint-Revision Baseline” (pages 23–24, lines 1231–1266). Changes are highlighted in blue.

Q3(a): How stable are the main gains and evolution-variant differences across repeated runs or paired confidence intervals?

Answer: We evaluated the main comparison, evolution variants, and Figure 3(a–b) SkillBank versions with seeds 42, 43, and 44, fixing each trained checkpoint. Mean ± sample SD measures evaluation variability, not independent-training variability. Overall averages eight benchmarks equally per seed. Scores are percentages; differences are percentage points.

BackboneMethodSeed 42Seed 43Seed 44Mean ± SD
Qwen3-VL-8BSkillRL13.1213.2512.8813.08 ± 0.19
Qwen3-VL-8BSkillRubric17.8817.6417.3817.63 ± 0.25
Qwen3.5-9BSkillRL25.2424.5624.7224.84 ± 0.36
Qwen3.5-9BSkillRubric28.7928.3528.0328.39 ± 0.38

SkillRubric wins on every seed for both backbones. Within-seed gains average 4.55 ± 0.19 points on 8B and 3.55 ± 0.24 on 9B, ranging 4.39–4.76 and 3.31–3.79, respectively.

Evolution variants.

VariantHQwen3-VL-8BQwen3.5-9B
GRPO-Rubric∞\infty13.38 ± 0.4625.27 ± 0.21
Guidance evolution only1016.94 ± 0.2426.43 ± 0.34
Rubric evolution only1016.17 ± 0.4826.49 ± 0.56
Joint revisions—14.56 ± 0.2626.72 ± 0.23
Ours514.27 ± 0.3226.46 ± 0.38
Ours1017.63 ± 0.2528.39 ± 0.38
Ours2016.25 ± 0.4226.87 ± 0.29

GRPO-Rubric fixes the SkillBank; H follows the manuscript’s alternating-update definition. Joint revisions propose both components every 10 policy-optimization steps. Default H=10 achieves the highest means, exceeding joint revisions by 3.07/1.67 points on 8B/9B, and both single-component variants and alternative intervals, supporting alternating validation across these settings.

SkillBank versions, Figure 3(a–b).

SkillBank versionQwen3-VL-8B-InstructQwen3.5-9B
Without skill guidance7.37 ± 0.3318.68 ± 0.37
v0v_010.23 ± 0.2822.54 ± 0.38
v1v_111.92 ± 0.4923.56 ± 0.29
v2v_212.72 ± 0.2424.34 ± 0.13
v3v_314.45 ± 0.1825.16 ± 0.07
v4v_416.06 ± 0.1126.12 ± 0.12

Cross-seed mean Overall increases monotonically from v0v_0 to v4v_4, gaining 5.83/3.58 points on 8B/9B, preserving progressive improvement across versions.

Q3(b)&W2: What is the total training cost relative to the strongest baseline, including synthesis, verification, and validation?

Answer: We compare SkillRL, the strongest baseline, across synthesis, verification, validation, and shared RL workload, using logged step timings, trajectory statistics, validation budgets, and API/tool pricing assumptions.

Cost componentUnit8B: SkillRL8B: SkillRubric9B: SkillRL9B: SkillRubric
Synthesis — teacher data, initial SkillBank, evolution proposals$2,2502,2482,2502,248
Verification — outcome and process judging$92799321
Validation — API/tools$853511740
Validation — GPU overheadGPU-hours2249220131
Shared RL — rollout tools$418418577577
Shared RL — training computeGPU-hours1,1571,1571,1441,144
Combined at $1/GPU-hour$4,1434,2294,2984,360
Relative combined cost—1.000×1.021×1.000×1.014×

8B/9B denote Qwen3-VL-8B/Qwen3.5-9B.

  • Synthesis: Both share teacher collection and initial SkillBank construction; evolution proposals add a few dollars. Each configuration includes $2,247 one-time preparation for standalone comparison, though artifacts are reused across runs.
  • Verification: Both use the same outcome judge. SkillRubric adds GPT-5-mini coarse matching and Gemini 3.5 Flash fine verification: $270/$312 extra for 8B/9B, estimated from sampled text lengths and a 2,048-token output budget per stage, excluding image tokens.
  • Validation: SkillRL uses periodic rollouts with tool/judging costs; SkillRubric uses cached rescoring for rubric calibration and paired frozen-policy rollouts for guidance validation. GPU overhead covers extra co-evolution time; API/tools cover both validation stages. Savings are 132/171 GPU-hours for 8B/9B.

Shared SFT compute is excluded. SkillRL validation time uses ordinary-step rollout timings. Totals precede rounding, so displayed components may differ from totals by one unit. At an illustrative $1 per H20 GPU-hour, combined cost rises 2.1% on 8B and 1.4% on 9B, with process-verification costs partly offset by validation savings.

The breakdown and assumptions appear in Appendix B.9, “Training Cost Analysis” (page 27, lines 1420–1449; Table 8 on the same page), highlighted in blue.

W2: The interaction-efficiency comparison against the original backbones also does not establish efficiency relative to the trained baselines.

Answer: Following the reviewer’s suggestion, we expanded the original comparison to include SkillRL, XSkill, Skill-SD, and Skill0. The tables below report average interaction turns on each benchmark, with Avg. denoting the unweighted average across all eight benchmarks.

Qwen3-VL-8B

ModelAgentVistaBrowseComp-VLVDR-BenchVisualToolBenchVisBrowseMMSearch+MMSearchExamTIR-BenchAvg.
Qwen3-VL-8B-Instruct6.74.63.98.45.85.04.67.95.9
SkillRL5.53.64.06.24.23.33.86.74.7
XSkill5.64.54.86.56.04.64.76.95.5
Skill-SD4.24.34.25.15.65.04.43.84.6
Skill05.33.95.66.34.24.43.45.44.8
Ours (w/ skill)5.93.33.16.15.23.23.16.04.5

Qwen3.5-9B

ModelAgentVistaBrowseComp-VLVDR-BenchVisualToolBenchVisBrowseMMSearch+MMSearchExamTIR-BenchAvg.
Qwen3.5-9B10.76.89.69.511.39.79.68.99.5
SkillRL7.97.78.711.78.89.28.89.39.0
XSkill7.95.46.910.99.75.66.74.37.2
Skill-SD6.67.39.65.410.49.610.14.47.9
Skill02.58.47.42.73.111.24.52.05.2
Ours (w/ skill)4.84.75.73.16.44.65.51.64.6

As shown in the tables above, SkillRubric achieves the lowest average interaction count on both backbones. On Qwen3-VL-8B, it averages 4.5 turns, compared with 4.7, 5.5, 4.6, and 4.8 for SkillRL, XSkill, Skill-SD, and Skill0, respectively. On Qwen3.5-9B, it averages 4.6 turns, compared with 9.0, 7.2, 7.9, and 5.2. The reductions are modest on 8B and larger on 9B, with benchmark-specific variation. Together with the higher success rates in Table 1, these results support improved task performance with fewer interactions on average relative to the trained baselines.

We have expanded Table 4 (page 10) and updated the “Interaction efficiency” paragraph in Section 4.4 (page 10, lines 506–512) to include the trained-baseline comparisons. The additions and revisions are highlighted in blue.

Reviewer_hAyQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

SkillRubric proposes a framework for training multimodal tool-using agents using reusable skills that jointly specify actor-facing guidance and evaluator-facing rubrics. Skills are distilled from successful and unsuccessful trajectories generated by multiple teachers. Successful trajectories additionally initialize the student through supervised fine-tuning.

During reinforcement learning, the agent receives retrieved guidance, while a coarse-to-fine multimodal verifier evaluates the corresponding criteria and assigns responsibility-weighted completion and incremental-progress signals to individual turns. These process advantages are combined with outcome advantages. The framework alternately updates guidance through frozen-policy paired rollouts and rubrics through offline evaluation on shared trajectories.

Critical strengths
  1. The novelty of this paper is pretty good among the experience-related papers I have reviewed. The authors find that previous works have used these two paradigms to deal with this problem: a. Synthesize past experience into operational recommendations to help the agent execute tasks more effectively. b. Evaluate intermediate steps using specific scoring criteria—for example, "Were entities correctly identified?" or "Was evidence obtained to support the conclusion?" And I totally agree that these two ways are not the ideal method to deal with this problem. They would lead to a fixed paradigm, and the generated trajectories are not that useful for further training. This paper derives guidance and evaluation criteria together from skills, which is a useful way to connect skill-conditioned execution with process supervision.

  2. The empirical improvements are broad across the evaluated settings. SkillRubric improves the eight-benchmark average over SkillRL from 13.12 to 17.88 with Qwen3-VL-8B and from 25.24 to 28.79 with Qwen3.5-9B. The skill-guided variant performs best among the listed trained methods on every benchmark under both backbones. Four benchmarks are reserved entirely for out-of-distribution evaluation, providing useful evidence beyond held-out tasks from the training datasets.

Critical weaknesses
  1. The central shared-source alignment claim is not directly isolated. Under the motivation of the paper, the rubrics should be a general rubric that could be distilled from different sources. The current comparisons establish that adding the proposed rubric supervision can help, but they do not establish that deriving rubrics from the same procedural source as guidance is responsible for the improvement. A strong control would retain the same guidance, verifier, reward formulation, and compute budget while generating rubrics independently from the task and available training evidence.

  2. The reliability of the process-credit mechanism is insufficiently measured. The method depends on criterion applicability, evidence-state classification, and responsibility attribution by external models. The appendix provides detailed instructions and illustrative cases, but does not quantitatively establish whether the verifier identifies the correct evidence or assigns credit to the responsible turns.

Questions for authors
  1. What is the specific benefit of deriving guidance and rubrics from the same skill? Could the authors compare skill-derived rubrics with independently generated task rubrics while keeping the actor guidance, verifier, reward formulation, and supervision budget fixed?

  2. How reliable are the verifier’s criterion-level judgments and turn-level credit assignments? Could the authors provide a human-audited evaluation or controlled evidence-perturbation study to assess whether the verifier identifies the correct evidence and responsible turns?

2 comments

We appreciate the reviewer’s positive assessment of our work. Thank you for the constructive suggestions and we address your questions below.

Q1&W1: What is the specific benefit of deriving guidance and rubrics from the same skill? Could the authors compare skill-derived rubrics with independently generated task rubrics while keeping the actor guidance, verifier, reward formulation, and supervision budget fixed?

A1: We thank the reviewer for suggesting this comparison. Deriving guidance and rubrics from the same skill provides two practical benefits:

  • Narrower search space and more applicable supervision. Guidance narrows the actor’s search toward relevant solution strategies. Independently generated rubrics may assess intermediate steps on paths the actor never takes, leaving checkpoints inapplicable or missing the progress it actually makes. Shared-source rubrics assess the evidence expected from the guided strategies, making process scores more applicable and interpretable while allowing alternative valid executions.
  • More targeted diagnosis and skill revision. Criterion-level scores help identify which parts of a skill fail to produce the expected evidence and where execution breaks down. This feedback supports targeted guidance revisions beyond selecting skill variants solely by their final task scores.

Following the reviewer’s suggestion, we conducted an independent-rubric ablation on both backbones, keeping actor guidance, the verifier, and reward formulation fixed under comparable rollout and external-model budgets. To construct the independent rubrics, we use GPT-5.5, the same synthesizer used for SkillBank construction. For each training task, the generator receives the task, its reference answer, and nine teacher trajectories (three from each of the three teacher models) containing reasoning text, tool calls, tool observations, and final responses. These inputs include image references but no image pixels, and no additional outcome labels are supplied. The generator directly produces three to five method-agnostic checkpoints without accessing the extracted skill, actor guidance, or existing rubric. The prompt permits alternative valid solutions and excludes task-specific answer values from the checkpoints. Both approaches therefore use trajectory evidence; the comparison tests whether deriving rubrics through the procedural abstraction shared with actor guidance improves their effectiveness. We disable co-evolution in both conditions to isolate the effect of shared-source construction from subsequent guidance and rubric updates. Actor guidance and rubrics remain fixed throughout policy training. The skill-derived results correspond to the original GRPO-Rubric ablation (H=∞H=\infty). The results are summarized below:

BackboneRubric sourceIIDOODOverall
Qwen3-VL-8BIndependently generated13.6810.0611.87
Qwen3-VL-8BSkill-derived15.4611.7013.58
Qwen3.5-9BIndependently generated22.8725.1023.99
Qwen3.5-9BSkill-derived25.6025.3725.49

Under this controlled setting, skill-derived rubrics improve overall performance by 1.71 percentage points on Qwen3-VL-8B and 1.50 points on Qwen3.5-9B, with higher IID and OOD averages on both backbones. These results support the benefit of deriving evaluation criteria from the procedural abstraction shared with actor guidance.

We have expanded Table 2 (page 8) and updated Section 4.3, “Component contributions” (page 8, lines 417–424). The protocol for independent rubric generation and the controlled training setup are provided in Appendix B.8, “Independent Rubric Construction” (page 26, lines 1380–1403). The additions and revisions are highlighted in blue.

Q2&W2: How reliable are criterion judgments and turn attribution? Can human evaluation or evidence perturbation validate the identified evidence and responsible turns?

A2: Following the reviewer’s suggestion, we conducted human evaluation and evidence perturbation to assess progress detection and turn attribution.

Human evaluation. Two CS PhD annotators evaluated the same 326 criterion–turn pairs from trajectories with incorrect final answers. Fixed verifier predictions were compared against each annotator separately. Correct detection requires identifying both the criterion showing evidence-supported progress and its contributing turn.

MetricAnnotator AAnnotator B
Human-labeled positive events7267
Verifier-predicted positive events6464
True positives / False positives / False negatives58 / 6 / 1455 / 9 / 12
Precision90.6%85.9%
Recall80.6%82.1%
F185.3%84.0%
False-positive rate2.4% (6/254)3.5% (9/259)

The verifier achieves 85.9–90.6% precision and 80.6–82.1% recall, with false-positive rates of 2.4–3.5%, supporting recognition and localization of valid progress despite incorrect final answers.

To illustrate this ability, a BrowseComp-VL case asks whether buses pictured in Guatemala are locally called chicken buses and whether residents consider the term inappropriate. A retrieval turn distinguished the local name camionetas from the tourist term chicken bus, but the final answer incorrectly identified the latter as the local name. For C2—organizing evidence into decision-relevant constraints—the verifier recorded missing → supported at that turn, citing the distinction, while assigning C2 a final fail and a missed_opportunity to that turn. This distinguishes localized progress from final satisfaction; the naming evidence alone did not resolve the term’s appropriateness.

Controlled evidence perturbation. Each backbone was evaluated on 64 critical-evidence and 64 irrelevant-evidence deletion cases from cached trajectories, with no trajectory overlap between groups. We assessed only intermediate process criteria, withholding final answers and environment outcomes. Deletions removed a tool observation and its images while retaining the tool-call action. For critical cases, both prefixes ended at the affected turn to prevent later evidence leakage. GPT-5-mini generated references in separate calls, requiring valid evidence references and confidence ≥0.85 for both versions. Critical cases required a reference-state decrease after deletion and positive reference responsibility at the affected turn in the original prefix; irrelevant cases required unchanged reference states and positive-responsibility turn sets. Cases were selected before testing the verifier, without filtering on its performance. Each was evaluated under Original, Repeat (independent rescoring of the same prefix), and Perturbed, with Repeat measuring evaluator variability.

MetricQwen3-VL-8BQwen3.5-9B
Excess state decrease after critical deletion, 95% CI0.34 [0.25, 0.43]0.31 [0.18, 0.44]
State consistency: irrelevant deletion / repeat93.75% / 95.31%92.19% / 93.75%
Original responsibility-turn F1 against model-generated references: critical / irrelevant groups0.88 / 0.860.87 / 0.85
Positive-attribution withdrawal after critical deletion90.00% (54/60)89.66% (52/58)
Positive-attribution withdrawal under repeat evaluation5.00% (3/60)5.17% (3/58)
Responsibility-set F1: Original–Perturbed, irrelevant group0.930.92
Responsibility-set F1: Original–Repeat, irrelevant group0.950.94

Excess state decrease follows Appendix B.3: missing/contradicted = 0, hypothesis = 0.25, supported = 0.6, and verified = 1. For the target criterion at the affected turn, we subtract the Original–Repeat decrease from the Original–Perturbed decrease and average over 64 critical cases. Confidence intervals use 1,000 task-clustered bootstrap resamples. Withdrawal is measured on the same cases receiving positive attribution under Original: 60/64 for 8B and 58/64 for 9B. Critical deletion lowers evidence states beyond repeat variability and withdraws attribution in approximately 90% of eligible cases, versus approximately 5% under Repeat. Irrelevant deletion preserves states and responsibility sets close to the repeat baseline. With tool-call actions retained, this supports attribution based on returned evidence rather than tool execution alone.

Together, these studies support human agreement on localized progress and selective responses to evidence removal. Protocols and results appear in Appendix D.1 (pages 35–36, lines 1867–1931; Figure 4 on page 37) (human evaluation and case analysis) and Appendix D.2 (pages 36–38, lines 1933–2040) (evidence perturbation), highlighted in blue.

Reviewer_DRYhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillRubric derives actor guidance and evaluator rubrics from a shared library of procedural skills for multimodal agents. Retrieved guidance conditions rollouts, while a two-stage verifier uses tool observations and visual evidence to assign rubric satisfaction, turn responsibility, and incremental progress. These process advantages supplement an outcome-based RL signal. Guidance and rubrics are updated alternately: guidance candidates are compared through frozen-policy rollouts, and rubric candidates are checked on the same cached trajectories against a fixed outcome evaluator. Experiments use two backbones and eight benchmarks, including four reserved for out-of-distribution evaluation. The paper reports improvements over adapted skill-based baselines, component ablations, transfer of evolved guidance, and reduced interaction turns.

Critical strengths
  1. Using the same procedural knowledge for action guidance and process evaluation is a useful design. The rubric is framed around observable intermediate states rather than mandatory tool sequences. The alternating update procedure also addresses a real problem: changing both the behavior-generating guidance and its evaluator at once would make improvement difficult to interpret. Keeping a separate outcome signal provides an important anchor.
  2. The empirical evidence supports a benefit beyond extra inference-time prompting. The baselines share the SFT initialization and training setup, and Appendix B.6 explains their adaptations. SkillRubric improves across all eight benchmarks for both backbones; the variant without inference-time skills still exceeds the strongest baseline on average. Tables 2 and 3 separately examine rubric rewards and guidance/rubric evolution, making the experimental case more convincing than a comparison of the full system with the base model alone.
Critical weaknesses

The evidence for the reliability of intermediate credit is less direct than the evidence for improved final performance. Prompt B.4 supplies the fine verifier with the final response and environment outcome, while Appendix B.4 accepts rubric revisions partly through increased agreement with that outcome. This is not test-set leakage, and the prompt explicitly forbids hindsight attribution. Nevertheless, agreement with a visible outcome does not independently show that earlier evidence states and responsible turns were judged correctly. The qualitative examples help, but they do not measure this issue. This matters because the paper attributes the benefit specifically to evidence-grounded turn-level supervision. I find the performance gains credible; I am less certain how much the current validation establishes the proposed explanation.

Questions for authors
  1. On a sample of the existing cached trajectories, how stable are the intermediate state judgments and turn attributions when the environment outcome is hidden? An independent audit of those judgments would also address the concern. I am asking about the process labels, rather than requesting another end-to-end training study.
  2. What is the exact definition of the alignment statistic (a(C)) in Appendix B.4, and are the 128 calibration trajectories distinct from those used to diagnose and propose that rubric revision? These details determine how I interpret the update gate.
3 comments

We thank the reviewer for recognizing the strengths of our work. We are grateful for the constructive feedback and address your questions below.

Q1&W1: How stable are intermediate state judgments and turn attributions when the environment outcome is hidden?

A1: Following the reviewer’s suggestion, we conducted a paired offline re-evaluation of 256 cached trajectories, with 128 generated by each of Qwen3-VL-8B-Instruct and Qwen3.5-9B. Following the submission, we used GPT-5-mini for the Stage 1 coarse selector and Gemini 3.5 Flash for the Stage 2 verifier. Stage 1 was executed once per trajectory, and its outputs were fixed across three Stage 2 conditions: A, outcome visible; B, the explicit outcome field replaced with an empty string; and A′, repeat scoring with identical inputs to A. All other inputs—including the skill, rubric, process evidence, images, and final agent response—were held fixed, and request order was randomized per trajectory. Thus, A/B measures sensitivity to explicit outcome visibility, while A/A′ measures repeat-scoring variability. The results are summarized below:

MetricQwen3-VL-8B-Instruct: A/BQwen3-VL-8B-Instruct: A/A′Qwen3.5-9B: A/BQwen3.5-9B: A/A′
Criterion-verdict agreement96.98%97.24%97.04%98.02%
Intermediate-state agreement90.20%90.85%91.46%92.50%
Responsible-turn F10.9120.9240.9060.932
Turn–role F10.8870.8910.9030.915
State agreement excluding positions labeled missing in all three conditions86.53%87.62%88.26%89.15%
State agreement excluding the final recorded turn91.04%91.72%92.05%92.68%

For each metric, A/B and A/A′ use the same eligible units across all three conditions, with results computed per trajectory and then averaged. Unavailable outputs are excluded, while missing remains a valid evidence-state label. Criterion-verdict agreement compares categorical criterion judgments, and intermediate-state agreement compares evidence-state labels at selected evidence turns. Responsible-turn F1 compares attributed turn sets, while turn–role F1 additionally requires matching attribution roles. Intermediate-state agreement reaches 90.20% and 91.46%, only 0.65 and 1.04 percentage points below the respective repeat-scoring references. Responsible-turn F1 is 0.912 and 0.906, with corresponding gaps of 0.012 and 0.026. State agreement remains close to repeat scoring after excluding positions labeled missing in all three conditions (86.53% vs. 87.62% for 8B; 88.26% vs. 89.15% for 9B) and, separately, after excluding the final recorded turn (91.04% vs. 91.72% for 8B; 92.05% vs. 92.68% for 9B). With Stage 1 outputs, process evidence, and the final response held fixed, these results show that intermediate judgments and turn attributions remain stable under removal of the explicit environment outcome, supporting the robustness of our process supervision to explicit outcome visibility.

We have added the protocol, metric definitions, and full results to Appendix D.3, “Sensitivity to Explicit Outcome Visibility” (pages 38–40, lines 2042–2112; Table 11 on page 39), highlighted in blue.

Q2: What is the exact definition of the alignment statistic a(C)a(C) in Appendix B.4, and are the 128 calibration trajectories distinct from those used to diagnose and propose that rubric revision? These details determine how I interpret the update gate.

A2: The alignment statistic a(C)a(C) is the Pearson correlation between the weighted rubric scores and the cached outcome rewards. For trajectory ii, we compute

\qquad a(C)=\operatorname{Corr} _ {\mathrm{Pearson}}\left(\{s _ i(C)\} _ {i\in\mathcal{I}},\{y _ i\} _ {i\in\mathcal{I}}\right).$$ Here, $w_j$ is the weight of criterion $j$ in the evaluated rubric, and $z_{ij}$ is 1 for a pass judgment and 0 for a fail judgment. The set $\mathcal J_i(C)$ contains the applicable and verifiable criteria with valid judgments. Not-applicable and unverifiable criteria are excluded rather than treated as failures. The set $\mathcal I$ contains trajectories with valid aggregate scores under both the current and candidate rubrics, ensuring a paired comparison. The cached outcome reward $y_i$ is used without binarization. We compute the alignment gain as $\Delta a=a(\widetilde C)-a(C)$ and estimate its one-sided 90% lower confidence bound using 1,000 paired bootstrap resamples, with identical trajectory indices used for both rubrics in each resample. For data separation, the calibration procedure selects the 128 trajectories from rollouts collected **after the candidate rubric is proposed**, under the selected guidance version. Trajectories used to diagnose issues and propose that candidate are excluded by this temporal cutoff. This is trajectory-level separation, not task-level separation: the same task may appear in both sets through different rollouts. Successive calibration checks may reuse trajectories from the cumulative post-proposal pool. We have clarified the alignment definition and trajectory separation in **Section 3.3 (page 6, lines 298–305)** and provided the score definition, paired bootstrap procedure, and calibration-pool selection protocol in **Appendix B.4, “Rubric validation” (page 23, lines 1188–1228)**. The revisions are highlighted in blue.

Dear Reviewer,

We would like to kindly follow up on our rebuttal and ask whether our responses have addressed your concerns. We would greatly appreciate any further feedback or questions you may have.

Thank you for your time and consideration.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.