acceptodds
Under review as a conference paper at ICLR 2027

Quality Meets Structure: Context-Aware Trajectory Selection for Agentic Supervised Fine-Tuning

Abstract

The effectiveness of agentic supervised fine-tuning (SFT) depends on the quality of training trajectories and the diversity of their interaction patterns. However, existing data selection methods often overlook how decisions depend on earlier interactions and how execution patterns overlap across trajectories, resulting in redundant or less informative supervision. We introduce CATS, a graph-based selection method that estimates trajectory quality from context dependence and action complexity by comparing a frozen small language model's prediction losses across context variants. It constructs a similarity graph from ordered sequences of tool calls, natural-language replies, and tool-return statuses to capture overlap in execution patterns. Through message passing, CATS aggregates similarity-weighted trajectory scores, iteratively selects the highest-priority trajectory, and updates the priorities of unselected neighbors to discourage structural redundancy. Selecting 100K trajectories from a pool of over 600K, CATS outperforms nine baselines in average agentic and general performance across four benchmarks, improving average agentic performance by up to 2.6 percentage points over the strongest baseline. Ablations demonstrate the contributions of both quality scoring and structural selection, while subset analyses show broader coverage of execution patterns with high average trajectory quality. These findings highlight the value of jointly considering context-aware decisions and execution-pattern diversity for effective data selection in agentic SFT.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model, data selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PPMYOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The proposed CATS is a method for selecting to keep which multi-turn trajectory for agentic SFT. It use a small probing model to obtain a quality score for each trajectory. Then each trajectory is transformed a labeled sequence. A similarity graph is built based on the edit distance. The selection is conducted based on the calculated scores. It demonstrates good performance across different benchmarks.

Critical strengths
  1. It provides a method that is scalable and could be applied to real problems.
  2. The performance gains across different base models and budgets are consistent.
Critical weaknesses
  1. The ablation experiments don't reveal that the core components matter.
  2. The comparison with some backbones should be shown in the paper.
Questions for authors
  1. Could you compare the performance with D2 pruning itself?
  2. Why do all variants with single component difference fall below Random? But why quality-only and graph-only ones can match it?
  3. How about the consumed token per selected subset?
2 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part1)

W1. Effectiveness of the core components

Concern: The existing ablations do not adequately establish the contributions of the core components.

Random selection, graph selection alone, quality scoring alone, and full CATS achieve aggregate scores of 31.7, 32.1, 32.0, and 34.7. We further isolate the score and graph through a crossed comparison.

Probe NLL with an embedding graph scores 32.7; CATS scoring with the same graph scores 33.5. With the structural graph, probe NLL scores 33.1 and CATS scores 34.5. ToolRM with the structural graph scores 33.6. Comparisons with the graph held fixed support the scoring contribution, while comparisons with the score held fixed support the structural representation.

The estimated average marginal effects of the score and graph are +1.1 and +0.7 percentage points, respectively. The interaction estimate is +0.6 points. These estimates support component contributions without establishing a statistically significant interaction.

W2. Comparisons across backbones

Concern: The paper should compare CATS with baselines on multiple backbones.

BackboneBest evaluated baselineCATSDifference, percentage points
Qwen3.5-4BLONGEST: 34.134.7+0.6
Gemma-4-12BATLAS: 21.624.0+2.4
Qwen3.5-35B-A3BATLAS: 38.641.2+2.6

CATS exceeds the best evaluated baseline on all three backbones. On Gemma, the advantage also persists under budget controls: CATS and ATLAS score 24.0 and 22.0 under equal supervision, and 23.9 and 22.2 under equal training cost.

Initial checkpoint performance and valid tool call rates can help explain differences between models, but do not replace task success rates. Model comparisons also require explicit checkpoint identifiers, interface templates, and call validation rules. Fewer baselines were evaluated on the MoE model, so our claim is limited to the models and baselines tested.

AuthorsOpenReview ↗

Official Comment by Authors (Part2)

Q1. Direct comparison with D² Pruning

Question: Can CATS be compared directly with D² Pruning?

We adapt D² using conventional probe NLL, an embedding graph, and the same message passing selector, while fixing the candidate pool, backbone, and supervision budget. The D² adaptation scores 43.4 on τ³, 22.0 on Vita, and 32.7 on μAGENT. CATS scores 45.6, 23.4, and 34.5, respectively, improving by 2.2, 1.4, and 1.8 percentage points.

This comparison supports the added value of the proposed score and structural representation within the inherited selection framework. We attribute the selector to D² and distinguish it from our contributions. The comparison applies to the adaptation described here.

Selection takes 16.0 wall hours for D² and 17.8 for CATS; SFT costs are 1,400.0 and 1,333.3 GPU hours. These quantities are reported separately. Converting selection wall time into GPU hours requires the corresponding hardware allocation, and neither cost component alone establishes total efficiency.

Q2. Why some partial variants perform worse than Random

Question: Why do some partial variants underperform Random, while graph selection alone and quality scoring alone perform similarly to Random?

Removing a signal entirely differs from retaining an incomplete signal. A partial score can bias selection toward particular trajectories, while a coarse graph can group trajectories with different behavior. We therefore separate the effects of supervision volume, data distribution, and component design.

ConfigurationOriginal settingEqual supervisionAlso matching length and source
AComp only + full graph28.929.530.3
CDep only + full graph30.530.831.4
Full score + graph without order27.028.229.4
Full score + graph using tools only29.129.830.7
Random31.732.032.2
CATS34.734.534.2

The partial variants recover some performance after matching supervision, length, and source, but remain below Random and full CATS. These factors explain part, but not all, of the degradation.

Among 100 neighboring trajectory pairs from each graph, 76 are behaviorally equivalent for the full graph, compared with 57 for the graph without order and 54 for the graph using tools only. With the graph fixed, permuting the trajectory scores reduces performance from 34.5 to 32.4. These results support the structural representation's ability to distinguish behavior and the importance of assigning scores to the correct trajectories.

Q3. Tokens consumed by the selected subsets

Question: How many tokens does training actually consume for each selected subset?

The full 4B table appears in our response to Reviewer 87xq, Q1. Over five complete passes, the supervised/input totals for Random, LONGEST, and CATS are 2.00/5.00B, 4.50/11.00B, and 2.40/6.00B, respectively. These are conversions from the per-pass counts; actual batch counts take precedence when truncation, resampling, or budget stopping changes consumption.

BackboneSupervised tokens, BInput tokens, BSFT GPU hours
Qwen3.5-4B2.4006.0001,333.3
Gemma-4-12B2.5926.4803,200.0
Qwen3.5-35B-A3B2.4006.0002,051.3

We report supervised tokens, input tokens, and GPU hours separately. The AComp scoring mask is distinct from the SFT supervision mask. Counts across backbones use each model's tokenization and training configuration rather than extrapolating from another model's throughput.

We assess total efficiency through two separate comparisons. At equal total cost, normalized to 100 for both methods, the strongest baseline scores 33.8 and CATS scores 34.7. At equal performance, the normalized costs are 100 for the baseline and 93 for CATS, a 7% reduction. These results support higher performance at the same cost and lower cost at the same performance, respectively.

Total cost includes scoring, selection, and training, with reference generation and environment execution additionally counted for RL. Each comparison is normalized to its own baseline. The efficiency claim applies to the reported budgets and target performance, rather than every possible budget.

Reviewer_87xqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CATS, a data-selection method for agentic supervised fine-tuning based. It gives each trajectory a quality score based on both context dependence and action complexity, estimated using a frozen small LLM. It also constructs a trajectory-similarity graph and uses a message-passing procedure to selects trajectories to balance estimated quality and structural diversity. Authors evaluate CATS by selecting 100K trajectories from a pool of 603K synthetic agentic trajectories. Experiments cover two agentic benchmarks, two general reasoning benchmarks, and three backbone configurations. CATS generally obtains the strongest average performance .

Critical strengths
  • This is an important and well-motivated problem. For agentic task training, how to select SFT trajectories that are most informative and capture diverse action patterns directly affects the training performance and efficiency.

  • The evaluation is comprehensive and shows good performance compared with several related prior work baselines. The paper compares end-to-end selection time and shows a competitive time spend.

Critical weaknesses
  • The 100K trace based comparison does not control the training budget. The paper explicitly notes that supervised-token counts vary with trace length and procedure. CATS may favor complex and context-dependent trajectories. Consequently, differences in downstream performance may partly reflect differences in supervised tokens, or total training compute rather than trajectory selection quality alone. The paper should report more selected data statistics, including supervised assistant tokens, total processed tokens, average trajectory length, and training time for every selected subset. An equal training-compute (especially trainable token count) comparison would complement the current comparison and strengthen the paper.

  • The gain is modest and performance uncertainty is unknown. On Qwen3.5-4B, CATS achieves better performance than LONGEST, but the gain is modest for most benchmarks. The paper does report variation across SFT seeds or average performance across a few saved checkpoints. The range of performance variability remain unknown.

Questions for authors
  1. How many trainable tokens, and how many total tokens are selected for SFT, for the subset selected by each method? Do CATS’s gains remain under an equal-supervised-token or equal-training-compute budget?

  2. How sensitive are the results to SFT random seeds and checkpoint selection? More specifically, is the 0.6-point µAGENT advantage over LONGEST on Qwen3.5-4B statistically reliable?

2 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part1)

W1. Fairness of the training budget

Concern: The gains may come from more supervised tokens or computation rather than better data selection.

We control cumulative assistant supervision and training GPU hours separately, and include random subsets matched in input length, assistant output length, and source distribution. Under equal supervision, CATS, LONGEST, and the matched random subset score 34.8, 33.7, and 33.0. Under equal training cost, they score 34.7, 33.8, and 33.1. CATS retains its advantage in both comparisons.

The same pattern holds on Gemma-4-12B. CATS and ATLAS score 24.0 and 22.0 under equal supervision, and 23.9 and 22.2 under equal training cost. These controls show that supervision volume, training computation, and the matched distributional factors do not fully explain the gains.

We also compare total pipeline cost. At equal total cost, CATS scores 34.7 versus 33.8 for the strongest baseline. In a separate comparison at equal performance, CATS requires a normalized cost of 93 versus 100 for the baseline, a 7% reduction. The accounting is described in our response to Reviewer PPMY, Q3.

W2. Magnitude and reliability of the gains

Concern: The 4B improvement is small, and the paper lacks analyses of training seeds and checkpoint selection.

The original aggregate scores are 34.7 for CATS and 34.1 for LONGEST. On τ³, the scores are 45.9 and 43.6; on Vita, they are 23.4 and 24.5. We report both benchmarks because the aggregate gain includes a tradeoff between them.

The original five training runs yield 34.5 ± 0.44 for CATS and 33.9 ± 0.49 for LONGEST. The interval for their difference is [−0.13, +1.33] percentage points and includes zero. The additional experiments yield mean scores of 34.9 and 33.9, a difference of 1.0 point. We retain both sets of results and describe the additional result as an improvement in mean performance; it does not change the statistical conclusion of the original comparison.

At 80%, 90%, and 100% of training, the mean gaps are 0.5, 0.6, and 0.6 points. The positive gap is therefore not confined to a single checkpoint. The final checkpoint is used for the main comparison, while the others assess sensitivity to training progress and are not treated as independent training runs.

Q1. Token counts and comparisons under equal budgets

Question: How many supervised and input tokens does each method select, and do the gains persist under equal budgets?

Token accounting must follow the tokenizer, template, truncation, packing, and loss mask used by each model. Selected token counts refer to one pass through the subset; cumulative consumption refers to the training batches actually processed. Input token counts include supervised assistant tokens, so the two columns must not be added. Steps excluded from the ATLAS loss may still be processed as context.

The 4B resource counts are as follows. B denotes 10⁹ tokens.

MethodSelected supervised tokens, BSelected input tokens, BSFT GPU hours for five epochs
Random0.4001.0001,010.1
LONGEST0.9002.2003,492.1
Perplexity0.5001.2501,462.0
SuperFilter0.4601.1501,277.8
AlpaGasus0.4201.0501,111.1
ToolRM0.4401.1001,192.4
KMeans0.4001.0001,010.1
InsTag0.4201.0501,111.1
ATLAS0.2001.2501,462.0
CATS0.4801.2001,333.3
FULL, 603,417 trajectories2.4146.0346,095.1

LONGEST contains 1.875 times as many supervised tokens per pass as CATS and requires approximately 2.62 times its SFT GPU hours over five epochs. At a fixed supervision budget of 2.40B tokens, the per-pass counts correspond to approximately 6.00 effective epochs for Random, 2.67 for LONGEST, and 5.00 for CATS. These conversions describe the nominal budget; matching must use the tokens actually processed.

CATS and LONGEST score 34.8 and 33.7 under equal supervision, and 34.7 and 33.8 under equal training cost. The gains persist under both controls.

Q2. Random seeds and checkpoint selection

Question: Is the original 0.6 percentage point improvement reliable?

The original difference interval, [−0.13, +1.33] points, includes zero, so the original improvement is not statistically significant. The additional experiments yield mean scores of 34.9 for CATS and 33.9 for LONGEST. This supports an observed mean improvement of 1.0 point but does not, by itself, establish significance across training seeds.

Checkpoint selection follows the training protocol: the final checkpoint is used for the primary comparison, and intermediate checkpoints are used for sensitivity analysis. Repeated evaluations of the same trained model remain grouped and are not counted as independent training runs. This separates variation due to training, evaluation, and checkpoint selection.

Thank you for the response. The new results on token counts and comparisons address my concern on fairness of the training budget. I will raise score to 3.

Reviewer_JURuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CATS, a trajectory selection method for agentic supervised fine-tuning that combines trajectory scoring with structural diversity. It scores trajectories using a frozen small model’s prediction losses to estimate context dependence and action complexity. It then constructs a similarity graph from ordered sequences of tool calls, natural-language replies, and tool-return statuses, and applies message passing to select trajectories while reducing structural redundancy.

Experiments select 100K trajectories from a pool of approximately 603K and evaluate three model backbones on two agentic and two general benchmarks. CATS achieves the highest reported average agentic and general performance across the tested backbones, improving average agentic performance over the strongest compared baseline by 0.6 to 2.6 percentage points.

Critical strengths
  1. The paper addresses a practical problem in agentic SFT. Considering both the training value of individual trajectories and redundancy in interaction patterns provides a reasonable basis for selecting data under a limited budget.

  2. CATS achieves consistent improvements in average agentic and general performance across three backbones. The results on Gemma-4-12B and Qwen3.5-35B-A3B are particularly encouraging.

  3. The ablations examine the scoring components, structural representation, and graph selection procedure. The budget analysis further demonstrates that the method remains useful as the selected dataset grows.

Critical weaknesses
  1. The message-passing selection procedure follows D2 Pruning, limiting the algorithmic novelty. Comparisons with D2 Pruning and existing quality scores combined with the same structural graph are missing, making it difficult to isolate the contribution of the proposed scoring method.

  2. Selecting the same number of trajectories does not ensure comparable training costs, since trajectory lengths differ. The paper reports neither the training token counts nor the training time for each subset, making the claimed efficiency gains difficult to assess.

  3. The evaluation is limited to selecting demonstrations for SFT. Whether the proposed scores are useful for selecting tasks or rollout trajectories in RL remains untested, limiting the conclusions to the SFT setting.

Questions for authors

Please see the weaknesses.

1 comment
AuthorsOpenReview ↗

Official Comment by Authors (Part1)

W1. Contribution beyond D² Pruning

Concern: CATS inherits the D² selector, but lacks a direct comparison and an isolated evaluation of its scoring function.

CATS adopts the message passing selector from D². We will make this attribution explicit. Our contributions are the context dependency and action complexity scores for agent trajectories, together with a structural representation that preserves interaction order. To isolate their effects, we hold the candidate pool, selector, backbone, subset size of 100K trajectories, and supervision budget fixed, and vary only the score or graph representation.

ScoreGraph representationμAGENT
Probe NLL, D² adaptationEmbedding graph32.7
CATSEmbedding graph33.5
Probe NLLStructural graph33.1
ToolRMStructural graph33.6
CATSStructural graph34.5

Full CATS exceeds the D² adaptation by 1.8 percentage points. With the structural graph fixed, CATS scoring improves over probe NLL and ToolRM by 1.4 and 0.9 points, respectively. With CATS scoring fixed, the structural graph improves over the embedding graph by 1.0 point. These comparisons support contributions from both the scoring function and the structural representation beyond the inherited selector.

The interaction effect has a point estimate of +0.6 points. We distinguish this estimate from evidence of a statistically significant interaction.

W2. Training tokens and computational cost

Concern: Equal trajectory counts do not imply equal training budgets. Token counts and training time are missing.

We agree that trajectory count alone is insufficient. For Random, LONGEST, and CATS, the supervised/input token counts per pass are 0.40/1.00B, 0.90/2.20B, and 0.48/1.20B, respectively. Five epochs of SFT cost 1,010.1, 3,492.1, and 1,333.3 GPU hours. The 17.8 hours in the original Table 3 measure wall time for scoring and selection only, not SFT cost.

SettingBaselineCATSDifference, percentage points
4B, equal supervised tokensLONGEST: 33.734.8+1.1
4B, equal training GPU hoursLONGEST: 33.834.7+0.9
Gemma, equal supervised tokensATLAS: 22.024.0+2.0
Gemma, equal training GPU hoursATLAS: 22.223.9+1.7

On 4B, random subsets matched to CATS in length and source distribution score 33.0 under equal supervision and 33.1 under equal training cost. CATS exceeds these controls by 1.8 and 1.6 points. The gains therefore persist after controlling supervision, training cost, and these distributional factors, and appear on both backbones. Total pipeline costs are discussed in our response to Reviewer PPMY, Q3.

W3. Applicability beyond SFT

Concern: The use of CATS for RL task selection and rollout selection has not been established.

We evaluate offline RL task selection and online rollout selection separately. Offline task selection requires executable, resettable environments with reliable rewards; the static Nemotron trajectories alone do not provide such an environment. A controlled comparison requires shared reference trajectories, initial policies, GRPO implementations, evaluation tasks, and generation budgets.

In the original experiment with a 50M token budget, CATS and Random both reach 39.0% success, while the strongest baseline reaches approximately 39.3%. We retain this result alongside the additional experiments.

EvaluationBaselineCATS resultBudget accounting
Final success rateStrongest baseline: 39.3%40.8%Equal compute budget
Cost to reach the same success rateBaseline total cost5% to 10% lowerIncludes selection and execution costs

In the additional offline task selection experiment, Random, the strongest baseline, and CATS reach 39.0%, 39.3%, and 40.8%, respectively. CATS improves over the strongest baseline by 1.5 percentage points. Separately, the learning efficiency experiments show a 5% to 10% reduction in total cost to reach the same success rate. This cost includes reference generation, scoring, environment execution, and training.

Online selection retains complete GRPO prompt groups, controls total generation and update budgets, and counts the generation cost of discarded rollouts. Random group selection, difficulty based selection, and CATS achieve 39.4%, 39.8%, and 40.7% success, respectively. CATS improves over these baselines by 1.3 and 0.9 points.

Each improvement is measured against the baseline in the same experiment. We do not subtract CATS scores across the original and additional experiments to estimate a method effect. With approximately 500 evaluation tasks, one percentage point corresponds to about five tasks, so these results establish observed performance differences rather than statistical significance or a general guarantee across RL settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.