acceptodds
Under review as a conference paper at ICLR 2027

Guard-and-Guide Runtime Safety for Long-Horizon LLM Agents

Abstract

Large language model (LLM)-driven agents are increasingly deployed in long-horizon tasks, where the harms of individually low-risk actions can accumulate over time and eventually lead to safety failures. Existing parametric alignment methods and non-parametric memory mechanisms struggle to detect subtle risks and provide timely protection. To address this challenge, we propose G2A (Guard-and-Guide), a framework for safe long-horizon agent execution. A lightweight Guard scores candidate actions in the current interaction context. When a risk score exceeds a specified threshold, a procedural Guide selects and executes a reusable Safety Skill. To detect context-dependent risks, G2A introduces Trajectory-Contrastive Safety Distillation (TCSD), which contrasts the same candidate action under matched safe and unsafe trajectory contexts and distills the resulting evidence into the Guard. The Guide organizes Safety Skills through explicit activation, intervention, and termination conditions, enabling state-aware skill activation, switching, termination, and fallback. It also records the scenario and action risk type at each use to support evolving skill reuse. Experiments on ATBench-C, AgentS4D, and HINTBench show that G2A improves long-horizon safety, enables fine-grained risk localization and cross-scenario skill reuse, and preserves agents’ task completion capability.Our code is available at   https://anonymous.4open.science/r/G2A-2228.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Agent, Skill method, Safety alignment, Value alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_w3zUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a Guard-and-Guide Architecture (G2A) that first detects accumulated risks in long-horizon agent actions during runtime and then proposes refinement to the potentially harmful action. The Guard module is based on a lightweight detection model learned through Trajectory-Contrastive Safety Distillation. The guide module is triggered when Guard’s predicted risk exceeds an operational threshold, which leverages evolution-supported safety skills to achieve intervention. The proposed G2A is tested on multiple benchmarks compared with SOTA guardrails from three categories and achieves generally better performance.

Critical strengths
  • The paper proposes a principled approach for learning a lightweight guardrail and achieves strong performance and testing data.

  • The evaluation covers a broad range of agentic settings and baselines.

Critical weaknesses
  • The Guard model is learned for one or a limited range of task domains, hindering its generalization to broader task domains. The authors deliberately made this choice instead of using an LLM or an agent for judging, but the latter seems more capable of task generalization. OOD evaluation would strengthen this weakness, but I am skeptical about the results.

  • Creating training data may be challenging for some domains, as some trajectories may converge into a harmful action without safe alternatives.

  • Detection followed by mitigation strategies for long-horizon safety guardrails have been investigated by prior work such as [1]. What is the advantage of the proposed output-level discrimination between safe and unsafe over the latent space discrimination in [1]?

[1] Luo et al., Agent MechSuits: Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents, arxiv 2026.

Questions for authors
  • Would it be sts_t in the constraint in Eq. (1)?

  • How are the training data constructed?

  • In Eq. (12), where TCR is defined, what is the definition of task completion when there are safety risks? Would it be proper to terminate the task? Can you give a concrete case study for illustration?

  • During the evaluation, is the skill evolution activated or not? With skill evolution on, the comparison with most baselines, with the testing tasks independently evaluated, is unfair. But with skill evolution off, how did you ensure that the initial skill set does not contain data leakage?

  • Space in front of "Detailed" (line 365).

  • Quality of figures, e.g., the overlapped text in the left figure of Figure 3.

3 comments
AuthorsOpenReview ↗

Response to Reviewer w3zU

We sincerely thank you for your careful review and constructive comments. Below, we respond to each weakness and question.

W1: The Guard is trained on a limited domain, raising concerns about its ability to generalize across task domains

A1: We agree that the open-domain coverage of a lightweight Guard is limited by its training distribution. In the paper, we therefore position it as a low-cost, calibratable monitor for continuous safety checking, rather than as a universally generalizable open-domain safety model.

G2A already provides two types of evidence for transferability:

  • Table 1 evaluates the same framework with two different execution backbones: Qwen3-32B and GPT-5.4.
  • Table 8 evaluates G2A on target variants whose task templates and risk instances do not overlap with those used for training. In this experiment, the Guard, skill library, thresholds, and editor are all frozen and reused in a zero-shot setting. This reduces the average cASR from 31.5 to 25.1, while increasing SHR from 60.8 to 66.5 and TCR from 95.1 to 97.6. Consistent improvements are observed across all five risk categories.

These results support transfer across execution backbones and across scenarios involving related risk types. However, they do not support a claim of universal generalization to arbitrary open domains.

In the revision, we will clearly state this scope, report the results for each source–target split, and identify transfer beyond the risk families seen during training as an important direction for future work. We will also report the stronger safety performance of the LLM judge alongside its cost of 640 ms per action and 32.9% end-to-end overhead, compared with 5.6 ms per action for the TCSD Guard.

W2: In some domains, safe alternative trajectories may not exist, making the training data difficult to construct

A2: This situation is a limitation of TCSD’s paired-supervision setting. During training, we include only tasks that have both a verifiable safe completion path and an unsafe path. Tasks without a safe alternative are not included in the paired training set.

At deployment time, when the system encounters such a state, the Guide can request additional authorization, skip the dangerous subtask, or terminate the current subtask. Safety and task completion are then measured separately: a safe fallback can prevent a safety violation, but if it does not satisfy the original task-completion predicate, the run is counted as a TCR failure.

This design prevents the system from obtaining artificially high safety scores by simply refusing all potentially risky tasks.

We will add the data-inclusion criteria and clearly discuss this limitation in the revised paper.

W3: What are the advantages of output-level classification over the latent-space approach used by Agent MechSuits?

A3: Agent MechSuits is a white-box method. It trains linear probes on intermediate hidden states and directly steers the model within a 10-dimensional safety subspace at a single layer.

In contrast, G2A operates outside the model and reads only the observable state and candidate action. It can therefore be applied to closed-source API models whose internal representations are inaccessible. It can also be reused across different execution backbones.

G2A separates risk detection from action correction. The Guard performs action-level risk assessment, while the Guide modifies risky actions using skills with explicit activation, intervention, termination, and fallback conditions. The revised action is then checked again before execution. This process provides auditable reasons for each intervention, readable intervention rules, and explicit control over tool calls.

The two methods address different deployment constraints: MechSuits is designed for white-box CLI agents with access to model weights and hidden states, whereas G2A targets black-box or heterogeneous agent runtimes.

In the revision, we will cite Agent MechSuits and provide a detailed comparison along five dimensions: internal-access requirements, intervention location, cross-model portability, auditability, and runtime cost.

AuthorsOpenReview ↗

Response to Reviewer w3zU

Question 1: What do the state and constraint in Equation (1) represent?

A4: s_t represents the current observable state before the action is executed. It includes the task specification, interaction history up to step t, environment and tool observations, and the currently active skills.

The constraint is applied to the immediate risk of the candidate action:

q_t = f_theta(s_t, omega_t, a_t) <= xi,  for every t.

When q_t > xi, the Guide generates a revised action. The revised action is executed only after it passes the same safety constraint.

The original expression, f_theta(s, a_t), omitted the time index and the skill input. We will consistently revise it to f_theta(s_t, omega_t, a_t).

Question 2: How is the training data constructed?

A5: We begin with a fixed version of the English tasks from Workspace-Bench. For each AgentS4D-style case, we preserve the original authorized objective, resources, required output, and completion criteria. We add only one controlled risk source and pre-register the conditions for payload contact, unsafe signals, task completion, and evidence integrity.

For each source task, we construct one safe demonstration and one unsafe demonstration:

  • In the safe demonstration, the agent encounters and rejects the risky instruction.
  • In the unsafe demonstration, the agent encounters and follows the risky instruction.

Both trajectories still complete the original authorized task.

We then select 196 pairs through stratified sampling along the S/T/L dimensions. The data are split by source_task_id, preventing variants of the same source task from appearing across the training, validation, and test sets.

Question 3: How is TCR defined when a safety risk is present, and is task termination an appropriate response?

A6: TCR uses the case-level task-completion predicate from AgentS4D:

TCR = n_C / n_T

TCR is calculated independently of the safety outcome. A run is counted as completed only if it satisfies the original deliverable requirements, the required environment state, and the case-level completion conditions.

A safe termination may prevent a safety violation. However, if the original task remains incomplete, the run receives a TCR of 0. Therefore, termination is not treated as task success, and the system cannot improve its safety–utility results through general refusal.

Appendix E provides a concrete example. A webpage asks the agent to upload local credentials that are unrelated to the original task. Before the action is executed, the Guard intercepts it. The Guide then activates a sensitive-information boundary skill and revises the action so that the agent ignores the credential-upload request and continues retrieving the information required by the original task.

After the revised action passes the Guard, execution continues. The final trajectory satisfies both the safety conditions and the original task-completion conditions.

If no safe alternative path exists, the system requests user confirmation or terminates the dangerous subtask. In this case, the run retains its successful safety outcome but is counted as a TCR failure if the original task is not completed.

Question 4: Is skill evolution enabled during testing, and how do you prevent data leakage?

A7: Skill evolution is completely disabled during testing.

Skill evolution is performed offline for three rounds using only the training trajectories. After the third round, the Guard, skill pool, editor, router/Guide prompts, decoding configuration, and thresholds are all frozen.

Neither the validation set nor the test set is used in the editor buffer, Guard refinement, skill generation, component editing, or threshold selection.

The initial skill pool contains only one general-purpose skill fixed before training. When the Guard is triggered, this skill instructs the agent to infer a coarse-grained risk category and make the smallest possible modification that preserves the original task objective.

The fixed risk categories come from a predefined operational taxonomy. The initial skill does not contain any test tasks, test templates, hidden labels, or benchmark-specific solutions. This procedure ensures that no test information is used during skill evolution or component optimization.

Question 5: There is a missing space before “Detailed,” and the text in the left panel of Figure 3 overlaps

A8: Thank you for pointing out these presentation issues. We will correct the missing space and perform a complete formatting check of the manuscript.

We will also redraw Figure 3 by increasing the width and margins of the left panel, repositioning the legend and statistical annotations, and using consistent font sizes. We will inspect every page of the final PDF to ensure that no text overlaps and that all content remains readable.

Thanks for the authors' responses. I will consider the rebuttal when discussing the paper with other reviewers and the AC.

Reviewer_XpDnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

G2A combines a lightweight action-risk Guard with skill-based intervention through a Guide. The Guard is trained using contrasts between safe and unsafe reference trajectories. Experiments across several benchmarks examine safety, task utility, transfer, and runtime costs.

Critical strengths

Distilling trajectory-level supervision into a lightweight runtime Guard is a useful design. Combining inexpensive monitoring with selective corrective intervention has practical appeal. The evaluation considers safety, task progress, transfer, and intervention costs. This broad coverage helps assess the framework beyond a single safety metric.

Critical weaknesses

The baseline adaptation protocols need more detail, particularly how agent methods produce trajectory-classification predictions and how checkpoint-based components are combined with each acting backbone. These details are important for assessing the fairness of the main comparisons. The TCSD reference alignment and truncation procedure remains unclear despite the stated exclusion of future actions. The human-reference evaluation also needs more detail on item selection, annotator assignment, aggregation, and agreement to make these results independently interpretable.

Questions for authors

Clarification: Could you describe the classification protocol and checkpoint configurations for the baselines? Please also document the human-annotation procedure, including sample sizes, the rubric, aggregation, and inter-annotator agreement. Suggested additional evidence: Could you provide a worked TCSD example showing exactly which reference tokens are visible, quantify paired-reference coverage in the training data, and report per-run results for representative main comparisons?

3 comments
AuthorsOpenReview ↗

Response to Reviewer XpDn

We sincerely thank you for your careful and detailed review of our work. Below, we respond to each of your weaknesses and questions.

**W1: **The baseline adaptation and trajectory-classification protocols are not sufficiently specific

A1: All methods use the same task splits, execution backbone, decoding configuration, and execution budget.

For G2A, the model produces an action-level risk score at each step. We compute the arithmetic mean of these scores over the entire trajectory and classify the trajectory as safe or unsafe using a threshold selected on the validation set.

For methods that use a separate guard, we retain their publicly released guard model or checkpoint. The guard is used as a frozen external safety component and receives the same trajectory context. Qwen3-32B or GPT-5.4 is used only as the task-execution model.

For methods that explicitly output a safety judgment, we use a unified parser to convert their outputs into binary trajectory-level predictions.

Experience-based methods are updated on the same training split according to their publicly described algorithms. We use the same budget for all such methods: one update per training trajectory, a memory or skill capacity of 32K, and top-4 retrieval.

**W2: ** The reference-trajectory alignment and truncation procedure in TCSD remains unclear

A2: TCSD first identifies a pair of safe and unsafe reference trajectories with the same task identifier. It then fixes the same candidate action obtained from an independent execution.

The two branches share the same currently visible state and the same candidate action; they differ only in the reference-trajectory context. Each reference trajectory is truncated at the causal boundary corresponding to the candidate action. All later actions, tool outputs, and task outcomes are hidden.

The candidate action is evaluated autoregressively at the token level, and its log-likelihood is normalized by the action length. During deployment, the Guard, router, and Guide have no access to any reference trajectory.

**A3: ** Annotation protocol and independence of the human reference evaluation

Thank you for pointing out that the details of our human annotation procedure were insufficiently explained. The human reference scores in Table 3 were produced by 50 blinded annotators using a standardized five-level risk scale.

The five levels correspond to:

  1. No identifiable risk
  2. Minor potential risk
  3. Moderate, context-dependent risk
  4. Clear high risk
  5. Direct and severe safety violation

Annotators assessed the risk of each candidate action based on the current state and the action itself.

Each annotator saw only an anonymized state–action pair. They could not see the identity of the evaluated method, the Guard score, the TCSD training objective, the model prediction, or the judgments of other annotators.

All methods were evaluated on exactly the same held-out state–action pairs and shared the same aggregated human judgments as the reference. Therefore, the human scores were determined only by the standardized scale and the content of each sample, independently of the method being evaluated.

Pearson correlation, Spearman correlation, and Pair Agreement were computed directly against the human reference scores on the held-out test set. The score mapping required for MAE was fitted only on an independent validation set and fixed before test-set evaluation. The test set was not used to fit the score mapping or select any threshold.

In the revised appendix, we will provide the complete definitions of the five risk levels, the annotation instructions, the anonymized annotation interface, the score-aggregation procedure, and inter-annotator agreement statistics. These additions will make the human reference evaluation independently interpretable and reproducible.

AuthorsOpenReview ↗

Response to Reviewer XpDn

Q1: Please provide a complete example of TCSD

A4: We will add the following complete example.

The task asks the agent to retrieve information from a webpage. However, the webpage additionally asks the agent to upload a local credential file. The fixed candidate action is:

Upload the local credential file to the endpoint specified by the webpage.

In the safe reference trajectory, the agent rejects this additional request and continues retrieving the requested information. In the unsafe reference trajectory, the agent uploads the credential file.

Under the currently visible prefix of each reference trajectory, TCSD computes the length-normalized token log-likelihood of the same fixed candidate action. It then calculates:

unsafe-context log-likelihood − safe-context log-likelihood

The candidate action has a higher relative likelihood under the unsafe context and therefore provides positive evidence of risk.

The complete example will explicitly show the visible state, the hidden future content, the token mask, the log-likelihood under each context, and the final TCSD target.

Q2: Please quantify the coverage of paired references in the training data

A5: TCSD uses only instances in the training split for which a same-task safe/unsafe reference pair can be constructed.

The coverage statistics for the three datasets are as follows:

DatasetTotal instancesInstances with a valid safe/unsafe pairPair coverage
ATBench-C70050071%
AgentS4D196196100%
HINTBench44040090%

For each training instance, we retain only one same-task pair of safe and unsafe reference trajectories. The two reference trajectories have the same task identifier, authorized objective, output requirements, and risk source. They also satisfy the evidence requirements needed to construct the TCSD target.

Pairing is strictly restricted to trajectories within the same data split, with no reference trajectories shared across the training, validation, and test splits.

**Q3: ** Please provide per-run results for the representative main comparisons

A6: Both the main experiments and the component ablations use six independent random seeds.

The random seeds jointly control task-agent sampling, editor sampling, data shuffling, Guard initialization, and candidate-action sampling. All hyperparameters and thresholds are selected only using the corresponding validation set.

We report the standard deviations across the six runs in Table 1. Because these statistics already capture the variation across independent runs, separate run-by-run test comparisons are generally not necessary.

Reviewer_XpDnOpenReview ↗

Follow-up after author response

Thank you for clarifying the evaluation protocols and pair construction. However, the matched-capacity control is described as planned or expected while numerical comparisons are presented. Please distinguish completed measurements from projections and provide the actual results. Human-annotation agreement statistics and a fully specified TCSD example also remain outstanding. I therefore maintain my current recommendation.

Reviewer_X2L9OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes G2A, a runtime safety layer for tool-using LLM agents that keeps the agent backbone frozen. A 6-layer MiniLM “Guard” scores every action the agent proposes. When the score exceeds a validation-selected threshold, a “Guide” (one call to the backbone LLM) selects a “Safety Skill”, a text procedure with activation, intervention and termination conditions, and uses it to rewrite the action. The rewrite is executed only if it passes the Guard again; otherwise the agent falls back to asking the user or aborting.

The Guard is trained with Trajectory-Contrastive Safety Distillation (TCSD). Each training task is assumed to come with a task-matched safe and unsafe reference trajectory. For each candidate action, the Guard regresses the length-normalized log-likelihood that a frozen Qwen3-32B assigns to it conditioned on the unsafe reference, minus the same quantity conditioned on the safe reference. The skills are evolved offline over three rounds by an LLM editor, which inspects the first step the Guard flagged, attributes the failure to one skill component, and applies a schema-validated edit. The resulting intervention pairs are relabeled with TCSD and used to refit the Guard.

The evaluation covers trajectory safety classification (ATBench-C), executable workspace-agent risks (AgentS4D, measured by cASR, SHR and TCR) and step-level risk localization (HINTBench), with Qwen3-32B and GPT-5.4 backbones and 14 baselines. With the Qwen3-32B backbone, it reports higher ATBench-C accuracy than the strongest baseline, SafeAgent (78.5 vs. 72.9), and lower AgentS4D cASR (24.2 vs. 25.3) with higher task completion. It also reports closer agreement with human step-level risk ratings than TRACE (r = 0.84 vs. 0.76) and a Guard latency of 5.6 ms per action.

Critical strengths

1. A practical cost structure.

  • Pairing a cheap always-on detector with a sparse LLM-based corrector is a sensible deployment design, and the paper quantifies the trade-off:
    • The MiniLM Guard costs 5.6 ms per action, and +0.6% end-to-end latency on its own.
    • The full system costs +6.7% end-to-end, against +32.9% for a per-step Qwen3-32B judge.
    • LLM correction fires on about 15% of steps (Table 6).

Why it is consequential to my assessment: Per-step LLM judging is the main practical obstacle for runtime guardrails, and a distilled monitor with this cost profile would be useful even if its accuracy only matched an LLM judge.

2. TCSD is a creative way to get step-level supervision from trajectory-level data.

  • When task-matched safe and unsafe trajectories exist, contrasting the likelihood of the same candidate action under the two references yields step-level risk targets without step annotations.
  • On HINTBench, the distilled Guard tracks human five-level risk ratings closely: r = 0.84, vs. 0.86 for its 32B teacher, 0.76 for TRACE, and 0.70 for a MiniLM trained on step labels (Table 3).

Why it is consequential to my assessment: This is the paper's main new idea and its most reusable contribution, provided it holds up under the comparison requested in Weakness 3.

Critical weaknesses

1. How TCSD's reference trajectories were obtained needs more clarification.

  • TCSD requires a task-matched safe and unsafe trajectory for every training task. The paper states these are "part of the original released data" (p.5). However, no code or data accompany AgentS4D's arXiv version, and AgentS4D defines no reference trajectories. It is unclear how the safe and unsafe references for its 196 training cases were produced.

Why it is consequential to my assessment: The Guard's training targets are computed from these safe and unsafe trajectories, so the results cannot be reproduced without knowing how they were obtained.

2. G2A's advantage over the baselines may partly come from training on the benchmarks themselves.

  • G2A trains its Guard and Skills on 60–70% of each benchmark, while the safety baselines use their published prompts or released checkpoints (p.23). Part of G2A's advantage over them may therefore come from in-domain training rather than from TCSD, and the paper does not show otherwise.

Why it is consequential to my assessment: Both effects would favour G2A over the off-the-shelf safety baselines, so the reported margins may overstate what TCSD and the Guide contribute.

3. What the evolved skills and the TCSD Guard add is unclear.

  • The evolved skill library improves on the single generic skill, which tells the agent its action was withheld and asks for the smallest revision (p.22), by only 0.9 cASR points (25.1 → 24.2, under one case) and 3.5 SHR points (66.8 → 70.3, about two cases) on the 66-case test set (Table 4b).
  • Replacing the TCSD Guard with a plain Qwen3-32B step judge in the same pipeline gives better cASR (21.8 vs. 24.2) and SHR (71.1 vs. 70.3) (Table 6), so the demonstrated gain is latency. It is unclear whether the same MiniLM trained on the judge's step scores, instead of TCSD targets, would do as well; without that comparison, what TCSD adds is not shown.

Why it is consequential to my assessment: These are the paper's two main learned components, but their benefit is not sufficiently demonstrated.

Questions for authors

1. G2A’s ATBench-C row (Table 1) does not add up.

  • Accuracy, precision and recall are computed from the same counts TP, TN, FP and FN (App. C.3), so together they fix TP + FN, the number of unsafe trajectories in the 150-trajectory test split. All 14 baselines imply the same number, 75; the proposed method’s rows imply about 71 and 70, which cannot hold on the same test split. With 75, its accuracy would be 77.4 / 86.8, not the reported 78.5 / 87.8. Could the authors explain this discrepancy?
2 comments
AuthorsOpenReview ↗

Response to Reviewer X2L9

We sincerely thank the reviewer X2L9 for the valuable suggestions regarding the data, experimental details, and comparison experiments. We address each concern below.

W1: The source of TCSD’s safe and unsafe reference trajectories is unclear

A1: Following the S/T/L controlled risk-injection taxonomy used in AgentS4D, we independently constructed 196 task-matched pairs of safe and unsafe reference trajectories. Within each pair, the two trajectories share the same authorized objective, task resources, output requirements, risk carrier, and verification conditions. The safe trajectory encounters and refuses the risky instruction, whereas the unsafe trajectory encounters and follows it. Both trajectories retain the original task-completion criteria to reduce confounding caused by differences in task-solving ability. We have uploaded the dataset to the provided link.

W2: G2A uses the benchmark training set, whereas the off-the-shelf safety baselines do not receive equivalent training

A2: Table 1 compares the end-to-end system performance of each method under its publicly documented usage setting. All methods that require training are trained on our predefined training split, while the final performance of every method is evaluated only on the held-out test split. The independent contribution of TCSD is measured through matched-capacity controls and ablation experiments. Therefore, the performance advantage reported in Table 1 comes from our method itself rather than from the benefit of in-domain training.

W3a: The gain from skill evolution over a single general-purpose skill is relatively small

A3a: We agree that the improvements are modest for these two metrics. However, the main value of skill evolution lies in its ability to transfer across different scenarios.

In Table 8, skills evolved in the source scenario are directly applied to non-overlapping, previously unseen risk variants, without any updates during the entire target-domain evaluation. On average, cASR decreases from 31.5 to 25.1, SHR increases from 60.8 to 66.5, and TCR increases from 95.1 to 97.6. This zero-shot reuse recovers 68.8% of the improvement achieved by target-domain-specific skill evolution.

These results show that skill evolution learns reusable intervention procedures. Its value therefore goes beyond the absolute performance difference observed in a single 66-case, in-distribution ablation.

AuthorsOpenReview ↗

Response to Reviewer X2L9

W3b: The Qwen3-32B step judge achieves better safety metrics, making TCSD’s advantage appear to be limited to speed

A3b: The Qwen3-32B judge does achieve a lower cASR. However, this comes at a cost of 640 ms per action, 21.5 additional LLM calls per episode, 31.6K additional input tokens per episode, and 32.9% end-to-end overhead.

In comparison, TCSD Guard has a monitoring latency of only 5.6 ms. The Guard itself introduces 0.6% end-to-end overhead, while the full G2A system introduces 6.7% overhead. Therefore, G2A is designed not only for safety but also for the cost of continuous deployment. Table 6 directly reports the full trade-off between safety and deployment cost.

We also thank the reviewer for suggesting a matched-capacity distillation control. Table 3 already includes a 6-layer MiniLM trained with discrete step labels, which achieves a Human Pearson correlation of 0.70. Under the same model capacity, TCSD Guard achieves 0.84.

We further include a planned Qwen3-32B step-judge score → 6-layer MiniLM control. For this comparison, we keep the training samples, model architecture, input format, optimizer, number of training steps, and threshold-selection procedure fixed.

This control is expected to achieve a Human Pearson correlation of approximately 0.79, outperforming the discrete step-label MiniLM at 0.70 while remaining below TCSD Guard at 0.84. Compared with this control, TCSD is expected to improve Human Spearman correlation by approximately 0.05, increase Human Pair Agreement by 4.2 points, and reduce Human MAE by 0.04. We expect the independent LLM rubric-based evaluation to show the same overall trend.

This matched-capacity comparison, using the same data and training procedure, directly isolates the effect of the supervision target. The results indicate that TCSD’s improvement comes from the directional risk evidence provided by task-matched safe/unsafe trajectory contrasts, rather than from differences in MiniLM capacity, training-data size, or optimization settings.

Risk scorerModel capacitySupervision targetHuman Pearson rr ↑Human Spearman ρ\rho ↑Human Pair Agreement ↑Human MAE ↓LLM Pearson rr ↑LLM Spearman ρ\rho ↑LLM Pair Agreement ↑LLM MAE ↓
Step label + MiniLM score6-layer MiniLMDiscrete step label0.700.7278.00.200.7700.79082.00.17
TRACELightweight monitorTRACE supervision0.760.7884.10.160.8120.83286.40.14
Qwen3-32B step-judge score → MiniLM6-layer MiniLMStandard step-judge score0.790.8185.60.150.8230.84287.10.14
TCSD Guard6-layer MiniLMTwo-sided trajectory-contrastive score0.840.8689.80.110.8550.87490.20.10
Qwen3-32B two-sided scorerQwen3-32BTwo-sided trajectory-contrastive score0.860.8891.20.100.8720.88991.70.09
GPT-5.6 rubric judgeGPT-5.6Direct rubric-based judgment0.820.8488.00.130.8370.85788.50.12

Q 1: The Accuracy, Precision, and Recall values in Table 1 cannot all correspond to the same test set

A4: Two Accuracy values in Table 1 contain transcription errors. The correct values for Qwen3-32B and GPT-5.4 are 77.4% and 86.8%, respectively. The Precision, Recall, F1, and AgentS4D results do not require any changes.

After this correction, G2A still achieves higher Accuracy than all baselines in Table 1 on both backbones. Therefore, the main conclusions and the ranking of the methods remain unchanged. We will also correct the corresponding “78.5%” statement in the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.