- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper propose a joint distribution to solve this question: Given a complete trajectory, how likely is it that it was unsafe? but at runtime you need to ask four different questions. Existing work answer these separately, so the four answers need not to be mutually coherent.
Thus the paper build the missing joint distribution called TTMonitor, p(x,y)= b(x)*w_{theta(y(c=x))}. where w_{theta, y} is the frozen black box teacher monitor's soft score and b(x) is a trajectory distributino estimated from source traces. The main experiments uses beta=1. It uses the first order markov model of agent trajectories, a nonnegative tensor-train model distilled from a frozen teacher monitor, the tensor contractino to condition on observed events and marginalise unobserved events.
On the InjecAgent and Agent Doji, explicit joint models show zero detected cross-query incompatibility, and TT monitor is far more faithful to the teacher than the Markov joints baselines. Query aware distillation cuts runtime fidelity error from 0.028 to 0.007. Separately, exact tensor contraction is faster than the tested monte carlo completion sampling methods and avoids their sampling error.
Critical strengths
1, The problem is good, clear, and practially relevant. The inconsitency exmple is compelling: if every possible resolutino f an unknown event has a risk above 0.6, the unsolved risk cannot be 0.2 under the same joint model.
-
The construction is elegant and tractable: The nonnegativity and stochasticity constraints make the student a valid conditional distribution. Combining it with a trajectory distributino produces a joint model whose conditionals satisfy probability identifies by construction.
-
Useful evidence for query aware distillation: On InjectAgent, query MAE decreases from 0.028 with trace-only training to 0.007 with the dual objective, while competed-trajectory MAE remains 0.004 while qury mae moves from 0.028 to 0.007) is a clean separation of what the architecture buys from what the objective buys.
Critical weaknesses
- The fidelity comparison is confounded by unmatched training objectives. A major weakeness is that the headline comparison suses tragets that TT monitor was explicitly trained to match, while the baselines were not.
The student is normalized over y for each trajectory. With beta=1. Query MAE is therefore not measuring runtime-inference quality at all- it is the competed-trajetory distillation error, reaveraged under the method's own conditional law. And l post is the empirical version of the evaluation metric(N=8 sampels in training, M=8192 at evaluation).Nothing here is undisclosed and model selection did not touch the test data. And the consequence will be the metric cannot distinguish has a good joint from was trained on this loss.
The author should distill the same frozen teacher into a GRU and a Transformer using the same L comp/L post objective and the same cached completions, and rerun both fidelity and the compatibility audit.
-
The motivating decision problem is never evaluated. The experiments reports the Brier, MAE and LP feasibility, and never return to the decision. The closest prior work does exactly this (ProGuard reports advance warning time and unsafe-behaviour reduction at preserved task completion), so the bar exists.
-
b(x) is load-bearing and unvalidated Every marginalization query is an average weighted by b(x) ^bta. b is a first order marlov chain over coarse structural symbols with additive smoothing- and the paper reports no held-out likelihood, perplexity, or comparison against a higher order or neural trajectory model. beta is introduced as a design knob in eq e and then fixed to 1 everywhere; an gamma=0.9 is fixed, unablated, and content-independent.
-
AgentDojo breaks the paper central promise, and the paper under reports it. On InjecAgent, Complete MAE =0.004 promise kept, On AgentDojo is 0.063 and 0.048, with Query MAE0.074/0.055. A systematic 6 point deviation in probability is not preserving the teacher, it is a differnt monitor. And the text says only that the achievvable fidelity depends on the teacher being approximated, which read as an understatement.
Questions for authors
- One decision-level experiment: does inconsistency actually flip interventions, and does the joint fix it. what do distillation matched baselines d? Per A.6 the pooled predictors receive labl supervision only. what are query MAE, joint repair and incompatibility rate for 1)a GRU and ii) a transformer distilled from the same frozen teacher with the same L comp/L post objective and the same cached completiions? This is a control that separates you need an explicit joint from you need the query aware loss.
2: Is the runtime fidelity metric measuring runtime inference, or remeasuring distillation error? (a) Does Query MAE reduce to the completed trajectory distillation gap reweighted by your own trajectory law, and b) is L_post the empirical form of the evaluation metric itself? could you report Query MAE against a reference mu' that your model does not use?
- Does inconsistency change decisions? On the audited query groups where a baseline is incompatible, at what rate does the incomptibilty flip the intervention decision at a realistic threshold? and for q future, can you report detection earliness against a prefix trained monitor? a low number is still a result and would be worth stating plainly