- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies why agentic safety fails to generalize across tasks. The authors first provide theory on an analytically tractable model, namely the linear-quadratic control with H∞-robustness setting of Section 2.3. In this setting, within the mapping from the task parameter Q to the optimal controller, the Lipschitz constant under safety requirements can be arbitrarily larger than without safety requirements; the two mappings are denoted Ksafe and Kunsafe. This result is given by Theorem 1 in Section 3.3, with the global explicit upper bound on Kunsafe in Lemma 1 of Section 3.2 as an intermediate step. The authors argue from this that cross-task generalization of safety is structurally more difficult, independently of how good the training method is. Experiments validate this in three settings, namely linear-quadratic control in the infinite-sample and finite-sample regimes, neural-network quadcopter navigation with two safety variants covering banned regions and disturbance handling, and an LLM agent in a CRM benchmark, with a LLaMA-3.2 student and a GPT-5.2 teacher. Table 1 shows that imitating safe and unsafe teachers is similarly easy on training tasks, while on unseen tasks the safe teacher is clearly harder to generalize.
Critical strengths
1.A core contribution of this paper is to elevate "safety generalizes more poorly" from an empirical observation to a provable structural statement. The infinite-sample rows in Table 1, rows 1 to 4, give a separation of 1265.53 versus 14.16, with a design matching the model of Section 2.3, and rule out insufficient training as an explanation. This is a main ground for my support of acceptance. 2.Lemma 1 stands independent of the main conclusion. It gives a global explicit Lipschitz bound for the mapping from Q to the LQR, see Eq. 3, a useful complement to the classical local or implicit characterizations. Section 4.1 further shows the conclusions survive when the assumptions are violated, which is a treatment that deserves credit. 3.The theoretical part of the appendix is complete and self-contained, and it lays out the theorem's conditions in a form that can be checked item by item. The proofs supply the intermediate steps, with separate lemmas for the contraction property of the Riccati operator and for the bounds on its solution, and the background on LQR and H∞ is developed from first principles in Appendix B. This lets my concerns about the theorem's conditions be traced to specific statements rather than left as conjecture.
Critical weaknesses
1.The theorem's conclusion is narrower than the title suggests. Section 3.3 itself concedes that the separation coefficient is not always greater than one. Together with the commutativity assumption, the theory in effect covers only systems that are essentially diagonalizable. Whether the headline claim, that safety generalizes more poorly, extends to general systems therefore remains open, and this is the direct reason I cannot give Clear acceptance. 2.The central metric in the empirical part is not scale-normalized, and the coverage explanation is not ruled out. The imitation errors are not normalized by the scale of the target mapping, and safe teachers, which must avoid banned regions, naturally produce demonstrations with narrower state coverage. As a result, "safety is intrinsically harder to learn" and "data coverage is worse" cannot be told apart. The support that the last two experiments provide for the headline claim is weakened accordingly. 3.The evidence in the main text is thin. Only the linear-quadratic setting has a figure, while the quadcopter and CRM settings appear as a single table row. The paper's claim is about a phenomenon, yet readers can verify it only through one aggregate number, and learning curves or example trajectories would be more convincing.
Questions for authors
1.Can you provide a hand-checkable example, with A, B, D, R listed, for which the separation coefficient exceeds one? 2.Is there a parameter example in which the Lipschitz constant of Ksafe is smaller than that of Kunsafe? If so, where is the boundary of the separation result? 3.Safe and unsafe teachers have different demonstration distributions by construction. Can a distribution-aligned control separate mapping complexity from coverage? 4.After switching to relative error normalized by the target scale, which of the three settings still show the separation? 5.The Lipschitz result predicts that generalization difficulty grows with the distance of a task from the training set. Can the test errors in Table 1 be stratified by that distance? 6.Do both teachers use the same network capacity and hyperparameters? If the safe setting were given more capacity, would the separation shrink? 7.If the hard avoidance constraint is replaced by a safety penalty in the objective, does the separation persist? That would separate the role of hard constraints from that of safety-informed data. 8.CRM is the only modern agentic setting, yet its test gap is the smallest. Can that setting support a universal claim? The following are suggestions for future work: Extending the separation theorem to the non-commuting matrix case.
4 comments
Dear Reviewer mC6T,
Thank you again for your review. As the discussion period ends on November 18, we wanted to kindly check whether our response has addressed your concerns. If so, we would greatly appreciate it if you would consider raising your score. We would be happy to clarify anything further.
Best wishes,
Authors
Thank you for the detailed response. On the experiments, the control in the linear-quadratic setting is clean and the coverage question is handled there. The empirical part is still thin, in that each setting is reported at a single operating point, so the evidence shows that the two targets differ in difficulty rather than that the safety requirement accounts for the difference. I find the experimental side not yet complete, and I am keeping my rating at 2.
Thank you for the follow-up. As you note, the linear-quadratic setting is cleanly controlled, and our other settings follow the same design: both students face the same systems and tasks with the same learner, and imitate (near-)optimal teachers that differ only in whether they ensure safety. It is therefore the safety requirement that accounts for the difference in difficulty between the two targets.
Thank you again for the constructive discussion.
Thank you for your detailed review and your support for acceptance, and for recognizing that our work elevates "safety generalizes more poorly" "from an empirical observation to a provable structural statement." You noted that what stands between our work and clear acceptance is whether our headline claim "extends to general systems." We have added results that directly support this (marked in blue in the revised PDF), as detailed below. We hope that, together with the strengths you describe as "a main ground for [your] support of acceptance," this allows you to recommend clear acceptance.
First, the new Remark 1 characterizes exactly when the coefficient of Theorem 1 exceeds one. This regime is broad, e.g., including all admissible systems when A, B and R are aligned and control authority is limited. Second, the new Theorem 2 shows that on the tasks diagonalized by the eigenbasis of the system, under a mild non-degeneracy condition, the Lipschitz constant is more than twice as large with safety requirements as without. We have also adjusted the phrasing of our claims accordingly.
Theorem 2 also yields a hand-checkable example. In one dimension, it gives Lip(Ksafe)≥(1+1/∣a∣)⋅Lip(Kunsafe), e.g., a ratio of 3 for a=1/2 and b=d=r=1. In closed form, Ksafe(q)=−min{2q,1/2} has slope 2, while Kunsafe(q) is smooth with slope at most ∣ab∣/(r(1−a2))=2/3. The true ratio is thus exactly 3: the bound is tight.
Remark 1 delineates the boundary of our bound, not of the separation: across all linear-quadratic configurations we evaluated, including ones violating Assumptions 1–3, we have not encountered a case where Lip(Ksafe)<Lip(Kunsafe). We thus conjecture that the separation holds in general. The non-commuting case you suggest is a natural next step toward proving it, which we leave for future work.
All errors in the continuous settings are already normalized by the target scale (Appendix I), so all three settings show the separation under this metric. Regarding coverage: in the linear-quadratic setting, both students are trained on the same tasks and states, and the separation persists. In the quadcopter setting, the fraction of states on unseen tasks with a training state nearby (Table 7) differs little between the students, and not consistently in either direction: the risk-avoidance student is best covered at small radii, yet generalizes far worse.
We have added example quadcopter trajectories of both students on seen and unseen tasks (Figure 8).
Our Lipschitz argument bounds the error on an unseen task by its distance from the training tasks, scaled by the Lipschitz constant. Both students are trained and tested on the same tasks, so these distances are identical, and the gap stems from the Lipschitz constant. We therefore test this factor directly: across random linear-quadratic systems, the difference between the two students' generalization gaps (unseen minus seen tasks) grows with the ratio of Lipschitz constants (Figure 2).
In the linear-quadratic setting, the teachers are the exact optimal controllers; in the quadcopter setting, they share an architecture and reach comparable performance; and in CRM, both are the same LLM, differing only in the prompt. Moreover, all data we use comes from tasks the teachers perform well on, so targets are (near-)optimal throughout. Teacher capacity is thus not a factor, and both students share the same architecture and hyperparameters.
The risk-avoidance teacher is in fact trained with a soft penalty for entering banned regions rather than a hard constraint (Appendix I.4), and the risk-handling teacher has no banned regions. Both exhibit the separation.
Our claim is qualitative rather than universal: safety requirements make generalization across tasks harder, even when imitation on seen tasks is equally easy. CRM exhibits this: the safe student's error grows by 57% from seen to unseen tasks, compared to 7% for the unsafe student. CRM errors are also bounded, so gaps are naturally smaller in scale than in the other settings.