- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the safety implications of converting natural-language robot commands into structured representations before applying safety rules. The authors argue that this conversion can lose safety-relevant information, allowing euphemistically rewritten harmful commands to evade detection, a phenomenon they call schema laundering.
Critical strengths
A potentially useful empirical observation. The comparison between prose-based policy evaluation and typed policy evaluation raises an important question: whether the benefits of structured safety checks come from the structure itself or from the underlying policy. The reported difference on euphemistically rewritten commands motivates further investigation.
Broad experimental coverage. The paper evaluates multiple safety benchmarks, includes comparisons with both rule-based and learned defenses, and examines adaptive rewriting, ablations, and error cases. The attempt to distinguish in-domain performance from out-of-domain behavior is also useful.
Critical weaknesses
- The methodological novelty of TELOS appears limited relative to its claims
TELOS assembles several well‑established building blocks: conservative union‑based composition of safety signals, deterministic physical‑hazard invariant checks, LLM‑driven harm assessment, perception‑based conditions, and selective defer‑to‑observe outcomes for failed grounding. While these individual components are reasonable, the paper does not sufficiently articulate what constitutes its core methodological advance over prior embodied‑agent guardrails and selective‑classification literature. Specifically, it remains ambiguous whether the contribution lies in individual modules, or in the strict one‑way compose‑release discipline (blocks can only be added; only perceived evidence can lift a block, never textual prompting). This conceptual distinction needs clearer elaboration and positioning against prior work.
- The experimental results do not fully support the broad claims about generalization and superiority
The paper reports competitive performance across several datasets, but the results are more nuanced than the abstract and conclusion suggest.
- The evidence for robustness against adaptive attacks is limited
The adaptive evaluation uses 100 command pairs and an attacker that rewrites refused commands for up to five rounds. TELOS achieves an attack success rate of 5/100, compared with 7/100 for the policy-only baseline.
- Writing and presentation create unnecessary barriers for readers.
Questions for authors
see the Weaknesses