- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
TutelaBench evaluates LLM-based planners in simulated elderly-care and child-care environments. It combines scripted human interactions, conflicting requirements, evolving environmental events, and safety failures caused by both actions and omissions. Task performance and safety compliance are scored separately from simulator states. Experiments with eight open-weight models and five agent architectures show that relatively high task performance does not imply consistently safe behavior, and that explicit guidelines and critics can improve repeated-run safety.
Critical strengths
- The benchmark is a meaningful extension of safety evaluation to care obligations. It includes omissions, proactive intervention, social pressure, and conflicting instructions. These features capture important failure modes that evaluations focused on isolated dangerous requests can miss, making the benchmark a contribution.
- This paper introduces a benchmark that evaluates task performance and safety compliance as separate dimensions. Separating task performance from safety compliance prevents task completion from obscuring violations. Simulator-based scoring makes outcomes inspectable, while repeated-run metrics expose inconsistency that average scores alone would hide.
Critical weaknesses
-
Scenario and scoring validation are insufficiently established. The main text does not adequately explain how requirements are translated into scoring conditions, how acceptable alternative behaviors are handled, or how scenarios are verified to admit a fully compliant solution under their observation and timing constraints. This matters particularly for omissions and proactive behavior, where failures should reflect missed opportunities based on available information.
-
Model comparisons are confounded by changes in the interlocutors. Section 3 states that the evaluated model also generates residents’ free-text replies. Changing the planner therefore changes part of its interaction environment. State-machine constraints improve control, but differences in dialogue clarity or informativeness may still influence outcomes. The use of LLM-based dialogue-transition classification introduces another potential source of variation. Without evidence that results are robust to a fixed interlocutor and transition classifier, the reported differences cannot be attributed solely to planner capability.
-
The architecture comparisons do not isolate the sources of improvement. Critic-based agents receive additional inference calls, while simulated time is decoupled from computation time. Their gains therefore combine guideline auditing with additional deliberation that incurs no simulated delay. Comparisons under matched inference budgets are needed to separate these effects. Likewise, the Robot architecture jointly adds memory, review, and self-checking, leaving their individual contributions unclear.
Questions for authors
-
What validation establishes that each scenario has a fully compliant solution under the available observations and deadlines, and how are alternative compliant behaviors recognized?
-
Which model performs dialogue-transition classification, and is there evidence that model rankings remain stable with a fixed interlocutor and classifier?
-
Do critic-based improvements persist under comparable inference budgets?
3 comments
Thank you for the detailed response. Some of my concerns remain. The added checks do not establish that every scenario has a fully compliant solution or that reasonable safe alternatives receive credit. Also, the fixed-interlocutor rerun is complete for the baseline tasks, but not yet for the safety-critical results. I may be missing something, but based on the current evidence, I cannot rate the paper above 3.
Thank you for the follow-up. As noted in our general response, the fixed-interlocutor rerun is now complete for the safety-critical suite as well, and the conclusions hold. On your two remaining concerns:
Every scenario has a fully compliant solution. We now check the strictest criterion: a run with full task credit and no safety violation at all. Every one of the 14 baseline and 33 safety-critical scenarios has such runs, each produced by at least 7 of the 40 model–architecture configurations and at least 5 of the 8 models. These are actual benchmark runs under the real observations and deadlines, so a compliant solution demonstrably exists.
Reasonable safe alternatives receive credit. Scoring reads the simulation state and never compares against a reference trajectory. A safety requirement fails only when a stated violation occurs, e.g. medication mixed into food. Duties are checked by their effect, e.g. "the physician knows about the refusal", so it does not matter whether the agent messages him or walks over, or how it phrases the report. An alternative loses credit only where the requirement itself prescribes a procedure, such as checking the order sheet before giving a drug; there, the requirement rules it out, not the scorer. Two elderly-care practitioners who reviewed five scenarios judged none of their 21 requirements wrong (Appendix, Practitioner Review).
We hope this addresses your remaining concerns and would be grateful if you would reconsider your rating.
Dear Reviewer,
Thank you for your thorough review and the helpful comments. We address each of your points below.
{Likewise, the Robot architecture jointly adds memory...} We agree that evaluating additional architectures would strengthen the paper. However, it is not feasible to cover all possible architectures, so we focused on those we considered most relevant and informative. We would also like to clarify that the goal of our paper is not to isolate or quantify the contribution of individual components. Instead, we aim to show that all evaluated methods, regardless of their sophistication, fail to behave safely. Since even the most elaborate configurations exhibit these failures, we do not expect other configurations to resolve them. Moreover, several components are only meaningful in combination. For example, the review step relies on the memory system to function properly, so evaluating it in isolation would not result in a sensible configuration.
{Clarify how requirements translated into scoring condition.} We are happy to clarify the benchmark setup and the scoring conditions. In TutelaBench, all safety goals are weighted equally, since the violation of any safety-critical goal indicates unsafe behavior and is thus unacceptable. Assigning them different weights would relativize the absolute importance of each. For safety scenarios, the only goal contributing to the task score is typically whether the system keeps the person involved by talking to them rather than resolving the safety issue silently. For non-safety scenarios, the task is typically more complex, and the task score is distributed across several sub-goals.
{1.} We believe that this is one of the crucial points of TutelaBench. This is why the correctness of the scenarios is verified by humans. Each scenario is also checked statically (every condition refers to an existing state, object or location). The time limits are generous: completing a task correctly typically takes only about 50% of the allotted time, so low scores are not caused by strict deadlines. To verify they are solvable for the agents we only use scenarios which least once have full score, independent on agent and model. During the design phase we tried to limit the space of alternative compliant solutions, focusing on an obvious guideline following solution.
We agree that safe behaviors other than the prescribed ones may exist. TutelaBench measures whether a planner reliably complies with the requirements it has been given, in a world with other people and a running clock. We do not claim that our rule set is the correct standard of care; standards differ between institutions and jurisdictions, which is why requirements are configurable. We do claim that reliable compliance with stated requirements is a necessary condition for deploying an LLM planner in care, because a deployer can only vet, certify, and oversee behavior that was specified in advance. This is also why we do not accept alternatives where a requirement prescribes a procedure. Contacting the prescribing physician in the medication example is a reasonable action, but it replaces the prescribed check with the agent's own judgment, and an agent that does so in one run gives no assurance about the next. Human care work has the same structure: verifying a medication against the order before administering it is mandatory however experienced the caregiver is. We will make this scope explicit in the introduction and in Section 2, and, in the released scenario files, mark each requirement as procedural or outcome-based so that users can report the two separately.
\textit{2. Which model performs dialogue-transition classification, and is there evidence that model rankings remain stable with a fixed interlocutor and classifier?}
In the first iteration the model under test performed the dialogue-transition classification. We reran the experiments with the simulated humans, both their replies and the dialogue-transition classifier, served by one fixed model (Qwen3.6-27B), independent of the model under test, and raised the runs per configuration from 10 to 15. The baseline suite is complete (8 models, 5 architectures, 14 scenarios, 15 runs: 8,400 runs) and is reported in the updated Table 3 and the appendix tables; the safety-critical suite is still running and will be added to the final version. For more see Q2 from canU.
{3.} We did not isolate these two factors, and we agree that the critic’s gain reflects both guideline auditing and the additional deliberation it allows at no cost in simulated time. We will clarify this in the results section and no longer attribute the gain to auditing alone. Importantly, this does not affect our main conclusion. Even the critic architecture, which uses roughly 2.5 times as many LLM calls as plain ReAct, still violates at least one requirement in about a third of its runs.