acceptodds
Desk rejected at ICLR 2027

SaplingGuard: A Multidimensional-Profile-Aware Multi-Agent Guardrail for Developmentally Safe Adolescent–LLM Interaction

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

As adolescents increasingly use LLMs in everyday life, ensuring safe and developmentally appropriate responses has become essential. However, existing LLM guardrails primarily target explicit harmful content in isolated prompts or responses and are less effective at identifying implicit, context-dependent developmental risks. To address this limitation, we propose SaplingGuard, a plug-and-play, profile-aware and dialogue-aware guardrail that requires no modification to downstream model parameters. SaplingGuard decomposes adolescent safety intervention into three specialized agents for user profile construction, context-aware risk assessment, and intent-preserving prompt optimization. Together, these agents leverage the current prompt, preceding dialogue, and structured user characteristics to identify contextual risks and guide downstream response generation. We evaluate SaplingGuard on SaplingBench, which contains 276 three-turn dialogues spanning seven categories of developmental risk. Across ten adolescent profile conditions and nine open- and closed-source downstream LLMs, profile-aware retrieval improves the Major Hit rate from to . End-to-end intervention further reduces the average harmful response rate from to and increases the average safety score from to . These results show that user-profile and dialogue context provide complementary signals for identifying implicit developmental risks, and that SaplingGuard can serve as an effective external safety layer for adolescent–LLM interaction.Our code and benchmark are available at https://anonymous.4open.science/r/Sapling-1A07/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLMs, Adolescent, Safety, Guardrail

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_eabJOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SaplingGuard, a plug-and-play, profile-aware and dialogue-aware guardrail that requires no modification to downstream model parameters.

Critical strengths
  1. The paper addresses an important and practically relevant problem: ensuring developmentally appropriate and safe interactions between adolescents and LLMs.
  2. The authors introduce SaplingBench, a new benchmark for evaluating implicit developmental risks in multi-turn conversations.
  3. The overall framework is intuitive and straightforward.
Critical weaknesses
  1. False positives and unnecessary interventions are not evaluated. Although the experiments demonstrate reductions in harmful responses, they do not adequately establish whether these improvements come at the expense of benign interactions. Reporting false-positive rates, together with inappropriate refusals or unnecessary rewrites on benign requests, is important for assessing the guardrail’s practical usability.

  2. Intent preservation and response utility require more direct evaluation. The paper should assess whether rewritten prompts preserve the user’s legitimate intent and whether the resulting responses still meaningfully address the original request. Responses that deviate from the user’s actual needs may be safe while offering limited utility. Moreover, in the component ablation with Llama-3.1-70B as the downstream model, replacing the Optimization Agent with deterministic injection of matched safety policies yields a Harmful Response Rate of 4.46% and a Harmful Dialogue Rate of 10.76%. With the Optimization Agent, these rates increase to 5.27% and 13.33%, respectively. Although the full framework achieves a higher average safety score, it also introduces additional model inference. I would therefore appreciate a clearer explanation of this trade-off, supported by independent evaluations of intent preservation, response utility, and inference overhead, to better establish the practical value of this component.

  3. The reliability of profile construction and maintenance is insufficiently evaluated. Profile awareness is a central feature of the proposed method, but the profile-aware experiments start from ten manually specified initial profiles (Table 5). Section 3.2 explains that fields can be initialized as unknown when prior information is unavailable and subsequently updated through interaction. However, the paper does not separately evaluate the accuracy of profile extraction and updating under this cold-start setting, where all fields are initially unknown. This limits the evidence supporting the Profile Agent’s robustness in realistic, open-ended interactions.

  4. The retrieval scores are not sufficiently specified or analyzed. Although the paper identifies bge-base-en-v1.5 as the embedding model, it does not provide the precise computation of Score_prompt and Score_profile. For example, is cosine similarity used? How is the structured profile P_t serialized and encoded into an embedding? These details can affect candidate-rule retrieval and, consequently, downstream risk assessment, and should be clearly documented. Furthermore, the method linearly combines the prompt-based and profile-based relevance scores, but lacks a systematic sensitivity analysis of α and a clear justification for its default value.

  5. Several aspects of the presentation could be improved.

    a) Equation (12) is identical to Equation (5), while Equations (3) and (10) overlap in their presentation of candidate-set construction at different levels of detail. The mathematical exposition could be streamlined by better integrating the overall workflow with the component-specific definitions.

    b) Figure 1 appears in Section 1, but its first explicit reference in the main text occurs in Section 3. Introducing and discussing the figure where it first appears would improve the connection between the text and the illustration.

Questions for authors
  1. What is the false positive rate (FPR) and over-refusal rate of SaplingGuard on benign adolescent interactions? Under a controlled and acceptable FPR, does the proposed framework still maintain its reported defense effectiveness?

  2. How does the prompt rewriting perform in terms of intent preservation and downstream helpfulness? Specifically, is there a quantitative evaluation ensuring that the LLM's response does not drift to an irrelevant topic merely to achieve safety?

  3. In Table 3 (Panel B), why does the full SaplingGuard yield higher Harmful Response and Harmful Dialogue Rates than the variant using deterministic policy injection? What specific failure modes or interaction dynamics might explain this increase?

Reviewer_gtUoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Existing guardrails, such as Llama Guard and WildGuard, mainly detect explicit harmful content in individual prompts or responses, but often miss implicit risks that depend on the user's background and conversation history. For adolescents, seemingly harmless advice about dieting or family relationships may reinforce unhealthy behaviors.

Therefore, the authors propose SaplingGuard, a plug-and-play guardrail that works with any downstream LLM without modifying its parameters. It contains three different agents: a Profile Agent, a Risk Agent, and an Optimization Agent. The system also carries forward previously identified risks to better handle multi-turn conversations. The main contributions of the paper include the guardrail framework, a taxonomy of seven adolescent-specific risk categories grounded in developmental psychology, and SaplingBench, a benchmark of 276 three-turn dialogues designed to evaluate implicit risks. The authors further evaluate the framework using ten synthetic adolescent profiles and nine downstream LLMs.

Critical strengths

(1) The paper addresses an important problem of implicit, context-dependent safety risks for adolescents with a modular, model-agnostic framework.

(2) The evaluation covers nine downstream LLMs and ten adolescent profiles, showing consistent safety improvements across models.

Critical weaknesses

(1) The main weakness of the paper is about the lack of comparisons with external baselines. The evaluation only compares SaplingGuard against an unguarded model and its own ablations. Comparisons with existing guardrails such as Llama Guard, WildGuard, PSG-Agent, or even simple system-prompt interventions are missing.

(2) The paper's own results suggest that a fixed-template approach can outperform the proposed LLM-based optimizer. It weakens the justification for the added complexity.

(3) Although profile-aware safety is the paper's main contribution, the experiments do not clearly demonstrate its benefits. All ten profiles are adolescents aged 13–17, with no adult control group, and harmful rates vary little across profiles (4.83–6.40%). This raises questions about whether the system meaningfully adapts to individual users.

(4) For ablations, removing the Profile Agent eliminates both stable user characteristics and dynamic conversation information. Therefore, it will be difficult to determine if the improvements come from personalization or simply access to dialogue history. Moreover, removing the agent changes the end-to-end harmful rate by only about 1.4 percentage points, with no reported variance or statistical significance.

(5) The experiments focus on reducing harmful responses without adequately assessing whether the guardrail preserves helpfulness or unnecessarily modifies benign prompts. It is unclear whether the safety improvements come at the cost of response utility.

Questions for authors

(1) How often does the Optimization Agent choose "rewrite" vs. "keep," overall and at Turn 1? Have you evaluated SaplingGuard on benign dialogues to measure over-refusal or unnecessary rewriting? How do you verify that rewritten prompts preserve the user's legitimate intent?

(2) Does the evaluator see the user's original prompt or the rewritten one? For the 98.8% agreement, how many annotators were involved, what was their expertise, and was agreement measured on the five-point or binary scale? Could you report a chance-corrected statistic such as Cohen's kappa?

(3) Turn 1 is described as an innocuous opening, yet 9.65% of unguarded Turn 1 responses are scored harmful. What makes these responses harmful, and does this suggest Turn 1 is not innocuous or that the judge over-flags?

(4) Could you provide example entries and describe who wrote and validated them? Were they written independently of SaplingBench? How do you expect SaplingGuard to handle risks outside the seven categories?

Reviewer_SS8rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SaplingGuard, a multi-agent guardrail framework for improving the safety of adolescent interactions with LLMs. The framework consists of three agents responsible for user profile construction, contextual risk assessment, and prompt optimization. The authors also introduce SaplingBench, a benchmark containing 276 three-turn dialogues covering seven categories of developmental risks. Experiments across nine downstream LLMs show that the proposed approach reduces harmful responses and improves safety scores. Additional retrieval experiments and component ablations investigate the contributions of user profiles and different guardrail components.

Critical strengths

The paper addresses an important problem that has received relatively limited attention: implicit developmental risks in adolescent–LLM interactions. Incorporating user profiles and dialogue history into safety assessment is a meaningful direction beyond conventional prompt-level moderation.

The evaluation covers nine downstream LLMs and includes several component ablations. The consistent improvements in harmful response rates, together with the retrieval experiments, provide useful evidence for the potential effectiveness of profile-aware safety intervention.

Critical weaknesses

The methodological novelty is somewhat limited, as the framework mainly integrates existing techniques, including user profiling, retrieval-based risk assessment, and LLM-based prompt rewriting. More importantly, the experimental evaluation lacks direct comparisons with existing guardrail approaches, particularly personalized or context-aware methods such as PSG-Agent. Without these comparisons, it is difficult to determine the additional benefit of the proposed multi-agent framework.

The safety evaluation relies heavily on a relatively small, constructed benchmark and an LLM-based evaluator. Although human agreement is reported, the validation of the scoring rubric and its generalizability to real adolescent interactions remain unclear. In particular, the rubric assigns a lower score to empathetic correction than to immediate rejection of harmful premises, which may not always reflect better developmental guidance. The lack of independent real-world evaluation and benign-query controls also limits the conclusions about practical safety and utility.

Questions for authors

Could the authors provide more details on the human validation of the safety evaluation, including annotator expertise, agreement measures, and the justification for assigning higher scores to immediate rejection than to empathetic correction? It would also be useful to evaluate the framework on independently constructed dialogues and benign queries to better assess generalization and potential over-intervention.

Table 3 shows that removing the Optimization Agent reduces the harmful response rate from 5.27% to 4.46%, although the average safety score decreases. Could the authors further discuss this trade-off and provide additional evidence that LLM-based prompt optimization offers meaningful benefits over deterministic safety-policy injection?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.