- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
HiRoute uses a classifier to activate a shared safety prompt and mix risk-specific soft prompts over a frozen LLM. Experiments on three backbones report higher average safety and safe-response helpfulness, with lower over-refusal than the compared defenses.
Critical strengths
-
Reports substantial safety improvements over the base models.
-
Evaluates safe-response helpfulness, general utility, and over-refusal alongside safety.
Critical weaknesses
-
The motivation for separate experts is weak. A shared prompt still conditions on the request and can produce tailored safe responses. The paper does not establish an advantage over one prompt trained on the same combined data, with the same gate and training budget. Figure 2’s separate diagnostic does not provide this comparison.
-
The over-refusal comparison is confounded by gating. No gate-matched results are reported for competing defenses, leaving the source of HiRoute’s advantage unclear.
-
All evaluated backbones are from 2023–2024. The benefit on recent and more relevant models remains untested.
-
The GCG evaluation uses transferred attacks. Adaptive attacks against the complete defense remain untested, as the authors acknowledge.
Questions for authors
-
Does HiRoute outperform a single prompt under matched data, gating, and training budget?
-
Does the over-refusal advantage persist when competing defenses use the same gate?
-
Do the gains hold on a recent instruction-tuned model?
-
How does the complete defense perform under adaptive attacks, including attempts to bypass the router?