- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DepthGate extends instruction-data separation from input embeddings to intermediate Transformer layers through role-conditioned gated rotations. Schedule Discovery selects model-specific intervention schedules using held-out evaluation and matched adaptation. Experiments across five models demonstrate improved separation and generally reduced prompt-injection attack success rates.
Critical strengths
1 Multi-layer steering is a clear contribution. The results convincingly demonstrate improvements over embedding-only ASIDE in separation and resistance to many prompt-injection attacks. 2 Evaluation across five models and multiple direct and indirect attack benchmarks provides substantive evidence for the effectiveness of the overall approach.
Critical weaknesses
1 The independent contribution of Schedule Discovery is insufficiently isolated. Existing uniform-all-layer controls jointly change intervention placement, active-layer count, and cumulative gate strength. Comparisons with random and regularly spaced schedules under matched intervention and selection budgets are needed to establish whether the discovery heuristic contributes beyond multi-layer steering itself.
2 Retained ability to use task-relevant data is insufficiently evaluated. SEP Utility measures instruction-side probe success, while AlpacaEval contains no data field and leaves steering inactive. The indirect-injection benchmarks involve task-relevant data but report ASR without corresponding legitimate-task performance. These evaluations therefore do not adequately establish whether active steering preserves accurate use of data while rejecting embedded instructions, leaving the robustness–utility trade-off relative to ASIDE unresolved.
3 Key ablations have limited cross-model coverage. Although the main results cover five models, the uniform-gate, schedule-matching, and role-conditioning controls primarily use Qwen2.5 7B. Complete-system results do not establish that individual design choices have comparable effects across model families. This is a secondary concern.
Questions for authors
Do discovered schedules outperform random and regularly spaced alternatives with matched active-layer counts, gate values, and adaptation budgets? How does DepthGate compare with ASIDE on legitimate-task success for paired clean and attacked data-dependent tasks? Do the key ablation conclusions hold on latest model (qwen3/qwen3.5/qwen3.8 or others published in last six month)?
Response to rebuttal
The experiments author added solved part of my concerns.
However, the experiments showing that its not a pareto frontier of existing similar methods(aka ASIDES), it seems like it use a more complicate and delicate methods and some utiltiy to exchange for a safety improvement. for asr and clean task successful rate, its not a big improvments compared with existing model.
I would like to raise my rating to 2.5. Considering there is only 4 rating scale, I lean to maintain my rating decision.