- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StageBreak, a framework for diagnosing jailbreak failures in MLLMs. The authors decompose the model's safety process into four stages: URDG. Based on prerequisite constraints, the framework attributes successful jailbreak attacks to the earliest operational failure stage.
Methodologically, the authors combine hidden-state probing, refusal likelihood margins, constrained adversarial optimization, and low-rank activation interventions to enable stage-targeted attacks and failure attribution. The authors further employ targeted repair, reverse transplantation, and cross-stage controls to validate the attribution results, and evaluate their stability across multiple models, attack settings, and external benchmarks.
Critical strengths
S1. Clear motivation and well-organized presentation.
The paper addresses a meaningful limitation of conventional jailbreak evaluation, which primarily relies on Attack Success Rate (ASR) without investigating the underlying causes of safety failures. Moving beyond binary attack success toward fine-grained failure diagnosis is a worthwhile research direction. The paper is generally well structured, and the proposed pipeline is presented in a relatively clear and systematic manner.
S2. Attempts to validate attribution through independent auditing and interventions.
The authors separate attack optimization from stage auditing, reducing the risk of using identical measurement instruments for both attack construction and evaluation. Furthermore, targeted repair, reverse transplantation, and cross-stage controls are introduced to investigate the relevance of the attributed representations rather than relying solely on probe-based classification.
S3. Broad experimental coverage.
The experiments cover multiple multimodal models, attack methods, and external benchmarks, along with various ablation studies and cross-model transfer analyses. These experiments provide a substantial empirical basis for examining the applicability of the proposed framework.
Critical weaknesses
W1. Limited methodological novelty and insufficient justification of the framework's complexity.
The proposed framework primarily combines existing techniques, including hidden-state probing, likelihood-based refusal measurement, constrained optimization, and SVD-based activation intervention. Moreover, the R/D/G decomposition is conceptually similar to prior structured safety-decision frameworks.[1] The paper does not sufficiently clarify what fundamentally new methodological contribution emerges from this combination. In particular, it remains unclear whether stage-specific attribution and intervention offer meaningful advantages over simpler, stage-agnostic alternatives.
W2. Insufficient validation of stage attribution and limited scientific insights.
The R/D/G stages are operationally defined through heterogeneous measurements and manually specified logical gates, without sufficient evidence that they correspond to distinct causal safety mechanisms. Although the authors provide intervention experiments and cross-model analyses, these results primarily demonstrate consistency with the proposed attribution criteria rather than independently validating the underlying stage decomposition. Furthermore, several experimental findings follow naturally from the predefined constraints and evaluation metrics, offering limited new insights into the mechanisms of multimodal jailbreak failures.
[1]. Gu, T., Huang, K., Li, L., Luo, R., Huang, S., Wang, Z., ... & Wang, Y. (2026). From sparse decisions to dense reasoning: A multi-attribute trajectory paradigm for multimodal moderation. arXiv preprint a
Questions for authors
Q1. Necessity of stage-specific decomposition.
Can the authors compare StageBreak against a simpler stage-agnostic baseline, such as a unified safety direction or a pooled clean–attack activation subspace? Would the proposed stage-specific interventions still provide significant benefits under matched experimental settings?
Q2. Validity and scientific insights of stage attribution.
What evidence supports the claim that R/D/G represent distinct causal failure mechanisms rather than different operational measurements of a shared safety-related representation? Beyond validating the predefined attribution criteria, what new insights into multimodal jailbreak failures does the framework reveal?
Response to Reviewer dNaK
We thank the reviewer for recognizing StageBreak's motivation, independent auditing, and broad experimental evaluation. We particularly appreciate the questions concerning methodological novelty, the necessity of stage decomposition, and the scientific insights supported by our framework.
1. Methodological contribution and necessity of stage decomposition (W1, Q1).
StageBreak's central contribution is prerequisite-preserving failure attribution: identifying which safety criterion becomes invalid while its upstream prerequisites and the original risk semantics remain intact. This goes beyond detecting safety-related signals or combining multiple safety measurements.
Equations (6)–(9) formalize sample-level eligibility, independent stage auditing, and constrained stage-targeted optimization. Importantly, separating attack steering from auditing prevents the optimization objective from directly determining its own evaluation outcome.
We appreciate the connection to Gu et al. (2026). Their UniMod framework develops structured reasoning trajectories and multi-attribute supervision to improve multimodal moderation. StageBreak addresses a different question: when an existing MLLM produces a harmful response under attack, which safety criterion failed, and can the corresponding representation be identified through independent auditing and intervention? Thus, the distinction lies in the research objective, attribution requirements, and interventional validation, rather than simply the number of modeled stages.
To directly test whether stage-specific decomposition provides benefits beyond a unified safety representation, we conducted an additional free-decoding evaluation on matched attacked samples using rank-2 interventions.
For R/D/G, strict ASR without intervention is 0.718/0.744/0.701. Unified rank-2 repair reduces these values to 0.523/0.551/0.536, while stage-specific repair further reduces them to 0.359/0.387/0.418, yielding additional absolute reductions of 0.164/0.164/0.118.
These results provide direct behavioral evidence that stage-aware intervention can outperform a pooled, stage-agnostic subspace under matched intervention capacity. Crucially, these improvements are measured from freely decoded responses, rather than being determined by the attribution rules.
2. Attribution validity and scientific insights (W2, Q2).
StageBreak identifies functionally distinguishable failure types and examines their representation-level relevance through interventions that provide evidence beyond the predefined audit criteria.
Three findings are particularly informative.
First, matched-stage repair achieves recovery rates of 0.720/0.690/0.660 for R/D/G, whereas applying a mismatched stage basis to the corresponding failure group yields target recovery no greater than 0.120 (Table 3). This demonstrates intervention selectivity beyond the stage labels themselves.
Second, intervention effects exhibit a structured asymmetry: R-targeted repair can also influence downstream D/G behavior, while D- and G-targeted interventions have much weaker effects on already violated upstream criteria. This reveals stage-dependent functional relationships without requiring a strictly sequential internal causal architecture.
Third, cross-model transfer distinguishes harmful endpoint transfer from failure-identity retention (Figure 3). Transferred Strict ASR is 0.460/0.481/0.444 for R/D/G, while Stage Match among harmful transferred successes reaches 0.828/0.838/0.799. These results reveal that successful transfer and preservation of the diagnosed failure identity capture different aspects of model vulnerability.
Importantly, the attribution gates determine which failures are validly assigned, but do not prescribe the observed intervention outcomes or cross-model transfer behavior. Together with the additional free-decoding comparison, these findings provide evidence of stage-dependent behavioral sensitivity and structured relationships among safety failures, rather than merely reproducing predefined classification rules.
We hope these clarifications and additional behavioral evidence better convey StageBreak's methodological contribution and scientific insights. We would be grateful if the reviewer could reconsider the assessment in light of these results.