- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies localized safety intervention for LLM responses containing both benign and harmful content. The authors propose SafeLLaDA, a diffusion-based model that detects unsafe spans and regenerates them while preserving the surrounding context. They also introduce OmniSafeBench to evaluate safety, fidelity, fluency, and efficiency.
Critical strengths
1. The paper identifies a useful practical setting where an otherwise helpful response contains only localized unsafe content. Looking beyond full refusal is a useful direction.
2. The key insight of diffusion-based localized editing is practically reasonable. Only regenerating selected spans is a natural fit for this post-hoc intervention.
Critical weaknesses
1. The local editing strategy seems to rely on an assumption that authors do not sufficiently validated: the preserved context remains valid after an unsafe span is replaced. SafeLLaDA edits only the detected span and keeps the rest of the response unchanged, but an autoregressive suffix may depend on the original content of that span. Even if the suffix is safe in isolation, it may no longer be a valid continuation after the replacement.
2. The paper does not clearly distinguish iterative editing from repeated safety checking over the full response, so I am not sure how final safety is ensured after an intervention. If detection and editing can repeat when unsafe content remains, the stopping condition and stability across multiple interventions should be clarified. If detection is performed only once, the paper should justify why the initially identified span is sufficient to determine all necessary edits.
Questions for authors
1. My understanding is that SafeLLaDA edits the identified unsafe span while keeping the rest of the response unchanged. I am concerned about the assumption that the originally safe context remains valid after the replacement. For an autoregressive response, the later suffix may heavily depend on the edited content, even if the suffix itself is safe. Have you tested cases where the edited span is important for the later context? A few examples would be helpful.
2. I would like to know the difference between iterative editing and repeated safety checking in SafeLLaDA. Like, after one localized edit, does the system recheck the response and intervene again, if unsafe content remains? If so, what is the stopping condition? I am also curious whether a single detection&editing is sufficient.
3. Can you show what localized intervention looks like in practice? How many unsafe spans are typically detected, and roughly how much of the response is edited? For multiple spans, are they edited jointly or separately?
4. The paper would benefit from separating unsafe-span localization from subsequent rewriting. The current end-to-end results make it hard to tell whether a failure comes from localization, span boundaries, or the replacement itself. Reporting these components separately would make the failure modes easier to understand.