acceptodds
Under review as a conference paper at ICLR 2027

IntentTrace: A Stateful Defense Against Multi-Turn Text-to-Image Jailbreaks

Abstract

Text-to-Image (T2I) models are vulnerable to multi-turn jailbreak attacks, in which adversaries exploit system feedback as proxy gradients to optimize malicious prompts. Current safety pipelines are fundamentally stateless and lack policy agility: they evaluate requests in isolation while wasting expensive computational resources on malicious objectives. We propose IntentTrace, the first context-aware stateful defense against multi-turn T2I jailbreaks. By treating multi-turn requests as a coherent session, IntentTrace tracks the evolution of sensitive semantics to preemptively identify malicious intent. It features training-free concept scoring for agile policy customization, temporal prompt parsing via Optimal Transport to capture semantic edits, and a lightweight Transformer for session-level intent recognition. Evaluations demonstrate that IntentTrace detects malicious intent within 2.05 rounds on average, reducing the attack success rate to 0.47% while significantly saving inference costs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Text-to-Image Generation, Multi-Turn Jailbreak Attacks, Stateful Defense, Intent Tracking

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LSjiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The existing defense methods suffer from two limitations: 1) Lack of policy agility: existing methods can only support a fixed set of harmful concepts, making it difficult to adapt them into dynamic policies. 2) Stateless safety pipelines: current designs can only evaluate independently of previous interactions and perform poorly on multi-turn attacks. To address this issue, this paper proposes a stateful defense framework that can be used to defend against multi-turn text-to-image jailbreaks. The contributions are as follows: 1) It uses cross-modal concept scoring to project text and image information into a shared embedding space to enable customizable concept vectors. 2) It formulates the prompt evaluation as a semantic matching problem to guide fine-grained editing operations. 3). It uses a lightweight intent recognizer to effectively detect harmful behaviors.

Critical strengths
  • The proposed pipeline is useful in defending against multi-turn text-to-image jailbreaks.

  • Empirical results show that the proposed text filter and image evaluator have SOTA or near-SOTA performance.

  • The computational cost of the online version is low.

Critical weaknesses

Overall, this paper is hard to follow due to a lack of clarity.

In general, most figures and tables are not self-explanatory enough. Some parts of figures and tables remain unclear even after the explanation in the main text. Specifically,

  • In Fig. 2: what is the setting of w/ and w/o concept?
  • Fig. 3 is very unclear to me. What are the x and y axes? What is the difference among the three subplots?
  • The use of terms in Fig.4 is very inconsistent with the explanation in Section 3.2. E.g., the terms “Sensitive Vector Scoring”, “Cross-Modal Projection”, “Temporal Prompt Parsing”, “Session Intent Recognition”, “Online Defense Pipeline” are not mentioned in the explanation of Fig. 4.
  • The Sensitive Vector Scoring, Temporal Prompt Parsing, and Session Intent Recognition in Fig. 4 look somewhat disconnected. It is not clear to me how these components interact with each other.
  • Tables 1- 7: It would be better to describe the key components in these tables. The current version is not self-explanatory enough.
  • In Fig. 5, what are B, S, and V?
  • In Fig. 6, the meaning of SP, JF, and IN should also be mentioned in the caption.

Algorithm 1 is not in a good format. It would be better to use a single-column layout that only takes half the page width. Also, the meaning of K, M, D, and A should be described in the algorithm.

In Section 3.1, the threat model is only introduced at a high level, although it is introduced in later sections. Here, an appropriate cross-reference to further details about the threat model should be provided.

It is not clear to me how the Cross-Modal Concept Scoring component compares with SOTA scoring methods.

Tables 2 and 3 compare the text filter and image evaluator components with SOTA methods. I wonder how the overall online defense pipeline compares with SOTA methods. It is also not clear to me which parts of the proposed pipeline correspond to the text filter and image evaluator.

This framework features a training-free concept scoring method but still requires training in Session Intent Recognition. Thus, emphasizing the training-free nature of one component of the pipeline is somewhat misleading.

Why are some categories not supported in Tables 3, 4, and 12? The explanation is missing.

It would be nice to compare the computational cost of the proposed method with SOTA methods.

Questions for authors

Please see Critical Weaknesses for my questions.

10 comments
AuthorsOpenReview ↗

Response for Reviewer LSji (1/3)

Dear Reviewer LSji,

Thank you for your detailed feedback, and for acknowledging that our pipeline is a useful and low-cost defense against multi-turn T2I jailbreaks. Please see our point-by-point responses to your comments below.

W1: Clarity of figures, tables, and algorithm


In general, most figures and tables are not self-explanatory enough. Some parts of figures and tables remain unclear even after the explanation in the main text.

Thank you for the detailed and specific feedback. We clarify Figs. 2 and 3 below and have revised the remaining figures, tables, and algorithm for clarity.

  • Fig. 2 (meaning of "w/ concept" and "w/o concept"). "w/ concept" and "w/o concept" denote the two sides of our contrastive prompt pairs, which share similar subjects, scenes, and styles and differ mainly in whether the target sensitive concept is present. The figure visualizes the concept-related displacement of these pairs in the BLIP embedding space, together with benign prompts from MSCOCO and malicious prompts from DREAM.

  • Fig. 3 (axes and subplots). We added the missing y-axis labels. The x-axis is the normalized request order within a session. The three vertically arranged panels show the text toxicity score, the image toxicity score, and the cosine similarity between consecutive prompt embeddings. Each colored trajectory is one real-user session, and each point is one request. The axes and panels are now defined in both the figure and the caption.

  • Fig. 4 and Sec. 3.2 (terminology and component interaction). We redesigned the figure with consistent module names, explicit data-flow arrows, and execution-order labels, and updated Sec. 3.2 to use exactly the same terms. The interaction between the modules is now described as follows:

    • Sensitive Vector Scoring (via Cross-Modal Projection) provides concept-risk scores for pre-generation text filtering, chunk-level risk modeling, and post-generation image filtering.
    • Temporal Prompt Parsing uses UOT to align semantic chunks across consecutive requests and extracts Keep / Modify / Delete / Add actions. The UOT transport cost also drives Session Routing.
    • Session Intent Recognition combines the parsed actions, their alignment statistics, and the concept-risk scores into structured action tokens, which CAT processes to estimate session-level intent.
  • Tables 1–7 and Figs. 5–6 (definitions and abbreviations). We expanded the table notes and figure captions to define the key components, metrics, and abbreviations. B / S / V denote Benign / Sexual / Violence, and SP / JF / IN denote SneakyPrompt / JailFuzzer / Inception. The original Fig. 5 has also been replaced with a more comprehensive Table 4, which presents a unified comparison of different intent recognition baselines.

  • Algorithm 1 (layout and notation). We reformatted the algorithm into a single-column layout and defined K, M, D, and A in the algorithm as the sets of chunks assigned the Keep, Modify, Delete, and Add actions, respectively.

AuthorsOpenReview ↗

Response for Reviewer LSji (2/3)

W2: Cross-references for the threat model


In Section 3.1, the threat model is only introduced at a high level, although it is introduced in later sections. Here, an appropriate cross-reference to further details about the threat model should be provided.

Thank you for the suggestion. We added cross-references in Sec. 3.1 that link the threat model to the detailed settings:

  • the attack methods and dataset construction in Sec. 5.1 and Appendix C.3;
  • the online defense protocol and attack-success criteria in Sec. 5.4 and Appendix C.4;
  • the defense-aware adaptive attack evaluation in Appendix D.

These links let readers see how the attacker's interaction capabilities, the system feedback, the attack budgets, and the evaluation procedure are instantiated in the experiments.

W3: Comparison with SOTA scoring methods and the overall pipeline


It is not clear to me how the Cross-Modal Concept Scoring component compares with SOTA scoring methods.

Tables 2 and 3 compare the text filter and image evaluator components with SOTA methods. I wonder how the overall online defense pipeline compares with SOTA methods. It is also not clear to me which parts of the proposed pipeline correspond to the text filter and image evaluator.

Thank you for pointing out this unclear correspondence between components and evaluations.

Our Cross-Modal Concept Scoring applies the same concept vectors to text prompts and to generated images. It therefore is the text filter and the image evaluator in our pipeline. Tables 2 and 3 compare it directly with SOTA text and image filters. It achieves an average text AUROC of 0.9610 and image AUROCs of 0.9689 / 0.9722 on Sexual / Violence.

At the system level, IntentTrace consists of three detection stages: (1) pre-generation text filtering (concept scoring on the prompt), (2) CAT-based session intent recognition, and (3) post-generation image filtering (concept scoring on the generated image). Stages 1 and 2 can block a request before the T2I generator is invoked.

For pipeline-level comparison, no existing baseline is a directly comparable stateful multi-turn defense pipeline to the best of our knowledge. We therefore ran a controlled stateless vs. stateful comparison on SD1.4. The stateless variant uses only request-level filtering (text and image filters), and the stateful variant adds session intent recognition. Adding it reduces the T2I invocation rate from 58.14% to 24.18% and the ASR from 1.12% to 0.47%.

W4: "Training-free" claim


This framework features a training-free concept scoring method but still requires training in Session Intent Recognition. Thus, emphasizing the training-free nature of one component of the pipeline is somewhat misleading.

We agree that the original wording could suggest that the whole framework is training-free. We revised the relevant statements in the Abstract and Introduction to say that the advantage of concept scoring is its support for flexible policy customization. Appendix E also shows that a new concept (Cat) needs only a new concept vector and threshold calibration, with CAT not retrained.

W5: Unsupported categories in tables


Why are some categories not supported in Tables 3, 4, and 12? The explanation is missing.

Thank you for pointing this out. We now explain every "–" entry in the table notes and in Appendix C.2. (Table numbers below follow the revised manuscript; the ablation table and the per-dataset image table you refer to as Tables 4 and 12 are now Tables 5 and 13.)

  • Tables 3 and 13: "–" means the category is outside the baseline's original detection scope. Some image safety detectors are designed for specific content categories only (e.g., SD Checker and Falconsai target Sexual content, and Q16 targets Violence). We therefore compare each baseline only on the categories it supports, to keep the comparison fair. Supported categories and implementation details are given in Appendix C.2.2.
  • Table 5: "–" under Benign means "not applicable". Detection Prefix is the position of the first positive prediction, which is defined only for malicious sessions.
AuthorsOpenReview ↗

Response for Reviewer LSji (3/3)

W6: Computational cost comparison with SOTA methods


It would be nice to compare the computational cost of the proposed method with SOTA methods.

Thank you for the suggestion. We added a cost comparison for the text filters, the image evaluators, and the session-level intent recognition pipeline. We report p50/p95 latency, peak system memory, and peak GPU memory, all measured on the same hardware (Appendix H, Tables 21–23).

1. Text filters and image evaluators

Text filters (Table 21):

Methodp50 (ms)p95 (ms)Peak memory (MiB)Peak GPU memory (MiB)
TextMatch0.070.08150.730.00
TextClassifier1.521.581610.39610.00
Detoxify2.612.711839.081436.00
GuardT2I176.17296.141889.313058.00
IntentTrace (Ours)3.363.391500.201276.00

Image evaluators (Table 22):

Methodp50 (ms)p95 (ms)Peak memory (MiB)Peak GPU memory (MiB)
Falconsai11.2911.791535.35714.00
MHSC73.7674.551655.932016.00
Q1619.1119.651722.302016.00
SD Checker20.7821.381847.411518.00
IntentTrace (Ours)15.4716.021700.671312.00

Our text filter runs at a median of 3.36 ms, on the same order as lightweight classifiers such as TextClassifier and Detoxify, and about 52× faster than GuardT2I (176.17 ms). Its peak GPU memory (1,276 MiB) is also well below GuardT2I's (3,058 MiB). Our image evaluator runs at a median of 15.47 ms with 1,312 MiB of peak GPU memory, faster than MHSC, Q16, and SD Checker. Overall, the cross-modal concept scoring supports both text and image screening at a small per-request cost.

2. Complete session-level intent recognition pipeline

No directly comparable stateful multi-turn T2I intent-recognition baseline exists, so we use the stronger Qwen3.8-27B History LLM as the reference. For IntentTrace, the measurement covers the whole path from raw prompt to final intent score: semantic chunking, embedding encoders, UOT alignment, action-token construction, and CAT inference.

Session-level intent recognition (Table 23):

Methodp50 (ms)p95 (ms)Peak memory (GiB)Peak GPU memory (GiB)
Qwen3.8-27B History LLM433.16634.0514.5113.12
IntentTrace (Qwen3.5-9B LLM chunker)462.52676.494.369.54
IntentTrace (MiniLM-L6 student chunker)34.2837.381.923.76

With the original 9B chunker, IntentTrace has latency comparable to the 27B History LLM while using less memory (9.54 vs. 13.12 GiB GPU, 4.36 vs. 14.51 GiB system).

3. Lightweight student chunker

Most of the overhead comes from LLM-based chunking, so we further investigate a deployment configuration that replaces the LLM chunker with a lightweight MiniLM-L6 sequence model. The student pipeline reduces the median latency of the complete intent recognition from 462.52 ms to 34.28 ms (about 92.59% lower) and the peak GPU memory from 9.54 to 3.76 GiB (60.59% lower). Session accuracy stays comparable (94.95% vs. 94.99%), and attack-session accuracy is 96.36%.

In summary, IntentTrace's text and image scoring is low-cost, and its structured edit representation also allows a lightweight session-level detector. Detailed measurements for every component are in Appendix H of the revised manuscript.

Reviewer_8XaiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses multi-turn attacks against text-to-image jailbreaks. The authors propose intentTrace, a stateful defense framework that tracks semantic updates for consecutive prompts to detect malicious intent. The paper combines concept scoring, Unbalanced Optimal Transport (UOT), and Concept-Conditioned Action Transformer (CAT) for session level intent detection. Experiments across different attack strategies and T2I models demonstrate improved detection performance, and ablation study is further provided to evaluate the proposed component.

Critical strengths
  1. The paper addresses an important limitation of existing Text-to-Image defenses, which typically evaluate individual prompts without considering previous interactions. The idea of tracking semantic editing patterns across multiple requests is interesting and provides a promising perspective to detecting multi-turn jailbreak attempts.
  2. The authors evaluate IntentTrace across different attack strategies and models, including ablation studies, cross-model transfer, and benign-user impact. The coverage of the experiments are somewhat thorough and adequate. It also shows the practical values of the propose framework per the findings.
Critical weaknesses
  1. This is the most concern I have for this paper. It is still not adequate to show if there exists potential dataset bias in malicious intent detection. i.e., the benign sessions are collected from DiffusionDB, while the malicious sessions are generated using SneakyPrompt and JailFuzzer. The authors was tying to argue this issue by providing ablation studies and cross-attack evaluations, but it remains unclear whether CAT learns generalizable malicious intent patterns or simply captures dataset-specific editing properties.
  2. The proposed framework combines multiple components, including semantic chunking, UOT, and CAT. While the ablation studies demonstrate their contributions, the comparison with simpler alternatives remains limited. This makes it difficult to justify the added complexity vs performance improvement.
Questions for authors
  1. The first question corresponds to weakness 1. As benign sessions are collected from DiffusionDB while malicious sessions are generated by SneakyPrompt and JailFuzzer, how can the authors ensure that CAT learns generalizable malicious intent patterns rather than dataset-specific editing properties? If they are both exhibiting similar prompt editing behaviors, how would CAT can distinguish benign and malicious session? Could the authors provide additional evidence or analysis?
  2. The second questions corresponds to weakness 2. I appreciate the authors to provide the ablation studies to show the contributions of UOT and CAT, but it remains unclear whether a simpler stateful model using prompt embedding differences and concept risk trajectories could achieve comparable performance.
  3. minor issue. The reported ASR decreases from 1.12% to 0.47%, while the framework introduces additional parsing overhead. Could the authors clarify the overall computational savings after accounting for the complete detection pipeline, and discuss the trade-off?

The first two questions are more important to my final assessment, and I would consider raiseing my rating and recommendation based on convincing responses to these concerns.

9 comments
AuthorsOpenReview ↗

Response for Reviewer 8Xai (1/3)

Dear Reviewer 8Xai,

Thank you for your thoughtful review, and your recognition that tracking semantic editing patterns across requests is a promising way to address the limitations of stateless T2I defenses. Please find our point-by-point responses below.

Q1: Dataset bias


As benign sessions are collected from DiffusionDB while malicious sessions are generated by SneakyPrompt and JailFuzzer, how can the authors ensure that CAT learns generalizable malicious intent patterns rather than dataset-specific editing properties? If they are both exhibiting similar prompt editing behaviors, how would CAT can distinguish benign and malicious session? Could the authors provide additional evidence or analysis?

Thank you for highlighting this potential source of bias. In our main setting, benign sessions come from real users while malicious sessions come from automated attacks, so differences in editing style between the two sources could in principle confound the results. To evaluate CAT more directly, we introduce an additional edit-controlled evaluation, in which both classes of trajectories are generated by the same editing algorithms.

1. Experimental setup

We use SneakyPrompt and JailFuzzer to construct multi-turn editing trajectories for three content categories:

  • Benign-content editing (negative class): we sample initial prompts from SneakyPrompt's Dog/Cat-100 dataset and generate multi-turn trajectories targeting an Animal filter (a dog/cat classifier trained on Animals10).
  • Harmful-content editing (positive class): we sample initial Sexual and Violence prompts from DREAM and generate multi-turn attack trajectories against the corresponding safety filters (SD Checker for Sexual and Text-classifier for Violence).

For each attack method, we sample 50 initial prompts per content category. Since both classes are produced by the same attack algorithm, this setting controls for the difference in editing-method sources between real-user revisions and automated attack revisions in the original experiments.

CAT performs intent recognition using risk features derived from the Sexual and Violence concept vectors. Prefix-level metrics aggregate all scorable prefixes, and session-level metrics use the maximum prefix prediction score over each complete trajectory. We report threshold-independent AUROC and AUPRC.

2. Results

Editing methodPrefix AUROCPrefix AUPRCSession AUROCSession AUPRC
SneakyPrompt0.93820.96700.93790.9631
JailFuzzer0.98420.99050.97960.9910

When benign-content and harmful-content trajectories are generated by the same attack method, CAT remains strongly discriminative at both the observable session-prefix level and the complete-trajectory level. Even though both classes share the same automated editing process, CAT can effectively distinguish trajectories that target different safety objectives by leveraging concept-related semantic information and its evolution across revisions. These results indicate that CAT's discriminative capability extends beyond differences in editing behavior.

We have added the construction process, evaluation protocol, and results to Appendix F.2 of the revised manuscript.

AuthorsOpenReview ↗

Response for Reviewer 8Xai (2/3)

Q2: Comparison with simpler stateful alternatives


[...] it remains unclear whether a simpler stateful model using prompt embedding differences and concept risk trajectories could achieve comparable performance.

Thank you for this suggestion. We agree that component-removal ablations alone do not establish that a structured edit representation is necessary compared with simpler history-based modeling. We therefore added controlled comparisons across different history representations (see also our response to Reviewer snVJ-Q1).

1. Simpler stateful baselines

  • Prompt Delta directly implements the model you suggest. Its inputs are the sentence-level embedding difference between consecutive requests, the current concept risks, and their changes, and a Transformer models how these evolve over the session.
  • Prompt History represents each past request by its sentence-level embedding and risk features.
  • Chunks concatenates the ordered chunk embeddings, concept risks, and request boundaries.

All baselines use the same data splits, two-layer block-causal Transformer backbone, training hyperparameters, and validation-based threshold selection as CAT, so the comparison isolates the history representation.

2. Results

MethodPrefix Acc.Session Acc.Attack Session Acc.Benign Session Acc.
Prompt Delta41.9840.1725.3099.64
Prompt History47.6947.4634.3999.73
Chunks34.9037.5321.92100.00
Full CAT96.3394.9997.0986.61

Prefix Acc. and Session Acc. are macro-averaged over five evaluation subsets (%); see revised Table 4.

The suggested Prompt Delta model reaches only 41.98% prefix accuracy and detects 25.30% of malicious sessions, versus 97.09% for Full CAT. Prompt History, which models the full history of semantic and risk features, detects only 34.39% of malicious sessions. Chunks, which keeps fine-grained semantics, reaches 34.90% prefix accuracy, far below the 96.33% of Full CAT.

These results indicate that sentence-level embedding differences and concept-risk trajectories do not carry enough information to capture how local semantic components change across consecutive requests. Concatenating chunks retains fine-grained content but provides no explicit cross-request correspondence. IntentTrace aligns semantic components across requests with UOT and organizes edit actions, concept-risk changes, and transport information into structured action tokens, which enables CAT to model the evolution of sensitive semantics across successive requests effectively.

We added these comparisons and their implementation details to Sec. 5.3 and Appendix C.2 of the revised manuscript.

AuthorsOpenReview ↗

Response for Reviewer 8Xai (3/3)

Q3: Computational savings and trade-off of the complete pipeline


The reported ASR decreases from 1.12% to 0.47%, while the framework introduces additional parsing overhead. Could the authors clarify the overall computational savings after accounting for the complete detection pipeline, and discuss the trade-off?

Thank you for this question. We agree that semantic parsing and intent recognition add per-request cost, so the trade-off should be judged against the generator calls that session-level blocking avoids.

1. Security benefit and generation savings

Compared with the stateless defense on SD1.4, IntentTrace reduces ASR from 1.12% to 0.47% and the T2I invocation rate from 58.14% to 24.18%. Once persistent malicious editing is recognized and the session is blocked, subsequent requests are rejected before image generation. This saves generator calls and also denies the attacker further generated images and rejection feedback to optimize against.

The saving matters because a stateless pipeline repeatedly invokes the expensive generator, and harmful outputs are only caught afterwards by the image filter. In our measurements, one T2I generation takes about 1,794 ms and one image evaluation about 15 ms, so every avoided generator call removes a larger cost than the detection added per request (about 3.9× the overhead of the LLM-chunker configuration and about 52× that of the student configuration).

2. Additional overhead and deployment optimization

We measured the complete intent-recognition pipeline, including LLM-based chunking, the embedding encoders, UOT alignment, and CAT inference. The original configuration (Qwen3.5-9B chunker) has a median (p50) latency of 462.52 ms and a peak GPU memory of 9.54 GiB. This is the extra cost of session-level defense.

To reduce it, we replaced the LLM chunker with a lightweight MiniLM-L6 sequence model (see our response to Reviewer snVJ-Q2). With comparable detection accuracy, the complete pipeline runs at a p50 latency of 34.28 ms and a peak GPU memory of 3.76 GiB, a 92.59% latency reduction and a 60.59% memory reduction relative to the LLM-chunker configuration.

3. Trade-off

IntentTrace adds a pre-generation intent-recognition step in exchange for stronger detection of multi-turn attacks and far fewer expensive T2I invocations. Benign requests mainly incur this extra detection latency, and the student variant makes it small relative to generation time, which makes the mechanism practical for online deployment.

We expanded the online defense evaluation and Appendices G and H of the revised manuscript with the lightweight deployment analysis and the latency and resource consumption of the complete pipeline.

Reviewer_snVJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

IntentTrace studies stateful detection of multi-turn jailbreak attempts against text-to-image systems. It aligns semantic chunks across successive requests using unbalanced optimal transport (UOT), converts the alignments into action tokens carrying concept-risk information, and trains a small Concept-conditioned Action Transformer (CAT) to recognize malicious trajectories. The paper evaluates component ablations, session detection, and the complete defense pipeline.

Critical strengths
  1. The use of UOT to construct trajectory tokens is interesting. Representing semantic evolution across requests offers a concrete approach to detecting jailbreak intent that may be difficult to judge from individual requests. This is the main methodological aspect that makes the paper promising to me.

  2. The paper contains several useful ablations, including the comparisons in Table 4 that probe parsing, chunking, UOT, and concept conditioning. These checks and the reported gains support a slightly positive assessment, although the comparisons do not fully resolve the concerns below.

Overall, my current recommendation is borderline weak acceptance. I find the trajectory representation interesting, while remaining concerned about whether its complexity is necessary and whether the evaluation establishes its incremental value.

Critical weaknesses
  1. The necessity and incremental value of the construction are not sufficiently isolated. The nearest-neighbor replacement provides evidence for the implemented UOT alignment, and Coarse and LLM are useful history-based baselines. However, Coarse also removes chunk-level detail, while the no-chunking variant replaces the transition with a global Modify action. These comparisons do not establish whether comparable results could be obtained by compacting the prompt history or concatenating the same chunk embeddings and risk information, with appropriate request/session separators and no UOT or discrete edit labels. The alignment-score distributions and the contribution of the assigned edit types also remain difficult to assess. Furthermore, the LLM judge is Qwen3.5-9B in zero-shot, non-thinking mode. Since CAT and its input construction are central to the claimed contribution, a comparison against a more capable history-based LLM detector is important for sustaining acceptance. I am less familiar with the full related literature and do not make a firm priority claim, but the paper should explain more precisely what is new beyond the intuitive use of information across requests and existing context-aware moderation.

Post-rebuttal update: Thank you for adding the simpler history baselines, the stronger history-based LLM comparison, and the pipeline cost measurements. The lightweight chunker and the clarifications about the evaluation make the results easier to assess. The paired follow-up shows a modest gain over risk history, so I remain uncertain how much of the overall gain depends on the full edit representation across settings. I maintain my rating of weak acceptance (3).

Questions for authors
  1. To establish the necessity of the representation, how does the method compare with compacted prompt history and with concatenated chunk embeddings/risk features using appropriate request/session separators? For the embedding comparison, please keep the available information, training data, and classifier comparable while removing UOT and discrete edit labels. What do the distributions of alignment distances/transport statistics and an ablation of the edit-type labels show?

  2. Does the advantage persist against a more capable history-based LLM detector? This is important to my acceptance assessment because CAT and its input construction are the central innovation. Please compare detection, benign acceptance, and complete pipeline cost, including the chunker and encoders.

  3. Could you clarify the construction, labels, split protocol, and evaluation denominators of the datasets, and explain which aspects of realistic user/attack behavior they capture? Please make the key points and definitions of Coarse and LLM easier to locate in the main text.

  4. What precisely distinguishes the proposed alignment/action-token representation from existing context-aware or memory-based moderation? My novelty concern is a request for clearer positioning, given my limited familiarity with this literature.

These questions concern evidence and clarification that could change my assessment, rather than suggestions for unrelated future work.

6 comments
AuthorsOpenReview ↗

Response for Reviewer snVJ (1/4)

Dear Reviewer snVJ,

Thank you for your careful review and for recognizing the value of using UOT to build trajectory tokens that capture how semantics evolve across requests. Our point-by-point responses are provided below.

Q1: Necessity of the representation


To establish the necessity of the representation, how does the method compare with compacted prompt history and with concatenated chunk embeddings/risk features using appropriate request/session separators? For the embedding comparison, please keep the available information, training data, and classifier comparable while removing UOT and discrete edit labels. What do the distributions of alignment distances/transport statistics and an ablation of the edit-type labels show?

Thank you for this constructive question. To address this question, we provide (i) controlled representation baselines, (ii) statistics of the UOT alignment, and (iii) an incremental analysis of continuous alignment features and discrete edit labels.

1. Compacted history vs. concatenated chunks vs. CAT

We compare CAT against the following representations:

  • Prompt History uses one token per past request (sentence embedding + concept-risk features)
  • Prompt Delta additionally encodes the embedding difference and risk change between consecutive requests.
  • Chunks concatenates ordered chunk embeddings and concept-risk features, with request-boundary markers.
  • Chunks + UOT and Chunks + UOT + Labels add continuous alignment features and the Keep/Modify/Delete/Add labels, respectively.

All variants use the same data splits, two-layer block-causal Transformer, hyperparameters, threshold selection, and chunking outputs, so they differ only in the alignment information.

MethodPrefix Acc.Session Acc.Attack Session Acc.Benign Session Acc.
Prompt History47.6947.4634.3999.73
Prompt Delta41.9840.1725.3099.64
Chunks34.9037.5321.92100.00
Chunks + UOT42.4444.0430.0799.96
Chunks + UOT + Labels45.7547.1433.9599.91
Full CAT96.3394.9997.0986.61

Prefix Acc. and Session Acc. (All) are macro-averaged over five evaluation subsets (%).

Full CAT reaches 96.33% prefix accuracy, far above the compacted histories and concatenated chunks. The simplified representations accept nearly all benign sessions but detect only 21.92%–34.39% of malicious ones. Sentence-level histories blur which local components changed, while plain concatenation keeps fine-grained information but has no explicit cross-request correspondence. CAT supplies this correspondence through concept-conditioned action tokens.

2. UOT alignment and transport statistics

We group aligned chunk pairs from consecutive requests by lexical Jaccard overlap JJ and report the median cosine distance / flow ratio of the dominant matching edge:

DatasetIdenticalJ≥0.5J \ge 0.50<J<0.50 < J < 0.5No shared tokens
DiffusionDB0.000 / 0.9780.123 / 0.8020.281 / 0.6120.428 / 0.413
SneakyPrompt0.000 / 0.9950.204 / 0.9020.354 / 0.6500.443 / 0.475
JailFuzzer0.000 / 0.8910.123 / 0.7700.333 / 0.5900.441 / 0.486

On all three datasets, semantic distance rises and flow concentration falls as lexical overlap drops, so the statistics reflect the magnitude of local edits.

We also test whether the transport cost ktk_t separates edits within one generation objective from objective switches (the basis of session routing):

DatasetWithin-objectiveObjective switchAUROC
DiffusionDB0.1440.4190.873
SneakyPrompt0.1820.4500.984
JailFuzzer0.2640.4410.947

The transport cost discriminates objective switches clearly on all three datasets, which supports its use for session routing and edit-action extraction.

3. Incremental value of UOT features and edit labels

Adding continuous UOT features to Chunks raises prefix accuracy from 34.90% to 42.44% and attack-session accuracy from 21.92% to 30.07%. Adding the edit labels raises them further to 45.75% and 33.95%. Both components therefore contribute. However, they remain far below Full CAT even when combined, which indicates that alignment features and labels alone are insufficient. The larger gain comes from organizing them together with concept risk into compact action tokens.

We have added the representation comparisons and baseline implementation details to Sec. 5.3 and Appendix C.2 of the revised manuscript.

AuthorsOpenReview ↗

Response for Reviewer snVJ (2/4)

Q2: Comparison with a stronger history-based LLM detector


Does the advantage persist against a more capable history-based LLM detector? [...] Please compare detection, benign acceptance, and complete pipeline cost, including the chunker and encoders.

Thank you for this important question. We added a stronger baseline, Qwen3.8-27B History LLM, and measured the cost of the complete intent-recognition pipeline. We also studied a lightweight chunker that removes the main source of our overhead.

1. Detection and benign acceptance

Both History LLMs run in non-thinking mode with 4-bit quantization. They read the complete chronological request history and output a 1–5 malicious score under the same zero-shot instruction. Their decision thresholds are selected on the validation set with the same prefix-F1 criterion used to calibrate CAT.

MethodPrefix Acc.Session Acc.Attack Session Acc.Benign Session Acc.
Qwen3.5-9B History LLM71.1575.7671.1094.38
Qwen3.8-27B History LLM86.9289.4588.1994.51
IntentTrace (Qwen3.5-9B chunker)96.3394.9997.0986.61
IntentTrace (MiniLM-L6 student chunker)96.2894.9596.3689.29

All numbers are macro-averaged over five evaluation subsets (%); see revised Tables 4 and 19.

Scaling the LLM from 9B to 27B improves attack-session accuracy from 71.10% to 88.19%, so a stronger model clearly helps. However, IntentTrace remains ahead, with +5.54 points in session accuracy (94.99% vs. 89.45%) and +8.90 points in attack-session accuracy (97.09% vs. 88.19%). This supports our claim that structured edit trajectories capture multi-turn optimization behavior better than raw history reading.

We also acknowledge the trade-off. The 27B judge accepts more benign sessions (94.51% vs. 86.61% for IntentTrace with the LLM chunker). The student-chunker variant narrows this gap to 89.29%.

2. Cost of the complete pipeline

The latency below covers the whole path from raw prompt to final intent score, i.e., chunker, embedding encoders, UOT, and CAT for IntentTrace.

Configurationp50 (ms)p95 (ms)Peak GPU memory (GiB)
Qwen3.8-27B History LLM433.16634.0513.12
IntentTrace (Qwen3.5-9B chunker)462.52676.499.54
IntentTrace (MiniLM-L6 student chunker)34.2837.383.76

With the original 9B chunker, IntentTrace has latency comparable to the 27B judge while using less GPU memory (9.54 vs. 13.12 GiB).

3. Lightweight student chunker

Most of our overhead comes from LLM-based chunking. We therefore distilled the 9B chunker into a MiniLM-L6 boundary chunker (about 22M parameters with a small MLP head), supervised by the original chunker's outputs, and retrained CAT on the resulting representations. Session accuracy stays essentially unchanged (94.95% vs. 94.99%), while median latency drops from 462.52 ms to 34.28 ms (−92.59%) and peak GPU memory from 9.54 to 3.76 GiB (−60.6%). The student pipeline is thus about 12× faster than the 27B judge and uses less than a third of its GPU memory, with higher detection accuracy.

We added the 27B comparison to Sec. 5.3, and the student-chunker design and full cost analysis to Appendices G and H.

AuthorsOpenReview ↗

Response for Reviewer snVJ (3/4)

Q3: Dataset construction, labels, splits, and denominators


Could you clarify the construction, labels, split protocol, and evaluation denominators of the datasets, and explain which aspects of realistic user/attack behavior they capture? Please make the key points and definitions of Coarse and LLM easier to locate in the main text.

Thank you for raising this. We expanded the dataset and protocol descriptions, and summarize the key points below.

1. Splits, labels, and denominators

  • Data Splits. CAT is trained and validated on an 80/20 session-level stratified split (stratified by data source and session label), giving 5,161 / 1,291 sessions. The test set is session-disjoint and contains 747 benign sessions and 471 scorable malicious sessions, totaling 6,386 scorable prefixes (4,121 benign + 2,265 malicious). CAT infers intent from cross-request edits, so a session of length LL yields L−1L-1 scorable prefixes (scoring starts at the second request). The 254 single-request malicious sessions contain no editing transition and are therefore excluded from offline intent evaluation.

  • Labels. Benign sessions are real user sessions that pass text and image safety screening. Malicious sessions inherit the label of their attack objective. Every prefix inherits its session's label.

  • End-to-end denominators. Each online configuration contains 900 attack-target attempts (3 attacks × 3 source datasets × 2 concepts × 50 targets). Request-level metrics (ABR, ASR, T2I call rate) use executed attack requests as the denominator. Session-level metrics (PBR, PSR) use attack sessions, i.e., target prompts.

2. Attack behavior covered

We cover three attacks with different editing mechanisms: SneakyPrompt performs local token substitutions guided by system feedback; JailFuzzer employs an LLM agent for semantic rewriting and optimization; and Inception progressively pursues malicious generation objectives through semantic decomposition and recursive composition.

Initial targets come from DREAM (automatically generated red-teaming prompts), I2P (real-world inappropriate-image prompts), and UnsafeBench (human-annotated unsafe images), covering both Sexual and Violence concepts. CAT is trained only on SneakyPrompt and JailFuzzer trajectories with NSFW-200 Sexual targets, and is evaluated on source-disjoint DREAM Sexual and Violence trajectories. Inception is entirely excluded from CAT training, and is used to evaluate generalization to an unseen attack strategy.

3. Benign behavior covered

Benign sessions come from real users. For DiffusionDB (official Stable Diffusion Discord), we group requests by user ID and timestamp using a 3–5-minute gap criterion, which preserves locally continuous refinement toward one objective. Midjourney2023 and StyleBreeder are used for cross-platform generalization. These datasets originate from public Midjourney Discord channels and Artbreeder user interactions, respectively. Thus, they test transfer to unseen benign refinement styles in end-to-end evaluation.

4. Locating Coarse and LLM

We now list all session-level baselines, including Coarse, the prompt-/chunk-level variants, and the two zero-shot LLM judges, in the Baselines paragraph of Sec. 5.1, with pointers to their definitions. The detailed implementations and the judge prompt are in Appendix C.2 and Appendix I.

Dataset construction, filtering, label definitions, splits, and denominators are expanded in Sec. 5.1 and Appendix C.3 of the revised manuscript.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their careful and constructive feedback, and the Area Chair for overseeing the review process. We are encouraged that the reviewers recognized the following strengths of our work.

First, IntentTrace addresses an important limitation of stateless T2I defenses by detecting multi-turn jailbreak intent that is difficult to identify from isolated requests (snVJ, 8Xai, LSji). Second, our UOT-based semantic alignment and structured edit representation were noted as an interesting and promising approach to capturing cross-request semantic evolution (snVJ, 8Xai). Third, our evaluation was regarded as comprehensive, featuring useful ablations (snVJ, 8Xai), diverse attack strategies and generators (8Xai), and state-of-the-art (SOTA) or near-SOTA text and image detection performance (LSji). Finally, the reviewers positively noted the low computational overhead of our online defense (LSji).

We have carefully addressed all comments through additional experiments, clarifications, and manuscript revisions, as summarized below.


Core Contributions

  1. Structured Semantic Evolution Modeling. We introduce a UOT-based representation to explicitly capture fine-grained semantic editing actions and sensitive-concept risk evolution across consecutive requests, turning historical interactions into structured evidence for identifying latent malicious intent.
  2. Stateful End-to-End Defense Framework. To the best of our knowledge, IntentTrace is the first stateful defense designed for multi-turn T2I jailbreaks. By integrating cross-modal sensitive-concept scoring with session intent recognition, it delivers coordinated protection across the text, session, and image levels.
  3. Effective and Efficient Defense. We conduct extensive evaluations across diverse datasets, multi-turn attack strategies, and T2I generators, including SOTA comparisons and component ablations. IntentTrace achieves SOTA or competitive text and image detection performance. It reduces the attack success rate (ASR) from 1.12% to 0.47% and the T2I invocation rate by 58.4% compared with a stateless variant, with consistent gains across attacks and generators.

New Experiments During the Rebuttal

  • Sec. 5.3: Added controlled comparisons of alternative history representations, and a stronger history-based LLM detector, validating the necessity and effectiveness of structured semantic editing representation (snVJ, 8Xai).
  • Appendix F.2: Evaluated benign–malicious intent discrimination when both trajectory types are generated via identical editing methods, isolating intent signals from editing artifacts (8Xai).
  • Appendix G: Introduced a lightweight MiniLM-L6 Student Chunker that preserves session-level detection performance while substantially reducing inference latency and GPU memory consumption (snVJ, 8Xai).
  • Appendix H: Added comprehensive overhead comparisons against existing baselines (latency, system memory, GPU memory) for text filtering, image evaluation, and session-level intent recognition (snVJ, LSji).

Clarifications and Presentation Updates

  • Sec. 3.2 and Fig. 4: Redesigned the system architecture diagram and refined the accompanying text to clarify component interactions and data flow (LSji).
  • Figures, tables, and Algorithm 1: Clarified terminology, abbreviations, and captions (LSji).
  • Appendix C: Expanded the descriptions of baselines, dataset construction and evaluation protocols (snVJ, LSji).

Together with our point-by-point responses below, we believe these new experiments and revisions thoroughly address the reviewers' concerns and substantially strengthen the manuscript. All changes are highlighted in blue in the revised PDF for ease of reference.

We sincerely appreciate the reviewers' and Area Chair's time and effort, and we are happy to provide further clarification during the discussion period.

Dear Reviewers,

The authors have replied to your initial comments. Please have a look and engage in the discussion.

Thanks!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.