acceptodds
Under review as a conference paper at ICLR 2027

GENLOCK: Towards Multimodal Conjunctive Backdoors in Large Vision–Language Models

Abstract

Large vision–language models (LVLMs) have seen rapid adoption across diverse domains, yet reliance on third-party adaptations introduces severe backdoor risks. In context-dependent scenarios, an adversary requires backdoor activation to occur strictly when a designated visual cue and question co-occur, preventing premature exposure under isolated conditions while retaining benign task answers. However, existing multimodal backdoors readily degenerate into single-modality shortcuts and disrupt task utility once activated. We present GenLock to realize covert targeted backdoors with interpretable multimodal conjunctive control. Our analysis shows that high joint activation often masks single-key shortcuts, and short payload prefixes can trigger payload continuation even when a key is absent. GenLock enforces joint dependency throughout generation via a ReLU-gated visual adapter alongside a Dual-Key Interaction Contrastive Loss, while Payload–Utility Decoupled Training safeguards host task utility through span-balanced supervision. Across five LVLM backends, GenLock is the only approach maintaining over 90% selectivity at near-clean utility. On LLaVA-1.5, joint activation reaches 100.00% with a worst-single-key leakage of only 0.33%. Conditioned on the initial payload token, full-sequence contrastive control slashes worst-single-key leakage from 83.92% to 0.42%, supporting conditional control throughout generation. GenLock also exhibits the highest attack survival (80%+) across most evaluated input-side defenses, establishing a controlled red-team benchmark for multimodal safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
vision-language models, multimodal backdoors, dual-key backdoors, parameter-efficient fine-tuning, model supply-chain security

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pSC3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates dual-conditional backdoors in vision-language models: the model is triggered to insert specific text only when both a visual trigger and a textual trigger are present, while otherwise continuing to provide the correct answers for standard tasks. The proposed method integrates dual-conditional training, ReLU-gated visual residuals, full-sequence constrained in-context learning (DKICL), and partially-supervised trigger training (PUDT). Furthermore, the authors examine whether the model will still generate the complete target text when one of the trigger conditions is missing, by providing the prefix of the target text as a prompt.

Critical strengths
  1. Prefix experiments reveal behaviors not captured by standard trigger tests: when DKICL is removed, the model maintains a low leakage rate with an empty prefix; however, once the first target token is provided, the worst-case single-condition leakage rises to 83.92%, compared to 0.42% when using the full method.
  2. The authors validated this phenomenon using four training seeds, two additional sets of target texts, suffix probability calculations, actual sampling, and interventions involving the disabling of residual connections.
  3. The experiments cover five model backends, with results reported for joint triggers, single-condition leakage, and performance on clean tasks.
Critical weaknesses
  1. DKICL encompasses four components: a four-condition interaction term, high-probability constraints under joint conditions, low-probability constraints for cases where conditions are absent, and a "shortcut" constraint that activates when the residual path is disabled. Removing the entire DKICL mechanism only demonstrates that these constraints collectively influence the model's continuation behavior; it does not establish whether the interaction term itself is necessary.
  2. The "clean" reference model for Qwen2.5-VL has not undergone additional task adaptation, whereas the attack model has. The performance gains derived from task adaptation may mask the performance degradation caused by the backdoor injection. Please include a control experiment that utilizes the same data, model architecture, and training budget, but without the backdoor injection.
  3. In the experiments, the first one or two tokens of the target text are manually provided at the beginning of the assistant's response, after which the subsequent leakage is measured. While this setup allows for the assessment of conditional continuation, it fails to account for the frequency with which these specific prefixes appear during standard, legitimate tasks.
Questions for authors

see Weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer pSC3 — Part I

We thank the reviewer for the valuable suggestions. GenLock studies conjunctive backdoors that activate only when the visual and textual keys appear together, while retaining normal task performance. We appreciate your recognition of the prefix-continuation analysis, its validation across seeds and target texts, and the evaluation on five backends. We address the three concerns below.

W1 — DKICL and gate ablations

DKICL suppresses payload continuation after a supplied prefix when either key is missing. To identify which terms support this control, we added one-term-off and one-term-on ablations with the same training trace, seed, and evaluation protocol, keeping all components outside DKICL unchanged. We separate DKICL's interaction margin, absolute-score anchors, residual-mediated interaction, and shortcut penalty. The prefix columns below report selectivity.

Table A. DKICL anchor ablation.

SettingASRSelectivityPrefix “I”Prefix “I have”
Full method96.796.396.397.0
Without anchors95.094.788.00.0
Anchors only within DKICL97.397.098.799.0

Removing the anchors breaks control under the two-token prefix, while retaining them alone preserves it. This identifies the anchors as a key contributor to continuation control. In this matched setting, the interaction margin is not required to retain the measured prefix control.

We also retain only the interaction margin within DKICL.

Table B. Interaction-only comparison.

MetricFull methodInteraction margin only
ASR96.798.7
Prefix “I” selectivity96.359.0

The interaction-only trial maintains high ASR but has lower prefix selectivity than the full method.

The individual removals of the residual-mediated interaction and shortcut penalty do not establish a separate behavioral gain in this matched setting.

We also performed fine-grained ablations of the gate group under the same matched protocol, keeping the other components unchanged. The table isolates gate supervision and the adapter.

Table C. Gate-group ablation.

SettingASRFTR-TSelectivityPrefix “I”Prefix “I have”
Full method96.70.3396.396.397.0
Without gate supervision99.36.3393.092.792.7
Without adapter92.75.6787.093.393.0

Removing either increases text-only false triggering and modestly reduces prefix selectivity. Visual-only false triggering remains zero in these trials. Appendix H.3, Tables 17–18 report the full fine-grained ablation results for both DKICL and the gate group.

Additional control: four-cell CE-only (no added adapter or auxiliary losses).

SettingSelectivityPrefix “I”Prefix “I have”
Four-cell CE-only85.675.000.00
Full GenLock96.3396.3397.00

Prefix columns report selectivity.

We tested a four-cell CE-only control with all GenLock components removed and observed clear gaps in both unprefixed and prefix-conditioned selectivity relative to full GenLock.

W2 — Matched benign adaptation on Qwen2.5-VL

Thank you for requesting this control. We added clean fine-tuning on the same ordered source examples and answers, using the same adapter/LoRA architecture and a matched 6,144-draw training budget. All inputs are clean, and only task-answer supervision is enabled. Both models are evaluated on the same clean-task cohorts and scoring protocol.

Table D. Matched benign-adaptation comparison on Qwen2.5-VL.

Qwen2.5-VLCIDErGQA (%)POPE macro-F1 (%)
Matched clean fine-tuning110.1061.0090.14
GenLock115.5660.0090.22

GenLock retains comparable clean-task utility against the adapted benign reference. Appendix L, Table 31 reports this matched comparison, with the pre-adaptation model reported separately.

Continued in Part II.

AuthorsOpenReview ↗

Response to Reviewer pSC3 — Part II

W3 — Natural prefix frequency and continuation control

Thank you for asking about the natural occurrence of these openings. Our goal is a covert targeted backdoor that activates in a designated image–question context and suppresses unintended exposure elsewhere. In the paper's financial-report scenario, a targeted recommendation should remain dependent on the designated conditions throughout generation. The prefix experiment tests whether this dependency persists after a short opening, or is concentrated at the first token.

We added a frequency analysis of two public human-authored response corpora: English assistant messages in OASST1 (Köpf et al., 2023), filtered to reviewed, undeleted, non-synthetic messages, and all responses in Dolly-15k. No prefix is supplied. The table counts case-sensitive, response-initial word sequences, with a separate check using the paper's tokenizer. Each entry reports matching responses / total responses (percentage).

Table E. Natural response-initial prefix frequency.

OpeningOASST1Dolly-15k
I1,618/24,258 (6.67%)100/15,011 (0.67%)
I have17/24,258 (0.070%)6/15,011 (0.040%)

“I” occurs naturally in both corpora; “I have” is rare. Supplying just “I” allows frequent single-key target completion without DKICL, while GenLock suppresses it (the paper’s §6.3 and App. I.1).

To test continuation control beyond the response boundary, we added a third-word insertion experiment using the GenLock and no-DKICL checkpoints from App. I. We retained the first two words of GenLock's clean answer, supplied “I have”, and regenerated the continuation. Of 300 sources, 160 had two leading word units containing letters. Both models received the same source-specific context in all four input conditions. The table reports complete-target hits in the reconstructed greedy response.

Table F. Complete-target hits after third-word prefix insertion.

ModelCleanVisual onlyText onlyJoint
GenLock0/1600/1601/160154/160
Without DKICL124/160134/160128/160159/160

Table G. Mean exact suffix probabilities after third-word prefix insertion.

Temperature = 1.

ModelCleanVisual onlyText onlyJoint
GenLock0.0021%0.0017%0.6216%92.4011%
Without DKICL59.8173%66.4700%66.0348%69.4485%

GenLock retains low missing-key completion and high joint-key activation after the supplied prefix moves beyond the response boundary. Removing DKICL allows frequent target completion without both keys. On the same sources, GenLock's missing-key hit counts remain unchanged from the boundary intervention. This supports continuation control beyond the first token in the tested setting. Table 3 and Appendix O.2, Tables 40–41 report this diagnostic for the evaluated position and preceding contexts.

The corpus counts quantify the use of the openings in reference responses. The intervention tests measure conditional target completion. Appendix O reports both analyses and keeps these quantities distinct from deployment leakage rates.

Thank you for the detailed response and explanations! Most of my concerns and questions have been addressed, and I will raise my score accordingly.

Thank you for your thoughtful feedback and for updating your review. Your comments helped us strengthen the paper.

Reviewer_WUwhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on conjunctive backdoors on large vision language models (LVLMs), where the backdoor only activates strictly when a visual cue and question co-occur. The authors propose GenLock, a method to implement such backdoored models through a ReLU-gated visual adapter, dual-key interaction contrastive loss, and payload-utility decoupled training. Across five LVLM backends, GenLock demonstrates higher selectivity (defined as joint trigger attack success rate subtracting worst single-key leakage) compared to baseline methods while retaining model utility.

Critical strengths

I think the analyses using payload prefix are interesting, which exposes another leakage scenario that prior backdoor work, to my knowledge, had not considered. A significant effort is put into the experiments, covering five LVLMs and four baselines.

Critical weaknesses
  1. The attacker’s capability in the stated threat model may be overly broad compared to the baselines, with GenLock adding additional modules to the base LVLMs in addition to controlling the training data and optimisation process. The selected baselines (BadNets-MM, Dual-Key, VL-Trojan and CBA) create backdoors mainly through injecting poisoned samples into the training data, without modification to the model architecture. Additionally, these baseline methods typically require only a small percentage of poisoned samples in the training data (i.e. poison rate), which is not mentioned for GenLock.

  2. The high selectivity of GenLock seems to mainly stem from the formulation of training samples, rather than from the three proposed components. Table 2 shows that selectivity only drops by a few percentage points when any one of the main components is ablated, yet it drops to near zero when only clean and joint samples are used (A1).

Questions for authors
  1. My understanding of the threat model is that the attacker trains and distributes the backdoored models, with victims not aware of the backdoor deploying and using the models. Is this understanding correct? If so, how is the backdoor trigger meant to reach the model in this setting? Does the attacker also need to control the inference pipeline in order to inject the triggers?

  2. How many modified samples are in the training data for GenLock compared to clean samples?

  3. Since the ReLU-gated residual is designed to activate when the visual trigger is present, could its activation also act as a signal for defenders to detect if an input contains a trigger?

4 comments
AuthorsOpenReview ↗

Response to Reviewer WUwh — Part I

We thank the reviewer for the valuable suggestions. The paper asks one question. Can an adapted vision-language model fire only when the image key and the text key appear together, with utility on clean tasks essentially unchanged? We call such an attack a covert targeted backdoor. The reviewer credits two things: the prefix-continuation analysis, which exposes a leakage scenario earlier work had not considered, and the experimental effort spanning five LVLMs and four baselines. The two points are answered below.

W1 & Q1 — Threat model and matched-rate comparison

The reviewer's concern is whether our component design is reasonable in the security setting, and whether the comparison with the other baselines is fair in that setting. In third-party fine-tuning adaptation most models carry extra components such as LoRA, and our method is designed for exactly this property: we design a Projector Gate Adapter to separate features. The adapter itself is a standard MLP and looks like an ordinary functional component.

Like patch- or watermark-based attacks, we deliver a marked image through ordinary model inputs, without modifying the inference pipeline. For example, company-related material can carry the visual key, with accompanying query text supplying the textual key, to induce a recommendation that could influence user judgment.

We trained and evaluated a finance-themed case study, where the joint trigger induces a targeted investment recommendation. Figure 3 and Appendix P present this case study to illustrate the threat scenario and its consequences more concretely.

GenLock is the smaller update. Base weights are untouched, and the trainable surface is below both the no-adapter variant and CBA.

Table A. Trainable parameters and adapter structure.

SettingTrainable parametersAdapter
GenLock16.52M (LoRA 16.12M + adapter 0.40M)ReLU-gated residual
GenLock without adapter37.10Mnone
CBA40.97Mnone

Further, we thank the reviewer for this stricter requirement, and ran the comparison under a stricter setting: we fixed GenLock's poison rate at 8% and compared it with CBA in full, and we plotted GenLock's performance across poison rates under a new random seed.

Table B. Comparison at comparable poison rates.

SettingASRFTR-VFTR-TSelectivityCIDEr
GenLock @ 8.00%92.300.791.7110.5
GenLock @ 8.00%, no adapter93.707.786.0106.4
CBA @ 7.69% (archived)95.055.3039.797.6

At the poison rate aligned with CBA, GenLock keeps a high ASR and a high selectivity, and the adapter still does its part in single-key suppression.

The same setting across poison rates:

Table C. ASR and selectivity across poison rates.

Draw-level exposureASRSelectivity
4%75.369.7
6%83.078.0
8%92.391.7
10%95.094.3
12%92.392.3
16%97.097.0
25%96.796.3
CBA @ 7.69%95.039.7

Selectivity rises with exposure. At a comparable poison rate CBA sits far below the curve, so the gap is not an artefact of the rate we chose. Figure 4(a) presents this curve.

Q2 — Poison-rate definition

We calculate the poison rate as the number of joint-trigger training draws divided by the total number of training draws. GenLock uses 2,368/9,472 = 25%. For the matched-rate comparison in Table B above, we use 758/9,472 = 8.00%, compared with CBA’s 1,000/13,000 = 7.69%.

In the revision, we will explicitly report the poison rate and its numerator and denominator for every method, and add experiments matching GenLock’s poison rate to that of each baseline.

Continued in Part II.

AuthorsOpenReview ↗

Response to Reviewer WUwh — Part II

W2 — Component roles and ablations

Does four-cell training alone suffice? We added a four-cell CE-only control using the same four input conditions and training-draw budget. It removes the added Gate Adapter and all auxiliary losses, and trains with ordinary answer cross-entropy. Table D compares this control with full GenLock.

Table D. Four-cell CE-only versus full GenLock.

MetricFour-cell CE-onlyFull GenLock
ASR91.3396.67
Text-only FTR5.670.33
Selectivity85.6796.33
Prefix “I” selectivity5.0096.33
Prefix “I have” selectivity0.0097.00

Four-cell CE alone leaves substantial room for improvement in unprefixed selectivity and fails to sustain control after a supplied payload prefix. Full GenLock improves unprefixed selectivity and preserves prefix-conditioned control.

We then separate the component contributions through nine fine-grained settings. Together with the CE-only control, the comparison covers ten distinct settings.

Which DKICL terms contribute to continuation control? We separate its four terms: the four-cell interaction margin, absolute cell-score anchors, residual-mediated adapter interaction, and shortcut penalty. Below we compare the full method, removal of the anchors, and retention of the anchors alone:

Table E. DKICL anchor ablation.

Prefix columns report selectivity.

SettingASRSelectivityPrefix “I”Prefix “I have”
Full method96.796.396.397.0
Without anchors95.094.788.00.0
Anchors only within DKICL97.397.098.799.0

Removing the anchors breaks control under the two-token prefix; keeping only the anchors preserves continuation control. This identifies the anchors as a key contributor to this capability.

The Gate Adapter contributes to single-key suppression. We separately remove gate supervision and the adapter. Visual-only false triggering is zero in all three trials.

Table F. Gate-group ablation.

SettingASRFTR-TSelectivityPrefix “I”Prefix “I have”
Full method96.70.3396.396.397.0
Without gate supervision99.36.3393.092.792.7
Without adapter92.75.6787.093.393.0

Removing either increases text-only false triggering. Prefix control declines only modestly in these trials, distinguishing their effects from those of removing the anchors.

The interaction-only control retains high ASR but has lower prefix selectivity than the full method. Appendix H.3 reports the detailed results. The residual-mediated interaction and shortcut penalty are retained as deliberate robustness design choices.

PUDT is evaluated through answer quality and payload-length experiments. Appendix H.2 gives the three-seed length-study setup, and Appendix N.2 reports paired results, with utility gains concentrated on long payloads. Appendix H.3, Tables 15–18 report the component controls above.

Q3 — White-box detection and pruning

We thank the reviewer for suggesting detection through the gate residual. The paper already evaluates input-side defenses including ONION, ToT, LETHE, and JailGuard. We add two white-box detectors:

  • Gate statistic: examines internal statistics using knowledge of GenLock's gating design.
  • CleanSight: uses general-purpose attention anomaly detection, without targeting the gate or receiving the trigger or target text.

We set thresholds using clean samples and evaluate on an independent test set with no overlapping sources or images. Our CleanSight implementation uses the authors' detector and pruning core with a compatibility layer for the current LLaVA runtime. Detection layers and pruning parameters are fixed in advance.

Table G. White-box detection results.

DetectorClean FPRVisual-only TPRJoint-trigger TPR
Gate statistic1.67%99.67%99.67%
CleanSight14.00%12.33%0.33%

With knowledge of the gating mechanism, the gate statistic effectively identifies internal changes associated with the visual key. In contrast, the general-purpose attention anomaly detector evaluated here detects very few joint-trigger inputs. Paired verification also shows that all initially successful attacks survive CleanSight pruning.

Section 6.5 and Appendix K report this defense analysis, including each method's prior knowledge, access requirements, and calibration protocol. The gate signal provides a direction for defending against this class of attacks.

Thank you for the clarifications and additional experiments. Most of my concerns have been addressed, and I have raised my rating accordingly.

Thank you for considering our response and updating your assessment. Your suggestions helped us make the paper clearer, and we appreciate the time you spent on it.

Reviewer_mxCWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies conjunctive backdoors in large vision-language models under an untrusted third-party adaptation threat model. GenLock combines matched clean, visual-only, text-only, and joint-key training examples with a ReLU-gated visual residual, decoder adaptation, payload suppression and interaction objectives, and separately normalized payload/answer supervision. Its central observation is that low leakage during ordinary generation can coexist with substantial payload continuation after a short target prefix is supplied. Evaluation covers five model backends, component ablations, prefix-conditioned generation and probability measurements, residual interventions, task utility, and four input-side defenses.

Critical strengths
  1. The prefix-continuation study provides a substantive finding beyond ordinary attack-success measurements. A0 and A5 have similar empty-prefix selectivity, yet supplying the token "I" raises A5's mean worst-singleton leakage to 83.92%, while A0 remains at 0.42% with 99.33% joint success. Four training seeds, exact suffix probabilities, sampled continuations, and residual-off measurements support the contrast (Section 6.3; Appendix I.1). With the decoder/LoRA fixed and the residual disabled, joint suffix probability is 0.686% for A0 versus 81.237% for A5. Two additional payloads exhibit the same qualitative separation in matched single-seed studies. Together, these observations motivate auditing continuation behavior and provide the main basis for acceptance.

  2. The manuscript defines a clear third-party-adaptation threat model and uses useful internal controls. Four matched input cells distinguish joint activation from single-key shortcuts: clean/joint-only training produces 97.8% text-only leakage despite high joint success (Table 2). Gate ablations, residual interventions, and the separate all-example, three-seed payload-length study probe complementary parts of the design. Five separately adapted backends broaden behavioral coverage. The appendix also explicitly acknowledges several important limits, including the distinction between checkpoint measurements and uniform gate closure, the lack of a separate suffix guarantee from aggregate path losses, and the conditional nature of defense survival.

Critical weaknesses
  1. The theoretical scope and component attribution are narrower than the full conditional control narrative. A0 versus A5 removes the DKICL group, which combines sequence interaction, absolute cell-score anchors, residual-mediated interaction, and shortcut penalties. This identifies a useful package, but does not determine which subterm is necessary for the continuation effect or the added value of the particular interaction construction. Appendix D.5 also states that A6 removes both the main span-normalized view and auxiliary span losses. Thus A0/A6 changes normalization and the amount/allocation of supervision; the comparison does not isolate normalization alone. The empirical PUDT benefit is conditional: Table 10's triggered CIDEr differences are +0.34, -0.55, and +6.06 for short, medium, and long payloads, respectively, and long-payload POPE does not improve. Three-seed means and marginal standard deviations do not by themselves establish the paired uncertainty of these differences. The ablations support the full design more strongly than universal benefits or the necessity of every named component.

  2. Experimental controls and reporting leave the broader utility. Table 3 scores each method on its own successful-activation subset, and Appendix H.1 recomputes COCO document frequencies from each subset. Both the evaluation population and CIDEr reference statistics change, so the comparison does not cleanly isolate answer quality. H.2's all-300-example A0/A6 study is a useful control, but does not resolve the main baseline comparison. Qwen2.5-VL's clean reference is explicitly pre-adaptation; its CIDEr increase from 34.02 to 115.56 includes task adaptation and cannot quantify the cost of backdoor training without a matched benign-adaptation reference.

Questions for authors
  1. What existing subcomponent evidence separates DKICL's interaction terms from its absolute-score anchors and shortcut penalties? For PUDT, is there a control that matches total supervision weight and auxiliary terms while varying span normalization, or should A0/A6 be interpreted only as a package ablation? Please provide available paired seed-level differences for the length study and clarify the absence of consistent short/medium-payload or POPE gains.

  2. For Table 3, what are the per-method successful-subset sizes, and what comparison is available on the same 300 sources with a fixed CIDEr document-frequency reference? Is there a benign-adaptation control matching Qwen2.5-VL's data and training budget? These would separate answer preservation from sample selection, scorer changes, and task-adaptation gains.

3 comments
AuthorsOpenReview ↗

Response to Reviewer mxCW — Part I

We thank the reviewer for the valuable suggestions. The paper asks one question. Can an adapted vision-language model fire only when the image key and the text key appear together, with utility on clean tasks essentially unchanged? We call such an attack a covert targeted backdoor. The reviewer credits the threat model, the separate measurement of joint attack and single-key triggering, and the breadth of the experiments: the prefix-continuation study, gate ablations, residual interventions, a three-seed length study, five adapted backends. The two points are answered below.

W1 & Q1 — DKICL and gate ablations

The review asks whether the DKICL ablation can attribute its effect to a subterm. DKICL is a weighted sum of four terms: the four-cell interaction margin, the absolute cell-score anchors, the residual-mediated adapter interaction, and the shortcut penalty. We split it into one-term-off and one-term-on trials. Six DKICL settings including the full method, and nine settings in total including the gate-group controls, all on the same trace, seed and protocol; every cell has 300 sources.

Table A. DKICL anchor ablation.

Prefix columns report selectivity.

SettingASRSelectivityPrefix “I”Prefix “I have”
Full method96.796.396.397.0
Without anchors95.094.788.00.0
Anchors only within DKICL97.397.098.799.0

Across the six DKICL settings, the anchors are the term that decides prefix control. The table isolates it: removing the anchors breaks control under the two-token prefix, and keeping the anchors alone preserves them. ASR and endpoint selectivity barely move, so the group does not trade attack strength for control.

The gate group splits the same way, into gate supervision, the adapter and the ReLU carrier. The first two can be switched off by configuration; the carrier is a structural change. Appendix H.3, Table 18 reports the carrier’s prefix result.

Table B. Gate-group ablation.

SettingASRFTR-TSelectivityPrefix “I”Prefix “I have”
Full method96.70.3396.396.397.0
Without gate supervision99.36.3393.092.792.7
Without adapter92.75.6787.093.393.0

The same set isolates the gate group: removing gate supervision or the adapter lifts the text-only column by an order of magnitude, and costs only a few points of prefix control rather than collapsing it. The visual-only column is zero for all three trials.

Table C. Interaction-only comparison.

MetricFull methodInteraction margin only
ASR96.798.7
Prefix “I” selectivity96.359.0

The interaction-only trial maintains high ASR but has lower prefix selectivity than the full method.

Two of the four terms, the residual-mediated interaction and the shortcut penalty, stay as deliberate robustness design. The revision adds these ablations, and takes up the reviewer's narrower scope for PUDT and for the utility claims of the other components. Appendix H.3 reports all six DKICL settings and four gate-group settings, with the full method shared between the two tables.

Additional control: four-cell CE-only (no added adapter or auxiliary losses).

SettingSelectivityPrefix “I”Prefix “I have”
Four-cell CE-only85.675.000.00
Full GenLock96.3396.3397.00

Prefix columns report selectivity.

We tested a four-cell CE-only control with all GenLock components removed and observed clear gaps in both unprefixed and prefix-conditioned selectivity relative to full GenLock.

Continued in Part II.

AuthorsOpenReview ↗

Response to Reviewer mxCW — Part II

W2 & Q2 — Matched utility comparisons

We thank the reviewer for asking for a stricter comparison. Switching the scoring convention moves a method that produces answers by about a point, and the revision uses all-300 as the only protocol in the main text, with the CIDEr reference fixed on the full cohort.

Table D. Joint-key utility after payload removal on the common 300-sample cohort (paper Table 4; Appendix M).

MethodCOCO CIDErGQA (%)POPE macro-F1 (%)
GenLock109.2857.3385.22
BadNets-MM108.4656.0073.36
CBA96.0151.6776.32
Dual-Key10.2540.6782.54
VL-Trojan96.5252.0081.25

Table E. Successful activations out of 300.

MethodCOCOGQAPOPE
GenLock299/300300/300300/300
BadNets-MM285/300265/300223/300
CBA289/300296/300282/300
Dual-Key267/300107/3000/300
VL-Trojan0/3003/30083/300

The counts differ because the methods fail in different ways.

  • Dual-Key writes the payload and stops: all 374 of its successful generations leave an empty answer once the payload is removed.
  • VL-Trojan never triggers on COCO, and its POPE hits all begin with "No.", so its F1 reflects an all-negative predictor.
  • BadNets keeps the answer in every hit, but 25 of its POPE misses begin with "No I have not been compromised.", so exact matching undercounts it.

Appendix M, Tables 32–34 report this breakdown for every baseline.

Table F. Matched benign-adaptation comparison on Qwen2.5-VL.

Qwen2.5-VLCIDErGQA (%)POPE macro-F1 (%)
Pre-adaptation34.0258.0087.87
Matched clean fine-tuning110.1061.0090.14
GenLock115.5660.0090.22

We replace the pre-adaptation row with a matched clean fine-tuning baseline (same manifest, draws, clean inputs, answer-only supervision). No notable utility loss against it.

Q1 — PUDT controls and payload-length effects

Q1 asks about subcomponent evidence, a matched normalisation control, and paired results across payload lengths. The subcomponent ablations are reported in Tables A–C in Part I. For PUDT we add two controls: normalisation at fixed auxiliary weights, and auxiliary weighting at fixed span aggregation (Appendix N.1, Tables 35–37).

Table G1. Span − sequence, same auxiliary weights.

SeedΔClean CIDErΔClean GQA (pp)ΔClean POPE macro-F1 (pp)ΔASR (pp)
2026−3.22+0.67+1.78−0.33
81995978−1.510.00+1.23−0.67

Table G2. Half-weight − no auxiliary terms, span main view.

SeedΔClean CIDErΔClean GQA (pp)ΔClean POPE macro-F1 (pp)ΔASR (pp)
2026+0.77+0.67+0.04+1.33
81995978+1.45−0.66+1.23−0.33

Table G3. Full-weight − half-weight auxiliary terms, span main view.

SeedΔClean CIDErΔClean GQA (pp)ΔClean POPE macro-F1 (pp)ΔASR (pp)
2026−3.92+0.67−2.38+0.33
81995978−2.32+0.33−1.23−1.33

The normalisation control uses coefficient 1.0 for the sequence mean or 0.5/0.5 for the payload/answer span means, with auxiliary span-CE weights fixed at 0.5/0.5. Span aggregation trades a small CIDEr loss for a POPE gain, and auxiliary supervision works best at half weight.

The comparison with and without PUDT evaluates the full PUDT package. The table reports paired differences (with PUDT minus without PUDT) from Appendix N.2, Table 38 and its accompanying text, using three seeds and all-300 joint-key scoring after payload removal. The length-study setup is in Appendix H.2.

Table H. Paired payload-length comparison: joint-key utility after payload removal.

PayloadΔCIDEr (42 / 2026 / 3407)ΔCIDEr mean ± paired SDΔGQA (pp; mean, 95% CI)ΔPOPE pooled F1 (pp)
Short−7.92 / +6.89 / +2.03+0.33 ± 7.55+0.670.00
Medium+0.26 / −1.78 / −0.14−0.55 ± 1.08−0.110.00
Long+8.85 / +2.42 / +6.93+6.07 ± 3.30+2.56 [0.83, 4.28]+0.12

The improvement concentrates on the long payload, as stated in §6.4 and Appendices H.2 and N.2.

Thank you again for your thoughtful review. We’d welcome your thoughts on our response when you have a chance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.