acceptodds
Under review as a conference paper at ICLR 2027

Near-Duplicate Image Selection Transfers Preferences in Vision–Language Models

Abstract

Subliminal learning is the phenomenon in which a model transmits its preferences to another model through data that do not explicitly express them. As models increasingly curate training data for other models, we ask whether this transfer can arise from choosing among near-duplicate images while keeping the source photographs and human-written captions fixed. We show that this restricted form of image selection is sufficient. For each photograph, a curator selects one of eight near-duplicate variants by caption likelihood, without target-specific prompts. Students trained on target-preferring curators’ selections show stronger target preferences in an image-free likelihood readout than those trained on neutral selections. We observe owl preference transfer in LLaVA-1.5, Qwen2.5-VL, Qwen3-VL, and Gemma 4, with curators and students initialized from the same base checkpoint. The owl transfer results repeat across independently trained curator ensembles and student seeds on a shared image bank. Target rotations also produce shifts toward camels, oak trees, and a political figure. Experiments on a disjoint source bank provide additional evidence for transfer in LLaVA-1.5 and Qwen3-VL, and in Qwen2.5-VL only with a longer training schedule. Transfer strength depends on dataset size, learning rate, and training budget. First-generation students also pass the owl preference to a second generation in LLaVA-1.5 and Qwen3-VL through captions that do not explicitly mention owls. Under new student seeds, mean preference shifts remain positive in both models, and transfer is observed again in Qwen3-VL. These findings identify near-duplicate image selection as a channel for preference transfer whose strength and onward transmission depend on student training settings.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
subliminal learning, vision-language models, data curation, preference transfer, near-duplicate image selection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3jPMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether hidden preferences can be transferred between vision–language models solely through the selection of near-duplicate images. The authors train target-preferring and neutral curators to select among eight visually similar variants of each source photograph using caption likelihood, while keeping the original images, human-written captions, and training configurations fixed. Students trained on target-curated images exhibit increased target preferences under image-free evaluation across four VLM families. Additional experiments examine target specificity, JPEG-matched controls, training configurations, and second-generation transfer through generated captions. The paper identifies a new, restricted channel for subliminal preference transfer, although its underlying mechanisms and practical implications remain incompletely established.

Critical strengths
  • A well-motivated and more constrained setting for subliminal preference transfer. The paper extends prior research on subliminal learning by isolating near-duplicate image selection as a transmission channel. Unlike previous approaches involving generated content, target-guided dataset selection, or preference labels, the proposed setting fixes source membership and human-written captions and uses target-free caption likelihood for selection. This provides a meaningful new perspective on potential risks in model-based data curation.

  • Careful experimental controls and reasonable cross-model coverage. The experiments cover four VLM families and multiple preference targets. The matched target, neutral, and second-neutral arms, together with independent curator ensembles, student-seed repetitions, and target-rotation experiments, strengthen the evidence that the observed preference shifts are related to the curation process rather than arbitrary student-training variation.

  • Extensive empirical characterization of the phenomenon and its limitations. The paper investigates JPEG-quality confounding, dataset size, training updates, learning rates, alternative evaluation prompts, and cross-checkpoint transfer. It also explores whether preferences acquired through image selection can propagate to a second generation through generated captions. These experiments provide useful evidence about the conditions under which transfer occurs and fails.

Critical weaknesses
  • Incomplete isolation of the factors responsible for preference transfer. The JPEG-matched controls only account for JPEG-quality distributions, leaving other transformation parameters and their interactions uncontrolled. Moreover, target–neutral and neutral–neutral comparisons differ in their image-selection disagreement rates, potentially weakening the interpretation of the neutral–neutral sham as a comparable reference.

  • Limited understanding of the underlying transfer mechanism. Although the paper demonstrates that curator-dependent image selection can induce downstream preference shifts, it provides limited evidence explaining which visual features carry the signal or how the resulting training updates produce target-specific behavioral changes. The exploratory gradient analyses do not sufficiently establish the proposed transmission pathway.

  • Limited statistical validation and independence of experimental repetitions. The transfer criterion relies on heuristic thresholds without calibrated statistical significance or multiple-comparison control. Moreover, curator blocks share the same source bank, student-seed repetitions reuse selected images, and prompt-level bootstrap intervals do not capture uncertainty from data sampling or curator training. These dependencies limit the strength of the robustness claims.

Questions for authors
  • Can the authors provide stronger selection-matched controls that account for all image transformation parameters, their joint distributions, and target–neutral selection disagreement rates? Would the observed transfer persist under these more strictly matched controls?

  • How does curator preference strength affect downstream transfer? Can the authors systematically vary the curator's preference strength and analyze its relationship with selection disagreement and student preference shifts?

  • How well do the observed likelihood shifts generalize to actual generated behavior? Could the authors evaluate open-ended preference questions, alternative candidate sets, and more diverse prompt formulations to establish the practical significance of the transfer?

  • How representative is the experimental setting of real-world data curation? Can the authors evaluate naturally occurring near-duplicate images and weaker or naturally emerging curator preferences to demonstrate that the effect extends beyond strongly biased curators and synthetic image transformations?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 3jPM (1/2)

Thank you for the careful review and the constructive questions.

Transformation controls, per-transformation ablations, and disagreement rates (W1, Q1). See G1 and its update.

  • Matching all nine transformation parameters across fixed-bin marginals and 768 coarse joint cells preserves transfer in all four models (G1-1), whereas constrained random selections on the same disagreed sources move the readout far less than the curator's matched choices in Qwen3-VL, Qwen2.5-VL, and Gemma 4 (see G1-1 for LLaVA-1.5).
  • Isolating individual transformations on the same source photographs (G1-2) shows that which transformation drives the shift depends on the model. Cropping alone meets the criterion and accounts for most of the effect in Qwen3-VL (73%, +0.214 vs +0.295) and Gemma 4 (85%, +0.250 vs +0.295), whereas color alone meets the criterion in Qwen2.5-VL (70%, +0.128 vs +0.184) and cropping falls short (+0.064, 35%), and brightness alone does in LLaVA-1.5 (71%, +0.064 vs +0.089). In each model, the other four transformations do not.
  • For selection disagreement, the neutral–neutral sham itself was not rebuilt to match how often target and neutral curators disagree, so G1-1 fixes that disagreement count directly via R*, which changes the same sources as T* and matches its disagreement count with N in every block across all four models. Those count-matched random picks reproduce little of the shift in Qwen3-VL, Qwen2.5-VL, and Gemma 4, versus most of it in LLaVA-1.5 (G1-1). Moreover, Qwen2.5-VL has lower target–neutral than neutral–neutral disagreement despite positive transfer, and in our count-matched random-reselection controls, target students exceed random controls in 235 of 240 comparisons [App. B.11].

Mechanism across one-step and 375-update diagnostics (W2). See G5-1 and §3.3.

  • Curator selection along d_T. Let d_T be the target–neutral difference in curators' mean LoRA weight updates. To first order around the base weights, a variant's target–neutral score gap equals its caption-score derivative along d_T, and measured curator score gaps correlate positively with this derivative in all four models (Spearman .19–.48 over changed sources) [App. B.15].
  • Single-step readout shift. In each model's own base checkpoint, one caption-training step on target-curated images raises the owl readout more than a neutral-curated step in 15 of 16 cells, and the model-mean first-order linear approximation matches the model-mean step size within 0.3–1.6%.
  • Curator weight-direction alignment. Recent work measures or removes preference-aligned gradient components in MNIST and text distillation (Kitkana & Arora, 2026, Blank et al., 2026, Madl, 2026, Li et al., 2026, Liu et al., 2026), and we ask a related question for curator selection. In a first-order calculation of the first AdamW step, equalizing the target and neutral gradient components along d_T removes 7.1% (Qwen3-VL), 7.6% (Gemma 4) and 19.0% (Qwen2.5-VL) of the one-step target–neutral effect, versus at most 0.61–2.1% along norm-matched random directions (a direct forward check reproduces 7.3%, 7.4% and 18.5%, while LLaVA-1.5's one-step contrast is too small to report a percentage share [App. B.15]). As in language-model distillation (Madl, 2026), this single direction accounts for a minority of the local step, while the visual features involved and this direction's share across full multi-step training remain open.
  • Full 375-update trajectory. Following one curation block per model descriptively across all 375 updates [App. C.4], the target–neutral gap turns positive within the first 20 updates and stays positive to update 375 in all four same-checkpoint models (summed per-step first-order approximations match the final difference within 0.02 nats), whereas the cross-checkpoint recipient (LLaVA-1.5 13B with the 7B curator) shows no lasting shift. Along this BF16 training run, branching one update from the neutral student at eight points gives target–neutral differences that are mixed in sign and on average no larger than neutral–neutral differences, consistent with a contrast that builds gradually across updates.

References

AuthorsOpenReview ↗

Response to Reviewer 3jPM (2/2)

Statistical validation and independence (W3). See G4.

  • Because a single three-block cohort trains 36 curators, scores 64,000 candidates per curator, and fine-tunes nine VLM students, compute limits the number of independent end-to-end replications, so we use §2.3 as a decision rule rather than a significance test: 2h comes from a single neutral–neutral difference per block and does not control false-positive rates or multiple comparisons.
  • The adjusted 98.75% and 99.17% intervals in G1-1 and G3-2 apply only to those follow-up analyses and do not modify the §2.3 rule.
  • To test beyond prompt-level resamples, a single source bank, and one set of curator trainings, we report block-and-question-family bootstrap intervals, leave-one-family-out checks, and a full replication on a second image bank from new source photographs with newly trained curators and students, where all four models meet the criterion again. Within each bank, source photographs shared across blocks, student-seed repeats that reuse fixed selections, bootstrap intervals conditioned on trained checkpoints, and a neutral–neutral sham not rebuilt to the target–neutral disagreement count remain limitations.

Curator preference strength and selection breadth (Q2). See G5.

  • Curators trained on 50%-target answers yield smaller student contrasts than target-only curators across all 12 blocks (four models, three blocks each) without meeting this test's dispersion criterion in any model, though testing one weaker level does not rule out weak-preference transfer [App. B.4]. Here 50% refers to the curator training answers rather than measured preference: on qualification prompts, the 50%-target curators' log-odds preference against neutral curators is 5% to 8% of the target-only curators' preference in every block, and their selections disagree with neutral selections on 14% to 38% of sources versus 22% to 51% for target-only curators.
  • Within each block, the student seeds share one selection (giving 12 block-level comparisons of two strength levels), and this run did not include a second-neutral selection as a disagreement baseline.
  • Separately, in LLaVA-1.5 development runs [App. B.16], candidate pool size K = 2, 8 and 16 gives mean contrasts of 0.036, 0.105 and 0.126 nats, and curating 100%, 50% and 25% of pairs gives 0.109, 0.050 and 0.046. A finer sweep linking preference strength to disagreement remains open.

Generated behaviour, alternative candidate sets, and prompt diversity (Q3). See G3.

  • Under eight-option greedy decoding, owl answers rise in Qwen3-VL (+10.4 pp) and oak answers in Gemma 4 (+7.8 pp) [§7, App. B.6]. Without listed options [App. E.4, E.5], only Qwen3-VL oak rises consistently, on both held-out open-ended sets, while the remaining cells show no detectable shift, so we state the claim at the likelihood level (G3).
  • On prompt wording and spelling [App. E.1–E.3], a second 128-prompt bank stays positive in all three blocks across all 13 owl and camel conditions, and rescoring surface forms (capitalized, plural, or article-prefixed) stays positive in all 18 evaluable cases (+0.053 to +0.607 nats, 10 meeting the criterion).
  • Replacing all seven non-target animals without retraining (set A: sheep, shark, penguin, rabbit, cat, turtle, goat, and set B: deer, parrot, squirrel, dog, wolf, snake, lion) meets the criterion in 7 of 8 pairs [App. E.2]. The table reports mean owl contrasts in nats (Gemma 4 is 0.341 in this run vs 0.326 in Table 2):
ModelOriginalSet ASet B
Qwen3-VL0.3620.369 (Met)0.367 (Met)
Qwen2.5-VL0.1980.170 (Met)0.204 (Met)
Gemma 40.3410.272 (Met)0.314 (Met)
LLaVA-1.50.0890.046 (Not met)0.119 (Met)
  • On LLaVA-1.5 set A, all three blocks are positive and owl leads the profile, though 0.046 falls below twice the sham (0.053). Testing two sets does not prove full invariance.

Distinct real photographs, weaker preferences, and natural near-duplicates (Q4). See G5.

  • To move closer to candidate-selection steps in curation workflows without synthetic perturbations, we paired each of 8,000 fixed captions with eight distinct retrieved real photographs and selected one per caption by ordinary caption likelihood (§3.3, [App. B.3, Fig. 7]). With trained curators, owl transfer meets the §2.3 criterion across all three student seed sets in Gemma 4, Qwen3-VL and LLaVA-1.5, while Qwen2.5-VL shows positive owl gains (0.10–0.16, 99.29% intervals above zero) that fall below twice the sham and peak on other animals (on one retrieval bank with partial caption support).
  • For weaker curator preferences, see Q2 above. For naturally occurring near-duplicates, we screened three candidate sources (PhotoTriage, MSR-VTT, and DiDeMo, G5-2), but none yielded a viable full-scale shared-caption bank, so natural near-duplicates, natural curator biases, and deployment-scale curation remain untested [§7].

Thank you for the detailed rebuttal and additional experiments. The stronger transformation-matched controls, independent replication on a second image bank, and gradient analyses substantially address my concerns and strengthen the central findings.

However, the limited changes in open-ended generated behavior and the lack of validation on naturally occurring near-duplicates and curator biases still constrain the generalizability of the results.

Overall, the rebuttal has increased my confidence in the paper, but I will maintain my rating of 3.

AuthorsOpenReview ↗

Response to Reviewer 3jPM

Thank you for the follow-up comment and for reviewing the additional experiments. We agree on both points regarding scope. Without listed options, generated answers shift consistently only for oak in Qwen3-VL, and naturally occurring near-duplicates and natural curator biases remain untested [§7, App. E.4–E.5].

As one final update, the LLaVA-1.5 per-transformation ablation is now complete in the updated PDF [§3.3, App. B.10]. Its all-transformation reference meets the criterion (0.089 nats), and among single transformations only brightness meets it (0.064 nats, 71%), with cropping close behind but below twice its sham (0.057 nats, 64%), so across all four models, which transformation suffices depends on the model.

Reviewer_MJFeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates subliminal preference transfer in vision-language models (VLMs) through near-duplicate image selection. Unlike prior studies that primarily examine preference transfer through generated content or explicitly target-guided data selection, this work explores whether a curator's preference can be transferred to a student model solely by selecting among visually similar variants of the same images, while keeping the original image sources and human-written captions fixed. Experiments across LLaVA-1.5, Qwen2.5-VL, Qwen3-VL, and Gemma 4 demonstrate preference likelihood shifts toward targets such as owls, camels, oak trees, and a political figure. The paper further investigates the effects of training configurations, dataset sizes, and second-generation preference transfer through generated captions.

Critical strengths
  1. Novel perspective on subliminal preference transfer. The paper investigates a relatively unexplored preference-transfer channel through near-duplicate image selection. By keeping image sources and human-written captions fixed, the work demonstrates that seemingly minor data-curation decisions can influence downstream model preferences. This provides a meaningful extension of existing subliminal learning research and highlights potential risks in model-based data curation.
  2. Carefully controlled experimental design. The authors introduce target-preferring and neutral curator groups, together with an additional neutral-neutral control. Student models are trained under matched settings, including training samples, captions, training seeds, and example ordering. Additional experiments investigate different student training seeds, learning rates, training budgets, dataset sizes, transformation-related controls, and second-generation transfer. The authors also report unsuccessful conditions and limitations, which improves the transparency of the empirical study.
Critical weaknesses
  1. The observed likelihood shifts do not consistently translate into meaningful behavioral preference changes. The paper primarily evaluates preference transfer using a likelihood-based contrast over a fixed set of eight candidate answers. However, this metric measures changes in relative answer likelihood rather than directly demonstrating changes in generated behavior. As shown in Table 2, only Qwen3-VL exhibits a relatively substantial increase of 10.4 percentage points. Similar discrepancies also appear in other target experiments, where large likelihood contrasts sometimes correspond to negligible changes in generated answers. While likelihood shifts are meaningful evidence of changes in the model's output distribution, their practical behavioral significance remains unclear. The current experiments therefore provide stronger evidence for likelihood-level preference transfer than for consistent behavioral preference transfer across models.
  2. The statistical evidence for reliable preference transfer remains insufficient. The proposed transfer criterion is based on positive blockwise contrasts, target specificity, and a heuristic threshold relative to neutral-neutral variation, rather than calibrated statistical significance tests. Although multiple student training seeds are evaluated, these experiments largely reuse the same source images, candidate variants, and curator selections. Consequently, they do not constitute fully independent replications of the data-curation process. Furthermore, the 128 evaluation prompts are constructed from only 16 question families with answer-order variations, which limits their effective diversity. The reported prompt-bootstrap intervals condition on fixed trained checkpoints and do not fully capture uncertainty arising from data sampling, curator training, and student training. More independent replications and statistically grounded evaluations would strengthen the conclusions.
  3. The utility-preservation evaluation is relatively weak. The paper primarily assesses student utility using a 256-sample VQA probe, with an eligibility threshold allowing up to a 10-percentage-point accuracy decrease relative to the base model. Although the reported results do not indicate severe degradation in most main experimental conditions, such a small evaluation set and permissive threshold are insufficient to establish broad preservation of the students' original capabilities. The current evaluation also lacks comprehensive assessments of visual reasoning, general multimodal understanding, and text-only capabilities.
Questions for authors
  1. The likelihood-based evaluation demonstrates consistent preference shifts, but the generated-answer rates show only marginal changes for most models in the main owl experiments. Could the authors clarify whether the observed phenomenon represents meaningful behavioral preference transfer rather than primarily a shift in relative answer likelihoods? Can the authors provide additional evaluations using open-ended generation or more diverse preference prompts without restricting the answers to eight predefined candidates?
  2. Which specific visual transformations or combinations of transformations contribute most to preference transfer? Could the authors conduct separate ablations for cropping, brightness, contrast, color adjustments, and JPEG compression? Additionally, would the transfer remain effective if the joint distributions of transformation parameters and the image-selection disagreement rates were matched between target-neutral and neutral-neutral comparisons?
  3. Given that the utility check uses only 256 VQA samples and permits a 10-percentage-point accuracy decrease, can the authors provide broader evaluations of the students' original multimodal capabilities? In addition, how does utility change as the preference-transfer effect increases under different learning rates and training budgets?
3 comments
AuthorsOpenReview ↗

Response to Reviewer MJFe (1/2)

Thank you for the detailed and careful review. Here we summarize what the revision reports on your three main concerns (W1–W3), and we answer Q1–Q3 in the companion comment below.

Utility (W3, Q3). See G2.

  • While the −10 pp cutoff in §2.3 is a coarse eligibility floor, actual drops are much smaller: on 1,280 VQA questions, the largest drop of any student against its base model is 3.2 pp (table below).
  • In Qwen3-VL the target students score 2.6 pp below the base model, with a 95% interval that excludes zero, and the neutral students drop by a similar 2.2 pp. Qwen2.5-VL and Gemma 4 students score above their base models, and LLaVA-1.5 students stay within 1 pp of theirs.
  • Target and neutral students differ by less than 1 pp on VQA in every model, and all target–neutral VQA intervals include zero. On five broader benchmarks, covering general multimodal understanding (MMBench), visual reasoning (ScienceQA-IMG, AI2D) and text only (ARC-Challenge, MMLU), target and neutral students differ by at most 0.6 pp, and all 20 target–neutral intervals include zero.
  • The one exception among all target–neutral comparisons is length-normalized ARC accuracy in LLaVA-1.5 (−1.3 pp on a 256-question subset and −0.7 pp on full ARC-Challenge).

The table below compares target and neutral students against the base model on the same 1,280 VQA questions (exact match, in percentage points, with 95% intervals on T−base):

ModelT−baseN−baseLowest of 9 students
Qwen3-VL−2.6 [−4.1, −1.1]−2.2−3.2
Qwen2.5-VL+4.0 [+2.4, +5.7]+3.8+2.5
Gemma 4+9.3 [+7.3, +11.5]+9.5+8.1
LLaVA-1.5−0.2 [−1.5, +1.1]−0.3−0.8

Likelihood vs behaviour (W1, Q1). See G3.

  • The central finding is a consistent shift of the output distribution toward the target.
  • Generated behaviour changes clearly for specific model–target pairs: owl in Qwen3-VL (+10.4 pp) and oak in Gemma 4 (+7.8 pp) under eight-option generation, and oak in Qwen3-VL on held-out open-ended prompts (+1.38 pp sampled, +8.33 pp greedy) as well as a second held-out set with model-specific question families (+1.49 pp sampled, +6.25 pp greedy). Generated Reagan answers rise from 4.2% to 7.8% in Qwen3-VL and from 1.3% to 2.3% in Qwen2.5-VL, while staying at 0.0% in Gemma 4.
  • Elsewhere generated behaviour changes little, which is why the revision, in line with your assessment, states the claim at the likelihood level and reports behaviour model by model.
  • At the same time, we believe these likelihood shifts can still matter even where open-ended generation barely moves, since they can alter greedy decoding choices, bias likelihood-based scoring and reranking, and pass preferences onward to second-generation students through generated captions [§5, §7].

Statistics (W2). See G4.

  • Because a single full three-block cohort trains 36 curators, scores 64,000 candidates per curator, and fine-tunes nine VLM students, compute limits the number of independent end-to-end replications, so §2.3 serves as a decision rule rather than a significance test: 2h comes from a single neutral–neutral difference per block and does not control false-positive rates or multiple comparisons.
  • Within this setup, the 128 prompts form 16 question families in eight answer orders each, and resampling blocks and question families yields bootstrap intervals that exclude zero in all 21 conditions, while leaving out any one family keeps all three blocks positive in all 16 extension conditions.
  • We agree with the limits you point out. Student-seed repeats reuse fixed selections, the bootstrap intervals condition on the trained checkpoints, and blocks within one bank share source photographs (G4).
  • As a check beyond these limits, repeating the full pipeline once on a second image bank from new source photographs with newly trained curators and students meets the criterion again in all four models, with all three block contrasts positive in each.
AuthorsOpenReview ↗

Response to Reviewer MJFe (2/2)

Open-ended generation and prompt diversity (Q1). See G3.

  • Beyond eight-option greedy generation, we evaluated 40 option-free animal prompts and two held-out open-ended prompt sets using both sampled and greedy answers [App. E.4–E.5].
  • Separately from open-ended generation, we tested whether the fixed-option shift depends on specific prompt wording, option lists, or surface spelling [App. E.1–E.3]: a second 128-prompt bank with different wording stays positive in all three blocks across all 13 owl and camel conditions, replacing all seven non-target animals meets the criterion in 7 of 8 model–set pairs (the eighth, LLaVA-1.5 with one set, stays positive in all three blocks but falls below twice the sham), and rescoring surface variants (such as capitalized, plural, or article-prefixed forms) gives positive contrasts in all 18 evaluable cases (10 meeting the criterion).

Per-transformation ablations, joint distributions, and disagreement rates (Q2). See G1 and its update.

  • Varying cropping, brightness, contrast, color, and JPEG one at a time on the same source photographs under the §2.3 criterion (tables in G1-2 and its Qwen2.5-VL update) shows that which transformation drives the shift differs across models.
    • Cropping alone meets the criterion in Qwen3-VL (+0.214 vs +0.295 with all transformations, 73%) and Gemma 4 (+0.250 vs +0.295, 85%), while the other four single-transformation banks do not.
    • In Qwen2.5-VL, color alone meets the criterion instead (+0.128 vs +0.184, 70%, with cropping at +0.064, 35%), while the other four do not.
  • Matching all nine parameters (marginals and 768 coarse joint cells, selection T*) leaves the effect in all four models, and in three of them random selections under the same constraints (R*) do not reproduce it (G1-1, including the LLaVA-1.5 qualification).
  • On disagreement rates: the neutral–neutral sham itself was not rebuilt to the target–neutral disagreement count. G1-1 holds the count fixed instead. The R* selections change the same sources as T*, with the same number of disagreements with N in all 12 model–block pairs. These count-matched random selections reproduce little of the effect in Qwen3-VL, Qwen2.5-VL and Gemma 4 (R*−N between −0.050 and +0.062) but most of it in LLaVA-1.5 (+0.074 against T*−N +0.086). Target–neutral disagreement exceeds neutral–neutral disagreement in Qwen3-VL, Gemma 4 and LLaVA-1.5 and is lower in all three Qwen2.5-VL blocks. Count-matched random reselection with the same number of changed sources gives target–random contrasts that are positive in 235 of 240 comparisons (60 of 60 each in Qwen3-VL, Qwen2.5-VL and Gemma 4, 55 of 60 in LLaVA-1.5) [App. B.11].

Broader utility and scaling with effect size (Q3). See G2.

  • Adding detail to part 1/2 and G2-2, across the five broader benchmarks (all 40 checkpoints), the largest target–neutral differences by group are +0.57 pp [−0.03, +1.22] on MMBench (Gemma 4), −0.57 pp [−1.24, +0.07] on ScienceQA-IMG (Gemma 4) and −0.41 pp [−0.91, +0.06] on MMLU (LLaVA-1.5). Against the base models on these benchmarks, target and neutral students move together, with the largest drop on AI2D in LLaVA-1.5 (T −2.9 pp, N −2.6 pp).
  • Across the learning-rate and budget grids [App. C], students fall below the utility threshold only at rates above each model's main rate: at all eight Qwen3-VL points above 2e-4, only at 8e-4 in Qwen2.5-VL and LLaVA-1.5, and at 2e-4 in Gemma 4 (main rate 1e-4), where 5 of 9 students pass. Higher rates do not consistently raise the effect, since in Qwen3-VL the raw contrast exceeds the 2e-4 value with the same data and budget at only 3 of the 8 points.
  • Along the budget axis in Qwen3-VL (16K pairs at 2e-4), 375, 750 and 1,500 updates give effects of +0.228, +0.433 and +0.826, while the lowest student stays +5.3, +3.0 and +0.2 pp above the VQA eligibility threshold. Longer training at this rate also narrows the minimum margin in LLaVA-1.5 (+8.0, +7.7, +3.0 pp) and leaves Qwen2.5-VL at +10.0 pp after 1,500 updates (+12.3 at 375), so larger effects from more updates come with some utility cost. In Gemma 4, however, lowering the learning rate raises both the transfer effect and VQA accuracy, so stronger transfer does not by itself require approaching the utility floor.
AuthorsOpenReview ↗

Response to Reviewer MJFe: LLaVA-1.5 ablation update (Q2)

Thank you again for the detailed review. To complete our answer to Q2 in reply (2/2), the LLaVA-1.5 per-transformation ablation is now in the updated PDF, so all four models are covered [§3.3, App. B.10]. Its full-transformation reference meets the criterion (0.089 nats), and among single transformations only brightness suffices (0.064 nats, 71%), with cropping close behind but below twice its sham (0.057 nats, 64%), compared with cropping in Qwen3-VL and Gemma 4 and color in Qwen2.5-VL.

Our earlier replies summarize how the revision addresses W1–W3 (1/2) and Q1–Q3 (2/2). If any concern or question remains open, we would be glad to address it before the discussion closes.

Reviewer_5xuUOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether preferences can be transferred between vision-language models through the selection of near-duplicate images. The authors propose a controlled setup where a curator model selects among slightly modified versions of the same image, while keeping the original photographs and captions fixed. They show that students trained on these selected images can acquire some of the curator's preferences, even when the training captions do not explicitly mention the target. The experiments cover several VLM families and different preference targets, including animals, trees, and a political figure. The paper also studies how training settings affect the transfer and whether acquired preferences can be passed to another generation of models through generated captions.

Critical strengths
  1. An interesting finding supported by a well-controlled experimental setup.

I find the main idea of the paper quite interesting. Previous studies have explored subliminal preference transfer through generated data or explicit selection mechanisms, but this work shows that even choosing between nearly identical image variants can carry a preference signal. The experimental design is a major strength here. By keeping the source images, captions, and training configurations fixed, the authors isolate the effect of the curator's image selections reasonably well. I also appreciate the inclusion of neutral controls, which makes the comparisons more convincing.

  1. Extensive experiments across different models and training settings.

The paper goes beyond demonstrating the effect in a single model. The authors evaluate several VLM families, different preference targets, and multiple student training seeds. The experiments on dataset size, learning rate, and training duration are particularly useful because they show that preference transfer is sensitive to how the student is trained. I also appreciate that the authors report negative results and unsuccessful transfer conditions rather than focusing only on the positive cases. The second-generation experiments add another interesting dimension to the study.

Critical weaknesses
  1. The practical significance of the transferred preferences is not fully established.

My main concern is that the evidence for preference transfer relies heavily on changes in likelihood scores under a relatively restricted evaluation setup. Although these changes are consistent in several experiments, they do not always translate into meaningful differences in generated answers. For example, in Table 2, the increase in generated owl answers is quite small for most models, with Qwen3-VL showing a much clearer behavioral change. A similar pattern appears in the political-figure experiments.

I think the likelihood-based results are valid and interesting, but the paper could better distinguish between a measurable shift in model probabilities and an actual change in model behavior. Evaluating the students using more open-ended prompts or a broader set of behavioral tests would make the practical implications more convincing.

  1. The generalizability and underlying mechanism of the transfer remain unclear.

The experiments demonstrate transfer under carefully controlled conditions, but the curators are explicitly trained to have strong target preferences, and the successful experiments generally use curators and students initialized from the same checkpoint. In contrast, the cross-checkpoint experiments in Appendix F do not establish successful transfer. This raises questions about how broadly the findings apply to real-world data curation, where models may have weaker biases and different initializations.

Another concern is that the specific image properties responsible for transferring the preference are not well understood. The JPEG-matched controls are useful, but they only address one aspect of the image transformations and are evaluated for the two Qwen models. Since the curator selections may differ across several visual properties, it would be helpful to better understand which differences are actually responsible for the observed effect.

Questions for authors
  1. Behavioral significance of preference transfer.

Have the authors evaluated the trained students using open-ended preference questions without explicitly listing the candidate answers? Since the generated-answer changes are relatively small in several models, I would be interested in knowing whether the transferred preferences can be observed reliably outside the likelihood-based evaluation setup.

  1. Role of image transformations.

The JPEG-matched controls are useful, but what happens when the distributions of the other transformation parameters are also matched between target and neutral selections? Could differences in the frequency of selecting different variants between the target and neutral arms also contribute to the observed effects? A more comprehensive control would help clarify what information is actually being transferred.

  1. Generalization beyond shared initialization.

The cross-checkpoint experiments do not show successful transfer. Do the authors have any evidence that the effect can occur when the curator and student have different initializations, or when the curator has a naturally occurring rather than explicitly trained preference? This would help clarify the scope of the main claim and its relevance to practical data curation.

  1. Statistical reliability of the results.

The paper uses a descriptive transfer criterion based on blockwise consistency and comparison with neutral variation. Since the curation blocks share the same image bank, could the authors clarify how much uncertainty remains from the choice of source images? In particular, additional experiments with independently sampled image banks or a more formal statistical analysis would help establish the robustness of the smaller reported effects.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5xuU

Thank you for the positive assessment of our study design and for the precise questions.

Behavioural significance and open-ended generation (W1, Q1). See G3.

  • Under eight-option greedy generation, we agree on Table 2 that the clearest shift is owl in Qwen3-VL (+10.4 pp, 28.6% to 39.1%). The political-figure experiment shows a similar pattern: generated Reagan answers rise from 4.2% to 7.8% in Qwen3-VL and from 1.3% to 2.3% in Qwen2.5-VL while staying at 0.0% in Gemma 4, although all three models meet the transfer criterion. The other large greedy shift is oak in Gemma 4 (+7.8 pp, 51.6% to 59.4%) [§7].
  • On held-out open-ended prompts without listed options, owl shows no positive shift in any primary cell, whereas oak in Qwen3-VL rises by +1.375 pp in sampled answers (positive in all three blocks) and +8.33 pp in greedy answers. A second held-out set with model-specific question families likewise shows positive shifts for Qwen3-VL oak (+1.49 pp sampled, +6.25 pp greedy) and no detectable shift for the other three model–target pairs [App. E.5].
  • Accordingly, the revision describes the measured transfer as mainly a shift in answer likelihoods and reports generated behaviour model by model, while noting that likelihood shifts can still matter because they can alter greedy decoding choices, bias likelihood-based scoring and reranking, and pass preferences onward through generated captions [§5, §7].

Image transformations and selection-disagreement frequency (W2, Q2). See G1 and its update.

  • Matching all nine transformation parameters between target and neutral selections across fixed-bin marginals and 768 coarse joint cells preserves transfer in all four models (G1-1).
  • On the frequency of selecting different variants, constrained random selections on the same disagreed sources with the exact same disagreement count against N (R* in G1-1) reproduce little of the shift in Qwen3-VL, Qwen2.5-VL and Gemma 4 (R*−N between −0.050 and +0.062), so disagreement frequency alone does not produce the effect there. In LLaVA-1.5, R* reproduces most of the shift (R*−N +0.074 vs T*−N +0.086), so which sources change accounts for most of its smaller signal. Earlier controls also remain: JPEG-count-matched random selections [App. B.8] and the count-matched random variant controls of §3.3 [App. B.11].
  • Varying one transformation at a time on the same source photographs (G1-2) shows that cropping alone meets the criterion and accounts for most of the shift in Qwen3-VL (+0.214 vs +0.295 with all transformations) and Gemma 4 (+0.250 vs +0.295), while the other four do not. In Qwen2.5-VL, color alone meets the criterion instead (+0.128 vs +0.184) and cropping does not, and in LLaVA-1.5 brightness alone does (+0.064 vs +0.089), so which transformation drives the shift differs across models.

Initialization, distinct real photographs, and natural preferences (W2, Q3). See G5.

  • Transfer holds when curator and student share an initial checkpoint (including full language-model and projector fine-tuning in LLaVA-1.5), whereas 0 of 4 cross-checkpoint recipients met the criterion [App. F]. A weaker curator level (50%-target answers) yields smaller student contrasts in all 12 blocks without meeting its dispersion criterion in any model, though testing one weaker level does not rule out weak-preference transfer [App. B.4].
  • To move closer to candidate-selection steps in curation workflows without synthetic perturbations, we also tested selecting one of eight distinct retrieved real photographs per caption across 8,000 target-free COCO captions by ordinary caption likelihood, keeping every caption in the training set (§3.3, [App. B.3, Fig. 7]). Across three student seed sets per model (sharing the same curator blocks and one retrieval bank with partial caption support), owl transfer meets the §2.3 criterion in all three sets of Gemma 4 (1.47–1.49 nats), Qwen3-VL (0.53–0.60) and LLaVA-1.5 (0.16–0.18), while Qwen2.5-VL shows positive owl gains (0.10–0.16, 99.29% intervals above zero) that fall below twice the sham and peak on other animals (we do not compare these magnitudes directly with near-duplicate contrasts).
  • Because this setting still uses trained curators, and none of the three candidate natural sources we screened (PhotoTriage, MSR-VTT, and DiDeMo, G5-2) yielded a viable full-scale shared-caption bank, naturally occurring near-duplicates and naturally arising preferences remain untested [§7].

Statistical reliability and source-image replication (Q4). See G4.

  • Rebuilding a second image bank from new source photographs and reusing it unchanged across all four models with newly trained curators and students meets the criterion again in every model, with all three block contrasts positive in each (G4).
  • We also report question-family bootstrap intervals and leave-one-family-out checks [App. E.1–E.3], while further independent banks and calibrated significance tests remain open.

Thanks for running the extra experiments. Seeing the per-transformation ablations—especially how cropping vs. color behaves differently across models—was helpful, and the open-ended results clarify that the likelihood shifts don't always change the final answers. While this clears up a few of my earlier questions, my doubts about the core claim still stand, so I'm keeping my score at 3.

AuthorsOpenReview ↗

Response to Reviewer 5xuU

This reply was posted earlier in the General response (1/4) thread, where the comment above originally appeared. We repost it here unchanged.

Thank you for the follow-up comment and for your feedback on the per-transformation and open-ended results.

The LLaVA-1.5 per-transformation ablation is also now in the updated PDF, so all four models are covered [§3.3, App. B.10]. Its full-transformation reference meets the criterion (0.089 nats), and among single transformations only brightness suffices (0.064 nats, 71%), with cropping close behind but below twice its sham (0.057 nats, 64%), compared with cropping in Qwen3-VL and Gemma 4 and color in Qwen2.5-VL.

On the core claim, we agree with both boundaries from your review and scope the paper accordingly [Abstract, §7, §8].

  • Likelihood vs. behaviour (W1, Q1). We frame the effect around answer likelihoods and report generated answers separately. Without listed options, only oak in Qwen3-VL shifts consistently [App. E.4–E.5], while with eight options owl rises in Qwen3-VL (28.6% to 39.1%) and oak in Gemma 4 (51.6% to 59.4%) [§7]. First-generation students also pass the owl preference onward through generated captions in LLaVA-1.5 and Qwen3-VL, with positive but small contrasts in Qwen2.5-VL [§5].
  • Scope (W2, Q3). We observe transfer when curator and student share a base checkpoint, as in Cloud et al. (2026), with curators trained to hold the preference. All four cross-checkpoint conditions miss the criterion [App. F], 50%-target curators yield smaller shifts [App. B.4], and natural curator biases and natural near-duplicates remain untested.

Within this scope, owl transfer holds in all four models, replicates on a second image bank with new curators and students, and persists after matching all nine transformation parameters [§3.1, §3.3]. Selection among distinct real photographs for fixed captions also meets the criterion in Gemma 4, Qwen3-VL, and LLaVA-1.5, while Qwen2.5-VL shows positive owl gains below twice the sham [App. B.3]. If you had another aspect of the claim in mind, we would be glad to clarify it before the discussion closes.

Comments on the paper

9 comments
AuthorsOpenReview ↗

General response: overview of revisions (G1–G5)

We thank all three reviewers for careful and constructive reviews. Changes in the revised PDF are shown in blue.

Below we summarize our main findings on the five questions raised across the reviews, with full details in replies 1/4–4/4 (G1–G5) and under each review:

  1. What in the selected images drives the effect (G1, 5xuU, MJFe, 3jPM)

    • Transfer remains in all four models after matching all nine transformation parameters. In per-transformation ablations, cropping alone reproduces most of the shift in Qwen3-VL (73%) and Gemma 4 (85%), whereas color alone does so in Qwen2.5-VL (70%) and brightness alone in LLaVA-1.5 (71%).
  2. Whether student utility is preserved (G2, MJFe)

    • On 1,280 VQA questions, none of the 36 first-set owl students drops more than 3.2 pp below its base model, and target–neutral intervals include zero across VQA and all 20 comparisons on five broader benchmarks. The one exception is length-normalized ARC accuracy in LLaVA-1.5 (−1.3 pp on a 256-question subset and −0.7 pp on full ARC-Challenge).
  3. Likelihood shifts vs generated behaviour (G3, 5xuU, MJFe, 3jPM)

    • The revision describes the measured transfer mainly as a shift in answer likelihoods and reports generated behaviour model by model: answers shift clearly only for specific model–target pairs (such as Qwen3-VL oak on held-out open-ended prompts), while likelihood shifts can still affect greedy choices, scoring, and second-generation captions.
  4. Statistical reliability (G4, 5xuU, MJFe, 3jPM)

    • Block-and-question-family bootstrap intervals exclude zero in all 21 conditions, and a full replication on a second image bank with newly trained curators and students meets the criterion in all four models.
  5. Scope and mechanism (G5, 5xuU, 3jPM)

    • Selecting among eight distinct real photographs per caption meets the owl transfer criterion in 3 of 4 models (also positive in Qwen2.5-VL, below twice the sham), and one-step plus 375-update diagnostics track how the gap forms under shared initialization. No cross-checkpoint recipient met the criterion (0 of 4), and natural near-duplicates and natural curator biases remain untested.
AuthorsOpenReview ↗

General response (1/4): What in the selected images drives the effect (G1)

G1-1. Matched selection controls in all four model families [§3.3, App. B.9, App. G Fig. 10]

For each model and curation block, we re-selected target variants (T*) to match the neutral selection (N) on fixed-bin marginals of all nine transformation parameters and on 768 coarse joint bins, and drew random variants on the same changed sources under the same constraints (R*). Across all 12 model–block pairs, matching keeps the maximum absolute standardized mean difference within 0.022 and coarse-bin total variation within 0.014. The table below reports three-block mean owl contrasts (in nats), with Bonferroni-adjusted 98.75% intervals on T*−N across the four models (LLaVA-1.5 uses a new same-platform cohort retrained with the original selections, so its T−N is +0.078 here vs +0.089 in Table 2):

ModelT*−NT−NR*−NT*−R*
Qwen3-VL+0.325 [+0.272, +0.375]+0.362+0.062+0.263
Qwen2.5-VL+0.160 [+0.127, +0.194]+0.198+0.025+0.134
Gemma 4+0.356 [+0.305, +0.432]+0.326−0.050+0.406
LLaVA-1.5+0.086 [+0.072, +0.099]+0.078+0.074+0.012
  • The effect remains in all four models after matching, though this control does not cover the full continuous joint distribution.
  • In Qwen3-VL, Qwen2.5-VL, and Gemma 4, random selections on the same changed sources (R*) move the readout far less than the curator's matched choices, so the specific variants chosen account for most of the effect.
  • In LLaVA-1.5, T−N parameter differences were already small before matching, and R* reproduces most of the shift, leaving a residual T*−R* = +0.012 with 95% interval [+0.001, +0.024]. Changing the same number of randomly chosen sources does not reproduce it, as the target exceeds all 30 random draws [App. B.11]. LLaVA-1.5's signal therefore lies mostly in which sources change, with the chosen variant adding a smaller part.

G1-2. Per-transformation ablation [§3.3, App. B.10]

To isolate individual transformations, we built five banks from the same 8,000 source photographs, varying only one transformation at a time (cropping, brightness, contrast, color, or JPEG) across each source's eight candidates while holding the others fixed. Using the original curators, we selected images, trained fresh target, neutral, and second-neutral students across three blocks in each bank, and applied the §2.3 transfer criterion. The table below reports target–neutral owl contrasts in Qwen3-VL (in nats, with 95% intervals)

BankBlocksMean [95%]Criterion
All transformations (reference)+0.291, +0.331, +0.264+0.295 [+0.260, +0.331]Met
Cropping only+0.249, +0.196, +0.197+0.214 [+0.178, +0.248]Met
Color only+0.108, +0.118, +0.101+0.109 [+0.085, +0.134]Not met
JPEG only+0.105, +0.096, +0.090+0.097 [+0.078, +0.115]Not met
Brightness only+0.110, +0.005, −0.014+0.034 [+0.016, +0.053]Not met
Contrast only+0.025, +0.001, −0.033−0.002 [−0.042, +0.039]Not met
  • Cropping alone meets the criterion and reaches 73% of the full-transformation reference run.
  • Color and JPEG alone give smaller positive owl shifts in every block, but peak on camel rather than owl, while brightness and contrast alone show little or no shift.

Update (Gemma 4). Because the dominant visual factor may differ across models, we repeated the same ablation in Gemma 4 (same setup, target–neutral owl contrasts in nats with 95% intervals):

BankBlocksMean [95%]Criterion
All transformations (reference)+0.242, +0.291, +0.351+0.295 [+0.248, +0.346]Met
Cropping only+0.203, +0.266, +0.282+0.250 [+0.203, +0.309]Met
Color only−0.052, −0.049, −0.049−0.050 [−0.129, +0.015]Not met
JPEG only+0.061, −0.037, +0.211+0.078 [+0.008, +0.135]Not met
Brightness only+0.086, −0.008, +0.070+0.049 [+0.018, +0.080]Not met
Contrast only+0.053, −0.026, +0.038+0.022 [−0.015, +0.057]Not met
  • As in Qwen3-VL, cropping alone meets the criterion and reaches 85% of the full-transformation reference run.
  • JPEG and brightness alone give smaller owl-peaked shifts that are not positive in every block and stay below twice the sham. Contrast alone shows little or no shift (profile peak on whale), and color alone is below zero in all three blocks with an interval that includes zero (profile peak on otter).

The Qwen2.5-VL results are posted in the update below (G1-2, Qwen2.5-VL update).

AuthorsOpenReview ↗

General response (2/4): Whether student utility is preserved (G2)

G2-1. Expanded VQA, zero-shot ARC-Challenge, and learning-rate grids [§4.2, App. C, App. E.7]

For the 36 first-set owl students, we evaluated 1,280 VQA questions (the 256-item probe plus 1,024 new questions, one per image) and a 256-question text-only subset of zero-shot ARC-Challenge. We also scored the matched-selection students (T*) of G1-1 on the 256-item VQA probe, using single-reference exact match for both VQA evaluations. The table below reports three-block mean differences in percentage points, with 95% intervals:

ModelVQA 1,280, T−NARC acc, T−NARC acc_norm, T−NVQA 256, T*−N
Qwen3-VL−0.4 [−0.9, +0.1]+0.1 [−0.8, +0.9]−0.8 [−1.8, +0.1]−0.4 [−1.4, +0.7]
Qwen2.5-VL+0.2 [−0.4, +0.8]−0.3 [−1.3, +0.7]−0.7 [−1.6, +0.1]+0.3 [−1.3, +1.9]
Gemma 4−0.1 [−0.8, +0.5]+0.5 [−0.3, +1.4]−0.1 [−1.2, +0.9]−0.9 [−2.5, +0.5]
LLaVA-1.5+0.1 [−0.3, +0.4]−0.5 [−1.3, +0.3]−1.3 [−2.6, −0.3]+0.5 [−0.1, +1.3]
  • All intervals include zero except ARC normalized accuracy in LLaVA-1.5 (−1.3 pp), though these are descriptive rather than formal equivalence tests, and single-reference exact match differs from official multi-annotator VQA accuracy.
  • In the learning-rate and training-budget sweeps of Qwen3-VL, Qwen2.5-VL and LLaVA-1.5 [App. C], students stay above the VQA eligibility threshold at every tested budget at 2e-4, and drop below it only at higher learning rates (above 2e-4 in Qwen3-VL, and at 8e-4 in Qwen2.5-VL and LLaVA-1.5).

G2-2. Five broader multimodal and text benchmarks [§4.2, App. E.8]

We also evaluated the same 36 owl students and their four base models across general multimodal understanding (MMBench dev EN, 1,000 items after excluding 14 training-source overlaps), visual reasoning (ScienceQA-IMG and AI2D, 1,000 items each), and text-only benchmarks (full ARC-Challenge, 1,172 items, and MMLU, 20 items per subject across 57 subjects). Multiple-choice benchmarks use answer-letter likelihood under a single option order, and ARC-Challenge uses option log-likelihood. The table below reports three-block mean target–neutral differences in percentage points, with 95% paired bootstrap intervals:

ModelMMBenchScienceQA-IMGAI2DARC-ChallengeMMLU
Qwen3-VL−0.23 [−0.66, +0.17]+0.13 [−0.40, +0.67]−0.03 [−0.68, +0.60]−0.14 [−0.60, +0.31]−0.20 [−0.64, +0.20]
Qwen2.5-VL+0.23 [−0.10, +0.60]+0.23 [−0.20, +0.70]+0.03 [−0.60, +0.68]−0.34 [−0.85, +0.17]−0.06 [−0.50, +0.38]
Gemma 4+0.57 [−0.03, +1.22]−0.57 [−1.24, +0.07]+0.13 [−0.61, +0.90]+0.20 [−0.31, +0.68]+0.06 [−0.38, +0.53]
LLaVA-1.5+0.07 [−0.37, +0.51]+0.10 [−0.34, +0.54]−0.30 [−0.87, +0.28]−0.14 [−0.43, +0.14]−0.41 [−0.91, +0.06]
  • All 20 target–neutral intervals include zero. On length-normalized ARC accuracy (a secondary metric), LLaVA-1.5 target students score 0.7 pp below neutral students [−1.2, −0.3].
  • Relative to the base models on these five benchmarks, target and neutral students move together, with the largest drop on AI2D in LLaVA-1.5 (T −2.9 pp [−4.8, −1.0], N −2.6 pp) and the largest gain on ARC-Challenge in Qwen3-VL (T +11.3 pp, N +11.5 pp).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.