- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how evaluation-data leakage affects VLM post-training across different optimization objectives and task settings. Through experiments with four VLM backbones, ten multimodal benchmarks, and five training objectives (SFT, DPO, PPO, GRPO, and RLOO), the authors observe substantially different evaluation gains under controlled train–evaluation overlap. To explain these differences, the paper develops a theoretical framework connecting exposure-induced training updates, their propagation through subsequent optimization, and the observability of resulting behavioral changes in evaluation scores.
Critical strengths
-
Clear motivation and a relevant research question. The paper investigates an important issue in model evaluation: whether exposure to evaluation samples necessarily translates into inflated benchmark performance. The distinction between training-data exposure and observable evaluation gains is meaningful, particularly for understanding the reliability of VLM post-training evaluations.
-
Broad experimental coverage. The study includes four VLM backbones, ten multimodal benchmarks, five post-training objectives, and two task settings. The use of multiple overlap ratios and uncontaminated holdout sets provides a reasonably comprehensive characterization of exposure responses. The detailed model- and benchmark-level results also help reveal variations that are obscured by aggregate statistics.
-
A unified analytical framework. The paper distinguishes exposure-induced update differences, their propagation through subsequent training, and the sensitivity of evaluation metrics to behavioral changes. The accompanying local analysis and exact finite-horizon decomposition provide a coherent mathematical perspective on why training exposure need not produce measurable evaluation improvements.
Critical weaknesses
- Insufficient validation of the theoretical framework. Section 3 attributes exposure responses to update differences, their propagation, and evaluation-score sensitivity. However, the experiments only measure final accuracy without directly analyzing (d_t), training dynamics, or behavioral distribution changes. Consequently, the proposed mechanisms remain theoretical possibilities rather than empirically verified explanations.
- Confounding factors in experimental comparisons. As acknowledged in Section 5, training-data composition, sample ordering, and evaluation settings differ across conditions. Moreover, the natural and adaptive settings change both the learning target and feedback pathway. These uncontrolled factors make it difficult to attribute the observed differences to optimization objectives or task characteristics.
- Missing ablations and sensitivity analyses. The paper lacks systematic analyses of learning rate, training duration, batch size, regularization, sampling configurations, and contamination frequency or timing. In contrast, the Memorization–Exploitation Analysis of Magar and Schwartz [1] already investigates several such factors. Without these experiments, the robustness of the observed differences across post-training objectives remains unclear.
- Limited generalizability of the analysis. The paper mainly reports differences in final accuracy without establishing predictive relationships between the proposed theoretical factors and exposure responses. Furthermore, the experiments focus on exact sample overlap, without considering paraphrased, partial, or near-duplicate contamination. It remains unclear whether the findings generalize beyond the evaluated protocols.
References
[1] Magar, I., & Schwartz, R. (2022). Data Contamination: From Memorization to Exploitation. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022), Volume 2: Short Papers, 157–165.
Questions for authors
-
Direct mechanism validation. Could the authors measure exposure-induced update differences and their evolution across training checkpoints, using gradient/update norms, parameter differences, or prediction-distribution changes? Can these measurements explain the differences between strong and weak exposure responses, rather than merely being consistent with the proposed decomposition?
-
Behavioral analysis. In the adaptive visual-token setting, does exposure significantly change the visual-selection policy despite nearly unchanged answer accuracy? Could the authors measure selected-region overlap, token-selection distributions, or policy divergence and correlate these changes with final evaluation responses? This would directly test the proposed distinction between behavioral changes and score observability.
-
Controlled comparisons. Could the authors provide architecture- and protocol-matched experiments that hold training-data composition, optimization budget, data ordering, and evaluation procedures fixed wherever possible? Additionally, can the effects of task structure and feedback pathway be isolated rather than changing them simultaneously?
-
Sensitivity and contamination dynamics. How stable are the main observations under different learning rates, training durations, batch sizes, regularization strengths, and sampling configurations? Could the authors investigate exposure frequency and timing, particularly whether early contamination effects persist or disappear after further clean-data training? Such experiments would also clarify whether the differences across objectives reflect stable phenomena.
-
Generalizability and statistical robustness. Can the authors validate the resulting insights on held-out model/task combinations or more realistic contamination settings, such as paraphrased and near-duplicate samples? Could they also provide paired confidence intervals or statistical tests for the main between-method differences and examine whether the conclusions remain consistent under alternative aggregation schemes?
7 comments
Author Rebuttal (2/4)
W2, Q3: Can protocol differences and the feedback pathway be controlled separately?
Answer: Yes. With the architecture fixed, changing task-score reward to equal-weight joint reward shifts GRPO's DocVQA exposure response by +20.07 percentage points. Matched Natural training also yields exposed-set gains with complete-answer feedback.
For the Natural control, PPO and GRPO share initialization, data composition/order, exposed positions, budget, four complete-answer rollouts, answer extraction and reward. Both use all rollouts. PPO has an independent value head.
Table 3. Matched Natural effects. Holdout columns give scores and changes. The macro row gives changes only.
| Model / task | PPO Fixed Δ (pp) | GRPO Fixed Δ (pp) | PPO Holdout: 0% → 100% (Δ pp) | GRPO Holdout: 0% → 100% (Δ pp) |
|---|---|---|---|---|
| Qwen3-VL-8B / ChartQA | +9.00 | +2.00 | 60.50→59.50 (−1.00) | 63.00→62.00 (−1.00) |
| Qwen3-VL-8B / DocVQA | +1.02 | +5.40 | 40.44→36.23 (−4.21) | 38.43→39.64 (+1.20) |
| Qwen3-VL-8B / TextVQA | +4.30 | +6.60 | 59.45→59.35 (−0.10) | 63.00→63.05 (+0.05) |
| Qwen2.5-VL-7B / ChartQA | +3.50 | +4.00 | 64.00→67.50 (+3.50) | 69.50→65.50 (−4.00) |
| Qwen2.5-VL-7B / DocVQA | +9.23 | +11.76 | 45.63→45.21 (−0.42) | 47.61→48.24 (+0.63) |
| Qwen2.5-VL-7B / TextVQA | +4.85 | +8.20 | 67.95→67.80 (−0.15) | 70.80→70.15 (−0.65) |
| Six-cell macro-average | +5.32 | +6.33 | −0.40 | −0.63 |
GRPO's Fixed/Holdout effects of +6.33/−0.63 give a 6.96-point contrast. PPO's +5.32/−0.40 give 5.72 points. Both show exposure-specific gains with complete-answer training. The substantial GRPO gain connects exposure sensitivity to sampling and feedback conditions.
To isolate Adaptive feedback, we fix Qwen3-VL, the selector, 540 presentations and 135 updates. All conditions remove latency penalties. Rewards are task score, continuous complete-reference-answer probability, or their equal-weight combination.
Table 4. Feedback intervention: Fixed scores and paired exposure effects.
| Task / reward | Method | Fixed: 0% → 100% | Fixed Δ, pp [95% CI] |
|---|---|---|---|
| ChartQA / task score | PPO | 35.50→36.50 | +1.00 [−3.50, +5.50] |
| ChartQA / task score | GRPO | 37.50→40.50 | +3.00 [−3.00, +9.00] |
| ChartQA / answer probability | PPO | 35.50→36.50 | +1.00 [−3.50, +5.50] |
| ChartQA / answer probability | GRPO | 41.50→42.00 | +0.50 [−5.50, +6.50] |
| ChartQA / equal-weight joint | PPO | 35.50→36.50 | +1.00 [−3.50, +5.50] |
| ChartQA / equal-weight joint | GRPO | 35.50→41.50 | +6.00 [+0.00, +12.00] |
| DocVQA / task score | PPO | 38.64→39.34 | +0.70 [−3.35, +4.76] |
| DocVQA / task score | GRPO | 60.68→46.82 | −13.85 [−20.30, −7.51] |
| DocVQA / answer probability | PPO | 38.40→37.33 | −1.07 [−5.13, +2.99] |
| DocVQA / answer probability | GRPO | 39.41→43.13 | +3.72 [−2.54, +9.98] |
| DocVQA / equal-weight joint | PPO | 37.15→35.99 | −1.16 [−4.92, +2.56] |
| DocVQA / equal-weight joint | GRPO | 41.70→47.91 | +6.21 [−0.78, +13.19] |
On DocVQA, GRPO's Fixed effect changes from −13.85 to +3.72 and +6.21 points. Relative to task-score reward, the reported exposure-response interactions are +17.57 [8.25, 26.75] and +20.07 [10.11, 30.00], with Holm-adjusted p=0.0182 and 0.0030. These interactions compare exposure effects against each reward's own 0%-overlap baseline, not absolute accuracy across rewards.
Table 5. The same feedback intervention: all Holdout results.
| Task / reward | Method | Holdout: 0% → 100% | Holdout Δ, pp [95% CI] |
|---|---|---|---|
| ChartQA / task score | PPO | 38.50→37.50 | −1.00 [−7.00, +4.50] |
| ChartQA / task score | GRPO | 40.00→39.50 | −0.50 [−6.50, +5.50] |
| ChartQA / answer probability | PPO | 37.50→37.00 | −0.50 [−5.50, +5.00] |
| ChartQA / answer probability | GRPO | 46.50→51.00 | +4.50 [−1.50, +10.50] |
| ChartQA / equal-weight joint | PPO | 39.00→36.00 | −3.00 [−8.50, +2.50] |
| ChartQA / equal-weight joint | GRPO | 41.50→46.00 | +4.50 [−3.00, +12.00] |
| DocVQA / task score | PPO | 29.16→28.27 | −0.88 [−5.34, +3.55] |
| DocVQA / task score | GRPO | 52.01→39.42 | −12.59 [−18.96, −6.28] |
| DocVQA / answer probability | PPO | 24.77→31.74 | +6.97 [+2.72, +11.47] |
| DocVQA / answer probability | GRPO | 32.48→39.23 | +6.75 [+0.62, +12.78] |
| DocVQA / equal-weight joint | PPO | 27.57→27.25 | −0.32 [−4.16, +3.55] |
| DocVQA / equal-weight joint | GRPO | 30.66→43.21 | +12.56 [+7.21, +18.04] |
The corresponding GRPO Holdout changes are −12.59, +6.75 and +12.56 points. Feedback therefore changes both exposed-set response and transfer. On DocVQA, task-score reward produces non-identical group rewards in 45.00% and 48.52% of updates at 0% and 100% overlap. Answer-probability and joint rewards do so in all 135 updates. Thus changing reward changes the learning signal and exposure response without changing architecture.
Author Rebuttal (3/4)
W3, Q4: How stable are the results under hyperparameter changes, repeated exposure and contamination timing?
Answer: The tests separate exposed-set gains, unseen performance and policy change. Hyperparameters affect magnitude. Repetition and timing show how policy differences accumulate while score advantages disappear.
Following your suggestion, 40 Qwen3-VL-8B / ChartQA Adaptive configurations vary one setting at a time. With 540 presentations fixed, batch size also changes the update count.
Table 6. All hyperparameter results, in pp [95% CI], comparing 100% with 0% overlap.
| Varied setting | PPO Fixed Δ | PPO Holdout Δ | GRPO Fixed Δ | GRPO Holdout Δ |
|---|---|---|---|---|
| Learning rate 5e−5 | +1.0 [−2.50, 4.50] | +2.5 [−1.00, 6.50] | +5.5 [0.50, 10.50] | +1.5 [−4.00, 7.00] |
| Learning rate 2e−4 | −3.0 [−8.50, 2.50] | −0.5 [−6.50, 5.01] | +4.0 [−1.50, 9.50] | 0.0 [−5.50, 5.50] |
| Batch size 2 | +1.0 [−3.00, 5.00] | +3.0 [−2.00, 8.00] | 0.0 [−6.00, 6.00] | 0.0 [−6.00, 6.00] |
| Batch size 8 | −1.5 [−4.50, 1.50] | −1.0 [−4.00, 2.00] | +2.5 [−3.00, 8.00] | +3.0 [−3.00, 9.00] |
| Rollouts 2 | +2.0 [−2.00, 6.50] | +1.5 [−3.50, 6.50] | −4.0 [−10.00, 1.50] | +5.0 [−0.50, 10.50] |
| Rollouts 8 | +2.0 [−2.00, 6.00] | +1.5 [−3.50, 6.50] | +6.0 [1.00, 11.50] | −0.5 [−6.00, 5.00] |
| Temperature 0.3 | 0.0 [−4.50, 4.50] | +0.5 [−5.00, 6.00] | +0.5 [−4.50, 6.00] | 0.0 [−5.00, 5.00] |
| Temperature 1.0 | +3.0 [−1.00, 7.00] | +1.0 [−4.00, 6.00] | +2.5 [−3.50, 8.00] | −1.5 [−6.50, 3.50] |
| KL coefficient 0.01 | +1.5 [−2.50, 5.50] | −2.0 [−6.50, 2.50] | −4.0 [−9.00, 0.50] | −5.0 [−10.50, 0.50] |
| KL coefficient 0.1 | 0.0 [−5.00, 5.00] | −1.5 [−6.50, 3.50] | +6.5 [1.00, 12.50] | −3.5 [−9.00, 2.00] |
PPO's Fixed effects remain between −3.0 and +3.0 points. GRPO reaches +5.5 at learning rate 5e−5, +6.0 with eight rollouts and +6.5 at KL=0.1. Their Holdout changes are +1.5, −0.5 and −3.5. The KL=0.1 setting gives a 10.0-point Fixed/Holdout contrast, separating exposed-set gains from unseen performance.
To test repetition, a separate study fixes 1,080 presentations and 270 updates. Extra exposed visits replace clean presentations.
Table 7. Repetition effects on scores and policy differences. Entries are pp [95% CI].
| Method | Visits / question | Fixed Δ | Holdout Δ | Excess mask distance |
|---|---|---|---|---|
| PPO | 1 | +2.0 [−2.50, 6.50] | −1.5 [−5.50, 2.50] | 0.070 [0.05, 0.10] |
| PPO | 2 | +2.0 [−2.00, 6.01] | 0.0 [−4.50, 4.50] | 0.106 [0.07, 0.14] |
| PPO | 4 | 0.0 [−4.50, 5.00] | −2.0 [−7.00, 3.00] | 0.277 [0.21, 0.35] |
| GRPO | 1 | −2.0 [−8.00, 4.00] | +0.5 [−5.00, 6.00] | 0.576 [0.45, 0.72] |
| GRPO | 2 | +0.5 [−5.00, 6.50] | +1.5 [−4.50, 7.50] | 0.993 [0.81, 1.20] |
| GRPO | 4 | 0.0 [−6.50, 6.00] | −2.0 [−7.50, 3.00] | 5.294 [4.58, 6.05] |
Four versus one GRPO visits raise excess mask distance from 0.576 to 5.294 points, about 9.2 times the original value, an increase of 4.718 points. Fixed changes from −2.0 to 0.0, with both intervals spanning zero. PPO mask distance also rises from 0.070 to 0.277. Repetition enlarges the policy difference without a comparable score increase.
To test timing, we expose the first or last 50 updates of a 135-update Adaptive run, then track the early-exposure run during clean continuation.
Table 8. Early and late exposure. Changes and mask distances are in pp.
| Method | Exposure window | Fixed: 0% → 100% | Fixed Δ [95% CI] | Holdout Δ | Excess mask distance [95% CI] |
|---|---|---|---|---|---|
| PPO | Early, steps 1–50 | 38.5→39.0 | +0.5 [−3.5, 4.5] | 0.0 | 0.003 [−0.005, 0.011] |
| PPO | Late, steps 86–135 | 33.0→35.0 | +2.0 [−2.0, 6.0] | +1.5 | 0.044 [0.026, 0.064] |
| GRPO | Early, steps 1–50 | 38.0→36.5 | −1.5 [−6.5, 3.5] | −1.0 | 1.670 [1.400, 1.964] |
| GRPO | Late, steps 86–135 | 35.0→40.0 | +5.0 [−0.5, 10.5] | +1.0 | 3.119 [2.760, 3.501] |
GRPO's late-exposure mask distance exceeds its early-exposure value by 1.449 points, 3.119 versus 1.670. The corresponding Fixed estimates are +5.0 and −1.5, with both intervals spanning zero. Table 9 traces how the early score advantage changes during clean training.
Table 9. Paired Fixed differences during early exposure and clean continuation, in pp.
| Method | Step 50 | Step 75 | Step 100 | Step 135 |
|---|---|---|---|---|
| PPO | −0.5 | −1.0 | +1.0 | +0.5 |
| GRPO | +6.5 | +3.5 | +3.0 | −1.5 |
GRPO's Fixed advantage falls from +6.5 through +3.5 and +3.0 to −1.5 points, an 8.0-point decline over 85 clean updates. Final parameter distance remains 0.1718 and excess mask distance 1.670 [1.400, 1.964]. Thus a score advantage can disappear while exposure-induced policy differences remain. Timing and clean continuation explain how an exposure effect becomes hidden from the final score.
Author Rebuttal (1/4)
W1, Q1: Can the proposed mechanisms be measured directly rather than inferred from final accuracy?
Answer: Thank you for suggesting direct mechanism measurements and controlled comparisons. The measurements show that exposure can leave parameter and visual-selection differences after its score advantage disappears. They directly test how exposure changes an update, how later training changes that effect, and whether changed behavior changes the scored answer.
To measure the local update effect, we branch from identical parameters, AdamW state and random-number state on Natural Qwen3-VL-8B / ChartQA. We test 32 exposed/clean question pairs at updates 200, 1,000 and 1,800. Two methods, two overlap conditions and three seeds give 1,152 paired diagnostics. To test persistence, Adaptive compares 50 exposed updates followed by 85 clean updates with a clean counterpart. Repeated mask sampling separately measures the final visual-policy difference.
Table 1. Direct update, continuation and policy measurements. ± is across-seed standard deviation. Brackets give 95% confidence intervals where available. n/r means not reported.
At 100% overlap, PPO's local parameter difference is approximately 8.6 times GRPO's. Only one of 288 exposed GRPO groups has mixed binary rewards, identifying why relative-advantage updates are rare. In Adaptive, clean continuation lowers GRPO's Fixed effect by 8.0 points, from +6.5 to −1.5, while parameter distance grows from 0.1223 to 0.1718. The experiment therefore separates retention of a learned change from retention of a score advantage. Q3 then tests the feedback explanation by changing the reward directly.
Q2: Does exposure change visual selection despite nearly unchanged answer accuracy?
Answer: Yes. In 83.59% of paired GRPO observations, the selected masks change but the scored outcome does not. The measured policy shift remains after subtracting the selector's ordinary resampling variation.
To test this distinction, we compare 0% and 100% overlap on the same 32 Fixed and 32 Holdout questions, sampling each condition 16 times per model. Excess mask distance is cross-condition mean mask disagreement minus within-condition mean disagreement. Question-paired bootstrap gives the 95% confidence intervals.
Table 2. Visual-policy differences and scored answers on Qwen3-VL-8B / ChartQA.
GRPO's excess mask distance is 2.6188 points [2.2683, 2.9937], and its mean action-probability difference is 4.718 points. Its Fixed effect is +3.5 [−2.5, +9.5], and the mask/score Spearman correlation is 0.150 [−0.097, 0.397]. The policy interval lies entirely above zero while the score interval spans zero. Together with the 83.59% unchanged-outcome rate, this directly shows what final accuracy misses: exposure can change the visual information selected without changing the evaluation's scored answer.