acceptodds
Under review as a conference paper at ICLR 2027

VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning

Abstract

Visual captioning converts visual information into text, requiring both completeness of salient content and correctness of expressed claims. Multimodal large language models (MLLMs) have advanced captioning through scaling and highquality data, while reinforcement learning (RL) offers a way to refine these factual dimensions. However, directly judging captions with an MLLM can yield coarse quality scores, while visual question answering (VQA) rewards supervise only queried facts. Both routes struggle to provide fine-grained rewards for individual omissions and errors. We propose VCap, a Witness–Adjudicator factcomparison reward that uses reference facts as inspection cues and visual evidence to adjudicate omissions and contradictions. References may be incomplete or imperfect; rewarding supported details beyond them enables weak-to-strong learning. A unified hypergeometric model explains how partial factual checks detect both coverage deficits and erroneous assertions under explicit sampling assumptions. Global and local temporal comparisons extend the same reward protocol from images to videos. Training Qwen3-VL-8B with VCap yields the highest aggregate scores among the evaluated models on CapMAS, DecapBench, and VDC. Ablations support the complementary reward components, and human assessment supports more complete captions. RL also outperforms backbone best-of-8 distillation, while the learned captions and policies benefit supervised training and visual question answering.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Caption, Reward Modeling, MLLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Q14rOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes VCap, a reward design for image and video captioning. A frozen multimodal judge uses the visual input and a reference caption to assess correctness, completeness, and text quality. The scores are combined to train a captioning model with GRPO. The paper also presents a hypergeometric analysis of factual checks and regenerates references for a second training round. Experiments include captioning benchmarks, component ablations, human assessment, distillation, and QA transfer.

Critical strengths
  1. Clear presentation: The paper is generally easy to follow. The roles of the reference caption and visual input are clearly described, and the limitations of the theoretical analysis are acknowledged.
  2. Broad experimental evaluation: The paper evaluates both image and video captioning and includes reward ablations, human assessment, distillation, and QA transfer. These experiments provide evidence that the overall training recipe is useful.
Critical weaknesses
  1. Limited novelty and incomplete positioning: The method mainly combines reference-assisted judging with GRPO, while the hypergeometric analysis is not directly used in reward computation. Recent work such as CCCaption, ClaimDiff-RL, and CapRL++ should be discussed to clarify the differences in completeness, correctness, and factual reward design.

  2. Insufficient validation of visual adjudication: The paper does not directly test whether the judge can correct deliberately erroneous references. Human evaluation also checks only model-proposed issues, leaving missed errors unmeasured. Stronger controls and an independent factual audit are needed to support the claimed mechanism.

Questions for authors
  1. How does VCap differ from the recent captioning methods listed above? Could a matched reference-assisted judge baseline establish the benefit of the proposed formulation?
  2. Does the judge remain reliable when references contain factual errors or omissions? Could an independent human audit check errors outside the model-proposed list?
5 comments
AuthorsOpenReview ↗

Response of Submission402 for Reviewer Q14r

We thank the reviewer for recognizing that the respective roles of the reference caption and the visual input come through clearly, since that separation of roles is what the rest of the method rests on. We are also grateful for the comments on the relation between the analysis and the implementation, the positioning against related work, judge reliability, and the coverage of our human-evaluation protocol. We did a substantial amount of additional work on these questions, and the findings have improved the paper.

Weakness I (a). Hypergeometric analysis not directly used in reward computation

It is true that no slot extraction is performed and no hypergeometric quantity is computed at training time (Section 2.2). The hypergeometric structure is nonetheless realized in the way VCap's reward is constructed. The model characterizes the statistical structure of the protocol (supplying both visual and partial textual evidence, using the reference as a per-claim inspection cue with the image adjudicating) and that protocol is precisely what the reward implements, whether or not a hypergeometric quantity is computed explicitly at runtime. Section 2.3's Scope paragraph limits the analysis to the reference-guided channel rather than to the judge's full behaviour.

We ran a new experiment to test this directly: degrading exactly this structure (judge compares the candidate directly against the image, with the reference as a weak anchor rather than a per-claim inspection cue) while holding images, GRPO hyperparameters, and judge fixed:

Comparison protocolCLAIRCoverageFactualityCapMAS AvgPrecisionRecallDCScoreDecapBench Avg
Qwen3-VL-8B-Instr (no RL)86.1971.3076.2577.9182.7355.6165.7169.17
Reference as anchor only (image-direct)86.7272.4478.1979.1285.1757.8268.7871.50
VCap (e1), structured text comparison89.1473.9581.4881.5287.8058.8369.5273.32

Removing the structure loses ground on all eight metrics, with the correctness axes (Factuality, Precision) and the completeness axes (Coverage, Recall) dropping together, the direction a real mechanism, not a cosmetic framing, would predict.

A second, more direct test of the model's own prediction. The hypergeometric account says the reference opens a detection channel: facts the reference covers should be easier to check than facts it does not. We measured this directly (full table in W2a/Q2a below). On inconsistencies, detection is 81.0% when the reference provides a textual cue versus 75.5% when it provides none; on omissions, 89.0% when the reference covers the fact versus 70.2% when it does not. The channel the analysis describes is visible in the measurement, and the residual detection in the no-cue regime is what the Scope paragraph attributes to direct visual inspection.

We will soften the title's framing in-text, add this ablation to Section 3.4, and revise the title accordingly.

AuthorsOpenReview ↗

Response II

Weakness I (b) and Question 1 (a). Positioning against CCCaption, ClaimDiff-RL and CapRL++

We confirm the omission and have added all three to Related Work. We have also run matched comparisons against ClaimDiff-RL-style and CapRL++-style rewards (Q1b below).

All three differ from VCap in where the factual inventory comes from and in what arbitrates. CCCaption derives visual queries from the image with multiple LVLMs and scores a caption by the share it answers, with a separate correctness judge on sub-caption queries; CapRL++ is reference-free, converting the image into a multiple-choice bank that a vision-free LLM answers from the caption alone, with format and length rewards in place of an explicit correctness term. In both, the factual source is the image rather than a reference, no visual arbiter is present at scoring time, and the question set bounds what can be checked. ClaimDiff-RL is the closest prior work: its judge receives the image, the candidate, and a reference from a strong external model (Gemini-3-Pro in the original), enumerates every candidate-reference difference and verifies each against the image. That protocol is more explicit and finer-grained than VCap's three holistic integer scores, and VCap belongs to the same family of reference-anchored comparison with visual adjudication, a family VCap did not introduce. What separates them is that VCap's reference is deliberately weak and self-generated, and serves as a one-directional inspection cue on the candidate rather than as one side of a symmetric comparison.

Across all three, the factual-comparison channel is demonstrated on images only; CapRL++ extends to video, but through a timestamp-format reward. VCap applies the same factual-comparison protocol to video, and gives an account of why the reward works.

Q1b below gives the empirical counterpart: with the reference source held fixed at VCap's weak self-generated caption, difference (1) is not cosmetic, it is where the ClaimDiff-RL-style protocol loses most of its effect.

Question 1 (b). Matched reference-assisted judge baseline

We have added the RL version of the baseline described below: same judge, same images, same reference, same GRPO setup as VCap (e1), but with the judge instruction stripped of VCap's comparison rules, dimension definitions, and "image is ground truth" instruction, replaced by a single overall 0 to 10 score:

RewardCapMAS AvgDecapBench AvgFactualityPrecision
Qwen3-VL-8B-Instr (no RL)77.9169.1776.2582.73
Generic reference-assisted judge78.8071.2775.4385.43
VCap (e1)81.5273.3281.4887.80

The generic matched baseline underperforms VCap on every metric, and notably its Factuality (75.43) falls below the untrained backbone (76.25), a reference in the prompt without an explicit comparison protocol and without "the image decides" can make factuality worse than no training at all. This is the matched baseline literally requested, and it separates the proposed formulation from "any judge with a reference in the prompt."

We also completed a matched comparison against the two closest published protocols. Every row uses the same reference caption as VCap (e1) (the weak, self-generated one) not ClaimDiff-RL's original strong external reference or CapRL++'s image-derived question bank, so the only variable changing across rows is the comparison protocol:

RewardCLAIRCoverageFactualityPrecisionRecallDCScore
Qwen3-VL-8B-Instr (no RL)86.1971.3076.2582.7355.6165.71
Generic reference-assisted judge87.0673.9075.4385.4357.1166.20
ClaimDiff-RL-style86.3572.2676.6084.4357.6165.87
CapRL++-style (paper default N=8)86.7871.8477.4986.7956.3765.27
VCap (e1)89.1473.9581.4887.8058.8369.52

Under the weakened reference, the ClaimDiff-RL-style reward barely moves CLAIR/Factuality/DCScore off the untrained backbone (all within 0.5 points), which is the regime its pairwise difference-enumeration protocol assumes away: it was built around a strong external anchor. The CapRL++-style reward gets closer on Precision but not on Coverage/Recall/DCScore, with DCScore falling below the untrained backbone, which is what VCap's mechanism reduces to at a small sampled fact count and with no visual arbiter for errors outside the sampled questions. Taken together, the comparison points to VCap being the more robust of the three to both the reference and the reward model. A reference caption can be supplied in whatever form the reward model is invoked, and the amount of factual content it carries is not capped in advance; a VQA bank fixes both how many facts can be checked and the form in which they are checked, so whatever falls outside the sampled questions is unreachable.

AuthorsOpenReview ↗

Response III

Weakness II (a) and Question 2 (a). Does the judge remain reliable when the reference has errors or omissions?

To settle this, we ran an in-depth independent audit. Using the uncorrected 8B backbone caption as the reference, itself independently audited at 9.43% error rate and 23.24% missed-fact rate, i.e. genuinely "containing factual errors or omissions", we scored VCap (e2) with the 32B judge and stratified its true errors and omissions by whether the reference could help:

CaseTrue countCaught by the judge-assisted protocolCatch rate
Candidate's inconsistency is also present in the reference211466.7%
Inconsistency, reference gives no cue at all13910575.5%
Inconsistency, reference gives a cue423481.0%
Omission, reference also misses it63444570.2%
Omission, reference does not miss it31027689.0%

The first row is the direct test of your scenario: when the candidate repeats an error the reference also contains, a judge relying on text agreement sees agreement and misses it, while ours still flags two thirds of these cases from the image. Overall detection is 75.7% on inconsistencies and 76.4% on omissions.

Two limits: the shared-error slice is small (n=21), so 66.7% carries a wide interval; and the measurement used a fact-enumerating judge instruction for human verification rather than the three-score training-time reward instruction, so it characterizes the judge's detection capability, not training-time behaviour verbatim.

Neither input is dispensable, though each ablation read alone suggests the other is. The no-cue rows (75.5%, 70.2%) might read as the judge doing nearly as well with no reference; retraining says otherwise, since Table 4's "−reference caption" row lands at 77.65/69.52 CapMAS/DecapBench Avg against the untrained backbone's 77.91/69.17, with Coverage and Recall falling below backbone (70.24 vs. 71.30, 53.24 vs. 55.61). Conversely, Table 4's "−reference image" row costs only 0.48/1.66, which makes visual adjudication look redundant, but that is measured with the reference and the comparison rule still in place, and on most samples the reference carries the signal.

The two are not in conflict because they quantify different things. The table above is recall given that a human-audited error exists at a location; it says nothing about precision when the judge has no reference to bound what is worth checking, and precision is what sets within-group contrast under GRPO. The asymmetry between the axes is the mechanism: an error can be checked against the candidate's own claims, an omission only against some notion of what should have been said. That is where the gap is widest (89.0% versus 70.2%) and exactly the axis that falls below backbone without a reference. The reference supplies the completeness inventory and the image adjudicates correctness, and the minority of cases where the reference is silent or wrong is what determines whether the method degrades gracefully under a weak reference. The revision will report both and scope the visual-grounding claim accordingly. If you consider this evidence insufficient, we will prioritize a larger stratified test with deliberately injected reference errors in the remaining discussion period.

Reviewer_ezjCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VCap, a reference-guided reinforcement-learning framework for image and video captioning. Existing caption rewards are either coarse MLLM quality scores or VQA checks limited to a question pool. Theoretically, a unified hypergeometric model treats completeness and correctness as the same detection event with different defect sets, and shows that a partial reference can still expose omitted facts and erroneous claims. In the method design, VCap trains captioners with a Witness–Adjudicator fact-comparison reward using frozen off-the-shelf models: (1) a frozen Qwen3-VL-32B-Instruct judge compares a candidate caption against a reference caption and the raw visual input, using the reference as an inspection cue and the image or video as ground truth, and scores factual correctness, completeness, and auxiliary text quality so that visually supported details absent from the reference remain eligible for reward; (2) the same scoring protocol is extended to video by combining a full-video global check with a caption-defined local segment check; and (3) image and video policies are trained separately from Qwen3-VL-8B-Instruct with GRPO, where VCap (e1) uses offline references generated by the original 8B backbone and VCap (e2) is retrained from that backbone using references regenerated by e1. The framework is evaluated on CapMAS and DecapBench for images and on VCapsBench and VDC for videos, together with reward-component ablations, human assessment, best-of-8 distillation, and QA transfer without task-specific fine-tuning.

Critical strengths
  1. Extensive empirical evaluation and strong results: The paper evaluates VCap on both image and video captioning, with additional human assessment and QA transfer experiments (Figure 3, Table 7). It examines key components of the training recipe: the comparison between e1 and e2 keeps the backbone fixed to assess the effect of reference regeneration (Section 3.2), while Tables 4–5 ablate the reference, visual input, individual score dimensions, and local video term. Table 6 further compares VCap RL with best-of-8 SFT using the same backbone. Best-of-8 self-distillation underperforms the original model (CapMAS / DecapBench Avg: 77.65 / 67.97 vs. 77.91 / 69.17), suggesting that the gains cannot be explained by simply imitating selected outputs from the backbone. VCap (e2) achieves 83.53 / 73.67 on CapMAS / DecapBench Avg and improves VDC Avg from 30.78 to 36.01. These results demonstrate the effectiveness of VCap (Tables 2–3).
  2. Solid theoretical analysis of reference-guided factual checks: Completeness and correctness are put under one hypergeometric detection model with different defect sets. The analysis explains why a partial reference can still expose omissions and errors, and why a higher-coverage regenerated reference should strengthen that signal.
Critical weaknesses
  1. Limited methodological novelty relative to existing LVLM-as-a-Judge rewards: The training reward is a frozen Qwen3-VL-32B-Instruct prompted with the visual input, a reference caption, and a candidate caption, then linearly combining three qualitative integer scores. Section 2.3’s fact-slot model is not used at training time: no slots are extracted and no hypergeometric quantity is computed. What is new in the implemented method is therefore small: a reference in the prompt and a named completeness/correctness split, not a new reward mechanism.
  2. The self-evolve claim is not adequately verified: Figure 2(c) presents reference regeneration as an iterative (Self-Evolve ···) process, but the paper reports only e1 and e2. Two points are not enough to read a trend. The self-evolve formulation is therefore not adequately verified.
Questions for authors
  1. Captioning scores of the reward model: Tables 2 and 3 report Qwen3-VL-8B-Instruct and Qwen3-VL-235B-Instruct, but not Qwen3-VL-32B-Instruct, which is the frozen judge used in training (Section 2.2). Could the authors add this model’s CapMAS, DecapBench, VCapsBench, and VDC scores under the same evaluation pipeline, so that the trained 8B policy can be compared with its supervisor?
  2. Self-evolve trend beyond e2: Figure 2(c) presents reference regeneration as an iterative (Self-Evolve ···) process, but only e1 and e2 are reported. Could the authors add e3 under the same recipe on CapMAS, DecapBench, VCapsBench, and VDC, or plot these metrics over further rounds ene_n, so that the self-evolve claim can be checked as a trend rather than a two-point comparison?
6 comments
AuthorsOpenReview ↗

Response of Submission402 for Reviewer ezjC

We thank the reviewer for recognizing the analysis: completeness and correctness are placed under a single detection model that differs only in the defect set, and that unification is what we most hoped a reader would take from Section 2. We are also grateful for the comments on the novelty of the reward mechanism and on the evidence behind the self-evolve claim. We ran a substantial set of experiments on both, and the findings have improved the work; they are reported below.

Weakness I (a). The fact-slot model is not used at training time; therefore no new mechanism?

It is true that at training time we do not explicitly extract fact slots or compute any hypergeometric quantity. Section 2.2 states this plainly ("Factual comparison takes place within this prompted judgment, with no explicit exhaustive fact graph."). The hypergeometric structure is nonetheless realized in the way VCap's reward is constructed.

Our position. The hypergeometric model characterizes the statistical structure of the comparison protocol itself, not a computation the judge must literally run. As long as a protocol (a) supplies both visual evidence and a partial textual reference, and (b) uses that reference as an inspection cue to check the candidate's claims, with the image adjudicating disagreements, that protocol has the structure of sampling without replacement from a population of facts, whether or not anyone computes the hypergeometric CDF at runtime. We will add a sentence to Section 2.3/Section 1 making this explicit.

On scope: Section 2.3 limits the analysis to the reference-guided channel, noting that the judge can inspect additional visual facts and that its overall accuracy and the trained policy's gains require empirical evaluation. We are not claiming the hypergeometric analysis explains the judge's entire behaviour, only the value of the specific design choice of using a partial reference as an inspection cue. We agree the title ("Hypergeometric Rewards") invites the stronger, incorrect reading; we will soften the in-text framing in the abstract and introduction, and we will revise the title accordingly.

We ran a new experiment to test this directly. To turn the point from a definitional dispute into an empirical one, we ran an ablation that degrades exactly the structure in question, holding everything else fixed (same images, same GRPO hyperparameters, same judge):

Comparison protocolCLAIRCoverageFactualityCapMAS AvgPrecisionRecallDCScoreDecapBench Avg
Qwen3-VL-8B-Instr (no RL)86.1971.3076.2577.9182.7355.6165.7169.17
Reference as anchor only (image-direct)86.7272.4478.1979.1285.1757.8268.7871.50
VCap (e1), structured text comparison89.1473.9581.4881.5287.8058.8369.5273.32

"Reference as anchor only" keeps the same inputs but has the judge compare the candidate directly against the image, with the reference serving only as a weak anchor rather than a per-claim inspection cue. Degrading the structure loses ground on all eight metrics, 2.40/1.82 points on the CapMAS/DecapBench averages, with the correctness axes (Factuality 81.48 → 78.19, Precision 87.80 → 85.17) and the completeness axes (Coverage 73.95 → 72.44, Recall 58.83 → 57.82) dropping together. If the structured-comparison protocol were interchangeable with "a reference in the prompt plus two named score dimensions," removing the structure should not systematically hurt both axes, but it does. We propose adding this to Section 3.4 and the appendix. We are grateful for this challenge: it is what produced the E9 ablation, which turns the hypergeometric account from a framing into a claim that can be tested and, as it happens, supported.

A second, independent line of support comes from detection rates we measured during the discussion period: on the subsets where the reference does provide a textual cue, the judge catches 81.0% of inconsistencies and 89.0% of omissions, versus 75.5% and 70.2% where it does not. That is the reference-guided detection channel the hypergeometric analysis describes, measured directly, with the gap between the two regimes being exactly what the model predicts.

AuthorsOpenReview ↗

Response II

Weakness I (b). Implementation novelty is small (reference in prompt plus named split)

It is true that the reward takes that form, and Appendix A.4 already places VCap in the broad LVLM-as-a-judge family. What we think is underweighted:

  1. The specific comparison rule ("the image is the source of truth; if the reference conflicts with the image, prioritize the image; a candidate detail absent from the reference but visually supported remains valid") is a design choice distinguishing this from reference-matching rewards (CIDEr, embedding similarity, SFT-to-reference).
  2. This "simple" choice is where almost all of the measured gain is: Table 4 shows the same judge, same image, with vs. without the reference differs by 3.87/3.80 points on CapMAS/DecapBench Avg, and without the reference the gain nearly disappears (77.65 vs. the backbone's 77.91).
  3. The video global+local dual-scale check, the weak-reference regeneration loop (e1→e2), and the completeness/correctness ablation (Table 4: removing completeness costs Coverage/Recall 1.01%/3.57%; removing correctness costs Factuality/Precision 5.45%/0.84%) are each small but jointly load-bearing.

That said, we accept that none of these pieces is individually unprecedented. ClaimDiff-RL in particular uses a closely related "reference as comparison anchor, image as adjudicator" idea, We have cited it and run a matched comparison against it (Q1b below). We will reframe the contribution as: a reward protocol that transfers cleanly from images to video, an analysis of why a weak self-generated reference suffices, and a systematic empirical evaluation, not a claim of a wholly new computational mechanism.

We also ran the matched-protocol comparison this point calls for. Every row uses the same judge (Qwen3-VL-32B-Instruct), the same images, and the same reference caption as VCap (e1) (the weak, self-generated reference), so the only variable changing is the comparison protocol itself:

RewardCLAIRCoverageFactualityPrecisionRecallDCScore
Qwen3-VL-8B-Instr (no RL)86.1971.3076.2582.7355.6165.71
Generic reference-assisted judge87.0673.9075.4385.4357.1166.20
ClaimDiff-RL-style (judge swapped to Qwen3-VL-32B-Instruct)86.3572.2676.6084.4357.6165.87
CapRL++-style (VQA-from-reference, paper default N=8)86.7871.8477.4986.7956.3765.27
VCap (e1)89.1473.9581.4887.8058.8369.52

Both alternatives trail VCap substantially. The ClaimDiff-RL-style reward barely moves CLAIR/Factuality/DCScore off the untrained backbone once the reference is weakened to match VCap's, which is not the regime its pairwise difference-enumeration protocol was built for: that protocol assumes a strong externally-generated anchor. The CapRL++-style reward gets close on Precision but not on Coverage/Recall/DCScore, with DCScore falling below the untrained backbone, which is what VCap's mechanism reduces to at a small sampled fact count with no visual arbiter. Taken together, the comparison points to VCap being the more robust of the three to both the reference and the reward model. A reference caption can be supplied to the judge in whatever form the reward model is invoked, and the amount of factual content it carries is not capped in advance; a VQA bank fixes both how many facts can be checked and the form in which they are checked, so whatever falls outside the sampled questions is unreachable no matter how capable the judge is. Protocol details for both settings, including the respect in which the CapRL++-style reward is an adaptation rather than a reproduction, will be included in the revision.

AuthorsOpenReview ↗

Response III

Weakness II and Question 2. Self-evolve claim verified on only two points (e1, e2)

We call the procedure self-evolving because the supervision is regenerated by the policy itself: e1 is trained against references from the original 8B backbone, e2 against references written by e1, and no external model or human annotation enters at any round. The loop demonstrably improves the supervision, with reference error rate falling 9.43% → 6.71% → 4.59% and audited fact coverage rising 69.52% → 75.65% → 78.56%, and the policy improves with it: e1 already exceeds GPT-5.4 on both image-captioning averages, and e2 extends the gain over the backbone to 5.62% on CapMAS and 4.50% on DecapBench. An 8B captioner surpassing the model that produced its own supervision is what the term is meant to denote.

Two points do leave the shape of that curve underdetermined, so we ran a third: e3 follows the same recipe on the same 29k COCONut images, with the reference regenerated by e2.

SetupReference fromCapMAS AvgDecapBench AvgCoverage
VCap (e1)Qwen3-VL-8B-Instr81.5273.3273.95
VCap (e2)VCap (e1)83.5373.6774.18
VCap (e3)VCap (e2)82.7573.1074.54

e3 improves Coverage slightly (74.54 vs. 74.18) but regresses on the aggregate CapMAS/DecapBench Avg relative to e2, and the cause is caption length: e3's captions grow to 874 words per image on average, up from 524 at e2, spreading single facts across more text, which costs detail both under the judge and under the QA-style evaluation metrics.

This is the behaviour the loop is designed around rather than an unexpected failure. RL fine-tuning of an 8B policy degrades once the number of updates grows large, particularly when successive rounds revisit the same 29k images, and the failure mode is drift in generation style rather than a visible drop in reward. Round 3 shows that signature: the completeness term continues to be satisfied while the policy drifts toward longer text rather than more accurate text. We therefore run the loop for a small number of rounds by design, rather than treating round count as a parameter to push.

The reference-quality audit quoted above covers all three generations, so the round-3 regression is not caused by the reference degrading: the reference keeps getting better while the candidate text gets longer and more diffuse, and that is the operative tension at round 3. This also bounds the claim. We demonstrate two productive rounds and a measured saturation point at the third, which is what Figure 2(c)'s "Self-Evolve ···" ellipsis should say instead of implying unbounded continuation; we will replace it accordingly.

On the video modality: the e3 run so far covers images only; the corresponding VCapsBench and VDC numbers are being produced and we will post them in this thread before the discussion period closes.

Reviewer_tbt8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes VCap, an RL method designed to make image and video captions more complete and factually accurate. During GRPO training, a frozen multimodal judge compares candidate captions against reference captions while checking the actual image or video to catch omissions and errors. Importantly, references are used as inspection cues rather than strict ground truth, which allows the model to be rewarded for valid visual details missing from the reference. The authors also provide a hypergeometric model to explain how partial references help detect defects, extend the reward to video via global and local checks, and test iterative reference regeneration. Experiments on Qwen3-VL-8B show solid gains across image and video benchmarks, backed by ablations, human evaluation, distillation, and QA transfer—though the improved coverage in video comes with a higher rate of factual inconsistency.

Critical strengths
  1. Using the same Qwen3-VL-8B backbone, VCap improves performance across both image and video benchmarks. Ablations confirm that removing references clearly hurts results, and dropping correctness or completeness reduces factuality or recall, respectively. These results show the practical value of reference-guided judging and the complementarity of the two rewards, serving as the primary evidence for acceptance.
  2. The reward framework extends cleanly from images to videos, with local evaluation helping capture fine-grained video details. The learned captions also transfer most of their gains to student models via distillation and modestly improve downstream QA. Importantly, deployment only requires the 8B captioner without references or a judge, making the method practically useful despite an remaining coverage–inconsistency trade-off in video.
Critical weaknesses
  1. Unclear definition of “weak references” and confounded weak-to-strong gains. While the theoretical analysis assumes references are incomplete but factually correct, model-generated references inevitably contain errors. The paper neither evaluates the quality of these references nor tests the judge's robustness when the reference is wrong. Moreover, training relies on a larger 32B judge without an 8B judge baseline, making it hard to tell whether the "weak-to-strong" gains come from the proposed reference mechanism or simply from distilling a stronger 32B evaluator.

  2. Unproven advantage of the caption format over other fine-grained rewards. Both reference captions and VQA question pools suffer from partial factual coverage. While ablating the reference confirms that extra cues help, it does not prove that formatting these cues as a free-form caption is superior to using explicit rubrics or VQA pairs derived from the same facts under a matched budget.

Questions for authors

1.How do the original and e1-generated references compare in terms of factual accuracy and coverage? Specifically, when a reference contains a hallucination, does the judge reliably favor a visually grounded candidate over one that copies the reference's mistake? 2.Keeping the judge and training setup fixed, how do 32B-generated references compare with self-generated (8B) references in both final performance and compute cost? 3.Do the RL gains still hold if the 32B judge is replaced by the 8B backbone model (while keeping the reference source fixed)? 4.Under a matched setup (same judge, visual input, factual coverage, and compute budget), how does VCap compare against dynamic-rubric or VQA rewards constructed from the exact same reference facts?

I am open to raising my score if the authors can convincingly address these concerns during the rebuttal.

6 comments
AuthorsOpenReview ↗

Response of Submission402 for Reviewer tbt8

We thank the reviewer for recognizing the deployment property: at inference only the 8B captioner is needed, with no reference and no judge, which is the practical constraint the whole design is organized around. We are also grateful for the comments on the definition of weak references, the attribution of the gains, reference quality, judge scale, and the reward format. We ran a substantial set of experiments on these questions during the discussion period, and the findings have improved the work; we report them below together with the limits of each.

Weakness I (a). Unclear definition of "weak references"; are the gains confounded?

It is true that the paper never gives "weak reference" or "weak-to-strong" an operational definition. We will add the following to Section 1/Section 3.2 of the revision:

Both ends of the pair are defined on the two axes the paper already evaluates captions on, correctness and completeness: how many of the facts a caption states are inconsistent with the image, and how many of the image's salient facts it fails to state. Section W1b below reports both for every reference we use.

  • Weak reference = a reference that is measurably wrong and measurably incomplete on those two axes, and that carries no privileged information beyond what the policy itself could produce. VCap (e1)'s reference is the original 8B backbone's own caption, audited at 9.43% of its stated facts inconsistent with the image and 23.24% of the salient facts missing; VCap (e2)'s reference is the previous round's policy. Neither is complete and neither is correct.
  • Strong = the trained policy ends up ahead of the model that generated its own reference on those same two axes, and ahead of the frozen judge's captioning ability on the held-out benchmarks. VCap (e1) reaches 6.71% error and 75.65% audited fact coverage, and e2 reaches 4.59% and 78.56%, against the reference generator's 9.43% and 69.52%.
  • The judge (Qwen3-VL-32B-Instruct) is explicitly not part of the weak/strong pair. It is a fixed external comparator.

On the substantive "confounded gains" claim (that it is hard to tell whether the gain comes from the proposed reference mechanism or simply from distilling a stronger 32B evaluator) we offer the following evidence:

  1. Reference ablation, judge held fixed (Table 4, "−reference caption"): same 32B judge, same images, only the reference removed. On the image task this removes almost all of the gain (CapMAS/DecapBench Avg 77.65/69.52 vs. backbone 77.91/69.17, vs. full VCap (e1) 81.52/73.32). This rules out "any judge that looks at the image is enough."
  2. Judge-scale ablation, reference held fixed (W1c/Q3 below): keeping the reference and swapping the 32B judge for the 8B backbone itself recovers almost all of the gain (81.42 vs. 81.52 CapMAS Avg). This rules out "the gain is really just distillation from a stronger 32B evaluator."
  3. Quantified, round-by-round dose-response (W1b/Q1 below): we audited reference quality at each round, error rate 9.43% (original) → 6.71% (e1) → 4.59% (e2), audited fact coverage 69.52% → 75.65% → 78.56%, against a correspondingly measured trend in downstream policy quality (CapMAS/DecapBench Avg 77.91 → 81.52 → 83.53, with e3 marking the point at which caption-length inflation offsets further reference-quality gains).

(1) and (2) each rule out one of the two obvious confounds with a matched ablation, and (3) adds a quantified trend across three rounds rather than a single before/after comparison. We will restructure Section 3.2/Section 3.4 so the three are presented together.

A further note. Table 4's other modality ablation, "−reference image", costs only 0.48/1.66 CapMAS/DecapBench Avg (81.04/71.66 vs. 81.52/73.32), against 3.87/3.80 for removing the reference. In aggregate the reference is load-bearing and the image is not. The two ablations measure different populations: Table 4 withholds the image from a judge that still has the reference and the comparison rule, which on most samples is enough on its own, whereas the stratified detection rates in W1b/Q1 isolate the minority of samples where the reference is silent or wrong, and there the image is the only arbiter available. Our conclusion is that the image's aggregate contribution is small while its contribution on that minority is decisive, and the revision will state both rather than the second alone.

We will also state plainly that Section 2.3's theoretical analysis assumes the reference is "incomplete but factually correct" and does not model reference errors. The narrower question (robustness to one specific wrong claim in the reference) is addressed in W1b/Q1 below.

AuthorsOpenReview ↗

Response II

Weakness I (b) and Question 1. Reference quality, and judge robustness to reference errors

Part 1, reference quality, directly measured. We ran a 100-image human audit (annotators blind to any model-proposed issue list) against a checklist of 5,344 salient facts. The checklist was assembled for this audit by applying the fact-extraction protocols of CapMAS and DecapBench to these images, with every extracted fact human-verified; it is not a merge of either benchmark's released annotations.

SourceWordsCorrect factsInconsistenciesError rate ↓Audited fact coverage ↑MissedMissed rate ↓
Qwen3-VL-8B-Instr (original reference)3223,7153879.43%69.52%1,24223.24%
VCap (e1)4894,0432916.71%75.65%1,01018.90%
VCap (e2)5244,1982024.59%78.56%94417.66%

Error rate = inconsistencies / (correct + inconsistent) claims; audited fact coverage = correct facts / 5,344. We use "audited fact coverage" rather than "coverage" throughout this response to avoid confusion with the CapMAS Coverage metric, which is a different quantity with a different denominator.

The e1-generated reference is measurably better than the original 8B reference on every axis, and e2 improves further. This answers the first half of Q1 with a direct audit rather than the benchmark-score proxy we previously relied on. We are grateful for this question: the paper asserted that reference quality improves across rounds, and the audit it prompted turns that assertion into a measured relation between reference quality and downstream policy quality, which is now one of the paper's load-bearing results rather than a claim made in passing.

Part 2, does the judge favor a grounded candidate over one that copies a reference's mistake? To settle this, we ran an in-depth independent audit; the relevant evidence is one slice of it. Using the uncorrected original 8B caption as the reference, itself audited at 9.43% error rate and 23.24% missed-fact rate, while scoring VCap (e2) with the 32B judge, we isolated the cases where the candidate's error is the same error present in the reference, i.e. exactly the "copies the reference's mistake" scenario, since a judge relying on text agreement would treat these as consistent and miss them entirely:

CaseTrue countCaught by the judge-assisted protocolCatch rate
Candidate's inconsistency is also present in the reference211466.7%
Inconsistency, reference gives no textual cue at all13910575.5%
Inconsistency, reference provides a cue423481.0%
Omission, reference also misses the fact63444570.2%
Omission, reference does not miss it31027689.0%

The first row is the direct answer: when the candidate repeats the reference's own mistake, text comparison alone would find nothing wrong, yet the judge still flags it two times out of three by weighing the image. The remaining rows show the same pattern the hypergeometric account predicts: where the reference offers no cue the judge still catches a majority (75.5% and 70.2%) from the image alone, and where it does provide a cue the rate is visibly higher (81.0% and 89.0%).

Two limits: (a) the shared-error sample is small (n=21), so 66.7% carries a wide interval; (b) this measurement used a judge instruction modified to enumerate every fact for human verification, rather than literally replaying the three-score training-time reward instruction, so strictly it characterizes the same judge model's detection capability rather than a verbatim replay of training-time reward behaviour on this scenario.

These catch rates do not make the reference dispensable. The no-cue rows (75.5%, 70.2%) might read as the judge doing nearly as well with no reference; retraining says otherwise, since Table 4's "−reference caption" row lands at 77.65/69.52 CapMAS/DecapBench Avg against the untrained backbone's 77.91/69.17, with Coverage and Recall falling below backbone (70.24 vs. 71.30, 53.24 vs. 55.61).

The two measurements quantify different things. The table above is recall given that an error is known to exist; it says nothing about precision when the judge has no reference to bound what is worth checking, and precision is what sets within-group contrast under GRPO. The asymmetry is the mechanism: an error can be checked against the candidate's own claims, an omission only against some notion of what should have been said. That is where the gap is widest (89.0% versus 70.2%) and exactly the axis that falls below backbone once the reference is removed. If you consider this insufficient, we will run a larger stratified test with deliberately injected reference errors under the training-time scoring instruction.

AuthorsOpenReview ↗

Response III

Weakness I (c) and Question 3. Is the gain just distillation from a stronger 32B judge?

We have filled in the missing runs across judge scale and the presence of the reference caption on the image task, keeping data, GRPO hyperparameters, and reward weights identical to VCap (e1):

SetupJudgeRef.CapMAS AvgDecapBench Avg
Qwen3-VL-8B-Instr (no RL),,77.9169.17
32B judge, no reference32B✗77.6569.52
VCap (e1)32B✓81.5273.32
8B judge, reference8B✓81.4273.09
8B judge, no reference8B✗76.7869.30

Two things follow. (1) With the same 32B judge and the same images, removing only the reference removes almost all of the gain (77.65 vs. the backbone's 77.91), so on the image task the gain is not "32B judge reads the image and scores well," it comes with the reference. (2) Replacing the 32B judge with the 8B backbone itself, while keeping the reference, recovers almost all of VCap (e1)'s gain (81.42 vs. 81.52), and the 8B judge even wins on Coverage and DCScore (74.22/69.77 vs. 73.95/69.52). The mechanism is not contingent on judge scale. The 8B-judge-no-reference setting shows the same reference-dependence pattern as the 32B case (76.78 vs. 81.42): without a reference, the 8B judge misses more fine detail in direct image comparison, and the policy learns to describe only the main content to avoid being penalized for errors, which costs coverage.

We will add this table to Section 3.4 and rewrite the "−reference caption" discussion to make the judge-scale point explicit; previously we had only the reference-ablation half of this argument.

A caveat on video: on video, Table 5's "−reference caption" row already shows a substantial gain from the 32B judge alone (VCapsBench AR 65.75 vs. backbone 63.28, full 71.34), so the "not just distillation" argument is strongest on images and weaker on video. We will say this in the text.

Question 2. 32B-generated references versus self-generated (8B) references

We generated references with Qwen3-VL-32B-Instruct on the same 29k COCONut training images under matched decoding, kept the judge at 32B, and trained one image RL run otherwise identical to e1:

SetupReference fromCapMAS AvgDecapBench Avg
VCap (e1)Qwen3-VL-8B-Instr81.5273.32
VCap (32B reference)Qwen3-VL-32B-Instr81.5773.22

The two are within noise of each other (+0.05 CapMAS, −0.10 DecapBench), consistent with the fact that the 8B and 32B reference generators' own benchmark scores are also close (77.91/69.17 vs. 78.43/68.24). This supports the "weak references are already sufficient" framing, upgrading the reference generator does not materially change downstream policy quality here, so the method is not relying on reference strength to work.

On compute: reference generation is a one-off offline pass over the 29k training images, performed once before RL and never repeated per step, per epoch, or at deployment. Moving it from the 8B backbone to a 32B model multiplies the cost of that single pass by roughly the parameter ratio while leaving the RL loop, the judge cost, and the deployed model untouched. Since the measured quality difference is within noise, the 4× more expensive reference pass buys nothing here, which is the practically relevant answer. We will report the exact GPU-hours for both reference passes in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.