acceptodds
Under review as a conference paper at ICLR 2027

Momentum-Coupled Rubric Adaptation for Detailed Image Captioning

Abstract

Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy, information coverage, and clarity. Compared with conventional methods that rely mainly on high-quality supervision or holistic rewards, rubric-based reinforcement learning decomposes these requirements into explicit criteria and provides targeted, structured feedback. However, existing methods often use separate models for caption generation, rubric construction, and judging, which may lead to inconsistent interpretations across roles. Some dynamic rubric methods alternate updates between the caption policy and rubric generator while keeping the judge fixed, but staged optimization may still leave rubric construction and judging out of step with policy optimization. We propose MoCo Rubric, a two-stage framework that coordinates these roles. First, role-conditioned, shared-parameter multi-task supervised fine-tuning equips a single vision–language model to serve as the Caption Policy, Rubric Generator, and Rubric Judge. Then, the Generator constructs rubrics online from captions sampled by the current Policy, reference captions, and image evidence. The Judge provides rubric-based rewards, and only the Policy receives GRPO updates. As Policy updates change the candidates being evaluated, we use an exponential moving average of the Policy parameters to update one momentum model shared by the Generator and Judge. This gradual transfer lets both rubric roles track Policy updates without separate RL optimization while smoothing parameter changes that could disrupt their rubric capabilities under direct synchronization. Under bounded Policy updates, we derive an upper bound on the parameter gap between the Policy and momentum model and analyze the smoothing effect of momentum updates. Across five captioning benchmarks, MoCo Rubric achieves an average pairwise win rate of 72.83%, the best mean rank in blind ranking, and the highest average score in caption-based question answering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Detailed Image Captioning, Rubric-Guided Reinforcement Learning, Momentum Adaptation, Vision-Language Models.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iUjgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MoCo Rubric for rubric-based reinforcement learning in detailed image captioning. The method first uses shared-parameter, multi-role supervised fine-tuning to establish caption generation, rubric construction, and judging capabilities. It then applies reinforcement learning only to the caption policy, while updating the parameters shared by the Generator and Judge through an exponential moving average (EMA). Together with online rubrics constructed from current policy candidates, the framework aims to coordinate evaluation with the evolving caption policy and improve caption quality.

Critical strengths
  1. The paper presents a reasonable method design for coordinating caption generation and evaluation. Shared multi-role initialization provides a common foundation, online rubrics adapt to current candidates, and momentum updates allow the evaluation roles to gradually track policy changes. These components form a coherent framework for adaptive training feedback.
  2. Evaluation across caption quality and downstream tasks supports the method’s overall effectiveness. Pairwise comparisons, blind ranking, and dimension-specific scores assess caption quality, while caption-based question answering further examines the utility of the preserved visual information. These complementary evaluations provide broad empirical support for the reported improvements.
  3. Detailed ablations support the contributions of the main components and their combination. Comparisons cover shared versus separate initialization, offline versus online rubrics, and frozen, fully synchronized, and EMA-updated evaluation roles. Further comparisons of Generator and Judge updates provide evidence supporting joint momentum updates.
Critical weaknesses
  1. The motivation is not sufficiently established. The paper motivates its approach through inconsistent interpretations when separate models perform caption generation, rubric construction, and judging. However, a strong independent Judge can also provide complementary scrutiny of the generator’s outputs. Further justification is needed to explain why differences across roles constitute a problem and why greater consistency should improve task performance. Figure 1 illustrates an error missed by an overly broad rubric, but does not sufficiently establish whether the underlying issue is cross-role inconsistency rather than inadequate rubric quality or judging capability.
  2. The connection between the motivation and the observed gains lacks direct mechanistic analysis. The experiments support the effectiveness of the complete method, but do not clearly establish whether the gains arise from improved consistency. Section 3.4 analyzes parameter tracking and smoothing, without directly connecting these properties to changes in rubric quality or Judge accuracy. In particular, why parameter updates optimized for caption generation benefit the other two roles requires further explanation. The relationship among the motivation, method design, and performance improvements therefore remains unclear.
Questions for authors
  1. Why is cross-role inconsistency a central problem that needs to be addressed? The introduction on page 2 motivates the method through semantic inconsistency across models and illustrates the issue with the undetected counting error in Figure 1. A common reason for using a strong independent Judge is to provide complementary scrutiny from a perspective different from that of the generator. Why should such differences harm training feedback, and why should shared initialization and coordinated evolution improve it? How can the failure in Figure 1 be distinguished from insufficiently accurate or comprehensive rubrics?

  2. Could an additional evaluation step for tied rollout groups yield similar training benefits?The experimental analysis reports fewer rollout groups with identical rewards under the proposed method, allowing more groups to provide relative reward signals. If reducing tied groups is a major source of improvement, could effective training signals be recovered by keeping the Generator and Judge parameters fixed and adding an evaluation step only for tied groups—for example, reconstructing the rubric or applying finer-grained judgments? What additional benefits do joint EMA updates provide beyond this approach?

  3. Why is the Judge configured without direct access to the image?Section 3.3 specifies that the Judge evaluates candidate captions using only the captions and rubrics, without directly accessing the image. Normally, a Judge considers the task input when assessing whether an output satisfies the task requirements. For image captioning, the original image is essential for determining factual accuracy and completeness. Why does the Judge obtain image information only indirectly through the Generator’s rubric, rather than inspecting the image directly? What motivates this input restriction?

  4. Can rubrics constructed from a subset of candidates fairly evaluate the entire group?Section 3.3 randomly selects four of the sixteen candidate captions to construct a rubric, which is then applied to all sixteen candidates. The criteria may focus on issues exposed by the selected candidates while missing important quality differences in the remaining candidates. What motivates the choice of four candidates? How does the method ensure that the resulting criteria also apply to candidates excluded from rubric construction, without introducing subset-dependent bias into the within-group reward ranking?

7 comments
AuthorsOpenReview ↗

Response to Reviewer iUjg — Part 1/4

Q1. Why is cross-role inconsistency a central problem that needs to be addressed? The introduction on page 2 motivates the method through semantic inconsistency across models and illustrates the issue with the undetected counting error in Figure 1. A common reason for using a strong independent Judge is to provide complementary scrutiny from a perspective different from that of the generator. Why should such differences harm training feedback, and why should shared initialization and coordinated evolution improve it? How can the failure in Figure 1 be distinguished from insufficiently accurate or comprehensive rubrics?

A1. Thank you for highlighting our motivation and the interpretation of Figure 1. We agree that an independent Judge can offer complementary scrutiny, and differences between models are not inherently harmful. Our term “semantic inconsistency” was too broad: it could imply alignment of internal model representations, whereas our concern is how rubric criteria are specified and applied to produce useful rewards as the Policy evolves. We therefore frame the motivation as rubric-feedback coordination across criterion specification, criterion application, and rubric-role adaptation. Figure 1 illustrates a criterion-specification failure; Tables 1 and 2 examine Judge-dependent feedback and its effect on learning, while Table 3 examines continued adaptation.

 

Q1.1 Why is cross-role inconsistency a central problem that needs to be addressed?

A1.1 The concern is not that different models must agree on every caption. Rather, the Judge’s application of a rubric determines the relative rewards used to update the Policy. Even when the Generator specifies the same criterion and pass/fail rule, different Judges may assign different binary decisions to a caption. This can change the caption’s reward relative to other rollouts and, in turn, which outputs GRPO reinforces. To measure how sensitive these signals are to Judge choice, we hold both candidate captions and rubrics fixed and replace only the Judge. The captions come from the Policy after Shared Multi-Role SFT and 300 RL steps, and the rubrics are produced by the shared SFT Generator. Table 1 reports differences relative to the shared SFT Judge.

Table 1. Judge-dependent changes in training feedback.

Replacement Rubric JudgeCriterion-level disagreement rate (%)Mean absolute reward differenceMean absolute GRPO advantage differenceAdvantage sign-flip rate (%)
GPT-5.59.100.07830.538920.95
Gemini 3.1 Pro8.610.07390.508119.55

Replacing the Judge changes 8.61%–9.10% of criterion-level decisions and reverses the signs of nonzero GRPO advantages for 19.55%–20.95% of candidates. Thus, Judge choice changes relative training signals even when rubric content is identical. This comparison measures feedback sensitivity.

 

Q1.2 Why should differences in criterion interpretation and evaluation preferences between the Generator and Judge harm training feedback?

A1.2 Judge-dependent decisions can harm training if they rank lower-quality captions above better ones, because GRPO reinforces captions according to their relative rewards. We therefore test the training consequence rather than assuming that disagreement is harmful. In Table 2, all runs continue the same 300-step Policy checkpoint for 50 steps with the same training images and optimization settings. Each Generator’s rubrics are generated offline and reused unchanged across Judges; only the Policy is optimized.

Table 2. Policy performance under different Generator–Judge configurations.

Rubric GeneratorRubric JudgePixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
Shared starting checkpoint—70.2861.1272.3360.1873.8067.54
SFT (shared)SFT (shared)70.3466.1372.6759.5776.2068.98
SFT (shared)GPT-5.569.0863.9374.0058.7573.4067.83
SFT (shared)Gemini 3.1 Pro70.4861.5272.0060.3673.6067.59
GPT-5.5SFT (shared)69.6464.2372.0059.9773.7067.91
GPT-5.5GPT-5.569.5463.7374.1760.0074.4068.37
Gemini 3.1 ProSFT (shared)70.5862.8372.1760.3974.3068.05
Gemini 3.1 ProGemini 3.1 Pro70.5463.9373.6760.3674.0068.50

With the shared SFT Generator’s rubrics fixed, the shared SFT Judge yields an average Policy win rate of 68.98%, compared with 67.83% for GPT-5.5 and 67.59% for Gemini 3.1 Pro. This shows that criterion application affects subsequent Policy performance without changing rubric content.

AuthorsOpenReview ↗

Response to Reviewer iUjg — Part 2/4

Q1.3 Why should shared initialization and coordinated evolution improve it?

A1.3 Shared Multi-Role SFT gives the Policy, Generator, and Judge a common parameter initialization while teaching their distinct tasks through role-specific supervision. This provides a compatible starting point for transferring Policy updates; it does not assume that the roles subsequently make identical judgments. As the Policy changes, we construct rubrics from its current candidates and gradually transfer its parameter updates to one momentum model used by both the Generator and Judge. The two rubric roles therefore use the same lagged checkpoint within each rollout group, while EMA limits abrupt changes from direct parameter copying. The original matched component ablation supports this design: with shared initialization and online rubrics, joint EMA increases the two-benchmark mean win rate from 66.61% with frozen rubric roles to 72.14%; updating only the Generator reaches 69.07%. We further test continued adaptation from the 300-step Policy checkpoint by comparing ten additional MoCo steps with rubric roles frozen at different earlier checkpoints (Table 3).

Table 3. Continued adaptation versus frozen rubric-role parameters.

Parameter setting for the Rubric Generator and Rubric JudgePixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
Shared starting checkpoint70.2861.1272.3360.1873.8067.54
SFT (shared)69.7662.4074.0861.3674.7568.47
Momentum checkpoint at step 10069.3564.6070.9060.8273.1567.77
Momentum checkpoint at step 20068.7565.2074.0859.2173.7568.20
Momentum checkpoint at step 30072.1863.2074.2560.6474.3568.92
MoCo Rubric72.3865.2072.9162.2574.1569.38

Continued MoCo updates reach 69.38%, compared with 68.92% for the strongest frozen setting in this short continuation experiment. Together, these results indicate the usefulness of a shared starting point and continued joint adaptation.

 

Q1.4 How can the failure in Figure 1 be distinguished from insufficiently accurate or comprehensive rubrics?

A1.4 We agree that Figure 1 does not distinguish cross-role interpretation differences from an insufficiently specific rubric. The original criterion checks whether a caption mentions a grid of cars; it does not require the correct number of columns. The Judge’s pass decision therefore follows the written criterion, while the column-count error goes undetected. We will revise the figure discussion to present this as a criterion-specification failure. Tables 1 and 2 address a separate question by holding rubric content fixed and testing criterion application, while Table 3 examines adaptation as the Policy evolves.

AuthorsOpenReview ↗

Response to Reviewer iUjg — Part 3/4

Q2. Could an additional evaluation step for tied rollout groups yield similar training benefits? The experimental analysis reports fewer rollout groups with identical rewards under the proposed method, allowing more groups to provide relative reward signals. If reducing tied groups is a major source of improvement, could effective training signals be recovered by keeping the Generator and Judge parameters fixed and adding an evaluation step only for tied groups—for example, reconstructing the rubric or applying finer-grained judgments? What additional benefits do joint EMA updates provide beyond this approach?

A2. Thank you for suggesting a targeted reevaluation of tied rollout groups. We agree that when all sixteen candidates receive the same reward, their within-group GRPO advantages vanish, and a more discriminative rubric could recover a training signal. To test this alternative, we keep RubiCap’s Generator and Judge fixed and update only its Policy: whenever a group is tied, we reconstruct its rubric once using the image, two reference captions, original rubric, and the same sixteen candidates, then rescore the group; no further retry is made if it remains tied.

As shown in Table 4, this retry improves RubiCap’s average win rate from 69.93% to 70.84%. It is therefore a useful intervention, although its mean remains below MoCo Rubric’s 72.83% in this comparison.

Table 4. Rubric retry with fixed Generator and Judge parameters.

MethodPixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
RubiCap70.1467.8072.3363.5775.8069.93
RubiCap + rubric retry71.1069.0872.1664.7077.1770.84
MoCo Rubric73.9470.3478.0063.3878.5172.83

The two procedures address different scopes of feedback. A retry acts only when a group is tied, whereas joint EMA updates the Generator and Judge throughout training, including for groups that already have nonzero reward variation. Table 1 shows why the latter groups can still matter: changing only the Judge under a fixed rubric reverses the signs of nonzero advantages for approximately 20% of candidates, although this diagnostic does not determine which Judge is correct. Separately, the matched component ablation in the original paper shows that joint EMA improves the two-benchmark mean from 66.61% with frozen rubric roles to 72.14% under the same shared initialization and online-rubric setting; updating only the Generator reaches 69.07%. Together, these results show that retrying ties can help, while continued, joint rubric-role adaptation has benefits in the evaluated setting that are broader than the tested tie-only procedure.

 

Q3. Why is the Judge configured without direct access to the image? Section 3.3 specifies that the Judge evaluates candidate captions using only the captions and rubrics, without directly accessing the image. Normally, a Judge considers the task input when assessing whether an output satisfies the task requirements. For image captioning, the original image is essential for determining factual accuracy and completeness. Why does the Judge obtain image information only indirectly through the Generator’s rubric, rather than inspecting the image directly? What motivates this input restriction?

A3. Thank you for emphasizing the importance of the image in assessing caption factuality and completeness. The image is used in our reward pipeline: the Policy sees it when generating captions, and the Generator examines it when constructing image-specific criteria from the image, reference captions, and current Policy candidates. We give the Judge the narrower task of checking each caption against those written criteria, so that rewards reflect one explicit rubric shared by the rollout group rather than an additional, implicit visual assessment by the Judge. This division follows RubiCap’s text-only judging setup (Huang et al., 2026, Appendix G) and the self-contained evaluation principle of Rubrics as Rewards (Gunjal et al., 2026, Sec. 3.1). We agree that this design cannot detect a visual error omitted from the rubric; we will make this limitation explicit when describing the Judge’s input.

Reviewer_HDitOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a Rubric-Based Reinforcement Learning (RL) approach for image captioning. Specifically, the authors propose to use a unified backbone for the three main roles of Rubric-Based RL, i.e., the Caption Policy, which generates the captions, the Rubric Generator, which generates the caption assessment criteria, and the Rubric Judge, which evaluates the captions produced by the Policy using the criteria provided by the Generator. The roles are assigned using specific prompts, and using a common backbone helps in aligning the three roles. Training is based on two stages. The first stage is standard SFT, while the second is based on RL. Specifically, in the RL stage, only the Policy is directly optimized (using GRPO), while the parameters of both the Generator and the Judge are updated using EMA.

Critical strengths

The proposed method seems to be original.

The results are good, showing that the proposed approach clearly outperforms the compared methods, including other recent rubric-based approaches.

Critical weaknesses
  1. There are some aspects of the method that are not very clear—especially for a reader like me, who is unfamiliar with rubric-based methods—and these should be better clarified. For example:
  • If the Generator is able to establish effective criteria for evaluating the captions produced by the policy, what need is there for the Judge? Why we cannot use the Generator to directly produce the rewards? More generally, what is the advantage of having three distinct roles, as opposed to using only the Policy?
  • What are the reference captions C^ref (line 247) and how are they selected?
  • Why you use a random subset S_t rather than the full Y_t in line 246?
  • Why the Judge has no access to the image?
  • Since the rewards are produced by the Judge, how do you avoid a Judge-Generator-Policy collapse? For instance, the Generator could produce trivial criteria, which are always satisfied by the Judge, and then, the rewards used for the Policy update will not be significant…
  1. In the experiments, it would be interesting to include some RL based captioning methods as those mentioned in Sec. 2 (e.g., (Xing et al., 2026), (Yang et al., 2026a), (Tang et al., 2026), (Ye et al., 2026)), possibly using GRPO as the optimization objective. This would help to understand the contribution of a rubric-based approach compared to using GRPO with, for instance, rewards derived from question-answering utility or other means.
Questions for authors

Please, see the Critical Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer HDit — Part 1/3

Thank you for identifying the methodological details that need clarification.

Q1.1. If the Generator is able to establish effective criteria for evaluating the captions produced by the policy, what need is there for the Judge? Why we cannot use the Generator to directly produce the rewards? More generally, what is the advantage of having three distinct roles, as opposed to using only the Policy?

A1.1 Thanks for the insightful comment. The reward computation proceeds through three distinct functions: the Policy generates candidate captions, the Generator constructs a common rubric, and the Judge makes binary decisions for each caption and criterion, which are then aggregated into weighted rewards. A possible simplification as suggested is to merge rubric construction and application into one reward-producing operation, asking the Generator to score each caption directly or prompting the Policy to score its own outputs.

The differences between our three-role design and the two-role direct-scoring alternative described above lie in how rewards are constructed. In our design, the Generator constructs one rubric from the image, reference captions, and sampled Policy candidates; the Judge then applies that rubric to every caption in the rollout group, and a fixed weighted rule aggregates its binary criterion-level decisions into rewards. In the direct-scoring alternative, the Generator instead returns an overall score for each caption, leaving the individual criterion decisions and their aggregation implicit. Our design thus gives within-group comparisons a stated common standard and makes each reward traceable to specific criteria.

RubiCap provides supporting evidence for this distinction. In its 7B GRPO experiments, Direct-Likert assigns an overall score without a reference caption, while Reference-Likert includes a reference caption but still assigns an overall score rather than explicit criterion-level decisions. With the same base captioner trained on 50,000 images per dataset, rubric-based rewards achieve CapArena win rates of 70.8% and 64.4% on PixMo-Cap and DenseFusion, compared with 66.2% and 59.2% for Direct-Likert and 67.2% and 50.6% for Reference-Likert. These results suggest that providing a reference for holistic scoring does not reproduce the benefit of explicit rubric-based feedback in that setting; because the reward providers also differ, they support the structured reward design rather than establish that three functional roles are necessary (Huang et al., 2026, Sec. 3.1 and Appendix H, Table 2).

 

Q1.2. What are the reference captions C^ref (line 247) and how are they selected?

A1.2 Thank you for asking us to clarify how the reference captions are constructed. CrefC^{\mathrm{ref}} contains two model-generated descriptions of each training image; they provide the Generator with detail cues beyond the sampled Policy captions, while the image remains the factual basis for the rubric. In our implementation, we use the same captioning prompt and procedure as for the Policy to generate one caption with GPT-5.5 and one with Gemini 3.1 Pro, and supply both to the Generator.

AuthorsOpenReview ↗

Response to Reviewer HDit — Part 2/3

Q1.3. Why you use a random subset S_t rather than the full Y_t in line 246?

A1.3 Thank you for asking how we choose the candidates used to construct each rubric. The subset StS_t gives the Generator examples of the current Policy’s captions to compare for image-verifiable errors, omissions, and differences in specificity; we sample uniformly from the N=16N=16 rollouts, while the image and two reference captions remain available in every setting. Using M=4M=4 provides comparative examples with a shorter input, whereas M=16M=16 exposes the Generator to every rollout but substantially lengthens its input; the effect of that additional context is an empirical question.

Table 1 compares different subset sizes. Every setting retains the image and two reference captions, and all sixteen rollouts are scored against the resulting rubric.

Number of candidate captions (MM)PixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
062.7555.1168.5048.8369.2260.88
268.8367.9473.0060.3272.4368.50
473.8970.3478.0063.3878.4772.81
870.8567.5474.6761.4075.6570.02
1669.4367.7478.5059.7874.2569.94

We conducted an additional ablation varying the number of current-Policy captions supplied to the Rubric Generator (M∈{0,2,4,8,16}M \in \{0,2,4,8,16\}); every setting retains the image and two reference captions, and scores all N=16N=16 rollouts against the resulting rubric. The results are reported in Table 1. Table 1 shows that the average win rate peaks at M=4M=4: 72.81%, compared with 69.94% when all sixteen candidates are used and 60.88% when none are provided. These results indicate including current-Policy examples while showing that more inputs do not necessarily improve performance, motivating our choice of M=4.

 

Q1.4. Why the Judge has no access to the image?

A1.4 Thank you for asking us to clarify the Judge’s input. The Policy uses the image to generate captions, and the Generator uses the image to formulate explicit, image-grounded rubric criteria; the Judge then checks whether each caption satisfies those criteria using the caption and rubric alone, rather than independently verifying the image. This follows RubiCap’s text-only judging setup (Huang et al., 2026, Appendix G) and the self-contained evaluation principle of Rubrics as Rewards (Gunjal et al., 2026, Sec. 3.1), although it means the Judge cannot detect visual errors that the rubric fails to capture.

 

Q1.5. Since the rewards are produced by the Judge, how do you avoid a Judge-Generator-Policy collapse? For instance, the Generator could produce trivial criteria, which are always satisfied by the Judge, and then, the rewards used for the Policy update will not be significant…

A1.5 Thank you for highlighting the risk of a self-reinforcing reward loop. Such a collapse could occur if the Generator produces trivial criteria, the Judge accepts captions indiscriminately, and the Policy consequently receives uninformative rewards. To limit this risk, Shared Multi-Role SFT supervises rubric construction and criterion-level judging as distinct tasks; the Generator prompt also requires image-verifiable distinctions and discourages easy-to-pass criteria, while the Judge must provide a rationale for each binary decision. During RL, only the Policy receives reward-driven gradients, whereas the Generator and Judge receive gradual EMA updates rather than being directly optimized to increase rewards.

If the Generator routinely produced trivial criteria that the Judge marked as satisfied for every caption, all rollouts in a group would receive the same reward, yielding zero within-group variance and no relative GRPO advantage. We examine this failure mode in Appendix B.3: only 0.46% of MoCo Rubric groups have zero reward variance, compared with 3.59% for RubiCap, indicating that most observed groups retain discriminative training signals.

AuthorsOpenReview ↗

Response to Reviewer HDit — Part 3/3

Q2. In the experiments, it would be interesting to include some RL based captioning methods as those mentioned in Sec. 2 (e.g., (Xing et al., 2026), (Yang et al., 2026a), (Tang et al., 2026), (Ye et al., 2026)), possibly using GRPO as the optimization objective. This would help to understand the contribution of a rubric-based approach compared to using GRPO with, for instance, rewards derived from question-answering utility or other means.

A2. Thank you for suggesting a comparison with RL captioning methods that use alternative rewards. Our method optimizes the Caption Policy with GRPO using weighted rewards from image-specific rubric criteria and binary criterion-level judgments. In contrast, CapRL and CapRL++ reward captions according to how accurately a vision-free model answers image-related multiple-choice questions from the captions; CCCaption uses visual-query coverage and hallucination checks, while BalCapRL combines utility-aware correctness, reference coverage, and linguistic quality (Xing et al., 2026; Yang et al., 2026a; Tang et al., 2026; Ye et al., 2026).

Reproducing these methods with our Policy checkpoint and training images would require constructing their respective reward inputs and verification pipelines, such as questions with answers or generated visual queries; BalCapRL also uses a different reward-normalization strategy. We therefore evaluate the released CapRL-InternVL-8B and CapRL-Qwen3VL-4B checkpoints under our common caption-evaluation protocols, adding both to the expanded caption-based question-answering comparison in Table 2.

Table 2. Expanded comparison on caption-based question answering.

MethodCaptionQABLINKTextVQADocVQAChartQAAverage
Base80.4147.0558.0069.1861.2863.18
ShareGPT4V-7B50.8243.3030.5714.838.6829.64
RICO-Flash-7B70.7243.1152.5865.5957.8057.96
OmniCaptioner-8B65.8342.4348.5350.6316.3644.76
JoyCaption-8B58.4245.1531.7716.119.6032.21
MetaCaptioner-8B71.3244.7253.1359.4858.7657.48
CapRL-InternVL-8B78.8248.6254.7271.4061.7663.06
CapRL-Qwen3VL-4B85.3751.0057.0875.2862.5666.26
Qwen3-VL-32B82.9250.9456.6975.0662.1665.55
RubiCap83.2849.0556.8577.9064.4466.30
EvoLM82.0049.0057.0877.2162.4465.55
SFT (shared)81.0547.6956.4476.1363.1664.89
MoCo Rubric84.2450.1058.3379.0564.1267.17

MoCo Rubric achieves the highest five-task average in Table 2 at 67.17%, compared with 66.26% for CapRL-Qwen3VL-4B and 63.06% for CapRL-InternVL-8B. The Qwen3VL-4B variant performs better on CaptionQA and BLINK, while MoCo Rubric performs better on TextVQA, DocVQA, and ChartQA. Together with the caption-quality comparison in the original Table 1, these results show that rubric-based GRPO achieves strong caption quality and downstream utility relative to the evaluated RL-trained models.

Reviewer_r2FoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper improves detailed image captioning through shared multi-role supervised fine-tuning, followed by GRPO optimization of only the caption policy. Online rubrics are constructed from current candidates, reference captions, and images. the generator and judge share a momentum model updated through an exponential moving average of policy parameters, aiming to coordinate rubric generation and judging while smoothing updates Across five benchmarks, the framework achieves a 72.83% average pairwise win rate against Qwen3-VL-8B-Instruct, exceeding RubiCap’s corresponding rate by 2.90 percentage points

Critical strengths
  1. The paper is clearly written and well organized. The motivation and proposed framework are presented in a straightforward manner
  2. The experimental evaluation is fairly comprehensive, covering multiple benchmarks
Critical weaknesses
  1. I am not convinced that the proposed strategy effectively addresses the stated problem. The underlying issue appears to be poorly designed rubrics rather than inconsistent interpretations across roles, which calls into question whether the approach targets the actual source of the problem. The authors have not demonstrated that the performance gains arise from reducing semantic inconsistency across roles rather than improving the specificity or coverage of the rubrics.

  2. The experimental comparisons are incomplete. In particular, why are the CapRL models omitted from the comparison in Table 2?

Questions for authors

None

5 comments
AuthorsOpenReview ↗

Response to Reviewer r2Fo — Part 1/2

Thank you for the questions about our motivation and the completeness of the comparisons.

Q1. The underlying issue appears to be poorly designed rubrics rather than inconsistent interpretations across roles, which calls into question whether the approach targets the actual source of the problem. The authors have not demonstrated that the performance gains arise from reducing semantic inconsistency across roles rather than improving the specificity or coverage of the rubrics.

A1. Thank you for this helpful comment on our motivation. Rubric specificity and coverage may contribute to performance, alongside how those criteria are applied during training. Our term “semantic inconsistency” was too broad, as it could also refer to alignment of internal model representations. Our concern is how image-specific criteria capture useful distinctions, how the Judge applies them to produce relative rewards, and how the rubric roles adapt as the Policy evolves. We now frame the motivation as rubric-feedback coordination across criterion specification, criterion application, and rubric-role adaptation. To examine these aspects separately, Tables 1 and 2 hold rubric content fixed while varying the Judge, and Table 3 examines continued rubric-role adaptation.

(1) Criterion application under fixed rubric content. We first examine whether training feedback depends on criterion application when rubric specificity and coverage are held fixed. We use captions generated by the Policy after Shared Multi-Role SFT and 300 RL steps, and rubrics generated by the shared SFT Generator. Holding both captions and rubrics unchanged, we replace only the Judge and compare its outputs with those of the shared SFT Judge.

Table 1. Judge-dependent changes in training feedback.

Replacement Rubric JudgeCriterion-level disagreement rate (%)Mean absolute reward differenceMean absolute GRPO advantage differenceAdvantage sign-flip rate (%)
GPT-5.59.100.07830.538920.95
Gemini 3.1 Pro8.610.07390.508119.55

Rewards are weighted means of binary criterion decisions, and GRPO advantages standardize those rewards within each image group. Table 1 shows 8.61%–9.10% criterion-level disagreement and 19.55%–20.95% opposite nonzero advantage signs when the Judge is replaced. Thus, even with identical criteria, Judge choice can change which captions receive positive or negative training signals.

(2) Consequences for Policy learning under fixed rubrics We next test whether the Judge-dependent feedback observed in Table 1 has consequences for Policy learning. All runs continue the same 300-step Policy checkpoint for 50 steps with the same training images and optimization settings. Each Generator’s rubrics are produced offline and reused unchanged across its Judge comparisons, and only the Policy is updated. Comparing Judges under the same Generator therefore keeps rubric content fixed during this test.

Table 2. Policy performance under different Generator–Judge configurations.

Rubric GeneratorRubric JudgePixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
Shared starting checkpoint—70.2861.1272.3360.1873.8067.54
SFT (shared)SFT (shared)70.3466.1372.6759.5776.2068.98
SFT (shared)GPT-5.569.0863.9374.0058.7573.4067.83
SFT (shared)Gemini 3.1 Pro70.4861.5272.0060.3673.6067.59
GPT-5.5SFT (shared)69.6464.2372.0059.9773.7067.91
GPT-5.5GPT-5.569.5463.7374.1760.0074.4068.37
Gemini 3.1 ProSFT (shared)70.5862.8372.1760.3974.3068.05
Gemini 3.1 ProGemini 3.1 Pro70.5463.9373.6760.3674.0068.50

With rubrics from the shared SFT Generator held fixed, the shared SFT Judge yields an average Policy win rate of 68.98%, compared with 67.83% for GPT-5.5 and 67.59% for Gemini 3.1 Pro. This shows that Judge choice affects subsequent Policy performance even when rubric specificity and coverage are unchanged.

AuthorsOpenReview ↗

Response to Reviewer r2Fo — Part 2/2

(3) Continued adaptation as the Policy evolves Finally, criterion construction and application must remain useful as the Policy’s captions change during training. Shared Multi-Role SFT provides a common, role-capable starting point, and Momentum Transfer gradually updates the Generator and Judge together as the Policy evolves. We test the value of continued updating by starting from the same 300-step Policy checkpoint and comparing ten additional MoCo updates with runs whose rubric roles are frozen at different SFT or momentum checkpoints.

Table 3. Continued adaptation versus frozen rubric-role parameters.

Parameter setting for the Rubric Generator and Rubric JudgePixMo-CapDenseFusionCapArenaCompreCapDOCCIAverage win rate (%)
Shared starting checkpoint70.2861.1272.3360.1873.8067.54
SFT (shared)69.7662.4074.0861.3674.7568.47
Momentum checkpoint at step 10069.3564.6070.9060.8273.1567.77
Momentum checkpoint at step 20068.7565.2074.0859.2173.7568.20
Momentum checkpoint at step 30072.1863.2074.2560.6474.3568.92
MoCo Rubric72.3865.2072.9162.2574.1569.38

Continued MoCo updates achieve an average win rate of 69.38%, compared with 68.92% for the strongest frozen setting in this ten-step continuation. This supports continued rubric-role adaptation in the tested setting.

 

Q2. The experimental comparisons are incomplete. In particular, why are the CapRL models omitted from the comparison in Table 2?

A2. Thank you for pointing out this omission. The original Table 2 prioritized rubric-based methods and core controls under space constraints, but leaving out CapRL made the downstream comparison with alternative RL rewards incomplete. We have now evaluated both CapRL-InternVL-8B and CapRL-Qwen3VL-4B using the same caption-based question-answering protocol as the other methods: each model generates a caption without seeing the question, and the same fixed text-only answering model uses that caption as its sole visual input. The expanded results are reported in Table 4.

Table 4. Expanded comparison on caption-based question answering.

MethodCaptionQABLINKTextVQADocVQAChartQAAverage
Base80.4147.0558.0069.1861.2863.18
ShareGPT4V-7B50.8243.3030.5714.838.6829.64
RICO-Flash-7B70.7243.1152.5865.5957.8057.96
OmniCaptioner-8B65.8342.4348.5350.6316.3644.76
JoyCaption-8B58.4245.1531.7716.119.6032.21
MetaCaptioner-8B71.3244.7253.1359.4858.7657.48
CapRL-InternVL-8B78.8248.6254.7271.4061.7663.06
CapRL-Qwen3VL-4B85.3751.0057.0875.2862.5666.26
Qwen3-VL-32B82.9250.9456.6975.0662.1665.55
RubiCap83.2849.0556.8577.9064.4466.30
EvoLM82.0049.0057.0877.2162.4465.55
SFT (shared)81.0547.6956.4476.1363.1664.89
MoCo Rubric84.2450.1058.3379.0564.1267.17

MoCo Rubric achieves the highest five-task average of 67.17%, compared with 66.26% for CapRL-Qwen3VL-4B and 63.06% for CapRL-InternVL-8B. CapRL-Qwen3VL-4B performs better on CaptionQA and BLINK, while MoCo Rubric performs better on TextVQA, DocVQA, and ChartQA. We will include the full comparison in the revised paper.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer,

Thank you for your careful review of our manuscript. We would appreciate your feedback on whether the additional experiments and clarifications address your main concerns. Should these findings change your assessment, we would be grateful if you would consider updating your rating accordingly.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank the Reviewers for their careful reading and constructive comments. In response, we have revised the manuscript and strengthened both the technical presentation and experimental validation. Here we summarize the important changes as follows:

(1)Motivation and training feedback (R1-Q1 and R3-Q1). Our term “semantic inconsistency” was too broad, as it could also refer to alignment of internal model representations. We now frame the motivation more precisely as rubric-feedback coordination across criterion specification, criterion application, and adaptation as the Policy evolves. We clarified Figure 1 as an illustration of the importance of precise rubric criteria and added fixed-rubric experiments showing how Judge choice changes GRPO training signals and subsequent Policy performance (Tables 1–2 in our responses to R1-Q1 and R3-Q1). A continuation experiment further evaluates joint rubric-role adaptation against frozen checkpoints (Table 3 in our responses to R1-Q1 and R3-Q1).

(2)Completeness of the comparisons (R1-Q2 and R2-Q2). We provide caption-based question-answering results for comparison methods, including the CapRL variants trained with question-answering utility rewards.

(3)Methodological choices and the retry alternative. We examine the number of candidates used for rubric construction (R2-Q1.3 and R3-Q4) and test whether retrying tied rollout groups with fixed rubric models can provide similar benefits to MoCo Rubric (R3-Q2). We also clarify the role definitions, reference captions, image inputs, and reward-collapse diagnostic (R2-Q1 and R3-Q3).

AuthorsOpenReview ↗

Reminder to reply

Dear Reviewer, we would be grateful if you could let us know whether our response addresses your concerns or whether any points would benefit from further clarification before the discussion closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.