- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies reinforcement learning for detailed whole-image captioning. Completeness queries (CQs) are constructed offline and answered from the caption alone to assess information coverage. The current caption is also decomposed into faithfulness queries (FQs), which are checked against the image online. The two query types are sampled under a fixed budget. After filtering out full-score FQs, the remaining scores are aggregated into a reward used to train Qwen3.5-4B with GRPO. The paper introduces the DQ-CAP-40k training set and DQ-Bench evaluation set, and reports results on eight PRISM question-answering evaluations, field-level diagnostics, and component ablations.
Critical strengths
- The supervision addresses both omissions and factual errors. Hard CQs target content that the base model omits or misstates, while online FQs check factual claims in the current caption. These signals are complementary, and the fixed query budget helps control the cost of each evaluation. The design addresses a meaningful tension between coverage and accuracy in detailed captioning.
- Two complementary evaluations show improvements. Relative to Qwen3.5-4B, the average PRISM score increases from 61.7 to 63.1. On DQ-Bench, Correct rises from 85.5% to 94.1%, Omit falls from 9.4% to 2.1%, and Wrong falls from 5.1% to 3.8%. The final diagnostic judge differs from the training verifier. These results provide empirical support for better coverage and error control.
Critical weaknesses
- The reward can penalize a factual correction. Eqs. (7) and (8) remove full-score FQs from both the numerator and denominator. With the default , consider six CQs, of which two are answered correctly, and seven FQs, of which six receive full scores and one receives . The reward is then . Correcting only the last FQ to obtain a full score, while holding the CQ outcomes and decomposition fixed, reduces the reward to . This is a counterexample to the stated formula, not an observed training failure. The aggregate improvement in Table 4 does not rule out this defect in the objective. The implementation and its practical impact need to be checked.
- The contribution of the new design choices is not sufficiently isolated. CCCaption already combines completeness-oriented question answering, factual verification, and dynamic sampling. Table 1 uses results from that paper without matching the base model or training conditions. The ablations also do not separately test hard CQs and decomposition granularity, making their individual contributions difficult to assess.
- The evidence from DQ-Bench also needs further validation. Each field receives a single label, but the rules for fields containing multiple facts are not fully specified. Wrong is a field-level proportion, not a per-fact error rate. Table 5 examines changes in the base model's scores across judges rather than the stability of method rankings. Using Gemini consistently in the main evaluation is a reasonable control, but agreement between the field labels and human judgments has not been established. CAPability's human annotations of visual elements could provide an additional check
Questions for authors
- Reward implementation: Does Eq. (8) match the implementation? How often does the counterexample above arise in actual rollouts? How is the reward defined in the FQ-only setting when every FQ receives a full score? How are the null scores allowed in the appendix handled?
- Supervision reliability: Have you checked whether the decomposer omits erroneous details? How well do FQ verification scores agree with human judgments? Do the four rollouts for the same image share CQs to reduce the effect of query difficulty on within-group rankings?
- Controlled comparison: Is there a dual-reward control with the same base model, query pool, and budget? Can the available results distinguish the contributions of hard CQs, decomposition granularity, and random mixing?
- Evaluation validity: How is a field labeled when it contains correct facts, omissions, and errors together? Are human checks or cross-judge method rankings available? Was the configuration in Table 6 selected on data separate from the final test set?
2 comments
-
For human verification of the automated evaluation, we report the detailed results of a blinded study in our response to Reviewer oNKa, Comment 2. We randomly sampled images from the evaluation set and independently annotated all valid attribute-level records. The Cohen's values between the human reference labels and Gemini-3.1-Pro, Seed-2.0-Lite, and Qwen3.5-9B are 0.7730, 0.7195, and 0.1017, respectively. Gemini-3.1-Pro achieves the highest observed agreement and Cohen's , indicating substantial agreement between its field-level evaluations and human judgments and supporting its use as our primary evaluator.
As a complementary validation independent of DQ-Bench, we additionally evaluate the models using the human-annotated visual elements provided by CAPability. The Qwen3.5-4B baseline obtains an average F1 score of 72.7, whereas DQ-RL achieves an average F1 score of 74.4, corresponding to an improvement of 1.7 points. This result shows that the improvement of DQ-RL over the same base model is not limited to the automated DQ-Bench evaluation and is also observed under an external evaluation based on human-annotated visual elements. We will add the human-verification protocol, agreement results, and complementary CAPability evaluation to the revision.
Thank you for asking about the reward implementation details. We provide a detailed response regarding whether Eq. (8) matches the implementation, the empirical frequency of the non-monotonic counterexample, and the all-full-score case in the FQ-only setting in our response to Reviewer exDs, Comment 1. Briefly, the boundary-case description following Eq. (8) omitted the all-correct fallback used in the implementation. When all FQs receive full scores, full-score exclusion is disabled for that rollout, and all FQs are retained in both the numerator and denominator. Consequently, an all-correct rollout in the FQ-only setting receives reward α, and no 0/0 case occurs. We also quantify the frequency of the non-monotonic counterexample in actual rollouts and will correct the description and add the implementation details and statistics in the revision. A
nullscore allowed by the appendix prompt indicates that the corresponding FQ cannot be reliably judged from the image; it is not treated as a numerical faithfulness score of zero. We therefore neither convertnullto zero nor include it in reward aggregation. A rollout containing anullscore or another invalid reward output is marked as invalid and excluded from subsequent training updates. Our training statistics show a valid reward-data rate of 99.68%, with only 0.32% invalid data. The invalid rate coversnullscores together with other reward-computation errors. Therefore, 0.32% is the overall invalid reward rate and an upper bound on the incidence ofnullscores. We will explicitly document the handling ofnullscores and the resulting valid-data rate in the revision.For FQ verification, we selected 100 FQs and manually rescored them using the same five-level criteria as the automated verifier, {0,0.25,0.50,0.75,1.00}. The mean automatic and human scores are 0.7450 and 0.8175, respectively. The mean signed bias is −0.0725, indicating that the automated verifier is slightly more conservative. The mean absolute error is only 0.0875, which is smaller than the score-bin interval of 0.25. Moreover, the exact score agreement is 88.00%, and Cohen's κ is 0.7477. These results demonstrate substantial agreement between the automated FQ verifier and human judgments, supporting its use as an automated reward judge during training. We will add this human-verification protocol and its results to the revision.
Regarding the contribution of Hard CQ, we compared the model performance with and without Hard CQ, and the results are shown in the table. After introducing Hard CQ, Correct increases from 89.7% to 92.2%, an improvement of 2.5 percentage points, while Wrong decreases from 8.8% to 5.9%, and Omit remains basically unchanged. It can be seen that Hard CQ can bring an improvement in Correct. For the contribution of random mixing, we supplemented an ablation experiment with a fixed CQ/FQ ratio. The experimental results show that, under the same query budget and CQ/FQ configuration, random mixing compared with fixed-quota sampling raises Correct from 93.6% to 94.1% and reduces Wrong from 4.7% to 3.8%. This shows that random mixing can more fully utilize the supervision signals of the two types of queries, further improving the overall caption quality and factual reliability. The framework also allows adjusting λ according to application needs: increasing λ increases the sampling frequency of CQ, placing greater emphasis on completeness, while decreasing λ increases the sampling frequency of FQ, placing greater emphasis on factual accuracy. The parameter α independently adjusts the weight of FQ signals in the aggregated reward. Therefore, we emphasize that DQ-RL focuses on unified sampling . By contrast, CCCaption follows an idea similar to DAPO, performing online data filtering after each epoch, DQ-RL separately designs its reward and excludes full-score FQs to increase the discriminability of the resulting reward signals. Since CCCaption has not released the relevant code or weights, we cannot make a direct and fair comparison. | Settings | Correct↑ | Omit↓ | Wrong↓ | |---|---|---|---| | without Hard CQ | 89.7% | 1.5% | 8.8% | | Hard CQ | 92.2% | 1.9% | 5.9% | | without random | 93.6% | 1.7% | 4.7% | | random mixing | 94.1% | 2.1% | 3.8% |