- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SPROUT, a training framework for improving the faithfulness of detailed image captions generated by masked diffusion vision-language models. The authors construct five levels of caption complexity from DOCCI through progressive simplification and assign each word a tier. A tier-dependent masking schedule and corresponding loss weighting shift supervision of detail tokens toward contexts in which the ground-truth sentence skeleton is more likely to be visible. The method also incorporates same-tier block masking and termination supervision, while retaining the backbone’s original confidence-based decoding procedure. Experiments on LLaDA-V compare SPROUT with the pretrained backbone, same-data uniform SFT, data-ordering curricula, and several masking and tier-construction variants. The results show improvements on multiple hallucination and detailed-captioning metrics. An analysis of token commitment order also shows a stronger skeleton-before-detail tendency after training.
Critical strengths
-
SPROUT makes effective use of the partially observed contexts available in masked diffusion training. Proposition 1 distinguishes supervision allocation from simply reducing the loss weight on detail tokens: the integrated coefficient is preserved while the surrounding context changes.
-
Relative to same-data uniform SFT, SPROUT improves AMBER coverage from 59.8 to 67.2 and THRONE F1 from 72.0 to 75.9, while reducing OpenCHAIR from 13.7 to 10.5 (Table 1). This supports a contribution from the schedule beyond block masking alone. The caption-quality results in Table 2 provide complementary evidence.
Critical weaknesses
-
Proposition 1 establishes greater visibility of the ground-truth skeleton during training. It does not establish increased reliance on visual evidence or ensure that the skeleton is correct during free generation. Similarly, Figure 4 shows average commitment order across tiers, but does not by itself establish that generating the skeleton earlier causes fewer hallucinations in subsequent details. The paper lists tier-stratified object- and attribute-level hallucination evaluation among its contributions, yet I could not find the corresponding error rates, coverage measurements, or uncertainty estimates in the supplied manuscript. The reversed-schedule control mainly reports commitment order rather than the associated faithfulness–coverage trade-off. This leaves the connection between the observed ordering and the performance gains unresolved.
-
The hallucination improvements are also not uniform: in Table 1, SPROUT has a CHAIR_s of 28.0, compared with 18.8 for uniform SFT. Overall, the evidence more strongly supports improvements on several aggregate metrics than the proposed fine-grained mechanism.
-
Severe degradation in several ablations remains insufficiently explained, and important controls and implementation details are missing. In Table 4, combining the tier-conditioned schedule with per-token masking reduces AMBER coverage to 22.5 and the average object count to 0.7. Shuffled, frequency-based, and POS-based tiers also lead to substantial under-description. These results reveal sensitivity to the design choices, but describing them as evidence of strong component coupling does not rule out premature termination, optimization instability, or configuration effects.
-
Section 3.2 states that termination supervision is ablated separately, but I could not locate that result. It is also unclear whether uniform SFT uses exactly the same termination-token filling and training canvas.
Questions for authors
-
Could you provide the tier-stratified object and attribute hallucination evaluation described in the paper, including coverage, for SPROUT, uniform scheduling with block masking, and the reversed schedule? How are tiers assigned to generated captions in Figure 4, how many examples are evaluated, and how stable are the differences in commitment order?
-
Do all controlled variants in Tables 1, 4, and 5 share the same termination-token filling, loss normalization, training canvas, and decoding settings? What are the results of the separate termination-supervision ablation mentioned in Section 3.2? For the severely degraded variants, do output lengths and premature-termination rates explain the behavior, and does it recur across random seeds?
-
The reverse transition in Appendix A.1 uses tier-dependent (r_{k,i}), and joint unmasking depends on the span partition. Both are derived from the target caption. Should Equation (4) therefore be understood as a NELBO for a model conditioned on these tiers and partitions? If it is intended to describe a model conditioned only on the image and prompt, how are the additional variables defined or marginalized?