- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RUNG, an ordered visual code that places caption-relevant information in early prefixes and reserves later rungs for image detail. It frames this design as successive refinement under a distortion schedule that shifts from caption supervision to pixel reconstruction. The theoretical results characterize when nested caption stages can be rate-optimal, illustrate how pixel-first ordering can omit caption information at low rates, and bound the pixel-rate cost of words-first ordering. Experimentally, the paper reports stronger prefix–caption alignment and text-to-image compositional performance than matched reconstruction-ordered alternatives, with little reconstruction loss at full code length.
Critical strengths
- The idea of explicitly choosing what early visual-code prefixes represent is compelling.
- The theory motivates the successive-summary caption ladder, and the predicted fidelity trade-off is tested experimentally.
- The controlled ordering comparison retrains models under matched architectures, data, and capacity, supporting the reported GenEval gain from 0.76 to 0.83. The evaluation covers generation, caption consistency, reconstruction, editing, and correction.
Critical weaknesses
-
Appendix J uses a language-model cross-entropy in a Barber–Agakov-style estimate. Since can exceed , the result is not necessarily a mutual-information lower bound. The relative ranking is supported by the InfoNCE check, but Figure 2a’s absolute values and the 2.9× ratio need more careful qualification.
-
Proposition 3 concerns optimal information rates, whereas Figure 2c uses learned reconstructions and sums of marginal FSQ entropies. The plotted estimated “lower bound” is not guaranteed to be a lower bound either. These curves are useful diagnostics, but should not be presented as a direct verification of the proposition.
-
LadderCap deliberately puts counts and spatial relations in short captions—precisely the attributes emphasized by GenEval. Existing ablations test how the ladder is constructed, but not which facts are prioritized. A matched successive-summary ablation with different early-caption priorities would help isolate this choice.
-
The paper reports latency and compares against best-of- with the same verifier, but its network-evaluation counts combine operations with different costs. A verifier-cost breakdown and a latency-matched best-of- comparison would strengthen the claim.
Questions for authors
-
The contribution of the proposed ordering is not sufficiently isolated. In Table 1, the words-first model receives caption supervision, while the pixel-first model does not. The improvement may therefore reflect both the additional language supervision and the ordering of information within the code. Could the authors include a comparison using the same caption data and supervision budget, with the language loss applied only to the full code? This would help clarify the additional benefit of supervising prefixes at different lengths.
-
The interpretation of the information estimate needs clarification. Appendix J uses a language model’s cross-entropy to estimate caption entropy. Since cross-entropy generally upper-bounds the true entropy, substituting it into the expression does not by itself preserve the mutual-information lower bound. Could the authors clarify the justification for calling this estimate a lower bound?
-
The effectiveness of prefix-level verification needs further analysis. The correction procedure checks attributes decoded from an intermediate prefix, but the final image is generated after additional blocks are sampled. How often does a prefix pass verification while the final image still violates the prompt? It would be useful to distinguish errors in the attribute checker from changes introduced during later generation. The authors should also clarify whether the 80% agreement threshold can allow important counting or spatial-relation errors to pass.