acceptodds
Under review as a conference paper at ICLR 2027

ZoomCap: Evidence-Routed Visual Zooming for Recoverable Visual Information Transfer

Abstract

Detailed captions transfer visual information into vision-language model supervision, but quality is easily conflated with length: long single-pass outputs still miss small objects, text, and spatial relations or promote uncertain observations into confident claims. We formulate fine-grained captioning as recoverable visual information transfer, in which a teacher acquires missing evidence under a finite observation budget and converts it into reusable supervision. ZoomCap uses a draft caption to route conditional knowledge retrieval, OCR-sensitive reading, coverage-gap recovery, and adaptive visual zooming, allocating deeper crops only to the most complex eligible region, deriving spatial relations from grounded geometry, and fusing non-duplicate evidence. We also introduce FactCycle, which scores caption-derived atomic claims on two blind anchors–source image (truthfulness) and reconstruction (realizability)–separating fabrication from generator loss claim by claim; human and multi-judge comparisons complement this fact-level view. Against Qwen3.5-397B-A17B captions, ZoomCap achieves an 81.9% human tie-adjusted preference score with a teacher over 10x smaller, and FactCycle counts 52.4 transferred claims per image vs. 31.5 for ScaleCap. The evidence also transfers: in continual pre-training, ZoomCap-450K improves the 11-benchmark average by 1.31-2.38 points over the strongest competing corpus and Ling-Tiny-VL-v3 gains 2.4 points on a production monitor suite at 60B tokens; in medical pre-training, 2.64M ZoomCap-captioned images improve five VQA benchmarks by 3.04 points (vs. 1.76 for same-image Gemini-3.1-Pro captions).

Primary area:
foundation or frontier models, including LLMs
Keywords:
image captioning, multimodal large language models, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_CeooOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ZoomCap, which uses an initial image caption to guide retrieval, OCR, and local zooming, then incorporates geometric constraints to generate detailed captions. It also introduces FactCycle, which evaluates factuality and recoverable information through atomic claims assessed against the original and reconstructed images. Experiments cover caption quality, human preferences, ablations, and downstream training across multiple backbones and application settings.

Critical strengths
  • The overall effectiveness is supported by several types of evaluation. The paper combines automatic evaluation, human preferences, and downstream training, reporting gains across three backbones and both general and medical domains. These results provide useful evidence for the practical value of the generated captions.
  • The approach to evidence acquisition and budget control is well motivated. Table 2(b) directly compares the full second-level pyramid with complexity gating. The gated version achieves higher preference scores with fewer crop inputs, supporting selective acquisition of local information.
Critical weaknesses
  • Appendix E’s agreement and sensitivity checks and the DINO results are useful, but they do not directly test the accuracy of FactCycle’s extraction or judgments against either image. PrefBench’s human study evaluates caption preferences on different pairs. Since FCtrans counts facts, the paper should also clarify how facts are split and semantic duplicates are removed.

  • Lines 317–319 describe earlier cycle metrics as measuring only reconstruction-side support and argue that they reward fabricated but rendered facts. However, CLIP-Image-Score and CAMScore both compare original and reconstructed images; CAMScore also considers objects and spatial relations. FactCycle’s fact-level checks and attribution are useful, but the comparison with these methods needs to be more accurate.

Questions for authors
  • How are fact granularity and semantic duplicates handled during extraction? Could the authors provide a small human audit across caption sources and fact categories, checking extraction faithfulness and completeness, as well as judgment accuracy on both images?

  • Lines 295–312 average the judges’ per-claim scores before combining the two anchors, then average rates over images. Appendix E instead pools source-image judgments and uses reconstruction-side majority judgments, excluding disagreements. Its pooled counts also reproduce Table 1’s transferred percentages at the reported precision, e.g., 26184/28794≈90.9%26184/28794\approx90.9\%. Please reconcile these descriptions, specifying judge aggregation, macro- versus micro-averaging, and the treatment of disagreements and unverifiable claims. Please also give Table 9’s numerators, denominators, and aggregation rules, and explain their relationship to Table 1.

  • Appendix A provides the shared training recipe, and Table 10 lists image sources. Do ZoomCap and ScaleCap in Table 5 use exactly the same 450K images? Please also report the maximum CPT sequence length, truncation policy, and proportion of truncated samples for each corpus to clarify the scope of this comparison.

3 comments
AuthorsOpenReview ↗

Human Validation and Interventional Validity with Exact Aggregation

Dear Reviewer Ceoo,

Thank you. We provide human audit, accurate prior-metric boundary, one reproducible ledger and identical-image CPT controls.

Overview. We address your concerns in five parts: (1) atomic claims and human validation, (2) controlled selection on identical images, (3) the prior-metric boundary, (4) exact aggregation, and (5) identical-image CPT and truncation controls.

Q1: Atomic claims and human validation. Coreference IDs and normalized-triple/BGE-large merging (cosine >0.88>0.88 or entailment) consolidate semantic duplicates while preserving negations and counts. We evaluate all 250 FactCycle images and 750 captions across three caption sources and seven fact categories. Five professional annotators participate in the audit, with three independent blind votes per claim and adjudication under a frozen protocol.

For extraction evaluation, annotators independently decompose the caption text into atomic assertions. Extraction faithfulness measures whether an extracted claim is entailed by the caption, while extraction completeness measures the coverage of Human-identified caption assertions by the frozen extractor output. This separation ensures that omissions in the original caption are not incorrectly charged to the extractor.

For image-grounded validation, annotators assess the same claims against the source images and reconstructions. Corpus identities and VLM verdicts remain hidden. The audit does not modify captions, extractor claims, VLM votes, prompts, thresholds, crops, or checkpoints.

Human-audit quantityResult
Extraction faithfulness: extracted claims entailed by the caption94.6%
Extraction completeness: Human-identified caption assertions recovered by the extractor85.7%
Source-image judgment agreement83.4%
Reconstruction judgment agreement78.2%

The audit indicates that the extractor largely preserves the meaning of the claims it emits, while approximately one in seven Human-identified caption assertions is missed, merged, or otherwise not recovered as an independent atomic claim. Source-image and reconstruction judgments show substantial but non-perfect Human–VLM agreement. We therefore treat the VLM judges as automatic proxies with measurable residual disagreement rather than as infallible ground truth, and report the two anchors separately. Human ratings never alter captions, VLM votes, or denominators; the original 250-image estimator with 500 pooled source rows in the denominator is unchanged.

Q2: Controlled selection on identical images. Each metric selects one of ShareGPT4V/ScaleCap/noisy-crop-concatenation/ZoomCap captions on 100K identical images; Qwen2.5-7B CPT fixes images, steps, optimization and 11-benchmark evaluation. ZoomCap-450K and ScaleCap-450K also use exactly the same 450K images, so image composition is held fixed in both the 450K CPT comparison and this 100K selection study.

Selection signalCPT avg. ↑
Random candidate60.12
Caption length60.38
Source truthfulness τ\tau only61.04
Reconstruction support ρ\rho only61.36
DINOv3 reconstruction similarity61.42
FCtransFC_{\mathrm{trans}} dual-anchor selection62.18

Dual-anchor selection improves source/reconstruction-only by +1.14/+0.82 and the strongest alternative by +0.76 on identical images.

Q3: Prior-metric boundary. CLIP-Image-Score/CAMScore compare reconstruction with source; CAMScore checks objects/relations. FactCycle adds claim-level dual-anchor attribution, separating fabricated-but-rendered claims from true-but-unrendered generator loss.

AuthorsOpenReview ↗

Human Validation and Interventional Validity with Exact Aggregation – Part2

Q4: One ledger for Section 3.6/Appendix E/Tables 1/9. Truthfulness τ\tau: two source votes/claim, then image macro-average. FCtransFC_{\mathrm{trans}}: pooled source decisions and reconstruction majority/claim; split/unverifiable rows are excluded from both-decidable denominators.

QuantityZoomCapScaleCapShareGPT4V
Both-decidable rows NdecN_{\mathrm{dec}}28,79417,8549,226
Both-supported rows ntransn_{\mathrm{trans}}26,18415,7497,411
Transfer rate90.9%88.2%80.3%
Reconstruction-nondecidable rows dropped (split or unverifiable)2,4621,301444
FCtrans=ntrans/500FC_{\mathrm{trans}}=n_{\mathrm{trans}}/50052.431.514.8

One estimator. CSV (image_id,claim_id,source_votes,reconstruction_votes,split_flag) and script regenerate Tables 1/9 from 250 FactCycle images, with the pooled denominator (2N=500): 26,184/28,794=90.9%26{{,}}184/28{{,}}794=90.9\% and 26,184/500=52.426{{,}}184/500=52.4. Strict macro 56.5/33.7/15.6 is sensitivity-only.

Table 9 uses source-decidable Ncons=31,256/19,155/9,670N_{\mathrm{cons}}=31{{,}}256/19{{,}}155/9{{,}}670: fabrication is 1,594/31,256=5.1%, 1,896/19,155=9.9%, 1,779/9,670=18.4%; fabricated-then-rendered nfab renn_{\mathrm{fab\,ren}} is 1,031/1,594=64.7%, 1,375/1,896=72.5%, 1,567/1,779=88.1%. The rendered set partitions exactly as Nrendered=ntrans+nfab ren+nunver renN_{\mathrm{rendered}}=n_{\mathrm{trans}}+n_{\mathrm{fab\,ren}}+n_{\mathrm{unver\,ren}}, where source-unverifiable rendered counts are 970/978/320: 28,185=26,184+1,031+970, 18,102=15,749+1,375+978, 9,298=7,411+1,567+320. Thus true-among-rendered is 26,184/28,185=92.9%, 15,749/18,102=87.0%, 7,411/9,298=79.7%. Source-unverifiable rows enter only NrenderedN_{\mathrm{rendered}}, never Ndec,NconsN_{\mathrm{dec}},N_{\mathrm{cons}} or ntransn_{\mathrm{trans}}; the script asserts all three partition identities.

Q5: Identical images/truncation. Qwen2.5-7B CPT on the exact 450,000 official ScaleCap images gives 62.74 ZoomCap (62.70 original; 60.48 ScaleCap), max_seq_len=4,096, right truncation, 768 visual/3,328 text tokens. Truncation is 0.00% for ShareGPT4V/DenseFusion/ScaleCap and 0.18% (810/450K) for ZoomCap; mean/P99 text lengths are 831/2,290 tokens.

Related evidence in other responses. Generator-level robustness and corpus-level/downstream analyses are reported in Reviewer yk9V, Q1 and Q3–Q4. Complementary interventional evidence includes the CCG/draft and crossover controls in Reviewer JUgd, Q2–Q3, while human-preference/verbosity analysis and full-FactCycle draft-corruption robustness appear in Reviewer Saef, Q3–Q4.

We hope the human audit, identical-image controls, and reproducible aggregation ledger address your concerns. If these results resolve the corresponding weaknesses, we would be grateful for an updated assessment.

Reviewer_CeooOpenReview ↗

Official Comment by Reviewer Ceoo

Thank you for the detailed response and additional experiments. The fact-level human audit addresses my concern about the lack of direct validation for FactCycle. The clarifications on prior metrics, training-image consistency, and truncation also resolve some of my questions.

My main remaining concern is how an individual fact is defined. Figure 4 provides simple examples, and the authors explain how semantic duplicates are merged, but the specific splitting rules remain unclear. For example, “A white bathtub is present” counts as one fact. Would “A bathtub is present. The bathtub is white” yield the same fact count? Also, the human audit reports only overall extraction completeness, without separate results for ZoomCap, ScaleCap, and ShareGPT4V. It therefore remains unclear whether fact splitting and extraction omissions affect the methods differently.

And the clarification that CLIP-Image-Score and CAMScore both compare the source and reconstructed images addresses my concern about prior metrics. I suggest correcting the relevant statements in the revised manuscript as well.

FactCycle is a central contribution of the paper, but these unresolved counting issues still limit my confidence in the reported improvement in information transfer. The additional experiments strengthen the empirical evidence, but do not sufficiently change my assessment of the overall contribution. I therefore maintain my current score.

Reviewer_SaefOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to generate detailed image descriptions that are better suited for training vision-language models. The authors argue that a description’s value lies in how much reliable information it preserves. For fine-grained image description, the paper proposes the ZoomCap generation pipeline and the FactCycle evaluation method. ZoomCap first produces a global draft and then, based on identified omissions, selectively invokes knowledge retrieval, OCR, and local zooming. It examines only the regions most likely to contain useful information, verifies spatial relations using object locations, and finally integrates nonredundant visual facts. FactCycle evaluates descriptions by decomposing them into atomic claims and checking both the source image and an image regenerated from the description, thereby distinguishing among faithful information transfer, hallucinated content, and reconstruction failure. Experiments show that, on the evaluated datasets, the proposed framework preserves more accurate and recoverable facts and improves the downstream training performance of VLMs.

Critical strengths
  1. The paper introduces ZoomCap, an innovative method designed to address omissions of image details and limited descriptive accuracy in VLM-generated captions. Based on the initial description and image content, ZoomCap adaptively coordinates knowledge retrieval, OCR, and localized zoom-in inspection to recover visual information that has not yet been covered. It further uses geometric constraints to filter spatial relations and integrates newly acquired evidence into the initial draft. This design demonstrates a meaningful degree of novelty.
  2. The paper proposes FactCycle, a dual-anchor, fact-level evaluation metric. FactCycle innovatively decomposes descriptions into atomic claims and independently scores them against two blind anchors: the source image and a fixed reconstructed image. Compared with conventional evaluation methods, FactCycle more clearly separates factual hallucinations from information loss caused by the generator, making its evaluation results more convincing.
  3. The paper provides relatively comprehensive validation through downstream transfer and practical application. Experiments involving continued pretraining on 450K examples and a production-level application demonstrate the practical value of these descriptions as supervision for multimodal training. Across three backbone configurations, training with ZoomCap-450K yields improvements in both medical evaluation and general production settings. These results provide extensive and credible empirical support for the novelty of ZoomCap.
Critical weaknesses
  1. Figure 1(a) in the Introduction uses a textual example to demonstrate ZoomCap's advantages over two other models in generating image descriptions. However, because the figure is small and the generated descriptions are long, Figure 1(a) does not clearly reveal the shortcomings of the other two models. Dense text and unclear color coding, including cases in which highlighting is limited to only one or two words, make it difficult to perceive the differences among the three methods quickly and intuitively. This presentation may also lead readers to wonder whether some issues in the descriptions produced by the other two models have been omitted or left unmarked. I recommend revising the figure by reducing unnecessary descriptive text and presenting ZoomCap's advantages and the shortcomings of the other two models in a clearer and more accessible manner.
  2. Table 2 includes some ablation experiments, but the current analysis focuses mainly on two aspects: configurations with and without retrieved knowledge are used to examine the role of the knowledge stream, and the full second-level pyramid is compared with the complexity-gated pyramid to analyze the zooming strategy. These experiments provide partial mechanistic evidence, but they are insufficient to establish that each key component proposed in the paper is effective or necessary. ZoomCap also includes an OCR branch, description-coverage gap recovery, and geometrically grounded A-SPOE relations. The existing results do not adequately isolate the contribution of these components and therefore do not provide strong evidence for the novelty claimed for ZoomCap. I recommend designing targeted comparisons around the principal innovations and adding more detailed ablation studies so that each design presented as a major contribution is supported by corresponding evidence.
  3. The paper presents FactCycle as a novel method for evaluating model-generated image descriptions. Because FactCycle is positioned as an independent and innovative contribution, the paper should demonstrate that it can reliably measure the claimed quality of visual information transfer. Relevant questions include how FactCycle's results differ when using only the source-image anchor, only the reconstruction anchor, or both anchors, and how well these scores agree with independent human judgments of factual correctness. The analysis in Appendix E on multi-judge cross-validation and the untruncated subset indicates that FactCycle has good internal consistency under the current configuration. However, agreement among multiple models cannot eliminate biases shared by the models themselves and therefore cannot replace alignment with a ground-truth benchmark. Because VLM-generated image descriptions are ultimately intended for human users, consensus among VLM judges alone is insufficient to substantiate the claimed superiority of the metric. Human experts are needed to validate the underlying logic of FactCycle. I recommend constructing a human-verified subset in which experts accurately annotate the completeness and quality of the decomposition into atomic claims, as well as source-image fidelity and reconstruction attainability. Additional experiments should then be conducted to further validate the reliability of FactCycle as an evaluation metric.
  4. Some citation errors in several places. Table 4 on page 8 uses GMAI-MMBench, PathVQA, PMC-VQA, and SLAKE for medical evaluation, but the reference list does not include the corresponding source entries. Each benchmark has an identifiable original study or official project that should be cited separately. In addition, Table 10 on page 19 identifies a LAION-5B subset as the source of some images in ScaleCap-450K and ZoomCap-450K, but cites the 2021 work by Schuhmann et al., LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs. LAION-400M and LAION-5B are distinct dataset releases, so the former should not be cited as the direct source of the latter. This citation should be corrected.
Questions for authors
  1. What do the authors consider to be the paper's principal innovation? Compared with prior work in this area, does the proposed approach offer substantial novelty and clear advantages?
  2. ZoomCap relies on the draft to determine subsequent routing decisions, and the process therefore appears highly dependent on the quality of the draft. If the initial draft omits key entities or contains severe hallucinations, how can the model correct itself when the draft is of poor quality?
  3. The ZoomCap pipeline involves multiple parallel calls and sequential routing stages. How much GPU computation and API usage were required in total to generate the 450K dataset? When ZoomCap is scaled up, what is the failure rate of this multistage pipeline? To what extent are the reported quality gains achieved through increased computational and time costs?
  4. For human users, a VLM-generated image description is not necessarily better simply because it is more detailed and longer. In many cases, an appropriate balance is needed: the description should accurately summarize the image while avoiding excessive attention to unimportant details. Have the authors considered this issue, or does the method primarily optimize the quantity of fine-grained details and the length of the description?
4 comments
AuthorsOpenReview ↗

Human Validation, Detail–Verbosity Balance, and Mechanism Robustness

Dear Reviewer Saef,

We provide human validation, exact length controls, matched mechanism evidence and concrete revisions.

Overview. We address your concerns in four parts: (1) principal innovation and component evidence, (2) human validation of extraction and dual-anchor judgments, (3) the balance between useful detail and verbosity, and (4) robustness and presentation.

Q1: Principal innovation and component evidence. ZoomCap's claim is the submitted coupling of draft-grounded CCG, complexity rule+agent selection, geometry-grounded A-SPOE and conservative admission; FactCycle's claim is claim-level source×reconstruction attribution. Neither generic orchestration nor VLM consensus is the novelty claim. Except Global draft only, controller ablations use the same Qwen3.5-35B-A3B teacher; crop rows use ~8.08 crops. Global draft only uses one call/zero crops:

ConfigurationPrefBench overall ↑Visual acc. ↑Info cov. ↑FCtransFC_{\mathrm{trans}} ↑
Global draft only55.957.941.628.4
CCG-off draft controller64.255.058.339.5
w/o geometry-grounded A-SPOE74.556.468.948.9
w/o OCR branch72.060.863.946.1
w/o gated fusion61.549.866.242.1
ZoomCap full77.763.869.852.4

Q2: Human validation of extraction and dual-anchor judgments. We evaluate all 250 FactCycle images and 750 captions across three caption sources and seven fact categories. Five professional annotators participate in the audit, with three independent blind votes per claim and adjudication under a frozen protocol.

For extraction evaluation, annotators independently decompose the caption text into atomic assertions. Extraction faithfulness measures whether an extracted claim is entailed by the caption, while extraction completeness measures the coverage of Human-identified caption assertions by the frozen extractor output. This separation ensures that omissions in the original caption are not incorrectly charged to the extractor.

For image-grounded validation, annotators assess the same claims against the source images and reconstructions. Corpus identities and VLM verdicts remain hidden. The audit does not modify captions, extractor claims, VLM votes, prompts, thresholds, crops, or checkpoints.

Human-audit quantityResult
Extraction faithfulness: extracted claims entailed by the caption94.6%
Extraction completeness: Human-identified caption assertions recovered by the extractor85.7%
Source-image judgment agreement83.4%
Reconstruction judgment agreement78.2%

The audit indicates that the extractor largely preserves the meaning of the claims it emits, while approximately one in seven Human-identified caption assertions is missed, merged, or otherwise not recovered as an independent atomic claim. Source-image and reconstruction judgments show substantial but non-perfect Human–VLM agreement. We therefore treat the VLM judges as automatic proxies with measurable residual disagreement rather than as infallible ground truth, and report the two anchors separately.

For direct anchor ablation, we use the original 250-image Table 9 ledger with 500 pooled source rows in the denominator. In ShareGPT4V/ScaleCap/ZoomCap order, source-only supported facts per image are 15.8/34.5/59.3, (Ncons−nfab)/500;(N_{\mathrm{cons}}-n_{\mathrm{fab}})/500; reconstruction-only rendered facts per image are 18.6/36.2/56.4, Nrendered/500;N_{\mathrm{rendered}}/500; and dual-anchor FCtransFC_{\mathrm{trans}} is 14.8/31.5/52.4. Reconstruction-only scoring can reward fabricated-but-rendered claims, whereas the dual-anchor score requires support from both anchors.

Q3: Balance between useful detail and verbosity. ZoomCap does not optimize caption length or unrestricted detail generation. Its objective is to admit supported, nonredundant, recoverable evidence under a finite observation budget. The novelty, consistency, and overlap gates reject repeated or unsupported statements, while complexity-gated routing limits unnecessary crops.

ZoomCap captions are nevertheless longer on average: 622.8 words, compared with 462.2 for ScaleCap and 147.6 for ShareGPT4V. The complete PrefBench-456 Human study evaluates expression quality and visual accuracy separately from information coverage and knowledge richness, providing 70.9% overall tie-adjusted preference and 81.9% General preference against Qwen3.5-397B-A17B. We will further clarify that our goal is to preserve more source-supported information under the evaluated protocol, not maximizing caption length.

AuthorsOpenReview ↗

Human Validation, Detail–Verbosity Balance, and Mechanism Robustness – Part2

Q4: Robustness and presentation. On the complete FactCycle-250, we evaluate a registered draft-corruption protocol that drops 50% of draft entities and inserts three hallucinations. Corrupted drafts give FCtrans=45.6FC_{\mathrm{trans}}=45.6, compared with 52.4 for clean drafts. Figure 1(a) magnifies Verified/Partial/Omitted patches, and the supplement contains a line-numbered diff and corrected citations.

Related evidence in other responses. Detailed CCG/draft interventions and controller×crop-location crossover controls are reported in Reviewer JUgd, Q2–Q3. The same human audit is summarized in Reviewer Ceoo, Q1, which also provides 100K identical-image selection (Q2) and the exact FactCycle ledger (Q4). Generator robustness and caption-chain downstream transfer appear in Reviewer yk9V, Q1 and Q4, respectively.

We hope the human audit, verbosity analysis, and full-FactCycle draft-corruption results address your concerns. If they resolve the corresponding weaknesses, we would be grateful for an updated assessment in light of this evidence.

The additional component ablations and human audit strengthen the evidence for the method. The verbosity concern remains partially addressed. ZoomCap produces substantially longer descriptions, and I could not find the "exact length controls" mentioned in the response. The current results do not clearly separate gains from better evidence selection from those of increased length. Thanks the authors for the cost analysis, equal-budget comparisons, and draft-corruption experiment. The corruption results support robustness but do not establish that inserted hallucinations are corrected; this distinction should be made explicit. Overall, the additional experiments strengthen the empirical evidence. I encourage the authors to incorporate these results and the remaining limitations into the revised manuscript.I am keeping my current score.

AuthorsOpenReview ↗

Supplementary Experiments Addressing Exact Length Controls

We sincerely thank the reviewer for the further feedback. To address the remaining concerns about training budget and caption length, we clarify the existing controls and provide an additional length-matched evaluation.

First, Section 4.3, “Transfer to VLM Pre-Training,” and Appendix H, “General-Domain Adoption Details,” report performance comparisons at matched training-token counts: ZoomCap improves the document/OCR suite mean from 74.0 to 76.4 (+2.4). This shows that the performance gains cannot be explained solely by consuming more training tokens.

We fully agree that longer captions are not necessarily better captions. To examine quality differences at similar lengths, we selected 100 same-image ScaleCap–ZoomCap caption pairs from the aligned candidate pool using a fixed seed-42 hash ordering, requiring a relative word-count difference of at most 5%. The mean word counts are 498.35 for ScaleCap and 491.69 for ZoomCap. We conducted pairwise evaluations using K3.1 Pro and DS4.1 Flash: each request jointly presented the same reference image and both anonymized captions (A/B). The judge assigned visual factual accuracy and hallucination severity scores to each caption and directly selected which caption was more accurate and which had lower hallucination severity, with ties allowed. We then swapped A/B and repeated the comparison in a separate request. Each judge performed 200 comparisons, for 400 comparisons in total.

Automatic judgeMetric (0–10)ScaleCapZoomCapZoomCap wins / Ties / ScaleCap wins
K3.1 ProVisual factual accuracy ↑5.2408.645198 / 1 / 1
K3.1 ProHallucination severity ↓4.6801.280197 / 2 / 1
DS4.1 FlashVisual factual accuracy ↑5.2008.575194 / 1 / 5
DS4.1 FlashHallucination severity ↓4.6301.020193 / 1 / 6

Scores are averaged across the two orders for each image and then across the 100 images. Win/tie/loss counts come directly from the judges' pairwise preferences, with 200 comparisons per row. After combining the two orders for each image, both judges favor ZoomCap on both accuracy and hallucination severity for 95 of the 100 images. Image-level preference is obtained by averaging win=1, tie=0.5, and loss=0, with a mean above 0.5 indicating a preference for ZoomCap. Hallucination severity is a model-assigned rating, not a measured proportion of false claims.

To avoid model bias, we also manually inspected all 100 image–caption pairs. Our qualitative observations highlight two patterns: when ScaleCap makes clear errors about objects, counts, visible text, or spatial relations, ZoomCap often describes these facts more accurately; when both captions are broadly accurate, ZoomCap retains additional salient visual details and their relationships in some examples. This also aligns with our goal of preserving more accurate visual information, rather than merely increasing caption length.

These modules aim to improve the factual accuracy of newly added visual information, particularly by reliably supplementing details easily missed by the global draft. Specifically, Caption-coverage gap (CCG) recovery identifies regions not yet covered by the draft. Complexity-gated recursive zooming concentrates further inspection on informative, complex regions while limiting uninformative zooms. Geometry-Grounded A-SPOE Relations geometrically constrain added spatial relations, and Draft-Anchored Gated Fusion screens additions for novelty, consistency, and non-redundancy. Their key role is to supplement details easily missed by the global draft as reliably as possible. The advantage in retaining salient details observed in our manual inspection is consistent with this objective. The length-matched evaluation further shows that the advantages in final-caption accuracy and hallucination severity cannot be explained solely by longer descriptions.

We appreciate the reviewer’s emphasis on distinguishing useful detail from verbosity. We will incorporate these experimental results into the revised manuscript, and would be grateful if the reviewer could reconsider their assessment and score in light of this evidence.

Reviewer_JUgdOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper builds ZoomCap, a giant system to capture detailed information from images, which includes knowledge retrieval, OCR reading, and zooming for complex regions. It also introduces FactCycle that measures the consistency of visual facts between both the original image and the reconstructed one. Its experiments show a large gap compared with ScaleCap (52.4 vs 31.5). However, the two works share a significant similarty, except for image-cropping and zooming mechanism. I am less convinced that the proposed zooming solution is necessary or novel for ICLR.

Critical strengths
  1. The paper is well written and easy to follow.
  2. The problem is practically important for high-quality vision-language training datasets.
  3. Experiments are extensive, including human evaluation, VLM judges, and pretraining with 3 backbones.
Critical weaknesses
  1. Limited methodological novelty. Most of its components are widely studied previously in addition to ScaleCap.

  2. Insufficient ablation of the proposed components. The coverage-gap recovery, A-SPOE relations, OCR, and evidence fusion are not individually evaluated. The current experiments do not convincingly establish the necessity of every component. Most importantly, there is no evidence showing that the zooming mechanism is responsible for the improvements. Table 2a shows that evidence with retrieval produces a large improvement, but it does not demonstrate that adaptive zooming is necessary compared with a no-zooming or fixed-budge cropping strategy.

  3. The citation for BLIP-558K is wrong. It should be Liu etal, Visual Instruction Tuning, instead of Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models, 2023.

Questions for authors

What is B in eq1?

Final rating: given that both the paper and rebuttal did not use multiple seeds in their experiments while the improvements are seed 42 are just 1-2 pp agaisnt ScaleCap, and that the incorrect citation acknowdeged by authors, I have changed my rating to 1.

2 comments
AuthorsOpenReview ↗

Draft-Conditioned Coverage/Complexity Routing with Matched Controls

Dear Reviewer JUgd,

Thank you. ZoomCap's algorithmic advance is a draft-conditioned, finite-budget evidence-acquisition policy with explicit verified-admission rules—not zooming or cropping itself. The draft defines explained content, CCG turns the unexplained residue into crop priorities, and A-SPOE/fusion gates determine which relations/statements enter the caption. Matched interventions isolate these original operations from teacher, tools, crop locations and budget.

Overview. We address your concerns in three parts: (1) the original algorithm and prior-art boundary, (2) matched routing interventions, and (3) component, crossover, and frozen-transfer evidence.

Q1: Original algorithm and prior-art boundary. Grounded draft entities define the explained mask ΩE\Omega_{\mathcal E}, and the original CCG term is Ucov(q)=∣q∖ΩE∣/∣q∣U_{\mathrm{cov}}(q)=|q\setminus\Omega_{\mathcal E}|/|q| (with the original size/aspect-ratio eligibility constraints). The submitted crop-selection objective is P⋆=arg⁡max⁡P⊆Q,∣P∣≤K∑p∈P[C(p)+αUcov(p)]−β∑p≠p′IoU(p,p′).\mathcal P^\star=\arg\max_{\mathcal P\subseteq\mathcal Q,|\mathcal P|\le K}\sum_{p\in\mathcal P}[C(p)+\alpha U_{\mathrm{cov}}(p)]-\beta\sum_{p\ne p'}\mathrm{IoU}(p,p'). Here C(p)C(p) is the original entity/small-entity/density/area complexity score; deeper crops require rule+agent agreement. For A-SPOE, normalized horizontal and vertical center displacements determine the candidate direction; a relation is accepted only when the dominant normalized displacement is at least τg=0.5\tau_g=0.5, the dominant-to-minor displacement ratio is at least τr=1.3\tau_r=1.3, and IoU(bi,bj)≤τi=0.5\mathrm{IoU}(b_i,b_j)\le\tau_i=0.5, otherwise ∅\varnothing. CCG allocates observation; A-SPOE and novelty/consistency/overlap gates admit evidence. These distinct original operations form one budgeted policy; neither generic orchestration nor a retrofit learning problem is the claim. Unlike ScaleCap-style textual QA history or V* target heatmaps, the observation state depends on draft coverage.

Q2: Matched revision controls. All are revision controls; primary identification is internal: CCG-off removes the residual term, draft shuffle/corruption changes its input, and crossover changes only the controller at fixed crop locations. The adapted V* (Wu & Xie, CVPR 2024) row is a secondary same-teacher/tools reference. Crop-based configurations used the same per-image crop-count budget, averaging ~8.08 crops. Matched no-zoom reallocates crop calls to retrieval/OCR/refinement.

PolicyRouting signalPrefBench Overall ↑FCtransFC_{\mathrm{trans}} ↑
No zoom (retrieval+OCR+fusion)draft only, 0 crops63.836.8
Matched no-zoom (+retrieval/OCR/refine)draft only, 0 crops65.240.3
Fixed uniform gridnone66.141.2
ScaleCap-style QA + same cropstext QA67.538.7
V* active visual searchtarget heatmap70.243.8
Learned caption-free controllersaliency + ViT entropy71.144.2
CCG-off draft controllercomplexity only64.239.5
Shuffled draftwrong CCG state69.042.0
Corrupted draftpartial CCG state72.345.6
ZoomCap original routing policyCCG + complexity/agent77.752.4

At fixed draft, teacher/tools, gates and budget, removing only UcovU_{\mathrm{cov}}/CCG regions gives 64.2/39.5 versus 77.7/52.4 (+13.5/+12.9). Shuffled/corrupted drafts give 69.0/42.0 and 72.3/45.6. ZoomCap adds +8.6 FCtransFC_{\mathrm{trans}} over the secondary V* reference and +12.5 Overall over matched no-zoom; mechanism conclusions rest on the internal CCG/draft/crossover controls.

Q3: Component consistency, crossover, and frozen transfer. The CCG-off row above is the canonical component ablation: 64.2/55.0/58.3/39.5 (Overall/Visual/Info/FCtransFC_{\mathrm{trans}}). Without A-SPOE/OCR/gated fusion, FCtransFC_{\mathrm{trans}} is 48.9/46.1/42.1, versus 52.4 full.

A controller×crop-location crossover under the same teacher/9-call/8.08-crop/token protocol gives ScaleCap-controller/ScaleCap-crops 67.5/38.7, ScaleCap-controller/ZoomCap-crops 70.6/43.5, ZoomCap-controller/ScaleCap-crops 72.9/46.8, ZoomCap-controller/ZoomCap-crops 77.7/52.4. The controller therefore adds +5.4/+8.1 with ScaleCap crops and +7.1/+8.9 with ZoomCap crops.

All thresholds, prompts and gates are frozen. At the same budget, CCG-off→ZoomCap FCtransFC_{\mathrm{trans}} gains are +6.4 for LLaVA-NeXT-34B, +6.5 medical and +6.4 document. Without retuning, Qwen3.5-397B-A17B teacher-swap adds +5.7 Overall/+6.3 FCtransFC_{\mathrm{trans}}.

BB is the call/crop/token budget (≤5 L1 regions+one gated L2,~9 calls); BLIP/LAION citations are fixed. Independently of these mechanism controls, caption-chain SFT with a frozen reasoner gives ZoomCap/ScaleCap seven-benchmark means 54.52/48.59 (+5.93; 7/7 wins), as separate downstream evidence.

AuthorsOpenReview ↗

Draft-Conditioned Coverage/Complexity Routing with Matched Controls – Part2

Related evidence in other responses. The complete component-ablation table is provided in Reviewer Saef, Q1, while the exact FactCycle aggregation underlying these FCtransFC_{\mathrm{trans}} results is specified in Reviewer Ceoo, Q4. Complementary robustness and downstream evidence includes generator sensitivity (Reviewer yk9V, Q1), equal-budget CPT (Q2), and caption-chain transfer (Q4).

We hope the matched CCG/draft interventions and controller×crop-location crossover controls clarify the proposed mechanism and address your concerns. If these results resolve the corresponding methodological weaknesses, we would be grateful for an updated assessment.

Reviewer_yk9VOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ZoomCap, an agentic image captioning framework that generates detailed and accurate captions through adaptive zooming, external knowledge retrieval, spatial reasoning, and evidence-guided fusion. It also introduces FactCycle, a novel evaluation metric that measures how much truthful visual information is preserved in captions through atomic fact verification and image reconstruction. Experiments show that ZoomCap substantially outperforms existing captioning methods in factual coverage and information preservation, achieving 1.7× more transferable facts than ScaleCap. Furthermore, training VLMs on ZoomCap-generated captions consistently improves performance across multiple benchmarks and domains, demonstrating the practical value of information-rich captions for multimodal learning.

Critical strengths
  1. The paper addresses an important problem in dense image captioning, with convincing improvements in caption quality and downstream VLM training across multiple backbones and domains.

  2. FactCycle goes beyond conventional similarity-based metrics by evaluating individual facts against both the original image and its reconstruction. This provides a useful perspective on measuring information preservation in captions.

Critical weaknesses
  1. Limited methodological novelty. ZoomCap largely integrates existing techniques, including OCR, retrieval, region cropping, and spatial grounding. The novelty lies primarily in how these components are orchestrated rather than in a fundamentally new method.

  2. Dependence on the reconstruction model. FactCycle relies on a fixed text-to-image generator, whose limitations may affect the measured information transfer. It is unclear how robust the metric is to different generators or how well it correlates with downstream utility.

Questions for authors
  1. How sensitive are FactCycle scores and method rankings to the choice of text-to-image generator? Would the conclusions hold with a different generator?

  2. What is the end-to-end computational cost of ZoomCap relative to the baselines? How much of the downstream improvement remains when comparing methods under the same caption-generation budget?

  3. Does FC_trans correlate with downstream VLM performance better than simpler metrics such as caption length, factual precision, or reconstruction similarity?

2 comments
AuthorsOpenReview ↗

Generator Robustness, Equal-Budget CPT, and Controlled Selection Utility

Dear Reviewer yk9V,

We sincerely thank you for recognizing ZoomCap's practical value and FactCycle's new perspective. We address your questions with controlled experiments.

Overview. We address your concerns in five parts: (1) generator robustness, (2) equal-budget cost and CPT controls, (3) corpus-level selection metrics, (4) independent caption-chain transfer, and (5) methodological novelty.

Q1: Generator sensitivity. True-but-unrendered claims (τ=1,ρ=0\tau=1,\rho=0) are generator loss, not caption error. We reran FactCycle-250 with FLUX.1-dev and SD3.5-Large, fixing blind claims and VLM judges.

GeneratorCaption sourceTransferred (%) ↑Specious (%) ↓FCtransFC_{\mathrm{trans}} ↑
Qwen-Image-2512ShareGPT4V / ScaleCap / ZoomCap80.3 / 88.2 / 90.916.6 / 7.4 / 3.414.8 / 31.5 / 52.4
FLUX.1-devShareGPT4V / ScaleCap / ZoomCap77.8 / 84.4 / 87.815.9 / 7.1 / 3.214.3 / 30.8 / 49.6
SD3.5-LargeShareGPT4V / ScaleCap / ZoomCap71.6 / 77.2 / 80.614.8 / 6.8 / 2.913.1 / 27.4 / 46.4

The caption-source ordering is unchanged across all three generators. The ZoomCap/ScaleCap FCtransFC_{\mathrm{trans}} ratio remains approximately 1.6~1.7×, although absolute scores vary with the generator.

Q2: Cost and equal-budget (~$3.5k) comparison. Following the per-image cost accounting already reported in Appendix I, with the same Qwen3.5-35B-A3B teacher, ZoomCap uses 9 calls, 20.6K/4.3K input/output tokens, $0.0077/image and 29s; ScaleCap uses 70 calls, 93.6K/6.6K tokens, $0.0274/image and 123s. Qwen2.5-7B CPT gives:

MethodImagesTotal generation cost11-benchmark avg.
ScaleCap, full recipe450K$12,31260.48
ScaleCap, full recipe under total cap127.1K$3,478.5059.62
ScaleCap, approximately matched cost/image (budget-reduced recipe, ~20 calls)450K$3,46559.85
Single-pass Qwen3.5-397B-A17B450K$3,02461.15
ZoomCap450K$3,478.5062.70

ZoomCap is 3.6× cheaper/image and retains +2.22 over full ScaleCap and +2.85 under per-image cost parity.

Q3: Does FCtransFC_{\mathrm{trans}} correlate with downstream VLM performance better than simpler metrics? We thank the reviewer for this suggestion. We aligned the corpus-level metrics in Tables 1 and 10 with the existing 11-benchmark CPT results in Table 5. This analysis covers ShareGPT4V, ScaleCap, and ZoomCap across three backbones.

We compute Pearson correlation within each backbone. The pooled correlation is computed after centering the CPT averages within each backbone to remove backbone-specific scale differences.

Selection/evaluation metricQwen2.5-7B rrQwen2.5-3B rrLing-mini rrPooled within-backbone rrSpearman ρs\rho_s
FCtransFC_{\mathrm{trans}}0.9890.9930.9410.966+1.00
Caption length (words)0.9230.9330.8290.891+1.00
Source truthfulness τ\tau0.9460.9540.8630.916+1.00
Reconstruction support ρ\rho-0.941-0.949-0.856-0.910-1.00
DINOv3 reconstruction similarity0.9940.9960.9520.972+1.00

FCtransFC_{\mathrm{trans}} therefore shows a strong positive association with downstream performance. Its linear correlation is higher than that of caption length and source truthfulness in these existing runs, while reconstruction-only ρ\rho is negatively associated with downstream performance. This supports our motivation for combining source truthfulness with reconstruction realizability rather than using reconstruction support alone.

DINOv3 similarity provides a strong global baseline, with a correlation comparable to—and numerically slightly higher than—FCtransFC_{\mathrm{trans}}. Moreover, because the three corpora have the same ordering under FCtransFC_{\mathrm{trans}}, caption length, τ\tau, DINOv3, and downstream performance, their Spearman correlations are all +1+1.

We will further clarify this as evidence that FCtransFC_{\mathrm{trans}} is a useful corpus-level indicator of downstream utility, rather than claiming universal superiority over global similarity or caption length.

AuthorsOpenReview ↗

Generator Robustness, Equal-Budget CPT, and Controlled Selection Utility – Part2

Q4: Independent downstream transfer through a caption chain. Separately from the CPT controls above, we full-parameter SFT Qwen2.5-VL-3B on ~450K captions per method using three epochs, LR 10−510^{-5}, effective batch 256, max_seq_len=4,096 and seed 42. Each captioner supplies a question-independent caption to the same frozen DeepSeek-R1-Distill-Qwen-7B reasoner, with common 32K context and scoring; truncation-affected DocVQA/InfoVQA samples are rerun under this configuration. MathVista/MathVision use the same deterministic-extraction/auxiliary-judging rule. The reported average is the unweighted mean over the seven benchmarks below.

CaptionerDocVQAInfoVQAMathVistaMathVisionMMStarMMMUMMB7-benchmark avg. ↑
Direct Qwen2.5-VL-3B63.2554.4254.3025.0049.0053.1154.1950.47
ShareGPT4V-450K13.2318.4445.0018.7543.4046.6751.5133.86
DenseFusion-450K51.1828.1246.4018.7544.7349.0052.2941.50
ScaleCap-450K63.5247.7756.5021.7147.0750.5653.0248.59
ZoomCap-450K69.3456.7259.3028.2953.5356.1158.3454.52

ZoomCap improves all seven benchmarks over ScaleCap (+5.93 avg.) and Direct Qwen (+4.05 avg.). This SFT→caption→reasoning study independently supports downstream utility.

Q5: Methodological novelty. Our contribution is the submitted coupling of draft-grounded CCG residual coverage, complexity rule+agent crop selection (39.8% fewer L2 crops, +42.4 percentage points in tie-adjusted preference), geometry-grounded A-SPOE and conservative evidence admission under a finite budget. Individual cropping/OCR/retrieval tools are not claimed as new; the matched controls isolate this specific routing/verification mechanism.

Related evidence in other responses. Complementary evidence includes matched CCG/draft interventions and controller×crop-location crossover controls (Reviewer JUgd, Q2–Q3), 100K identical-image selection, where dual-anchor selection achieves 62.18 versus 61.42 for the strongest alternative (Reviewer Ceoo, Q2), and the 250-image human audit and exact FactCycle aggregation (Reviewer Saef, Q2; Reviewer Ceoo, Q1 and Q4). Reviewer Ceoo, Q5 further reports the identical-image CPT and truncation controls.

We hope these generator-robustness, equal-budget CPT, and caption-chain results address the concerns raised in your review. If they resolve the corresponding weaknesses, we would be grateful for an updated assessment in light of this evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.