acceptodds
Under review as a conference paper at ICLR 2027

Compact Chain-of-Thought Distillation with Visual Grounding Guidance for Vision-Language Models

Abstract

Multimodal chain-of-thought (CoT) reasoning has significantly improved the performance of large vision-language models (VLMs) in solving complex tasks. However, distilling this reasoning capability into small VLMs remains challenging. In this paper, we construct a large-scale multimodal CoT dataset comprising 120k samples, each featuring a compact CoT with a standardized structure, explicit length constraints, and region-level visual grounding annotations. Based on our dataset, we obtain two critical observations: (1) VLM layers vary substantially in their receptiveness to visual grounding knowledge; (2) Compact CoTs achieve greater reasoning efficiency and higher accuracy than lengthy native CoTs. Motivated by this, we propose the Visual Grounding-Guided CoT (VGG-CoT) distillation framework, which first identifies receptive layers via per-layer probing and restricts LoRA updates to these layers for efficient visual grounding transfer. Then, a standard LoRA distillation is performed over all layers to transfer multimodal reasoning capabilities. In this way, VGG-CoT decouples visual grounding from multimodal reasoning, tailoring the distillation process to the layer-wise receptiveness and limited capacity of small VLMs. Extensive experiments on several VLM architectures, including QwenVL and InternVL, demonstrate that VGG-CoT consistently outperforms native CoT distillation in terms of reasoning accuracy and token efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal chain-of-thought, knowledge distillation, vision-language model, visual grounding, visual function layer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XFsBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper constructs CM-CoT, a compact multimodal chain-of-thought dataset containing 120K samples, and proposes VGG-CoT, which first probes the receptiveness of individual decoder layers in a student vision-language model to visual grounding knowledge, then distills visual grounding only into the most receptive contiguous decoder-layer interval, and finally distills multimodal reasoning into all decoder layers.

Critical strengths

The problem is important and clearly motivated. Small VLMs indeed face difficulty in simultaneously learning fine-grained visual grounding and long-chain reasoning, and decoupling these two capabilities is intuitively well justified.

The method is simple, modular, and relatively easy to reproduce. Layer-by-layer probing, contiguous-interval selection, and two-stage LoRA distillation are all built on commonly used training components and do not require teacher logits or intermediate representations.

Critical weaknesses

The evaluation may suffer from teacher self-consistency bias. Seven of the 12 source datasets do not provide task-aligned ground-truth bounding boxes, so their reference grounding boxes are generated by the same teacher model used for annotation. A student that more closely imitates the teacher will naturally achieve a higher IoU, but this does not necessarily mean that it better captures the true visual evidence. The paper does not report any human verification or assessment of bounding-box annotation quality.

The compact-CoT experiments conflate multiple factors. The compact version simultaneously changes sequence length, annotation structure, visual anchors, self-correction fields, grounding crops, and the supervision format. Therefore, the accuracy improvement cannot be attributed specifically to “compactness.” Length-matched, structure-matched, and field-removal controls are needed.

Questions for authors
  1. How extensive was the human quality audit of the teacher-generated grounding boxes and compact chains of thought? Please report the audit sample size, annotation protocol, bounding-box IoU or inter-annotator agreement, and error rates for each task category.

  2. Could the authors provide controlled comparisons using length-matched Native CoT, compressed CoT without additional structured fields, structured CoT without length reduction, compact CoT without visual anchors, and compact CoT without self-correction fields? These experiments would help disentangle the effects of sequence length and annotation structure.

  3. Why does Native SeqKD perform approximately 40–55 ANLS points below the Original Model? Does the conclusion still hold when the generation length is controlled, the loss weight assigned to final-answer tokens is increased, or only the final answer is distilled?

  4. Please report the complete computational cost of layer-by-layer probing, including the number of training runs for each student model, NPU hours, total training tokens, and the additional overhead relative to standard all-layer LoRA training. Can the selected VGR layers be reused across datasets after a single probing procedure?

5 comments

Thank you for your constructive suggestions regarding annotation quality and the composition of our supervision signal. We have added human audits, model-based verification, and controlled ablation experiments to address each point below.

Q1: How extensive was the human quality audit of the teacher-generated grounding boxes and compact chains of thought? Please report the audit sample size, annotation protocol, bounding-box IoU or inter-annotator agreement, and error rates for each task category.

A1:

We report the human quality audit covering all seven teacher-box source datasets in Table R20. We checked item by item whether the teacher box contains the visual evidence needed to answer the question and whether the compact explanation is grounded in the image. Across 350 audited questions, 93.71% of teacher boxes are usable and 96.29% of compact CoTs are image-grounded, with a mean box IoU of 0.590. Table R21 further breaks down these rates by source dataset across document, scene-text, and spatial-reasoning tasks.

Table R20: Human audit of teacher boxes and compact explanations.

Human Audit ItemResult
Total audited questions350
Number of questions per source dataset50
Grounding Box Validity Rate93.71%
CoT Validity Rate96.29%
Mean IoU0.590

Table R21: Annotation quality across source datasets in the human audit.

SourceUsable BoxesImage-grounded compact CoT
DocVQA94%98%
DUDE90%92%
InfographicsVQA86%92%
SROIE100%100%
TextCaps98%98%
TextVQA94%98%
VSR94%96%

We further scaled up the verification to 7,000 examples using GPT and DeepSeek under the same criteria and report the results in Table R22. Both verifiers indicate that the teacher-generated boxes, compact CoTs, and answers are generally reliable.

Table R22: Annotation quality verification by GPT and DeepSeek.

Verifier ModelQuestionsUsable BoxImage-grounded compact CoTCorrect Teacher Answer
GPT-6.1-Sol700090.31%89.50%94.74%
DeepSeek-V4.1-Flash700092.80%91.24%95.44%

Q2: Could the authors provide controlled comparisons using length-matched Native CoT, compressed CoT without additional structured fields, structured CoT without length reduction, compact CoT without visual anchors, and compact CoT without self-correction fields? These experiments would help disentangle the effects of sequence length and annotation structure.

A2:

We conducted controlled comparisons on reasoning length, structural formatting, and individual field removal to examine how each component affects student performance. First, to separate the effects of reasoning length and structure, we retrained Native SeqKD at full scale alongside two controlled variants: compressing the reasoning without adding structured fields, and adding structured fields while preserving the original length. Both variants keep the teacher's native final answer and use the same single-stage all-layer training.

We also compare performance under different settings of the image token budget, which is defined as the maximum number of visual tokens per input image during inference. We evaluate each trained model with fixed weights at 256 and 100 tokens to check the supervision effects when more or less image detail is available.

We compare these variants with One-stage Compact in Table R23. One-stage Compact achieves higher ANLS and F1 than both controlled variants under both image token budgets.

Table R23: ANLS and F1 (%) for compressed CoT without additional structured fields, structured CoT without length reduction, and One-stage Compact on Qwen3-VL-2B.

image token budgetSupervision & MethodANLSF1
256Native SeqKD12.3115.44
Compressed CoT without additional structured fields11.7814.93
Structured CoT without length reduction12.7615.68
One-stage Compact69.1464.17
100Native SeqKD11.1912.73
Compressed CoT without additional structured fields10.9812.61
Structured CoT without length reduction10.6712.12
One-stage Compact61.0653.56

Second, we remove only the visual anchor or only the self-correction field at a time and report the MME, MMStar, and POPE results in Table R24.

Specifically, retaining the visual anchor improves position and scene recognition, helping the student answer from spatial layout and scene context. Retaining self-correction improves code reasoning and position recognition, supporting logical checking and visual judgment. Complete compact CoT also achieves higher MMStar coarse perception accuracy and POPE recall across all three sampling splits, identifying more present objects and reducing false negatives.

Overall, these results further verify that both visual anchors and self-correction contribute to the performance gains of compact CoT.

Table R24: Results for compact CoT without visual anchors and compact CoT without self-correction fields on Qwen3-VL-2B (image token budget 256). MME reports subtask scores, MMStar reports accuracy (%), and POPE reports recall (%).

SupervisionMME PositionMME SceneMME Code ReasoningMMStar Coarse Perception (%)POPE Random (%)POPE Popular (%)POPE Adversarial (%)POPE Overall (%)
Complete compact CoT133.33164.0077.5068.0092.0092.0792.2092.09
Compact CoT without visual anchors118.33162.2577.5066.8089.5389.6089.5389.56
Compact CoT without self-correction fields128.33164.0070.0066.8089.8789.9389.9389.91

Q3: Why does Native SeqKD perform approximately 40–55 ANLS points below the Original Model? Does the conclusion still hold when the generation length is controlled, the loss weight assigned to final-answer tokens is increased, or only the final answer is distilled?

A3:

The low performance of Native SeqKD can be partly attributed to the free-form answer patterns that the student learns by imitating the teacher's native CoT responses. Specifically, the student generates a long rationale followed by a final response that may include additional explanations or multiple candidate answers. Such responses may not conform to the concise, unambiguous answer format specified by these datasets. For example, the response in Table R25 gives both floor and table as possible locations. Extracting only floor would discard the alternative table and misrepresent the model's uncertain response as a single, definite prediction.

Table R25: A Native SeqKD response from Qwen3-VL-2B on V7W. The reference answer is Floor.

ItemContent
QuestionWhere is the power strip?
Native final responseThe power strip is on the wooden surface, likely the floor or a table, to the right of the hand holding the smartphone.

Since Native SeqKD produces long responses, we checked whether hitting the generation token limit accounts for the low performance. As shown in Table R26, the scores remain low at both image token budgets even when we evaluate only the untruncated responses.

Table R26: Truncation statistics and EM, ANLS, and F1 (%) on untruncated examples for Qwen3-VL-2B Native SeqKD. EM denotes Exact Match of the answer.

image token budgetTotalTruncatedUntruncatedEM (%)ANLSF1
256300001329286711.4912.7716.00
100300001807281931.1111.9713.57

We also evaluated three training variants of Native SeqKD while keeping the teacher's native final answer fixed. As shown in Table R23, shortening the reasoning without adding structured fields leaves both ANLS and F1 low. We see the same pattern in Table R27 when we increase the loss weight on final-answer tokens or distill only the final answer, keeping the other settings of the retrained Native SeqKD baseline fixed. By contrast, VGG-CoT trains the student to give concise answers grounded in visual evidence and achieves higher scores on both metrics.

Table R27: ANLS and F1 (%) when the loss weight assigned to final-answer tokens is increased or only the final answer is distilled. Qwen3-VL-2B is evaluated at an image token budget of 256. Scores are arithmetic means across datasets on the full test set.

Supervision & MethodANLSF1
Native SeqKD12.3115.44
Increased loss weight on final-answer tokens (×4\times 4)11.5514.51
Only the final answer is distilled10.0313.16
VGG-CoT67.0262.45
Reviewer_axwGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CM-CoT, a 120K-example dataset of compact structured reasoning and region annotations from twelve source datasets, and the two-stage VGG-CoT distillation method. A LoRA probe is trained at each decoder layer, and BIC segmentation of validation grounding scores selects a receptive layer interval. The first stage distills grounding into that interval. The second updates all decoder layers for reasoning, using the original image, grounding crop, and grounding annotation. Experiments on held-out CM-CoT samples compare grounding and answer quality across three student architectures and examine reasoning length and layer selection.

Critical strengths
  1. Compact grounded supervision and layer-selective adaptation address the limited capacity of small VLMs. CM-CoT covers several task families, and the probing analysis spans multiple training-set sizes and architectures, giving a broader basis for layer selection than a single sweep.

  2. The paper reports explicit training configurations and includes parameter-matched late-layer and middle-layer controls. Table 2 shows higher average answer scores than the late-layer control for all three students. Table 3 also makes clear that a simple middle-layer interval captures much of the benefit. The supplementary scripts allow the evaluation procedure to be inspected.

Critical weaknesses
  1. The released evaluation scripts may score methods on different subsets of the test set. In code/preprocessing/build_predicted_crops.py, an invalid predicted box or missing <find> field causes an example to be skipped. Then code/evaluation/answer_metrics.py averages over the predictions present for each system and excludes missing reference examples instead of counting them as failures. If these scripts were used for the reported results, grounding failures can disappear from the answer metric. The paper does not report per-method coverage, so this possibility remains unresolved and affects the central accuracy comparison.

  2. Answer formatting may account for part of the large native-versus-compact accuracy gap. The scorer extracts <result> content from structured outputs, but takes all text after </think> for native outputs, or an empty answer if that delimiter is absent. ANLS/F1 against short references can then penalize a correct but verbose answer. Table 3 also changes compact supervision and the two-stage procedure together, while the selected interval adds only 0.33 ANLS over the middle-layer control. These comparisons do not yet separate reasoning transfer from answer formatting or establish the added value of automatic layer selection.

Questions for authors
  1. For each method, how many of the 30,000 held-out examples have valid Stage-1 boxes and scored final answers? Were the released skip/exclusion rules used for Tables 2–3 and the significance tests? Please report scores on the same full test set, treating failed and missing outputs consistently.

  2. Does the native-CoT gap persist when both systems use, or are independently parsed into, the same short final-answer format? Please distinguish missing/truncated answers, formatting failures, and semantically incorrect answers.

  3. With compact data and answer extraction fixed, how does one-stage all-layer distillation compare with the two-stage method? Please also report uncertainty for the 0.33-ANLS gain over the middle-layer interval and the additional layer-probing cost.

9 comments

Thank you for pointing this out. We will update the ANLS scores in Figure 3 and the evaluation scripts in the supplementary material.

Q1: For each method, how many of the 30,000 held-out examples have valid Stage-1 boxes and scored final answers? Were the released skip/exclusion rules used for Tables 2–3 and the significance tests? Please report scores on the same full test set, treating failed and missing outputs consistently.

A1:

Across all three backbones, VGG-CoT produces valid Stage-1 boxes on over 99.7% of the test set, with only 0.003%–0.26% missing or invalid boxes. We audited the full test set and report the number of valid Stage-1 boxes, saved outputs, and valid final answers for every method in Table R8 (dashes indicate single-stage methods without Stage-1 grounding).

Table R8: Output coverage across the full 30,000-example test set. Valid boxes are from Stage-1 predictions. The Valid Answers column counts non-empty extracted final answers.

ModelMethodTotalValid BoxesSaved OutputsValid Answers
Qwen2.5-VL-7BOriginal Model30000—3000030000
Native SeqKD30000—3000028968
Token Swapping30000299992999929999
Few-Shot CoT30000224833000030000
Vanilla Layers30000299982999829998
VGG-CoT30000299992999929999
Qwen3-VL-2BOriginal Model30000—3000030000
Native SeqKD30000—3000029230
Token Swapping30000299232992829928
Few-Shot CoT30000296533000030000
Vanilla Layers30000299652996829968
VGG-CoT30000299222992429924
Middle30000299812998129981
InternVL3-1BOriginal Model30000—3000030000
Native SeqKD30000—3000028952
Token Swapping30000294392950429504
Few-Shot CoT30000290013000029994
Vanilla Layers30000297672976929769
VGG-CoT30000299402994129941

As can be observed, some baselines have higher rates of missing or invalid Stage-1 bounding boxes. On InternVL3, Token Swapping and Vanilla Layers lack valid boxes for 561 and 233 questions, respectively. On Qwen2.5-VL, Few-Shot CoT lacks valid boxes for 7,517 questions.

In the original submission, the crop-building pipeline skipped examples with missing or invalid Stage-1 boxes. For Tables 2 and 3, we averaged scores over the available final prediction records counted in the “Saved Outputs” column of Table R8. The significance tests used only examples for which both compared methods had saved predictions. Consequently, the original averaging rule favors methods with more missing final outputs. Following your suggestion, all methods and paired significance tests are now evaluated over the same full 30,000-example list, counting any missing or failed output as zero. The revised Tables 2 and 3 are shown below. VGG-CoT's average ANLS and F1 in Table 2 change by at most 0.05 percentage points and remain the highest across all three backbones. Specifically, on InternVL3, VGG-CoT outperforms Vanilla Layers by 2.62 ANLS points and 2.92 F1 points.

Table 2: Re-scored reasoning results on the full test set. All values are in percentages. Best results are in bold, and second-best are underlined.

Qwen2.5-VL-7B

ANLS

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model25.5047.2337.1376.1174.6687.9058.09
Native SeqKD5.7310.4420.308.880.6712.489.75
Token Swapping53.3751.2742.1176.7375.7785.46‾\underline{85.46}64.12
Few-Shot CoT20.9234.1935.4070.0932.5475.0044.69
Vanilla Layers66.88‾\underline{66.88}52.65‾\underline{52.65}44.64‾\underline{44.64}76.89‾\underline{76.89}79.17‾\underline{79.17}84.3267.43‾\underline{67.43}
VGG-CoT68.5052.9545.1977.0880.2584.9168.15

F1

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model21.4944.3940.9476.88‾\underline{76.88}74.6586.1857.42
Native SeqKD5.4114.0626.7413.432.1217.5013.21
Token Swapping43.4146.9144.3976.4675.7783.49‾\underline{83.49}61.74
Few-Shot CoT16.5233.3137.0271.1837.2275.2945.09
Vanilla Layers57.64‾\underline{57.64}47.83‾\underline{47.83}46.83‾\underline{46.83}76.7579.17‾\underline{79.17}82.4165.11‾\underline{65.11}
VGG-CoT58.9748.0747.3977.2080.2582.9065.80

Qwen3-VL-2B

ANLS

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model50.1153.0522.8276.1677.0385.8360.83
Native SeqKD6.9811.8722.159.520.6912.4710.61
Token Swapping41.6548.6029.4871.6375.0678.2857.45
Few-Shot CoT24.1640.6733.6270.069.3075.0142.14
Vanilla Layers65.16‾\underline{65.16}50.7941.52‾\underline{41.52}72.5879.57‾\underline{79.57}81.6465.21‾\underline{65.21}
VGG-CoT67.7151.62‾\underline{51.62}41.8174.52‾\underline{74.52}80.2183.15‾\underline{83.15}66.50

F1

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model38.3245.2823.4475.9176.9982.8557.13
Native SeqKD6.5315.4528.6214.092.2117.7114.10
Token Swapping31.5542.0430.5471.5775.1176.0754.48
Few-Shot CoT19.5137.5834.3870.0115.3174.2441.84
Vanilla Layers54.74‾\underline{54.74}44.4744.04‾\underline{44.04}71.6279.57‾\underline{79.57}78.2362.11‾\underline{62.11}
VGG-CoT57.9645.06‾\underline{45.06}44.2473.59‾\underline{73.59}80.2179.82‾\underline{79.82}63.48

InternVL3-1B

ANLS

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model31.2045.3638.0571.63‾\underline{71.63}71.99‾\underline{71.99}84.8657.18
Native SeqKD7.9112.9823.669.990.7313.3811.44
Token Swapping52.9244.6419.6766.3464.0774.8353.74
Few-Shot CoT17.9222.1724.8338.6114.5440.7726.47
Vanilla Layers64.98‾\underline{64.98}49.32‾\underline{49.32}42.83‾\underline{42.83}70.6571.4580.1363.23‾\underline{63.23}
VGG-CoT70.7950.5144.7371.8175.8981.34‾\underline{81.34}65.85

F1

ApproachOpenImagesV7WFlickr30kTextCapsCUBTextVQAAvg.
Original Model23.1342.3240.6872.4272.05‾\underline{72.05}84.9255.92
Native SeqKD7.4016.2929.4014.382.2818.8014.76
Token Swapping41.7336.6720.6164.9864.1473.1050.21
Few-Shot CoT13.6219.3426.0540.1111.1741.7425.34
Vanilla Layers56.28‾\underline{56.28}42.34‾\underline{42.34}45.24‾\underline{45.24}69.6271.4578.8960.64‾\underline{60.64}
VGG-CoT62.9843.5347.5471.19‾\underline{71.19}75.8980.23‾\underline{80.23}63.56

Table 3: Re-scored ablation results on the full test set

ConfigurationCoTTwo-stage[email protected][email protected]ANLSF1
Original Model——––65.4860.12
Native SeqKD✓—––12.4415.33
Base (L17–L27)✓✓49.4537.4065.8061.01
Middle (L9–L19)✓✓53.6742.2866.8362.17
VGR (L6–L16)✓✓54.0042.6267.0262.45

Q2: Does the native-CoT gap persist when both systems use, or are independently parsed into, the same short final-answer format? Please distinguish missing/truncated answers, formatting failures, and semantically incorrect answers.

A2:

VGG-CoT maintains a clear lead across all three backbones under the same short-answer extraction rules. We examine the gap by checking the native output format, measuring truncation, and applying the same text post-processing to both methods.

In our original manuscript, each system was parsed according to its output format: we extract the short answer inside the <result> tag for VGG-CoT and the free-form response after the </think> tag for Native SeqKD. As shown in Table R9, we checked all 30,000 test outputs from each of the three Native SeqKD models used for Table 2 and found no <result> tags. These models were trained to place the final response after </think>, so we extract it using that native separator. For each model, we found a non-empty final answer in at least 96.5% of its test outputs.

Table R9: Answer formatting of the Native SeqKD models used for Table 2. Valid answers are non-empty final answers extracted using each model's native format.

ModelTotalWith <result> tagValid Answers
Qwen2.5-VL-7B30000028968
Qwen3-VL-2B30000029230
InternVL3-1B30000028952

Since Native SeqKD produces long responses, we checked whether hitting the generation token limit accounts for the low performance. As shown in Table R10, the retrained Qwen3-VL-2B Native SeqKD model still scores low at both image token budgets when we evaluate only the untruncated responses.

Table R11 reports the number of untruncated Native SeqKD responses in each formatting and text-matching category under each image token budget. We first count formatting failures, where the reasoning tag is unclosed or the extracted answer is empty. We then classify the remaining responses by how they match the ground-truth answer, including exact matches, responses that contain the ground-truth answer (with additional text), and other mismatches. As can be seen, even after excluding truncated responses and formatting failures, fewer than 1.5% of answers matched exactly at different image token budgets.

Table R10: Truncation statistics and EM, ANLS, and F1 (%) on untruncated examples for Qwen3-VL-2B Native SeqKD. EM denotes Exact Match after answer normalization.

image token budgetTotalTruncatedUntruncatedEM (%)ANLSF1
256300001329286711.4912.7716.00
100300001807281931.1111.9713.57

Table R11: Counts of untruncated Native SeqKD responses by formatting and text-matching category under each image token budget.

image token budgetUnclosed reasoningEmpty AnswerExact matchesContains ground-truth answer (with additional text)Other mismatches
256604271586712371
1001903141273015130

It is worth noting that some responses contain the reference answer and still leave the answer uncertain. For example, the response in Table R12 gives both floor and table as possible locations. Extracting only floor would discard the alternative table and misrepresent the model's uncertain response as a single, definite prediction.

Table R12: A Native SeqKD response from Qwen3-VL-2B on V7W dataset. The reference answer is Floor.

ItemContent
QuestionWhere is the power strip?
Native final responseThe power strip is on the wooden surface, likely the floor or a table, to the right of the hand holding the smartphone.

Reviewer_6mevOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to teach smaller vision-language models to reason with short, visually grounded explanations. It introduces CM-CoT, a dataset of 120,000 examples with compact reasoning traces and region annotations. The proposed method, VGG-CoT, first measures how well different decoder layers learn visual grounding. It selects a group of layers for grounding training, then updates all decoder layers in a second stage for reasoning and answer generation. At test time, the student predicts a relevant image region and uses this visual evidence to answer the question. Experiments cover three student models and evaluate grounding, answer quality, compact versus native CoT supervision, and different layer choices.

Critical strengths

1.The dataset and compact-supervision results are useful contributions. CM-CoT combines short reasoning traces with visual grounding across several task types. Figure 3 compares compact and native CoT training under matched settings. In this validation experiment, compact supervision reduces the reported average response length from 349.35 to 111.99 tokens and improves answer accuracy. This provides direct support for the supervision design.

2.The method improves performance across three student models. In Table 2, VGG-CoT improves average ANLS over Vanilla Layers by 0.72, 1.31, and 2.48 points. The grounding results also improve. Together with the layer-wise analysis, these results support the value of applying grounding training to suitable layers.

Critical weaknesses

1.The added value of the two-stage framework is not clearly established. Figure 3 supports compact CoT supervision, but it does not compare the full framework against direct all-layer training on the same compact data on the held-out test set. Table 3 changes both the supervision and the training procedure. It therefore leaves the central method contribution unclear.

2.The large gains over native CoT distillation need to be interpreted with care. In Table 2, Native SeqKD performs far below the original student models. Beating this baseline shows that VGG-CoT avoids an unsuccessful transfer setting. It provides less evidence of an advance over a strong distillation method. The smaller gains over the compact-data baselines are more useful for judging the method contribution.

3.Shorter responses do not yet establish the efficiency of the full pipeline. Figure 3 measures response length in a separate compact-versus-native training comparison. The full method also predicts a region and processes the original image together with a crop. End-to-end timing or compute measurements are needed to show whether the full pipeline is cheaper. This limits the efficiency claim.

Questions for authors

1.Can you compare the full method with direct all-layer LoRA training on the same compact data? Please use the same backbone, held-out test set, and comparable training resources. Please state any differences in visual inputs or test-time processing. This is the comparison most likely to change my recommendation.

2.Is the gain over the Middle variant consistent across training runs or other backbones? A paired analysis of the existing test predictions would also help assess whether this small gain is reliable.

3.What is the end-to-end inference cost of the full method, including region prediction and the second image-processing stage? A comparison with the compact-only baseline would clarify the practical trade-off.

6 comments

Thank you for recognizing the value of compact grounded supervision. Below, we address your three questions regarding the comparison with direct all-layer training, consistency across repeated runs, and end-to-end inference cost.

Q1: Can you compare the full method with direct all-layer LoRA training on the same compact data? Please use the same backbone, held-out test set, and comparable training resources. Please state any differences in visual inputs or test-time processing.

A1:

We compare VGG-CoT with two controls: interleaved grounding-and-reasoning training and a One-stage Compact baseline. With interleaved grounding-and-reasoning training, the model alternates between visual grounding and question answering, then uses the same region-prediction and crop-based answering pipeline as VGG-CoT at test time. One-stage Compact baseline trains all layers on complete compact responses and generates the region, rationale, and answer from the original image in one call. VGG-CoT instead learns grounding first and reasoning second.

Specifically, we compare budgets of 256 and 100 visual tokens per image.

In Table R1, we first isolate the effect of training order while keeping the data, layer configurations, number of update steps, and inference pipeline fixed.

Table R1: Average ANLS and F1 (%) for Qwen3-VL-2B training-order controls on the datasets reported in Table 2.

image token budgetTraining OrderANLSF1
256Interleaved grounding and reasoning66.2863.27
Two-stage (Grounding-then-reasoning)66.5363.50
100Interleaved grounding and reasoning64.5360.80
Two-stage (Grounding-then-reasoning)64.6160.97

Grounding-then-reasoning achieves higher average ANLS and F1 under both image token budgets. At an image token budget of 256, the paired ANLS gain has a 95% bootstrap confidence interval (clustered by source image) of [0.05, 0.45] percentage points. This supports learning to locate visual evidence first before learning how to answer from it.

We next compare VGG-CoT with One-stage Compact on MMStar and POPE in Table R2 and on MME in Table R3. Both use the same backbones and compact training data, with matching learning rate, LoRA rank, and effective batch size. On all three benchmarks, both models use the same prompts and answer directly from the original image, without region prediction or cropping. This comparison examines the trained models' visual answering capability under identical inputs.

Table R2: MMStar and POPE results for Qwen3-VL-2B.

BenchmarkMetric (%)One-stage CompactVGG-CoT
MMStarAccuracy ↑39.2745.00
POPEAccuracy ↑85.3387.96
F1 ↑86.2688.06
False Positive Rate ↓21.4212.91

As can be seen, VGG-CoT answers more questions correctly on MMStar. It also achieves higher accuracy and F1 on POPE, while reducing the false positive rate by nearly 40%.

Table R3 reports scores on four MME subtasks: numerical calculation, poster recognition, object existence, and text translation. All three models achieve higher scores on numerical calculation and poster recognition. These results demonstrate that two-stage training improves visual question-answering performance with compact data and answer extraction held fixed.

Table R3: Scores of three models on the same MME subtasks.

ModelMethodNumerical CalculationPostersExistenceText Translation
Qwen2.5-VL-7BOne-stage Compact132.50163.27200.00185.00
VGG-CoT140.00165.65200.00185.00
Qwen3-VL-2BOne-stage Compact62.50146.60190.00125.00
VGG-CoT87.50148.98190.00170.00
InternVL3-1BOne-stage Compact32.50126.19190.00170.00
VGG-CoT40.00127.89195.00170.00

Finally, we compare the One-stage Compact baseline with VGG-CoT in Table R4 to examine the role of two-stage inference when image resolution is constrained.

As shown in the table, at the higher image token budget (256), VGG-CoT improves average F1, especially on SROIE. When the image token budget is reduced to 100, VGG-CoT achieves higher average ANLS and F1 than the One-stage Compact baseline. Its average F1 advantage is larger than at an image token budget of 256.

The reason lies in the fact that the original image preserves global context, while the located image crop increases the effective spatial resolution of the relevant visual evidence, helping retain details needed for answering. Taken together, these results demonstrate the advantages of the two-stage method over one-stage all-layer distillation when compact data and answer extraction are held fixed.

Table R4: ANLS and F1 (%) of Qwen3-VL-2B on representative datasets. Avg. is the arithmetic mean across the listed datasets.

image token budget 100

DatasetMethodANLSF1
DocVQAOne-stage Compact56.7342.37
VGG-CoT58.0047.02
InfoVQAOne-stage Compact48.0733.02
VGG-CoT46.0831.70
DUDEOne-stage Compact39.7029.20
VGG-CoT41.7932.38
SROIEOne-stage Compact62.0745.34
VGG-CoT69.2558.06
TextVQAOne-stage Compact76.4469.65
VGG-CoT78.5573.13
TextCapsOne-stage Compact70.1566.56
VGG-CoT70.4067.87
Avg.One-stage Compact58.8647.69
VGG-CoT60.6851.69

image token budget 256

DatasetMethodANLSF1
DocVQAOne-stage Compact81.0774.68
VGG-CoT81.4675.41
InfoVQAOne-stage Compact61.1450.63
VGG-CoT60.8250.62
DUDEOne-stage Compact55.1448.84
VGG-CoT54.6349.25
SROIEOne-stage Compact87.8178.91
VGG-CoT86.1082.50
TextVQAOne-stage Compact83.0879.60
VGG-CoT83.1579.82
TextCapsOne-stage Compact74.7373.59
VGG-CoT74.5273.59
Avg.One-stage Compact73.8367.71
VGG-CoT73.4568.53

Q2: Is the gain over the Middle variant consistent across training runs or other backbones? A paired analysis of the existing test predictions would also help assess whether this small gain is reliable.

A2:

Middle baseline (L9–L19) overlaps with VGG-CoT (L6–L16) in eight out of eleven layers and already covers most of the VGR interval, making it a strong baseline. This comparison tests the benefit of choosing the interval boundaries automatically. We ran four paired training runs using different random seeds, keeping all other configurations identical within each pair. As shown in Table R5, VGR achieves higher ANLS and F1 than Middle in all four runs.

In the main experiment, VGG-CoT's average scores over the same 30,000 test questions are 0.30 ANLS and 0.36 F1 percentage points higher than Middle (two-sided Wilcoxon p=0.0037p=0.0037 and p=0.0089p=0.0089, respectively).

Table R5: Four runs with different random seeds on Qwen3-VL-2B. SD denotes the sample standard deviation.

MetricRunVGRMiddlePaired Gain
ANLSRun 167.0266.83+0.19
Run 267.1766.81+0.36
Run 366.9266.74+0.18
Run 467.1867.02+0.16
Mean67.0766.85+0.22
SD0.130.120.09
F1Run 162.4562.17+0.29
Run 262.5362.09+0.44
Run 362.3862.20+0.18
Run 462.5662.42+0.14
Mean62.4862.22+0.26
SD0.080.140.14

Q3: What is the end-to-end inference cost of the full method, including region prediction and the second image-processing stage? A comparison with the compact-only baseline would clarify the practical trade-off.

A3: We measured the end-to-end inference cost of the full method on 240 examples. Compared to Compact Only, the one-stage method, the two-stage method takes only about 0.2 seconds more per question. Stage 1 generates only the short grounding output, and Stage 2 generates the compact rationale and answer. As reported in Table R6, we timed region prediction, cropping, second-stage image processing, and answering on the same hardware and questions after warm-up.

Table R6: Average end-to-end inference time (seconds).

image token budgetStage 1: Region PredictionCropStage 2: InferenceVGG-CoT totalOne-stage CompactAdded Time
2561.6810.0231.5063.2092.9980.211 (7.0%)
1001.6730.0241.5103.2073.0410.166 (5.5%)

End-to-end inference adds 5.5%–7.0% to total time, with the cropping step itself taking less than 0.8% of the total time. Thus, two-stage inference trades a modest amount of extra time to acquire and use local visual evidence, preserving useful detail under a limited image token budget.

Compact supervision also improves token efficiency. The matched experiment in Figure 3 directly compares the two supervision formats. As presented in Table R7, Compact CoT distillation cut generated tokens by roughly two-thirds, requiring fewer autoregressive decoding steps and making the CoT easier to inspect. This captures the token-efficiency benefit, while the end-to-end timing above reflects the small additional cost of incorporating local visual crops.

Table R7: Output length of Native CoT versus Compact CoT.

Output LengthNative CoTCompact CoT
Mean generated tokens349.35111.99
Standard deviation1170.968.08

Comments on the paper

1 comment

Thank you for this insightful observation. We agree that automated metrics such as ANLS reflect both gains from compact reasoning and final-answer extractability. To better distinguish the effect of compact reasoning from that of final-answer formatting, we conducted a controlled 2×2 ablation by pairing Native CoT with compact final answers, and Compact CoT with the teacher's native final answers.

Table R31: Qwen3-VL-2B results across four combinations of reasoning and final-answer supervision. The Compact/Compact row corresponds to One-stage Compact, which trains all layers on complete compact responses. ANLS and F1 (%) are averaged equally across the six datasets in Table 2.

ReasoningFinal answerANLS (256 image tokens)F1 (256 image tokens)ANLS (100 image tokens)F1 (100 image tokens)
NativeNative11.0114.3911.2014.10
CompactNative10.3514.0010.3613.55
NativeCompact66.6864.0164.3860.60
CompactCompact69.6466.8567.0463.30

As shown in Table R31, the results support your observation: the format of the final-answer target influences automated scores, shifting ANLS from about 11% with native answers to more than 64% with compact answers. Crucially, however, when both variants are supervised with the exact same compact final answers, Compact CoT still consistently outperforms Native CoT across both token budgets, improving ANLS by 2.96 and 2.66 percentage points and F1 by 2.84 and 2.70 percentage points at 256 and 100 image tokens, while requiring far fewer reasoning tokens.

To further examine the gap through human evaluation, we randomly sampled 50 questions from each of the six datasets in Table 2 and rated both models' answers on the same questions. We manually assessed correctness against the image, question, and accepted answers, and rated format separately against the expected answer type and level of detail. We rated correctness as incorrect, partially correct, or fully correct. A fully correct answer gets all parts of the question right. For example, naming only one of two requested objects is partially correct. We used the same three-level scale for format compliance. The “Both” column shows the percentage of answers that were fully correct and fully matched the expected format.

Table R32: Human evaluation of Qwen3-VL-2B answers at 256 image tokens across the six datasets in Table 2. Native and Compact denote Native SeqKD and One-stage Compact, respectively. All values are percentages, averaged equally across datasets.

DatasetFully correct (Native)Fully correct (Compact)Fully format-compliant (Native)Fully format-compliant (Compact)Both (Native)Both (Compact)
OpenImages64.0084.000.00100.000.0084.00
V7W70.0082.000.00100.000.0082.00
Flickr30k66.0054.0094.0094.0064.0052.00
TextCaps82.0086.000.00100.000.0086.00
CUB62.0082.000.0084.000.0082.00
TextVQA90.0092.000.0098.000.0092.00
Average72.3380.0015.6796.0010.6779.67

As shown in Table R32, human judges marked more answers as fully correct for One-stage Compact than for Native SeqKD, a difference of 7.7 percentage points. One-stage Compact's answers also matched the expected format more often, by 80.3 points. Both gains help explain the large gap in Table 2.

It is worth noting that our contribution is not limited to compact reasoning supervision alone. The structured CoT design, the two-stage distillation strategy that teaches grounding before reasoning, and VGR-guided layer selection jointly constitute our VGG-CoT framework. Together, these components not only contribute to better performance in specific VQA tasks, but also achieve higher token efficiency with fewer generated tokens and better preserve general multimodal capabilities, such as achieving higher MMStar accuracy than One-stage Compact under the same single-image setting in Table R2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.