acceptodds
Under review as a conference paper at ICLR 2027

VisCal: Calibrating Visual Intermediate-State Capacity by Downstream Task Utility

17 views

Abstract

Interleaved visual reasoning models generate intermediate visual states. Yet the number of visual tokens or latents assigned to each state is usually fixed or calibrated by reconstruction fidelity. Holding reasoning trajectories fixed while varying capacity, we find that fidelity does not reliably order downstream answer utility, useful capacity depends on the problem and reasoning state, and capacity effects interact across steps. Visual capacity is therefore a budgeted sequential decision rather than a representation-quality parameter. We introduce VisCal at two levels: VisCal-Ext keeps the backbone frozen and distills hindsight trajectory allocations into a lightweight causal controller, isolating the deployment value of utility calibration and joint allocation. VisCal-OPD places a budget-conditioned capacity action before visual generation and constructs an outcome-calibrated teacher from the correct answer and complete outcomes of structured cross-step branches. Distilling this teacher jointly into capacity, visual, and text policies lets the model learn what each visual state contains and how later reasoning uses it. Across four models and eight transfer benchmarks disjoint from the post-training pool, VisCal-OPD improves matched-budget accuracy by 5.14 percentage points over the released models and matches their accuracy with an estimated 65.1% fewer visual tokens or latents from one budget-conditioned checkpoint per model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal reasoning, interleaved visual reasoning, vision-language models, visual intermediate states, visual capacity allocation, multimodal post-training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_K1LCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how many visual tokens or latents should be allocated to intermediate states in interleaved visual reasoning. Through capacity interventions on fixed reasoning trajectories, the authors find that reconstruction fidelity does not reliably predict downstream answer utility, that useful capacity varies across states, and that capacity choices interact across reasoning steps. VisCal-Ext trains an external capacity controller using trajectory-level utility while keeping the backbone frozen. VisCal-OPD moves capacity selection before visual generation and jointly optimizes capacity, visual content, and subsequent text through outcome-weighted branch distillation. Experiments across four models and eight transfer benchmarks report an average accuracy improvement of 5.14 percentage points at matched visual capacity and an estimated 65.1% reduction in intermediate visual capacity at the released models’ accuracy.

Critical strengths

Beyond measuring the relationship between capacity and answer utility, the authors include question-image controls, nuisance controls, and free-generation validation. The external-controller experiments further hold architecture and training procedures fixed while changing the supervision target, providing meaningful evidence for utility-based supervision and joint allocation. This is the main reason I lean toward acceptance. Results across four backbones and eight transfer benchmarks, together with Standard OPD, single-step branching, and capacity-only ablations, make the contribution more substantial than capacity tuning for a particular model. The appendix also reports end-to-end costs, allowing readers to distinguish reductions in visual capacity from actual computational savings.

Critical weaknesses

Standard OPD uses the native capacity policy, whereas VisCal-OPD trains across varying capacities. The comparison does not fully isolate adaptive allocation from the benefits of capacity-varied training. The capacity-only ablation freezes both visual and text generation. It therefore does not establish whether both components need adaptation or whether updating either one would suffice.

Questions for authors

Does VisCal-OPD outperform a state-independent capacity schedule trained with the same capacity-varied branches and supervision? With capacity learning fixed, how does updating only visual or only text generation compare with updating both?

4 comments

Dear Reviewer K1LC,

Thank you for the careful and constructive review. We are grateful that you think our controlled comparisons provide meaningful evidence for utility-based supervision and joint allocation. We ran both suggested experiments and report them below:

  1. A state-independent capacity schedule (W1/Q1).
  2. Updating only the image or only the text generator (W2/Q2).

W1/Q1: Adaptive allocation vs. capacity-varied training

Thank you for this helpful question. VisCal-OPD outperforms a state-independent capacity schedule trained with the same capacity-varied branches and supervision. In this new variant, the capacity head reads only the step index and the capacity price. It does not read the question or the current reasoning state. At a given step and price, it assigns the same capacity to every question. Everything else is the same as VisCal-OPD.

The table gives VisCal-OPD's accuracy gain over this variant on the same held-out Zebra-CoT split as Table 4 (pp, 95% CI). γ sets the visual budget: γ = 1 is the original model's budget, and γ = 0 is the lowest budget the model reaches.

Modelγ = 0.25γ = 0.50γ = 0.75γ = 1.00
Bagel-Zebra-CoT+3.41 [1.62, 5.23]+3.18 [1.40, 4.99]+2.74 [0.95, 4.55]+2.96 [1.19, 4.75]
DeltaV+5.87 [4.02, 7.75]+5.32 [3.47, 7.19]+4.48 [2.63, 6.36]+4.21 [2.40, 6.05]

VisCal-OPD is significantly ahead at every budget in both models, so adaptive allocation adds a gain beyond capacity-varied training.

We added this variant to Table 4, Sec. 6.4, and App. F.4.

W2/Q2: Updating only visual or only text generation

Thank you for this suggestion. At every tested budget, updating both generators is better than updating either one alone. With capacity learning kept the same as in VisCal-OPD, we trained two new variants: one updates only the image generator, and the other only the text generator. We used Bagel-Zebra-CoT because its image and text generators have separate parameters. In DeltaV, image and text generation share one backbone, so updating one also changes the other.

The table gives VisCal-OPD's accuracy gain over each variant on the same held-out Zebra-CoT split as Table 4 (Bagel-Zebra-CoT, pp, 95% CI; γ as in W1/Q1).

VisCal-OPD overγ = 0.25γ = 0.50γ = 0.75γ = 1.00
Capacity + image generator+2.58 [0.82, 4.38]+2.57 [0.73, 4.49]+2.32 [0.50, 4.19]+2.63 [0.88, 4.42]
Capacity + text generator+4.65 [2.90, 6.45]+4.22 [2.42, 6.08]+3.07 [1.22, 4.97]+3.72 [1.92, 5.60]

Both components need adaptation; updating either one alone does not suffice.

We added both variants to Table 4, Sec. 6.4, and App. F.4.

Thanks for your detailed responses, which partially solve my concern. I will maintain my score.

Dear Reviewer K1LC,

Thank you for your reply and for maintaining your positive score. We understand that a remaining concern may be that the two new comparisons in our response (the state-independent schedule and updating only one generator) were evaluated on the in-distribution Zebra-CoT split. To address this, we are evaluating the same variants on the eight transfer benchmarks. We will post the results here before the discussion period ends.

(Follow-up to our message above.)

Dear Reviewer K1LC,

Thank you for your patience. We evaluated the variants from our response on the eight transfer benchmarks at the original model's budget. The table gives VisCal-OPD's accuracy gain over each variant (pp).

VisCal-OPD overModel4 visual-state-centric benchmarksAll 8 benchmarks (95% CI)
Capacity + image generatorBagel-Zebra-CoT+1.63+1.71 [0.98, 2.43]
Capacity + text generatorBagel-Zebra-CoT+2.94+2.20 [1.47, 2.90]
State-independent scheduleBagel-Zebra-CoT+0.92+0.60 [−0.08, 1.27]
State-independent scheduleDeltaV+1.95+1.38 [0.69, 2.06]

As on the held-out split, both components need adaptation: updating either one alone does not suffice. VisCal-OPD stays significantly ahead of each single-generator variant.

Against the state-independent schedule, VisCal-OPD stays significantly ahead on DeltaV, so adaptive allocation adds a gain beyond capacity-varied training on the transfer benchmarks as well. On Bagel-Zebra-CoT, it is still ahead, but the gap is not significant. As with the main results (Sec. 6.2), this gap is larger on the four visual-state-centric benchmarks, which depend most on generated visual states.

We added the per-benchmark results for each variant to App. F.8 (Table 26) of the revised paper.

We hope these results address your remaining concern. Thank you again for your constructive suggestions.

Reviewer_A1r5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies models that draw intermediate images, or visual latents, while they reason, and asks how many visual tokens each image should get. Today this number is fixed or chosen by reconstruction quality. The paper changes only the number of tokens while keeping the reasoning fixed. It shows that better reconstruction does not always give better answers, that the best number depends on the problem and step, and that choices at different steps affect each other.

It then proposes two methods. VisCal-Ext keeps the model frozen and trains a small controller to choose the number of tokens; the controller learns from the best choices found during training using the correct answer. VisCal-OPD trains the model itself to choose the number of tokens before drawing each image, and also updates how it draws images and writes text. Across four models and eight benchmarks, VisCal-OPD improves accuracy by 5.14 points at the same visual budget. It matches the original accuracy with an estimated 65% fewer visual tokens.

Critical strengths
  1. The diagnosis is careful and convincing. Better reconstruction and better answers disagree on 27–44% of capacity pairs, while control comparisons stay at or below 6.1%. This is the main reason I lean toward acceptance: it gives a clear, well-tested reason to choose capacity by answer usefulness rather than by reconstruction quality.

  2. The gains are consistent and well controlled. VisCal-OPD beats the original model in all 32 model–benchmark pairs, by 5.14 points on average at the same visual budget, and it stays 4.4 points ahead of a matched post-training baseline. This makes me confident that the method is useful in practice, not only in one setting.

Critical weaknesses
  1. It is not clear how much of the gain comes from better images. The capacity-only ablation freezes both the image and text generators, so its 5–6 point gap mixes better images with better text. This is why I am not more positive. The claim that the model learns better visual content is not yet supported, even though the overall gain is.

  2. Some details needed to reproduce the work are missing. The controller's two main inputs, visual change and text uncertainty, are never defined, and the "negative-aware update" used for image training is not explained. The training cost is also not reported, even though each training decision runs eight branches to the final answer. This makes the method hard to reproduce and compare against, which also keeps my rating at weak accept. I incline to accept, if provided these during rebuttal.

Questions for authors
  1. The capacity-only ablation freezes both the image and text generators, so it cannot show which one drives the gain. Can you add a variant that trains only the image generator, so the effect of better images can be measured on its own?

  2. Each model and method was trained once, so the intervals do not include retraining variation. How much do the main gains change if VisCal-OPD and Standard OPD are retrained with a few different seeds on at least one model?

  3. The controller uses visual change and text uncertainty as inputs, but the paper does not say how they are computed. How exactly are these two features calculated and normalized?

5 comments

Dear Reviewer A1r5,

Thank you for the detailed and constructive review. We appreciate that you recognized our careful diagnosis and well-controlled gains. Below we provide each item you requested:

  1. The image-only variant (Q1).
  2. Two more training seeds (Q2).
  3. The details needed to reproduce the work (W2/Q3): the two controller features, the negative-aware update, and the training cost. We apologize for the missing details, and we added all of them to the revision.

W1/Q1: How much of the gain comes from better images

Thank you for suggesting this variant. Training only the image generator already gives a significant gain, but the full gain needs both generators. We trained two new variants and evaluated them on the same held-out Zebra-CoT split as Table 4. We used Bagel-Zebra-CoT because its image and text generators have separate parameters. In DeltaV, image and text generation share one backbone, so updating one also changes the other. The table gives each variant's accuracy gain over capacity-only (pp, 95% CI).

TrainedOriginal model's budgetLowest tested budget
Capacity only (Table 4)0 (reference)0 (reference)
Capacity + text generator (new)+1.38 [−0.37, 3.21]+1.05 [−0.70, 2.89]
Capacity + image generator (new)+2.47 [0.67, 4.33]+3.12 [1.29, 4.99]
Capacity + both = VisCal-OPD (Table 4)+5.10 [3.33, 6.83]+5.70 [3.90, 7.57]

VisCal-OPD stays significantly above the image-generator variant at both budgets (App. F.4).

Table 20 also shows that the model learns better visual content. At the same prefix and capacity, we replace the original model's visual state with VisCal-OPD's and score it with the original model, so only the image changes. With VisCal-OPD's image, the original model assigns higher likelihood to the correct answer in 68.3% (Bagel-Zebra-CoT) and 74.9% (DeltaV) of states.

We added both variants to Table 4, Sec. 6.4, and App. F.4 (all four budgets), and described this setup in the Table 20 caption.

Q2: Retraining with different seeds

Thank you for this suggestion. The main gains are stable across training seeds. We retrained VisCal-OPD and Standard OPD on DeltaV with two more training seeds (three in total). The table gives the gains averaged over the eight transfer benchmarks at the same visual budget (pp):

Gain of VisCal-OPD overSeed 1 (in the paper)Seed 2 (new)Seed 3 (new)
Original model+5.23+5.11+5.43
Standard OPD+4.81+4.93+4.90

We added per-benchmark results for each seed to App. F.6.

W2/Q3: How visual change and text uncertainty are computed and normalized

Visual change measures how much the new visual state differs from the previous one:

  • For Bagel-Zebra-CoT and ThinkMorph, it is the cosine distance between the DINO features of the two states.
  • For Monet, it is the cosine distance between the two mean-pooled latent blocks.
  • For DeltaV, it is 1 − TSIM (DeltaV's own similarity between the new state and earlier states).

For each model, we min–max scale visual change with the 1st and 99th percentiles of the training set and clip it to [0, 1].

Text uncertainty is the entropy of the next-token distribution just before the visual state is generated. We divide it by the log of the vocabulary size, so it also lies in [0, 1].

We added the full formulas for both features to App. D.1.

W2: Negative-aware update

The update uses both better and worse branches (as in DiffusionNFT). We compare branches from the same prefix. Branches with above-average return pull the image generator toward their images, and branches with below-average return push it away. We use it only for the flow-matching image generators (Bagel-Zebra-CoT and ThinkMorph).

We added the full loss and its settings to App. E.2.

W2: Training cost

One full VisCal-OPD training run (including all branch rollouts) takes 6.8–25.2 hours on two 8×A800 nodes (109–403 GPU-hours), depending on the model.

Standard OPD runs the same number of complete branches per prefix (App. E.3), so VisCal-OPD needs no more branch rollouts than Standard OPD. One Standard OPD run takes 7.4–29.4 hours on the same nodes. This is 1.06–1.17 times the VisCal-OPD time for each model.

Both methods' costs for each model have been added to App. E.3.

(Follow-up to our response above.)

Dear Reviewer A1r5,

We have just finished the same experiments on Bagel-Zebra-CoT and report them below.

Q2: Retraining with different seeds (follow-up)

We also retrained VisCal-OPD and Standard OPD on Bagel-Zebra-CoT with two more training seeds (three in total). The table gives the gains averaged over the eight transfer benchmarks at the same visual budget (pp). As in the paper, each question's result is averaged over three rollout seeds (App. B).

Gain of VisCal-OPD overSeed 1 (in the paper)Seed 2 (new)Seed 3 (new)
Original model+4.89+5.10+4.98
Standard OPD+4.22+4.14+4.39

The main gains are stable across training seeds in both Bagel-Zebra-CoT and DeltaV.

Detailed results of each seed on each benchmark are in App. F.6 (Table 24) of the revised paper.

We would be happy to address any further questions.

Thank you for the detailed response and the new experiments. The image-only variant, the extra training seeds, and the added details on the controller features, the negative-aware update and the training cost answer my questions. One small follow-up: the image-only variant was tested on one model and on the in-distribution Zebra-CoT split. Does training only the image generator also help on the eight transfer benchmarks? I will keep my positive score of 3 (weak accept).

Reviewer_Th1BOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a method that studies the capacity allocation problem for intermediate visual reasoning models. It establishes diagnosis that fidelity of generated intermediate images does not reliably give signal on utility for downstream tasks. It then proposes a out-come weighted self distillation method using OPD from ground-truth samples to post-train both the reasoning model as well as the capacity-action head for capacity allocation during inference.

Critical strengths

The results are strong in the demonstrated experiment setup with up to 10pp boost in multi-modal reasoning benchmarks. It also shows a saving of ~65% of use of visual token capacity to achieve the same accuracy with compared baseline methods. It also uses ablation to show that joint capacity allocation and visual reasoning model updates for co-adaptation is important for accuracy boosts.

Critical weaknesses

My major concerns are three folds:

  1. Novelty relative to existing on-policy self-distillation. The use of privileged outcome information to construct a hindsight teacher for on-policy distillation is not itself novel. It has been used, and for multi-modal it was explored in methods like visual-OPSD. Outcome-calibrated distillation has also been investigated previously. The key distinction beyond capacity-aware actions and structured branch exploration needs to be better articulated.

  2. Limited post-training scale. The post-training experiments use only 9.6K training examples, selected around the model's competence boundary. It remains unclear whether the observed gains hold at larger post-training scales or across broader training distributions.

  3. Generalization to different task distributions. VisCal-OPD relies on ground-truth answers and downstream task outcomes to construct its hindsight teacher. Although cross-benchmark transfer is evaluated, it remains unclear how well the learned capacity policy generalizes to substantially different task distributions or domains not represented during post-training which is an essential question for the practical applicability of the method.

Questions for authors

Some presentation issue — Figure 2(c) is not explained in the figure caption.

Training-time compute overhead — Constructing the hindsight teacher requires multiple structured branch rollouts with complete continuations. The associated training-time computational cost is not sufficiently characterized relative to the inference efficiency gains.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Th1B (1/2)

Dear Reviewer Th1B,

Thank you for the thoughtful and constructive review. We appreciate your recognition of our strong results, visual-token savings, and ablations. Your questions helped us strengthen the paper. Below we address each point:

  1. A clarification of the key distinction from prior distillation methods (W1).
  2. New results with 2× training data (W2).
  3. New results on two unseen domains (W3).
  4. An explanation of Fig. 2(c) (Q1).
  5. The training cost (Q2).

W1: Novelty relative to on-policy self-distillation and outcome-calibrated distillation

Thank you for this question. We agree that the outcome-weighted hindsight teacher itself is not new, but the key distinction is what it teaches: how much visual capacity each generated visual state receives. Below we compare VisCal-OPD with two representative methods: Visual-OPSD for on-policy self-distillation with privileged information, and SPOT for outcome-calibrated distillation.

Teacher usesStudent learns
Visual-OPSDits own generated visual thoughtstext-only reasoning, without visual states
SPOToutcomes of a few next-token candidatestext tokens
VisCal-OPD (ours)the correct answer and outcomes of capacity branchesthe capacity of each visual state, its content, and later text

Neither Visual-OPSD nor SPOT decides visual capacity, and the models we study fix it or set it by reconstruction fidelity. Our diagnosis shows that fidelity often prefers the wrong capacity (Sec. 3). The best capacity also depends on the reasoning state and interacts across steps, so capacity-aware actions and structured branch exploration follow from these findings rather than being extra parts on top of the teacher.

Most of the gain comes from these two parts: VisCal-OPD improves the original models by 5.14 points, while Standard OPD, which uses the same teacher but has no capacity actions or capacity branches, improves them by only 0.80 (Table 3).

We apologize for the confusion, and we clarified this distinction in Sec. 1 and App. A.

W2: Limited post-training scale

Thank you for raising this point. The gains hold at 2× training data. We retrained DeltaV's VisCal-OPD from the original model on 19.2K questions: the original 9.6K training questions plus 9.6K new questions from the same Zebra-CoT pool. The table gives the accuracy gain over the original model at the same visual budget (pp).

Training dataMIRAViC-BenchRBench-VMathCanvasMathVistaM³CoTVisPuzzlesVStarAvg.
9.6K (Table 2)+3.60+9.03+6.81+7.58+1.53+7.02+5.88+0.35+5.23
19.2K (new)+3.72+9.37+7.22+7.98+1.47+6.91+6.05+0.17+5.36

The gain on each benchmark changes by less than 0.5 points (we also report three independent training runs at 9.6K with different seeds in App. F.6, and these changes stay within their range).

Broader training distributions. Our 9.6K training questions cover all four task categories of Zebra-CoT (scientific questions, 2D visual reasoning, 3D reasoning, and visual logic and strategic games) and 18 subdomains. All our training data come from Zebra-CoT, and we added this as a limitation to the Conclusion. Transfer beyond Zebra-CoT is tested on the eight benchmarks and two unseen domains (W3). None of them is selected around the model's competence boundary.

We added the 2× result to Sec. 6.2 and App. F.6, and the task categories to App. B.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer Th1B (2/2)

(Continued from the previous comment.)

W3: Generalization to different task distributions

Thank you for this question. The learned capacity policy generalizes to both unseen domains we tested. We evaluated our Bagel-Zebra-CoT and DeltaV VisCal-OPD models on two MMMU-Pro disciplines outside Zebra-CoT's task categories: Business and Health & Medicine (286 questions each). Capacity prices are not retuned on MMMU-Pro. The table gives the accuracy gain over the original model at the same visual budget, and the estimated visual-token saving at the original model's accuracy.

DomainModelAccuracy gain (pp, 95% CI)Estimated visual-token saving
BusinessBagel-Zebra-CoT+4.20 [1.40, 6.88]51.2%
BusinessDeltaV+3.61 [0.93, 6.18]48.0%
Health & MedicineBagel-Zebra-CoT+1.63 [−0.70, 3.96]68.2%
Health & MedicineDeltaV+1.28 [−1.17, 3.73]53.3%
  1. Business. Accuracy improves significantly in both models. The domain is new, but it still relies on chart reading and quantitative reasoning.
  2. Health & Medicine. The accuracy gain is smaller and not significant. The domain needs medical knowledge that Zebra-CoT does not provide. Even so, the learned capacity allocation generalizes: VisCal-OPD reaches the original model's accuracy with an estimated 53–68% fewer visual tokens.

Ground-truth answers are used only to build the teacher during training. At test time, the model answers without them. Training does need checkable answers, and we added this as a limitation to the Conclusion.

We added these results to Sec. 6.2 and App. F.7.

Q1: Figure 2(c) caption

Thank you for catching this! Panel (c) measures how often reconstruction fidelity and answer utility disagree on which of two capacities is better. It compares intermediate states with three controls:

  1. Question image: the same measurement, with the capacity change applied to the input question image instead of an intermediate state.
  2. Fidelity null: how often two fidelity metrics disagree with each other.
  3. Utility null: how often two equivalent ways of scoring the answer (such as reordering the answer options) disagree.

It shows that intermediate states disagree far more often than all three controls. We added the missing explanation to the Fig. 2 caption.

Q2: Training-time compute overhead

One full VisCal-OPD training run (including all branch rollouts) takes 6.8–25.2 hours on two 8×A800 nodes (109–403 GPU-hours), depending on the model. This cost is paid once per model. After training, the model can run at any visual budget by changing the capacity price, without retraining. At inference, VisCal-OPD reaches the original model's accuracy with an estimated 20.1% fewer FLOPs and 15.9% less wall-clock time (App. F.5).

Standard OPD runs the same number of complete branches per prefix (App. E.3), so VisCal-OPD needs no more branch rollouts than Standard OPD. One Standard OPD run takes 7.4–29.4 hours on the same nodes. This is 1.06–1.17 times the VisCal-OPD time for each model.

We added both methods' costs for each model to App. E.3.


We hope these new experiments and clarifications address your concerns, and we would be grateful if you would take them into account in your final rating.

(Follow-up to our response above.)

Dear Reviewer Th1B,

We have just finished the same experiment on Bagel-Zebra-CoT and report it below.

W2: Limited post-training scale (follow-up)

We also retrained Bagel-Zebra-CoT's VisCal-OPD from the original model on the same 19.2K questions. The gains hold at 2× training data in this model as well. The table gives the accuracy gain over the original model at the same visual budget (pp). As in the paper, each question's result is averaged over three rollout seeds (App. B).

Training dataMIRAViC-BenchRBench-VMathCanvasMathVistaM³CoTVisPuzzlesVStarAvg.
9.6K (Table 2)+4.76+8.00+6.10+6.41+2.47+6.61+3.17+1.57+4.89
19.2K (new)+4.95+8.23+6.39+6.72+2.27+6.53+3.28+1.40+4.97

The gain on each benchmark changes by at most 0.31 points (we also ran three independent Bagel-Zebra-CoT training runs at 9.6K with different seeds, and these changes stay within their range).

We added these results to App. F.6 (Table 24) of the revised paper.

We would be happy to address any further questions.

Dear Reviewer Th1B,

Thank you for your time and effort in reviewing our paper. As the discussion period ends in about one day, we would be grateful if you could let us know whether our response addresses your concerns. If it does, we would appreciate it if you could take it into account in your final rating. Thank you again for your help in improving our paper.

Best regards,

The Authors

Comments on the paper

1 comment

Dear Area Chair,

We thank you and all reviewers for the time and effort spent on our paper. We also appreciate that the reviewers found our diagnosis "careful and convincing" (A1r5), our controlled experiments "meaningful evidence for utility-based supervision and joint allocation" (K1LC), our results "strong" (Th1B), and the method "useful in practice, not only in one setting" (A1r5).

Below we summarize how we addressed each concern and where the revisions are.

Concerns and responses

Concern (reviewer)New experimentResponseWhere
How much of the gain comes from better images; whether both generators need training (A1r5, K1LC)Image-only and text-only variants, on the held-out split and the eight transfer benchmarksTraining only the image generator already helps, but training both generators helps much more.Table 4, Sec. 6.4, App. F.4, F.8
Adaptive allocation vs. capacity-varied training (K1LC)A variant with the same capacity-varied training whose capacity ignores the current state, on the held-out split and the eight transfer benchmarksAdapting capacity to each state helps more than capacity-varied training alone: VisCal-OPD stays ahead of this variant in all tests.Table 4, Sec. 6.4, App. F.4, F.8
Retraining variance (A1r5)Two more training seeds (three in total) for VisCal-OPD and Standard OPD on Bagel-Zebra-CoT and DeltaVThe gains are stable: the average gain changes by at most 0.32 points.Sec. 6.2, App. F.6
Post-training scale (Th1B)2× training data on Bagel-Zebra-CoT and DeltaVThe gains hold; every benchmark stays within the seed range.Sec. 6.2, App. F.6
Generalization to unseen domains (Th1B)Two MMMU-Pro disciplines outside Zebra-CoTAccuracy improves significantly on Business. On Health & Medicine, the accuracy gain is not significant, but the learned capacity allocation still generalizes: VisCal-OPD reaches the original model's accuracy with 53–68% fewer visual tokens.Sec. 6.2, App. F.7
Novelty relative to on-policy self-distillation and outcome-calibrated distillation (Th1B)–The key distinction is what the teacher teaches: how much visual capacity each generated state receives, which prior methods do not decide. Without capacity actions and capacity branches, the same teacher (Standard OPD) gains only 0.80 points over the original models, vs. 5.14 for VisCal-OPD.Sec. 1, App. A
Missing reproduction details (A1r5)–All requested details are added: the definitions of visual change and text uncertainty, and the negative-aware update.App. D.1, E.2
Training cost (A1r5, Th1B)–6.8–25.2 hours per run; Standard OPD takes 1.06–1.17 times as long.App. E.3
Fig. 2(c) caption (Th1B)–Added the missing explanation of the panel and its three controls.Fig. 2 caption

Together, the new experiments separate the contribution of each component and show that the gains are stable across training seeds and training-data scale.

Discussion

  • Reviewer A1r5 noted that the image-only variant, the extra training seeds, and the added details "answer my questions". We then answered their follow-up question on the eight transfer benchmarks.
  • Reviewer K1LC replied that our detailed responses "partially solve my concern". We then evaluated both new comparisons on the eight transfer benchmarks, in addition to the held-out Zebra-CoT split.

Thank you again for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.