acceptodds
Under review as a conference paper at ICLR 2027

GOGGLES: When Output Serialisation Changes the Benefits of Higher Rank in VLMs

Abstract

Vision Language Models (VLMs) can acquire dense visual perception skills by distilling specialized expert targets via Low-Rank Adaptation (LoRA). However, because autoregressive VLMs generate spatial predictions sequentially, a fundamental question remains: how does adapter capacity interact with output serialisation, the way structured targets are formatted into text token sequences? We address this with GOGGLES, a framework that evaluates detachable LoRA adapters on frozen VLM backbones. Our design philosophy isolates output serialisation from information content by holding teacher target resolutions strictly fixed while varying output organisation. We uncover three primary insights: i. Serialisation-Rank Synergy: lossless output restructuring disproportionately benefits low-rank adaptation, allowing low-rank models to approach the fidelity of high-rank counterparts; ii. Domain Specificity: this interaction is highly modality-dependent, holding strongly for continuous depth rasters but vanishing in discrete segmentation, revealing that sequence dynamics depend on spatial target structure; and iii. Fidelity-Efficiency Trade-offs: gains in expert-relative fidelity require additional inference prefills and compute, demonstrating that serialisation design must balance fidelity with deployability. Overall, GOGGLES establishes output serialisation as a critical dimension alongside adapter rank, demonstrating how target sequence structure can alter rank requirements in VLM perception.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, low-rank adaptation, expert distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Jvq5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the interaction between output serialisation and LoRA rank in VLMs, showing that losslessly splitting depth-map generation into four calls benefits low-rank adapters more than high-rank ones at the cost of additional image prefills, while no comparable interaction is observed for the tested compact segmentation masks.

Critical strengths
  1. Clear research question and target-preserving experimental design. The paper examines whether output serialisation changes the benefit of increasing LoRA rank, rather than simply whether chunking improves performance. Its format-by-rank comparison preserves the complete quantised target and matches image exposure and optimiser updates. This distinguishes overall format gains from changes in the marginal value of rank, while ruling out improvements obtained by discarding target information.

  2. Consistent empirical support for the main depth finding. The interaction is positive across all three training seeds, with per-seed paired-image confidence intervals above zero. A fit-free ordinal metric and parser-sensitivity checks further support the finding, reducing dependence on a single evaluation procedure. Beyond average fidelity, the paper also demonstrates a substantial reduction in low-fidelity complete predictions, showing that the improvement extends to the failure tail.

Critical weaknesses
  1. The practical benefit of preserving the complete target remains unclear. Predicting a low-resolution depth map in a single call and then upsampling already achieves a score comparable to four-call generation (0.648 versus 0.642). Although the simpler approach loses detail, the paper should demonstrate whether preserving the complete target actually improves fine-detail prediction or downstream performance.

  2. It is unclear whether parameter savings justify the additional computation. Low-rank chunked generation uses fewer adapter parameters but requires four image prefills. The reported timings are not controlled measurements under identical hardware conditions, leaving its practical advantages in latency, GPU memory usage, and throughput uncertain.

  3. Task differences are confounded with output length and encoding. Depth outputs are long, whereas segmentation masks use a compact representation of only 64 hexadecimal characters. The stronger interaction for depth may therefore reflect differences in output representation rather than inherent task properties. Comparing different encodings within the same task would help clarify this distinction.

  4. Capacity requirements have not been clearly distinguished from learning speed. The experiments compare only two ranks and train for half an epoch. Without longer training and intermediate-rank comparisons, it remains unclear whether chunking consistently reduces the required capacity or simply helps low-rank adapters learn faster under the current budget. Single-call generation might catch up with more training.

Questions for authors

Please address the questions and concerns raised in the Weaknesses section.

2 comments

W1 — what preserving the complete target buys

The 0.648-versus-0.642 comparison is a tie on a score that cannot see within-block structure, because both numbers average over cells whose teacher values the coarse arm represents by a single block mean. Measuring the fine detail directly, on the frozen 1,000-image set and all three seeds:

armwithin-block ordering accuracyresidual amplitude (teacher 7.43)skill vs zero-detail baselineδ1\delta_1 lost by smoothing its own detail
Pooled12/r16 (one call, upsampled)0.000 by construction0.00+0.00000.0000
Raw/r160.583-0.5924.08-4.17+0.285-+0.2980.020-0.024
Chunked/r160.648-0.6547.12-7.24+0.210-+0.2260.004-0.008
Raw/r2560.684-0.6894.81-4.87+0.415-+0.4220.041-0.044
Chunked/r2560.700-0.7037.04-7.12+0.348-+0.3590.029-0.031

Within-block ordering accuracy is the fraction of within-block cell pairs whose predicted order matches the teacher's, over pairs whose teacher values differ. Residual amplitude is the standard deviation of H(x)=x−upsample(2×2H(x)=x-\mathrm{upsample}(2\times2 block mean of x)x). Skill is $1-\lVert H(\text{pred})-H(\text{ref})\rVert / \lVert H(\text{ref})\rVert$, so zero means no better than predicting no detail at all. The last column rescores each arm's own prediction with its high-frequency residual removed.

Reading these two ways, as we think they should be read:

  • Complete target versus coarse target. Preserving all 576 values buys real within-block signal, and the coarse one-call alternative cannot represent it in principle: zero residual, zero skill, nothing lost by smoothing it away. This holds under both readings below.
  • Grouped versus one call at the same rank. Grouping produces detail whose amplitude is the closest of any arm to the teacher's (7.12-7.24 against 7.43) and whose within-block ordering is more often right (0.648-0.654 against 0.583-0.592 at rank 16). One-call predictions are over-smoothed: their residual amplitude is about half the teacher's. Shrinking towards the block mean lowers squared error, so the one-call arms also score higher on the RMSE-based skill column. Both columns follow from that single shrinkage, and we report both.

We also ran the downstream check on the full BLINK and CV-Bench sets. It holds the question, the images and the answering model fixed and varies only which readout supplies the depth descriptor, with every descriptor rendered by the same routine, so a difference between arms is a difference between readouts. Between supplying the complete target in four grouped calls and supplying it in one, answering through the frozen backbone, we see no detectable difference on either benchmark: +0.006+0.006 on BLINK and +0.000+0.000 on CV-Bench, and against the coarse one-call readout +0.012+0.012 and +0.001+0.001. The grouped configuration reaches those numbers with a sixteenth of the adapter parameters. Detaching the skill restores the frozen backbone's own answer on every one of the 3,160 questions, so what the skill adds while worn is also exactly what it takes away when removed.

W2 — do the parameter savings justify the extra computation

This is the controlled benchmark in our reply to Hmn1 (Q3), and it is now measured: same GPU, one process, one complete map as the request unit, the three arms interleaved, concurrency 1 and 8, with p50 and p95 latency, throughput and peak memory. The controlled benchmark replaces the earlier timings, which were amortised throughput at a fixed number of calls in flight, measured on different hosts. Peak memory stays in one class (19.0 to 23.5 GiB), and the grouped rank-16 configuration uses a sixteenth of the adapter parameters.

W3 — task differences versus output length and encoding

Expand12 is the within-task control for exactly this: same task, same 144 independent values, 1,727 tokens instead of 431. It shows the length axis alone does not account for the depth gain. The complementary test, the same task under a different encoding, is a segmentation replication with a verbose per-cell encoding; at the measured 19.4-23.7 s per optimiser update that is four cells of ten to twelve hours each plus its own registration, so we name it as the next experiment. From the compact-mask result we claim a boundary: at 16x16 packed masks the interaction stays inside the predefined ±0.02\pm0.02 region while rank still helps, which bounds where the depth pattern holds.

W4 — required capacity versus learning speed

The direct test is the same recipe trained twice as long: 3,750 updates, a full epoch, all four cells, seed 0. On the registered 1,000 images the full-epoch interaction is +0.0041+0.0041, no detectable difference on the primary metric, with a positive ordinal-accuracy companion (+0.0066+0.0066, interval excluding zero); from update 1,875 to 3,750 of the same trajectory it changes by +0.0199+0.0199, with an interval excluding zero. Update 1,875 of that trajectory is half annealed; with the cosine completed at 1,875 the interaction is +0.0353+0.0353, with an interval excluding zero (reply to Hmn1, W2/Q2). Rank 64 adds the intermediate point on the capacity axis, and it places the interaction in the lower segment: it is already present between ranks 16 and 64, where the ranks are close. We report both rank segments and both budget points.

Reviewer_RK6tOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how output serialization affects the marginal benefits in different LoRA ranks for dense visual prediction in VLMs. The proposed framework called GOGGLES compares generating the same target in one call (RAW) or four independent calls (CHUNKED). Experiments show that the latter could benefit lower-rank adapters more on depth estimation, but not on segmentation. Further analyses discuss possible explanations and computational costs.

Critical strengths

Controlled experimental comparison. This comparison across multiple seeds provides empirical support for rank-serialization interaction which the authors observed.

Critical weaknesses
  1. Limited insights and contributions. This paper observes the so-called rank-serialization interaction but the explanation is not sufficient, with respect to the underlying mechanism. Analyses are more on grouping benefits, not why lower-rank adapters benefit more.
  2. Insufficient experimental settings. The experiments cover only one backbone, with two LoRA ranks and two serialization strategies. Hence, the generalizability of the findings is not guaranteed. Another crucial problem is the lack of cost-matched comparisons and downstream validation.
Questions for authors
  1. Can the authors extend their experiments with more rank settings and training-budget comparisons to enhance what they have found in rank-serialization interaction?
  2. What advantages can CHUNKED provide compared with simpler alternatives, considering you mentioned additional computational cost?
1 comment

W1 — why low-rank adapters benefit more

Our account has three measured parts, and we have sharpened the second since submission.

  1. Per-call prediction load, not answer length. Expand12 keeps the 1,727-token raster but asks for 144 independent values, and it recovers most of the rank-16 gain (0.646-0.654 against Raw/r16's 0.541-0.553). The burden that matters is how many independent spatial values one answer has to commit to.
  2. Free-running history is where the formats diverge. With correct history the long raster is better at the next value, and the ordering reverses (−0.159-0.159 to −0.158-0.158 with gold prefixes against +0.011+0.011 to +0.018+0.018 free running, twelve adapters). The grouped format repeatedly restarts from a clean task-phase context instead of conditioning on its own drifting output.
  3. The gain lands where rank 16 fails. At the registered δ1=0.30\delta_1=0.30 threshold, grouping moves 88-118 images per seed from failure to success and 8-11 the other way, a net reduction of 80-110 failed maps out of 1,000.

W2, Q1 — ranks, backbone, cost matching, downstream

  • Ranks: rank 64, both formats, three seeds, both rank segments reported. The interaction is present between ranks 16 and 64 in every seed, and in the upper segment two of three seeds show no detectable difference.
  • Training budget: the full epoch, all four cells, seed 0. On the registered 1,000 images I(3750)=+0.0041I(3750)=+0.0041, no detectable interaction on the primary metric, with ordinal accuracy +0.0066+0.0066 (interval excluding zero). I(1875)I(1875) on the same trajectory is −0.0158-0.0158; a registered arm that completes the cosine at 1,875 gives +0.0353+0.0353, with an interval excluding zero, so that negative sign comes from the half-annealed point (reply to Hmn1, W2/Q2).
  • Second backbone: InternVL3.5-8B, same target and image-exposure budget, four cells, training seed 0, scored on the registered 1,000 confirmation images with the registered parser. The interaction is +0.030+0.030, with a positive interval. One call gains +0.134+0.134 from rank 16 to rank 256 and the grouped format gains +0.104+0.104, and the grouped format leads at both ranks (0.536 against 0.419 at rank 16, 0.640 against 0.553 at rank 256). On the primary backbone the same quantity is +0.031+0.031 to +0.037+0.037 across three seeds, so the interaction is not a property of one backbone.
  • Cost-matched comparison: the controlled benchmark in our reply to Hmn1 (Q3), now measured.
  • Downstream: the full BLINK and CV-Bench sets, with the question and the answering path held fixed and only the readout that supplies the depth descriptor varied (Chunked/r16, Raw/r256, Pooled12/r16, and no descriptor). Every descriptor is rendered identically, each adapter also answers with no descriptor supplied, and every item is answered both through its own adapter and through the frozen backbone, so the descriptor is read apart from the adapter. The grouped configuration carries a sixteenth of the adapter parameters, and in the cost benchmark of our reply to Hmn1 (Q3) it produces a complete map 3.7×3.7\times faster with one request in flight; answering through the frozen backbone, which readout supplied the descriptor makes no detectable difference downstream on either set: Chunked/r16 against Raw/r256 is +0.006+0.006 on BLINK and +0.000+0.000 on CV-Bench, and against Pooled12/r16 +0.012+0.012 and +0.001+0.001.

Q2 — what grouping buys over simpler alternatives

The cheap alternative is to predict a coarse map in one call and upsample it. It scores 0.648 against the grouped arm's 0.642 at 0.58x the recorded training time, and we report it as the strongest rank-16 operating point when a coarse map is acceptable. What it cannot do is carry any within-block structure: its high-frequency residual is identically zero, its skill against a zero-detail baseline is exactly 0.0000, and removing its detail costs exactly 0.0000 of registered fidelity, because there is none to remove. Every complete-target arm carries real within-block signal on the same 1,000 images: skill against the zero-detail baseline is +0.210 to +0.226 for Chunked/r16, +0.285 to +0.298 for Raw/r16, +0.348 to +0.422 at rank 256, and removing each arm's own detail costs 0.004 to 0.044 of affine δ1\delta_1. So the choice is not "0.648 versus 0.642": it is whether the application needs within-block structure at all.

Reviewer_Hmn1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how output serialisation interacts with LoRA rank in VLM-based dense spatial prediction. The authors introduce GOGGLES, a controlled framework comparing RAW, which generates the entire spatial grid in one response, and CHUNKED, which generates four interleaved spatial partitions independently while preserving the same complete target information. Experiments on Qwen3.5-9B with LoRA ranks 16 and 256 show that CHUNKED benefits low-rank adapters more strongly in depth estimation, producing a positive rank–format interaction of +0.031 to +0.037 across three seeds. However, the corresponding interaction is small for compact segmentation masks. Additional experiments investigate generation dynamics, prediction failures, and computational costs.

Critical strengths

S1. Well-controlled experimental design. The target-preserving factorial comparison is a key strength. By keeping the quantised teacher targets identical across RAW and CHUNKED, the authors avoid confounding output restructuring with information loss. The consistently positive depth interaction provides convincing evidence that output organisation changes the marginal benefit of higher LoRA rank under the tested conditions.

Critical weaknesses

W1. The underlying mechanism is not sufficiently disentangled. Although RAW and CHUNKED preserve identical target information, they simultaneously change per-call sequence length, prediction history, context resets, phase prompts, and the number of visual prefills. Therefore, it remains unclear whether the observed interaction is primarily caused by spatial output restructuring or by these other generation-related factors. Furthermore, the depth and segmentation experiments differ in output length, encoding, and target complexity. Thus, the absence of a comparable interaction in segmentation may reflect differences in prediction burden rather than an intrinsic modality-specific effect. More controlled ablations would strengthen the mechanistic interpretation. W2. Limited evidence for a general capacity–serialisation relationship. The main experiments consider only two LoRA ranks (16 and 256) under a fixed half-epoch training budget. Although the results establish an interaction between these configurations, they do not characterise how this relationship evolves across ranks or training stages. In particular, it is unclear whether CHUNKED genuinely reduces the representational capacity required for the task or simply provides more favourable optimisation dynamics under the current training budget. Intermediate-rank evaluations and learning curves would help distinguish these explanations. Additionally, CHUNKED requires four visual prefills, and its end-to-end accuracy–compute advantage over higher-rank RAW models remains unverified. These limitations weaken the generality and practical significance of the central claim.

Questions for authors

1.Can the authors disentangle output restructuring from independent inference calls and context resets? For example, does the interaction persist when multiple chunks are generated within a single context or when using two rather than four calls?

2.How does the interaction change across training stages or intermediate LoRA ranks (e.g., 64 or 128)? Does it persist with additional optimisation?

3.What are the end-to-end inference latency and computational costs of CHUNKED/r16 versus RAW/r256 under matched hardware and batching conditions? Does CHUNKED retain a favourable accuracy–compute trade-off?

4 comments

If the reviewer responds, I will consider re-evaluating the score.

Thank you for the detailed reading, and for the offer to re-evaluate.

W1, Q1 — separating output restructuring from calls, resets and prefills

We agree this is the central mechanistic question, and we address it with three pieces of evidence rather than one.

Independent values per call, not answer length, carry most of the low-rank gain. The paper's two coarse-target controls separate those two factors, and we restate them here because they bear directly on your question. On the same 500 images, at rank 16: one-call full-length Expand12 (144 independent values written out as a 1,727-token raster) reaches 0.654, 0.646, 0.653 across seeds, against one-call Raw (576 independent values, same length) at 0.541, 0.553, 0.544. Shortening the answer is therefore not necessary for the gain; reducing the number of independently predicted values per call is what moves rank 16, and Expand12 does that without changing call count, context resets or visual prefills at all.

Correct answer history reverses the format ordering, which rules out a trivial "four calls are simply easier" account: with gold prefixes the chunked-minus-raw exact-value gap is −0.1588-0.1588 to −0.1575-0.1575, while under free generation it is +0.0110+0.0110 to +0.0175+0.0175 (twelve adapters, 200 development images, Appendix H.2).

Single context, four phases: the interaction largely survives. We decoded the trained grouped adapters with all four phases generated inside one conversation instead of four independent calls -- one image prefill, the answer history retained, the partition and the prompts unchanged -- at both ranks and all three seeds, on 200 development images. Both columns below are that same set of 200 images.

seedII, four callsII, one context
0+0.0344 [+0.0082, +0.0619]+0.0079 [-0.0194, +0.0352]
1+0.0399 [+0.0109, +0.0696]+0.0516 [+0.0211, +0.0818]
2+0.0621 [+0.0336, +0.0905]+0.0372 [+0.0065, +0.0678]

Under single-context decoding the interaction keeps an interval above zero in seeds 1 and 2; in seed 0 the interval contains zero, so on that seed it is not detectable this way. Relative to its own four-call estimate, the single-context estimate sits above it in seed 1 and below it in seed 2, with no consistent direction. These adapters never saw a retained history during training, so the effect is evidently not an artefact of the context reset.

The fidelity level does fall when the phases share a context, by 0.075 to 0.116 across the six cells, and that is not a parsing artefact: four of the six cells truncate nothing at all, and where truncation occurs it is in the fourth phase, where the retained context is longest. We report this as an exploratory diagnostic with its limit stated: a single-context decode is out of distribution for these adapters, so the remaining difference could come from the unfamiliar output shape, the retained history or the single prefill, and this design does not separate them.

Two further comparisons, K=2K=2 and single-call phase blocks, each require training a new output format (a new target serialisation, both ranks, all seeds), so we state them as the next experiments and say what each one fixes: K=2K=2 varies the call count while holding the partition scheme, and single-call phase blocks remove the reset while holding the call count.

On depth versus segmentation: the two tasks do differ in output length and encoding as well as in task, which is why the paper carries a within-task control. Expand12 holds the task and the number of independent values fixed and varies only the answer length, and the length axis alone does not explain the depth gain. A cross-encoding segmentation replication would separate encoding from task directly, and we name it as the next experiment.

W2, Q2 — ranks and training stages

Intermediate rank 64, both formats, three seeds, registered recipe. Both segments of the rank axis were registered before the runs: I(16 ⁣→ ⁣64)I(16\!\to\!64) and I(64 ⁣→ ⁣256)I(64\!\to\!256), per seed, with paired image bootstraps on 500 held-out images, strict parser.

seedFF Raw/r64FF Chunked/r64I(16→64)I(16\to64)I(64→256)I(64\to256)
00.6010.693+0.021 [+0.004, +0.037]+0.014 [−0.004, +0.031]
10.6110.687+0.026 [+0.012, +0.040]+0.005 [−0.010, +0.020]
20.6070.692+0.025 [+0.009, +0.040]+0.021 [+0.008, +0.034]

The interaction is present in the lower segment in all three seeds. In the upper segment it is smaller, and in two of three seeds we see no detectable difference. So the effect does not depend on comparing the extreme ranks: it is already there between 16 and 64, where the two ranks are close, and most of the aggregate interaction the paper reports is accumulated by rank 64.

Doubled training budget. Your question of capacity versus optimisation speed is answered by running the same recipe twice as long: 3,750 optimiser updates, a full epoch over the 60,000 training images, with the cosine horizon set at that endpoint, for all four format x rank cells, training seed 0. All contrasts are on the registered 1,000 confirmation images, strict parser, corrected-v2 affine δ1\delta_1, with 95% paired image bootstraps.

contrastvalue95% paired interval
I(1875)I(1875), same trajectory−0.0158-0.0158[−0.0295,−0.0025][-0.0295, -0.0025]
I(3750)I(3750), full epoch+0.0041+0.0041[−0.0071,+0.0155][-0.0071, +0.0155]
I(3750)−I(1875)I(3750)-I(1875), paired+0.0199+0.0199[+0.0049,+0.0348][+0.0049, +0.0348]

At the full epoch the primary metric shows no detectable interaction, and the shipped parser reading agrees (+0.0047+0.0047). The two registered companions do not move with it in the same way: ordinal accuracy gives +0.0066+0.0066 with an interval that excludes zero, and tail fidelity gives +0.0140+0.0140 with an interval that includes zero. Along this trajectory, the interaction changes by +0.0199+0.0199 from update 1,875 to update 3,750.

Where the negative value at update 1,875 comes from: with the horizon at 3,750, update 1,875 is a point where the cosine is only half annealed, whereas the paper's recipe completes its cosine at 1,875. A diagnostic arm, registered before we read it, trains the same four cells on the same trainer for 1,875 updates with the cosine completing at 1,875:

contrast (seed 0, same 1,000 images)value95% paired interval
I(1875)I(1875), cosine completed at 1,875+0.0353+0.0353[+0.0237,+0.0468][+0.0237, +0.0468]
minus I(1875)I(1875) of the half-annealed trajectory+0.0511+0.0511[+0.0356,+0.0671][+0.0356, +0.0671]

Both registered companions agree (ordinal accuracy +0.0242+0.0242, tail fidelity +0.0600+0.0600, both with intervals excluding zero). So, on the same trainer and at the same update count, completing the cosine restores a positive interaction, and the negative sign at update 1,875 of the long trajectory comes from its half-annealed intermediate state. This diagnosis is on training seed 0, the seed run for this arm, and its scope is the schedule; I(3750)I(3750) stands as the full-epoch result. The interaction is also not tied to one backbone: on InternVL3.5-8B, it is +0.0301+0.0301 on the same 1,000 images, with an interval excluding zero.

Q3 — end-to-end latency and compute

We measured it. One complete 24×2424\times24 map is one request; for the grouped format all four phase calls and the reassembly are inside the timer. Three arms are interleaved per batch index on one B200, in one process, after two discarded warm-up requests.

armadapter paramsprefillsp50 latency, 1 in flightp95s/map, 8 in flightpeak GiBfidelity
Chunked/r1629.1 M411.55 s11.69 s2.01 s23.50.642
Raw/r256465.6 M143.00 s43.29 s5.86 s20.70.644
Pooled12/r1629.1 M110.84 s11.03 s1.48 s19.00.648

Four prefills do not cost four times the latency; they cost less latency than one call does. Grouping splits the same 1,728 answer tokens into four shorter sequences that decode in parallel, so the sequential decoding path is a quarter as long: at one complete map in flight, Chunked/r16 is 3.7 times faster than Raw/r256, and at eight in flight it is 2.9 times faster, with a sixteenth of the adapter parameters and fidelity that differs by 0.002 on the same 1,000 images. Peak memory stays in one class, 19.0 to 23.5 GiB.

The claim is about the measured latency, throughput and parameter count, which separate the two configurations by factors of 3.7, 2.9 and 16; at one map in flight the 24 latency samples per arm have non-overlapping p95 ranges. The 0.002 fidelity gap is reported as the measurement it is, not as statistical equivalence, since this response registered no non-inferiority margin. Peak memory is measured with a single arm resident.

What the existing logs already show, as amortised throughput: on one host at a fixed eight sequences in flight, the amortised wall time per complete map is 6.00 s for Raw/r16, 5.94 s for Raw/r256, 5.70 s for Chunked/r16 and 5.63 s for Chunked/r256 (200 development images on one host, same day; the 1,000-image runs on the same host give 5.96 / 5.76 / 5.73 s). Four prefills do not cost more wall time per complete map there, because the phase answers are short and batch together. That number is amortised throughput at a fixed number of calls in flight, not a per-request latency, which is why we ran the controlled benchmark above.

Comments on the paper

1 comment

Dear reviewers, thank you for three reviews that converge on the same two questions: what the interaction is made of, and what it costs. We ran new experiments for both. All of them use the registered instrument (corrected-v2 affine δ1\delta_1, strict parsing as the primary reading with the shipped reading as a sensitivity check), report every seed separately, and never pool seeds into an image sample.

New experiments in this response

#WhatAnswers
1Intermediate rank 64, both formats, three seeds, registered recipeHmn1 Q2, RK6t Q1, Jvq5 W4
2Doubled training budget (3,750 optimiser updates, one full epoch), all four format x rank cellsHmn1 W2/Q2, RK6t Q1, Jvq5 W4
3Controlled cost benchmark: one complete map as the request unit, same GPU, interleaved arms, concurrency 1 and 8, latency / throughput / peak memoryHmn1 Q3, Jvq5 W2
4Second backbone (InternVL3.5-8B), four cellsRK6t W2
5Fine-detail analysis of the complete-target questionJvq5 W1, RK6t Q2
6Single-context diagnostic: four phases generated in one contextHmn1 Q1
7Downstream check on the full BLINK and CV-Bench sets, each item answered both through the arm's adapter and through the frozen backboneRK6t W2, Jvq5 W1

Detachment is exact, measured item by item. The paper supports detachment with parameter hashes and identical logits at a set of probes. The downstream experiment gives the behavioural version at full scale: with each adapter disabled for the answer, all three skills reproduce the frozen backbone's answer on every one of 3,160 benchmark questions, so accuracy returns to 0.7452 on BLINK and 0.8669 on CV-Bench exactly. Taking the goggles off leaves nothing behind.

The main result replicates on images that were never part of it. The paper reports the depth interaction on a 1,000-image confirmation set fixed before any main output was read. Rescoring the same trained adapters on a disjoint 500-image set gives the same interaction in every seed, under both parser readings:

seedII (strict, 500 images)II (shipped)paper's II (1,000 images)
0+0.0345 [+0.0170, +0.0520]+0.0362+0.0371
1+0.0312 [+0.0140, +0.0476]+0.0312+0.0312
2+0.0452 [+0.0289, +0.0614]+0.0452+0.0452

Intervals are 95% paired image bootstraps (10,000 resamples) conditional on each trained model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.