- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes Kite, a construction for specialising a frozen multimodal host with external evidence while preserving host behaviour exactly when the specialist is inactive. Frozen DINOv3-7B features are routed as side input, outside the host token sequence, to a shared 1.88B-parameter trunk attached at five MoE stations of a frozen Qwen3.6-35B-A3B host. Prompts carry markers; at each station, marker states query the evidence via cross-attention and the trunk output is written back additively at marker positions through zero-initialised interfaces. Marker-free inputs bypass the branch by control flow, giving bit-exact host preservation (Theorem 1), verified empirically (Δ=0), with prefix KV-cache locality and runtime removability. Training is three-stage: EO expert training on a generated corpus with the host's native vision off; native-path scheduling with a sigmoid gate; benchmark fine-tuning. Deployment invocation uses a prompt-level router; the learned gate is a diagnostic. On a pre-locked new-site test, Kite reaches 0.429 versus 0.258 for a full-chain-matched host LoRA (+9.8pp on corner identification, n=458); mismatched imagery drops accuracy to 0.106, supporting evidence dependence. Benchmark results: VRSBench 0.627→0.772 (seven-category 0.814 vs published Earth-OneVision 0.803), RSVQA-LR 0.913, RSVQA-HR below dedicated specialists, EarthBench agent gains +17/+22pp with mixed tool-trajectory metrics. The paper also reports matched-LoRA ablations (some paired comparisons significantly favour LoRA), evidence-mismatch controls, station-count ablations, a pre-registered negative result on progressive evidence consumption (falsified and reported), a channel-competition analysis motivating "establish then fuse", runtime and memory costs, and a rerun reproducing headline numbers.
Critical strengths
S1. Verifiable exact preservation. Inertness is enforced by control flow rather than by a zero-valued additive branch, so equality is bitwise on a fixed configuration; prefix locality means KV caches computed by the host remain valid for Kite and vice versa; removal is a runtime switch; the marker occupies an unused vocabulary row so logits are unchanged. The property is verified (Δ=0) across every trained model, and the one condition that can break it in deployment — router false-fire — is quantified rather than hidden. Auditability of the inactive path is a real, checkable deployment property that the adapter literature generally does not provide.
S2. Evaluation discipline well above field norm. The pre-registered negative result fixes acceptance criteria before the experiments and reports the falsification; LoRA controls are matched across the full training chain (corpus, stages, step budgets) rather than taken off the shelf; test manifests are locked before evaluation; paired bootstrap CIs are used throughout; a full-recipe rerun reproduces the headline numbers (0.763 vs 0.764; OA 0.845 vs 0.844); leakage audits found 21–41% image overlap with training data, which was fixed and disclosed; scoring protocols are separated and their swings documented. These choices materially raise my confidence in the reported numbers, including the unfavourable ones.
S3. Transferable empirical findings. First, channel competition: an evidence branch co-trained from scratch alongside a strong native vision path is absorbed (write-back scale 0.0918→0.0002), whereas establish-then-fuse preserves it (0.0996→0.0977), with controls (input dropout does not help; a fully trained expert keeps its scale). This is a concrete, mechanism-backed lesson for grafting new input channels onto strong hosts. Second, the pre-registered architecture hypothesis (progressive evidence consumption) was tested with reusable probes and shown false: stations behave as a single on/off unit, with real but unstructured per-station causality. Both findings outlast this particular instantiation.
Critical weaknesses
W1. The title and abstract sell a general construction; every quantitative result comes from one host (Qwen3.6-35B-A3B), one evidence encoder (DINOv3-7B), and one modality (optical EO imagery). No second host or second evidence type appears; SAR and multispectral are explicitly deferred, and the boundary conditions of the channel-competition finding (does establish-then-fuse matter only when the native path is strong?) are unexplored. As written, this is convincingly an EO-specialization system paper; the general-method framing is the least supported part.
W2. Zero-initialized gated write-back is Flamingo; side networks are Ladder Side-Tuning; invocation markers and prefix reuse are Activated LoRA. The paper itself concedes that conditional activation is not unique to Kite and locates the distinction in "one shared expert reading external evidence and host states at sparse depths." Theorem 1 is a control-flow identity. This would be acceptable if the delta were validated across settings, but with n=1 (W1) the methodological novelty is thin.
W3. The +9.8pp headline is corner identification, a 20-class task, on a 458-question subset; under matched benchmark rows and steps on VRSBench, the joint model wins under the official protocol (0.815 vs 0.786) but loses significantly under the authors' frozen-host judge with shared extractors (−0.035, CI excluding zero), RSVQA-LR counting is at parity with LoRA, and on RSVQA-HR the model trails published specialists by a wide margin. The paper reports all of this honestly, but no single unified protocol establishes superiority over matched LoRA, and official protocols here credit a large share of answers by substring matching.
W4. The pipeline requires a 2.27M-row generated corpus with twelve template families, machine-checked acceptance rules and version freezing, three-stage scheduling, a diagnostic gate, and a prompt-level router with its own failure modes (1.3% false-fire under an EO-like header on general prompts, which forfeits exactness in deployment). Nothing in the paper indicates this budget transfers to a new domain at a cost below simply LoRA-tuning the host. The practical case for the construction — exactness plus an auditable evidence pathway — is real, but the paper would be stronger if it claimed that value proposition directly rather than the broader one.
Questions for authors
Q1. Do any results exist on a second host or second evidence modality, however small (a different LLM host, or multispectral tiles)? Relatedly, do you have evidence on when establish-then-fuse is necessary, e.g., whether it matters when the host's native vision path is weak?
Q2. On the 932-question LoRA-comparison subset, how do Kite and matched LoRA compare on the non-corner fields? This would show whether the specialist advantage extends beyond the narrowest task. Also, most configurations are trained once; how sensitive are the small-n paired comparisons (n=458) are to the training seed?
Q3. In deployment, exact preservation holds only when the router withholds markers. What is the operational contract when the router fires incorrectly on a general request — is there an acceptance-time mechanism restoring host behavior, or is bit-exactness conditional on router accuracy?
2 comments
Appreciation for your review and feedback
We sincerely appreciate your supportive evaluation and valuable input, which has helped us further improve the quality of our manuscript. As the discussion period is drawing to a close, we remain happy to provide further clarification if helpful. Thank you again for your time and consideration!
Recognition Results, Preservation Guarantees, and Evaluation Scope
Thank you for the review. Given the time constraints of the rebuttal period, we prioritized adding non-corner results, evidence controls, and three-judge scoring across all 37,409 VRSBench questions.
Q1 / W1: Other hosts, modalities, and native-vision strength
Our study focuses on a 35B host and optical Earth Observation (EO) imagery. To directly investigate the interaction between visual pathways, we conducted fixed-checkpoint intervention experiments (§5.3, Appendix E.4). These controls characterize the dynamic interaction between the host visual path and the expert module during inference. Our current experiments isolate and validate the core fusion mechanism at the 35B scale; given time constraints and training costs, determining when the expert must be established prior to fusion is left to future work.
Revised paper: §5.3 (p. 9) and Appendix E.4 (pp. 21–22) describe the intervention experiments; §6 (p. 10) states the scope limitations.
Q2 / W3: Non-corner fields and training seeds
On 466 retained non-corner judgments, Kite answers 176 correctly (37.77%) versus LoRA's 162 (34.76%), an advantage of +3.00 pp.
Across all 924 retained judgments including corner identification (excluding eight data-leakage instances from the initial 932-question set), Kite answers 339 correctly (36.69%) versus LoRA's 280 (30.30%), an overall gain of +6.39 pp.
On 458 corner questions, Kite yields a +9.8 pp gain (35.6% vs. 25.8%); on an independent 198-question subset, the visual-path mismatch rate drops from 0.429 to 0.106.
Regarding reproducibility and seed sensitivity: our paired confidence intervals establish significance across evaluation samples. While 35B multi-seed training is computationally infeasible within the rebuttal period (stated in §6), §4.2 confirms that Kite's gains remain robust across training configurations.
Revised paper: §5.1 (p. 8) adds full non-corner counts; Table 3 (p. 9) separates the recognition subsets. §6 (p. 10) states the seed sensitivity limitation; §4.2 (p. 7) reports reproducibility results from the rerun.
Q3: Router false positives and operational guarantees
We distinguish Kite's architectural guarantee from its empirical routing behavior as follows:
Bitwise preservation: When the router inserts no evidence markers, the expert module is strictly bypassed. Under the stated runtime configuration, the system output remains bitwise identical to the base host model.
If a false-positive insertion occurs, the expert module is activated, returning a specialist prediction rather than the default host output. No mechanism currently checks the answer and restores the host output after an erroneous invocation. If a missed invocation occurs, the system returns the host model's own answer.
The deterministic bypass guarantee and empirical routing accuracy are two distinct concepts.
Paper locations: §3.3, Theorem 1 (p. 4); §4.4, "Autonomous invocation" (p. 8).
W2: Prior components and methodological contribution
Kite builds upon established techniques including zero-initialized branches, side networks, and conditional activation. It combines evidence outside the host sequence with a single shared expert that reads evidence and host marker states at sparse depths.
Its demonstrated value lies in enabling specialist adaptation with an explicit, auditable inactive bypass path, while providing empirical findings on visual-path competition. Under our tested criteria, we found no evidence of progressive depth specialization.
Paper locations: §2 (pp. 2–3); §3.2 (shared expert formulation); Appendix A (depth-specialization tests).
W3: Scoring sensitivity
To eliminate evaluation bias, we evaluated both raw and extracted outputs using three distinct LLM judge models (GPT-4o-mini, the frozen 35B host, and DeepSeek-V4.1-Flash): Kite consistently outperforms LoRA across all judges and response formats, with Avg10 improvements spanning +3.23 to +3.67 pp. This cross-model consistency confirms that Kite's advantage is robust to evaluator choice and answer formatting.For transparency, we retain protocol-specific results and comparisons with dedicated RSVQA-HR specialists.
Revised paper: §5.1 and Table 2 (p. 8); Appendix E.3 (pp. 20–21). Benchmark-specific results remain in §4.2 and Appendix G.
W4: Cost and practical positioning
The value of this method lies in providing an auditable external-evidence interface and exact preservation when idle. The expert module trains 1.88B parameters versus 84.7M for LoRA; estimated visual-path compute costs are 6.8–27 TFLOPs versus 3.8 TFLOPs for LoRA. LoRA is lower in cost. These cost metrics are intended to characterize deployment trade-offs rather than demonstrate superior adaptation efficiency.
Paper locations: §5.1 (p. 8); Appendix E ("What the graft costs").
We thank the reviewer again for the constructive comments and would be glad to address any further questions or concerns.