acceptodds
Under review as a conference paper at ICLR 2027

EKT-Router: Expert–Evidence Alignment for Embodied Vision–Language Models

Abstract

Embodied vision-language models must answer questions from partial, evolving observations. Their success depends on both the reasoning capability selected and the evidence available to it. Expert routers adapt internal computation, while tool-augmented systems acquire external evidence; choosing these independently can leave a suitable expert without the evidence it needs. We introduce EKT-Router, a bounded framework that jointly selects a capability expert, optional knowledge, and perceptual tools. Outcome-derived signatures represent the overlapping strengths of General, Spatial Reasoning, and Spatiotemporal experts. A joint utility scores complete expert–evidence routes under a call budget. After acquisition, a packet verifier admits valid evidence with positive predicted expert-conditioned gain. The selected expert then answers once, reverting to the unchanged observation if all packets are rejected. Across 6 embodied QA benchmarks, EKT-Router achieves a mean score of 61.4, the highest among the compared systems, and averages 74.7 on 10 general vision–language benchmarks. With all external evidence disabled, question-conditioned expert selection alone improves over the best fixed expert by 1.75 points. Ablations show that learned routing is the dominant component and that knowledge and perception are complementary.

Primary area:
foundation or frontier models, including LLMs
Keywords:
embodied vision-language models, expert routing, tool-augmented reasoning, evidence selection, mixture of LoRA experts, embodied question answering, spatial reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6WwiOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EKT-Router, a framework that jointly selects reasoning experts and external evidence for embodied vlm. It introduces outcome-derived expert signatures, a joint utility-based routing mechanism, and a packet verifier to filter acquired evidence.

Critical strengths

The paper evaluates the framework across 16 benchmarks, including embodied and general vision-language tasks. The results demonstrate competitive performance against strong baselines.

Critical weaknesses

1.The framework combines established techniques, including expert routing, utility prediction, and evidence filtering. The methodological novelty beyond their integration is not sufficiently clear. 2. The contributions of the expert–evidence interaction term and packet verifier are not independently evaluated. Additional ablations are needed to establish the effectiveness of these key components.

Questions for authors

Although sequential expert-then-evidence routing has not been widely explored in embodied agents, similar approaches have been studied in other domains. How does the proposed joint routing framework demonstrate its advantages over these sequential methods?

[1]RouterRetriever: Routing over a Mixture of Expert Embedding Models

Reviewer_iiAqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies embodied question answering, where a model has to answer questions about a scene from a single image, a few frames, or a video. The authors argue that two decisions are usually made separately but should be made together: which specialized model answers the question, and what extra evidence (retrieved knowledge, tool outputs) that model receives. Their argument is that the value of a piece of evidence depends on which expert is consuming it.

The method, EKT-Router, uses three LoRA experts on a frozen Qwen3-VL-8B (general, spatial, spatiotemporal). A frozen Qwen3-VL-2B encodes the query, and a small learned head (about 1.2M parameters) scores every combination of expert, knowledge option and tool subset (Grounding DINO, Depth Anything V2, OCR). The score is a sum of four parts: an expert term built from "capability signatures" measured on diagnostic probes, a shared evidence term, an expert-evidence interaction term, and a cost penalty. The router is trained with a Bradley-Terry loss on pairs of executed routes, ranked by answer accuracy. After the tools run, a verifier picks which returned outputs to keep, with a fallback to no evidence at all. The chosen expert then answers once.

Validation covers six embodied QA benchmarks (61.4 average, reported as best among the compared systems), a ten-benchmark general VL suite (74.7), a no-evidence control where per-question expert selection gains 1.75 points over the best fixed expert, and ablations. According to the ablations, the router contributes the most and knowledge and tools help more together than alone.

Critical strengths
  1. The no-evidence control in Q2 is well designed. With knowledge and tools turned off for every candidate, per-question selection still beats the best single expert by 1.75 points, with a media-grouped bootstrap CI of [1.25, 2.26]. This is the cleanest result in the paper and it convinces me the experts are actually complementary, which is the precondition for everything else.

  2. The inference procedure is bounded and auditable. It makes a fixed number of provider calls (at most four), generates one answer, and always has an exact fallback to the expert's no-evidence path. For robotics deployment this matters more than a point or two of accuracy, and it is a real advantage over open-ended agent loops.

Critical weaknesses
  1. The paper's central claim is that expert and evidence have to be chosen jointly, but the parts that implement this are never tested in isolation. The interaction term Γ is what makes evidence value expert-dependent, and the packet verifier is the second main contribution. Neither has its own ablation, and Appendix B.3 says so directly. The existing ablations remove the whole router or whole evidence channels. That shows "a learned router plus tools helps." It does not show that the joint formulation helps beyond simpler alternatives, such as choosing the expert first and then the tools, or letting the model call its own tools under the same budget. Without these comparisons the main novelty claim is not supported.

  2. The baseline comparisons are hard to trust. In Table 1, Qwen3-VL-8B gets 42.9 on MMStar, 46.7 on AI2D and 32.8 on MathVista (64.6 average), which is below the older Qwen2-VL-7B and well below what I would expect from the official report. Every expert is built on this backbone, so if the baseline is mis-evaluated, the "no loss of generality" story falls apart. Separately, the newest proprietary baselines are Gemini-2.5-Pro, o3 and GPT-4o, while stronger models (e.g. the Gemini 3 and GPT-5 families) were public for months before the deadline. The six-benchmark comparison also mixes reproduced, API and copied numbers, and some margins are small (+0.4 on RoboVQA, +0.9 on OpenEQA). Together these make the "surpasses larger and proprietary systems" claim weak.

Questions for authors

Clarifications:

  1. How was Qwen3-VL-8B evaluated in Table 1, and why are its numbers so far below the official report? Which baseline entries in Figure 3 and Table 1 are reproduced, which are obtained via API, and which are copied from other papers?
  2. Can you add the following on the same checkpoints: (a) the additive utility without Γ, (b) a sequential selector that picks the expert first and then the tools, (c) the full model without the packet verifier, and (d) a single merged LoRA trained on the union of the expert data, given the same tools? These are what would show that the joint formulation, rather than tools plus stronger fine-tuning, is responsible for the gains.
  3. What is "w/o Router-Adapter" in Figure 5 exactly? Is it a fixed General expert, and are tools still on? Also, what benchmarks and protocol does the Q3 "separate frozen protocol" use, given that its 60.2 differs from Q1's 61.4?
  4. What is the oracle-routing upper bound, and how often does the router pick the best route? This tells us whether the frozen 2B descriptor plus a 1.2M-parameter head is limiting the result. Did you try a larger or fine-tuned descriptor, or reusing hidden states from the 8B backbone, which is already loaded?
  5. Section 4.6 calls the controller cost negligible, but it counts only the trainable heads. What are the end-to-end latency and compute, including the 2B descriptor pass and the tool calls?
  6. Table 2 says the deployed adapters use rank 128, but the sensitivity sweep in Figure 5 goes from 4 to 32 and peaks around 16. What did the sweep vary?
  7. Appendix A.1 says "General can also be paired with Knowledge or Tools." Can the spatial and spatiotemporal experts receive evidence too?

Suggestions:

  1. Please add at least one current frontier model to both result tables, ideally with the same tools under a matched call budget. If EKT-Router doesn't beat it, framing the paper around the small on-device regime, with latency numbers, would be a fair positioning.
  2. Results on very recent models released close to the deadline (e.g. GPT-6 Astra, Claude Opus 5.5) would be nice but are not expected.
Reviewer_ENVSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work addresses embodied QA task from partial observations with the proposed EKT-Router framework that jointly selects reasoning expert and external evidence. EKT-Router utilizes 3 LoRA experts trained for general, spatial and spatiotemporal reasoning on a shared frozen Qwen3-VL-8B-Instruct backbone. Each trained expert is represented by an outcome-/probe-derived signature to select expert knowledge and perceptual tools with a learned utility function. A packet verifier then evaluates acquired evidence before the selected expert answers. Experiments across 6 EQA benchmarks report an average score of 61.4, with additional experiments ablating expert routing, knowledge and perception.

Critical strengths

S1. The study identifies a relevant limitation of embodied QA systems. Jointly selecting reasoning experts and external evidence is a meaningful approach to handling different reasoning demands.

S2. The framework scores a complete expert–evidence tuple and separates acquisition from acceptance with an exact evidence-free fallback, which is a practical approach for bounded tool use

Critical weaknesses

W1. The main contribution of the expert and evidence alignment is not sufficiently validated. No experiment isolates the gain from the interaction term Γ or the Packet Verifier. Ablations for B + G and B + G + Γ should be provided to isolate the contribution of Γ.

W2. Figures 3 and 4 suggest that individual experts outperform the complete system on several benchmarks. For example, the Spatiotemporal expert scores 66.15 on VSI-Bench and 66.18 on RoboVQA, compared with 64.1 and 63.0 for EKT-Router. These raise concerns about the effectiveness of the proposed routing strategy. The study needs matched comparisons against fixed experts, independent expert and evidence selection, and routing without verification. The verifier’s acceptance behavior and calibration are also not evaluated.

W3. The cross-system comparison in Figure 3 cannot be verified. Each panel shows three comparators chosen by an unstated rule. Although L322 says all three are mixed, and no comparator’s six-benchmark mean is given, so ”highest among the compared systems” at L022 and L350 is uncheckable.

Minor Weaknesses:

  1. Continued pretraining (CPT) is mentioned at L076 to 077 without proper description of the data, parameters and objectives utilized. Without a description of the CPT, it is not clear whether General expert is a meaningfully different model from the backbone, and hence interpreting Findings 1, 3 or 7 becomes difficult.

  2. Table 1 shows performance of InternVL2.5-8B (avg. 74.4) and Qwen2-VL7B (avg. 72.7) higher than Qwen3-VL-8B (avg. 64.6) model on general vision–language evaluation. Despite the performance gap, the authors selected Qwen3-VL-8B, which needs explanation.

  3. L305 cites InternVL3.5 to the InternVL3 entry at L641.

  4. Please correct writing and grammatical issues in the manuscript. B is used for Bϕ, BT , Bext and the set B(a∗); the relation ⪯ is used at L243 before ts definition at L258; Bext is used at L269 before L283. No space after ”1” and ”2” at L049 and L051. ”Judgement” at L883.

  5. Sections 3.2–3.5 omit several details necessary to understand and reproduce the method. The number and composition of capability probes, signature dimension (r), score normalization, and learning procedure for projection (P) are not specified. The calibration loss (Lcal) is described but not explicitly formulated. Moreover, the values of (τ ,ϵ,λ,α,γ), the configured budget, and individual action costs are not reported. The paper states that these parameters are selected using held-out data but does not explain the selection criteria. The cost terms also need a clear definition and units to isolate measured computational costs from tuned utility penalties.

Questions for authors
  1. Could the authors independently evaluate the interaction term (Γ) and Packet Verifier? In particular, how does the complete method compare with additive utility and simpler evidence-filtering strategies?

  2. Are Figures 3 and 4 evaluated under identical protocols? Could the authors please report matched results for all three fixed experts and the complete system across the six primary benchmarks?

  3. How frequently does the Packet Verifier accept or reject acquired evidence? Could the authors also report its calibration results, as discussed in Appendix A.3? Flag For Ethics Rev

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.