acceptodds
Under review as a conference paper at ICLR 2027

MoHA: Model-Conditioned Harness Adaptation for Long-Video Agents

Abstract

Long-video understanding increasingly relies on agentic systems that actively acquire evidence rather than process fixed video representations. Their execution harnesses must coordinate language-level planning with video-level observation and evidence acquisition, yet are typically fixed across model stacks. We therefore introduce MoHA, a framework for model-conditioned harness adaptation that adapts a common initial execution harness to the specific planner and observer models it hosts. MoHA treats the harness itself as a model-conditioned optimization object: under the same task distribution and initialization, different planner–observer stacks can benefit from different executable harnesses. Starting from a common initial harness, MoHA analyzes failed calibration traces to diagnose model–harness mismatches and adapts both planner-side reasoning support, such as overview, memory, and verification, and observer-side capability assignments, including OCR and ASR specialists. After structural adaptation, MoHA separately calibrates observer execution configurations for the hosted model stack. Across three frozen planner stacks, MoHA improves Video-MME Full accuracy by 6.2–27.1 percentage points and Video-Holmes accuracy by 3.1–20.1 points over the fixed initial harness. Cross-model transfer further shows that the adapted harnesses generalize only partially to other models, highlighting the harness as a model-conditioned optimization layer for long-video understanding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
long-video understanding, multimodal large language models, multimodal agents, agentic video understanding, agent harness adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ac4POpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose a framework MoHA to adapt models into different harness to improve their performance on long video understanding tasks. By analyzing execution trace, it select modules like memory, overview, verification, and some other configurations. Methods show performance gain in Video-MME benchmark.

Critical strengths
  1. The objective is clear: adapting the harness to the underlying models for specific downstream tasks.
  2. The model selection is appropriate, covering representative open-source Qwen models and a closed-source GPT model.
Critical weaknesses

1.Important baselines are missing, how does Qwen Code + Qwen3.5 and Codex+GPT5.6 perform on benchmark test cases? 2.The method mainly selects predefined modules and tunes FPS and input scale, and test on the validation set. Methodological novelty is limited, with few new insights. 3.Lack of in-depth analysis how/why some modules benefit certain models and some don’t. 4.Details on the configuration of the optimizer and judge models are missing.

Questions for authors

Please address the weakness mentioned above

Reviewer_Ad4xOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how the harness of a long-video agent should depend on the models it hosts. MoHA starts from a minimal harness that only has search and observe, with all requests sent to a general Omni observer. It then adapts the structure for a specific frozen pair of planner and observer. An LLM judge diagnoses failed trajectories on 64 calibration questions from Video-Holmes and may recommend one component out of overview, memory, verification, OCR and ASR. A candidate is promoted only if it improves accuracy on a 32-question validation set, run with two repeats, by more than 2 percentage points. After the structure is frozen, FPS and visual scale are selected from a grid of 3 by 3 settings. With GPT-5.5, Qwen3.5-27B and Qwen3.5-9B as planners and Qwen3-Omni as observer, Table 1 shows that MoHA improves Video-MME Full by 6.2, 7.9 and 27.1 points and Video-Holmes by 4.2, 20.1 and 3.1 points over the initial harness while using fewer frames. In a cross-model transfer study, the matched harness performs best in every row, as reported in Table 2. I thank the authors for a clearly motivated study. The observation that the same component can have opposite effects on different models is interesting, and the validation-gated procedure is a sound design. I do, however, have substantial concerns about the reliability of the experimental conclusions. In some settings the initial harness is close to or below chance, so the largest gains may mainly come from recovering valid answers. The observer-only baseline is missing. Adoption decisions rest on a very small validation set, with no variance reported anywhere in the paper.

Critical strengths

The problem is decomposed cleanly, and adoption is gated on validation accuracy. The paper separates which capabilities are exposed, the structure, from how perception is executed, the configuration, and on page 6 it lets held-out accuracy, not the judge, decide promotion. The observer probe in Appendix A.3, which tests whether a failure can be fixed by sampling settings before adding a specialist, is a thoughtful design choice. The paper shows that harness components interact with the hosted model. The intervention table in Figure 3b shows the same component changing sign across planners. Verification gives +3.1 for Qwen3.5-27B but -21.9 and -26.6 for the other two. Each matched harness is best in its row of Table 2.

Critical weaknesses

The baselines make it hard to attribute the gains. In Table 1 on page 7, Qwen3.5-9B with the initial harness obtains 22.8% on Video-MME Long, below the 25% chance level, and Qwen3.5-27B obtains 17.9% on Video-Holmes, close to chance for six options. Figure 3a attributes 38%, 58% and 24% of failures to Termination, which covers step budget, parsing failure and no answer. I am concerned that the largest gains of +27.1 and +20.1 mainly reflect recovering valid answers. In addition, the observer is never evaluated on its own. For Video-MME without subtitles, Table 9 of the Qwen3-Omni technical report lists 70.5 for Qwen3-Omni-30B-A3B-Instruct. That protocol differs from yours, so this is not a direct comparison. Still, the figure is above the initial harness for both Qwen3.5 planners, which reach 64.0 and 32.4 on Full, and close to the 71.7 of GPT-5.5 with the initial harness. It is also above the 59.5 of MoHA with Qwen3.5-9B. A corrected initial harness and an observer-only baseline under the protocol of the paper would resolve this. The statistical support for model-conditioning is limited. As listed in Table 6, promotion uses 32 questions repeated twice with a threshold of 2 percentage points, so about two extra correct answers decide a promotion. The observer configuration is then picked as the best of nine options on the same set. No results report seeds or variance. The validation estimates appear noisy. Verification is -21.9 on GPT-5.5 validation in Figure 3b, yet Static-All, which includes it, is only 1.6 below MoHA on test in Table 3, which covers one stack. MoHA exceeds Shared by 0.7 on GPT-5.5 Full, and on Video-Holmes Shared is ahead of MoHA for two stacks, 52.1 versus 51.9 with GPT-5.5 and 30.9 versus 29.4 with Qwen3.5-9B. Multi-seed results would make the central claim much more convincing, ideally with the adaptation itself repeated and Static-All run for all planners.

Questions for authors
  1. Under the initial harness, what fraction of trajectories end without a valid answer, and how are these scored? What does the initial harness reach if invalid outputs are replaced by a forced answer or a random guess, or if the step budget is relaxed? How large are the gains of MoHA relative to that baseline?
  2. It would strengthen the paper to report Qwen3-Omni and Qwen2.5-Omni answering Video-MME and Video-Holmes directly under your frame and audio budget.
  3. Could the authors report the mean and standard deviation over several seeds for Tables 1 to 3? How often is the same structure selected when adaptation is repeated?
  4. How should the -21.9 validation effect of verification for GPT-5.5 be reconciled with Static-All being 1.6 below MoHA on the test set?
  5. With only 32 possible structures, how does diagnosis-based ranking compare with random ordering or exhaustive search under the same validation gate? For GPT-5.5, Memory is promoted first although Figure 3a shows no Retention failures. Which model serves as the diagnosis and probe judge?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Ad4x (1/2)

Thank you for recognizing the validation-gated design and for identifying the baseline and statistical issues affecting our results. Below, we separately address answer completion, evidence use, and comparison with direct observers.

W1 / Q1 — Invalid answers, corrected baselines, and the remaining gains.

A1: We report end-to-end accuracy within 16 planner calls, scoring invalid answers as incorrect. Qwen failures mainly exhaust this budget during search/observe calls or end without a parseable option; after the paper’s terminal-answer rescoring, unresolved outputs are:

PlannerVideo-MME LongVideo-Holmes
Qwen3.5-27B135/900 (15.0%)997/1,837 (54.3%)
Qwen3.5-9B526/900 (58.4%)170/1,837 (9.3%)

On Video-Holmes, the final planner output of Qwen3.5-27B still contains an observe or search request in 1,117/1,837 trajectories (60.8%). At the 16-call limit, a new observe/search request leaves no remaining planner call to integrate further evidence and submit an answer. If the final output also lacks an extractable answer option, the episode is scored as invalid. Therefore, the high invalid-answer rate of 54.3% for Qwen3.5-27B on Video-Holmes is observed.

Random-guess baseline. Replacing only unresolved H0 outputs with uniformly random options gives these expected accuracies:

BenchmarkPlannerH0 (%)Expected H0 (%)MoHA (%)Gap (pp)
Video-MME LongGPT-5.566.266.277.3+11.1 pp
Video-MME LongQwen3.5-27B53.957.665.2+7.6 pp
Video-MME LongQwen3.5-9B22.837.451.1+13.7 pp
Video-HolmesGPT-5.547.747.851.9+4.1 pp
Video-HolmesQwen3.5-27B17.928.438.0+9.6 pp
Video-HolmesQwen3.5-9B26.327.829.4+1.6 pp

Random replacement reduces the gaps, but MoHA remains above adjusted H0 expectations.

Relaxed call budget. We completed 16-to-32-call continuations for all 526 originally invalid Qwen3.5-9B H0 trajectories on Video-MME-v1 Long: 57/526 (10.8%) produced valid answers, 34/526 (6.5%) were correct, and 469/526 (89.2%) remained invalid. Doubling the budget still fails to rescue most cases, suggesting limitations in the planner’s effective use of H0. The corresponding 27B continuations are running.

Q2 — Direct-answer Omni baselines.

A2: We evaluated Qwen2.5-Omni and Qwen3-Omni with one direct-answer request per question, using up to 128 uniformly sampled frames and 300 seconds of audio:

SystemVideo-MME LongVideo-Holmes
Qwen2.5-Omni direct41.1%40.1%
Qwen3-Omni direct51.1%51.2%
GPT-5.5 MoHA77.3%51.9%
Qwen3.5-27B MoHA65.2%38.0%
Qwen3.5-9B MoHA51.1%29.4%

Qwen3-Omni matches 9B MoHA on Long and exceeds both Qwen planner stacks on Holmes. Video duration may partly explain this pattern:

BenchmarkMean (s)Median (s)Questions by video duration
Video-Holmes18519099.56% involve videos ≤300 s
Video-MME-v1 Long2,4662,423897/900 involve videos >30 min

The 300-second audio budget covers nearly all Holmes videos but only part of a typical Long video, making direct answering particularly competitive on Holmes.

Longer videos demand targeted observation and evidence integration across distant segments, motivating MoHA’s harness adaptation. Performance also depends jointly on the planner, observer, and harness: a strong observer cannot ensure accurate answers if the planner fails to invoke it appropriately or integrate its evidence. We will add these baselines and analysis to the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Ad4x (2/2)

W2 / Q3 — Repeated adaptation, Shared, and uncertainty.

A3: Adaptation across data splits. We reran full Qwen3.5-9B adaptation and evaluated each selected harness on three new random calibration/validation splits, with video-disjoint calibration and validation sets:

SplitModulesFPS/scaleLong accuracy
S0Overview + Memory + ASR2 / 0.7550.7%
S1Overview + Memory2 / 0.7550.4%
S2Memory + ASR0.5 / 0.546.7%
Original MoHA (reference)Overview + Memory + ASR2 / 0.7551.1%

S0/S1 remain close to the original result; S2 drops further, indicating split sensitivity. Memory appears in 3/3 runs, Overview and ASR in 2/3 each, and Verification in 0/3; each complete structure appears once. The corresponding 27B runs are ongoing.

Validation-set reliability. MoHA limits search to at most six rounds and two candidates per round, requiring an accuracy gain >2 percentage points averaged over two evaluations on the 32-question validation set. We also reran the full adaptation on three new splits to assess selection reliability empirically, as reported above. We will discuss the observed split sensitivity and limited validation size in the revised Limitations section.

Repeated benchmark evaluation. We are rerunning the full benchmark in dataset order with GPT-5.5 MoHA fixed. The table compares original and repeated scores on the same 200 questions per duration subset; Long200 is the first 200 of Long900, and Full aggregates these 600 questions:

EvaluationShort (200)Medium (200)Long (200)Full (600)
Original run80.0%79.5%88.5%82.67%
Repeat run81.5%78.5%89.0%83.00%
Change+1.5 pp−1.0 pp+0.5 pp+0.33 pp

The +0.33-point overall change suggests preliminary stability on this subset. The full three-run study for Tables 1–3 remains ongoing.

Comparison with Shared. MoHA does not always outperform it, but gains on Long are consistent: +5.2/+11.5/+9.0 points for GPT-5.5/27B/9B. This matches our long-video focus on acquiring and integrating evidence across distant segments.

Static-All across planners. We completed Video-MME-v1 evaluations for all three planners (accuracy, %):

PlannerShortMediumLongFull
GPT-5.571.066.975.771.2
Qwen3.5-27B67.650.837.652.0
Qwen3.5-9B59.348.644.350.7

MoHA has higher Full accuracy for all three planners, supporting selective adaptation over enabling all modules in these settings. We will add these results to the paper.

Q4 — Verification, Static-All, and the interpretation of Figure 3.

A4: Figure 3(b)’s −21.9-point effect is a local validation change from adding Verification to GPT-5.5; Static-All 75.7% vs. MoHA 77.3% (1.6 points) compares complete harnesses on Video-MME-v1 Long. Static-All enables all modules/specialists and calibrates observation settings, so structure, configuration, and evaluation data differ; module interactions can further change the gap. GPT-5.5’s strong reasoning may compensate for harness limitations, consistent with H0 already reaching 66.2% on Long.

Q5 — Search controls, the first Memory promotion, and judge identity.

A5: Random-search comparison. We completed three 9B random-proposal adaptation and benchmark runs (S0–S2), keeping MoHA’s intervention library, two-repeat validation protocol, and promotion gate. We randomized proposals, not merely diagnosis-derived rankings:

ExperimentSelected structureVideo-MME Long accuracy
MoHA (Qwen3.5-9B)Overview + Memory + ASR51.1%
S0 RandomOverview46.1%
S1 RandomMemory45.6%
S2 RandomMemory + OCR44.1%

27B controls are ongoing.

Exhaustive search. Validation requires 32 structures × 32 questions × 2 repeats = 2,048 trajectories, or 18,432 including all nine observer configurations, exceeding our candidate-validation budget.

Memory promotion and Retention failures. Figure 3(a) reports category shares among failed H0 calibration trajectories, not benchmark-wide invalid rates; the votes below come from GPT-5.5 adaptation. Invalid outputs are assigned to Termination. The judge recommends interventions from actual traces, without a fixed Retention-to-Memory mapping; held-out validation decides promotion. Thus, zero plotted Retention failures do not imply zero Memory recommendations. We thank the reviewer for flagging this ambiguity and will correct the data-source and aggregation descriptions.

Specifically, in GPT-5.5’s first round, Memory and Verification each received 9/18 valid votes; a deterministic tie-break tested Memory first. Memory was accepted at +4.7 points; Verification was rejected at −21.9.

Diagnosis and probe judge. We used frozen claude-opus-4-8 on calibration data, separately from the planner/observer.

Reviewer_ASMiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoHA, an offline procedure for adapting the execution harness of a frozen long-video planner–observer stack. An LLM diagnoses failed calibration traces and recommends additions from five predefined components. A voting rule orders the candidates, and held-out validation determines which additions to retain. After component selection, the procedure searches nine frame-rate and input-scale configurations. Experiments on Video-MME and Video-Holmes show improvements over a common initial harness, while cross-model transfer suggests that the preferred harness depends on the hosted models.

Critical strengths
  • The paper studies a practical question. Using the same initial harness and adaptation data across model stacks makes the experiments easier to interpret.

  • The cross-model transfer experiments are useful. Table 2 provides direct evidence that the hosted model and the selected harness interact, with the matched configuration performing best in each evaluated row.

  • The method improves the evaluated frozen model stacks without fine-tuning. The separation between adaptation data and benchmark test data is appropriate, and the reported reduction in sampled frames is encouraging.

Critical weaknesses
  • To my understanding, MoHA is mainly a configuration search over five predefined component switches and nine observer settings. The LLM recommends candidate additions, while the actual selection is determined by validation accuracy. The model-conditioned diagnosis–proposal–validation direction is already central to Self-Harness. MoHA adds observer-side choices, but the additional value of its particular search procedure is not established. Why is the LLM diagnosis needed in this small search space? A greedy or random search run separately for each model would also produce model-conditioned configurations. The authors should compare these alternatives using the same component library, data, and adaptation budget. Initial, Shared, and Static-All do not isolate the benefit of diagnosis and voting.

  • The selected configurations may be sensitive to the small validation set. Only 32 distinct examples are used, with two evaluation repeats, and the same set is repeatedly used for component selection and the final nine-way observer calibration. A gain of 3.1 percentage points corresponds to only two additional correct outputs across 64 evaluations, yet it passes the promotion threshold. Would another calibration/validation split select the same components? Repeated adaptation runs and uncertainty estimates are needed, particularly for the 1.6-point advantage over Static-All in Table 3.

  • The two-stage search design is not sufficiently justified. Components are selected under the initial 1 FPS/full-scale observer configuration and are never reconsidered after the observation settings change. However, changing frame rate or resolution can change whether overview, memory, or a specialist is useful. Similarly, the specialist probe only shows that some alternative configuration can rescue one observation; it does not establish that the final selected configuration will rescue it. The current ablations remove entire adaptation surfaces, but do not test the specialist gate or the ordering of the two stages. Would interleaved component–configuration search achieve similar or better results?

  • The interpretation of Figure 3 needs more care. The reported intervention effects are measured from different current harness states along different adaptation paths. For example, Overview is added first for Qwen-27B but after Memory for Qwen-9B. The sign changes may therefore reflect component interactions as well as differences between models. Table 2 supports model dependence of the complete harness, but does not establish that each individual component has the claimed model-specific effect. A common reference structure would make this analysis more convincing.

  • The cost of finding and executing the selected harness is not sufficiently reported. Frames/Q captures visual sampling, but does not include ASR processing, auxiliary model calls, token usage, or latency. The adaptation stage also involves repeated trace collection, diagnosis, probing, and validation. Since the proposed method adds these operations to a small configuration search problem, the authors should report total adaptation cost and compare search efficiency with simpler alternatives. Deployment cost should also be measured beyond sampled frames.

Minors:

  • Algorithm 1 collects recommendations using a set and set union, but vote counting requires preserving repeated recommendations. A list or multiset would be more appropriate.

  • Algorithm 1 does not explicitly enforce the two-candidates-per-round limit in Table 6. Please make the pseudocode consistent with the reported procedure.

Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer ASMi (1/2)

We Thank the reviewer for the careful review and for recognizing the practical value of frozen planner–observer harness adaptation. We provide our responses below.

W1 — Contribution and comparison with simpler search procedures.

A1: MoHA uses diagnosis to prioritize candidate interventions; held-out validation determines promotion. As discussed in Appendix C, MoHA extends Self-Harness’s model-conditioned, trace-driven perspective to long-video agents with learned perceptual backends and explicit observer-side adaptation. The contribution is its application to the structure and observation settings of frozen long-video planner–observer stacks, together with the empirical analysis of their interactions.

Random-search comparison. We completed all three Qwen3.5-9B random-proposal control experiments, including adaptation and benchmark evaluation, using the same component library, adaptation data, validation protocol, promotion rule, and observer-configuration budget. The results are:

RunSelected modulesVideo-MME Long accuracy
MoHA, Qwen3.5-9BOverview + Memory + ASR51.1%
S0 RandomOverview46.1%
S1 RandomMemory45.6%
S2 RandomMemory + OCR44.1%

S0–S2 denote three random-proposal adaptation runs; MoHA exceeds all three by 5.0–7.0 percentage points, supporting diagnosis-guided candidate selection under this protocol. The corresponding Qwen3.5-27B control is in progress.

Expanded module library. Each module family can contain multiple implementations, while the current paper uses one basic implementation per family. We are testing an expanded library to examine adaptation beyond these initial choices and have completed the 9B expanded-library experiment, which selects Memory + Overview + AVP-style Reflect [1] with 2 FPS and input scale 0.5. This configuration achieves 52.2% accuracy on the 900-question Video-MME Long test set, compared with 51.1% for the original MoHA (+1.1 percentage points), supporting the procedure’s applicability to an expanded module library. The corresponding 27B expanded-library experiment remains in progress. This experiment examines extensibility, while the random-proposal control addresses the value of diagnosis-guided search.

[1] Ziyang Wang, Honglu Zhou, Shijie Wang, Junnan Li, Caiming Xiong, Silvio Savarese, Mohit Bansal, Michael S. Ryoo, and Juan Carlos Niebles. Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding. arXiv preprint arXiv:2512.05774, 2025.

W2 — Sensitivity to the validation split and statistical uncertainty.

A2: We reran full Qwen3.5-9B adaptation and evaluated the selected harness for each of three new random calibration/validation splits, each video-disjoint with 64 calibration questions and 32 validation questions evaluated twice:

SplitModulesFPS/scaleLong accuracy
S0Overview + Memory + ASR2 / 0.7550.7%
S1Overview + Memory2 / 0.7550.4%
S2Memory + ASR0.5 / 0.546.7%
Original MoHA (reference)Overview + Memory + ASR2 / 0.7551.1%

S0 and S1 remain close to the original MoHA result, while S2 shows a larger decrease, indicating sensitivity to the adaptation split. This split sensitivity remains a limitation of MoHA, which we will explicitly discuss in the revised Limitations section.

Across S0–S2, Memory is selected in all three runs, Overview and ASR in two runs each, and Verification in none. The corresponding Qwen3.5-27B runs are in progress.

Evaluation stability with a fixed harness. We are rerunning the full benchmark in dataset order with GPT-5.5 MoHA fixed. The table compares original and repeated scores on the same 200 questions per duration subset; Long200 is the first 200 of Long900, and Full aggregates these 600 questions:

EvaluationShort (200)Medium (200)Long (200)Full (600)
Original run80.0%79.5%88.5%82.67%
Repeat run81.5%78.5%89.0%83.00%
Change+1.5 pp−1.0 pp+0.5 pp+0.33 pp

Overall accuracy changes from 82.67% to 83.00% (+0.33 percentage points); the full three-run study remains in progress. These preliminary results suggest stable aggregate performance across the two evaluations on this subset.

W3 — Two-stage ordering and the specialist probe.

A3: Two-stage ordering. Interleaved structure–configuration adaptation on Qwen3.5-9B selected Memory at 2 FPS and original input resolution (scale 1.0); benchmark evaluation is ongoing.

Specialist probe. The probe guides candidate generation but does not guarantee that the final observer configuration resolves the same observation failure; held-out validation separately determines promotion.

AuthorsOpenReview ↗

Response to Reviewer ASMi (2/2)

W4 — Common-reference intervention analysis.

A4: Figure 3(b) shows conditional effects along adaptation paths; different reference states confound model differences with component interactions. We evaluated both additions from a common Memory structure on the same 32 video-disjoint validation questions, with two repeats, fixed observation settings, and identical budgets.

PlannerMemoryMemory + OverviewMemory + Verification
Qwen3.5-9B24/6428/64 (+6.25 pp)7/64 (−26.56 pp)
Qwen3.5-27B27/6433/64 (+9.38 pp)24/64 (−4.69 pp)

Conditioned on this reference, Overview has positive point estimates for both stacks, while Verification is more negative for 9B. Figure 3(b) aims to highlight these model–module and module–module interactions: a module’s benefit depends jointly on the hosted model and the current harness and is not a fixed, additive contribution. We will label reference conditions and distinguish original adaptation trials from additional fixed-candidate validation.

W5 — Adaptation and deployment costs.

A5: Frames/Q covers only visual sampling; we report offline adaptation and deployment separately, including auxiliary calls, tokens, and audio. Each adaptation run has 384 calibration and 896 validation episodes: 320 structural validations plus 576 for the nine-configuration observer sweep.

Main-execution totals and per-episode averages (P/O = planner/observer; K = 1,000 tokens):

PlannerP/O calls, totalText tokens/episodeVideo tokens/episodeAudio s/episode
GPT-5.511,461 / 6,68332.7K16.4K153.8
Qwen3.5-27B16,597 / 9,82243.6K18.8K152.1
Qwen3.5-9B16,685 / 7,49425.9K8.36K63.5

Auxiliary calls and average tokens/call:

PlannerDiagnosis callsTokens/callProbe callsProbe text + video tokens/call
GPT-5.514862.5K54.00K + 1.65K
Qwen3.5-27B26247.1K443.54K + 1.35K
Qwen3.5-9B27955.1K142.90K + 0.89K

Diagnosis counts include retries; 9B probes are embedded calls, reported separately to avoid double-counting.

Deployment per question:

PlannerP/O calls/QText tokens/QVisual tokens/QAudio s/Q
GPT-5.55.87 / 4.9357.47K6.60K264.90
Qwen3.5-27B5.71 / 3.4538.80K23.21K111.39
Qwen3.5-9B4.43 / 1.9721.23K9.51K22.74

These tables show that offline adaptation incurs additional diagnosis, probing, and validation costs, while deployment calls and resource usage vary across planner stacks. We will include these cost tables in the appendix of the revised manuscript.

M1–M2 — Algorithm specification.

A6: Thank you for identifying these inconsistencies.

M1: Duplicate votes. We will replace set union with list append or a multiset to preserve duplicate votes, as the implementation already retains one vote per valid failed trajectory.

M2: Candidate limit and stopping rule. We will specify L = 2 (Table 6) and align the stopping condition with the implemented patience rule and six-round cap.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.