- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper models test-time decision-making with a latent state S, a goal G, base observations X_base, and an interface statistic Z that a test-time procedure conditions on. Proposition 1 telescopes excess risk into three terms:
- computational: R(δ_K) − R*(Z,G);
- representation: R*(Z,G) − R*(X_base,G);
- observation: R*(X_base,G) − R*(S,G).
Under log loss, the last two sum to I(Y;S|Z,G), which splits by the chain rule. Theorem 1 has three parts:
- Zero "decision deficiency" for all bounded losses holds exactly when the posterior is sufficient: p(y|Z,G) = p(y|S,G).
- The log-loss deficiency equals I(Y;S|Z,G).
- For bounded losses, the deficiency is bounded above via Pinsker, with a Fano lower bound for deterministic finite-label tasks.
Two further results motivate "bottleneck matching":
- Proposition 2: sampling more from the same state yields no new information (conditional data processing).
- Proposition 3: each operation class moves only its own term's floor. The classes are compute; fuse, which re-encodes within σ(X_base,G); and acquire, which expands the information set.
Bottleneck matching spends the next unit on whichever operation has the largest estimated risk reduction minus λ·cost, and stops otherwise.
ReLATE implements this as a one-step controller over a frozen VLM or VLA policy. Two pilot samples produce cheap features, and a learned router picks one of three arms:
- compute: majority vote over K fresh samples;
- acquire: a probe ranked by decision-relevant information gain (DRIG);
- stop.
Fuse is validated separately rather than routed, and the iterative form is not evaluated (L346–350). InfoGap has three parts:
- Exact factored POMDPs, where every Bayes risk and mutual information is computed by enumeration.
- Procedural two-view vision scenes. The families are occlusion, before/after, relational XOR, targeted probe, a tunable occlusion-leak family, and a compute-limited hard-count family. A constraint solver verifies single-view Bayes accuracy ≤ 0.6 and joint ≥ 0.95.
- Ancillary wrist-camera masking on OpenVLA-OFT in LIBERO.
Experiments cover:
- the information ceiling (Table 1, Fig. 2);
- fusion at fixed evidence (Fig. 7);
- decision-relevant vs. generic acquisition (Fig. 8);
- the router on a held-out pool of 1,800 instances (Tables 4–5);
- cross-model transfer (Table 8);
- embodied sensitivity (Table 6).
Critical strengths
The decomposition gives a clean vocabulary for something test-time-scaling papers routinely blur: evidence the system already holds but discards (fixable without acting on the world) versus evidence it never observed. The formal statements are correct under carefully stated assumptions. The paper uses conditional independence (Y ⊥ X_base | (S,G), ξ ⊥ (S,Y) | (Z,G); L211–214) rather than a naive Markov chain. Theorem 1 usefully ties the deficiency across loss families to a single conditional mutual information.
The best experiment is the occlusion-leak sweep. A real VLM's sampling plateau tracks a designed Bayes bound within 0.02 at four levels, which cleanly separates an information ceiling from a capability limit. The effect also runs in both directions:
- On synthetic Task D, Best-of-K goes from 0.494 to 1.000 while exact-DRIG probing stays at 0.494 (Table 3).
- On the compute-limited slice of the VLM pool, always-acquire gets 0.093 where the router gets 0.865 (L1234–1236).
Opening all four probes (0.793) is worse than opening the one decision-relevant probe (1.000), a nice demonstration that acquisition has to be targeted.
The 2026 works it positions against (e.g., Li & Shi; Sevetlidis 2026a/b; Jeong & Yoon) exist and are described accurately. The related-work section is direct about which identities are not new.
Critical weaknesses
Theory. The theoretical contribution is thin for a paper submitted under learning theory:
- Proposition 1 is a telescoping sum (L195).
- Theorem 1(ii) is the identity "log-loss Bayes risk = conditional entropy," which the paper credits to Sevetlidis 2026b (L157–160).
- Theorem 1(i), called "the nontrivial content" (L234), reduces to Brier-score propriety plus the tower property and the law of total variance (L897–913).
- Proposition 2 is conditional data processing.
- Proposition 3(ii) is immediate, because Z′ = X_base is admissible.
- Proposition 4 is Howard/Lindley value of information.
There is no generalization, sample-complexity, or rate result. The "decision deficiency" terminology is not related to Le Cam's deficiency or to the Variational Deficiency Bottleneck (Banerjee & Montúfar). "Fuse" is not positioned against the literature on early and late multimodal fusion.
Evidence. More importantly, the empirical tests of P1–P3 mostly replay how the benchmark was built:
- The solver enforces chance-level single views and near-perfect joint views.
- Each XOR view carries zero bits (L1074), so per-view late voting must sit at 0.5.
- Acquisition hands the model the missing view.
- The trivial "All views (K=1)" baseline already scores 1.000 everywhere at 64 tokens (Table 1).
Compute is only ever scaled over the starved single view. That cannot distinguish "sampling cannot help" from "sampling a view with zero decision information cannot help," and the second is true by definition.
The flagship P2 result (0.500 → 1.000 at fixed evidence) depends on the paper's taxonomy. A two-axis view could just as well call "one joint prompt instead of per-view votes" a better use of inference over evidence already held. The additive-vs-parity control (E.6) attributes the gain to relational structure, but it does not settle this. The paper itself concedes the result is "a restricted-class gap, not a bound on representation regret" (L445–446). Yet the abstract says "acquisition cannot explain" it and calls it a prediction a compute-versus-interaction view "cannot state" (L018–025).
The line between computational and representation regret also depends on an analyst-chosen Z, which can be moved for an end-to-end VLM. And the regret terms are never quantified on a real model: the logged regret fields "were not computed under the measurability convention of §3.1" (L1716–1718).
Router. The operational half does not deliver the three-way choice in the title:
- The router selects only among compute, acquire and stop; fuse is "validated separately, not as a routing arm."
- The relational encoder is validated on synthetic data only (L1600–1601).
- The iterative controller is not evaluated.
The router itself is effectively a 30-cell lookup table: an explicit per-cell lookup scores 0.937 (L1279–1281). A category-only baseline comes within 0.007. Leave-one-category-out falls to 0.695, with hard-count collapsing to 0.299 after 452 of 720 instances are misrouted to acquire (L1286–1295). The 0.938 headline is just the weighted average under an unmotivated 40/40/20 regime mix (0.4·1.000 + 0.4·0.865 + 0.2·0.958). The "half the generated-token cost" claim uses configured per-arm costs that count output tokens only. The authors state that the sign of the total-cost difference "is not identified" (L1153–1154).
Scope. The evidence base is narrow:
- Vision is procedural only, with no natural images.
- The tunable-p sweep is reported only for Qwen3-VL-8B (Fig. 2), although the abstract opens that sentence with "Across three VLMs."
- Transfer (Table 8) covers only the chance-level categories.
- Embodied evidence is ancillary, and its baseline was not reproduced (LIBERO-10 at 0.26 vs. about 0.95 published).
- The per-scene records behind Fig. 7 and the multi-model transfer logs are not in the supplement (L492–493).
The writing is dense, heavily hedged, and full of internal codenames (E5–E9, R10/R11, script names). The hard-count and saturated families make up 60% of the router pool, yet they are never specified beyond one-line labels.
Questions for authors
- Please scale compute on the joint interface where there is headroom, e.g., before/after joint at 0.929 (Fig. 7), and report it next to Table 1.
- Table 3 reports single-inference accuracy 0.961 [0.954, 0.968] on Task B (relational XOR). Appendix C says binary-label families are sampled at C_info ∈ {0.2, 0.5, ln 2} nats with Y = g(S) (L1093–1099). For binary Y, H(Y|Z,G) ≥ 0.2 nats implies Bayes error ≥ h⁻¹(0.2) ≈ 0.05, so accuracy is at most about 0.95. Which instances and ceilings does Table 3 use? For example, does column B pool the 4-label additive control?
- Why does acquisition accuracy fall from 1.000 to 0.966 as the leak level rises (L435–436), when joint Bayes accuracy is 1.0 throughout?
We would like to sincerely thank all three reviewers and the Area Chair for taking the time to read and evaluate our work.
We were encouraged by the reviewers' recognition of the importance of distinguishing different bottlenecks in test-time scaling and the usefulness of our decision-theoretic perspective. We also appreciate the positive feedback on our information-ceiling experiments.
Although some aspects of the theoretical formulation may not have come across as clearly as we intended, the reviews have given us a better understanding of where the paper needs to be strengthened. In particular, we agree that the work would benefit from deeper theoretical developments, more direct empirical evidence for representation regret, and a more thorough investigation of how these different bottlenecks can be identified in practice.
After careful consideration, we have decided to withdraw the current submission. Rather than making incremental changes during the rebuttal period, we would like to take more time to develop these ideas and address the reviewers' concerns more thoroughly.
We remain excited about the research question and believe that the feedback we received will help us make this work stronger.
Thank you again for your thoughtful comments, time, and consideration. We genuinely appreciate your efforts.