acceptodds
Under review as a conference paper at ICLR 2027

BRINE: Matched-State Counterfactual Supervision for Risk-Aware Cognitive Routing in Underwater Robots

Abstract

Long-horizon underwater control requires cognitive computation to be allocated by current risk and capability. Reactive control efficiently handles routine conditions, while degraded sensing and disturbances benefit from adaptive deliberation; learning this allocation requires evidence about the consequences of alternative cognitive branches. We introduce BRINE, a multimodal system that addresses this coupled decision, learning, and execution problem. From observed robot state, RGB images, point clouds, sonar, task commands, and episodic memory, BRINE determines when deliberation is needed and coordinates three role-specialized components: a state refiner, a safety verifier, and a capability planner. A safety-constrained action projection governs execution. To train the router, we construct matched-state counterfactual supervision by evaluating alternative expert combinations from the same simulator state and comparing their task, safety, and computation outcomes. We further formulate risk-shielded contextual-bandit refinement of the router and introduce role-aware mixed-precision execution that retains precision-sensitive semantic and safety computations in FP32. On 72 paired underwater tasks, the standard-precision system succeeds in 62 cases, compared with 48 for reactive control alone and 52 for always-on deliberation. Counterfactual supervision improves held-out expected return by 0.255 (95% confidence interval: 0.139–0.377) over matched risk and teacher supervision. In separate paired 24-task evaluations, contextual-bandit refinement reduces actual expert-forward duty from 0.609 to 0.482 (paired difference −0.127; 95% confidence interval: −0.180 to −0.074; Holm-adjusted p < 0.001), completes 24/24 tasks with zero collisions, and matches the reference route on all 287 high-risk decisions; mixed precision reduces peak temporary allocated memory by 7.13% and matches FP32 with 24/24 successes. These findings support counterfactual learning for selective, safety-conscious deliberation and mixed-precision execution with preserved behavioral semantics.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, conditional computation, counterfactual supervision, cognitive routing, safe robot control, multimodal perception, contextual bandits, mixed-precision inference, underwater robotics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_eS9AOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the problem of long-horizon underwater control where compute is limited and it needs to decide when to trigger and allocate compute to different modules (state refiner, safety verifier, and a capability planner). To do this, the authors present matched-state counterfactual supervision to learn a router that at runtime, will determine which modules to use. This is done, at simulation, at the same initial state roll out what using different modules would do, to generate labels. The authors then tested in a simulator across different tasks (with labels learned in the same tasks).

Critical strengths
  • Strengths
    • The approach seems well-motivated and sound, where users may want to learn how to best route and does so by learning from simulator-based rollouts of picking different modules from the same initial state.
    • Interesting application of underwater control which has many different types of risk and errors.
Critical weaknesses

Please see below for enumerated points. Major topic 1 concerns novelty against typical routing works and how this method improves upon it. For example, it seems conceptually similar to routing networks, if so, is the novelty in the underwater robot testing? Major topic 2 concerns evaluation and potential limitation of the approach. It is hard to understand for now, how this approach can handle even slight distribution shift in the task, as the task is trained and tested on the same tasks. In addition, this approach seems to be only possible with training in simulation, how would this scale to real world?

  • To improve
    • (major topic 1) Evaluation / Explanation of Novelty: How is this different from other router learning algorithms (Routing Networks, 2017) that uses the final task performance? It seems conceptually the same as what is proposed, but in the proposed work, it is tested on underwater robots. Clarifying this difference and adding a baseline to showcase the delta will be important. This is consequential as for now, it is unclear the novelty of the paper, both qualitatively and quantitatively.
    • (major topic 1) Evaluation: Adding a simple rule and threshold-based baseline (as this is from the introduction), one main paradigm this method would like to move away from.
      • For example, when risk is above a threshold (0.7), trigger all three modules.
    • (major topic 2) Evaluation: The current evaluation seems very in-distribution. As the method learns with data from the same task (while they are “different states”). I would like to see the split be based on having different training and testing task within a task family, such that the evaluation is not tested with the same task. Please also describe how the randomization between tasks is configured.
    • (major topic 2) Evaluation/Approach Question: How would this operate in the real world, where it is not possible to generate such matched-state counterfactual supervision? At minimum, a paragraph of explanation is needed, especially how far the difference between sim and real can be. It would be very helpful if there are real-world deployments of the method with simulator-trained router.
    • Motivation: It is hard to understand what makes the approach unique for underwater robots. Some explanation of this will be helpful.
    • Explaining motivation: Why is the FP32 detail important? It is repeated throughout the paper. What is unique about the approach that makes preserving FP32 now achievable that was previously not.
    • The writing was quite hard to parse, especially on the approach and experimental section. For example, the Figure 1 caption was hard to understand as it had many abbreviations. I would highly recommend structuring the paper such that the novelty is highlighted, and in the approach, describe the approach intuitively. Having some running examples would also help.
Questions for authors
  • How is this different from other router learning algorithms (Routing Networks, 2017) that uses the final task performance? Clarifying this difference and adding a baseline to showcase the delta will be important.
  • How does this work compare quantaitavley to a simple rule and threshold-based baseline? For example, when risk is above a threshold (0.7), trigger all three modules
  • How does this work perform when test set is less in-distribution? I would like to see the split be based on having different training and testing task within a task family, such that the evaluation is not tested with the same task. Please also describe how the randomization between tasks is configured.
  • How would this operate in the real world, where it is not possible to generate such matched-state counterfactual supervision? At minimum, a paragraph of explanation is needed, especially how far the difference between sim and real can be. It would be very helpful if there are real-world deployments of the method with simulator-trained router.
Reviewer_hwbJOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The manuscript introduces BRINE, a multimodal system designed to dynamically allocate computational and cognitive resources during long-horizon underwater control. Rather than relying entirely on reactive control (prone to failures under sensory ambiguity and disturbances) or always-on deliberation (computationally heavy and prone to unnecessary interventions), BRINE conditions a cognitive router on observed robot state, RGB images, point clouds, sonar, task commands, and realized episodic memory to select among six legal role-specialized expert combinations. Moreover, a risk-shielded contextual-bandit refinement of the router is formulated, which introduces role-aware mixed-precision execution that retains precision-sensitive semantic and safety computations. Empirical evaluations of BRINE across 72 simulated tasks in three scene families demonstrate superior task completion, reduced expert-forward duty, and lower peak memory compared to reactive control and always-on deliberation baselines.

Critical strengths
  1. Formulating embodied cognitive allocation over distinct functional roles (SR, SV, CP) rather than generic mixture-of-experts token routing is well-tailored to safety-critical robotics.
  2. The six-branch matched-state evaluation effectively exposes the downstream consequences of alternative cognitive routes under identical disturbance schedules, providing a robust offline supervision signal that achieves near-zero regret on held-out states (Table 5).
  3. The decoupling of the learned proposal router from execution authority ensures that learned policy optimization cannot bypass fundamental safety constraints.
  4. The paper presents an extensive array of evaluations, including 72 paired tasks across diverse scene families, ablation studies on role removal, external VLA comparisons, and rigorous statistical testing with Holm corrections.
Critical weaknesses
  1. The authors directly mentioned FP32 in abstract and introduction without any explanation of what it is (no reference for the term exists anywhere). Moreover, the line that says "... contextual-bandit refinement reduces actual expert-forward duty from ..." is not clear at all. What do you mean by reducing expert-forward duty? Another line that says "... mixed precision reduces peak temporary allocated memory..." also needs explanation: what is the temporary memory here? The abstract needs to be self-explanatory and for a general audience, and I think a short description of these terms can be incorporated without significantly increasing the abstract length.
  2. The methodology section is very poorly written. None of the following terms are explained anywhere, not even in the appendices: the current capability ktk_t from capability graph (the graph is not present anywhere, and no discussion of how this value is selected), the risk diagnostics r_t (no explanation of how it is measured or even what it means), the action uncertainty utu_t (no explanation of how it is measured), the action inconsistency dtd_t (no explanation), the failure evidence ftf_t (no explanation), the risk trend qtq_t (no explanation), the functions σ(),v()\sigma(), v() in ptupp_t^{up}, w()w() in ptrolep_t^{role}, the capability completion CiC_i. The authors must describe each of them (better with examples) in main paper if possible, or add them in appendices and mention in the paper that the details are in appendices, or just refer to a research paper where these terms are explained, and say that the authors are using them. Without these, the current methodology section is unacceptable.
  3. There are several inconsistencies in the methodology as well. cc is used for a task command in section 3.1, but c(m)=∣R(m)∣/3c(m) = |\mathcal{R}(m)|/3 is mentioned as nominal role cost in section 3.3 (not sure if they are the same). It is said that R(m)\mathcal{R}(m) is the role set in Table 1, but not clear if its length is just the number of roles, and why it is divided by 3. In 3.1, Ht\mathcal{H}_t is used for state-action history, but in 3.4, H\mathcal{H} is used for high-risk set.
  4. Need more explanations for the terms in Eq. 5. Though KL divergence is common in ML community, the high-risk and low-risk sets and how they are measured should be explained.
  5. Need references for the methodologies that are textually described in section 3.5. No references exist in that section. Also, no reference for section 4.1. A good scientific paper needs to add references for McNemar tests, Holm correction, etc., even if they are standard terms.
  6. In Figure 1, the refined representation is written as h^t\hat{h}_t, but it is simply written as hth_t everywhere in the paper.
  7. In Table 2, BRINE completed 10 more tasks than always-on, but it has a collision where always-on has none. Not sure if it is better to succeed on more tasks and risk collisions. Also not sure why Table 3 is separate; I feel that both can be combined into one table.
  8. A critical limitation is acknowledged in Section 4.1, which is self-explanatory: "Simulation time is paused during policy inference, and latency is measured separately in wall-clock time. The reported task outcomes therefore do not establish closed-loop performance under inference-induced delay."
Questions for authors
  1. Do you have the references for the terms that I mention for weakness 2? If so, how will you incorporate them in the main paper or appendices?
  2. in Section 3.3, how do you measure ∣R(m)∣|\mathcal{R}(m)|? Is it just the number of roles or something else? Also, why are you dividing by 3 to get c(m)c(m)? Is it for the three roles SR, SV, and CP?
  3. Is the task command cc same as the nominal role cost c()c()? If not, why are you using the same symbol? If yes, please explain.
  4. The normalized progress Pi(m)∈[−1,1]P_i(m) \in [-1,1] can have negative values. Usually, progress is positive; what does negative progress mean here?
  5. Why do you add capability completion, normalized progress, and role cost in Eq. 4? The range of normalized progress is given, but what is the range of the other two?
  6. The line in Section 3.3, "Actual expert-forward duty separately measures SR’s pre-routing diagnostic and all executed expert calls," is not clear at all. I found that the expert-forward duty is described in the text in section 4.1, but you must define this term before you write this line. Also, in the line "Binary upgrade supervision, conditional route supervision, and benefit prediction train the router alongside risk and memory regularization", what are risk and memory regularization? Where are they mentioned in the methodology? Are they related to the risk diagnostic rtr_t? It is vaguely written.
  7. What is the difference between state-action history Ht\mathcal{H}_t and high risk set H\mathcal{H}? Why is the same symbol used?
  8. I may have missed it, but since BRINE uses one of the routes from M0M_0 to M5M_5, what happens if it chooses M0M_0 or M1M_1? Does that mean it is unsafe or risky since safety verification (SV) is not present for these? Why should it select an unsafe route?
  9. Why can't we combine Table 2 and Table 3 and make it just one table? The 72 tasks and the three controllers are the same.
Reviewer_VSFhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers adaptive deliberation in underwater robots. This addresses the question of which cognitive operations a robot needs given its current risk and capabilities. The authors introduce a system coupling a reactive controller with a cognitive router which invokes a combination of expert roles. The system is evaluated empirically on simulated underwater scenes and compared with reactive control and always on deliberation.

Critical strengths

S1. The research question is clearly identified and well motivated. It was clear to me why adaptive deliberation is beneficial in these contexts.

S2. Results are tested for statistical significance.

Critical weaknesses

W1. There is insufficient detail on training and evaluation procedures, as is detailed in my comments below. Specifically, the design of the simulator environment and set of tasks against which the BRINE system is evaluated are not described.

W2. At a high-level the method’s section is reasonably well-described. However, I found too much detail was deferred to the appendix without signposting of where to find it e.g. the architecture of the routing network. This combined with the use of undefined notation/terms made it tricky to follow. In section 3.2, vv and ww are not defined and I am not certain what these represent. In section 3.4, the loss function for contextual-bandit refinement partitions fitting contexts using a “guard predicate”. This “guard predicate” is not defined before this use, nor could I find it explained later in the text.

W3. Most results are presented without confidence intervals.

Questions for authors

C1. L213-L215 discuss emergency evidence, but it isn’t clear to me what constitutes emergency evidence and what system this would come from.

C2. In L217, could you clarify what is meant by “calibrated action-prior correction”?

C3. What is the value of the expert-forward duty metric? There is no discussion of the importance of this metric for evaluation.

C4. In equation (4), how were the coefficients alpha, beta and gamma decided? Was there any tuning done to select these?

C5. How are collisions counted? Given collisions are measured out of 72 (the number of tasks), I understood that there could be at most one collision per task i.e. a collision meant immediate failure and was unrecoverable. Is this the case?

C6. In evaluation, what is goal progress and how does a “goal” differ from a “task”? It would be beneficial to the reader to more explicitly state the dataset of tasks/goals used for evaluation. I suggest that Figure 2 would be an appropriate place to do this as, in its current form, it doesn’t add much understanding.

C7. The simulator set up is not detailed. Appendix D.1 states that the environment uses Isaac Sim 4.5. Was the exact environment designed for this specific experiment?

C8. Discussion of results highlights the metrics where BRINE outperforms reactive control and always on deliberation. However, there isn’t acknowledgement of when BRINE does worse. In the results tables, the BRINE results are in bold when they are the best, but the other systems aren’t in bold when they are top. Further, I think it is misleading to format the BRINE results in bold where it performs equally as one of the other systems. For example, in Table 3, BRINE and Reactive Controller both have 48/48 successes but only the BRINE result is in bold.

C9. On L209, vv and ww are not defined.

C10. In equation (3), Ut\mathcal{U}_t and Uglobal\mathcal{U}_{global} are not defined.

C11. Minor Comments:

  • The diagram in Figure 1 contains a typo in the “Risk-Shielded Cognitive Router” block. I believe the ztz_t should match equation (1) and contain emb(kt)emb(k_t) not emb(ht)emb(h_t).
  • There is inconsistent notation between the description of BRINE’s components and Figure 1. In L158 the causal state-action history is denoted Ht\mathcal{H}_t whereas in Fig 1 it is written HtH_t.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.