- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents ReliAct, a lightweight framework for robust underwater vision-language-action control. It derives source-specific reliability targets from a frozen policy’s response discrepancy between paired clean and corrupted observations. An estimator predicts these scores from a short observation history, and a shared reliability vector regulates visual feature repair, multimodal routing, and action generation. The paper also introduces an underwater robustness benchmark with 18 conditions and 3,560 closed-loop episodes per model. ReliAct improves aggregate corrupted success from 26.7% to 36.0%, with 5.9% additional inference latency.
Critical strengths
-
Practical and well-motivated problem. The paper addresses both visual degradation and sensor failures, which are relevant challenges for underwater robotic control.
-
Coherent method design. Connecting policy-derived reliability to repair, fusion, and action generation provides a clear mechanism for adapting information use to degraded observations.
Critical weaknesses
-
The reliability definition lacks direct comparison. The paper does not compare independently trained models using alternative signals, such as sensor health, perceptual quality, or an unconstrained learned gate. Replacing the signal at inference with constants or random values does not establish the superiority of the proposed target.
-
Component contributions are not fully isolated. The incremental ablations also increase trainable capacity, and the RobustVLA baseline implements only part of the original method. Stronger controls would help attribute the gains.
-
Evaluation is limited to one backbone and simulator. Transfer to other VLA models and real underwater conditions remains untested.
-
Composite generalization is tested narrowly. Only three unseen combinations and two withheld corruption families are evaluated. Broader simultaneous failures and prolonged sensor outages remain unexplored.
-
Uncertainty estimates are incomplete. Training uses a single seed, and the aggregate bootstrap does not account for correlations across conditions sharing task seeds. The variability of improvements over the strongest baselines is therefore unclear.
Questions for authors
See Weaknesses.
2 comments
Thanks for the authors' rebuttal. I maintain my original positive score.
Response to Reviewer e9xD
Thank you for the comments. We address each concern below.
1. Direct comparison of reliability supervision. We independently retrained the added modules of four variants with the full ReliAct architecture, 8.6M trainable parameters, frozen U0, identical training data and single-source corruption sampling, AdamW, batch size 64, and 20,000 steps. "No Lrel" removes reliability supervision. "Policy-only" additionally removes clean-reliability regularization while retaining clean-policy consistency. The severity proxy uses q=1-s, with s normalized within each corruption family's training range and unaffected sources labeled 1. Severity is used only for training labels.
Entries are SR (%), on the Table 4 manifest: navigation/grasping/transport/tracking (G/R/T/F), 20 base scenes each; N=80/240/160/240 across columns. Severe visual comprises L3 turbidity/color shift/low-light; Sensor comprises IMU bias/DVL dropout. Methods share initial states, instructions, environment and corruption seeds.
2. Component attribution and RobustVLA. We retain the full architecture and all computational branches, fix reliability at only one location to the per-source training-target mean, and independently retrain with the same parameter budget and training steps. Fixing repair/routing/action-conditioning inputs yields composite SRs of 33.3/30.4/32.1%, versus 37.5% with all locations dynamic. Fixing repair lowers severe-visual SR from 31.7% to 27.5%; fixing routing lowers sensor SR from 43.8% to 35.6%. These controls test dynamic regulation at each location within the complete architecture.
RobustVLA-Input adapts input consistency to U0 without the original method's complete output-side components. Claims are limited to this implementation; the matched-architecture controls above provide the attribution evidence.
3. Backbone and simulation scope. Our empirical conclusions concern the evaluated U0/USIM underwater simulation tasks. Transfer to other VLA backbones and real underwater platforms has not been tested.
4. Unseen combinations and corruption families. Standard training uses only single-source degradation. Turbidity+low-light, turbidity+IMU, and low-light+DVL never occur jointly during training; their evaluated visual components are L2. Separately, LOCO excludes the entire motion-blur or DVL-dropout family. SRs are 42.5% and 40.7%, retaining approximately 76% and 78% of the full model's gains over U0.
Composites test combination transfer; LOCO tests withheld-family transfer. DVL outages last 0.5 s every 10 s. Prolonged outages and broader simultaneous failures are not covered by these experiments.
5. Paired uncertainty and training variability. The original condition-wise bootstrap did not account for correlations across conditions sharing a base scene. For the original Table 3 comparisons, we now resample base-scene clusters within each task, jointly retaining all conditions and both methods' outcomes. We use 10,000 replicates and the 2.5th/97.5th percentiles.
SameAug denotes Fixed IB+SameAug; Gated U0 denotes Reliability-Gated U0. Counts are both succeed/only ReliAct succeeds/only baseline succeeds/both fail. Gains use unrounded counts. Table 3 Sensor/Cross-modal each contain four tasks x 40 base scenes x two conditions. Cross-modal contains only turbidity L2+IMU and low-light L2+DVL; these manifests differ from Table 4.
These CIs describe scene-sampling uncertainty at fixed checkpoints. The L3 CI crosses zero, the Sensor CI touches zero, and the Cross-modal CI is strictly positive. Training variability is reported separately: both methods are independently trained using seeds 0/1/2.
For Table 3, Sensor gains are 4.06/3.44/4.69 pp (mean +/- sample SD: 4.06 +/- 0.63); Cross-modal gains are 5.94/5.31/6.88 (6.04 +/- 0.79). Their G/R/T/F gains are 12.50/-2.50/2.50/3.75 and 5.00/7.50/6.25/5.00, respectively. Thus seed-level gains are positive, with task-level variation.
On Table 4's manifest, mean +/- sample SD of gains is 3.61 +/- 0.24 pp (L3 vs SameAug), 5.42 +/- 0.36 (Sensor vs Gated U0), and 5.97 +/- 0.24 (three composites vs Gated U0). All three seeds show positive gains; composite gains are positive on all four tasks.