- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RN-VLA addresses fine-grained robotic manipulation through a modular pipeline combining Semantic Query Grounding, Cross-Modal Gated Modulation, and Reference Point Normalization. A task-conditioned perception frontend predicts a target-part bounding box. Its centroid is used to re-center geometric observations and proprioception, while visual and task features modulate point-cloud features through FiLM. A conditional diffusion policy generates action sequences.
The paper reports substantial improvements on seven atomic manipulation tasks, real-robot demonstrations, and improved success when atomic policies are composed into longer sequences. My main concerns are whether the improvements are sufficiently attributed to the proposed mechanisms, whether the input and ablation protocols isolate their contributions, and whether the evidence supports the stated precision and long-horizon claims.
Critical strengths
S1. Meaningful tasks and substantial reported improvements.
The tasks are interesting and practically relevant, particularly selecting and pressing a specific key among densely arranged alternatives. The reported average atomic success rate of 81.6% substantially exceeds the evaluated baselines. Real-robot results, including 70.0% success on Keyboard and 67.6% on Remote with predicted bounding boxes, provide useful evidence that the system can perform these interactions beyond simulation.
S2. The internal ablations support the usefulness of several design choices.
The target-centered representation and cross-modal modulation have substantial effects within the proposed system. On Keyboard, replacing the target reference with a workspace-fixed reference reduces success from 80.8% to 53.4%, while removing CMGM reduces it to 56.1%. These results support the utility of the components, although they do not fully establish the mechanisms or comparative advantages claimed.
Critical weaknesses
W1. Limited methodological novelty and insufficient attribution of the improvements.
The method primarily integrates established representation extraction, conditional feature fusion, coordinate re-centering, and diffusion-based control. These choices have reasonable motivations, but the paper does not sufficiently establish what new methodological insight the combination provides or why its particular feature-processing mechanisms are responsible for the reported fine-grained manipulation advantage.
Table 3 is especially relevant: compared with FVP-DP3, target-selection accuracy increases from 58.6% to 94.0%, whereas execution success conditional on correct selection increases from 84.0% to 86.8%. This suggests that improved target specification or localization is central to the overall gain. Since the correctly selected subsets may differ, these statistics are not a controlled causal decomposition, but they do call for a more careful distinction between selection and execution improvements.
The evidence does not clearly demonstrate that SigLIP-based fine-grained visual discrimination or the proposed visual-to-geometric modulation produces more accurate manipulation. Aggregate selection accuracy and attention maps do not establish whether the model reads distinguishing target details, exploits familiar spatial layouts, or primarily benefits from explicit target-box conditioning. Similarly, channel-wise FiLM modulation is not itself evidence of precise point-level spatial selection. The contribution should be positioned more carefully and supported by controls that isolate these explanations.
W2. The “without RGB” ablation and baseline input protocols are insufficiently specified.
Removing RGB reduces success only from 80.8% to 76.8% on Keyboard and from 83.9% to 79.8% on Safe. These relatively small differences require explanation given the central role assigned to visual semantics and cross-modal fusion.
RGB information can reach the policy both through the visual-fusion branch and indirectly through SQG-generated bounding boxes. The paper does not clearly specify whether “without RGB” removes both paths, whether SQG still processes RGB, or whether predicted, cached, or oracle boxes remain available. Consequently, the ablation does not establish the end-to-end contribution of RGB or SigLIP. Moreover, CMGM receives task and bounding-box information, so its larger ablation effect cannot be attributed specifically to RGB-driven modulation.
The side branch also directly supplies task, bounding-box, and proprioceptive information, and training explicitly includes cases with the fused visual condition removed. Its standalone capability is therefore an important missing reference.
Finally, each baseline's access to the intended target identity, language, bounding boxes, point clouds, and normalization should be specified. Otherwise, comparisons may combine differences in available information with differences in policy design. The box conditions also need reconciliation: Table 5 is labeled “Oracle BBox” but reports 80.8% for Keyboard, while Table 6 reports 80.8% with predicted boxes and 88.2% with oracle boxes.
W3. The precision and long-horizon claims are stronger than the evidence supports.
Successful interaction with small targets is meaningful, but task-success thresholds do not directly quantify localization or control precision. For example, depressing the correct key by 5 mm measures successful actuation, not a 5 mm targeting error. The paper lacks sufficiently direct measurements of target-centroid error, contact-point error, or end-effector accuracy to substantiate the claimed fine-grained precision and its source.
The long-horizon results show improvement, but do not establish reliable execution. Table 7 reports seven-step success rates of 22.5% and 33.6%, versus 13.4% and 22.3% for the strongest baselines. These gains are useful, but most sequences still fail. Statements about eliminating atomic errors or resolving error compounding should therefore be substantially qualified. The expression p^T is an independence-based reference, not a generally justified conservative lower bound.
The execution protocol also requires clarification. The main text states that composition requires no LLM planning or online verification, while Appendix A.1 and Appendix D describe an LLM planner and verification procedures. These may be offline or evaluation-only components, but their roles in the reported experiments must be made explicit.
Overall, I lean toward weak rejection. The tasks and performance gains are encouraging, but the current evidence does not sufficiently isolate the proposed contribution or support the breadth of the precision and composability claims. Clearer protocols and focused matched-information controls could materially change this assessment.
Questions for authors
Q1. Contribution and improvement attribution — corresponding to W1.
What is the principal methodological contribution beyond combining task-conditioned grounding, feature modulation, target-relative representations, and a diffusion policy? Please explain how the evidence separates improved target selection from improved control.
In particular, can you provide task-specific grounding diagnostics, such as target-confusion rates or localization errors on Keyboard, that support the claimed fine-grained visual discrimination? Please clarify whether evaluation distinguishes recognition of target details from reliance on familiar layouts. A focused comparison of the proposed modulation against simple fusion under the same target-box conditions would help establish its specific contribution.
Q2. RGB ablation and input protocol — corresponding to W2.
Please specify exactly what is removed in the “without RGB” experiment, including whether the model is retrained, whether SQG still receives RGB, how its bounding boxes are obtained, and what generates the FiLM conditions after RGB removal.
Please provide a per-baseline input inventory covering target identity, language, bounding boxes, RGB, point clouds, and coordinate/action normalization, and reconcile the oracle/predicted-box conditions in Tables 5 and 6.
How much performance is achievable from the side branch alone, or from a simple target-box-conditioned 3D policy? A representative matched-information control would help determine whether the additional visual-fusion machinery provides a meaningful advantage beyond the supplied target information.
Q3. Precision and long-horizon interpretation — corresponding to W3.
Beyond binary task success, what measurements establish the claimed spatial precision? Please report available localization or execution-error statistics, together with evaluation counts and uncertainty, and distinguish perception errors from errors after correct target localization.
For the long-horizon experiments, please specify whether sub-goal lists are predetermined, whether the LLM or verifier influences switching, stopping, or retries, and how the 5,800 long-horizon trajectories are used relative to the claim of atomic-only training. Please also explain the lower-bound interpretation of p^T and qualify the reliability claims in light of the measured 22.5% and 33.6% full-sequence success rates.