- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RoboRSI studies whether a robotic system can improve the strategy that guides its own subsequent modifications. It separates the execution system, improvement strategy, and accumulated experience while keeping model parameters and execution backends fixed. An inner loop proposes and evaluates system changes; an outer loop updates the improvement strategy using matched, isolated downstream improvement branches. Experiments cover four robotic benchmarks and include controls for access to historical experience and for continued strategy updating.
Critical strengths
-
A meaningful target for self-improvement. Separating the execution system from the strategy that selects and reviews modifications makes learning how to improve operational. Testing candidate strategies through isolated downstream improvement branches, while keeping validation traces branch-local, is a sensible design.
-
Controls support a benefit beyond additional history. With full historical access, the LIBERO-Pro control reaches 68.4%, versus 76.6% with the explicit strategy (§3.3). Updating the strategy improves Atomic-Seen success from 57.4% to 68.0% relative to keeping it fixed (Table 1). These are consequential gains for the central strategy-learning claim.
Critical weaknesses
-
The evaluated implementation is not clearly connected to the complete claimed mechanism. Proposal review and outcome analysis are central to the method, but Appendix E, p. 19, states that the released four-benchmark development backend does not directly dispatch PreCritic and Auditor. This does not prove their functionality was absent from the experiments. It does leave unclear which selection-and-review components generated the reported results, limiting attribution to the learned review strategy.
-
The controlled advantage needs a traceable resource and uncertainty comparison. Every candidate strategy is validated through two branches over at least 20 decisions. Equation 3 already includes proposal/review calls, implementation, outcome analysis, and robot execution in branch cost; the unresolved issue is total strategy-generation/validation expenditure and the units used to match budgets. History matching alone does not distinguish strategy benefit from additional development computation. Independent improvement-run variability is also unclear. Figure 2's LIBERO-Pro values of 64.7/76.5 differ from §3.3's 63.6/68.4/76.6 and need a configuration-level explanation.
Questions for authors
- Which backend and configuration produced each main result and ablation? Could you provide the mapping from the paper's roles to the executed components, together with one complete improvement trace showing proposal review and outcome analysis?
- What exactly is included in the matched improvement budget? Please report total model calls/tokens and robot executions, including both strategy-validation branches, and explain the cost units. A comparison at a common total budget on the existing controlled setting would directly address the attribution concern.
- How many independent improvement runs produced the final systems? Please clarify the Figure 2/Figure 3 discrepancy and provide run-level variability where repeated runs exist, or qualify the conclusions if only one learned trajectory supports a comparison.