- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Recursive State-Aware Credit assignment (RSAC), a method that iteratively improves a code-as-policy approach to robot learning. Given an initial policy expressed in code, RSAC will save checkpoints throughout execution as the policy progresses through stages of the code. With a handful of stages and checkpoints, RSAC then runs several rollouts from each checkpoint to estimate that checkpoint's individual success-rate. RSAC then applies an LLM to update the code to improve success rates from the various stage checkpoints, validating with full end-to-end rollouts to ensure that the fixes do not cause performance regressions.
Critical strengths
- The paper proposes an iterative improvement strategy to a code-as-policy approach to robot learning, which enables backprop-free and interpretable policies for robots.
- RSAC seems to perform well against baselines, and the components are ablated and demonstrated to contribute to overall success.
Critical weaknesses
- The paper is almost impossible to understand. Variables are acronyms are entirely undefined, even those which seem very important to the paper (for example "SA" is used frequently but is never defined, the variable in equation 3 seems important but is likewise not defined, and many others exist in the paper). The method is impenetrable, I would absolutely not be able to reproduce this.
- The results, like the method, are very difficult to parse and follow. The exact experimental setup is not clear -- the paper says that 3 iterations of program improvement were given, yet RSAC defines a method which seems to require significantly more than three rollouts to estimate success rates, make policy updates, validate that no updates have caused regressions, etc. At each update iteration, methods had equal rollout count, but it is not clear what this count was, or if it was fair to other methods (which may rely on more iterations and fewer rollouts).
Ultimately, the primary and overwhelmingly strong weakness of this paper is its legibility. This paper seems to have been entirely LLM generated after prompting with a code repo and results directory, from the text to the figures. It does not serve as a useful contribution to the community, as it is not clear enough to offer learnings or insights to the broader community.
Questions for authors
I do not know where to begin with questions. I would need a significant rewrite to have meaningful and interesting questions about the work.