- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes NEEDLE, an offline data rewriting method for imperfect robot demonstrations. It finds useful source-target pairs, generates short action bridges with a goal-conditioned inverse dynamics model, verifies them with an action-conditioned reachability model, and adds the accepted bridges as extra supervision for imitation learning. Experiments on real-robot and simulated manipulation tasks show consistent improvements over several baselines.
Critical strengths
-
The experiments are fairly thorough. The paper includes real-robot results, several baselines, and useful ablations, especially on the verifier and the different types of stitching.
-
The method can recover useful supervision from imperfect data without additional robot interaction. This is practically useful when failed or inefficient trajectories already exist in the dataset.
Critical weaknesses
-
The overall pipeline feels quite heuristic and relatively heavy. It requires candidate retrieval, bridge generation, verification, and several hand-designed stitching cases. It is not clear how well this would scale to much larger and more diverse robot datasets.
-
The verifier only checks local reachability. The method still assumes that after reaching the target, the successful continuation from that trajectory remains valid. For contact-rich manipulation, small differences in grasp or object state may break this assumption.
Questions for authors
-
How does the offline rewriting cost scale with dataset size, including candidate retrieval, IDM sampling, and verification?
-
Have the authors evaluated whether a bridge that reaches the target also preserves the validity of the target trajectory’s remaining actions?
-
How sensitive is the method to the manually defined S1/S2/S3 stitching categories? Would a more generic relabeling or conditioning strategy achieve similar gains?
3 comments
Response by authors (2/2)
“How sensitive is the method to the manually defined S1/S2/S3 stitching categories? Would a more generic relabeling or conditioning strategy achieve similar gains?” (Q3)
We appreciate the reviewer’s attention to detail in this question. The three categories capture different benefits of stitching: S1 skips inefficient or redundant actions within a trajectory, S2 connects to a shorter continuation in another successful trajectory, and S3 adds supervision from failed demonstrations. We ablate the effects of including different subsets (Mean ± standard error over five training seeds.):
| Included stages | Can success (%) ↑ | Square success (%) ↑ | Transport success (%) ↑ |
|---|---|---|---|
| S1 | 73.6 ± 1.2 | 40.8 ± 1.5 | 49.2 ± 4.5 |
| S2 | 80.4 ± 1.0 | 46.4 ± 2.6 | 46.4 ± 2.0 |
| S3 | 74.4 ± 1.2 | 40.4 ± 2.2 | 48.4 ± 1.7 |
| S1 + S2 | 81.6 ± 2.7 | 41.2 ± 1.6 | 45.6 ± 1.5 |
| S1 + S2 + S3 | 83.6 ± 1.7 | 47.2 ± 1.4 | 53.2 ± 2.2 |
| DP | 72.8 ± 0.8 | 39.6 ± 1.5 | 46.4 ± 1.6 |
Most subsets have higher mean success than DP, and using all three gives the highest mean on all three tasks. Their relative value differs across tasks. Section 4.3 (Figure 11) in the manuscript additionally compares against reweighting the original data without bridge actions, which underperforms the full method on all three tasks.
We present our results in Section 6, "Are the three categories needed" of https://needle-work.github.io/rebuttal/#rebuttal-algorithm-stages.
The three categories are the three ways a useful target can relate to a source, given that the target must lie on a successful trajectory: on the source's own successful trajectory (S1), on another successful trajectory from a successful source (S2), or on a successful trajectory from a failed source (S3). All three use the same retrieval, IDM, verifier, and selection rule. In earlier development, to test whether the categories themselves are needed, we replaced them with a single ranking of all accepted bridges by verifier score. This recovered part of the gain but fell short of the full method. The key role of the categories is therefore how they allocate the bridge budget: each category supplies a different kind of connection, and the ablation above shows that no single category, and no subset we tested, matches using all three.
“How does the offline rewriting cost scale with dataset size, including candidate retrieval, IDM sampling, and verification?” (W1, Q1)
We appreciate the reviewer for pointing out this important analysis. We highlight that our rewriting method is a one-time offline preparation step, and does not impact the actual duration of policy training, nor inference time. Downstream training uses the same policy architecture and training procedure as any other imitation learning policy.
On scaling: rewriting cost is dominated by scoring candidates with the IDM and verifier, and this grows linearly with the number of candidates (a least-squares fit of scoring time against candidate count gives R² = 0.998 on Can and 0.999 on Square, over 5k–200k candidates). Because each source receives a fixed candidate budget, the number of candidates, and so the cost, grows linearly with the number of source states rather than quadratically with the number of pairs. The GPU time per candidate (eight IDM samples plus verification) is similar across tasks: 7–15 ms on Robomimic and 7–9 ms on the real-robot data. Retrieval is the only step that grows faster than linearly, and it stays a small share of the total: 0.4–3.4 minutes on Robomimic (88k–137k states) and 28–44 minutes on the real-robot datasets (212k–409k states).
We report component durations measured on RTX PRO 6000 Blackwell GPUs:
| Component | Square (simulation) | Sweater Folding (real-robot data) |
|---|---|---|
| IDM training | 1.8 h | 9.2 h |
| Verifier training | 2.8 h | 9.2 h |
| Rewriting | 1.2 h | 5.3 h |
| Policy training | 1.4 h | 25.6 h |
The IDM and verifier can be trained independently. We report their component durations separately. On real-robot data, auxiliary training and rewriting together take about as long as one policy training run, and the prepared dataset is reused across policies.
In Section 8, "What does offline rewriting cost?" of https://needle-work.github.io/rebuttal/#rebuttal-algorithm-cost, we show a case study on Square (sim) and Sweater Folding (real) on the cost of running our method: training the IDM and verifier, which can be trained in parallel, and rewriting are one-time costs, shown next to one policy training run.
As we get close to the end of the author-reviewer discussion period, we wanted to (a) confirm that our above response answered at least some of the questions the reviewer had and (b) ask if there was anything else we could do to improve your assessment of our work. Regardless, thank-you again for the feedback you've already provided.
Response by authors (1/2)
We thank the reviewer for this important question. Yes, and our method is designed so that it does not depend on it: our training never assumes that the target's remaining actions stay valid after a bridge. Training windows stop the action target at the bridge endpoint (Figure 3 of the main paper), and the target's remaining actions are learned separately, paired with their own recorded observations. We evaluate this choice in two ways below; all results and visualizations are in the webpage section “What if bridges don't match the remaining actions?” https://needle-work.github.io/rebuttal/#rebuttal-continuation.
1. Downstream effect of the training design. Appending the target's remaining actions directly to the bridge, as if the connection were exact, lowers policy success on all three tasks (mean ± standard error over five training seeds):
2. Continuation from bridge landings. We measure how well each bridge landing supports the rest of the task. Open-loop replay: from the bridge landing, we replay the target trajectory's recorded remaining actions and compare the success rate to replaying the same actions from the exact target state. Closed-loop policy: from the bridge landing, a trained diffusion policy takes over, and we compare its success rate to the same policy taking over after the source's own recorded actions. A ratio of 1 means the bridge landing is as good a starting point as the recorded one:
Open-loop replay from a landing succeeds only about a third as often as from the exact target state, so the reviewer's concern is well founded: small differences at the landing can make the target's recorded actions invalid. This is exactly why we do not train on them. A closed-loop policy that takes over at the same landings succeeds as often as after the source's recorded actions (ratio 0.99–1.14).
Together, these results support training on bridges without assuming that their landings preserve the target's exact open-loop continuation: bridges do not need to perfectly connect to the target's suffix trajectory to be usable for policy learning. We see that the verifier play an important role in filtering out harmful errors, and our training method ensures we're able to learn well in spite of any remaining imprecision.