acceptodds
Under review as a conference paper at ICLR 2027

NEEDLEWORK: Offline Rewriting of Robot Data with Verified Local Stitches

Abstract

Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful. Trajectory stitching offers a way to compose these behaviors into improved training data, but identifying useful connections and verifying their feasibility is difficult in high-dimensional robot data, where many prior methods rely on low-dimensional state representations. We introduce NEEDLE, an offline dataset-augmentation algorithm that addresses these challenges by adding short, verified action bridges between recorded observations in high-dimensional robot demonstrations. First, NEEDLE identifies and creates connections that bypass suboptimal detours, broaden action coverage, and augment the original dataset with failed trajectories, using only RGB images, proprioception, and episode-level outcomes, without new environment interaction or privileged object state. Next, we present a sampling technique that incorporates accepted bridges into policy training without synthesizing intermediate images or discarding the original demonstrations, allowing policies to learn alternative actions while retaining the original dataset's coverage. On real-robot tasks, NEEDLE improves success rate over the strongest baseline on each task by an average of 21 percentage points. Supplementary materials are on needle-work.github.io.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Dataset Augmentation, Learning from Suboptimal Demonstrations, Trajectory Stitching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4XGnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes NEEDLE, an offline data rewriting method for imperfect robot demonstrations. It finds useful source-target pairs, generates short action bridges with a goal-conditioned inverse dynamics model, verifies them with an action-conditioned reachability model, and adds the accepted bridges as extra supervision for imitation learning. Experiments on real-robot and simulated manipulation tasks show consistent improvements over several baselines.

Critical strengths
  1. The experiments are fairly thorough. The paper includes real-robot results, several baselines, and useful ablations, especially on the verifier and the different types of stitching.

  2. The method can recover useful supervision from imperfect data without additional robot interaction. This is practically useful when failed or inefficient trajectories already exist in the dataset.

Critical weaknesses
  1. The overall pipeline feels quite heuristic and relatively heavy. It requires candidate retrieval, bridge generation, verification, and several hand-designed stitching cases. It is not clear how well this would scale to much larger and more diverse robot datasets.

  2. The verifier only checks local reachability. The method still assumes that after reaching the target, the successful continuation from that trajectory remains valid. For contact-rich manipulation, small differences in grasp or object state may break this assumption.

Questions for authors
  1. How does the offline rewriting cost scale with dataset size, including candidate retrieval, IDM sampling, and verification?

  2. Have the authors evaluated whether a bridge that reaches the target also preserves the validity of the target trajectory’s remaining actions?

  3. How sensitive is the method to the manually defined S1/S2/S3 stitching categories? Would a more generic relabeling or conditioning strategy achieve similar gains?

3 comments
AuthorsOpenReview ↗

Response by authors (1/2)

“Have the authors evaluated whether a bridge that reaches the target also preserves the validity of the target trajectory’s remaining actions?” (W2, Q2)

We thank the reviewer for this important question. Yes, and our method is designed so that it does not depend on it: our training never assumes that the target's remaining actions stay valid after a bridge. Training windows stop the action target at the bridge endpoint (Figure 3 of the main paper), and the target's remaining actions are learned separately, paired with their own recorded observations. We evaluate this choice in two ways below; all results and visualizations are in the webpage section “What if bridges don't match the remaining actions?” https://needle-work.github.io/rebuttal/#rebuttal-continuation.

1. Downstream effect of the training design. Appending the target's remaining actions directly to the bridge, as if the connection were exact, lowers policy success on all three tasks (mean ± standard error over five training seeds):

Training designCan success (%) ↑Square success (%) ↑Transport success (%) ↑
Diffusion Policy (DP)72.8 ± 0.839.6 ± 1.546.4 ± 1.6
Append target's remaining actions78.8 ± 0.842.0 ± 2.345.6 ± 1.2
NEEDLEwork83.6 ± 1.747.2 ± 1.453.2 ± 2.2

2. Continuation from bridge landings. We measure how well each bridge landing supports the rest of the task. Open-loop replay: from the bridge landing, we replay the target trajectory's recorded remaining actions and compare the success rate to replaying the same actions from the exact target state. Closed-loop policy: from the bridge landing, a trained diffusion policy takes over, and we compare its success rate to the same policy taking over after the source's own recorded actions. A ratio of 1 means the bridge landing is as good a starting point as the recorded one:

TaskOpen-loop replay of the target's actions: success from landing ÷ success from exact target ↑Closed-loop policy: success from landing ÷ success after source's recorded actions ↑
Can0.32 [0.15, 0.48]0.99 [0.82, 1.21]
Square0.33 [0.11, 0.48]1.01 [0.70, 1.47]
Transport0.37 [0.24, 0.49]1.14 [0.96, 1.45]

Open-loop replay from a landing succeeds only about a third as often as from the exact target state, so the reviewer's concern is well founded: small differences at the landing can make the target's recorded actions invalid. This is exactly why we do not train on them. A closed-loop policy that takes over at the same landings succeeds as often as after the source's recorded actions (ratio 0.99–1.14).

Together, these results support training on bridges without assuming that their landings preserve the target's exact open-loop continuation: bridges do not need to perfectly connect to the target's suffix trajectory to be usable for policy learning. We see that the verifier play an important role in filtering out harmful errors, and our training method ensures we're able to learn well in spite of any remaining imprecision.

AuthorsOpenReview ↗

Response by authors (2/2)

“How sensitive is the method to the manually defined S1/S2/S3 stitching categories? Would a more generic relabeling or conditioning strategy achieve similar gains?” (Q3)

We appreciate the reviewer’s attention to detail in this question. The three categories capture different benefits of stitching: S1 skips inefficient or redundant actions within a trajectory, S2 connects to a shorter continuation in another successful trajectory, and S3 adds supervision from failed demonstrations. We ablate the effects of including different subsets (Mean ± standard error over five training seeds.):

Included stagesCan success (%) ↑Square success (%) ↑Transport success (%) ↑
S173.6 ± 1.240.8 ± 1.549.2 ± 4.5
S280.4 ± 1.046.4 ± 2.646.4 ± 2.0
S374.4 ± 1.240.4 ± 2.248.4 ± 1.7
S1 + S281.6 ± 2.741.2 ± 1.645.6 ± 1.5
S1 + S2 + S383.6 ± 1.747.2 ± 1.453.2 ± 2.2
DP72.8 ± 0.839.6 ± 1.546.4 ± 1.6

Most subsets have higher mean success than DP, and using all three gives the highest mean on all three tasks. Their relative value differs across tasks. Section 4.3 (Figure 11) in the manuscript additionally compares against reweighting the original data without bridge actions, which underperforms the full method on all three tasks.

We present our results in Section 6, "Are the three categories needed" of https://needle-work.github.io/rebuttal/#rebuttal-algorithm-stages.

The three categories are the three ways a useful target can relate to a source, given that the target must lie on a successful trajectory: on the source's own successful trajectory (S1), on another successful trajectory from a successful source (S2), or on a successful trajectory from a failed source (S3). All three use the same retrieval, IDM, verifier, and selection rule. In earlier development, to test whether the categories themselves are needed, we replaced them with a single ranking of all accepted bridges by verifier score. This recovered part of the gain but fell short of the full method. The key role of the categories is therefore how they allocate the bridge budget: each category supplies a different kind of connection, and the ablation above shows that no single category, and no subset we tested, matches using all three.

“How does the offline rewriting cost scale with dataset size, including candidate retrieval, IDM sampling, and verification?” (W1, Q1)

We appreciate the reviewer for pointing out this important analysis. We highlight that our rewriting method is a one-time offline preparation step, and does not impact the actual duration of policy training, nor inference time. Downstream training uses the same policy architecture and training procedure as any other imitation learning policy.

On scaling: rewriting cost is dominated by scoring candidates with the IDM and verifier, and this grows linearly with the number of candidates (a least-squares fit of scoring time against candidate count gives R² = 0.998 on Can and 0.999 on Square, over 5k–200k candidates). Because each source receives a fixed candidate budget, the number of candidates, and so the cost, grows linearly with the number of source states rather than quadratically with the number of pairs. The GPU time per candidate (eight IDM samples plus verification) is similar across tasks: 7–15 ms on Robomimic and 7–9 ms on the real-robot data. Retrieval is the only step that grows faster than linearly, and it stays a small share of the total: 0.4–3.4 minutes on Robomimic (88k–137k states) and 28–44 minutes on the real-robot datasets (212k–409k states).

We report component durations measured on RTX PRO 6000 Blackwell GPUs:

ComponentSquare (simulation)Sweater Folding (real-robot data)
IDM training1.8 h9.2 h
Verifier training2.8 h9.2 h
Rewriting1.2 h5.3 h
Policy training1.4 h25.6 h

The IDM and verifier can be trained independently. We report their component durations separately. On real-robot data, auxiliary training and rewriting together take about as long as one policy training run, and the prepared dataset is reused across policies.

In Section 8, "What does offline rewriting cost?" of https://needle-work.github.io/rebuttal/#rebuttal-algorithm-cost, we show a case study on Square (sim) and Sweater Folding (real) on the cost of running our method: training the IDM and verifier, which can be trained in parallel, and rewriting are one-time costs, shown next to one policy training run.

As we get close to the end of the author-reviewer discussion period, we wanted to (a) confirm that our above response answered at least some of the questions the reviewer had and (b) ask if there was anything else we could do to improve your assessment of our work. Regardless, thank-you again for the feedback you've already provided.

Reviewer_ak1NOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how to stitch segments of suboptimal or even failed robot demonstrations to form an improved dataset. The paper proposes NEEDLE, an offline dataset augmentation algorithm that uses a learned IDM module and a verifier to construct action bridges that connect trajectory segments, in order to reduce detours, broaden action coverage, or reuse failed rollouts. In both simulation and real-world experiments, the pipeline is validated to improve policy performance and reduce execution time.

Critical strengths
  1. The paper studies an important and practical problem. The proposed approach is attractive because it can generate new action supervision from offline datasets without requiring additional environment interactions or privileged information. The presentation and visualization are very helpful for understanding.

  2. The experiments are comprehensive and well-designed. The authors evaluate the method on both real-world and simulated manipulation tasks, compare against several relevant baselines, and conduct ablation studies to validate key components. The additional analyses of execution efficiency and recovery from failure states help explain how the proposed method improves policy performance.

Critical weaknesses
  1. The candidate targets selection procedure is not sufficiently described. Given a source, the method must first identify candidate targets so that the IDM can generate the actions bridge. While the authors mention using visual encoder features to retrieve candidate targets, it is unclear how this retrieval is performed. In particular, for S2 and S3, is a target trajectory selected first and then a target observation is selected from that trajectory? What similarity metric or ranking criterion is used, and how are the candidate budgets allocated across sources? Since the quality of candidate retrieval directly affects which useful connections can be discovered, it's better to provide a more detailed description of this procedure, accompanied by an algorithm or a visualization.

  2. The method requires that the initial dataset contains useful segments that can be connected through short action bridges, but the impact of initial dataset coverage is not sufficiently studied. It would be helpful to evaluate how the number or diversity of successful demonstrations affect the number of accepted bridges and downstream policy performance improvements. This would help characterize the conditions under which the proposed method is effective.

I would be willing to increase my score if the concerns are sufficiently addressed.

Questions for authors
  1. Are the bridges mostly free-space motions, or do they also involve manipulation and interaction with objects? It will be helpful to visualize some representative (s, g) observation pairs.
  2. How many bridges are created, for example, in the Square task? It would be helpful to compare the sizes of the original and augmented datasets.
  3. Is it possible that for a highly isolated source trajectory, there is no targets that can be connected within the given horizon? Does this happen in the experiments?
  4. In real robot experiments, do the authors deliberately collect mixed-quality demonstrations?
5 comments
AuthorsOpenReview ↗

Response by authors (1/2)

“…the impact of initial dataset coverage is not sufficiently studied.” (W2)

We sincerely appreciate this thoughtful suggestion, as the experiments it led to has helped us strengthen the analysis in our work. From further experimentation, we solidify the intuition that stitching helps more as the quality of data decreases. Further, over different quantities of total data, stitching's gain is preserved. We show visualization of results in Section 3, "Impact of quality and quantity of data on stitching" in https://needle-work.github.io/rebuttal/#rebuttal-composition.

First, holding the number of successful demonstrations fixed, we vary the percentage drawn from the suboptimal pool (Mean ± std. error over five training seeds.):

Suboptimal demonstrationsDP success (%) ↑NEEDLEwork success (%) ↑Difference (percentage points) ↑
25%40.4 ± 1.639.2 ± 1.9−1.2
50%35.2 ± 1.240.8 ± 1.7+5.6
75%22.0 ± 1.727.6 ± 1.3+5.6
100%20.4 ± 0.724.8 ± 0.8+4.4

Stitching gives larger gains in the mixtures with more suboptimal demonstrations than in the 25% mixture. The gain is smaller when all demonstrations are suboptimal, likely due to having no demonstrations from the optimal pool to stitch onto. The corresponding demonstration-length analysis shows that in the presence of even small amounts of optimal demonstrations, the median length after stitching decreases substantially:

Suboptimal demonstrationsMedian recorded length (steps)Median shortest route after stitching (steps) ↓
25%143.5122.5
50%338.5133.5
75%495.5137.5
100%495.5386.0

Next, holding the mixture at 50% suboptimal demonstrations, we vary the number of successful demonstrations. We observe a positive mean gain at all three dataset sizes. (Mean ± std. error over five training seeds.):

Successful demonstrationsDP success (%) ↑NEEDLEwork success (%) ↑Difference (percentage points) ↑
2013.2 ± 0.516.8 ± 1.6+3.6
3035.2 ± 1.240.8 ± 1.7+5.6
40 (manuscript setting)39.6 ± 1.547.2 ± 1.4+7.6

“The candidate targets selection procedure is not sufficiently described.” (W1)

We thank the reviewer for pointing out this confusion from the paper. Per the reviewer’s suggestion, we have added both a clearer algorithm and animated visualization to Section 7, "How are target states retrieved?" in https://needle-work.github.io/rebuttal/#rebuttal-algorithm, and also made this addition to the manuscript.

To the reviewer’s questions: for S2 and S3 we do not select a target trajectory first. Target states are selected directly from all eligible states, pooled across trajectories. Exhaustive sampling would provide the broadest coverage of candidate pairs. However, generating and verifying bridges for all O(N2)O(N^2) source–target pairs is quadratically more expensive as datasets increase, so we distribute a limited budget across sources and visual distances.

First, we exclude target states that cannot provide the required step saving, though this restriction applies only to shortcuts within successful demonstrations (S1/S2), but not failure-to-success recovery (S3). Retrieval then uses cosine similarity in the frozen visual features the policy is trained on. Since the inverse dynamics model has a fixed horizon of HH, we calibrate a visual-distance cutoff from recorded observations HH steps apart. For S2 and S3, eligible target states are pooled across trajectories. We sample round-robin across visual-distance bins for each source, then allocate the overall candidate budget round-robin across source states with available targets.

“Are the bridges mostly free-space motions, or do they also involve manipulation and interaction with objects?” (Q1)

We thank the reviewer for pointing out this missing piece of analysis. Bridges include both free-space motions and object interactions. We provide a thorough breakdown of the different types of stitches in each task in Section 5, "Composition and total percentage of stitched dataset" on https://needle-work.github.io/rebuttal/#rebuttal-dataset-contact. Per the reviewer’s suggestion, we also add visualizations for representative (s,g)(s, g) observation pairs and stitch examples for each of these categories.

Using simulator state for analysis, we classify the 146 / 212 / 423 bridges whose stored actions match the selected training bridges exactly on Can / Square / Transport. We provide the following rounded percentages, together with representative bridge visualizations on the accompanying webpage:

Bridge behaviorCan (%)Square (%)Transport (%)
Free space708316
Touching without grasping872
Holding throughout11875
Picking up822
Releasing315
AuthorsOpenReview ↗

Response by authors (2/2)

“Is it possible that for a highly isolated source trajectory, there is no targets that can be connected within the given horizon?” (Q3)

We appreciate this attentive question. It is indeed possible for a trajectory to have no targets that can be connected within the given horizon. Taking this suggestion, we provide deeper analysis on exactly why this is the case in the latter half of Section 5 in "Composition and total percentage of stitched dataset" in https://needle-work.github.io/rebuttal/#rebuttal-dataset-isolated.

In our experiments, every successful demonstration has retrievable candidates, but some have all candidates rejected by the verifier. Counting the bridges the verifier accepts, before the training subset is selected:

TaskDemonstrations never used as a bridge source (of 40)Their lengths (steps), median [min–max]Other demonstrations' lengths (steps), median [25th–75th percentile]
Can12109 [98–119]420 [119–534]
Square18134 [123–155]540 [469–653]
Transport3413 [392–425]940 [425–1047]

Every demonstration without a retained source bridge is at or below its task’s median length. These demonstrations are consistently among the shortest, so they offer less opportunity for shortening, and their original recorded supervision remains available for training.

“It would be helpful to compare the sizes of the original and augmented datasets.” (Q2)

We agree that this is indeed a helpful number to include. Across the three Robomimic tasks, bridges consist of 19.6, 8.3, and 6.5% of the training windows on Can, Square, and Transport, respectively. On real robot tasks, 11.9% of the training windows on Sweater Folding include bridges, and 10.9% of the training windows on Dish Racking include bridges. The main Robomimic policies are trained on 254, 245, and 441 bridges on Can, Square, and Transport, respectively.

“In real robot experiments, do the authors deliberately collect mixed-quality demonstrations?” (Q4)

We thank the reviewer for this important clarification. We collect a mixture of suboptimal + optimal demonstrations for real robot experiments. Our suboptimal demonstrations are similar in nature to the mixed-human demonstration from Robomimic, consisting of some stalling, detours, and repetitive motions.

As we get close to the end of the author-reviewer discussion period, we wanted to (a) confirm that our above response answered at least some of the questions the reviewer had and (b) ask if there was anything else we could do to improve your assessment of our work. Regardless, thank-you again for the feedback you've already provided.

Reviewer_voEDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed NEEDLE: an offline method for improving robot policy training data by connecting useful segments from existing demonstrations. The motivation is that inefficient or failed demonstrations can still contain useful behavior that should not simply be discarded, and instead re-used.

The authors develop an inverse dynamics model based on diffusion, that proposes short action bridges between offline observations, and an action-conditioned verifier that selects which bridges to include in training. These bridges provide additional action supervision while preserving the original demonstrations, without generating intermediate images or requiring further environment interaction.

The paper evaluates the approach in simulation and on real-robot sweater-folding and dish-racking tasks, and demonstrates improvements over several baselines. Several ablations also evaluate whether the verifier and generated bridge actions contribute to these improvements, as compared to the gains coming only from reweighting the original data.

Critical strengths

The method addresses a practical problem: how can we recover useful behavior segments that can be present in inefficient or failed demonstrations. The approach is interesting in that it adds alternative action supervision without collecting any new robot interaction data, which is often operationally expensive.

The real-robot results are a clear strength of the paper. NEEDLE achieves higher success rates than the evaluated baselines on both sweater folding and dish racking. These gains provide encouraging evidence of the practical benefits of the approach on real-world tasks.

The ablations help explain where the improvements in the approach come from. They help understand the importance of the verifier-based selection stage of the method and show that generated bridge actions are useful beyond data reweighting.

The videos in the supplementary website -of teh real robot rollouts make teh qualitative results clear and easy to follow. The evaluation videos also make the robot’s behavior easier to inspect and provides clarity on the exact nature of the tasks evaluated.

Critical weaknesses

The results show that the verifier improves bridge quality, but accepted bridges can still fail to reach their targets and introduce incorrect action supervision. I would like to see a clearer analysis of when these errors hurt the learned policy, particularly on tasks that require precise manipulation.

The real-robot results are encouraging, but the evaluation is limited to two tasks and two baselines. This makes it difficult to judge how broadly the method applies. Evaluating more diverse tasks would be helpful in understanding the broader applicability of the method. Evaluation on diverse tasks is particularly important given the simplicity and elegance of the proposed method in terms of not being conceptually limited to a particualr category of tasks.

The tasks evaluated (and as seen from the video rollouts) are very quasi-static. It would be helpful to analyze the approach on more dynamic tasks and on tasks that require repeated motions (like pouring, wiping etc.) which are common in daily life, and where the applicability of the proposed stitching method is intuitively unclear.

Questions for authors

How do inaccurate accepted bridges affect downstream policy performance, especially on tasks requiring precise manipulation? It would be helpful to analyze which errors are most harmful, and provide qualitative analysis of this.

Could you evaluate on more real-world tasks? Additional tasks and baseline comparisons would help clarify how broadly the method is applicable.

How would the approach perform on more dynamic tasks or tasks involving repeated motions, such as pouring or wiping? How would stitching as in the proposed method be able to distinguish necessary repetitions from inefficient detours in execution?

6 comments
AuthorsOpenReview ↗

Response by authors (1/3)

“How do inaccurate accepted bridges affect downstream policy performance, especially on tasks requiring precise manipulation?” (W1, Q1)

We thank the reviewer for this keen question. We’d like to highlight that our training method does not assume an exact connection between a bridge and the target's remaining actions.

Our sampling method intentionally masks loss signal at actions after the bridge endpoint (Figure 3 of the main paper). The target's remaining actions are learned separately, paired with their own recorded observations. Appending those actions directly to the bridge reduces downstream success on all three tasks. We show all results and additional visualizations in the webpage section “What if bridges don't match the remaining actions?” https://needle-work.github.io/rebuttal/#rebuttal-continuation

We first evaluate the downstream impact of this training design. Appending the target's remaining actions to the bridge's action-training window reduces policy success relative to our method on all three tasks (Mean ± standard error over five training seeds.):

Training designCan success (%) ↑Square success (%) ↑Transport success (%) ↑
Diffusion Policy (DP)72.8 ± 0.839.6 ± 1.546.4 ± 1.6
NEEDLEwork without verifier72.0 ± 1.340.4 ± 1.220.4 ± 1.3
Append target's remaining actions78.8 ± 0.842.0 ± 2.345.6 ± 1.2
NEEDLEwork83.6 ± 1.747.2 ± 1.453.2 ± 2.2

Taken together, we see that the verifier play an important role in filtering out harmful errors, and our training method ensures we're able to learn well despite any remaining imprecision.

To understand this further, we measure how well each bridge landing supports the rest of the task, in two ways. Open-loop replay: from the bridge landing, we replay the target trajectory's recorded remaining actions and compare the success rate to replaying the same actions from the exact target state. Closed-loop policy: from the bridge landing, a trained diffusion policy takes over, and we compare its success rate to the same policy taking over after the source's own recorded actions. A ratio of 1 means the bridge landing is as good a starting point as the recorded one:

TaskOpen-loop replay of the target's actions: success from landing ÷ success from exact target ↑Closed-loop policy: success from landing ÷ success after source's recorded actions ↑
Can0.32 [0.15, 0.48]0.99 [0.82, 1.21]
Square0.33 [0.11, 0.48]1.01 [0.70, 1.47]
Transport0.37 [0.24, 0.49]1.14 [0.96, 1.45]

Together, these results support training on bridges without assuming that their landings preserve the target's exact open-loop continuation. This suggests that bridges do not need to perfectly connect to the target's suffix trajectory to be usable for policy learning, so long as they are properly filtered. We provide further evidence on the importance of this filtering in the response below.

On precise manipulation specifically: Dish Racking is our most precision-demanding real-robot task (grasping a plate by its thin rim, a bimanual handover, and inserting it between rack poles barely wider than the plate), and NEEDLEwork improves success there by 18 percentage points over the strongest baseline (88% vs. 70%).

AuthorsOpenReview ↗

Response by authors (2/3)

“It would be helpful to analyze which errors are most harmful, and provide qualitative analysis of this.” (W1, Q1)

We agree that this is a helpful analysis, and thank the reviewer for this suggestion. We analyze two different types of bridge errors that occur in our dataset: pose error, where the arm lands >5cm from the target location, and grasp error, where the gripper drops the object during the bridge. We find that grasp errors hurt the policy more, lowering its performance below DP. We show all results and additional visualizations in the webpage section “Which bridge errors hurt, and does the verifier remove them?” https://needle-work.github.io/rebuttal/#rebuttal-errors.

To test which error type matters, we keep the paper's Can, Square, and Transport training sets fixed in size and composition (same number of bridges per stage, same source states where possible) and swap their bridges for bridges of one error type. Each replaced bridge is paired with one of the same stage, similar length, and similar steps saved, so the two sets differ mainly in error type. All three sets are trained with the same recipe and training seeds (mean ± standard error over three training seeds).

Bridge setCan success (%) ↑Square success (%) ↑Transport success (%) ↑
NEEDLEwork (paper reference)83.3 ± 2.947.3 ± 1.851.3 ± 2.4
Pose-error bridges71.3 ± 2.438.0 ± 1.248.0 ± 2.3
Grasp-error bridges66.7 ± 1.831.3 ± 1.839.3 ± 3.5

Grasp errors cost more than pose errors on all three tasks (Can −4.7 ± 0.7, Square −6.7 ± 1.8, Transport −8.7 ± 5.8 percentage points), and training only on grasp-error bridges lowers success below DP on every task (DP: 72.0 / 40.0 / 47.3).

Our verifier admits far fewer false positives than the value function on both error types. Bridges make up 19.6 / 8.3 / 6.5 % of training windows; within those bridges, the error types are:

QuantityCan (%)Square (%)Transport (%)
Within 5 cm, no grasp error81.093.382.7
Pose error (>5 cm)11.05.712.7
Grasp error8.01.04.7

Together, grasp-error bridges make up only about 1.6%, 0.1%, and 0.3% of all training windows on Can, Square, and Transport.

We compare the amount of grasp errors falsely admitted by our verifier, compared to the value function learned and used by IDQL and AWR.

Candidate selectionCan grasp errors (%) ↓Square (%) ↓Transport (%) ↓
Kept by verifier8.01.04.7
Kept by value function19.39.018.7
Rejected by verifier27.317.731.7

We also calculate the average pose error of the end effector with the same selection criterion used by the value function above.

Candidate selectionCan (cm) ↓Square (cm) ↓Transport (cm) ↓
Kept by verifier2.852.193.46
Kept by value function4.293.5810.05
Rejected by verifier6.386.1717.16

“How would the approach perform on more dynamic tasks or tasks involving repeated motions, such as pouring or wiping?” (W2, W3, Q3)

We thank the reviewer for this suggestion. We find that our method also improves success on a dynamic, non-quasistatic task. We test our method in the third-party RoboDyna environment, where a robot places a cup onto a moving belt (Mean ± standard error over five training seeds.):

MethodSuccess (%) ↑Environment steps per successful trial ↓
Diffusion Policy88.4 ± 1.0281.3 ± 7.9
AWR85.2 ± 0.8273.9 ± 4.7
NEEDLEwork94.0 ± 1.1244.0 ± 10.6

We show visualizations of the task and rollouts in Section 4, "Experiments on dynamic and more diverse tasks." of https://needle-work.github.io/rebuttal/#rebuttal-dynamics.

AuthorsOpenReview ↗

Response by authors (3/3)

“How would stitching as in the proposed method be able to distinguish necessary repetitions from inefficient detours in execution?” (Q3)

We appreciate this nuanced question. Our stitching pipeline can distinguish necessary repetition from inefficient detours when there is observable progress in observation space. For this work, the verifier is conditioned on source and target observations and the proposed actions. As long as these image and proprioception observations provide meaningful signal to progress, a distinction should be possible. On the real robot this shows up as less repeated motion: pooled over both tasks, 9.2% of NEEDLEwork's gripper travel re-enters a previously visited 2 cm region, compared with 10.8% for DP and 16.7% for AWR (Appendix B.1).

Nevertheless, other modalities, such as force/torque, could help distinguish progress that is not observable from images alone. We agree that repeated-motion tasks with such hidden progress are an important direction for future work, and have added this point to the Limitations and Future Works section of our updated manuscript.

“Could you evaluate on more real-world tasks? Additional tasks and baseline comparisons would help clarify how broadly the method is applicable.” (Q2)

We agree that broader real-world evaluation is valuable. Our current evaluation covers six simulated benchmark tasks and two real-robot tasks (Sweater Folding and Dish Racking), and during the rebuttal we added a seventh third-party dynamic task (RoboDyna conveyor, see above), where NEEDLEwork improves success over both Diffusion Policy and AWR (94.0 vs 88.4 and 85.2). While we may not have sufficient time during the rebuttal session to collect, train, deploy, and evaluate multiple new real-world tasks, we will begin working on this suggestion on more dynamic tasks in real, and aim to include it in the final released manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response by authors

We thank the reviewers for their time, careful reading, and constructive suggestions. Each question has helped us strengthen the completeness of our experiments and analysis, which we have added to our updated manuscript. We highlight all changes in the manuscript in blue. For ease of browsing, we provide the numerical results in tables here and additional visualizations on the accompanying rebuttal webpage https://needle-work.github.io/rebuttal/.

Precision of Bridges

We would also like to use this space to address a common concern from reviewers regarding the precision of our proposed bridges, how we address them, and how exactly they fail. We would like to clarify that a key contribution of this paper is indeed how to handle imprecise bridges, and we do not assume them to be perfect. We thank the reviewers for their keen questions in ensuring we have more thorough analysis to address these concerns.

While we provide detailed responses to each reviewer, we would like to highlight the main experiments added in Section 4.4 of the manuscript, and Sections 1 and 2 in the rebuttal webpage. We quantify which types of bridges are truly harmful (thanks to reviewer voED’s suggestion), and highlight how our trained verifier greatly reduces the presence of these bridges in our final stitched dataset (thanks to reviewer voED and 4XGn). In particular, our verifier largely removes most of the harmful imprecisions from bridges; and especially when compared to the value function learned from baselines, our verifier admits far fewer false positives into the final dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.