- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper applies influence function approach to data curation in a multiple-domain setting: choosing source-domain data (e.g., simulation data) to complement target-domain dataset (e.g., real robot) for sim-and-real co-training.
The key finding is that existing influence function approach maximizes a single scalar objective associated with the overall improvement on the target-domain utility, which loses information about “how” each source-domain data point affects the utility of different target domain data points (influence pattern). The paper proposes a new utility function for data curation that reflects both overall improvement and the influence pattern. Maximizing the new utility function helps with selecting source-domain data with broader coverage of the influence pattern.
Critical strengths
- The work introduces a useful insight for data curation that the coverage of the selected dataset can influence the final co-training performance sometimes significantly. This coverage can be omitted by existing approach.
Critical weaknesses
- Presentation: The clarity of the presentation can be significantly improved in general, which currently makes it hard to give review feedback on the actual contributions made in this work.
- For example, starting from the second paragraph of the Introduction, the presentation becomes jargon-heavy with a lot of abstract concepts of existing and proposed methods without a clearly streamlined core message and clear explanation of what the jargons are. I recommend to move large part of the detailed explanation to Related Work and extract a few most important, focused messages and intuition to be presented in the Introduction. This also avoids a lot of repetition between the two sections.
- Mathematical clarity: There are also other relevant small things. For example, at line 162, the behavior cloning loss is not properly defined; at the next line, the return is not defined either. I encourage the authors to take greater care when writing mathematical expressions and take a pass to fix such places.
- Contribution relative to existing work: the paper makes it really hard to distill the actual contribution made in this work relative to existing work. For example: section 4.2 is in its essence taken from CUPID with the main novelty being applying the influence profile approach in CUPID to the multi-domain setting. However, everything is presented as “Approach” in this work.
- It is slightly confusing to call “source” and “target” domains in the single-simulator setting in the context of sim-and-real co-training. It would be good to clarify this setting as a “sanity check”.
- Figure 3: in large part of the settings, the error bars in this figure are heavily overlapping between methods, which suggest the number of random seeds being insufficient to support statistical significance. Hence, the claimed improvement by the proposed approach is not supported here.
- The implicit function theorem used to construct the influence function requires the Hessian matrix to be invertible. It is not tested and discussed anywhere whether and when this assumption is satisfied in practice.
Questions for authors
N/A
6 comments
Response to Reviewer QAkw - Part 2
W3: Methodological contributions relative to CUPID
R3: Thank you for highlighting the need to distinguish this contribution more clearly from CUPID.
First, we provide a new mathematical perspective that reveals diversity information left unused by CUPID's ranking rule. In Section 4.2, we express each demonstration's influence across target-policy executions as a profile. Our analysis shows that these profiles describe both individual learning value and differences in how demonstrations support target behavior. CUPID selects demonstrations using an aggregate utility score. These individual scores do not capture the relationships needed to assess redundancy and coverage within a selected dataset.
Second, we propose an influence-function-based decomposition to extract utility and coverage information. In Section 4.3, we separate each profile into its component along the return vector and an orthogonal component. The former determines trajectory-level utility, while the latter retains differences across target executions and provides a representation for measuring set-level coverage. This decomposition allows us to assess individual usefulness and collective diversity through the same closed-loop target behavior.
Third, we propose a performance-aligned subset optimizer that uses both measures during selection. In Section 4.4, we restrict coverage-driven selection to candidates with positive utility and sufficient alignment with target return. We then balance individual utility with the additional coverage each candidate provides. We update coverage gains as the selected set grows, so selection accounts for demonstrations already retained. This favors useful and complementary demonstrations while reducing redundancy.
Our experimental contribution is a systematic evaluation across single-simulator, sim-to-sim, and sim-to-real settings. To our knowledge, we provide the first systematic study establishing the value of simulation data curation for sim-to-real policy co-training. This contribution concerns both the new evaluation setting and the evidence that active curation can improve real-world performance.
W4: Source and target terminology in the single-simulator setting
R4: The single-simulator experiment serves as a sanity check of curation without domain shift. Thank you for this helpful interpretation and for highlighting the need to clarify our terminology.
We use “source” for the domain supplying candidate demonstrations and “target” for the domain in which the policy is evaluated. In the single-simulator setting, these domains share the same environment. The terms distinguish their roles in curation without implying a domain gap. We retain them to provide a consistent formulation across single-simulator, sim-to-sim, and sim-to-real settings.
Following your suggestion, we have explicitly stated in Section 4.1 that the single-simulator and sim-to-sim settings serve as sanity tests for our sim-to-real study. They evaluate curation without domain shift and under a controlled simulator shift, respectively.
Revisions: Section 4.1, clarification of the three source-to-target settings and the role of single-simulator and sim-to-sim experiments as sanity tests.
Response to Reviewer QAkw - Part 3
W5: Overlapping error bars and evidence across training seeds R5: Following your insightful advice, we have conducted five additional random seeds for the sim-to-sim experiments and combined them with the original three, yielding eight seeds per configuration. Thank you for encouraging us to strengthen the evaluation.
The expanded results show that TUCO maintains its strong performance across tasks and selection budgets while exhibiting the lowest aerage cross-seed standard deviation among the compared curation methods. Error-bar overlap is reduced in the updated comparison. These results provide stronger evidence for TUCO's overall performance and consistency across training runs.
The remaining variability is consistent with a recognized challenge in robot data curation: the scores used to select training data can themselves depend on stochastic learning and noisy influence estimation. DemInf reports substantial cross-seed instability for parametric mutual-information estimators (Hejna et al., 2025). CUPID discusses ensembling attribution scores across independently trained models to account for learning non-determinism (Agia et al., 2025), while DataMIL uses clustering to reduce noise in sample-level influence estimates (Dass et al., 2026).
During this rebuttal, we have updated Figure 3 and the corresponding experimental analysis with the eight-seed results and clarified the statistical convention in the caption.
Revisions: Figure 3, eight-seed results and caption; Section 5.1, updated Sim-to-Sim Co-Training analysis.
W6: Hessian invertibility and practical influence estimation
R6: Thank you for raising this point. Our implementation follows the practical influence estimator used by CUPID, and this approximation is already specified in Appendix B.2, Equations (11)–(12). The nonsingular-Hessian assumption in Appendix B.1 supports the exact influence-function derivation. The implementation uses a projected-gradient curvature matrix for efficient estimation.
Specifically, we follow CUPID (Agia et al., 2025) in using the TRAK-based approximation (Park et al., 2023). As defined in Appendix B.2, the matrix used for scoring is
=\widetilde{\mathbf G}^{\top}\widetilde{\mathbf G}+\eta\mathbf I_d,$$ where the rows of $\widetilde{\mathbf G}$ are projected source gradients. This is the same matrix construction and inversion used in CUPID's released TRAK implementation (`get_x_xtx_inv`). Our zero-damping setting also matches CUPID's provided experiment script (`lambda_reg=0.0`). In this setting, invertibility requires the projected gradient matrix to have full column rank. Positive damping makes the matrix positive definite. Thus, the relevant practical condition concerns the projected Gram matrix specified in Appendix B.2, rather than the full neural-network Hessian. **TUCO builds its utility–coverage decomposition and subset selection on these established influence estimates.** Appendix B.3 further states that the decomposition identities hold for any fixed influence profiles and a nonzero return vector. Our implementation therefore follows the estimation procedure used in prior work, while our contribution concerns how these estimates guide set-level curation. **References:** - Hejna, J., et al. (2025). [Robot Data Curation with Mutual Information Estimators](https://arxiv.org/abs/2502.08623). *Robotics: Science and Systems (RSS).* - Agia, C., et al. (2025). [CUPID: Curating Data your Robot Loves with Influence Functions](https://arxiv.org/abs/2506.19121). *Conference on Robot Learning (CoRL).* - Dass, S., et al. (2026). [DataMIL: Selecting Data for Robot Imitation Learning with Datamodels](https://proceedings.iclr.cc/paper_files/paper/2026/hash/033d9e8dbbbc0ad90e59222cf1db0fc2-Abstract-Conference.html). *International Conference on Learning Representations (ICLR).*
Response to Reviewer QAkw - Part 1
We thank the reviewer for your careful reading and insightful feedback. Below, we clarify the mathematical definitions and distinguish our methodological contributions from the attribution foundation provided by prior work.
W1: Clarity and organization of the presentation
R1: As you suggested, we have meticulously rewritten the abstract, Introduction, Preliminary, and Methodology to make the core idea and contributions easier to follow. Thank you for identifying how the abstract terminology obscured our message.
W2: Incomplete definitions of the behavior-cloning loss and return
R2: We have clarified both definitions in the revised paper. Thank you for highlighting these omissions. The imitation loss measures how well the policy reproduces demonstrated actions, while the return measures whether a complete target-domain execution succeeds.
Behavior-cloning loss. We now define ℓ(ot,at∣θ) as the supervised imitation loss for predicting the demonstrated action at from observation ot. For a demonstration ξ=(ot,at)t=1Tξ of length Tξ, the trajectory loss used in influence attribution is
=\sum_{t=1}^{T_\xi}\ell(o_t,a_t\mid\theta).$$ This definition explicitly connects the demonstration-level influence calculation to the policy's imitation training loss. **Target-domain return.** We now explicitly define the return of a target-domain rollout $\tau$ as $$R_{\mathrm t}(\tau):= \begin{cases} +1, & \text{if the target task is completed successfully},\\\\ -1, & \text{otherwise}. \end{cases}$$ The curation objective uses its expectation, $J_{\mathrm t}(\theta)=\mathbb E_{\tau\sim p_{\mathrm t}(\cdot\mid\pi_\theta)}[R_{\mathrm t}(\tau)]$, where $p_{\mathrm t}$ is the target rollout distribution. Under this definition, expected return equals twice the task success probability minus one, so maximizing it is equivalent to maximizing task success probability.