- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies compositional generalization in robot control: learning from a limited set of task combinations and transferring to unseen combinations of familiar factors. The proposed method uses a shared diffusion network trained with independent dropout of task factors and composes factor contributions during inference. Under approximate conditional independence and additional regularity assumptions, the paper analyzes score decomposition error and its propagation to closed-loop trajectory deviations. Experiments in simulated drone racing show that factor dropout improves generalization when training coverage is sparse. However, composed inference does not outperform joint conditioning of the same network, and the computed trajectory bounds do not establish task success in the evaluated settings.
Critical strengths
-
The paper addresses a meaningful problem. Collecting demonstrations for every combination of task factors becomes expensive as the number of factors grows. Exploiting task structure within a shared policy is a relevant direction for improving data efficiency in robot learning.
-
Some controlled experiments provide useful evidence about the proposed training procedure. Table 2 evaluates sparse task coverage using two covers and two training seeds, showing benefits from factor dropout. The controlled factor-interaction experiment also helps characterize when composed inference deteriorates. These results provide insight beyond a comparison against an unconditioned baseline.
Critical weaknesses
- The experimental evidence does not sufficiently establish the practical advantage of the complete method over simpler alternatives and representative existing approaches.
Most comparisons involve variants of the same diffusion architecture. The K-network baseline is a PoCo/DSE-style adaptation to the authors' factor decomposition, rather than separate evaluations of those methods. Its representativeness therefore requires further justification. The baseline without factor inputs also receives less task information, so improvements over that baseline alone do not isolate the value of the proposed mechanism.
Table 1 reports 87% held-out gate passage for composed inference, compared with 97% for joint inference using the same trained network. Joint inference without factor dropout also reaches 97%. Table 2 provides evidence for factor dropout under sparse coverage, but does not establish an advantage from explicit composition. For two factors, composed inference additionally requires three network evaluations per denoising step, compared with one for joint inference. The authors acknowledge that composition primarily provides analytical structure. However, the resulting rigorous trajectory bounds do not certify gate passage in the experiments, leaving the practical benefit of this additional computation insufficiently demonstrated.
The validation scope is also limited. All experiments concern simulated drone flight, without evaluation in another robotic domain or on physical hardware. Importantly, Appendix C states that the main visual checkpoints were trained without factor dropout and do not exercise the proposed composition mechanism. Those results therefore do not directly validate the complete method. Table 1 reports a single evaluation with gates scored independently, which does not establish full-race completion rates or robustness across independent training runs.
- The presentation makes it unnecessarily difficult to distinguish the central contribution, supporting evidence, and scope of the claims.
The introduction introduces score decompositions, error bounds, trajectory radii, and ODE sensitivity constants before sufficiently developing the problem and methodological intuition. It combines motivation, technical analysis, and detailed experimental interpretation, making the main argument difficult to follow. A clearer introduction would retain the essential intuition and move detailed mathematical conditions and bounds to the method section.
Although relevant references are included, the introduction does not sufficiently explain the precise limitations of closely related approaches or distinguish the contribution from existing score composition, condition dropout, and task-conditioned diffusion policies. The abstract reports qualitative improvements without concrete numerical results and evaluation conditions, making the magnitude of the empirical benefit difficult to assess.
Figure 1 requires substantial effort to relate multiple tasks, spatial views, and speed profiles. Table 1 highlights the proposed methods in bold even when their results are not the best, without clearly explaining the emphasis convention. Finally, the paper lacks a concise concluding synthesis that separates experimentally supported findings from conditional theoretical results. Together, these issues weaken the clarity of the contribution and its evidential support.
Questions for authors
-
Please clarify the differences between the implemented K-network baseline and the original PoCo/DSE methods. How do these adaptations affect its representativeness, data utilization, and optimization difficulty?
-
Beyond providing analytical structure, what demonstrated practical benefit does composed inference offer over joint inference using the same network? Given that the rigorous bounds do not certify gate passage, what concrete decision can the current diagnostic support?
-
Table 15 reports 36.7% success and 15.0% crashes for zero-shot visual transfer, whereas the “released (paper)” checkpoint in Table 16 reports 42.5% success and 5.8% crashes. Do these results use the same checkpoint and evaluation protocol? Please explain any differences in model versions or evaluation conditions.
-
Are results from independent training runs or full-race completion evaluations already available for the comparisons in Table 1? If so, do they preserve the reported performance relationships? This question concerns existing evidence rather than a request for an extensive new experimental study.
2 comments
Follow-up on the Author Response
Thank you for the detailed response and the substantial revision. The new domains, additional training seeds, race-level results, and the rewritten introduction address most of my concerns. I will raise my score to 2. My remaining concern is that composed inference still shows no practical advantage over joint readout, so the main empirical contribution rests on factor-dropout training.
Title: Response to Reviewer Gijr
Thank you for the review. The revision answers each point below with new evidence: two added domains exercise the complete method, the same controlled comparison gives the same answer in all three, and the introduction, abstract and conclusion are rewritten.
Weakness 1. The claims rest on the same network with and without factor dropout, and that comparison gives the same answer in all three domains. We agree that the baseline receives less information and that the K-network row is confounded. The comparison we rest on is the same network with and without per-factor dropout at sum-sized coverage, on two disjoint covers and two training seeds per domain: drone racing 3-16% to 36-50% held-out gates; tabletop 52-71% to 96-99%; LiDAR racing 26-81% to 88-97%, crashes 18-28% to 2-9%; with a third latent factor (wheelbase, 72 tasks) a cover of 8 tasks gives 85-92% against 61% without dropout and 30% for the baseline. We keep the vision study in the appendix as preliminary. The composed-versus-joint half of this weakness is answered under Q2.
Weakness 2. Introduction, abstract, Figure 1, Table 1 and conclusion are rewritten as asked. The introduction now states the problem, gives the method with the additive-score intuition, lists four findings labeled F1 to F4, each tagged on the paragraph of Section 4 that establishes it, and names the gap to score composition, condition dropout and task-conditioned policies. The abstract carries the numbers; Table 1 bolds the best entry per column; Figure 1 shows one held-out task per domain; a conclusion separates experimental findings from theoretical results.
Q1. The K-network row is PoCo/DSE-style composition over named task factors with everything else matched; it is not the comparison our claims rest on. PoCo composes policies trained on different data sources by weighting their scores at inference; DSE composes a base skill model with task-specific experts. Both compose separately trained networks with the additive rule we use. Our row keeps that rule and training scheme (12 networks: one unconditional, eight track-only, three aperture-only, each trained on the tasks carrying its value) and changes only what is composed: named task factors. Nothing makes separately fit scores additively compatible, so their sum is unconstrained on unseen pairs; and the row changes parameter sharing, data allocation and optimization at once. We say so in the revision and report three seeds (0-6 of 30 held-out gates, no held-out race).
Q2. No gain in success rate over joint readout, and the paper never claimed one; the demonstrated benefits are a rule for which corrections to compose when a factor is observable, and a per-task residual that predicts composition failure. First, the decomposition says which readout to use when a factor is determined by the observation. In LiDAR racing (Section 4.3) the scan already identifies the circuit. On a sum-sized cover each track is trained at one speed level, so the track token becomes a proxy for that level, and joint readout on held-out combinations is erratic (34-87%). Dropping the track correction leaves a single-factor conditional that completes 88-97% of held-out laps with 2-9% crashes, at one forward pass per step; a jointly conditioned network has no term to drop. Second, the residual eps_D predicts composition failure per task. Across all twelve factored LiDAR checkpoints (Appendix D.2), on the six dropout-trained ones, every task composed inference passes has eps_D at most 0.29 and 39 of the 42 it fails have eps_D between 0.47 and 2.4; the three exceptions are tasks where the joint readout also fails, so the composed score agrees with a wrong joint score. Without dropout the residual predicts nothing. The decision it supports is concrete: drop a correction whose factor the observation fixes, and compute the residual of a new combination before deploying; at the training-task level (0.1-0.2) composed inference passed every task, above 0.47 none. The tube certificate ranks held-out tasks by closed-loop sensitivity, and we present it only as that ranking.
Q3. Same checkpoint, two evaluation runs with different spawn draws. Table 15 of the submission (now Table 17) is the original evaluation; the released row of Table 16 (now 18) re-evaluates it under the retraining study's spawn seed. We say so in the caption of Table 18.
Q4. Yes. Four training seeds and race-level completion are in Table 1, and every ordering holds. Four training seeds for the factored and no-dropout rows, three for the K-network: factored joint 101-102 of 104 gates and 29 of 30 held out on every seed; composed 97-98 and 25-26; no-dropout joint 101-102 and 28-29. Races (all gates, no crash): factored joint 17-18 of 20 and 5 of 6 held out on every seed; composed 14-15 and 2-3; oracle 19 and 6; baseline 3 and 1; K-network 8-10 and 0. Baseline and oracle are single runs.