acceptodds
Under review as a conference paper at ICLR 2027

PROJECTION ORDER MATTERS: LEARNING FROM UNREACHABLE TARGETS

Abstract

Robot faces learning from human mouth motion, prostheses from able-bodied motion capture and impaired musculoskeletal models from a healthy expert learn from targets they cannot reach. Some existing methods filter such demonstrations or retarget each onto the feasible set before training. When every target is unreachable, nothing survives strict feasibility filtering, and it is unclear whether to project targets before or after learning. This order matters through two competing effects; we make three contributions. (1) An exact finite-rank theorem: when the reachable set is a linear subspace, projecting first keeps unreachable directions from consuming a rank-limited linear student's capacity. (2) On curved reachable sets, projection and conditional averaging need not commute, so projecting noisy targets can bias the learning target; constrained-Bayes distillation (CBD) estimates the conditional mean with an offline teacher, projects it and distils the feasible target into a small deployment model. In controlled sweeps, CBD improves on project-then-fit at every tested curvature, while raw fitting is the best of the three orders in most ample-capacity cells. (3) The inverse that projects a target also returns commands, useful supervision even where the projected outputs are worse labels. On a saturating trajectory plant, pretraining a feasible-output student on them turns the worst learner into the best at smaller widths, a gain compute-matched alternatives do not reproduce; its advantage over them replicates on a torque-driven two-link arm in a predeclared design. Projection order is thus a design choice, not a convention.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, imitation learning, constrained regression, reachable sets, knowledge distillation, projection order

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8LdXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies how the order of fitting, averaging, and projection affects learning from unreachable demonstrations. It explains competing capacity and statistical effects, proposes constrained-Bayes distillation (CBD) and command-based intialization, and compares them with existing training strategies. Its contribution is primarily an analysis of these tradeoffs on simulated experiments.

Critical strengths

The paper identifies the tradeoff in learning from unreachable targets: projecting demonstrations before training can preserve limited model capacity, but nonlinear projection can also bias the regression target. The theoretical analysis and controlled experiments explain why making training targets feasible does not necessarily improve the final predictions.

Critical weaknesses
  1. The presentation lacks a clear hierarchy of claims, which makes the central contribution hard to extract. The title suggests a focus on when and why the order of projection affects learning, but the experiments interleaves too many things. Projection-order effects, CBD, and command-based initialization are discussed along with a lot of rankings, exceptions, and sensitivity results without sufficiently prioritizing the takeaway. Although individual findings are listed in detail, the evidence are not synthesized into a concise interpretation of the pros and limitations of each method. Hence, it is a bit hard for the readers to follow.
  2. The connection between the motivating applications and validation remains uncertain. The physical-face measurements demonstrate a reachability mismatch but do not evaluate the compared training strategies or establish the projection-order bias in the setting. Hence, how this study can practically guide the real-world facial expression transfer remains unclear.
Questions for authors
  1. What is the central conclusion about projection order that the experiments establish? Please distinguish conclusions supported across benchmarks from observations specific to each detailed particular setting, and clarify how the command-initialization results support or complement the projection-order analysis.
  2. What do the physical-face measurements establish about the relevance of the proposed methods? Do your existing measurements support either capacity competition or projection-order bias in facial imitation?
  3. What is your actual advance over the closest prior methods?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 8LdX

Thank you for identifying the presentation problem. We have reorganized the Introduction and Experiments around one central insight: making a demonstration feasible can save a learner's capacity while changing the target it learns. CBD addresses the target bias; command initialization is a complementary use of the same inverse. Figures 3 and 6 now make the results and their behavioural meaning easier to see.

  1. What is the central conclusion about projection order?

The linear theorem isolates the capacity benefit of projecting first. On curved or bounded sets, the nonlinear analysis explains the competing effect of interchanging averaging and projection. The experiments then show when those effects matter for a finite student.

CBD improves on PTF pooled within each tested curvature of the curved benchmark and at every width on the face. PTF is never significantly the best of the three output-label orders on the curved benchmark, trajectory plant or face (Figure 3; Tables 1–2). Whether CBD also beats RAW depends on the setting: RAW takes 13 of 19 decided cells in the curved benchmark, and CBD the other six, all at the highest curvature or narrowest width (§5.1). RAW leads among the three output-label orders at every tested width on the trajectory plant and face.

The command experiments ask a closely related question: which product of an inverse solve is useful supervision? With the same inverse solve and second stage, command pretraining beats output imitation at every width and in every seed on the trajectory plant. It makes the feasible-output student the best learner at widths 8 and 16, and its advantage over compute-matched multi-start and curriculum transfers to the two-link arm (§5.2, Tables 1, 22, 24, 30).

  1. What do the physical-face measurements establish?

Our face is designed to be affordable: five actuators shape its mouth, yet it has to convey human speech. We have used the humanoid robot outdoors with passers-by for over a month. This is why the distinction between feasible movements and useful training targets matters to us in practice.

The original measurements established the reachability mismatch: 86% of held-out human frames lie beyond a fitted limit (Figure 1b). The new §5.3 adds a learning comparison on real speech and an estimate of projection-order bias, which is concentrated at jaw limits (Table 34). PTF is the worst output-label order at all four widths. At the smallest width, it trails RAW by 5.5% on the first model and 7.1% after re-measuring the face (Tables 2, 37).

Across 12 utterances played twice on the robot, PTF's mean peak opening above neutral is 12% below RAW's and below CBD's in every utterance (Table 38). The capacity mechanism is established by the rank-limited analysis; the new face experiments show the learning outcome and make the effect on speech visible.

  1. What is the advance over the closest prior methods?

Retargeting constructs feasible demonstrations before training (Yeh et al., 2025); extrinsic regression projects an estimated conditional mean (Lin et al., 2017); differentiable projection layers incorporate feasibility into training (Grontas et al., 2026; Utkarsh et al., 2026).

Our contribution is to analyse and compare projection orders for unreachable output supervision under finite student capacity. This yields the deployment-specific capacity result, the competing projection-order bias, and controlled comparisons that separate the quality of a population target from the performance of its fitted student. CBD implements the averaging-first target, and the command-versus-output experiment identifies which product of the inverse provides the better initialization. The revised §6 states this connection directly.

In the face experiment added in the Nov 13 revision, RAW had the lowest held-out error at all four widths. Among the two ways of making training labels feasible, CBD beat PTF, and the full ordering repeated with a refitted face model (Tables 2 and 37). Our contribution beyond retargeting and projection layers is to explain how the timing of projection changes what a student with limited capacity learns (§6). The command experiments then show how the same inverse can also supply a better starting point for a feasible-output student.

Before the discussion closes, could you let us know if any concerns remain after our response and the revised experiments? We would be glad to clarify them.

Reviewer_Tp7ZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to learn from demonstrations outside a system’s reachable output set. Projecting demonstrations before training can save model capacity, but nonlinear projection may introduce bias. The authors analyze the capacity effect for a rank-limited linear model and propose constrained-Bayes distillation (CBD), which estimates the conditional mean before projection and trains a small student on the projected targets. They also use commands returned by the inverse solver to initialize a command policy before end-to-end training.

Critical strengths

1.The paper gives a useful explanation of why projection can help or hurt learning. The finite-rank analysis shows how unreachable directions can consume model capacity, while the clipping example clearly illustrates the bias introduced by projecting noisy targets. 2.The command-pretraining experiments include relevant controls. Comparisons with longer training, compute-matched multi-start, curriculum learning, and output imitation provide evidence that the gains are not simply due to additional training or computation.

Critical weaknesses
  1. The role of CBD in the empirical results needs clarification. CBD does not consistently improve over RAW, and even Oracle CBD performs worse in the trajectory experiment. The largest gains come from command pretraining. This supports the value of a better initialization, but it is less clear what the experiments establish about the benefits of changing the projection order.
  2. The experiments rely heavily on aggregate error metrics, with no qualitative visualization of the simulated behavior. For the trajectory plant and two-link arm, the main text does not show target and executed trajectories or representative failure cases. This makes it difficult to understand how unreachable targets affect execution and what behavioral improvements the reported error reductions represent.
Questions for authors

1.Clarification: Oracle CBD underperforms RAW in the trajectory experiment despite using the true conditional mean. Can you quantify the roles of student approximation, optimization, and numerical projection error in this result? This would help clarify when changing the projection order is beneficial independently of command pretraining. 2.Could you show representative target and executed trajectories for the trajectory plant and two-link arm, comparing RAW, cold training, and command pretraining? Including both successful and failed cases would help clarify where the methods differ and what behavior command pretraining improves.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Tp7Z

Thank you for distinguishing the projection-order question from the command-initialization question. That distinction is central to the paper, and we have made it explicit in the revised Introduction and Experiments. We have also added the two pieces of evidence you asked for: quantitative diagnostics of the oracle-CBD reversal in Appendix I, and matched target/executed trajectories in Figure 6.

  1. Why does oracle CBD trail RAW, and what do the results establish about projection order?

Changing the order has a benefit independently of command pretraining: CBD beats PTF pooled within each tested curvature of the curved benchmark and at every width on the face (Figure 3; Table 2, t = 5.7–7.8). The face comparison repeats after re-measuring the plant (Table 37). In hardware playback, PTF's peak opening above neutral is below CBD's in all 12 utterances (Appendix C.9).

The oracle result addresses a different part of the question. In the population ideal, RAW followed by exact projection and oracle CBD target the same constrained Bayes predictor. Their finite students fit different functions: the raw conditional mean and its projected version. An oracle teacher therefore does not make the trained student optimal.

We examined numerical inversion at fixed predictions. On the trajectory plant at seed 0 and width 8, directly inverting the true conditional mean yields less than half the conditional MSE of either student. Increasing the search budget fourfold changes their errors by under 3% and leaves their ordering unchanged (Table 51).

The fifteen-seed replay then explains how the errors translate into task risk. At width 8, the oracle-minus-RAW gap is 1.02 ×10⁻²; the cross-term difference relative to the common numerical reference accounts for 0.90 ×10⁻² of it and favours RAW. The gap falls to 0.82 and 0.57 ×10⁻² at widths 16 and 32 (Table 50). Measured against each method's target function, the deployment inverse also removes about half of RAW's squared error, versus 9–20% of oracle CBD's (Table 28).

Finally, refitting frozen targets on one split with three initializations shows substantial variation at width 8; at width 32, RAW remains better in all three refits on the trajectory plant (Table 52). Width changes capacity and optimization together. These checks show how a better target can still yield a worse finite student, which is the point of the oracle comparison.

  1. Target and executed trajectories, including failure cases.

Figure 6 now compares RAW, cold feasible training and command pretraining on the trajectory plant and both arm regimes, at seed 0 and width 8. Each condition shows a typical context and the least favourable warm-versus-cold context among 128 held-out contexts. On the trajectory plant, the warm start tracks worse than cold training in that case (ΔMSE +0.030). Even the least favourable arm cases favour pretraining. The supplement contains full time and command traces for all 128 contexts per condition.

We also filmed the physical face. Its measured opening follows the modelled opening with a mean within-take correlation of 0.95 after the measured lag, and PTF's mean peak opening above neutral is 12% below RAW's (Table 38). This gives a physical example of how changing the labels affects execution.

Your oracle-CBD question prompted us to rerun the inverse search without retraining the students. On the trajectory plant at width 8 and seed 0, a longer search with multiple starts changed RAW's and oracle CBD's conditional MSE by less than 3%, and RAW still led. Directly inverting the true conditional mean gave less than half either student's conditional MSE (Appendix I, Table 51). RAW and oracle CBD share the same ideal deployed predictor, but their finite students fit different functions. Figure 6 in the Nov 13 revision shows matched target and executed trajectories on the plant and arm, including a plant case where cold training tracks better than command pretraining.

Is there anything in the oracle comparison that you would like us to explain further?

Before the discussion closes, could you let us know if any concerns remain after our response and the revised experiments? We would be glad to clarify them.

Reviewer_L264OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether unreachable demonstration targets should be projected onto the reachable set before learning or learned first and projected later. It proves that, for a linear reachable subspace and a finite-rank student, projecting first prevents unreachable directions from consuming model capacity. For curved or bounded reachable sets, projection and conditional averaging generally do not commute, creating projection-order bias. The paper proposes constrained-Bayes distillation (CBD), which estimates the conditional mean with a teacher, projects that estimate, and distils it into a deployment model. It also shows that the commands returned by the projection inverse can provide a useful warm start for a feasible-output student. Experiments on a parabola, a simulated actuator with saturation and a dead zone, and a two-link arm show that no order is universally best: RAW is strongest when capacity and curvature are favorable, while CBD is more useful under high curvature or limited student capacity.

Critical strengths
  1. The theoretical contribution is clear and its assumptions are carefully stated. The finite-rank capacity-drain theorem, curvature-induced bias, and first-order behavior at nonsmooth boundaries correspond to distinct mechanisms. The paper also distinguishes exact projections from practical approximate projectors.
  2. The experiments are carefully controlled. The authors vary curvature, noise, student width, and teacher quality, use multiple seeds and predeclared comparisons, include compute-matched controls, and report an independent reimplementation. These choices help separate target bias, capacity, and optimization effects.
Critical weaknesses
  1. Practical validity is supported mainly by synthetic and simulated benchmarks. The three physical faces are used only to estimate geometry and cannot measure the projection bias studied by the paper. Direct evidence that the proposed effect is large in realistic demonstration distributions is therefore missing.
  2. CBD is expensive and lacks a validated selection rule. It requires an additional teacher and projection calls. On the trajectory plant, CBD costs substantially more than RAW, and the paper does not provide a validated procedure for deciding in advance among RAW, PTF, CBD, and EMBED.
Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer L264

Thank you for recognizing the theoretical distinctions and the controlled comparisons. Your two concerns helped us focus the revision on the practical question: what does changing the projection order buy on a real system, and what does each choice cost?

Our face is deliberately designed to be affordable, with only five mouth actuators to convey human speech. We have used the humanoid robot in a public outdoor setting for over a month, speaking with passers-by. The revision now connects that application to controlled learning comparisons and measured execution on the robot.

  1. Practical validity on realistic demonstrations.

We added a speech-driven learning experiment on the deployed face (§5.3, Appendix C.8). Students train on 99 CREMA-D/RAVDESS speakers and are tested on 10 unseen speakers. Through the measured actuator model, PTF has 4.3–5.5% higher held-out error than RAW at all four widths (|t| ≥ 19), while CBD recovers 43–50% of that gap (Table 2). We also estimate the projection-order discrepancy with teachers trained on disjoint groups of speakers: it is three to four times larger at jaw limits than in the interior, with the same pattern in a check using CREMA-D's repeated sentences (Table 34, Appendix C.8).

We then measured the face again after re-mounting the jaw linkage. The refitted model predicts mouth shapes for 600 fresh commands with R² = 0.96, and the learning ranking repeats at every width (Tables 36–37). Finally, we played 12 held-out utterances twice on the robot. PTF's mean peak opening above neutral is 12% below RAW's, lower in 10 of 12 utterances, and below CBD's in all 12 (Table 38). The effect is visible in the executed mouth motion.

  1. CBD's cost and choosing among the methods.

We have made the cost and required model access explicit in §7. In the width-32 CPU trajectory experiment, CBD costs 8.63 s per seed versus RAW's 1.83 s when including one deployment batch. The extra 6.8 s is for the teacher and label construction; both methods deploy the same student architecture and inverse (Table 5). On the face, the teacher and label projection take 5 s and 60 s on one GPU (Appendix C.8).

The results help distinguish the choices. In the rank-limited linear setting, Theorem 2 favours PTF. On the trajectory plant and face, RAW beats the other two output-label orders at every tested width. CBD wins six decided cells of the curved benchmark, all at the highest curvature or narrowest width, and improves on PTF pooled within each tested curvature (§5.1).

A command-emitting student (EMBED) makes a different tradeoff: it uses a differentiable plant during training and avoids the deployment inverse. Pretraining it on the inverse's commands beats the compute-matched multi-start and curriculum controls on both simulated plants, and beats RAW at widths 8 and 16 on the trajectory plant (§5.2, Tables 1, 22, 24). This is a useful product of the inverse even where its projected outputs are less useful as labels.

We added the playback in the Nov 13 revision to check whether the learning differences carry over to the robot. On our face with five mouth actuators, we replayed 12 held-out utterances twice per method. PTF's peak opening above neutral was 12% smaller on average than RAW's, and smaller than CBD's in all 12 (Appendix C.9, Table 38). The learning comparison kept the same ordering with a refitted model of the face (Table 37). CBD's teacher and label projection add offline work; deployment uses the same student architecture and inverse as RAW (§7).

Is there a particular part of the hardware comparison you would like us to clarify?

Before the discussion closes, could you let us know if any concerns remain after our response and the revised experiments? We would be glad to clarify them.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank the reviewers for recognizing the theoretical insights and controlled experiments, and for pointing out where the practical contribution was difficult to see. We have revised the paper around that concern and added real-speech experiments and measured robot playback.

Our humanoid robot has been speaking with passers-by in a public outdoor setting for over a month. Its face was designed to be affordable: only five actuators shape the mouth, yet we want it to convey the motion of human speech. This is the practical problem behind the paper: what should a learner fit when the demonstrations ask for movements its hardware cannot make?

  1. Real speech and physical execution.

The new §5.3 compares the training orders using 99 training speakers and 10 unseen test speakers. On the measured face model, PTF has 4.3–5.5% higher error than RAW at every tested width, and CBD recovers 43–50% of that gap (Table 2). The ordering repeats after re-measuring the face (Table 37). Across 12 utterances played twice on the robot, PTF's mean peak opening above neutral is 12% below RAW's and below CBD's in every utterance (Table 38).

  1. What changing the order contributes.

The revision separates the benefit of changing output labels from that of command initialization. Figure 3 brings the projection-order results together; Figure 6 shows what the policies execute, including a case where pretraining loses to cold training. Appendix I quantifies the oracle-CBD gap and tests solver and fitting sensitivity. These additions make the connection between the theory, finite-student learning and behaviour easier to follow.

  1. A clearer account of the contribution.

The Introduction and Experiments now follow the capacity effect, the projection-order bias and the complementary command-initialization result. §6 clarifies the relation to recent projection layers, and §7 gives practical guidance. Substantive additions are marked in blue; all submitted results are retained.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.