- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes BReF, a residual correction module attached to a frozen multimodal trajectory predictor. The module uses a Neural ODE to update candidate trajectories along an artificially defined correction time, and controls the correction behavior through drift-magnitude regularization, gating, and zero initialization. The paper also proposes a “force-open gating” diagnostic that analyzes the dependence of different correction modules on gating protection by comparing prediction errors before and after removing gating. The authors report prediction accuracy, cross-backbone adaptation, variations in solvers and integration steps, and several distribution-shift experiments on nuScenes.
Critical strengths
-
It highlights internal behavioral differences that conventional prediction metrics may obscure. The paper shows that correction modules with similar normal prediction errors can exhibit markedly different performance degradation after removing gating. This provides an instructive perspective for analyzing the behavior of correction modules.
-
The experimental analysis covers multiple dimensions. The authors examine regularization weights, integration steps, solvers, input noise, geographic and behavioral splits, and report three training seeds for BReF. These experiments help understand the stability of the reported phenomena.
-
It has a degree of engineering practicality. Freezing an existing predictor and training only a lightweight correction module allows existing models to be reused. Adaptation results on multiple frozen predictors also provide some empirical support for the method’s portability.
Critical weaknesses
Weaknesses
-
The methodological contribution is relatively incremental. BReF mainly combines existing ideas such as residual correction, frozen backbones, gating, zero initialization, and Neural ODEs. The paper has not yet clearly established a substantive methodological advance of this combination over existing trajectory correction methods. Although the diagnostic of removing gating is somewhat illuminating, whether it can constitute a sufficiently independent and important contribution still requires more thorough argumentation.
-
There is a lack of cross-dataset validation, and the generality of the conclusions remains unclear. The experiments are concentrated on nuScenes. Adapting to multiple backbone models can support cross-architecture portability, but it cannot replace cross-dataset validation. Therefore, it is currently difficult to determine whether the observed gains and robustness depend on a specific data distribution, scene characteristics, or evaluation protocol, and the evidence is insufficient to support broader generalization claims.
-
The experiments have not sufficiently demonstrated the necessity of the overall design. Existing ablations show performance changes under different configurations, but they do not sufficiently rule out the possibility that simpler residual correction and regularization methods could achieve similar results. Thus, the current evidence shows more that the approach is effective under the tested settings, but it is not yet enough to prove that its complex design yields irreplaceable benefits or reveals a new mechanism of general significance.
-
There remains a gap between the scope supported by the experiments and the paper’s claims. The paper emphasizes robustness, stability, and deployment value, but the evidence mainly comes from open-loop prediction errors, gating-removal diagnostics, and artificial perturbation tests. These results cannot yet sufficiently demonstrate advantages under real distribution shifts, recursive use, or downstream planning. The relevant conclusions require more direct experimental support or a clearer limitation of their scope of applicability.
Questions for authors
What is the core novelty? Compared with existing residual learning and trajectory correction methods, what substantive methodological advance does BReF offer? Beyond combining existing components, what unique conclusion does the paper aim to establish?
How is the necessity of the current design validated? Can the current experiments demonstrate that the reported benefits depend on the overall design of BReF, rather than on simpler residual correction and regularization methods? What evidence could distinguish these explanations?
Do the conclusions hold across datasets? The current evidence is concentrated on nuScenes and is insufficient to determine whether these findings are general. Can the authors validate the core conclusions on other motion prediction datasets (e.g., Navsim, nuplan, bench2drive, Argoverse 2, Waymo Open Motion Dataset) with different scene and interaction characteristics? Or, could you provide strong enough justification that the experiments on nuScenes is sufficient?
Do the experiments sufficiently support the paper’s claims? To what extent can the current open-loop prediction results and gating diagnostics support the paper’s claims about robustness, stability, and deployment value? Can the authors provide validation corresponding to these claims, or explicitly narrow the scope of the conclusions?
4 comments
Response to Reviewer uUZx (2/3): is the design necessary, or would simpler residual correction do? (W3, Q2)
This is the right question. The paper's ODE-versus-MLP comparison was confounded: only the ODE had a magnitude penalty, because the smoothness term acts on the ODE's drift log, which an MLP does not have. We now test the simpler alternative directly. We add an explicit residual-energy penalty to an MLP refiner, and sweep it and the ODE's smoothness penalty over the same four weights. Each refiner is compared with its own zero-residual control (Δ minADE₅ in mm, 95% CI). "gate-forced" is the error added when the gate is forced open.
| w | ODE + smoothness: gain | MLP + energy: gain | ODE − MLP, minADE₅ | ODE − MLP, minFDE₅ | gate-forced, ODE / MLP |
|---|---|---|---|---|---|
| 0.003 † | −0.92 [−1.77, −0.09] | −0.25 [−1.18, +0.68] | −0.70 [−1.54, +0.16] | −2.09 [−3.91, −0.30] | +8.4 / +10.1 |
| 0.01 (BReF) | −0.92 [−1.65, −0.20] | −0.48 [−1.44, +0.47] | −0.44 [−1.35, +0.44] | −2.04 [−3.75, −0.36] | +5.6 / +2.3 |
| 0.03 † | −0.84 [−1.43, −0.25] | −0.33 [−0.89, +0.23] | −0.53 [−1.13, +0.06] | −0.14 [−1.39, +1.11] | +1.1 / +0.7 |
| 0.1 † | −1.12 [−1.62, −0.67] | −0.27 [−0.60, +0.04] | −0.87 [−1.32, −0.45] | −0.27 [−0.96, +0.39] | +0.6 / −0.1 |
† Trained on a second cluster with a different GPU model. Its backbone outputs differ from the first cluster's by only +0.02 mm [−0.00, +0.05].
What this rules out and what it does not:
- "Simpler regularisation achieves the same result" is not supported here. An energy-penalised MLP becomes gate-safe, but its gain has a clustered CI including 0 at all four weights. Every penalised ODE has a significant gain: four weights, and six runs including two more seeds at w = 0.01 (−1.04, −1.14).
- Head to head, the ODE is ahead at every weight. Its paired difference has a CI excluding 0 on at least one of minADE₅ / minFDE₅ at three of four weights, and it is never significantly worse on either.
- Against an unpenalised MLP, BReF is not more accurate (paired CIs include 0). Its advantage there is gate reliance alone: 4–7 mm versus 308–439 mm.
We will state the limits with the result. Each added weight is a single run, while the seed spread at w = 0.01 is 0.11 mm. The two penalties are different functionals, so equal weights are not equal regularisation; matching on realised correction size gives the same picture, as shown in our reply to th2h. The margin is small, 0.4–0.9 mm minADE₅. So the evidence supports the ODE parameterisation helps, at this size, on this backbone. It does not show that BReF yields irreplaceable benefits, and we will not claim that.
Response to Reviewer uUZx (3/3): cross-dataset evidence and the scope of our claims (W2, W4, Q3, Q4)
Cross-dataset (W2, Q3): conceded. All of our experiments use nuScenes. Testing many backbones shows portability across architectures; it is not cross-dataset evidence, and we should not have let the two blur together. We could not port the full chain (backbone pre-training, frozen backbone, refiners, and the paired per-sample evaluation) to Argoverse 2 or WOMD within the rebuttal window, and we will not present a partial run as validation. We therefore make two changes:
- every conclusion in the paper is now explicitly scoped to nuScenes;
- cross-dataset validation is listed as required future work, not implied coverage.
We do not claim that nuScenes alone suffices to establish generality.
Scope of the robustness, stability and deployment claims (W4, Q4): we narrow all three to what was measured.
- Robustness / OOD. Our "OOD" test added Gaussian noise to the backbone's output. That is a perturbation of the prediction, not a distribution shift of the input scene, and we relabel it as such. The geographic and manoeuvre splits are genuine distribution shifts. We will make only the narrow claim those splits support.
- Stability under recursive use. We replaced the extrapolated decay with a measurement of repeated application on the full val split.
- Applied repeatedly to the backbone's prediction (re-anchored), BReF converges to a fixed point within 2–3 steps. The fixed point lies 8 mm from the backbone, and minADE₅ stays within 0.05 mm of the deployed value.
- Applied to its own output (feedback), every refiner drifts: the step ratio exceeds 1 at every iteration, and accuracy degrades after the first one or two steps. BReF drifts 6.9 cm after 8 steps, against 22.0 cm for the MLP, but it does drift.
- We will claim the first behaviour and disclaim the second. This is open-loop self-iteration, not closed-loop control, and we will not call it that.
- Closed-loop and quantisation. We ran no planner-in-the-loop experiment and no low-bit quantisation experiment. Both motivations are removed from the abstract and introduction.
- Deployment value. We claim only what we measured. BReF trains 575,747 parameters (7.7% of the backbone's count) in 0.43× the wall time of joint training, and leaves every backbone weight untouched. It gives a small (~1 mm minADE₅) gain that is significant in each of three independent runs, and it transfers to a second backbone (LAformer: minFDE₅ −2.6 mm, p = 0.0014). We make no claim about downstream planning.
The diagnostic you singled out as the paper's strength is what we keep and sharpen. It shows that refiners with near-identical displacement error can differ by more than an order of magnitude in how much of their accuracy depends on the gate. We will present it as a behavioural probe, reported beside the correction size and a zero-residual control, so that it cannot be satisfied by a residual that is simply zero.
Please help to join the discussion
Dear Reviewer uUZx,
The discussion deadline is approaching. Please help to join the discussion and offer the reviewer's opportunity to further respond to your concerns.
Thanks!
AC
Response to Reviewer uUZx (1/3): what the contribution is (W1, Q1)
We thank the reviewer for a careful review. We agree with most of it, and we have narrowed the paper's claims accordingly. The other two reviewers raised overlapping points, so we ran new experiments; we summarise here what they establish.
About the new numbers. The original training machine no longer exists, and our archive kept only aggregate tables. We therefore rebuilt the full chain (LAformer stage-1 → frozen backbone → refiners) and ran every comparison below on it, with N = 9041 nuScenes val samples. 95% CIs come from a bootstrap clustered on keyframes, and we also report Wilcoxon signed-rank tests. The questions here are relative comparisons on identical samples, which the reproduction can answer; it does not certify the paper's absolute numbers.
What we are no longer claiming. Re-examination confirmed several overclaims, and we withdraw them:
What remains, and what we now claim. We agree the components (residual correction, frozen backbone, gate, zero-init, Neural ODE) are individually known. We do not claim a new component. We claim two findings, both now supported by paired, per-sample evidence:
Joint training. Matched joint training turned out to be backbone fine-tuning. In all three joint runs, the refiner's gate ended at exactly zero. Fine-tuning the backbone alone, with no refiner, matched the joint result to within 0.8 mm minADE₅. BReF's role is therefore narrower and, we think, clearer: a cheap correction for backbones that cannot be retrained. It trains 7.7% of the backbone's parameter count, never modifies a backbone weight, and gives a gain of about 1 mm minADE₅ that is significant in each of three independent runs (−0.92 / −1.04 / −1.14 mm, each CI excluding 0).
We will rewrite the abstract and introduction around these two findings and this scope.