- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies differentiable optimization on nonlinear optimal control problems when the inner iterative solver operates in the pre-convergence (transient) regime. In this regime, implicit differentiation (IFT) is biased because the fixed-point condition does not hold, while truncated unrolling misses the remaining gradient tail beyond the truncation horizon. The authors propose Spectral Extrapolated Phantom Gradient (SE-PG). The method extends the transient state by a -step damped phantom rollout and approximates the infinite-horizon gradient tail via linear extrapolation utilizing geometric contraction. Furthermore, the authors present Hybrid SE-PG, which adaptively blends SE-PG with standard IFT using a sample-wise convergence mask and a batch-wise convergence fraction. The approach is evaluated on control tasks (Cartpole, Pendulum) and robot manipulation tasks (DiffTORI).
Critical strengths
-
The problem is real and practically relevant. In differentiable MPC / trajectory optimization and DEQ-style models, inner solvers are routinely truncated under compute budgets or during early training when is poor, and both IFT and truncated unrolling are biased in this regime. A cheap correction for the truncated tail is a reasonable thing to want.
-
The core idea is lightweight in spirit: estimating a local contraction factor from two consecutive gradient increments (a power-iteration-like heuristic) and approximating the remaining infinite-horizon gradient tail via a simple geometric series (Eq. 11) does not require an explicit Jacobian inverse.
Critical weaknesses
Beyond presentation issues, the paper has internal inconsistencies between the method description, the algorithm, and the appendix derivations; the theory is close to tautological; the central gradient-fidelity experiment contains an unexplained failure mode; and the problem and experimental setup are essentially absent, making the work nearly impossible to reproduce.
W1. The core gradient quantity is undefined in the main text, and both possible readings conflict with the paper's claims
- is the core gradient variable heavily used in Sec. 4 and Algorithm 1, but is never formally defined in the main text. Only in Appendix B (Eq. 21) is it defined as , i.e., a full forward-mode Jacobian. Yet Algorithm 1 uses plus the tail term directly to form the gradient for parameter update. Either reading leads to a contradiction:
- If really is the Jacobian, propagating it costs JVPs/VJPs per phantom step, compared to VJP per step for standard UPG/TBPTT. This contradicts the repeated claim that SE-PG "preserves the computational efficiency of truncated differentiation" (Sec. 4.4, Line 321).
- If instead is the reverse-mode loss gradient , then: (a) calling backward() on gives only , so obtaining and for Eq. (10) requires running separate backward passes from steps and ; and (b) because is still moving, the outer gradient changes with , which breaks the constant-operator recursion indicated by Eq. (9).
W2. Algorithm 1 is internally inconsistent and not executable as written
- Algorithm 1 Line 8 instructs to "Compute at anchor to define operator and ". But these quantities are never used in Lines 9–13, and computing them directly contradicts Sec. 4.4 (Line 321) and Appendix B (Line 701), which state that SE-PG does not construct . Moreover, forming the full Jacobians and in Line 8 already incurs the heavy Jacobian construction cost that truncated differentiation is meant to avoid.
- Conversely, the quantities that Line 9 actually needs (, , ) are never computed anywhere in the algorithm.
- Line 5 computes unconditionally for every sample before the check in Line 6. For unconverged samples (), this expensive IFT solve is computed only to be multiplied by zero in Line 11.
W3. The batch-coupled weight is not justified and depends on the batch size
- only affects samples that already pass the convergence test (), for which IFT is already valid by the paper's own argument. Why should sample 's gradient be diluted by because other independent samples in the same mini-batch have not converged? If the concern is that is a noisy indicator, the natural fix is a per-sample soft weight based on sample 's own residual, not a batch statistic.
- The statistical properties of —in particular the conditional expectation —depend on the batch size , so the batch size directly affects the resulting gradient estimator. Yet the paper provides neither a theoretical analysis of this dependence nor even reports the batch size in the experiments.
- The ablation in Appendix G (Fig. 8d) sweeps in but omits (i.e., ), the uncoupled baseline. Without , the experiments cannot establish that introducing is effective at all.
W4. Theorem 4.1 is essentially tautological, and its premise is violated in the paper's own experiments
- Theorem 4.1 replaces the matrix recursion by a scalar AR(1) model (Eq. 17). Any sequence satisfies this identity if is defined as the residual. The theorem does not bound (the accuracy of a one-step norm ratio with ) nor (which absorbs both the non-normal, multi-mode structure of and the infinite-horizon linearization drift), so it does not tell the reader when SE-PG is accurate. The "Interpretation" paragraph only restates the statement.
- The constants hide the dependence on : since has and is clipped at in Eq. (10), and (these are not even shown in the proof in Appendix C). Fig. 6 (Appendix D) shows the measured repeatedly reaching , where both constants blow up and the geometric series does not converge, so the theorem's premise is violated in the authors' own experiments.
W5. Fig. 3 reveals an unexplained failure mode of SE-PG, and the Hybrid variant's good behavior appears to be IFT in disguise
- In Fig. 3, from roughly epoch 40 to 150 the cosine similarity of pure SE-PG (and UPG) with the reference gradient is strongly negative (about to ), while
mask_alphais already at 90–100%. That is, SE-PG produces gradients pointing against the reference precisely when the solver is nearly converged after steps. This is not discussed. Fig. 2(a) stops at epoch 40, just before this regime, where a drop to about is already visible. - In the same regime implies , so Hybrid SE-PG reduces to pure IFT, and comparing IFT against an IFT reference trivially yields cosine . The paper's own evidence therefore does not show that the SE-PG component contributes positively beyond the first ~30 epochs.
W6. Problem setup, forward solver, and experimental details are essentially absent
Essential setup details are missing from both the main text and appendices, and no code is provided:
- The bilevel problem and its dimensions () are never written down: the outer loss , the exact forms of and for Cartpole and Pendulum, and expert data generation are all omitted. For standard Pendulum () and Cartpole () with , the trajectory dimension is only –, making it unclear why Cartpole training in Fig. 4(a) takes ~16,000 s on an i9 + RTX 4060.
- The forward solver used in the main control experiments (Sec. 5.2) is never specified.
- Critical hyperparameters that directly govern the hybrid switching behavior—most notably the batch size and convergence tolerance in Eqs. (12)–(14)—are omitted, as are the optimizer, learning rate, in Eq. (10), etc.
W7. Presentation and notation
- Notation collisions: (planning horizon in Eq. 1 vs. solver truncation step in Sec. 4), (state dimension in Eq. 2 vs. batch size in Sec. 4.3), and (iteration index in Eq. 2 vs. phantom horizon in Sec. 4) are all overloaded across sections.
- Sec. 3 spends a page on textbook IFT yet never writes down UPG (Geng et al., 2021), the method SE-PG directly builds on.
- The term "tail" is used in Sec. 4.1 before it is defined, and Sec. 4.2 presents and Eq. (9) as part of the method without clarifying until the end of Sec. 4.4 (Line 321) that is only an analysis device and is never constructed.
- In Appendix B, Eqs. (22)–(23) omit the evaluation point of the partial derivatives, and while Eq. (26) uses "" when freezing the Jacobian at the anchor, Eqs. (28)–(30) and Eq. (9) in the main text silently revert to "".
Questions for authors
- Is in your implementation the Jacobian (Appendix B, Eq. 21) or the loss gradient ? In either case, how exactly are , , obtained, and at what cost relative to standard UPG?
- In Algorithm 1, why does Line 8 compute , , , and when Sec. 4.4 states that is never constructed and Line 9 does not use them, and why is computed unconditionally in Line 5 before the check?
- What assumption in Appendix B justifies freezing the Jacobians as constants (Eqs. 24–30) when the solver is in the pre-convergence regime and is still moving?
- In Theorem 4.1, under what conditions on the solver and trajectory are (with ) and the infinite series guaranteed to be bounded, especially when reaches as in Fig. 6?
- Why does
cos_SEPGbecome strongly negative after epoch 40 in Fig. 3 whilemask_alphais at 90–100%? In that regime, is the Hybrid curve anything other than pure IFT? - Why should a converged sample's weight depend on other samples through ? Please report () in the ablation of Fig. 8d.
- Could you provide the key missing experimental setups for Sec. 5, such as the problem and parameter dimensions, the inner/outer objective formulations, and the forward solver , together with the batch size and tolerance used?