-
The principal bounds in Section 5 are not established by the provided proofs, and parts of the analysis do not match the implemented optimizer.
In Appendix D.1, Eq. (53) bounds the squared norm of a sum by the sum of squared norms without cross terms or additional factors. This inequality is false in general: taking scalar values a=b=1 and c=0 gives 4 on the left and 2 on the right. Equation (56) similarly bounds the squared norm of accumulated gradient increments by the sum of their squared norms without the required factor or an orthogonality assumption. Consequently, Theorem 5.3 and its dependent results do not follow from the stated assumptions.
A separate problem occurs in Eqs. (49)–(50), where a second-order approximation is promoted to a strict upper bound using only the Hessian at the GD endpoint. Smooth gradients and bounded iterates do not justify this step. For example, f(x)=(1−cos x)^2 has a globally Lipschitz gradient and satisfies f′(0)=f″(0)=0, but f(δ)>0 for sufficiently small nonzero δ. Thus, the endpoint Hessian alone cannot provide the claimed bound. The proof also treats the finite-K GD endpoint as stationary without establishing this condition. Appendix D.13 acknowledges that the experimental endpoint is not stationary and that the corresponding quadratic estimate frequently fails to upper-bound the measured gap.
There are also substantive inconsistencies between the analyzed and implemented quantities. Equation (2) corresponds to descent on L_OW−(η_RL/η)V, whereas Definition 5.5, Eq. (63), and Appendix D.10 use L_OW+η_RL V. Moreover, Appendix D replaces the implemented six-dimensional critic state with a scalar semantic-drift statistic involving the next environment’s semantic distribution and uses them interchangeably without establishing the necessary relationship. These issues undermine the claimed loss-gap and gradient-flow explanation of the actual optimizer.
2.The remaining valid theory and empirical diagnostics do not establish the claimed connection between temporal value learning and improved detection at the reported operating point.
Theorem 4.2 concerns soft FPR at a fixed threshold, whereas the principal experimental metric is FPR@95 with a threshold calibrated to ID acceptance. These are materially different quantities. A common downward shift of all class logits leaves softmax probabilities and classification cross-entropy unchanged while increasing every energy score by the same constant. This lowers fixed-threshold soft FPR, yet leaves FPR@95 and AUROC unchanged after threshold recalibration. Therefore, even a simultaneous decrease in the theorem’s soft-FPR risk and preservation of classification risk does not establish improved detection at controlled ID acceptance.
In addition, the finite-window identities hold for arbitrary additive corrections and do not use the TD or Bellman structure to establish the required alignment. Table 7 reports critic sensitivity and mean semantic-OOD energy changes, but these do not verify the complete alignment, induced-drift, and curvature conditions in Theorem 4.2. The penalty sweep in Appendix D.11 is useful, but it does not isolate temporal bootstrapping from an adaptive instantaneous regularizer. A matched myopic critic or another temporal-control experiment is needed to substantiate the proposed explanation of why RL helps.
TD Ablation and Pseudo-OOD Sensitivity
We thank the reviewer for the careful review.
W1.1&Q1. Consistency of the update rule and objective. The implemented update corresponds to the locally frozen objective LOW−ηηRLVϕ(s) (Section 3). The effective gradient flow is now reported for the objective the update actually descends (Appendix D.10). Theorem 4.2, the main result, uses the update rule directly and is unaffected.
W1.2&Q2. Contribution of TD learning. The TD ablation confirms the role of the critic. Without bootstrapping (discount 0), RL loses its advantage on all three gradual streams (by 2.4, 1.5, and 1.0 points, all significant), and a larger step does not restore it. Bootstrapping from a random earlier state instead of the next one changes nothing, so the gain comes from the bootstrapped value estimation rather than transition order (Section 6.5).
W1.3&Q3: Sensitivity to pseudo-OOD selection. We varied the pseudo-OOD percentile and flipped 10% or 20% of the pseudo-labels for both optimizers (Appendix D.17). RL's advantage in FPR@95 at percentiles 0.90, 0.95, and 0.99 is 1.8, 2.0, and 1.1 points on Core50 and 1.8, 1.9, and 1.8 points on iCubWorld. Given 10% and 20% of the labels flipped, it is 4.0 and 3.8 points on iCubWorld, and almost zero on Core50 and OpenLORIS. The gain holds across thresholds on Core50 and iCubWorld, and on iCubWorld it grows under label noise. Core50 shows that pseudo-label quality is a lever for further gains.
Modest gains and DomainNet. Theorem 4.2 explains where gains appear. On DomainNet the measured value term is 10x smaller than on Core50, which accounts for the small difference we observe. We also extended the OOD-weight sweep: RL stays lower at matched weight on iCubWorld throughout and on Core50 up to a weight of 6, and we now state the gains at matched weight.
Linear head. The frozen-feature setting is where Proposition 4.3 makes the constants explicit. The fine-tuning results confirm this boundary and point to step-size control as a route to larger models (Section 6.4).