- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work studies prediction-space extrapolation from a single globally trained model to a weighted routed-expert setup. The routed experts are initialized from the same global checkpoint and fine-tuned on separate subsets. A scalar extrapolation coefficient is selected on a holdout set while the expert endpoints remain fixed. Under asymptotic linear expansion, the theory yields the long-run covariance form of the optimal coefficient. Within a two-state model with persistent routing mistakes, the optimal coefficient is shown to exceed one, and its variation with error persistence is characterized. Experiments include controlled simulations, four real datasets, budget-matched directional baselines, hard-routing baselines, and architecture ablations.
Critical strengths
-
The paper presents a useful formulation of route-conditioned extrapolation. Learned routes produce better prediction directions than the tested budget-matched random, temporal, and continued-training alternatives across four datasets.
-
The theory identifies how routing-error persistence affects correction strength beyond the marginal error rate. The explicit coefficient and controlled simulations provide an informative characterization within the stated two-state model.
-
The experimental design includes frozen-endpoint coefficient sweeps, matched directional controls, and hard-routing comparisons. Evaluation and certification use independent subjects or patients, appropriately accounting for dependence within units.
Critical weaknesses
-
The additional value of extrapolation over hard routing remains unclear. Table 5 reports nearly identical performance, limiting the demonstrated benefit of extrapolation beyond the learned routing and expert specialization.
-
The theory addresses composition correction, whereas route-specific fine-tuning dominates the real-data displacement. Although acknowledged, this gap limits the theory’s ability to explain the main empirical gains or guide coefficient selection.
-
The pilot estimator in Appendix B.3 appears to omit the normalization connecting the raw route score to the endpoint direction. This requires clarification because it affects the reported agreement between pilot and oracle correction.
-
Some empirical gains have limited statistical support. WISDM’s improvement over weighted averaging is small relative to its standard error, while PAMAP2 contains only nine independent subjects despite repeated data splits.
Questions for authors
See weaknesses.
2 comments
Response to reviewer KaRR – Part 2
On the ethics flag: all four datasets are public, de-identified, and licensed under CC BY 4.0. They are HAR-Transitions (UCI, Smartphone-Based Recognition of Human Activities and Postural Transitions), WISDM (the 2019 UCI Smartphone and Smartwatch Activity and Biometrics release), PAMAP2 (UCI, PAMAP2 Physical Activity Monitoring), and PTB-XL (PhysioNet, version 1.0.3). No new data were collected, and no attempt was made to re-identify participants. The revision states these licenses in the ethics statement and cites each dataset.
Please let us know whether these changes address your concerns.
Response to reviewer KaRR – Part 1
Thank you for the detailed review. The table summarizes our answers, and the text after it gives the derivations.
The pilot must use the actual endpoint direction. Appendix A.2 correctly gives ax⋆=hxcFx/(2Hx) and ax0=hxc/2, hence αx⋆=ax⋆/ax0=Fx/Hx. Appendix B.3's raw covariance-to-variance expression instead estimates a⋆ unless the direction scaling is included. The revision states this scaling in Appendix B.3. Recomputed with the scaled direction, the neural pilot estimates are 1.54 ± 0.04, 1.57 ± 0.05, 1.43 ± 0.07, and 1.48 ± 0.03 on HAR-Transitions, WISDM, PAMAP2, and PTB-XL, with block-bootstrap standard errors, against the reported 1.58, 1.61, 1.46, and 1.52. Each falls by 0.03–0.04, and each still exceeds one by more than six standard errors, so the pilot still indicates extrapolation beyond the aggregate.
For the precise statement, let centered stationary scores Ut,Vt have absolutely summable covariance sequences, and let the subtracted correction have influence a0Vt with a0=0. Then
\frac{\operatorname{LRCov}(U,a^0V)}{\operatorname{LRV}(a^0V)} =\frac{\operatorname{LRCov}(U,V)}{a^0\operatorname{LRV}(V)} =\frac{a^\star}{a^0}.$$ The leading squared-error coefficient is $\operatorname{LRV}(U)-2\alpha a^0\operatorname{LRCov}(U,V)+\alpha^2(a^0)^2\operatorname{LRV}(V)$. Differentiation proves the result for either sign of $a^0$, and the common Brier factor does not change the minimizer. If $a^0=0$, the direction vanishes and the first-order optimum is unidentified. Near-zero scaling requires a declared stability rule, not unguarded division. The finite-cell formula recovers $F_x/H_x$. A neural pilot must instead use its actual scaled vector direction, or a justified cellwise assembly. The recomputed pilot uses the scaled direction. The Brier normalization also needs to be explicit. For frozen displacement $\Delta=\hat f_{\mathrm{eq}}-\hat f_0$, $$R(\alpha)=A-2\alpha B+\alpha^2C,\qquad B=\tfrac12\mathbb E\langle e_Y-\hat f_0,\Delta\rangle,\qquad C=\tfrac12\mathbb E\|\Delta\|^2$$ for normalized, unprojected Brier risk, with $A=R(0)$. This follows by expanding $\tfrac12\|e_Y-\hat f_0-\alpha\Delta\|^2$ and agrees with Appendix A.5. If $C=0$, the endpoints agree almost surely. Simplex projection cannot increase distance to the one-hot target, so the same quadratic is an upper bound after projection; projected risk need not remain globally quadratic when clipping is active. Finally, WISDM's endpoint gain is small relative to its reported uncertainty, and PAMAP2 contains nine unique subjects. The unit-aware certification analysis correctly treats the small cohorts as information-limited. The supported conclusion is a useful tested direction with dataset-dependent evidence, not a reliable population improvement established by repeated splits.