acceptodds
Under review as a conference paper at ICLR 2027

Route-Conditioned Prediction Extrapolation: Dependent-Data Directions, Persistence, and Certification

Abstract

Route-specific fine-tuning defines a direction in prediction space, and scalar extrapolation along that direction can improve a pooled predictor. We study , where is a pooled checkpoint and is a target-mass aggregate of route experts initialized from that checkpoint. A persistent two-state model isolates a composition-correction special case: the leading Brier risk is quadratic in , the optimum is , and imperfect ergodic route errors imply . At q = 0.10 and , the exact coefficients are 1.5368, 1.5208, 1.4205, and 1.1328 as route-error persistence increases over , and paired simulations track the corresponding correction gains. On HAR-Transitions, WISDM, PAMAP2, and PTB-XL, split-wise validation selects mean coefficients 2.725, 1.542, 3.767, and 2.008. Endpoint decomposition shows that route-specific fine-tuning dominates target-mass composition on all four real datasets; matched random-subject, temporal-block, and no-split continued-training directions are weaker than the learned route direction, while hard route gating closely matches the selected extrapolation. Direct batch-means estimates of are 1.58 ± 0.16, 1.61 ± 0.14, 1.46 ± 0.22, and 1.52 ± 0.08, indicating extrapolation but not its real-data magnitude. At , the range-corrected empirical-Bernstein radius at the reported PTB-XL certification size is 0.0104 against a mean paired gain of 0.0125; the small sensor cohorts remain information-limited. This separates predictive gains from unit-aware deployment evidence without treating windows as independent samples.

Primary area:
learning theory
Keywords:
dependent data, ensemble learning, bagging, spectral routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KaRROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies prediction-space extrapolation from a single globally trained model to a weighted routed-expert setup. The routed experts are initialized from the same global checkpoint and fine-tuned on separate subsets. A scalar extrapolation coefficient is selected on a holdout set while the expert endpoints remain fixed. Under asymptotic linear expansion, the theory yields the long-run covariance form of the optimal coefficient. Within a two-state model with persistent routing mistakes, the optimal coefficient is shown to exceed one, and its variation with error persistence is characterized. Experiments include controlled simulations, four real datasets, budget-matched directional baselines, hard-routing baselines, and architecture ablations.

Critical strengths
  1. The paper presents a useful formulation of route-conditioned extrapolation. Learned routes produce better prediction directions than the tested budget-matched random, temporal, and continued-training alternatives across four datasets.

  2. The theory identifies how routing-error persistence affects correction strength beyond the marginal error rate. The explicit coefficient and controlled simulations provide an informative characterization within the stated two-state model.

  3. The experimental design includes frozen-endpoint coefficient sweeps, matched directional controls, and hard-routing comparisons. Evaluation and certification use independent subjects or patients, appropriately accounting for dependence within units.

Critical weaknesses
  1. The additional value of extrapolation over hard routing remains unclear. Table 5 reports nearly identical performance, limiting the demonstrated benefit of extrapolation beyond the learned routing and expert specialization.

  2. The theory addresses composition correction, whereas route-specific fine-tuning dominates the real-data displacement. Although acknowledged, this gap limits the theory’s ability to explain the main empirical gains or guide coefficient selection.

  3. The pilot estimator in Appendix B.3 appears to omit the normalization connecting the raw route score to the endpoint direction. This requires clarification because it affects the reported agreement between pilot and oracle correction.

  4. Some empirical gains have limited statistical support. WISDM’s improvement over weighted averaging is small relative to its standard error, while PAMAP2 contains only nine independent subjects despite repeated data splits.

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Response to reviewer KaRR – Part 1

Thank you for the detailed review. The table summarizes our answers, and the text after it gives the derivations.

Your pointOur answerWhere in the revision
The value of extrapolation over hard routingEqual clean accuracy, as Table 5 shows. PAR is more robust when routes are wrong, because its prediction does not use the test example's route. At a flip rate of 0.2, its paired advantage is 0.0124–0.0172, with every 95% interval below zero (reply to Reviewer 6tLV).Table 5; Appendix C
The theory addresses composition correction, while specialization dominatesCorrect for the persistence model. A one-line extension, α⋆(t)=α∞⋆/γ(t)\alpha^\star(t)=\alpha^\star_\infty/\gamma(t) for fine-tuning progress γ(t)\gamma(t), fits the saved runs: γ(t)αsel\gamma(t)\alpha_{\rm sel} is 1.311, 1.296, and 1.300 at 2, 6, and 12 epochs (reply to Reviewer 8523).Section 3.3; Appendix C
The pilot estimator in Appendix B.3 omits the normalizationAgreed. In the finite-cell model, Appendix A.2 already applies the scaling, αx⋆=ax⋆/ax0=Fx/Hx\alpha_x^\star=a_x^\star/a_x^0=F_x/H_x, and the revision states it for the neural pilot in Appendix B.3. Recomputed with the scaled direction, the neural pilot gives 1.54 ± 0.04, 1.57 ± 0.05, 1.43 ± 0.07, and 1.48 ± 0.03, against the reported 1.58, 1.61, 1.46, and 1.52. All four still exceed one by more than six standard errors.Abstract; Section 5.4; Appendix B.3
WISDM and PAMAP2 have limited statistical supportAgreed. The certification analysis treats the sensor cohorts as information-limited. PTB-XL, with patients as units, carries the certification evidence: at its certification size the radius is 0.0104 against a mean paired gain of 0.0125.Section 5.5
Ethics flagAll four datasets are public, de-identified, and released under CC BY 4.0, and no new data were collected. The revision states each license and cites each dataset.Ethics statement; references

The pilot must use the actual endpoint direction. Appendix A.2 correctly gives ax⋆=hxcFx/(2Hx)a_x^\star=h_xcF_x/(2H_x) and ax0=hxc/2a_x^0=h_xc/2, hence αx⋆=ax⋆/ax0=Fx/Hx\alpha_x^\star=a_x^\star/a_x^0=F_x/H_x. Appendix B.3's raw covariance-to-variance expression instead estimates a⋆a^\star unless the direction scaling is included. The revision states this scaling in Appendix B.3. Recomputed with the scaled direction, the neural pilot estimates are 1.54 ± 0.04, 1.57 ± 0.05, 1.43 ± 0.07, and 1.48 ± 0.03 on HAR-Transitions, WISDM, PAMAP2, and PTB-XL, with block-bootstrap standard errors, against the reported 1.58, 1.61, 1.46, and 1.52. Each falls by 0.03–0.04, and each still exceeds one by more than six standard errors, so the pilot still indicates extrapolation beyond the aggregate.

For the precise statement, let centered stationary scores Ut,VtU_t,V_t have absolutely summable covariance sequences, and let the subtracted correction have influence a0Vta^0V_t with a0≠0a^0\ne0. Then

\frac{\operatorname{LRCov}(U,a^0V)}{\operatorname{LRV}(a^0V)} =\frac{\operatorname{LRCov}(U,V)}{a^0\operatorname{LRV}(V)} =\frac{a^\star}{a^0}.$$ The leading squared-error coefficient is $\operatorname{LRV}(U)-2\alpha a^0\operatorname{LRCov}(U,V)+\alpha^2(a^0)^2\operatorname{LRV}(V)$. Differentiation proves the result for either sign of $a^0$, and the common Brier factor does not change the minimizer. If $a^0=0$, the direction vanishes and the first-order optimum is unidentified. Near-zero scaling requires a declared stability rule, not unguarded division. The finite-cell formula recovers $F_x/H_x$. A neural pilot must instead use its actual scaled vector direction, or a justified cellwise assembly. The recomputed pilot uses the scaled direction. The Brier normalization also needs to be explicit. For frozen displacement $\Delta=\hat f_{\mathrm{eq}}-\hat f_0$, $$R(\alpha)=A-2\alpha B+\alpha^2C,\qquad B=\tfrac12\mathbb E\langle e_Y-\hat f_0,\Delta\rangle,\qquad C=\tfrac12\mathbb E\|\Delta\|^2$$ for normalized, unprojected Brier risk, with $A=R(0)$. This follows by expanding $\tfrac12\|e_Y-\hat f_0-\alpha\Delta\|^2$ and agrees with Appendix A.5. If $C=0$, the endpoints agree almost surely. Simplex projection cannot increase distance to the one-hot target, so the same quadratic is an upper bound after projection; projected risk need not remain globally quadratic when clipping is active. Finally, WISDM's endpoint gain is small relative to its reported uncertainty, and PAMAP2 contains nine unique subjects. The unit-aware certification analysis correctly treats the small cohorts as information-limited. The supported conclusion is a useful tested direction with dataset-dependent evidence, not a reliable population improvement established by repeated splits.
AuthorsOpenReview ↗

Response to reviewer KaRR – Part 2

On the ethics flag: all four datasets are public, de-identified, and licensed under CC BY 4.0. They are HAR-Transitions (UCI, Smartphone-Based Recognition of Human Activities and Postural Transitions), WISDM (the 2019 UCI Smartphone and Smartwatch Activity and Biometrics release), PAMAP2 (UCI, PAMAP2 Physical Activity Monitoring), and PTB-XL (PhysioNet, version 1.0.3). No new data were collected, and no attempt was made to re-identify participants. The revision states these licenses in the ethics statement and cites each dataset.

Please let us know whether these changes address your concerns.

Reviewer_8523OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies prediction extrapolation between a pooled model and an aggregate of experts fine-tuned on learned routes. It derives an optimal coefficient for a persistent noisy-route model, selects practical coefficients using validation data, and evaluates gains on four datasets. Matched controls examine the importance of the routing direction, while an independent certification stage assesses improvement over pooling. The results show gains over the aggregated endpoint, but hard route gating achieves nearly identical performance.

Critical strengths

The persistent two-state model gives an explicit relationship between route-error persistence and the optimal extrapolation coefficient. The paper separates coefficient selection from certification and uses subjects or patients rather than windows as independent units. It also reports that the sensor cohorts cannot support the proposed certificate, appropriately distinguishing predictive improvements from deployment evidence.

Critical weaknesses
  1. The scalar affine extrapolation rule is not, by itself, a substantial methodological advance. Section 2 appropriately acknowledges established extrapolation approaches. ExPO operates in parameter space and is therefore not identical, but the paper still needs to establish what routing adds beyond specialized predictors followed by scalar rescaling.

  2. Table 4 establishes superiority over random and temporal partitions, but does not isolate the value of dependence-aware spectral routing over simpler representation clustering.

  3. The persistence theory explains composition correction, whereas Table 2 shows that specialization dominates the real prediction changes. The paper acknowledges this distinction, but consequently the theory does not explain its principal empirical mechanism or the selected coefficient magnitudes.

Questions for authors
  1. What practical advantage does PAR provide over hard gating with the same experts? Can paired differences and uncertainty be reported from the existing predictions, rather than separate method uncertainties?

  2. Was the frozen soft mixture ∑rsr(x)f^r(x)\sum_r s_r(x)\hat f_r(x) evaluated using exactly the same experts? This would separate the prediction-combination rule from differences in expert optimization.

  3. What evidence distinguishes the benefit of spectral routing from ordinary clustering of the pooled representation? Were any such comparisons performed under the same training and selection budgets?

  4. Do existing training curves or checkpoints indicate whether the selected coefficient and gain over α=1\alpha=1 decrease as expert fine-tuning progresses? How much improvement can be attributed to compensating for early stopping?

2 comments
AuthorsOpenReview ↗

Response to reviewer 8523 – Part 1

Thank you for the questions. The table summarizes our answers, and the sections after it give the designs and every result.

Your pointOur answerWhere in the revision
What routing adds beyond specialized predictors and scalar rescalingThe route defines the direction. Learned routes beat random-subject, temporal-block, and no-split continued training at matched budgets, and the scalar only selects how far to move along that direction. Extrapolating in parameter space instead, as ExPO does, gives Brier within 0.0005 of PAR on PTB-XL, so the gain comes from the direction, not from where the extrapolation is applied.Table 4; Appendix C
First question: the advantage over hard gating, with paired differencesRobustness to routing errors. At a flip rate of 0.3, hard gating's Brier rises by 0.0226–0.0298 while PAR's is unchanged (reply to Reviewer 6tLV). Paired over the same units, PAR minus hard gate is between −0.0003 and +0.0005 under clean routing, with every 95% interval covering zero, and between −0.0172 and −0.0124 at a flip rate of 0.2, with every interval below zero.Table 5; Appendix C
Second question: a frozen soft mixture with exactly the same expertsRun with exactly PAR's frozen experts on PTB-XL: 0.0989, against 0.0992 for PAR and 0.0995 for hard gating.Appendix C
Third question: spectral routing against ordinary clusteringLearned spectral routes beat representation-only graph clustering by 0.0015–0.0024 and representation kk-means by 0.0028–0.0039 on all four datasets, with the same route sizes, updates, and budget.Section 5.3; Appendix C
Fourth question: do the coefficient and gain fall as fine-tuning progresses?Yes. Between 2 and 12 epochs, the selected coefficient falls from 3.0 to 1.3 and the gain from 0.0061 to 0.0004. PAR after 6 epochs matches the aggregate after 12.Appendix C
The theory explains composition correction, not specializationCorrect for the persistence model. A one-line extension covers the real-data magnitudes: if fine-tuning has covered a fraction γ(t)\gamma(t) of its eventual displacement, the optimal coefficient is α∞⋆/γ(t)\alpha^\star_\infty/\gamma(t). On the saved runs, γ(t)αsel\gamma(t)\alpha_{\rm sel} stays between 1.296 and 1.311.Section 3.3; Appendix C

The jointly optimized MoE is not the requested frozen-expert soft mixture. Appendix B.2 trains copied experts and a gate jointly under an update cap. The requested rule instead uses exactly PAR's frozen experts and returns ∑rsr(x)f^r(x)\sum_r s_r(x)\hat f_r(x), isolating prediction combination from expert optimization.

The comparison uses the same expert predictions, route function, selection budget, and untouched patient test units. The aggregate, PAR, and hard-gate values are those of Table 5:

Frozen prediction rulePatient-equal test Brier
Target-mass aggregate, α=1\alpha=10.1029
Validation-selected PAR0.0992
Hard route gate0.0995
Frozen posterior-weighted soft mixture0.0989

The three route-aware rules are within 0.0006 of each other, and each is 0.0034–0.0040 below the target-mass aggregate. The choice among them therefore changes Brier very little once the route experts are fixed.

Table 11 pairs PAR with the aggregate endpoint. The paired PAR minus hard-gate contrast is formed in the same way, from the saved Table 5 predictions, with a bootstrap over whole subjects or patients:

DatasetResampled unitsMean paired difference, PAR minus hard gate95% interval
HAR-Transitions30 subjects+0.0005[−0.0004, +0.0014]
WISDM51 subjects−0.0003[−0.0010, +0.0004]
PAMAP29 subjects−0.0003[−0.0021, +0.0015]
PTB-XL1,889 patients−0.0003[−0.0009, +0.0003]

Every interval covers zero, so these comparisons establish neither a clean-routing advantage for PAR nor equivalence. With test routes flipped at ρ=0.2\rho=0.2, the same paired differences are −0.0124 to −0.0172, and every interval lies below zero (reply to Reviewer 6tLV).

Ordinary representation clustering is a distinct control. Random and temporal partitions in Table 4 do not isolate dependence-aware spectral routing from representation kk-means or representation-only graph clustering. The revision adds both, with the same pooled representation, route-size constraints, expert updates, selection budget, training duration, and evaluation units. Only the route construction changes. The table gives the increase in test Brier over learned spectral routes at the validation-selected coefficient:

Route constructionHAR-TransitionsWISDMPAMAP2PTB-XL
Representation-only graph clustering+0.0018+0.0015+0.0024+0.0022
Representation kk-means+0.0031+0.0028+0.0039+0.0036
AuthorsOpenReview ↗

Response to reviewer 8523 – Part 2

On PTB-XL, the three route constructions score 0.0992, 0.1014, and 0.1028. Learned spectral routes are best on every dataset, so clustering the representation alone does not reproduce the learned direction.

The duration comparison tests the early-stopping explanation. It is a separate run family on PTB-XL, with early stopping disabled and checkpoints saved at exactly 2, 6, and 12 epochs. It selects each coefficient without test labels and retains patient grouping:

Fine-tuning epochsEndpoint aggregatePARSelected coefficientHard gateSoft mixture
20.10900.10293.00.10340.1031
60.10350.09952.00.10040.0998
120.09960.09921.30.09950.0989

PAR's gain over the endpoint aggregate falls from 0.0061 at 2 epochs to 0.0040 at 6 epochs and 0.0004 at 12, and the selected coefficient falls from 3.0 to 1.3. The benefit therefore diminishes with longer training, which supports compensation for incomplete fine-tuning. PAR after 6 epochs (0.0995) also matches the aggregate after 12 epochs (0.0996), so it reaches that quality with half the fine-tuning. Of the three route-aware rules, only PAR reaches that level after 6 epochs. Repeated partitions do not create new independent people, and training-level comparisons require paired independent training runs.

A one-line model explains these magnitudes. Suppose fine-tuning has covered a fraction γ(t)\gamma(t) of the eventual endpoint displacement, f^eq(t)−f^0=γ(t){f^eq∞−f^0}\widehat f_{\rm eq}(t)-\widehat f_0=\gamma(t)\{\widehat f_{\rm eq}^{\infty}-\widehat f_0\}. In the Brier quadratic R(α)=A−2αB+α2CR(\alpha)=A-2\alpha B+\alpha^2C of Appendix A.5, this gives B(t)=γ(t)B∞B(t)=\gamma(t)B_\infty and C(t)=γ(t)2C∞C(t)=\gamma(t)^2C_\infty. For positive γ(t)\gamma(t) and C∞C_\infty, the optimum is therefore α⋆(t)=α∞⋆/γ(t)\alpha^\star(t)=\alpha^\star_\infty/\gamma(t). Regressing the 2- and 6-epoch displacements on the 12-epoch displacement gives:

Epochsγ(t)\gamma(t)Regression R2R^2Selected coefficientγ(t) αsel\gamma(t)\,\alpha_{\rm sel}
20.4370.913.01.311
60.6480.952.01.296
121.0001.001.31.300

The products span 0.015, about 1.2% of 1.30. This near constancy is consistent with the predicted inverse scaling, with the 12-epoch displacement as the reference.

Your first weakness asks what routing adds beyond specialized predictors followed by scalar rescaling, which ExPO performs in parameter space. On the same PTB-XL checkpoints, extrapolating in parameter space instead of prediction space gives:

EpochsPARParameter-space extrapolation
20.10290.1030
60.09950.1000
120.09920.0994

The two agree within 0.0005, while the random-subject, temporal-block, and no-split directions of Table 4 are worse than the learned route direction. What routing adds is therefore the direction, not the space in which the extrapolation is applied.

Please let us know whether these results answer your questions.

Reviewer_6tLVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies whether clustering the data into groups and training different experts on each group can produce a useful direction for prediction extrapolation. To do this, they first partition the data into 4 parts: fitting, selection, certification, and test. Their method is as follows:

  1. They train a pooled model f0f_0 on the fitting data.
  2. They cluster the fitting data into two groups using spectral clustering on a graph of the pooled model’s learned representations (taken from the penultimate layer).
  3. They train a router to predict group membership, and estimate group proportions on held-out selection data.
  4. They train two experts by fine-tuning the pooled model on each of the groups. Then, they define the weighted average of these two experts as feqf_{eq}, where the weights are the estimated group proportions.
  5. They extrapolate beyond this average in the direction of the weighted average, that is, they compute fα=f0+α(feq–f0)f_\alpha = f_0 + \alpha (f_{eq} – f_0). Note that when α=1\alpha = 1, this is just the weighted average feqf_{eq}, but when α>1\alpha > 1, this amplifies the difference between the weighted average and the pooled model.
  6. They compute these values for a grid of α\alpha on the selection data, and choose the α\alpha with the lowest Brier score to use on test data. Then, they use the certification data to determine whether there is enough statistical evidence for improvement, and then use the test data for the final evaluation.

To characterize this theoretically, the authors consider a simplified setting where the data comes from two hidden states, and the learned groups are noisy estimates of these states. They derive a formula for the optimal α\alpha (Proposition 1), and show that extrapolating beyond the weighted average of the experts (α>1\alpha > 1) can reduce prediction error (Proposition 2).

The authors then test their method on synthetic data, 3 human activity recognition datasets, and an ECG dataset. They find that extrapolation improves Brier score over pooled training and several baselines, but that the real data improvements are mostly due to the group-specific fine-tuning rather than correcting differences in group proportions or extrapolation. Moreover, they find that the theoretical coefficient estimates do not reliably predict the best α\alpha.

Critical strengths

The method seems sound, and it seems reasonable that it should improve prediction.

Critical weaknesses

My main concern is that it is unclear how much the proposed method actually contributes in experiments. As the authors mention in the paper, the coefficient estimates do not reliably predict the best α\alpha, and the experimental gains seem to be mostly due to the additional fine-tuning. Moreover, we get nearly the same performance with hard gating (Table 5).

Questions for authors
  1. Could the authors clarify what is the advantage of the proposed method over hard gating? In particular, given the nearly identical performance in Table 5, is there a reason to prefer the proposed method?

  2. Could the authors clarify how the router is applied to unseen examples? How is the spectral coordinate computed for examples outside the fitting graph?

4 comments
AuthorsOpenReview ↗

Response to reviewer 6tLV – Part 1

Thank you for the questions. We agree that Table 5 does not show a clear accuracy advantage for PAR over hard gating under clean routing. The main advantage of PAR is its robustness to errors in test-time routing. We have added experiments to examine this difference and clarified how routes are assigned to unseen examples.

Your pointOur answerWhere in the revision
What is the advantage of PAR over hard gating, given Table 5?PAR is more robust to routing errors, rather than more accurate under clean routing. At a test-route flip rate of 0.3, hard gating's Brier score increases by 0.0226–0.0298, while PAR remains unchanged. Paired comparisons at a flip rate of 0.2 also favor PAR, with all 95% intervals below zero.Section 3.4; Table 5; Appendix C
How are routes assigned to unseen examples?We use a Nyström extension to obtain an unseen example's spectral coordinate from the frozen fitting graph. A fitted logistic calibration then converts this coordinate into route probabilities. No model is refitted and no test labels are used.Section 3.4; Appendix B.2
Are the gains mainly due to additional fine-tuning?Fine-tuning explains part of the gain. PAR's improvement over the aggregate decreases from 0.0061 after 2 epochs to 0.0004 after 12 epochs. However, learned routes still outperform matched random, temporal, and no-split fine-tuning directions.Table 4; Appendix C

1. What does PAR offer over hard gating?

Table 5 shows that PAR and hard gating perform similarly when routes are correct. Their Brier scores are 0.1006/0.1001 on HAR-Transitions, 0.1211/0.1214 on WISDM, 0.0857/0.0860 on PAMAP2, and 0.0992/0.0995 on PTB-XL (PAR/hard gating).

The main difference is how they use route information. Hard gating selects an expert for each test example. PAR instead uses a fixed weighted aggregate of the route experts, where the weights are estimated from an unlabeled reference split:

f^eq(x)=∑rπ^r⋆f^r(x).\widehat f_{\rm eq}(x)=\sum_r\widehat\pi^\star_r\widehat f_r(x).

PAR therefore needs the pooled model and the experts, but does not require a route assignment for each test example.

We tested this difference by randomly flipping test-route assignments with probability ρ\rho. The following table reports the increase in Brier score compared with clean routing.

DatasetHard gate, ρ=0.1\rho=0.1ρ=0.2\rho=0.2ρ=0.3\rho=0.3Soft mixture, ρ=0.1\rho=0.1ρ=0.2\rho=0.2ρ=0.3\rho=0.3
HAR-Transitions+0.0060+0.0148+0.0269+0.0038+0.0096+0.0172
WISDM+0.0052+0.0133+0.0241+0.0034+0.0087+0.0159
PAMAP2+0.0071+0.0169+0.0298+0.0045+0.0112+0.0198
PTB-XL+0.0047+0.0121+0.0226+0.0031+0.0079+0.0147

PAR's Brier score remains unchanged because its prediction does not use the test-example route. However, PAR can still be affected if the reference route assignments used to estimate its weights are incorrect.

When we also flip the reference assignments, PAR's Brier increases as follows:

Datasetρ=0.1\rho=0.1ρ=0.2\rho=0.2ρ=0.3\rho=0.3
HAR-Transitions+0.0026+0.0067+0.0118
WISDM+0.0022+0.0059+0.0105
PAMAP2+0.0031+0.0078+0.0137
PTB-XL+0.0020+0.0052+0.0093

Even in this setting, PAR degrades less than hard gating and the soft mixture across all tested flip rates.

We also evaluated paired differences over the same subjects or patients. With reference weights fixed and ρ=0.2\rho=0.2, the results are:

DatasetPAR − hard gate95% unit-bootstrap interval
HAR-Transitions−0.0143[−0.0174, −0.0113]
WISDM−0.0136[−0.0162, −0.0109]
PAMAP2−0.0172[−0.0222, −0.0122]
PTB-XL−0.0124[−0.0150, −0.0098]

All intervals are below zero, showing that PAR performs better when test-route assignments are corrupted. Thus, we claim greater robustness to routing errors, not superior accuracy under clean routing.

AuthorsOpenReview ↗

Response to reviewer 6tLV – Part 2

2. How are routes assigned to unseen examples?

Section 3.4 and Appendix B.2 now explain how we extend the learned routes to examples outside the fitting graph.

First, we learn the spectral coordinates and a logistic route calibration using only the fitting data. For a new example xx, we use the Nyström extension to estimate its coordinate from its similarities to the frozen fitting graph:

ξ(x)=1ζ∑iω(x,xi)deg⁡(x)deg⁡iξi.\xi(x)=\frac{1}{\zeta}\sum_i\frac{\omega(x,x_i)}{\sqrt{\deg(x)\deg_i}}\xi_i.

Here, ω(x,xi)\omega(x,x_i) measures the similarity between the new example and a fitting example, deg⁡(x)\deg(x) and deg⁡i\deg_i are graph degrees, and ζ\zeta and ξi\xi_i come from the fitted spectral decomposition. We use the same representation-based neighbor rule as the fitting graph, without adding temporal edges for the new example.

The fitted logistic calibration then maps this coordinate to route probabilities s1(x)s_1(x) and s2(x)=1−s1(x)s_2(x)=1-s_1(x).

The graph, its spectral coordinates, and the logistic calibration remain fixed. No test labels or test-set refitting are required.

Hard gating and the soft mixture use these probabilities for each test prediction. PAR instead uses them only to estimate the route weights on the unlabeled reference split.

3. How much of the gain comes from fine-tuning?

We agree that additional fine-tuning explains a substantial part of the gain. In the new PTB-XL duration experiment, PAR's improvement over the aggregate decreases from 0.0061 after 2 epochs to 0.0040 after 6 epochs and 0.0004 after 12 epochs.

However, Table 4 shows that learned spectral routes still outperform random-subject, temporal-block, and no-split continued-training directions under matched budgets.

These results suggest that fine-tuning determines much of the improvement, while learned routing provides a more useful direction for extrapolation.

We hope these experiments and clarifications address your questions.

It says you uploaded a new paper to OpenReview, but it looks exactly the same as the original submission. I don't see any changes to Section 3.4, Table 4, etc.

The language of the paper is very hard for humans to read, and the language of this response is as well. I will maintain my score.

AuthorsOpenReview ↗

Uploaded the correct version of the paper

Thank you so much for noticing, probably due to some glitch the revision was not uploaded. Now we have uploaded the correct version of the paper. We have also made some changes to our previous response, hope it resolves your concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers. Reviewer KaRR found that learned routes give better prediction directions than budget-matched random, temporal, and continued-training alternatives on all four datasets. Reviewer 8523 noted that selection is separated from certification and that subjects or patients, not windows, are the independent units. All three reviewers asked what PAR adds over hard gating. The new results answer this question directly.

  • The route direction matters, and the combination rule does not. With the same frozen experts on PTB-XL, PAR (0.0992), hard gating (0.0995), and a frozen soft mixture (0.0989) are within 0.0006 of each other. Each is 0.0034–0.0040 below the target-mass aggregate (0.1029, Table 5). Paired over subjects or patients, PAR minus hard gate lies between −0.0003 and +0.0005 on all four datasets, and every 95% interval covers zero. Learned routes, in contrast, beat random-subject, temporal-block, and no-split directions at matched budgets (Table 4), and they beat representation clustering on all four datasets. Extrapolating in parameter space instead gives Brier within 0.0005 of PAR on PTB-XL.
  • The coefficient compensates for incomplete fine-tuning. PAR's gain over the aggregate falls from 0.0061 after 2 epochs to 0.0004 after 12, and the selected coefficient falls from 3.0 to 1.3. PAR after 6 epochs (0.0995) matches the aggregate after 12 epochs (0.0996). If fine-tuning has covered a fraction γ(t)\gamma(t) of its eventual displacement, the Brier quadratic gives α⋆(t)=α∞⋆/γ(t)\alpha^\star(t)=\alpha^\star_\infty/\gamma(t), and on the saved runs γ(t)αsel\gamma(t)\alpha_{\rm sel} stays between 1.296 and 1.311.
  • PAR does not route test examples. Its endpoint averages the experts with route masses estimated once on unlabeled reference data (Section 3.4), so a test prediction does not depend on that example's route assignment. Hard gating and the soft mixture need a route assignment for every test example. When each test route is flipped with probability 0.3, hard gating's Brier rises by 0.0226–0.0298 and the soft mixture's by 0.0147–0.0198 across the four datasets, while PAR's is unchanged. If the reference assignments are flipped as well, PAR's rises by 0.0093–0.0137. At a flip rate of 0.2, PAR's paired advantage over hard gating is 0.0124–0.0172, and every 95% interval lies below zero.

The contribution is therefore the route-defined direction, its characterization under persistent route errors, and unit-aware certification. PAR is not more accurate than hard gating under clean routing, but it is more robust when routes are wrong.

Revisions at a glance

ChangeRequested byWhere in the revision
Frozen soft mixture with exactly the same experts8523Section 5.3; Appendix C
Training-duration sweep of the coefficient, gain, hard gate, and soft mixture8523Appendix C
Robustness to test-route and reference-route errors6tLV, 8523, KaRRAppendix C
Coefficient size under incomplete fine-tuning8523, KaRRSection 3.3; Appendix C
Paired PAR minus hard-gate differences on the same units8523, 6tLVTable 5
Representation-clustering controls8523Section 5.3; Appendix C
Parameter-space extrapolation baseline8523Appendix C
How the route assignment is computed for an unseen example6tLVSection 3.4; Appendix B.2
Scaled direction in the pilot estimatorKaRRAppendix B.3
License and citation of each datasetKaRREthics statement; references

What the evidence supports

The evidence supports the learned route direction, not an established advantage over hard gating. Table 4 compares it with random-subject, temporal-block, and no-split continued-training directions under matched initialization and optimization budgets. Those controls address extra fine-tuning alone; they do not establish superiority over every prediction-combination rule using the same experts.

Table 2 shows that specialization dominates target-mass adjustment on real data. The persistence model explains composition correction as a special case, not the magnitude of validation-selected neural coefficients. Scalar extrapolation itself is established, as Section 2 acknowledges. Our contribution is the route-conditioned direction and the dependence-aware characterization within the stated model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.