- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper considers adapting a residual predictive distribution while keeping a supplied task marginal unchanged. Source information defines a family of conditional corrections, and a small target sample updates their weights. The analysis separates restrictions of the candidate family from estimation error, including a symmetry-based approximation barrier. Controlled PDE experiments examine the benefits and costs of the resulting adaptation procedure.
Critical strengths
-
The theoretical separation is the strongest part of the paper. Exact preservation of a task law, the approximation error imposed by a shared symmetry, and an expected estimation guarantee address different questions. The paper generally keeps those questions distinct. In particular, it does not replace the individual-candidate oracle with a stronger convex-mixture guarantee that the estimator cannot satisfy.
-
The experiments are reported with appropriate restraint. The authors show where prefix averaging loses to the final posterior, where broader directional coverage makes the aggregate result worse, and where improved log score does not improve an event prediction. Those outcomes help explain the construction's limits. Treating trained seeds as the unit of inference also avoids overstating the number of independent model replications.
Critical weaknesses
-
The practical payoff is less established than the mathematical result. The main application is restricted to one PDE and a projected residual representation. The preserved task distribution is supplied to the conditional procedure, so exact preservation does not establish that this law is well calibrated. Likewise, the main paired log-score comparison removes that common marginal. The event results in Section 5.5 are mixed and provide only limited evidence for better operational decisions. I regard this as a limitation of application scope, rather than an error in the preservation claim.
-
The estimator comparisons do not fully separate aggregation from source protection. The proposed construction gives substantial protection to the source reference, whereas the unregularized mixture fit and continuous alternatives do not match that protection. The advantage over those baselines is therefore harder to attribute to the chosen estimation rule. Moreover, the final posterior performs better than prefix averaging in the principal setting at several budgets. The authors acknowledge this, but it means the case for prefix averaging rests chiefly on its expected guarantee. A comparison with source-regularized fitting would make the practical choice easier to assess.
Questions for authors
- Can the absolute adaptation gains for the additional architecture be reported? The current difference-of-gains comparison does not by itself establish its effectiveness there.
- Was a source-regularized fit over the same candidate family evaluated? Existing results would help distinguish the contribution of source protection from the estimation rule.
- Is any assessment of the supplied task marginal's calibration or the absolute joint predictive score available? This would clarify what the conditional improvement means for the whole predictive distribution.
- How sensitive are the main gains to the prior mass assigned to the source reference? Please summarize any existing sensitivity analysis.
- What do the independent source-direction checks establish about selection stability? Please distinguish finite-difference accuracy from the nonlinear-response approximation discussed in Appendix A.16.
2 comments
Official Comment by Reviewer gWA6
Thanks to the author for the rebuttal. After reading the rebuttal carefully, I will maintain the score.
Response to Reviewer gWA6
Your review asks what the conditional construction contributes in practice, and how much of the estimator comparison comes from source protection. We now follow those questions through to physical scores, absolute joint predictions and matched regularized fitting. The applications retain useful task forecasts and adapt the joint information those forecasts leave open. Matched comparisons then measure how fitting rules use the same family and source prior.
W1. Practical payoff and application scope
The revision evaluates complete Burgers regional predictions and adds weather, hydrology, advection–diffusion and particles (Section 5.4; Table 23). On G5, original C774 lowers tail OR-event Brier by 3.31×10−4, with improvement interval [1.31,5.30]×10−4. Weather J121 prefix lowers gust CRPS by 10.7% and 3.49% at the two stations; the combined gain is 0.111 m/s [0.0127, 0.205]. The protocol's primary endpoint has gain 0.115 m/s with its own interval [-0.0126, 0.230]. In hydrology, retaining supplied volume model m1 lowers volume and peak-flow CRPS by 26.3% and 41.7% relative to an additional 32-label update with shape fixed. With volume fixed, J13 endpoint and MAP4 improve peak-date RPS by 6.16% and 8.14%; HEFS shape remains strongest (Tables 24, 27 and 28). Table 3 also evaluates a frozen whole-joint branch-selection policy directly in event Brier risk.
W2/Q2. Aggregation versus source protection
We added MAP fits with strengths 4, 16 and 64 over the same C774 candidates, prior, labels, marginal and completion. Their objective is conditional log-likelihood minus τKL(π∥w). Realized reference weights make the comparison concrete: prefix, final and endpoint average 0.773, 0.704 and 0.707; MAP4, MAP16 and MAP64 average 0.665, 0.774 and 0.797 over all 448 G5 contexts. At almost equal mean reference weight, prefix has mean NLL 9.805 versus MAP16's 9.814 (Tables 2, 11 and 22).
The stronger tail/combined performance of other estimators remains explicit. On combined shift, NLL differences from source are -0.215 for prefix, -0.219 for MAP4, -0.225 for final and -0.227 for endpoint. Prefix retains the expected source-reference guarantee and has lower null cost than MAP4 (+0.00184 versus +0.0204). These matched comparisons explain the practical tradeoff behind the estimator choice.
Q1. Gains within the additional architecture
C774 improves over the custom convolutional backbone's own source reference in each of its five trained instances for all three main shifts. Mean NLL differences are -0.254 [-0.423, -0.0845] for tail, -0.0207 [-0.0273, -0.0141] for dependence and -0.323 [-0.501, -0.145] for combined. Each checkpoint uses its own source chart and response direction. Table 17 retains all six controls and the separate FNO difference-of-gains comparison.
Q3. Quality of the task marginal and absolute joint prediction
We report weather task, conditional and joint NLL together with marginal calibration. At Klippeneck, with pooled source H fixed, an independent 2017 temperature update lowers task NLL from 1.828 to 1.793 and joint NLL from 4.517 to 4.482. Nominal 90% coverage moves from 95.5% to 93.2%. Conversely, with the supplied marginal fixed, J121-prefix adaptation lowers joint NLL from 4.525 to 4.253 (Table 25). For hydrological volume model m1, nominal 50% physical equal-tail coverage is 0.938, and the corresponding central randomized-PIT frequency is 0.758; the latter accounts for the zero-flow atom (Table 29). These measurements separate task-forecast quality from conditional improvement.
Q4. Reference-prior sensitivity
Raising reference mass from 0.5 to 0.95 lowers null cost from 0.00681 to 5.10×10−5 nats, while tail, dependence and combined gains remain 0.296, 0.0125 and 0.334 nats. Coarsening the radial grid from 17 to 9 nodes changes mean prefix NLL by at most 2.01×10−4 nats. Tables 18–20 report all priors, grids and estimators.
Q5. What the independent direction checks establish
The local Jacobian rule has finite-difference relative error [3.13,3.39]×10−7 and reproduces the selected forcing pair in 20/20 independent source checks, with direction cosines 0.993–0.99993. Agreement with the full nonlinear covariance-response criterion is 16/20; at finite correlations ±0.2, the 40 checks have response discrepancies of 1.37–7.11%. Appendix A.9 uses Price's identity to explain the curvature terms separating those criteria. The additional covariance-response subspace is a separate representation extension: its cosines are 0.372, 0.656, 0.408 and 0.922, with three of four sources failing the combined angle/eigengap check (Table 21).