- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies when a linear detector should assign positive or negative weights to historical verifier scores. Under a binary Markov state model with correlated observation residuals, the paper characterizes the signs of the optimal coefficients and shows that the normalized total weight placed on history decreases as observation quality improves.
The extensions give a necessary-and-sufficient innovation condition for the sign law and show that finite initialization can reverse signs or produce kernels with both positive and negative weights. The evaluation combines controlled simulations, numerical covariance checks, and comparisons of 27 aggregation methods across 6 observation settings on 2000 MATH/GSM8K problems.
Critical strengths
-
The theory explains the structure of the optimal weights beyond the standard covariance-inverse solution. Thm 1 determines the sign of every historical coefficient and establishes monotonicity of the total historical weight for arbitrary fixed class weights. Thm 2 identifies exactly which innovation profiles preserve this sign law across observation qualities. Thm 3 then explains why simply truncating a stationary filter cannot reproduce finite-start sign reversals. These results provide a substantive understanding of how the optimal weights behave.
-
The experiments distinguish the benefits of using history from the accuracy of the theoretical weight prescriptions. Problem-level nested fitting, EWMA controls tuned to the corresponding metric, and position-aware comparisons address important attribution questions. Table 2 shows improvements over scores based only on the current observation, while Table 8 and Appendix N.4 report cases where the fitted model’s prescribed weights fall short.
Critical weaknesses
my main concern: The practical value of the distinctive cancellation results remains unestablished. Every fitted stationary setting satisfies , placing these kernels in the positive-smoothing regime (Appndix M.6). Appendix N.6 finds no consistently supported subtraction direction under its stated descriptive rule. Moreover, the colored kernel falls below SNR-tuned EWMA in 5 of 6 within-position point estimates (Table 2), and substantial higher-lag residual discrepancies remain (Table 9). Thus, the experiments support beneficial history aggregation more strongly than practical use of cancellation or the exact prescribed weights. This limits demonstrated significance without contradicting the theoretical results.
Questions for authors
-
Do any fitted finite-start Fisher kernels assign negative weights at supported positions? If so, how do those kernels compare with nonnegative kernels optimized under the same fitted moments? Appendix N.6 examines individual perturbations around scoring based only on the current observation. Those comparisons do not determine the signs or benefits of the jointly optimal kernel.
-
Does relaxing the parametric moment model improve the empirical Fisher prescription? Appendix N.5 documents departures from the assumed covariance and conditional-mean structure, while Appendix N.4 shows that matching the evaluation objective does not eliminate the shortfall relative to tuned EWMA. Within the existing evaluation protocol, could a regularized Fisher rule estimated directly from training-set means and covariances over a short lag window clarify this discrepancy?