- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper compares two recursive filters for a stationary -state HMM with scalar Gaussian emissions and symmetric switching probability . One is exact Bayes, written in centered logits. The other, which the authors call the radial filter, replaces the exact transition-mixing step with a map that shrinks the centered logit vector along its own direction, capped through a at the confidence scale . Theorem 3 takes the input and shows that the two transition maps differ there by about in pairwise log-odds, with , while the KL between the corresponding softmax outputs goes to zero. Theorem 4 runs both filters on the same stationary path from a uniform start. It shows that the expected KL between their filtered posteriors at horizon tends to zero as , for fixed and fixed distinct means. The proof conditions on a switch-free window, shows that both filters put a margin growing with on the true state, and then uses softmax curvature on that event and a envelope on the rest. The experiments are a simulation with equally spaced means for , a set of binary scalar controls (tanh, hard clip, affine, identity), and Mamba, GRU, scalar S6 and finite-window models trained by distillation or end to end. The stated takeaway is that an unbounded gap between update maps does not by itself imply predictive loss.
Critical strengths
-
The proofs are careful. I went through Theorem 3 and the branch of Theorem 4 (Lemmas 8 to 20) and did not find errors. The pathwise Abel-summation bound in Lemma 9 handles the adaptive weights without a union bound over time, which is a nice touch.
-
The paper is explicit about scope. Table 1, Section 8 and Appendix F say what is and is not claimed, and the empirical protocol is clean: disjoint tuning and confirmation seeds, seed-level standard errors, per-seed results in the supplement.
-
The authors report results that cut against them, for example the learned Mamba arm missing the pre-specified criterion and the affine control beating tanh in the tail cells.
Critical weaknesses
My main concern is that the paper's message does not depend on the radial filter. Given the metric used for the internal gap and the asymptotic regime used for the KL, the message is close to immediate, and a much simpler filter shows the same thing.
-
Theorem 3 restates a known property of the Hilbert metric. is the range of , which is exactly the Hilbert projective distance between the two softmax outputs. That distance is unbounded near the boundary of the simplex while KL is not: for and , while . At both maps put a margin of order on the same coordinate, so Theorem 3 is this effect. The gap is also small: , so passes one nat only once (for ).
-
Neither theorem singles out the radial filter. The identity accumulator ignores the transition model and never recovers from a switch, yet it shows both effects more strongly. Its gap to exact mixing is about at ; for and I get with . The proof of Theorem 4 also appears to go through for it, because its margin is the same sum of log-likelihood ratios as in Lemma 15. In the paper's configuration (means , , ), I get for the identity accumulator against for the radial filter at . Appendix F.1.2 reports the same direction for the affine control. The radial construction is therefore not needed for the headline claim, and the paper does not say what it buys.
-
The regime excludes switches. The window has length while switches arrive about every steps, so the probability of a switch in the window is at most , and Theorem 4 reduces to sequential testing on i.i.d. data. The authors say it "does not characterize post-switch tracking" (line 285), and none of the simulated paths with contains a switch (Appendix F.1.2). Switches are where the filters differ. Conditional on one switch in the window, I get KL between 0.18 and 0.57 for the radial filter and about 0.75 for the identity accumulator. A result stated per switch would make the radial design matter. Short memory suffices in this regime (Sharan et al., 2018), so the result does not speak to the state-tracking separations that motivate the paper.
-
The connection to SSMs is weak. The radial filter is hand-built and is not shown to be implemented or learned by any architecture (lines 164-165). Trained end to end, Mamba misses the pre-specified five-of-six criterion, and under distillation both Mamba depths pass zero of six at the long endpoint. The closure statistic has a denominator of at and ranges from to ; raw KL to exact Bayes would be clearer. The next-observation NLL and calibration error named in the protocol were not measured.
-
The paper is hard to read. Much of the text lists what is not claimed, the abstract uses an undefined , and standard objects get new names (the "quotient distance in centered logits" is the Hilbert metric). Proposition 6, which Remark 7 says cannot give a lower bound, and the unfinished lower bound in Appendix A take space that could go to explaining why this radial map was chosen.
Questions for authors
-
Do Theorem 4 and the analogue of Theorem 3 hold for the identity accumulator and for the affine recurrence ? If they do, which property established in the paper does the radial filter have that these two lack?
-
What happens to the expected KL conditional on a switch in the window, or to the cumulative KL per switch over a stationary window much longer than ? Is there a regime, for example with shrinking as or growing, in which the radial filter provably beats the identity and affine filters and still matches exact Bayes?
-
Why this particular radial map? The paper says it matches the exact saturation scale and the midpoint slope. Is it optimal or canonical in any sense among maps that preserve direction?
-
Could you report raw KL to exact Bayes for every learned arm next to the closure values, together with the next-observation NLL and calibration error that the protocol specified?
-
Is there any result in the other direction, i.e. a condition on the update gap measured in a metric that stays bounded on the simplex (total variation between beliefs, say) that forces predictive loss under the path law? Even a simple version would make the message usable.