- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies non-Markovian evolving domain generalisation (EDG), where domains drift continuously within a regime but the drift mechanism itself can change abruptly. The authors propose a hierarchical switching state-space model in which a change indicator ct and a regime variable rt govern the transition statistics of a continuous domain state zt, and a Rao-Blackwellized particle filter (RBPF) jointly infers the discrete regime history and the continuous state. The method includes a hazard process, regime priors, an inner moment-matching update, particle weights, change-point readout, and resampling. The authors also construct abrupt EDG benchmarks (Circle, Sine, RMNIST, PowerSupply, Portraits, CalTran) with controlled shifts. Table 2 reports mean target accuracies of 88.63% on the original streams and 82.58% on the abrupt variants. The paper concludes that explicit regime and change-point inference improves post-change generalisation over a continuous-only SSM.
Critical strengths
Separating within-regime gradual drift from a change in the drift mechanism is well motivated, and the hierarchy in Section 3.2 gives the two effects separate, interpretable roles; this is more suitable for abrupt EDG than a single continuous random walk.
Section 3.3 gives a concrete RBPF procedure, including the inner moment-matching update, particle weights, change-point readout, and resampling; the authors are also candid about the limits of change-point identification (Section 5, Figure 5).
Critical weaknesses
- I had difficulty reconciling the filtering procedure with the EDG setting. Section 3.1 (p. 4) defines each domain as a labeled batch. Eq. (5) (p. 5) uses its labels in the likelihood, and Algorithm 1 (p. 6, steps 8-11) updates the posterior before predicting labels. The loop covers the full stream without a target-domain branch. Read literally, predictions can depend on labels from the domain being evaluated.
Section 4.1 (p. 7) rules out target labels for hyperparameter selection, but does not settle their use in filtering. Section 5 (p. 9) says predictions precede the next domain's observations. If the reported scores are one-step-ahead predictions, this indexing needs to be explicit in the algorithm and prediction formula. If there is instead a labeled support set, its separation from the scored examples and the baselines' label access need to be stated.
Eq. (5) also cannot by itself explain an unlabeled target update: marginalizing its unknown labels gives one. I cannot establish label leakage without the code, but the information used for the scored predictions is not clear enough to interpret Table 2.
- Table 3 (p. 8) matches the observation family and particle count; Appendix C (p. 14) lists D=4, K=600, and L=8 for both Circle/Sine methods. However, Section 3.3 (p. 6) describes the continuous-only control as a bootstrap filter, while the full method uses Gaussian moment matching. As written, the comparison changes both the regime variables and the treatment of the continuous posterior. Keeping the same inner update and fixing the regime would isolate switching more clearly.
The gains are uneven. Abrupt Circle rises from 43.70% to 95.90%, while Sine and PowerSupply gain only 0.88 and 0.57 points. Section 4.2 (p. 8) attributes the Circle result to the invert bit, also specified in Appendix B (p. 13). This supports the full mechanism on the constructed reversal, but does not isolate how much of the average 12.47-point gain comes from the change-point indicator itself.
- In Figure 5 (p. 9), the first dashed change marker appears at t=8 for both Sine and RMNIST. Table 1 (p. 7) and Appendix A (p. 13) specify t=12 and t=10, respectively. The second markers agree at t=16 and t=13, so a zero-based versus one-based indexing difference does not explain this. Were the curves generated with the same configurations as the accuracy tables? This needs resolving before the claim of detection without delay can be assessed.
The RMNIST curve also has additional high-probability alarms, as the caption acknowledges. Section 4.3 gives no detection-delay or false-alarm statistics, leaving the reliability of the detector difficult to judge.
Questions for authors
Does Table 2 score current-domain predictions after updating with Dt labels, or next-domain predictions before Dt+1 is observed? Which labels enter Eq. (5) and Algorithm 1? Please give the exact evaluation order, any support/query split, and the baselines’ label access.
Does the continuous-only control use the same Gaussian moment-matching inner update as the full method? If not, can you provide a matched control with fixed regime and the same inner update to isolate switching? Please also report variance or independent runs for the sub-percentage gains on Sine and PowerSupply.
Which first-change locations were used for Figure 5? Please reconcile t=8 with Table 1’s t=12 and t=10, and report detection delay and false alarms. Also clarify why Circle’s first change at t=10 lies within the 15/5/10 source prefix, while Section 4.1 says jumps are placed after the source split.