Forecasting Before the Outcome: Catalogue Gated Prequential Cognitive Performance under Participant and Domain Shift
Abstract
Online assistance needs a performance forecast before the response and a principled way to decide whether source knowledge transfers to the current participant. We introduce the Support-Aware Prequential Forecaster (\method), which uses a fixed catalogue of supported experiment identifiers to retain source predictions when transfer is expected and switches to hierarchical Beta adaptation for unseen tasks. Every route decision and probability is committed before the current response; feedback updates only subsequent trials. On an external auditory cohort with 13 participants and 28,452 trials, SAPF achieves NLL 0.066 and AUROC 0.929, while immediate source exclusion increases first-five-trial Brier by 0.039 relative to retaining the source. A feedback-weighted SafeMix baseline is useful when catalogue membership is misleading, reducing macro Brier from 0.113 to 0.071 under identifier conflict. Two-expert and temporal ablations show that the main gain comes from the route decision and from recovering source weight after transient failures. In a prefix-causal replay with a 10% budget cap, SAPF covers 42.2% of errors at a 5.8% call rate, compared with 9.0% coverage at a 9.5% call rate for random allocation. These results position pre-response forecasting as an explicit transfer-control layer: retain source knowledge when catalogue support holds, fall back at unsupported cold starts, and update only when later feedback justifies a change. Together, the results show how SAPF turns transfer uncertainty into an actionable routing decision for budgeted online assistance under participant and domain shift.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.