acceptodds
Under review as a conference paper at ICLR 2027

Forecasting Before the Outcome: Catalogue Gated Prequential Cognitive Performance under Participant and Domain Shift

Abstract

Online assistance needs a performance forecast before the response and a principled way to decide whether source knowledge transfers to the current participant. We introduce the Support-Aware Prequential Forecaster (\method), which uses a fixed catalogue of supported experiment identifiers to retain source predictions when transfer is expected and switches to hierarchical Beta adaptation for unseen tasks. Every route decision and probability is committed before the current response; feedback updates only subsequent trials. On an external auditory cohort with 13 participants and 28,452 trials, SAPF achieves NLL 0.066 and AUROC 0.929, while immediate source exclusion increases first-five-trial Brier by 0.039 relative to retaining the source. A feedback-weighted SafeMix baseline is useful when catalogue membership is misleading, reducing macro Brier from 0.113 to 0.071 under identifier conflict. Two-expert and temporal ablations show that the main gain comes from the route decision and from recovering source weight after transient failures. In a prefix-causal replay with a 10% budget cap, SAPF covers 42.2% of errors at a 5.8% call rate, compared with 9.0% coverage at a 9.5% call rate for random allocation. These results position pre-response forecasting as an explicit transfer-control layer: retain source knowledge when catalogue support holds, fall back at unsupported cold starts, and update only when later feedback justifies a change. Together, the results show how SAPF turns transfer uncertainty into an actionable routing decision for budgeted online assistance under participant and domain shift.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.