Consensus Forgets First: Online Adaptation of Frozen Predictors

When a pool of frozen models predicts the same target under distribution shift, the right adaptation memory is unknown and itself non-stationary. The Gram matrix of the pool's predictions, however, has a spectrum known before deployment, which acts as a prior on memory. With one covariance-inflation scalar, the dominant direction, the consensus level, carries the shortest memory while contrasts between models keep long memory. We take the ratios between memory lengths from the spectrum and hedge over their common scale by aggregating recursive least-squares experts online. After standardizing by the training scale, the layer has no tuned parameter. We prove deterministic oracle inequalities: safety against the best raw predictor, and path-length tracking against the best time-varying combination. On French electricity load (2019–2023) and NYC taxi ridership, the layer improves on every frozen base and on aggregation of the raw pool in every regime, and on validation-tuned adaptive filters overall. Without the inflation scalar, the same hedge loses to a validation-tuned Kalman filter.