When is Historical Feedback Enough to Change a Recommendation?

Historical feedback can help a recommender decide whether to change its recommendations. However, more likes need not mean more sustained viewing, and the relation between these responses can change over time. We study when historical observations provide enough evidence for an improvement despite this uncertainty. Our method estimates changes in the feedback relation jointly and uses the same calibration set for every proposed recommendation. It checks both whether the estimated gain is large enough and whether the current feedback still points in the right direction. The decision accounts for repeated observations from a user and for choosing how much history to use. Under the stated assumptions, the rule bounds the chance of accepting a change that falls short of the required gain. A KuaiRand-Pure study shows that the benefit is regime dependent: shared calibration is useful in a calibration-limited matched setting, while fixed borrowing, separate-stage fitting or additional current labels can be preferable elsewhere. These findings connect the value of historical feedback to the information still needed before a recommendation changes.