When Should an Agent Trust a Reward? Trajectory-Supported Learning under Corrupted Reward Observations

Reinforcement learning typically treats observed rewards as direct supervision, yet deployed agents may receive feedback corrupted during reward observation or transmission. Without structural assumptions, such corruption cannot be distinguished from rare but valid rewards using trajectories alone. We study a setting in which observed rewards equal clean environment feedback plus sparse or transient corruption that is weakly coupled to transition dynamics. We introduce ReConRL, which estimates trajectory support for each observed reward and modulates its influence on value learning rather than attempting to reconstruct an unobservable true reward. Support is estimated using temporally separated return evidence and transition-conditioned signals obtained through distinct estimation pathways, reducing dependence on self-consistency from a single critic. We construct matched cases that separate rare-but-valid rewards, frequent corrupted rewards, and equally unusual rewards with different trajectory support, directly testing whether rarity is confused with unreliability. Evaluation measures corrupted-reward acceptance, genuine-reward rejection, and policy return. This formulation studies when an observed reward is sufficiently supported by trajectory evidence to guide learning without assuming that unusual feedback is necessarily untrustworthy.