Tracing Context Ghosts in Agentic Replay Training

Leveraging past interaction trajectories as training data provides a scalable way for agents to improve from their own execution histories. Yet data reuse pipelines often sanitize or shorten the recorded context while retaining the continuation produced from the original context as the training target. This practice introduces a hidden mismatch where the retained target still encodes behavioral effects triggered by the absent context. We term this residue a context ghost. We isolate this mechanism by independently varying the context used to generate the target and the context retained to train the model. This explicitly contrasts models fine-tuned on identical sanitized prompts but supervised by targets generated with and without extraneous metadata. Comprehensive evaluations reveal that target provenance shapes learned behavior even when the original influencing context is absent. Furthermore, simply cleaning the training prompt while keeping old targets amplifies these unintended behavioral shifts. Regenerating targets under cleaned contexts largely mitigates the problem and improves task success. Ultimately, robust agent replay training requires alignment between training inputs and the exact conditions that produced their supervision.