Trajectory Editing as Posterior Inference for Offline Reinforcement Learning

Offline RL performance depends on the quality of trajectories in a fixed dataset. Nevertheless, practical datasets often contain structural deficiencies along two axes: spatial gaps, where useful locomotion segments lack connecting transitions, and temporal sparsity, where manipulation demonstrations under-sample critical phases such as contact, grasping, and alignment. These deficiencies limit downstream policy learning, reducing returns or success rates. We introduce trajectory editing to repair offline data without environment interaction by bridging disconnected segments or densifying critical phases. The key challenge is to satisfy the prescribed anchors of each edit while remaining within the support of the offline dataset and consistent with its dynamics. We propose Bayesian Inference for Trajectory Editing (BITE), a unified framework that treats the edited trajectory as a latent variable to be inferred. BITE learns a prior over plausible trajectories from the offline dataset and constructs an anchor-consistency likelihood from the edit specification, which induces a posterior that trades off data plausibility and edit satisfaction. We instantiate this prior with a diffusion trajectory model and derive a lifted posterior sampler that propagates clean-level edit observations through the reverse diffusion chain using backward sequential Monte Carlo. The resulting posterior inference procedure achieves sampling consistency as the particle population grows and its edit posteriors are provably as accurate as the learned prior, with no dependence on any edit distribution. Experiments on D4R locomotion and RoboMimic manipulation tasks show that BITE delivers the strongest performance gains among baselines. Our code is publicly available at https://anonymous.4open.science/r/bite-055D.