TRIVE: Token Responsibility Modeling for Streaming Egocentric Video Editing

Egocentric video editing seeks to modify object appearance while preserving the recorded hand–object interaction, offering a practical route to visual data augmentation for augmented reality and embodied policy learning. This setting is challenging because latent video tokens near interaction boundaries often mix editable object content with hand structure that should remain unchanged. We introduce TRIVE, a framework for streaming egocentric video editing that models each token with soft object, interface, hand, and background responsibilities. These responsibilities form a shared representation from which we derive three operation-specific permissions: source retention, target-appearance access, and reference eligibility. We instantiate these permissions through role-aware attention control and a role-guided denoising update, including directional filtering that suppresses source corrections opposing the observed source-to-target response. To reduce appearance drift across causal blocks, we further construct a source-addressed appearance anchor from reliable object tokens and enable broader role-conditioned retrieval, including mixed interface tokens. TRIVE operates on a frozen streaming backbone and applies its responsibility-based controls during inference. Across egocentric and robot-manipulation benchmarks, TRIVE improves editing fidelity, source preservation, and temporal consistency over the evaluated baselines, while complementary hand- and interaction-focused diagnostics further support its preservation capability. Real-robot experiments further suggest that its edited demonstrations can improve policy performance under held-out visual conditions.