MINT: Meeting-time INdicators for Truncation in Multi-Step Off-Policy RL

Multi-step off-policy TD methods correct the behavior–target mismatch through a product of per-step trace coefficients, yet this product inherently amplifies variance or discards long-horizon signal, making the cap length a sensitive hyperparameter. We propose MINT (Meeting-time INdicators for Truncation), which replaces coefficient products with a binary coupling indicator: one until the behavior and target trajectories meet, zero thereafter. Because the indicator is idempotent, variance depends only on the meeting-time survival probability—not on ratio products—and a single action sample estimates the per-step coupling probability without mixing-time knowledge. MINT contracts in supremum norm to $Q^{\pi }$ under arbitrary behavior policies; empirically, MINT outperforms handcrafted multi-step baselines across MuJoCo and DeepMind Control tasks without cap-length tuning.