Exchange, Not Gating: Event-Guided Deblurring and Deraining

Event cameras record brightness changes asynchronously at high temporal resolution, making them a natural guide for RGB restoration tasks such as motion deblurring and image deraining. Existing methods, however, still couple the two modalities through three heuristic choices: they process event voxels with blocks designed for images, control event injection with hand-designed activity masks and unconstrained gates, and decode fused features without directional guidance. We replace these choices with three complementary mechanisms. VIBE provides event-native processing by separating polarities and conditioning the event stream on the mass, signed mass, temporal centroid and temporal spread of each pixel's event histogram, thereby removing the baseline's hand-designed blur-and-mask pipeline. COEX replaces independent gates with a per-channel $2\times 2$ mixer projected toward the doubly stochastic manifold, encouraging convex cross-modal mixing while approximately preserving the sum of the two inputs. COMPASS predicts a per-pixel distribution over five fixed shifts from event features and their moment condition, realigning fused features without regressing offsets. The resulting single-frame model achieves the best event-guided results on GoPro, HighREV and NTURain: on GoPro, it reaches $37.72$ dB compared with $37.10$ dB from the official TRM-UNet weights, while a larger configuration reaches $37.97$ dB.