LumiLens: Choreographing Spatially-Varying Light and Lens for Generative Monocular Video Re-rendering

Our perception of the real world is dynamic, jointly shaped by geometry, intrinsic attributes, and lighting. Building upon this vision, we present LumiLens, the first generative framework that internalizes this intricate interplay for monocular video re-rendering that simultaneously controls viewpoint and fine-grained, parametric spatially-varying local illumination. It is driven by two key observations: (1) unlike global relighting which essentially acts as a uniform 2D filter, local relighting is a high-frequency near-field 3D rendering problem where every pixel receives a distinct light vector, demanding geometry-aware spatial reasoning; and (2) consistent with the principles of radiance transport, surface appearance is inherently defined by the coupling of 3D geometry, lighting, and viewing direction, necessitating precise camera control to synthesize reflections that faithfully respond to perspective shifts. Existing works fail to achieve such joint control or suffer from the "Baked-In Reflection", manifesting as structural stretching artifacts and original highlights that fail to respond to viewpoint changes. To bridge this gap, we introduce a Physically-Grounded Dual-Stream Conditioning Mechanism that decouples spatial illumination modeling from intrinsic scene semantics. The Spatial Stream explicitly encodes the spatial illumination distribution of light-geometry interactions via warped depth and parametric light proxies, while the Global Stream injects global lighting conditions and anchors inherent scene structure and dynamic semantics by integrating the source video. To tackle the scarcity of paired training data, we introduce a scalable Omni-View Illumination (OVI) pipeline based on Unreal Engine 5. By employing a Decoupled Light Rendering strategy, we synthesize over 134K physically disentangled video pairs. We validate the effectiveness of our approach through extensive experiments, demonstrating that LumiLens successfully disentangles illumination from geometry. This capability enables not only the precise simultaneous manipulation of lighting and viewpoint but also the flexible, independent control of each dimension across both synthetic and real-world scenarios.