CoHSPAD: Continuous Human Gaussian Reconstruction from Raw Monocular 1-bit SPAD Streams
Abstract
Monocular human reconstruction from conventional RGB video is limited by finite frame rates and exposure time. Rapid motion causes severe blur, degrading the image quality needed for reliable pose estimation and appearance recovery. SPAD cameras offer an alternative: binary photon-arrival observations at very high temporal resolution, which remain informative under low light and rapid motion. However, a single 1-bit frame is extremely sparse, and independent per-frame optimization of SMPL parameters and 3D Gaussians is easily destabilized by noisy photon gradients. We present CoHSPAD, a monocular framework for 3D Gaussian reconstruction of dynamic humans and static scenes with continuous human motion recovered directly from raw SPAD streams. Built on a differentiable SPAD imaging model, we supervise rendering directly in the 1-bit raw sensor domain. Instead of independent per-frame poses, we model motion as a continuous SMPL trajectory with sparse temporal knots that share supervision across the stream and stabilize pose gradients. We introduce a Multi-Scale Temporal Fusion Network (MTF-Net) to aggregate long and short streams per-timestamp, providing a more stable signal than a single binary frame while avoiding blur from naive temporal averaging. Experiments on our newly collected CoH-Synthesis and CoH-Real show that CoHSPAD remains robust and temporally consistent under inaccurate initial SMPL poses and rapid motion.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.