SMART-GS: Blur-Guided Gaussian Splatting reconstruction

A blurred image does not tell 3D Gaussian Splatting how to learn the sharp details, while a deblurring network could not add enough details that are consistent across views. Both problems are strongly related and a joint optimization by no means straightforward. We propose SMART-GS, an offline deblurring 3DGS method in which image blur guides the optimization process without assuming a particular type of blur. We use the amount of blur in the input image to define how much each observed frame and a corresponding deblurred image should supervise the optimization method. The blurriness of the current rendered image then measures how many sharp details have been recovered which we use to update the supervision loss for recovering more details in the blurred image. We also use the blurriness of both images to guide 3DGS densification so the optimizer learns both what to trust and where to grow in a complex input video sequence. One set of sharp 3D Gaussian representations explains captured and deblurring observations through ordered, learned blur formation. At inference, only this sharp representation remains. SMART-GS reaches state-of-the-art deblurring performance on five real- world benchmarks: motion blur, defocus blur, extreme motion blur, TUM-RGBD, and the ScanNet dataset. We achieve 27.74 dB PSNR higher than the state-of-the- art methods on the extreme motion blur sequences. We also study how supervision and capacity affect the optimization of large indoor scenes.