Automatic Neural Baking
Abstract
Baking precomputes an object's appearance into a texture on a mesh, but baked formats cannot represent sharp view-dependent reflection. Neural and splat representations can, yet they place appearance in the space around the object rather than on its surface, so there is nothing to bake. We identify two independent obstructions to the natural fix, binding splats to the faces of a known mesh, and prove both are fundamental rather than incidental. First, coplanar primitives admit no compositing order that is both view-stable and geometrically meaningful, and no cheap, per-primitive perturbation of the depth, such as a normal offset, can repair this. Second, reflection about a fixed normal is a rotation of the view sphere, so any per-primitive view-dependent color model is band-limited at a fragment regardless of how many primitives are used: adding primitives adds no angular bandwidth. We resolve both with Automatic Neural Baking, a construction that works rather than the only one that could. An on-manifold kernel field, 2D Gaussian disks anchored to mesh faces and blended by an order-free partition of unity, removes the ordering obstruction; factoring view dependence out of the elements into a shared environment map and a FastKAN decoder removes the locality bound. The result composes directly with the traditional rendering pipeline, baking splat-quality, real-time appearance onto a mesh.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.