Mikaela Angelina Uy
3D Vision Reading Group | 3D Vision Reading Group webpage.
Description: NeRF Revisited: Fixing Quadrature Instability in Volume Rendering
Neural radiance fields (NeRF) rely on volume rendering to synthesize novel views. Volume rendering requires evaluating an integral along each ray, which is numerically approximated with a finite sum that corresponds to the exact integral along the ray under piecewise constant volume density . As a consequence, the rendered result is unstable w.r.t. the choice of samples along the ray, a phenomenon that we dub quadrature instability . We propose a mathematically principled solution by reformulating the sampl
One of the key underpinnings of the recent advances of coordinate-based representations, e.g. NeRFs, is volume rendering. Volume rendering enables end-to-end differentiable rendering, and hence has made learning of 3D geometry and appearance from only 2D images possible. In practice, the volume rendering integral is evaluated with quadrature resulting in the expressions shown below.
These expressions are what we've gotten used to, which is the exact integral under the piecewise constant assumption to opacity and color. However, this seemingly simple, innocuous assumption can result in the rendered image being sensitive to the choice of samples along the ray. While this does not necessarily cause a practical issue in classical rendering pipelines, it has surprising consequences when used in neural rendering.
Mikaela Angelina Uy
3D Vision Reading Group | 3D Vision Reading Group webpage.