nesf3d.github.io - NeSF: Neural Semantic Fields

home > domains > nesf3d.github.io

Description: Neural Scene Representations for Semantic Segmentation of 3D Scenes using soley 2D supervision.

nerf (138) nesf (1)

Example domain paragraphs

We present NeSF , a method for producing 3D semantic fields from posed RGB images alone. In place of classical 3D representations, our method builds on recent work in implicit neural scene representations wherein 3D structure is captured by point-wise functions. We leverage this methodology to recover 3D density fields upon which we then train a 3D semantic segmentation model supervised by posed 2D semantic maps. Despite being trained on 2D signals alone , our method is able to generate 3D-consistent semant

Given a pre-trained NeRF model, we sample its volumetric density grid to obtain the 3D scene representation. This grid is converted to a semantic-feature grid by employing a fully convolutional volume-to-volume network thus allowing for geometric reasoning. The semantic-feature grid is in turn translated to semantic probability distributions using the volumetric rendering equation. Note the semantic 3D UNet is trained across all scenes in the TrainScenes set, though not explicitly depicted for the sake of s

It is borrowing the source code of this website. We would like to thank the Utkarsh Sinha and Keunhong Park for all their help.

Links to nesf3d.github.io (3)

• kylegenova.com Kyle Genova
• msajjadi.com Mehdi S. M. Sajjadi | Personal website of Mehdi S. M. Sajjadi.
• msmsajjadi.github.io Mehdi S. M. Sajjadi | Personal website of Mehdi S. M. Sajjadi.