Zhiwen Fan's Homepage
Pradyumna Chari
Shijie Zhou - Homepage
Description: Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields
feature (406) distillation (91) 3d gaussian splatting (5) 3dgs (1) radiance fields (1) feature fields (1)
3D scene representations have gained immense popularity in recent years. Methods that use Neural Radiance fields are versatile for traditional tasks such as novel view synthesis. In recent times, some work has emerged that aims to extend the functionality of NeRF beyond view synthesis, for semantically aware tasks such as editing and segmentation using 3D feature field distillation from 2D foundation models. However, these methods have two major limitations: (a) they are limited by the rendering speed of Ne
We adopt the same 3D Gaussian initialization from sparse SfM point clouds as utilized in 3DGS, with the addition of an essential attribute: the semantic feature . Our primary innovation lies in the development of a Parallel N-dimensional Gaussian Rasterizer, complemented by a convolutional speed-up module as an optional branch. This configuration is adept at rapidly rendering arbitrarily high-dimensional features without sacrificing downstream performance.
Feature 3DGS empowers pixel-wise semantic scene understanding from any novel view, even with unseen labels, by mapping semantically close labels to similar regions in the embedding space of 2D foundation models.
Zhiwen Fan's Homepage
Pradyumna Chari
Shijie Zhou - Homepage