Yi Ren (任意) - Homepage
Zhenhui Ye (叶振辉) - Homepage
Weichuang Li
Description: Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis.
nerf (138) one-shot (16) real3d-portrait (1)
Paper arXiv Code Rebuttal Demos Abstract One-shot 3D talking portrait generation aims to reconstruct a 3D avatar from an unseen image, and then animate it with a reference video or audio to generate a talking portrait video. The existing methods fail to simultaneously achieve the goals of accurate 3D avatar reconstruction and stable talking face animation. Besides, while the existing works mainly focus on synthesizing the head part, it is also vital to generate natural torso and background segments to obtai
To address these limitations, we present Real3D-Potrait , a framework that (1) improves the one-shot 3D reconstruction power with a image-to-plane (I2P) model that distills 3D prior knowledge from a 3D face generative model; (2) facilitates accurate motion-conditioned animation with an efficient motion adapter (MA) ; (3) synthesizes realistic video with natural torso movement and switchable background using a head-torso-background super-resolution (HTB-SR) model ; and (4) supports one-shot audio-driven talk
To show the overall performance of Real3D-Portrait , in the following video, we provide demos of 10 unseen identities driven by 6 audio clips from various languages .
Yi Ren (任意) - Homepage
Zhenhui Ye (叶振辉) - Homepage
Weichuang Li