Hazarapet Tunanyan
Description: We demonstrate a plug-and-play method which outperforms the latest few-shot personalization alternatives of diffusion models.
nerf (138) d-nerf (47) nerfies (46)
We present Specialist Diffusion , a style specific personalized text-to-image model. It is plug-and-play to existing diffusion models and other personalization techniques. It outperform the latest few-shot personalization alternatives of diffusion models such as Textual Inversion and DreamBooth , in terms of learning highly sophisticated styles with ultra-sample-efficient tuning.
Samples generated by our models fine-tuned on the three datasets. The left column shows the dataset on which the model is trained on, and the top row shows the text prompt used to generate the image.
Comparison of fine-tuning the Stable Diffusion model. Three rows represent three different, rare styles models personalization, using only a handful of samples ( even less than 10 ). All examples are generated using the same text prompt. Object specific DreamBooth performs poorly when being applied to capturing styles. Textual Inversion achieves neat performance on some styles, but fails on more unusual styles such as “Flat design” . Specialist Diffusion (rightmost) succeeds to capture those highly unusual,
Hazarapet Tunanyan