Jindong Gu
Description: Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
safety (4257) transparency (204) text-to-image (28) diffusion models (23) responsible ai (14) explainability (7) multimodal generative models (2)
Diffusion-based models have gained significant popularity for text-to-image generation due to their exceptional image-generation capabilities. A risk with these models is the potential generation of inappropriate content, such as biased or harmful images. However, the underlying reasons for generating such undesired content from the perspective of the diffusion model's internal representation remain unclear. Previous work interprets vectors in an interpretable latent space of diffusion models as semantic co
The quality of a caption can be evaluated by assessing the image reconstruction produced by a text-to-image model. The best text for an image is one that leads to the most accurate reconstruction of the original image
Method : The generated image is compared with the input image using a similarity function based on CLIP image embeddings. Human-annotated captions serve as ground truth representations of the input image to evaluate the quality of the generated caption.
Jindong Gu