Introduction
- A Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models.Various image synthesis with our proposed IP-Adapter applied on the pretrained text-to-image diffusion model and additional structure controller.
- The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt. The proposed IP-Adapter consists of two parts: a image encoder to extract image features from image prompt, and adapted modules with decoupled cross-attention to embed image features into the pretrained text-to-image diffusion model.
Features
Generalizable to Custom Models
Once the IP-Adapter-FaceID is trained, it can be directly reusable on custom models fine-tuned from the same base model.
Structure Control
The IP-Adapter-FaceID is fully compatible with existing controllable tools, e.g., ControlNet and T2I-Adapter.
Better align with the reference image
Our method not only outperforms other methods in terms of image quality, but also produces images that better align with the reference image.
Image-to-Image and Inpainting
Image-guided image-to-image and inpainting can be also achieved by simply replacing text prompt with image prompt.
Multimodal Prompt
Due to the decoupled cross-attention strategy, image prompt can work together with text prompt to realize multimodal image generation.
More superior results
Compared with other existing methods, our method can generate superior results in both image quality and alignment with multimodal prompts.
More controllable
You can adjust the weight of the face structure to get different generation!
How to use IP-Adapter-FaceID
- You only need to upload your photo and enter the desired scene description to generate your photo in the specified scene.
- Upload several of your photos for a result that better matches your appearance
- You can also try the advanced configuration for more controllable outcomes
Can be used in Comfy UI，SD
- use IP-Adapter-FaceID through ComfyUI IPAdapter plus in comfyui.
- and the Contorlnet plugin for SD Web UI (ip-adapter-faceid_sd15) is also supported
Login with Google