SDXL: How to use Stable Diffusion

Recently, a WordPress plug-in was developed based on Stable Diffusion XL (SDXL). To summarize by the way.

1. What is Stable Diffusion?

Stable Diffusion is an open source large model for image generation.

The currently mainly used version is Stable Diffusion XL, also known as SDXL.

Official website:Stability AI

2. Is Stable Diffusion free?

Stable Diffusion and other large models of Stability AI (including video, voice, 3D) are free and open source. You can deploy the code directly to your own computer or server from the official website.

However, most of the time, we will not deploy it ourselves because the cost is relatively high and the difficulty is relatively high. Therefore, more often than not, when developing a product based on Stable Diffusion, we need to call the official website or the third-party Stable Diffusion API, but in this case, we need to pay the interface usage fee.

3. How to deploy Stable Diffusion? (generated by Google Gemini)

Deploy the Stable Diffusion model

Way 1: Use a pre-built Docker image

1. Pull the Docker image:
   • docker pull stabilityai/stable-diffusion:latest
2. Create and start the container:
   • docker run -it --gpus all stabilityai/stable-diffusion:latest

Method 2: Manual deployment

1. Install dependencies:
   • Python 3.10.x
   • PyTorch 1.13.0+
   • torchvision 0.14.0+
   • ax-platform 0.3.11
   • transformers 4.28.0+
   • accelerate 0.9.1+
   • kornia 0.6.2+
   • einops 0.4.1+
   • huggingface_hub 0.10.0+
2. Clone the Stable Diffusion repository:
   • git clone https://github.com/huggingface/diffusers
3. Install the Stable Diffusion library:
   • cd diffusers
   • pip install .
4. Download model weights:
   • wget https://huggingface.co/CompVis/stable-diffusion-v1-5/resolve/main/sd-v1-5.ckpt
5. Deployment model:
   • Create a Python script that loads the model and sets inputs for it.
   • use diffusers.pipelines.stable_diffusion.StableDiffusionPipeline Class runs image generation.

Instructions for use

After deploying the Stable Diffusion model through the above steps, you can use the following command to generate the image:

from diffusers import StableDiffusionPipeline # Load model pipeline = StableDiffusionPipeline.from_pretrained("YOUR_MODEL_PATH") # Run image generation image = pipeline("YOUR_PROMPT")

hint

• use --gpus all Flag to utilize all available GPUs.
• Adjustment batch_size and num_outputs Parameters to control the quantity and quality of generated images.
• use save_image() Method saves the image to local disk.

4. How to use Stable Diffusion API?

https://replicate.com/

You can easily deploy your own open source large models on this website. You can also use the API of large models deployed by others.

1. Log in to this website using your GitHub account

2. Enter the dashboard

3. Select an image generation model

https://replicate.com/collections/text-to-image

All recommended image generation models are listed here.

4. Using the model’s API

There are multiple models of stable diffusion and sdxl, just choose one. We can choose the first one. Click in and find its http API page:

stability-ai/stable-diffusion – Run with an API on Replicate

Here is the API and calling code example of this model.

You can also use packaged js, python and other SDKs.

5. Summary

Stable Diffusion is free, open source, and universal. SDXL is also the most widely used large image generation model at present. It can help you implement various image generation AIGC products.