> For the complete documentation index, see [llms.txt](https://textopia.gitbook.io/textopia.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://textopia.gitbook.io/textopia.ai/introduction/stable-diffusion-for-image-synthesis/concept-of-stable-diffusion.md).

# Concept of Stable Diffusion

**1. Latent Diffusion Process:**

* **Diffusion Probabilistic Model:** We can model the image creation process as a gradual addition of noise. Let x\_0 be the original clean image and <mark style="background-color:blue;">**x\_t**</mark> be the image with noise added at step t (t goes from 0 to T, where T is the total number of steps). The diffusion process defines the probability of transitioning from a noisy image <mark style="background-color:blue;">**(x\_t)**</mark> to a noisier version <mark style="background-color:blue;">**(x\_t-1)**</mark> at the previous step:

<mark style="background-color:blue;">**p(x\_t-1 | x\_t) = eps \* N(mu\_t(x\_t), Σ\_t(x\_t))**</mark>

Here:&#x20;

* eps is a noise level hyperparameter between 0 and 1.
* <mark style="background-color:blue;">**N(mu\_t(x\_t), Σ\_t(x\_t))**</mark> represents a Gaussian noise distribution with mean <mark style="background-color:blue;">**mu\_t(x\_t)**</mark> and standard deviation <mark style="background-color:blue;">**Σ\_t(x\_t)**</mark>. These are typically functions of the current noisy image x\_t, often implemented using neural networks.
* **Reverse Process (Denoising):** Stable Diffusion aims to achieve the opposite – denoising the image starting from a highly noisy version <mark style="background-color:blue;">**(x\_T).**</mark> This is achieved by learning an inverse process <mark style="background-color:blue;">**p(x\_t | x\_t-1)**</mark>. Generative models like U-Nets are used to estimate this conditional probability.

**2. Text Encoding and Conditioning:**

* **Text Encoder:** A separate neural network <mark style="background-color:blue;">**(often a Transformer model)**</mark> takes your text description as input and encodes it into a latent representation, z. This essentially captures the semantic meaning of your words.
* **Conditioning the Denoising Process:** The latent representation z is incorporated into the denoising process to guide the model towards generating an image that aligns with your text description. This can be achieved through various techniques, such as concatenating z with the noisy image representation at each step or using an attention mechanism.

**3. Loss Functions:**

During training, the model learns by comparing the generated image <mark style="background-color:blue;">**(x\_0)**</mark> with real images and their corresponding text descriptions. Here are some common loss functions used:

* **Pixel-wise Loss Functions:** These measure the difference between the generated image pixels <mark style="background-color:blue;">**(x\_0)**</mark> and the corresponding pixels in a real image. Examples include <mark style="background-color:blue;">**Mean Squared Error (MSE)**</mark> or L1 loss.
* **Perceptual Loss Functions:** These losses compare the generated image and real image through pre-trained image recognition models. This encourages the generated image to not only look similar to the real image but also activate similar neurons in a pre-trained convolutional neural network, ensuring semantic similarity.

**4. Optimization:**

The entire model is optimized using gradient descent algorithms. The gradients of the chosen loss function with respect to the model parameters (weights and biases in the U-Net, text encoder, etc.) are calculated and used to update these parameters in a way that minimizes the loss function. This iterative process continues for a large number of training steps, allowing the model to learn how to generate realistic images conditioned on text descriptions.