The forward process

One step, then many

The forward process is a Markov chain with one rule. A Markov chain is a sequence of steps where the next state depends only on the current one. At each step, scale the current sample by the square root of one minus beta, then add Gaussian noise with variance beta. The scaling is what keeps the variance from growing. Without it, the sample would inflate a little at every step and end far from a standard normal. With it, a unit-variance input stays unit-variance throughout. That is the variance-preserving form used by almost every image model.

Beta is small, typically between one in ten thousand and two hundredths, so a single step is barely visible. That is the point. The chain destroys structure little by little, and each transition is close enough to invertible that a network can learn the inverse. The forward process has no parameters at all. It is a fixed recipe chosen before training, and it is the same for every image.

The closed form

You never have to run the chain to get a noisy sample. Because Gaussian noise adds up cleanly, the marginal at step t has a closed form (an exact formula you can evaluate in one go). The marginal is the distribution at that step, on its own. x_t equals the square root of alpha_bar times x_0, plus the square root of one minus alpha_bar times a fresh noise sample. Alpha_bar is the product of every one minus beta up to t. One draw, any timestep, no loop. Training uses this constantly, because it lets a batch sample a different t for every example at no cost.

The two coefficients are a trade. Alpha_bar is the share of the original signal that survives. One minus alpha_bar is the share that is noise. Their squares sum to one, so the variance is preserved exactly. At t equal to zero, alpha_bar is essentially one and the sample is clean. At the final step it is near zero and the sample is almost pure noise. Everything between is a blend, and the blend is what the model learns to separate.

Build the cumulative table

import torch
from diffusers import DDPMScheduler

scheduler = DDPMScheduler(num_train_timesteps=1000, beta_schedule="linear")
betas = scheduler.betas
alphas = 1.0 - betas
alphas_cumprod = torch.cumprod(alphas, dim=0)

print("first beta :", betas[0].item())
print("last beta  :", betas[-1].item())
print("alpha_bar at 0   :", alphas_cumprod[0].item())
print("alpha_bar at 999 :", alphas_cumprod[-1].item())

Signal to noise

The ratio alpha_bar over one minus alpha_bar is the signal-to-noise ratio, and it is the most useful single number about a schedule. It falls steadily from a very large value to almost zero. When it crosses one, the sample holds equal parts signal and noise. Below that, the noise dominates. Most of the interesting denoising happens in a narrow band around that crossing. Schedules and samplers spend a lot of effort on where it sits for that reason.

Alpha_bar is cumulative, not per step. Confusing the two is the most common off-by-one in a diffusion implementation. A beta of 0.02 does not mean two percent of the signal is gone at that step. It means two percent of the current variance is replaced, and the effect compounds. Print the curve before you trust a schedule. Check the last value too: a schedule that ends at alpha_bar of 0.01 has not quite reached pure noise, and a sampler that assumes it has will leave structure behind.

The forward process is not the model

It is tempting to think the forward process is something the model learns. It is not. It is a fixed corruption, chosen by you, and the model only ever learns its inverse. That separation is what makes the maths tractable. The training distribution at every timestep is known exactly, so the loss can be computed in closed form. It also means a checkpoint is tied to the schedule it was trained on. Change the betas and the same weights are denoising a different problem.

In practice you precompute alpha_bar once, store it as a tensor, and index it by t. The chain itself is only used when you want to visualise the corruption or check an implementation. Everything else reads the cumulative table: training, and the closed-form sampling updates. Keep that table in the checkpoint metadata. It is part of the model in every way that matters.