How to get intermeidate output images

🧨 Diffusers
1

Is it possible to get the images at each denoising step via the Diffusers library? I am sure I’ve seen it done but can’t find where or how.

2

Hi @dkackman!

You might want to look at the callback mechanism, which sends intermediate latents to a function you specify. You could then decode the latents in that function and visualize them as you need.

This notebook includes a section about callbacks that demonstrates how to use that feature.

Good luck!

2 Likes
3

Oh perfect. I was unclear on how to transform the latents into an image but this exactly what iI was looking for.

vae = pipe.vae
images = []

def latents_callback(i, t, latents):
    latents = 1 / 0.18215 * latents
    image = vae.decode(latents).sample[0]
    image = (image / 2 + 0.5).clamp(0, 1)
    image = image.cpu().permute(1, 2, 0).numpy()
    images.extend(pipe.numpy_to_pil(image))

prompt = "Portrait painting of Jeremy Howard looking happy."
torch.manual_seed(9000)
final_image = pipe(prompt, callback=latents_callback, callback_steps=12).images[0]
images.append(final_image)
image_grid(images, rows=1, cols=len(images))
1 Like