Using ControlNet

This guide was made using the version 1.0 ControlNet models, but all the concepts still apply.

Fair warning, ControlNet takes up a lot more VRAM than a regular image generation. You may not be able to get as big of an image as you would get normally.

Start by placing an image into the Starting Image space.

Open up the ControlNet tab and choose one of the models from the drop-down list. Leave 'Use Preprocess ControlNet' unchecked for now.

You can preview the preprocessed image for the selected model with the preview button. You can also save this preprocessed image to use or edit later

Preview Result

Save for later

This is convenient for making edits to some preprocessed images to add in details, or remove defective parts more easily. These images get saved in a folder called ControlNet in your listed image output folder.

Enter the prompts you want, choose your favorite model, and enter other settings as you would for regular text to image. Image Variation will currently have no effect on ControlNet images. You can add on Loras though.

Using these ControlNet models will take a bit longer to generate. But eventually it will output an image very similar to the input base image, with some alterations from your prompt and base model.

Experiment trying out all of the different models, and even chain the other ControlNet models by right clicking on the generated image and setting it as the new starting image.