Using ControlNet

Now that we're past all of that install stuff this next part will seem easy.

This guide was made using the version 1.0 ControlNet models, but all the concepts still apply.

Fair warning, ControlNet takes up a lot more VRAM than a regular image generation. You may not be able to get as big of an image as you would get normally.

Start by placing an image into the Starting Image space. Note, when using Controlnet in the Paint Tab the area inside the bounding box is used instead of the starting image.

Open up the ControlNet tab and choose one of the models from the drop-down list. Leave 'Use Preprocess ControlNet' unchecked for now.

You can preview the preprocessed image for the selected model with the preview button. You can also save this preprocessed image to use or edit later

Preview Result

Save for later

This is convenient for making edits to some preprocessed images to add in details, or remove defective parts more easily. These images get saved in a folder called ControlNet in your listed image output folder.

Enter the prompts you want, choose your favorite model, and enter other settings as you would for regular text to image.

Image Variation (from 0.05 to 0.99) will alter how much of the base image is used in the final ControlNet images. Setting variation to 1 will use just the ControlNet preprocessed image.

You can also add on Loras.

Using these ControlNet models will take a bit longer to generate. But eventually it will output an image very similar to the input base image, with some alterations from your prompt and base model.

Experiment trying out all of the different models, and even chain the other ControlNet models by right clicking on the generated image and setting it as the new starting image.