ControlNET is the next step in AI that allows for additional control in diffusion models (like Stable Diffusion).
For example, ControlNET enables you to add a text prompt to Stable Diffusion and create a link between img+text to img.
Paper: https://arxiv.org/abs/2302.05543
Code: https://github.com/lllyasviel/ControlNet
Sandbox: https://huggingface.co/spaces/hysts/ControlNet
P.S. By the way, you’ve probably seen videos where scenes change frame by frame; they are made using this technology.
