Rethinking Classifier-Free Guidance for Diffusion Models: Alternative Approaches and Strategies for Conditional and Unconditional Generative Setups

In the world of generative models, Classifier-Free Guidance (CFG) has been a hot topic of discussion lately. With diffusion models gaining popularity, researchers have been exploring alternative approaches to CFG, especially for situations where conditioning dropout is not feasible. This follow-up blog post delves into various methods for implementing CFG in purely unconditional generative setups.

One of the key challenges faced in CFG is the inability to apply it when conditioning dropout is not possible. To address this limitation, recent works have introduced alternative approaches such as using impaired or inferior models as substitutes for the unconditional model. These inferior models can be either conditional or unconditional and are designed to have some bottleneck compared to the conditional model.

To avoid confusion between conditional and unconditional models, researchers have introduced the concept of positive and negative models in CFG. The positive model typically refers to a regular diffusion model, while the negative model is a modified version of the positive model.

One approach to implementing CFG in unconditional generative setups is Self-Attention Guidance (SAG). SAG leverages self-attention maps to modify the predictions of the negative model, allowing for guidance in both conditional and unconditional models.

Another approach, Attention-based self-guidance: perturbed self-attention (PAG), involves impairing the attention module in the UNet to create a negative term for the CFG equation. By perturbing the self-attention matrices, PAG aims to guide the models in a training-free manner.

Autoguidance takes a different approach by using an inferior version of the denoiser model as the negative/guiding model. By limiting the model’s capacity or training time, Autoguidance enables CFG-like guidance for unconditional image synthesis.

Independent condition guidance (ICG) is another training-free solution that involves sampling a random condition as a negative for conditional models. This approach provides an alternative for models that have not been trained with conditional dropout.

Smoothed Energy Guidance (SEG) manipulates self-attention blocks via blurring to create a negative model from the base model. This method is tuning- and condition-free and only requires the tuning of the standard deviation parameter (sigma).

In conclusion, while there is no one-size-fits-all solution for replacing vanilla CFG, the methods discussed in this article offer interesting alternatives for implementing guidance in diffusion models. Further research and experimentation are needed to explore the full potential of these approaches in the field of generative modeling. If you found this blog post helpful, consider sharing it on social media or donating to help us reach a broader audience.