An overview and list of free resources to help you get started.
Easiest way to play with these models is to sign up for a Google Colab account.
Couple ways to generate an image from text
- Inference from a model trained for this task
- Guiding an image representation with CLIP
All the generated images below are from prompt
Winds of Winter
Very quick and easy to generate lots of images. It's recommended to generate 100 or so images using model as generation is quite cheap and later rank them using CLIP to get a good dozen generations.
One of the best text to image model available right now.
Pretty good text to image model. Only Older cogview model has been released publicly. You can try newer model which is faster and better only at online demo.
Online Demo: https://agc.platform.baai.ac.cn/CogView/index.html
They have trained 2 models, XL and XXL. Only XL has been released so far which has 1.3 billion params. Pretty amazing results, similar to cogview 2.
They also released Ru-CLIP.
Clip Conditioned Decision Transformer
Guiding Image Representation with CLIP
It's more like model training rather than model inference. As expected, it takes a while to generate an image.
Image representation can be as simple as RGB array, set of bezier curves to latent representations of various AEs/GANs like VQGAN, BigGAN, StyleGAN, OpenAI dVAE.
This doesn't need much GPU VRAM, works really well on just 4GB of it. Good for getting high resolution outputs.
Vector Strokes Optimization
CLIPDraw by Kevin Frans
OpenAI dVAE Optimization
Colab of Implementation by Rivers Have Wings: https://colab.research.google.com/drive/10DzGECHlEnL4oeqsN-FWCkIe_sq3wVqt
Rivers Have Wings coded up an implementation which connected VQGAN with CLIP. Almost every other VQGAN CLIP notebook is derived from it. Most derivatives have different learning rate strategies, image augmentations, optimizers and generally results are much better.
Much better global coherency compared to CLIP guided GANs. Generation may not follow prompt as well as CLIP guided GAN's do.
Colab Implementation by nshepperd: https://colab.research.google.com/drive/10dvDxcS4e4anlwpJE2yLBjq0O1vKqxdn
We have an app which provides an easy way to play with text to image algorithms. If you would like to give it a go, sign up at https://snowpixel.app/ and send us your sign up email at firstname.lastname@example.org. We'll add couple free credits to your account to try it out.