Image GPT
OpenAI showed that GPT architecture trained on image pixels (iGPT) generates coherent image completions and samples, with generative quality correlating to supervised classification accuracy.
Excerpt
We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting.
Read at source: https://openai.com/index/image-gpt