Computers can Draw (draft)

January 1, 2017

Survey on the attempts to build an AI artist

History

• AARON (Harold Cohen)
• “The Painting Fool” (Simon Colton)
• “The Next Rembrandt” (TU Delft)
• Machine learning

Modern advances

Non Photo-realistic Rendering (NPR)

• physic-based: NPR book
• data-driven: texture/style transfer, e.g. Artomatix (Erric Risser et al), Neural Art (Gatys et al)

Generative model of images

• Restricted Boltzman Machine (RBM)
• Variational Auto Encoder (VAE), Deep Recurrent Attentive Writer (DRAW)
• Generative Adversarial Network (GAN)
• PixelRNN
• PPGN: Generative net + conditioned + activation maximisation + prior structure = photo-realistic image generation!! (inner-thought: 18-month ago this vague idea of “creative AI” drove me to machine/deep learning, and here we are!!)

Applications in graphics

• iGAN (Jun-Yan Zhu et al)
• StyLit (Jakub Fišer et al)
• Colorisation (various) (note-to-self: r/colorization might be a good place to crawl labelled data)
• Texture propagation (Satoshi Iizuaka et al)
• Sketch cleaning (Satoshi Iizuaka et al)
• Super-resolution
• …