Style2Paints: Transferring Artistic Styles to Oil Paintings Using Text Prompts
Abstract
Style2Paints is a generative model that transfers artistic styles to oil paintings using text prompts. It was developed by researchers at the Computer Graphics and Visualization chair of the Technical University of Munich, Germany. The researchers trained the model on 44,000 oil paintings from the WikiArt dataset, which contains artworks from the 18th to the 21st century.
Model Architecture
The Style2Paints model architecture is based on StyleGAN2, a generative adversarial network (GAN) that can generate realistic images. StyleGAN2 consists of two neural networks: a generator and a discriminator. The generator generates images from a random noise vector, and the discriminator distinguishes between generated and real images.
The Style2Paints model extends the StyleGAN2 architecture with a text encoder. The text encoder converts a text prompt into a vector, which is then fed into the generator together with the random noise vector. The generator uses the text embedding and the random noise vector to generate an image.
Results
The Style2Paints model is able to generate realistic oil paintings that exhibit the artistic style described in the text prompt. For example, if the user inputs “sunflower painting in the style of Van Gogh”, the model will generate an oil painting of sunflowers in the style of Van Gogh.
The Style2Paints model is also able to transfer the sentiment and emotion conveyed in the text prompt to the oil painting. For example, if the user inputs “sad landscape painting”, the model will generate an oil painting of a landscape that conveys a sense of sadness.
Applications
The Style2Paints model can be used for a variety of applications, including:
- Art creation: Artists can use the Style2Paints model to create oil paintings in different artistic styles.
- Education: Students can use the Style2Paints model to learn about different artistic styles.
- Entertainment: People can use the Style2Paints model to create fun and interesting images.
Conclusion
The Style2Paints model is a generative model that transfers artistic styles to oil paintings using text prompts. The model is able to generate realistic oil paintings that exhibit the artistic style and sentiment described in the text prompt. The Style2Paints model can be used for a variety of applications, including art creation, education, and entertainment.
