Last year, the arrival of powerful AI models capable of generating original images based on text descriptions amazed people with their uncanny ability to simulate different artistic styles.
But for artists, that wonder was a nightmare, with the easy, startlingly accurate mimicry threatening their livelihood.
Now, a team of University of Chicago computer scientists have built a tool that protects artists from the absorption of their style into these AI models. Called Glaze, the software “cloaks” images so that models incorrectly learn the unique features that define an artist’s style, thwarting subsequent efforts to generate artificial plagiarisms.
The research, developed by the SAND Lab research group led by Neubauer Professors of Computer Science Ben Zhao and Heather Zheng, gives artists a countermeasure against generative art platforms such as DALL-E, Midjourney, and Stable Diffusion, which have exploded in popularity.
“Artists really need this tool; the emotional impact and financial impact of this technology on them is really quite real,” Zhao said. “We talked to teachers who were seeing students drop out of their class because they thought there was no hope for the industry, and professional artists who are seeing their style ripped off left and right.”
In 2020, SAND Lab developed Fawkes, an algorithm for cloaking personal photographs so that they could not be used to train facial recognition models. The research was covered by the New York Times and dozens of international outlets, and the software received nearly one million downloads. So when DALL-E and similar applications broke out last fall, SAND Lab started receiving messages from artists hoping that Fawkes could be used to protect their work.
However, merely adapting Fawkes for artistic images proved insufficient. Faces have a small number of features, such as eye color or nose shape, that models use to make their identifications, and slightly perturbing these features is an effective protection. But an artist’s style can be defined by a large number of characteristics, such as brushstroke, color palette, shadowing, or texture. To effectively cloak an artist’s work, the most important features that make up their unique style would have to be determined.