The Art of Predicting Tastes in Art

BY WHITNEY CLAVIN

The thick brush strokes and soft color palettes of a Monet. The bold colors and abstract shapes of a Rothko. A new Caltech study, which appeared in the journal Nature Human Behaviour in May, shows that a simple computer program can accurately predict which paintings a person will like.

The study enlisted more than 1,500 volunteers through Amazon’s crowdsourcing platform Mechanical Turk to rate paintings in the genres of impressionism, cubism, abstract, and color field. The volunteers’ answers were fed into a computer program which, after this training period, could predict the volunteers’ art preferences much better than would happen by chance.

Area Broken by Perpendiculars, Joseph Schillinger, 1934. An example of abstract art.

“I used to think the evaluation of art was personal and subjective, so I was surprised by this result,” says lead author Kiyohito Iigaya, a postdoctoral scholar in the laboratory of Fletcher Jones Professor of Psychology John O’Doherty, an affiliated member of the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech.

“The main point is that we are gaining an insight into the mechanism that people use to make aesthetic judgments,” says O’Doherty.

In the study, the team programmed the computer to break a painting’s visual attributes down into what they called low-level features—traits like contrast, saturation, and hue—as well as high-level features, which require human judgment and include traits such as whether the painting is dynamic or still.

“The computer program then estimates how much a specific feature is taken into account when making a decision about how much to like a particular piece of art,” explains Iigaya. “Once the computer has estimated that, then it can successfully predict a person’s liking for another previously unseen piece of art.”

The researchers discovered that the volunteers tended to cluster into three general categories: those who like paintings with real-life objects, such as an impressionist painting; those who like colorful abstract paintings, such as a Rothko; and those who like complex paintings, such as Picasso’s cubist portraits. The majority of people fell into the “real-life object” category.

Seconnet Point, Rhode Island, Worthington Whittredge,1880

The researchers found that they could also train a deep convolutional neural network (DCNN), a type of machine-learning program, to how to predict the volunteer’s art preferences with a similar level of accuracy. In this case, the deep-learning approach did not include any of the selected low- or high-level visual features used in the first part of the study, so the computer had to “decide” what features to analyze on its own.

“In deep-neural-network models, we do not actually know exactly how the network is solving a particular task because the models learn by themselves much like real brains do,” explains Iigaya.

In another part of the study, the researchers demonstrated that their simple computer program, which had already been trained on art preferences, could accurately predict which photos volunteers would like.

The study, “Aesthetic preference for art can be predicted from a mixture of low- and high-level visual features,” was funded by the National Institute of Mental Health (through Caltech’s Conte Center for the Neurobiology of Social Decision Making), the National Institute on Drug Abuse, the Japan Society for Promotion of Science, the Swartz Foundation, the Suntory Foundation, and the William H. and Helen Lang Summer Undergraduate Research Fellowship. Other Caltech authors include Sanghyun Yi, Iman A. Wahle (BS ’20), and Koranis Tanwisuth, who is now a graduate student at UC Berkeley.