An Intelligent Algorithm Made A Discovery That Slipped Past Art Historians For Years

Could a computer program influence how we understand art history and the canon? Or, could an artificially intelligent algorithm do the work of art experts for them? One particular researcher project doesn't quite suggest such a reality, but it does demonstrate that machines can highlight subtleties within arts and culture that humans have previously never noticed.

In a paper titled "Toward Automated Discovery Of Artistic Influence" by Babak Saleh and a team of computer science researchers at Rutgers, the academics explained how they used nuanced imaging technology and classification systems to robotize the process of understanding how famous artists have influenced and inspired one another.

For their research, the team chose 1,700 paintings by 66 artists, covering the 15th to the late 20th century. Using a technique that analyzes visual concepts called "classemes"—wherein objects, color shades, subjects' movement, and more are marked—the researchers created a list of 3,000 classemes for each painting, data which The Physics arXiv Blog compares to a vector. Then, they used an artificially intelligent algorithm to evaluate the vectors and look for similarities or overlapping qualities among the 1,700 paintings. ArXiv adds, "To create a ground truth against which to measure their results, they also collate expert opinions on which these artists have influenced the others."

Though it sounds like Pandora's Music Genome Project, the project is not as simple as "if you made this, you must be influenced by that." First, Saleh and his team's program was able to recognize the influence of individual paintings on wider movements—such as the impact of Picasso's Spanish Still Life: Sun And Shadow and George Braque's Man With A Violin (both made in 1912) on the Cubist movement.

Secondly, the machines were able to recognize similarities between paintings that had similar imagery, but very different styles, such as Vincent van Gogh's Old Vineyard With Present Woman (1890) and Joan Miro's The Farm (1922). Also worth adding is that their algorithms identified artistic influences that corroborate with expert opinions, such as Klimt being influenced by Picasso and Braque.

Where the data analysis systems actually take things to a new level is a connection discovered between two paintings that historians had previously never described. Frederic Bazille's Studio 9 Rue de la Condamine (1870) and Norman Rockwell's Shuffleton's Barber Shop (1950) are from different centuries and art movements, but they definitely share visual elements. "After browsing through many publications and websites, we concluded, to the best of our knowledge, that this comparison has not been made by an art historian before," Saleh told arXiv.

So, does this mean machines should be used to fill in the gaps art historians may have missed?

Griselda Pollock, a professor at the University of Leeds, aptly explains why using algorithms are by no means foolproof in an article on The Conversation:

"To study art history, we need to know about economics, politics, literature, philosophy, languages, theologies, ideologies while also studying to understand how art thinks. Art thinks through making, through forms, through materials. And over the past century, art history has been enriched by feminist, post-colonial, queer, and trans-national perspectives. We no longer hunt for connections – we ask questions. We are not diagnosticians seeking for common symptoms. We are not criminologists tracing clues that link a with b.

Even at the most basic level, machines would not be helpful in developing these larger narratives. The idea that machines can see or notice what human beings do not is a fallacy, because the machine is only doing what it is told – and it is the programmers who are setting parameters. But those parameters are based on a woefully old-fashioned and dull misunderstanding of what art historians do, and what they look for."

Saleh and the researchers never claim their system could replace an arts expert, but it is interesting that their process highlights the mathematical aspects of image making, and how more math can be integrated to spotlight them. Ultimately, though, arts analysis is a highly subjective act, even when you have the quantifiable aspects down to an automated science.

via arXiv/Medium

Related:

How Art Experts Discovered A Hidden Portrait Beneath A Picasso Masterpiece

Science Suggests Humans Are Biologically Wired To Like Art

Study Shows That Artists Actually Have Different Brains