An A.I. Curated a Magazine Using Image Recognition Technology

EyeEm is a photography community and marketplace of over 18 million photographers. It also publishes a magazine, also called EyeEm. For its fourth issue, Machina: A Curation of Real Photography by a Machine, the company turned to an artificial intelligence powered by computer vision, EyeEm Vision, to curate the magazine, selecting the photographs it feels are the best aesthetically and most impactful.

Now, before the inner smartphone photographer in you rolls your eyes, understand that it is pretty neat that a machine can, in some ways, learn to identify photographic aesthetics like a human. Sure, an A.I. cannot truly exercise a similar series of complex calculations of why an image might be great or resonant, but it’s certainly intriguing to see where humans are in imbuing machines with mental processes. And that is where the A.I-curated issue of EyeEm is conceptually and aesthetically interesting.

Paul Aguirre-Livingston, EyeEm’s Associate Creative Director, tells The Creators Project that EyeEm had been interested in how A.I. can be used to augment and simplify creative workflow for photographers and editors alike. On a curatorial level, they wanted to understand what makes a photo beautiful or interesting. On a technical level, they wanted to know how to better index photos for discover and concepting as well as editing and buying.

“[It] only felt natural to put our own technology to the test using our 18 million photographers—an amazing talent pool on its own,” says Aguirre-Livingston. “The magazine itself was conceived as a way to further showcase the visual stories within our community, and EyeEm Vision really mirrors that effort. Both projects, in my eyes, have the a similar mandate: To surface the best, the hidden, the unseen.”

EyeEm Vision recognizes thousands of objects and concepts like feelings, moods, and so on. It can also recognize the potential aesthetics of a photograph. To do this, it assigns each photo an aesthetic score from 0 to 100, which Aguirre-Livingston explains more or less “predicts” the “curatorial preference of that photo”—so, whether it’s “good” or “bad.”

“Of course, this score isn’t a definitive assessment and photographers are beautifully sensitive beings,” he explains. “It’s meant as more of a guideline, a starting point for discussion or evaluation.”

“Since the training set of the default version of our aesthetic algorithm is trained by curators inside EyeEm, it has strong rooting in EyeEm, giving high scores to authentic moments in our community photographers' demography,” adds Appu Shaji, Head of EyeEm’s Research & Development, whose sight.io became EyeEm Vision. “For example, there is an emphasis on travel photos being preferred. The original motivation of building personalized aesthetics is rooted in it, since aesthetics is subjective, it is core to understand the user's motive and inner thoughts.”

The A.I.’s curation also depends on the feature. The cover shot, photographed by the surrealist Japanese photographer Horoyoi, was chosen from a pool of users that had created some of the best portrait photos uploaded in the last six months. The A.I. then selected the five best of these 213 uploads before making the final decision.

“We did a larger profile with London-based photographer Lucy Ridgard, who shot a series of portraits of the next generation of teens emerging from her little hometown,” Aguirre-Livingston says. “To find her, we applied a wide net to find the best from "all new users in the last year" based on the average aesthetic score of their entire profile.”

This resulted in another list of five stunning profiles (one of which was, shockingly, an EyeEm intern). Lucy had joined the EyeEm community five months ago to participate in the site’s annual photo competition, and the curatorial team (which didn’t include EyeEm Vision) did not select her as a finalist. But EyeEm Vision found her work.

“To say she was a hidden gem is a true understatement, and her story and creative process is quite fascinating,” says Aguirre-Livingston. “Lucy still doesn’t know how or why we selected her, I’m waiting until she gets our copy or reads this. But, the remaining four photographers not selected for print will be profiled in a series on our blog in the coming weeks.”

For Aguirre-Livingston, an A.I.-curated issue of EyeEm was a chance to examine the intricacies and interactions of the technology from different points-of-view, and with different exercises. Apart from personal views of A.I., beauty and curation, the issue is aimed at understanding if the technology will be helpful to people.

Shaji sees the issue as a humble call for action from the magazine’s readers to envision a bigger photographic tomorrow facilitated by the technology. “[We should] ponder upon the potential negative impact of technology, and how to stay clear of it,” he says. “In short, imagine a better future for the photographic medium together collectively.”

The A Curation of Real Photography by a Machine issue of EyeEm magazine launched on November 3rd.

Related:

MIT's Nightmare Machine Made Its Scariest Image Yet | Insta of the Week

'Westworld' Jumpstarts the Conversations We Need to Be Having About A.I.

This Emotion Detector Wants to Cheer Up Your Web Browsing Experience