[Exclusive] Robots And Choreography Abound In Update To Ballet Masterpiece

Director Tarik Abdel-Gawad creates a stunning choreographic masterpiece around Tchaikovsky's "Francesca da Rimini" using a giant robotic camera.

by Emerson Rosenthal

July 9, 2014, 5:20pm

Images by Alexander Reneff-Olson

Star-crossed lovers. Immaculate dance moves. Giant robots. If it sounds like the plot of the newest Guillermo del Toro movie, you wouldn't be too far from the truth. The reality, however, might be even more exciting:

Tarik Abdel-Gawad, creative and technical director of the award-winning projection-mapping spectacular, Box, has finally released his pre-Box masterpiece, a robot-aided update of Tchaikovsky's ballet fantasia, Francesca da Rimini, a piece as revered for its heartwrenching subject matter as for its incredibly precise choreography. With choreography from Yuri Possokhov, and starring San Francisco Ballet dancers Maria Kochetkova and Joan Boada, Abdel-Gawad took the piece one step further, employing a massive, robot-controlled camera to capture the deftness of the dance.

Says Abdel-Gawad, “The film itself brings the viewer closer to a ballet performance than is possible on a stage. Using a robot allows the camera to be choreographed as well as the dancers, achieving spectacular shots designed specifically for this performance. The end result is a film that makes viewers feel they’re in the room dancing with the performers.”

The Creators Project spoke to Tarik Abdel-Gawad about his newest release, Francesca da Rimini, and what it was like to balance a choreographed performance between humans and robots. Below, Francesca da Rimini and a stellar behind-the-scenes featurette.

The Creators Project: You previously used robots to help you capture the projection-mapped surfaces in Box— from lighting to programming to the blocking, how was this process different?

Tarik Abdel-Gawad: Box was about creating a digital world and bringing it to life in a physical space. With Francesca we started with real choreography, captured it digitally and animated camera motion in 3D. Then we united the digital and the physical by shooting the real performance with a motion control camera.

The robot never appears on screen, it is used as a tool for camera control, but also as part of an experiment with the dancers. Entering the project I did not know how well we could synchronize the dancers and robot, how the dancers would perform in close proximity to industrial machinery, or whether the dancers could reproduce the same choreography over and over again. A lot of these questions and their answers are captured in the behind the scenes video. In the end, with enough attempts and extreme resilience on the part of Masha and Joan, it proved a success.

Was it more difficult to "humanize" the movement of the robotic camera? Or to update the ballet for this highly-technologized format?

The camera motion was designed to move in rhythm with the choreography, following the dancers like another performer that counters and amplifies their movement. We didn’t want to change the choreography to accommodate the tech. The challenge was to perform it with an intimidating robotic partner. It was more difficult for the dancers to adapt to this environment than it was for the robot.

What were some of the difficulties you encountered? i.e. How far did you get using solely motion capture, and how much had to be adjusted on-set?

The motion capture process had to be repeated. The partnering in the choreography required enormous strength and involved a great deal of contact and lifting. One of the motion capture technicians actually said that the dance was more “violent” than the fight scenes they are more accustomed to capturing. Using an active marker system helped us get a clean capture.

Throughout the project there was a lot of communication with the dancers about the process but nothing could prepare them for the actual experience. I think only after they completed the shoot did the significance of each step become clear. The dancers’ job was the most difficult. The choreography is highly technical and challenging even in normal circumstances. Dancing in perfect sync with a robot and a previously captured performance took the challenge to another level.

Ballet and robotics are both precise, practiced, mechanical performances. Were there any "magic," spontaneous, or surprising moments?

Just seeing it come together feels magical. Masha (Maria) is a perfectionist. When we finally got the shot - and there were many takes - it was as if she had defeated the robot.

Your last film, Box, earned you a place in the Saatchi & Saatchi New Directors Showcase. Did this achievement help you in pulling Francesca da Rimini together, and given you access to further projects?

Francesca was shot prior to Box and was a very different type of project. It began as an experimental collaboration with a friend, Maria Kochetkova, shot over a single weekend with a small crew. It’s only being released now because projects like Box require so much time and focus.

It took a while to come back to the project but I felt it was necessary to share for the performance alone. Despite the scale of the technology involved, it was always intended to blend in with the choreography, and not overshadow the dancers. We wanted to capture the rehearsal atmosphere, where the focus is on perfecting motion apart from stagecraft and theatrics. Our goal was to use technology to bring the viewer into the choreography itself and see the classical art form from new angles.

You describe this project as an experiment. How do you plan to apply what you learned here to your next projects?

The project demonstrated that it was possible to synchronize robotic motion with extremely complex athletic choreography. I started with a camera, but in the future similar techniques could be applied to an entire stage.

Below, we were lucky enough to snag some awesome snapshots of Francesca da Rimini in action, and from behind-the-scenes. Images courtesy of Avantgarde, taken by Alexander Reneff-Olson.