FYI.

This story is over 5 years old.

Entertainment

Guest Column: The Magical Magic Of Augmented Reality

Artist and researcher Helen Papagiannis takes a look at the magic of early cinema and how it relates to the developing field of AR.

"That's for me, what a great trick!" This was the phrase French filmmaker Georges Méliès is said to have exclaimed upon seeing cinema for the first time at a preview of the Lumière brothers' invention in 1895. I too felt this exact way upon first encountering Augmented Reality (AR).

Méliès, often referred to as the 'father of special effects,’ became famous for the trick-film, utilizing a stop-motion and substitution technique through the splicing of film and multiple exposures. It is important to note that before he became a filmmaker, Méliès was a magician, bringing many of his tricks from the stage to the new medium of cinema.

Advertisement

Méliès is an important figure for AR's prehistory and its future, with lessons to be learned to advance AR in creating new conventions and exploring content alongside technology.

Unlike the Lumières' approach to cinema, which was rooted in 'documenting everyday actualities,’ Méliès was interested in the fantastical illusions cinema could give form to. He artfully utilized the stop-camera technique to create magical effects, producing appearances, disappearances and sleight-of-hand-tricks.

Méliès: Les Cartes Vivantes (1905)

Méliès' film Les Cartes Vivantes, or The Living Cards (1904), is included in my FogScreen and RFID installation, The Amazing Cinemagician (a homage to Méliès). In the film, Méliès appears as a magician performing a card trick; however, this magic trick is far from ordinary. In addition to objects shifting forms, appearing and disappearing, the inanimate becomes animate, with the figures on the cards coming to life. For example, with the use of stop-substitution, a live woman dressed in costume emerges from the Queen of Hearts card. The trick continues with the King of Diamonds, using the same effect. Méliès uses the newfound medium of cinema to extend magic into novel, seemingly impossible visualities.

AR, too, is about creating impossible visualities. In some ways, we can consider AR to be a real-time stop-substitution that layers content dynamically atop the physical environment and creates virtual actualities with objects—shifting forms, appearing and disappearing—as Méliès first did in cinema.

Advertisement

Helen Papagiannis: The Amazing Cinemagician

I liken AR, at this time, to cinema in its infancy, when there were as yet no conventions. AR, like cinema when it first emerged, has commenced with a focus on the technology with little consideration to content. While Méliès' contemporaries were focused on exploring the technical possibilities of the emerging medium, Méliès was arguably the first to use it as a means for personal expression and give precedence to substantial content.

As I’ve stated in the past, AR content needs to catch up with AR technology. As a community of designers, artists, researchers and commercial industry, we need to advance content in AR and not stop with the technology, but look at what unique stories and utility AR can present.

Méliès serves as an inspiration to my practice in AR in that the technology of film inspired stories and the direction of the work, but Méliès also gave himself the freedom of experimentation and creativity to move beyond the constraints and invent new ways of applying the technology, thereby establishing new conventions and stylistic attributes for the medium of film. This is the path we must heed with AR.

Magic and AR go together like two peas in a pod. There’s certainly something entrancing and extraordinary about the technology, particularly when experienced for the first time; a feeling of amazement and wonderment arises, while marveling at the illusion which is presented.

Advertisement

Helen Papagiannis at TEDxYorkU 2010: How Does Wonderment Guide the Creative Process?

The term Magic appears quite often in the lexicon of AR, which is a personal project I am working on to identify and build a language or vocabulary for AR as the medium evolves. Another term is Enchantment, which arose in a Twitter conversation between myself (@ARstories), @JohnCHavens and @TrakLord around the discussion of another name for Augmented Reality to better describe the medium. Other terms that were explored were Enhanced, Entrancing and Expanding Reality.

But getting back to the term Magic: we see this in the name of AR applications such as the HIT Lab NZ’s MagicBook (2001) and even in the name of an AR research lab, the Magic Vision Lab at University of South Australia. I’ve had the pleasure of touring both labs and experiencing their magical projects first hand. I certainly had an entrancing experience while visiting the Magic Vision Lab, experimenting with haptics and AR.

Above: A still from the Magic Vision Lab’s haptics AR demo.

Donning a head-mounted display and using a haptic stylus called Phantom, I was able to touch virtual objects and receive tactile feedback, as though these were real, physical objects I was interacting with. This completely threw off my sensibilities of the real, having difficulty distinguishing between what was real and what was virtual. This experience signified an important shift for me in the medium of AR: in the past, the only tactile component of AR was that which physically existed in our environment. Now being able to touch and feel virtual objects in AR, we have entirely new possibilities to further engage our tactile senses within this medium.

Advertisement

On the topic of the tactile, paper engineering and pop-up books have been a great love of mine. My recent AR pop-up book, Who’s Afraid of Bugs? uses the iPad 2 and iPhone 4 to seek and explore virtual creepy crawlies that you can interact with. With the possibilities of haptics in AR, perhaps you’ll be able to not only see these critters crawling over your hand, but actually feel them too. Now wouldn't that be magical?

Helen Papagiannis: Who’s Afraid of Bugs? (2011)

Magic can be associated with gimmickry, but it need not be. As AR evolves into a new medium and finds its footing, moving beyond gimmickry into utility and storytelling, I only hope that we don’t abandon magic and continue to remain enchanted by the magical magic of AR.

The tile of this article is in play with: Gaudreault, André. "Méliès the Magician: The Magical Magic of the Magic Image." Early Popular Visual Culture 5.2 (2007): 167-174.

Helen Papagiannis is a designer, artist, and PhD researcher specializing in Augmented Reality (AR) in Toronto, Canada. Helen has been working with AR since 2005, exploring the creative possibilities for AR with a focus on content development and storytelling. She is a Senior Research Associate at the Augmented Reality Lab at York University, in the Department of Film, Faculty of Fine Arts. Helen has presented her interactive artwork and research at global juried conferences and events including TEDx (Technology, Entertainment, Design), ISMAR (International Society for Mixed and Augmented Reality) and ISEA (International Symposium for Electronic Art). Prior to her augmented life, Helen was a member of the internationally renowned Bruce Mau Design studio where she was project lead on "Massive Change: The Future of Global Design." Read more about Helen's work on her blog and follow her on Twitter: @ARstories.