Meet The Man Behind The Fantastical Technology In The Avengers, Rise Of The Planet Of The Apes, 2012 And More [Q&A]

The movies will always be the world’s favorite form of escapism, and the further they take us from reality, the better. There was a time when films depicted a distant technological future, a world capable of feats the viewers would never see actualized in their lifetimes, but as the development of new technology continues to accelerate, we now end up using tools that were quite literally first conceived in the realm of science fiction. The touchscreen, 3D printing, and robot assistance—once unlikely dreams depicted in sci-fi novels and films—are all en route to ubiquity in real life.

As this turnover from fictional to real quickens, films continue to challenge our conventions of what’s possible, though now we view them with the knowledge that these fantastical creations may in fact be sitting on our desks sooner rather than later. Our graphical user interfaces, to take but one example, have gone from green text on a static black screen with a single point of control to the touchscreen tablet computer in a mere three decades. So, what’s next?

Enter Jayse Hansen, fictional graphic user interface designer extraordinaire. He created the interface Tony Stark uses to control his Iron Man suit in The Avengers, the scientific visualizations in Rise of the Planet of the Apes, as well as dynamic diagrams of the Ark in 2012.

What Hansen adds to films is not only the element of futurism in the depiction of the tools themselves, but also a vision for our sense of connection with them. In these films, the humans using his fictional technology do so with understanding and fluency, something we could hardly imagine having with a touchscreen before we ended up carrying them around in our pockets. As daily users of technology, we crave this type of prescient prediction. Perhaps that’s why The Avengers just became the third-highest grossing film of all time.

We spoke to Jayse about the thought process behind his work, some fun behind-the-scenes knowledge of how it’s integrated into film, and how exactly someone ends up being a fictional GUI designer.

The Creators Project: You’ve got a pretty distinct style that fits with pop culture’s and the technology industry’s perceptions of a futuristic look.
Jayse Hansen: Thanks. That may actually just be because I'm kind of a geek with all things tech.

What interfaces, real or fictional, inspired this look? Dare I say Terminator 2?
Possibly. Terminator 2 was awesome. James Cameron is one of my favorite directors actually—love his attention to detail while keeping the big picture in focus at the same time. When I was a kid, I remember loving the red “machine-vision” when he scanned people's clothing to determine if it would fit him or not. I haven't seen that in a while, but I'm sure that's been gestating like some gooey alien in the back of my head somewhere.

As far as inspirations, well, I'm definitely addicted to all interfaces in films—good and bad. I wanted to make my own little contribution, so I got into this field. For the glass screens in The Avengers’ Helicarrier, I referenced real screens on aircraft carriers, but also experimental things being done with real glass screen displays.

Production wanted the Helicarrier interfaces to be high-tech, but very hard and militaristic, and a step away from the sleekness of the Stark interfaces, which we also featured in the Science Lab when Tony is breaking into S.H.I.E.L.D.'s computers.

Looking outside of the film world is sometimes the most inspiring, so we also referenced medical optical equipment, steampunk props, precision scopes and combat aviation systems.

Venti Hristova (Cantina's VFX supervisor) found a great artist named Tatiana Plakhova, and she was a huge inspiration. What she does with lines and dots is simply stunning. For structure and data visualization, a lot of my design inspiration came from things being done currently in network mapping, infographics and generative art. It's a great time to be inspired right now. So much work out there is gorgeous.

Were you a fan of comic books or sci-fi as a kid? Is it a trip for you to be designing visuals for Marvel super hero movies?
Of course! It's a bit of a dream job, for sure. It's definitely very fun to “have to” go to a comic book store to do your research. Such a chore, right? I've always been a fan of these characters, so working on these films and adding my own personal design flair into their universe is quite an honor and such a privilege.

How did you get started with designing fictional interfaces? What was the first film or commercial project you worked on?
An awesome screen-design company called G-Creative took a chance on me with a referral from one of the best FUI (Faux User Interface) designers in the world named Mark Coleran. G-Creative has done screen designs for some of the biggest blockbusters in film history, so I was kind of walking on air when they came to me to concept out some looks for Deadpool's distorted vision and Stryker's command center for X-Men Origins: Wolverine.

Based on that work, G-Creative hired me again for the film 2012 and a few other films like Rise of the Planet of the Apes. I still consider myself rather new to the game, but I love focusing on this kind of work.

How much room for creativity do you generally have when creating a fictional interface for a major film?
There's quite a bit actually. It's part of why I love it so much. You're usually being asked to tell part of the story with the screens. So, for instance, you might need to quickly explain a complex story-point and make it immediately clear in two seconds, or elevate the threat level of the scene. It's a challenge, and it has to fit in with the film's look and work with the actors' performances.

They're relying on you or your team to provide the research, design, and animation that should go on each screen. So, sometimes you're brainstorming things like, "What would Fury need to have on his screen when the Helicarrier becomes invisible? What kind of stat windows would he have when engine number three goes down? Do his maps show currently deployed S.H.I.E.L.D. agents all over the world or other Helicarriers?" There's a lot of back and forth between the team, the director, and the VFX supervisor to figure all of that out. It's quite a fun challenge.

Do you generally create the interface first, which is then incorporated into scenes, or do you add the graphics to the footage?
A lot of times the design is live for filming on set, but I usually do graphics that are added in after the scenes are filmed. It's done both ways depending on the needs of the film. With Rise of the Planet of the Apes, for instance, I did temporary graphics for them to film and pace the action, but then I also did the final graphics that were composited in after filming.

For the film 2012, G-Creative did all the on set playback monitors for the rescue arks and various other scenes explaining in scientific terms how the world will end. After filming was completed, they had a number of screens that needed storytelling elements and we did those as post-replace.

With The Avengers, Cantina Creative designed all the on set playback monitors before they filmed. These were considered the “background” screens; the ones that all of the Helicarrier personnel are sitting at. But all of the glass monitors in the Science Lab with Stark and Banner, or the ones that Fury and Hill used were simply blank glass panels on set. So all of them, and of course the HUDs, were done in post by compositing them into the footage after everything was shot and roughly edited together.

How did you go about designing the symbology and layout for Iron Man’s in-helmet layout?
The HUD was a fantastic challenge because it had to be new and yet also respect what was established before. Initially we did go all out and created some organa-tech experimental looks and animations, but since The Avengers was not really exploring Stark's neural-interface as a story-point, it was decided to make the HUD less organic and more tech.

There were two Iron Man films before that set the groundwork for the basic look and feel, starting with Dav Rauch and Kent Seki's excellent work on the first to determine how the HUD even worked and lived on screen. In Iron Man II, Stephen Lawes, Venti Hristova, Sarah Blank and Takashi Takeoka (of Cantina Creative, formerly from the PLF) took on the design of several new HUDs like the Mark V, Mark VI, and War Machine. So just like the physical suit design, each new version has its designer's unique mark on it, while still remaining recognizably Iron Man. So that was the challenge with the new Mark VII HUD—to honor the established design language, but make it new, updated, exciting and more three dimensional.

It was clear from the beginning that production didn't want to just load the HUD with gorgeous visual clutter that didn't mean anything. Janek Sirrs (VFX Supervisor) and Joss Whedon (Director) liked to know why everything was on screen and what its function was. So I created an 11×17 printed HUD bible that listed every function for every widget that I designed.

To design to that level of detail I loaded three notebooks full of sketch ideas and research on combat aviation HUDs, commercial HUDs, experimental aircraft HUDs and flight systems and even solicited situational feedback from an A10 fighter pilot to help flesh it out as realistically as possible.

Your visualizations, like Deadpool’s vision in Wolverine and the cancer cells in Wall Street 2, are a bit different than your other work in that they are more organic images. Is your process different when recreating an actual organic scene and structure? Does one style come more naturally to you than the other?
I definitely love organic, but it's also much harder to do. I did a lot of organic looks in Rise of the Planet of the Apes as well, and enjoy the challenge.

My process for organic things is still very structured, because often the director and or VFX supervisor will want to make very precise changes. For instance, in Wall Street II, Oliver Stone was extremely specific in how each artery would behave and how fast it would grow.

In Apes, it was a similar situation. For the neuron scene I needed to be able to control each dendrite and produce various amounts of growth and pacing. And doing organic in such a way that is modifiable is where the process gets tricky.

I'd say that tech comes much more naturally to me. Organic just keeps me balanced. I'd love to combine tech and organic a lot more in the future.

The interfaces you have designed are, of course, fictional. Do you see the potential for your film designs to ever be used in actual consumer-focused technology?
Well, I won't say there's not potential… But real user interface design is a completely different art form. It's a field I respect a lot, but it's different from what I do. In film, there is really no such thing as UX (User Experience) because the user will never make a mistake. They are actors acting. And, quite conveniently, in three quick moves they can make a computer explore 25,000 cell phone cameras and locate a suspect via facial recognition from an image reflected off the eyeball of a nearby cat. So, we break some rules.

With Iron Man's HUD, the idea is that Tony Stark designed it only for his personal user experience. So he really has no concern if you or your grandma can use it. And he's extremely comfortable with mass amounts of data display. To him, it's a relatively simple display with what he needs, when he needs it.

So the challenge is different; it's more about story than user experience. And I totally love it that way.

At the same time, having that freedom allows you to explore a lot and make discoveries you might not normally make if you're too focused on too many rules. So some of that could make it into the real world. I have recently been working on several real-world designs for the military, and they specifically ask for the more exploratory dynamic approach I've done in films. The challenge of bridging the two is not a small one, but I think it does mean some very exciting solutions to issues that haven't been able to be solved via more traditional means.

Bret Victor, who was an interface designer for Apple, made several interesting points about the direction in which interfaces are headed and the lack, in our current technology, of tactile response that fits our instinct for handling physical objects. What’s your opinion on the direction in which interfaces are headed? What traits do you believe interfaces will have in 10 years that they don’t now?
Ah, yes, I read that a while back. It struck me because I was cautiously excited about the development of haptic feedback for keyboards on screens because screen-keyboards have always bothered me. In fact, like a lot of people, I've always been sensitive to touch, whether it's playing piano and how the difference in the weight of the keys between my solid baby grand and the feathery plastic keys of my synth affect playing style. Or how going from textured paper to a smooth Wacom to a glossy iPad affects the way you draw. So, I love the idea of tactile response. But I haven't seen… or felt, rather, a great solution yet. There are also some interesting things being done with air, and that may hold promise, especially for interactive dimensional displays.

But at the same time, we have a generation growing up that is extremely comfortable with glossy touch screens. They're starting to use them from the time they're two years old, so while they may not have the tactile feedback of a hammer, they're extremely useful tools as-is. Just as UI's no longer have to be so grounded in physically relatable terms like “desktop,” “folder,” “trash bin,” etc. to be understandable, I think gestures and touch screens don't require true tangibility to be truly useful. As a culture, I think we're moving beyond that need.

But there is a definite need to do things like touch-type on screens, just as a basic starting point. And, I do think it'd be fantastic one day if we could be physically turning holographic jog-wheels and playing tricked out holographic cyber-pianos!

What projects are you currently working on/will you be working on next?
I've been working on some very cool real UI's for the military that I can't say much about, and have a few film projects I'm super excited about. It appears that screens and holographic displays are becoming more and more integral to films and storytelling, and that's awesome. This is a great time to be jammin' on this type of work.

The Avengers is ™ & © 2012 Marvel and Subs.
Wall Street II: Money Never Sleeps ©2010 Twentieth Century Fox Film Corporation. All Rights Reserved.
2012 is © 2009 Sony Pictures Digital Inc. All rights reserved.
Rise of the Planet of the Apes ©2011 Twentieth Century Fox Film Corporation. All Rights Reserved.

@ImYourKid