Immerse Yourself in a 3D Cloud of Creative Coding Culture

From its surface to its core, CLOUDS is a reflection of the community it examines. You can navigate through hours of interviews with creative coding luminaries in a futuristic game-like interface, and explore their ideas as digital creations inhabiting the same space. CLOUDS is an immersive documentary created by artist James George and filmmaker and media historian Jonathan Minard. Available for download today on Mac, PC, or Oculus Rift, the film encompasses interviews with people in the creative coding industries, covering a mind-boggling gamut of topics. Interview subjects appear as ethereal digital creatures that you can navigate around within a virtual space. The futuristic experience was made with 3D video captured using DepthKit, an open source toolkit created by James George that uses a Microsoft Kinect to augment a normal digital camera with depth data—creating, in effect, a three-dimensional digital reproduction of the subject.

At any time you can easily navigate to a clip from a different interview that addresses a related topic, or you can enter an immersive visualization created by the artist being interviewed. CLOUDS was built around the same open source platforms (notably openFrameworks, Processing, and Cinder) that stand at the center of the community that the documentary explores. The 40 software artists interviewed in the film also use these tools in their work, and many have helped to create them.

The Creators Project caught up with James George and Jonathan Minard to learn more about the ideas behind CLOUDS:

The Creators Project: In the film you see the people living as digital creatures in a digital world. You don’t try to hide the digitalism behind a curtain. How did you make that decision to make it very clearly virtual?

Jonathan Minard (JM): We were really inspired by cyberpunk fiction, and at the 2010 Vimeo Awards, Bruce Sterling had described the camera of the future as a kind of device that could capture all the photons in the room and assemble a kind of complete recording of one time and place. And I think RGBD and this 3D scanning technique suggests that possibility, but it certainly wasn’t there as we were developing and exploring it. And so we decided to settle on exactly what you described, on a kind of aesthetic which is transparently digital, and which acknowledges its artificiality. It’s a kind of fragmented digital world, with bits of code and things floating around. That was very much the idea. Because we knew that was attainable, but also we want people to kind of have an awareness of what they’re experiencing—that they’re having an experience which is computationally generated, which is created from algorithms running in real-time.

Bruce Sterling appears in CLOUDS among your interview subjects, and you’ve also mentioned him as a personal influence. But he’s a science fiction writer, not a coder. How does science fiction play into the ideas behind CLOUDS?

James George (JG): We were interviewing people who work and think about the future, and that really use technology and programming as a way to engage with science fiction in a very tactile way. You can make prototypes that suggest futures that aren’t here yet, but you still can kind of touch it or glimpse it—versus just writing about it. So in some ways that’s really the way that we write science fiction. We write prototypes for futures that don’t exist yet, but that allow you to interact with a small sliver of that future. CLOUDS is really like that. So of course there’s not very much of the person there that we filmed. It’s just a sort of shimmering holographic point cloud, but you can actually imagine the extrapolation of that technology into full-capture and full 3D worlds that are actually filmed rather than rendered in 3D.

So CLOUDS is a very unconventional format. How is thinking about a network different than thinking about a traditional narrative?

JM: Documentaries tend to follow the thread of an idea. When we were conducting the interviews with these artists and technologists and started to realize that we didn’t have to constrain the content of the conversation—that we could have this sprawling, free-flowing conversation and include all of it—that was quite liberating. I think somewhere halfway into the production we started to think about it in that way, that we’re creating a network. And even actively trying to connect threads of ideas between those conversations among the artists—to explore the ways their work is connected, the way their ideas are connected, the way that the code that they’re developing is shared with this community. And so I think that on many levels CLOUDS reflects the ideas of networks. Open source tools have really flourished on the internet, because you have communities of people who are suddenly able to share their ideas and actualized metaphors of code. And so we constructed it in that same way.

Is part of the motivation behind this whole project just to expose people to creative coding?

JG: Absolutely. I kind of made it in the way where I wish it would’ve been something I saw much longer ago, because I studied computer science and was working at tech companies, and always had this kind of video and artistic bent—and I never really knew how to merge those two. It was really through finding the openFrameworks community when I was finished with school that my life changed. I wish I would’ve had something like this to really explore that network. And really Twitter was that to me. I would click from person to person. I don’t know if you saw the visualization of all the people [seen above]? That’s actually a data visualization of their actual Twitters. We scrapped all of the people in the documentary’s tweets, and each line represents a tweet from one person to another. And then we run through the history through 2012 and 2013, and that’s all the flashes of them talking to each other, and we don’t need to explain it, but really it is a visualization of that network that they’re actually out there communicating with each other, and I want this film to be an invitation to that. You can watch it and download the tools. The code for the film itself is open source. So you can actually see how the visualizations were created. So for people who want to engage with it on that level, the door is open. And that’s important—it’s in the tradition of the community that it’s open.

Click here to visit the official site for the CLOUDS interactive documentary.

Related:

New Exhibition Pushes The Binary Boundaries Of Creative Coding

Surreal 3D Video Game Puts an Architectural Spin on the Exquisite Corpse

Interactive SXSW Installation Uses Kinect To Beam Users Into CGI Worlds

Practice Sousveillance With The SlyPhone Camera Attachment