Images viaGenuinely new experiences are rare; even new works of art, intentionally or otherwise, reference past artistic creations. But when I entered the latest version of SoundSelf, Robin Arnott's multisensory Oculus Rift experience, which fuses sound, vision, and vibration into one sublime biofeedback loop, I had to discard everything I'd seen before to describe what I'd experienced. The closest comparison could be a combination of six hours spent in an isolation chamber, a visit to La Monte Young's Dream House, and an eye-straining optical illusion— as far as personal experiences go, the term sui generis doesn't even cut it.
Advertisement
As Arnott lowered the new Oculus Rift onto my head, fixed headphones onto my ears, attached two microphones around my voice box, then positioned my back against a flat, soft subwoofer, I didn't know what to expect. This wasn't helped by the facts that SoundSelf began with a black void, and that the ambient sounds of the room I was in that could barely be heard through the headphones.Arnott told me to hum monotones and hold them until I wanted to try out some other notes. If I modulated my voice too much, he cautioned, the experience wouldn't work as well. So, I hummed— and suddenly, a brand new, fifteen-minute cavalcade of kaleidoscopic, geometric shapes and colors burst and drifted into my field of vision, merging with my voice, and the vibrations coming out of the subwoofer.
Some people might say that SoundSelf is a “mind-melting experience.” This, in my opinion, isn't quite accurate. The game, which Arnott describes as something like a trance or virtual meditation, doesn't melt the mind so much as activate and elevate consciousness to a new type of reality. Melting implies mental overload or breakdown. SoundSelf, on the other hand, pulls the user into a beautiful and dynamic multi-sensory reality, one that creates a sense of calm peacefulness.The word “psychedelic,” in Arnott's opinion, is a horrible aesthetic crutch that's overly used when talking about non-chemical trips. SoundSelf, comprised of original software written in C++ (one that builds the system, the other that visualizes sound), clearly isn't meant to be a virtual substitute for mind-altering chemicals. Since the game is the result of arranged computer code, its new realities are fundamentally different. Arnott's hope is that in a few years developers and users will create a new vocabulary for trance-inducing virtual reality experiences.
Advertisement
“With SoundSelf, it's such a difficult thing to describe, and one of the only things I can try to leverage a little is the 'star gate scene' in 2001: A Space Odyssey,” said Arnott. “So, the way I try to pull that into the description is I describe it as 'an odyssey of light and sound' and try to make people make the connection.”
Arnott, who created sound design for the mind-bending experimental games Antichamberand The Stanley Parable, originally became interested in gaming during his time at New York University's film school, Tisch. In his last year at NYU, the school founded the NYU Game Center, which he describes as the current equivalent of the university's film school in the 1980s. This game center helped hurdle Arnott down his current course in experimental gaming experiences.Before working on the above games, Arnott created his own, Deep Sea, where the player, underwater and blinded with a gas mask, fights a sea monster that he or she can only hear. Arnott used two little microphones embedded in breath-in/breath-out ports to allow the computer to sense players' breathing.The player aims and fires a weapon, hoping to hear the creature cry out in pain. More often, the only sound the player hears is “their shot disappearing uselessly into the void.”Deep Sea is ultimately, as Arnott told me, about being “vulnerable.” Though it was very much a game (as well as an installation piece), its concept and design would have a big impact on the SoundSelf development process.
Advertisement
Arnott also explained that his game design background allowed him to think about SoundSelf's user interaction as “flow,” which he said game designers have down to a discipline. Flow can be thought of as a player interacting in the moment with the game system. Side effects of these systems' interactivity, Arnott said, are trance states. So, what he is trying to do with SoundSelf is combine gaming's interactivity discipline with what monks are trying to do with singing bowls or mandalas. A clash of two different technologies, as it were.Originally, Arnott programmed SoundSelf's visuals himself. Never too confident when it came to designing visuals or deploying color, it took about a year before Arnott felt he was proficient at programming dynamic shapes and movement. Eventually Arnott enlisted programmer Evan Balster to help broaden SoundSelf's current visual palette through C++ programming, which had to visualize the game's music as geometry in a generative way.
When Arnott finally started offering demos at gaming conferences like E3's Indiecade booth, the input he got was invaluable. It also proved that SoundSelf challenged gamers in ways they'd never been challenged before.“There is a lot of interesting things to explore other than what it feels like to be powerful, such as what it feels like to be weak,” said Arnott. “With SoundSelf, when gamers come into it they realize that it's an interactive system responding to their voice, and they'll try to take control of it. They'll try to explore it and understand what it does, playing it like an instrument or using it like a gun in a shooter game, and it doesn't work like that.”
Advertisement
Arnott said that this type of gamer will then give up. But when they finally do let go, they find they're able to fully fall into SoundSelf's virtual reality.
Maybe because I am not a rabid gamer, my entry into SoundSelf's latest virtual reality experience was smooth. That I immediately let go might have also had something to do with the fact that I knew the game had close analogues with non-gaming, mind-bending experiences.In SoundSelf's design, Arnott attempted to reach the six qualities of a spiritual experience: a sense of unity, an intuitive sense of deep truth, sacredness, positive mood, transcendence of time and space, and ineffability. Arnott was interested in how these qualities can be experienced across cultures in a number of different ways, and wondered if it could be done in a virtual environment. While he said SoundSelf doesn't achieve an intuitive sense of deep truth or sacredness, he believes the game hits the other qualities pretty well.
Having experienced SoundSelf, I would agree. While the fusion of beautifully rendered, three-dimensional visuals, monotone singing, and the subwoofer's vibrations was indescribable, it didn't produce feelings of deep truth or sacredness. But this might have more to do with VR being in its infancy than any real faults with SoundSelf's design or concept. In the future, it could be possible for Arnott and other virtual reality developers to reach those lofty goals.
Advertisement
What is encouraging is that there were moments in SoundSelf, especially when the visuals resembled soft or silky holographic shapes, where I felt a sense of unity. In these moments, SoundSelf's synchronized input and output—the humming, the virtual shapes flying around, and the vibrations—combined to produce a feeling that I was in another space, but still simply existing in or seeing just another aspect of reality within the cosmos.
Equally as important, the game induced a beautiful and seamless trance that made me forgot that I was in virtual reality. Though fleeting, the moments were powerful; something for Arnott and his team to build on in future SoundSelf iterations. The only other way to try and describe SoundSelf is by comparing it to a synaesthetic experience, where sight, sound, and other sensations blur into one.“Synaesthesia is one of the dragons I'm chasing, which puts you off balance in the way you perceive things,” said Arnott. “With SoundSelf, the hope is that you experience a oneness of self while you're also experiencing a oneness of sensation.”While SoundSelf is not pure synaesthesia at this point, technological advances in devices that create virtual sensations—from sound to touch, though perhaps not smell—should help Arnott further refine his virtual odyssey. And when the Oculus Rift consumer model comes in the near future, then we can all trip the light fantastic in SoundSelf's ever-evolving world.
ONE EMAIL. ONE STORY. EVERY WEEK. SIGN UP FOR THE VICE NEWSLETTER.
By signing up, you agree to the Terms of Use and Privacy Policy & to receive electronic communications from Vice Media Group, which may include marketing promotions, advertisements and sponsored content.
