** PLEASE READ THE CAPTION TO UNDERSTAND THIS VIDEO **
The left clip is a segment of a Hollywood movie trailer that the subject viewed while in the magnet. The right clip shows the reconstruction of this segment from brain activity measured using fMRI. The procedure is as follows:
[1] Record brain activity while the subject watches several hours of movie trailers.
[2] Build dictionaries (i.e., regression models) that translate between the shapes, edges and motion in the movies and measured brain activity. A separate dictionary is constructed for each of several thousand points at which brain activity was measured.
(For experts: The real advance of this study was the construction of a movie-to-brain activity encoding model that accurately predicts brain activity evoked by arbitrary novel movies.)
[3] Record brain activity to a new set of movie trailers that will be used to test the quality of the dictionaries and reconstructions.
[4] Build a random library of ~18,000,000 seconds (5000 hours) of video downloaded at random from YouTube. (Note these videos have no overlap with the movies that subjects saw in the magnet). Put each of these clips through the dictionaries to generate predictions of brain activity. Select the 100 clips whose predicted activity is most similar to the observed brain activity. Average these clips together. This is the reconstruction.

For a related video see: http://www.youtube.com/watch?v=KMA23JJ1M1o
For the paper (Nishimoto et al., 2011, Current Biology) go to:http://dx.doi.org/10.1016/j.cub.2011.08.031
For more information about this work, please check our lab web site:http://gallantlab.org

Movie reconstruction from human brain activity

In what is perhaps the most significant breakthrough in cognitive neuroscience since the MRI itself, scientists at UC Berkeley have figured out a way to reconstruct brainwaves into digital video. Stop.

In a landmark study [published Thursday](http://sites.google.com/site/gallantlabucb/publications/nishimoto-et-al-2011), for the first time in human history, _actual_ visual experiences have been reconstructed as movies. [Satoshi Kon](http://en.wikipedia.org/wiki/Paprika_(2006_film)) just had a coronary.

The study involved three subjects spending extended periods of time inside MRI machines watching movie trailers. The resulting brainwaves were then referenced against a special computer software that uses a bank of 18 million seconds of random YouTube clips as its palette. Most simply put, brainwaves are broken down into voxels (3D pixels), and the voxels are then matched up with clips that cause similar brain activity. The resulting videos are low-quality, but with a hauntingly uncanny similarity to the images they were reconstructed from. Thoughts, essentially, as movies. And that just happened.

Below is a 30-second video showing the initial image and resulting reconstructions, alongside the similar brain-wave inducing clips used in the study. While there&#x2019;s no word yet on when the software used will be open-sourced (jokes!), here&#x2019;s to the last few days of your thoughts being trapped inside your own head. This is, officially, the future.

For an explanation of the procedure please see the other video on this channel: http://www.youtube.com/watch?v=nsjDnYxJ0bo
This video is organized as follows: the movie that each subject viewed while in the magnet is shown at upper left. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured activity as possible. The clips used to fit the model, the clips used to test the model and the clips used to reconstruct the stimulus were entirely separate.) The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject. 
For the paper (Nishimoto et al., 2011, Current Biology) go to: http://dx.doi.org/10.1016/j.cub.2011.08.031
You can find more information about this work at our laboratory web site: http://gallantlab.org

Movie reconstructions from human brain activity: 3 subjects

Place your comments about the tremendous implications this has about collective unconscious theory below. Seriously. This actually just happened.

(via [Gizmodo](http://gizmodo.com/5843117/scientists-reconstruct-video-clips-from-brain-activity))

FYI.

This story is over 5 years old.

UC Berkeley Recreates Thought As Digital Video

FYI.

This story is over 5 years old.

UC Berkeley Recreates Thought As Digital Video

ONE EMAIL. ONE STORY. EVERY WEEK. SIGN UP FOR THE VICE NEWSLETTER.