Feindel Virtual Brain and Mind Lecture Series: Reconstructing visual and subjective experience from the brain

Yukiyasu Kamitani presents, "Reconstructing visual and subjective experience from the brain".

Registration via Eventbrite.

Livestreaming via Vimeo. 

Speaker: Yukiyasu Kamitani, Ph.D.

Professor, Graduate School of Informatics, Kyoto University, Kyoto, Japan

Department Head and ATR Fellow, ATR Computational Neuroscience Laboratories

Abstract: The internal visual world is thought to be encoded in hierarchical representations of the brain. However, previous attempts to visualize perceptual contents based on machine-learning analysis of fMRI patterns have been limited to reconstructions with low level image bases or to matchings to exemplars. While categorical decoding of imagery contents has been demonstrated, the reconstruction of internally generated images has been challenging. I introduce our recent study showing that visual cortical activity can be decoded (translated) into the hierarchical features of a pre-trained deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features. Next I present a novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers. Our method was able to reliably produce reconstructions that resembled the viewed natural images. While the model was solely trained with natural images, it successfully generalized to artificial shapes, indicating that our model was not simply matching to exemplars. The same analysis applied to mental imagery demonstrated rudimentary reconstructions of the subjective content. Our method can effectively combine hierarchical neural representations to reconstruct perceptual and subjective images, providing a new window into the internal contents of the brain.

The Feindel Virtual Brain and Mind (VBM) Seminar Series will advance the vision of Dr. William Feindel (1918–2014), Former Director of the Neuro (1972–1984), to constantly bridge the clinical and research realms. The talks will highlight the latest advances and discoveries in neuropsychology, cognitive neuroscience, and neuroimaging.

Speakers will include scientists from across The Neuro, as well as colleagues and collaborators locally and from around the world. The series is intended to provide a virtual forum for scientists and trainees to continue to foster interdisciplinary exchanges on the mechanisms, diagnosis and treatment of brain and cognitive disorders.