Natural sounds can be reconstructed from human neuroimaging data using deep neural network representation
Fig 1
Schematic overview of the proposed sound reconstruction pipeline.
(A) DNN feature extraction from sound. A deep neural network (DNN) extracts auditory features at multiple levels of complexity using a hierarchical framework. (B) Sound reconstruction. The reconstruction pipeline starts by decoding DNN features from fMRI responses using trained brain decoders. The audio generator then transforms these decoded features into the reconstructed sound.