Natural sounds can be reconstructed from human neuroimaging data using deep neural network representation
Fig 6
Reconstructions with leave-category-out analysis.
Each panel corresponds to a sound category excluded during training. The upper row in each panel illustrates the spectrograms of the original stimuli. The middle row displays the reconstructed spectrograms generated by decoders trained on the full dataset. The bottom row depicts the reconstructed spectrograms obtained using decoders trained on a dataset where data from the test category were excluded during training (ROI: AC; DNN layer: Conv5). Audio examples of the reconstructed sounds are accessible at https://www.youtube.com/watch?v=znm6NWL1YYY. Adjacent to the spectrograms, the upper bar graphs represent the identification accuracies achieved using decoders trained on the full dataset, while the lower bar graphs show the accuracies for decoders trained on the leave-category-out dataset. Error bars represent the 95% CIs. Each color in the bar graphs corresponds to a different subject. The data underlying this figure are provided in S2 Data.