Evidence for distributed and localized semantic knowledge representation

Critical to understanding brain and behavior is clarifying the principles of human knowledge representation. A plethora of scientific findings from neuroimaging, lesion studies, and computational modeling has sparked debates on the degree to which functions and knowledge are localized to specific regions or distributed across multiple regions of the brain cortex [13–18]

Although this commentary is not meant to serve as an exhaustive review, it bears mentioning that on the one hand, since the early work of Broca, Penfield, Meynert, and Wernicke, among others, it has been shown that multiple regions in the human cortex are necessary for or have selective responses to sensory, motor, perceptual, linguistic, or abstract cognitive functions [13,15,18–24]. Indeed, as of today, the dominant model for the representation of semantic knowledge in the human cortex proposes the involvement of multiple spatially localized, functionally specialized but interconnected brain regions (Fig 1A). Several regions of the cortex have been reported to respond preferentially to objects, such as animate or inanimate artifacts, and to motor planning for object-centered actions, among others. Over the years, this model has received substantial evidence from multiple measurement methodologies, both from lesion studies as well as from neuroimaging [18,19,22–26].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Evidence for distributed or localized information representation in the human brain.

(A) Localized representation. The evidence for information representation localization in the human brain is pervasive. This panel shows some of the approximate regions representing different types of information such as information about object form, motion, and object use, as well as information integration. The localized knowledge representation model is perhaps the currently dominant model. It has received converging support from multiple methodologies, suggesting that focalized brain areas represent very specific types of information. (B) Distributed representation. Evidence for distributed brain response to semantic tuning during passive viewing of movie clips [27]. The authors used PCA to map a four-dimensional semantic space and projected each brain location tuning vector onto the second, third, and fourth principal components—visualized as mixtures of red, green, and blue colors. Voxels with similar tuning projected to nearby points in the semantic space are assigned similar colors in this representation and nicely cluster across the cortical mantle. For example, whereas cortical locations colored along the yellow–green axis are more selective to animated objects, body parts and locations colored along the purple–red axis are more selective to spatial locations and movement. (C) Semantic space organization. Objects viewed during movie clips mapped onto the nodes and hierarchical structure of the WordNet lexicon [28]. This visualization allows us to appreciate how semantically neighboring words are represented across spatially neighboring cortical locations as shown in panel A. Figures generated using http://gallantlab.org/index.php/brain-viewer/. Fig 1A generated combining summary figures reported in [23] and [24]. Fig 1A brain picture modified from a publicly available file through the Wikimedia Foundation. PCA, principal component analysis.

https://doi.org/10.1371/journal.pbio.2005758.g001

On the other hand, perhaps following the legacy of Flourens and Lashley, much work has been reported that is consistent with the existence of distributed representations spanning across multiple cortical regions sharing some degree of functional overlap[14,29,30]. Indeed, human knowledge could, in principle, be distributed dynamically and ubiquitously throughout the cerebral cortex, even though some regions of the cortex might respond preferentially to specific categories of actions or stimuli. In fact, compelling reports have measured semantic knowledge signals distributed across the entire human cortex [16,17,27]. Using functional neuroimaging and advanced computational methods, it has been shown that cortical signals map semantically neighboring concepts to spatially neighboring regions. Furthermore, the semantic representation of the cortical signals can change relative to the allocation of attention during behavioral tasks at hand. These results suggest that the space of semantic knowledge could, in principle, be mapped smoothly and dynamically across the human brain cortex. Fig 1B shows an example of the spatial distribution of brain responses to semantic categories; panel C shows the corresponding organization of the semantic knowledge. Color clusters demonstrate continuous but segregated distributions of semantic knowledge. The results can also be taken as an indication that multiple signals could coexist within each brain area. Dynamic signal multiplexing such as the one reported by [16,17,27] has been previously demonstrated in the primate brain [31]. The widespread distribution of semantic signals across the human cortex suggests leveraging the network neuroscience paradigm [9,32,33]. Critical to such a goal is understanding the relation between semantic representation and the pattern of structural brain connections and the white matter tissue within.

Mapping disconnection syndromes via representational dissimilarity

The work of Fang and colleagues [34] attempts to relate semantic knowledge—the meaning of words describing objects, things, or actions, such as “dog,” “the sky,” or “travel”—to the structural brain substrates in the human white matter network to demonstrate that semantic representation is not limited to cortical regions but also extends into the connecting white matter pathways [34]. The authors use a clever cocktail of machine learning and computational methods, combining more classical brain lesion data in a group of patients to predict the patients’ object-naming performance (Fig 2A). The pattern of brain lesions was mapped to white matter connections as defined in an atlas [34]. After that, a support vector machine classifier and representational dissimilarity analysis—the inverse of the correlation between brain and behavioral measurements [35–37]—were used to predict patients’ object-naming performance from the brain lesion pattern data in each white matter connection (Fig 2A). In addition, the authors mapped the semantic similarity of a large set of objects in an independent set of control subjects. This mapping allowed them to represent the rich semantic knowledge space of the objects using representational dissimilarity, and such representation was necessary to connect brain connections to behavior (Fig 2). Finally, the two measurement domains for brain and behavior obtained with different data modalities were combined. The authors computed the correlation (Fig 2C, Brain and behavior) between the object-naming dissimilarity matrices for each white matter connection collected on the patients’ brain data (Fig 2A, Brain), with the behavioral dissimilarity matrix describing the objects’ semantic space collected in the control subjects (Fig 2B, Behavior). The procedure allowed the authors to characterize the potential involvement of each connection in the semantic knowledge representation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Semantic knowledge representation in distributed brain connections.

(A) Brain lesion patterns predict patients’ object-naming performance. Patients’ object-naming performance was predicted using a classification algorithm. The characteristic pattern of brain lesion location and the connections overlapping with the lesions are used in combination with the patients’ naming performance of different objects to predict patients’ naming performance of other objects (e.g., classification performance of object 1 is used to predict objects 2, etc.). (B) Objects’ semantic knowledge structure via behavioral similarity ratings by control subjects. Objects’ similarity was estimated behaviorally in control subjects. Subjects judged objects’ similarity along multiple dimensions (e.g., usage, visual, etc.). (C) Bridging across brain and behavior using representational similarity analysis. The dissimilarity of the objects’ naming prediction from the brain lesion pattern data (A) and the independent objects’ dissimilarity ratings from behavioral responses (B) are correlated. The brain connections overlapping with the lesion pattern were identified. Brain and behavior dissimilarity patterns (1- Spearman’s correlation coefficient) were used to identify connections significant in predicting semantic knowledge. Brain connection visualization generated using open services and data available at brainlife.io [38].

https://doi.org/10.1371/journal.pbio.2005758.g002

This approach is a clever extension of previous methods using representational dissimilarity analysis on functional neuroimaging data [35] that can be thought of as a modern, computational incarnation of the classical disconnection syndrome analysis [39,40]. The authors identified a series of connections likely to contribute to communicating information relevant for semantic knowledge (see Fig 3 of Fang and colleagues [34]). Several of these connections appear to project to the temporal lobe and map across regions spanning multiple cortical lobes. This is consistent with previous reports on the role of the temporal, parietal, and occipital lobes in language processing, memory, and knowledge representation [22,25,41,42]. Importantly, the large extent of connections and the areas to which they project, speak to the models introduced in Fig 1. Fig 3A shows a map of some of the principal cortical regions whose connections are identified by Fang and colleagues as relevant [34]. This figure was reconstructed for illustration purposes only. To do so, the regions touched by the connections reported by Fang and colleagues were highlighted in different colors [34]. The figure also shows some of the major known human white matter tracts (Fig 3B). These tracts were not reported by Fang and colleagues [34] but were segmented here for illustration purposes using open services and data available at brainlife.io—a modern open cloud services platform for brain analysis—and established methods for human white matter mapping [43–46]. Whereas Fang and colleagues focus on brain connections [34], in the next section, we propose a series of white matter tracts potentially involved in knowledge representation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. White matter tracts putatively involved in semantic knowledge representation.

(A) Some of the cortical areas with connections identified by Fang and colleagues [34] that could potentially play a role in the representation of human semantic knowledge. (B) Major white matter tracts terminating within and between some of the cortical areas identified by Fang and colleagues (A) [34]. The superior lateral fasciculus, the arcuate fasciculus, the inferior longitudinal fasciculus, the inferior fronto-occipital fasciculus, and the uncinate fasciculus all may play a role in coordinating information processing and supporting the computations involved in establishing our sense of knowledge. Tract visualization generated using open services and data available at brainlife.io [38]. Fig 3A brain picture modified from a publicly available file through the Wikimedia Foundation.

https://doi.org/10.1371/journal.pbio.2005758.g003

A proposal for white matter tracts potentially involved in semantic knowledge representation

A large body of literature supports the view that long-range brain connections wrapped in myelin sheaths are often bundled together. These bundles of coated axonal projections are generally referred to as white matter tracts or fasciculi [1,5,7,43,47,48]. Fang and colleagues [34] studied brain connections by looking at the cortical terminations of the connections passing through the lesion areas. Their connectivity analyses did not identify, however, which of the known human white matter tracts are involved in the measured behaviors. Yet in principle, this information would be possible and important to obtain because such tracts are highly studied and could provide valuable knowledge for understanding brain disease and even for planning and guiding surgical procedures [5,7,43,49].

A full such analysis of the white matter tracts shared by the reported connections would require substantial additional work and data well beyond that by Fang and colleagues [34]. Nonetheless, I performed some qualitative analyses and attempted to propose some candidate white matter tracts that might be relevant to the findings of Fang and colleagues [34]. Fig 3B shows some of the major white matter tracts whose terminations are known to reside within the cortical regions significantly involved in knowledge representation as reported by Fang and colleagues ([34]; see their Fig 3A and 3B). Given the pattern of connectivity identified, it is likely that at least the inferior fronto-occipital fasciculus, the arcuate fasciculus, the inferior longitudinal fasciculus, the uncinate fasciculus, and the superior lateral fasciculus could be involved in semantic knowledge representation (Fig 3B). Additional white matter tracts could, in principle, also include the vertical occipital fasciculus and the posterior arcuate [50–53]. This is, inevitably, a large number of large white matter tracts. It suggests that a complex behavior, such as knowledge representation, might be implemented by the coordinated activity of multiple brain areas communicating via several of the major brain highways. Given the large-scale involvement of distal cortical areas in human semantic knowledge representation, clarifying the role of the white matter tracts could become critical in understanding the fundamental mechanisms of knowledge representation because of their significance in large-scale brain communication. Importantly, understanding brain communication would inform clinical diagnoses with the potential to clarify the relation between the brain and psychiatric diseases. Fang and colleagues [34] bring a modern, computational approach to one of the classic patient literatures on the disconnection syndromes that is foundational to our understanding of many types of brain disease [39,40,54].

Understanding the human brain is a priority for the scientific community not only because it defines us as species but because psychiatric diseases have profound effects on society. For example, just in Europe, yearly estimates suggest that up to 200 million individuals may be impacted by brain disease, affecting up to 25% of the European population over the course of their lifetime [55–59]. Such impact accounts for up to 800 billion euros in annual costs, representing nearly 38% of the European gross domestic product. These estimates may, unfortunately, be destined to rise because of the steady aging of the European population. The statistics for Europe are just a reflection of a worldwide phenomenon, highlighting the urgent need for better understanding of the human brain. Computational neuroimaging provides some of the most promising approaches to address such profound societal needs [55–59].

We are, in many ways, still in the early stages of understanding the role of human white matter in knowledge representation. More generally, both the structure and functional role of white matter are poorly understood and still reside within the least explored side of brain science. Indeed, only recently have we become aware of the critical role that white matter plays in establishing successful, healthy trajectories across long-term life events such as development and learning and in the aging brain. The properties of white matter tissue within the major white matter tracts have been shown to be associated with human cognitive, perceptual, and emotional abilities and to be critical for a healthy brain [7,8,47,60–62]. Yet much more research will be needed to understand the human white matter.

I like to think about the role of myelin and white matter in the brain as that of a tuning mechanism for a musical instrument. The whole orchestra can be in the theater ready to play a concert, but the performance will fall short unless all instruments are well tuned. The white matter acts as that tuning mechanism that permits the symphony of the brain to attain its most masterful form. Our understanding of brain function will be incomplete without understanding the significance of white matter tissue organization and the pattern of connections it supports [7]. The work of Fang and colleagues [34] helps us move forward by proposing new ways to understand the human brain and cognition. Their results suggest that if knowledge is widely distributed across the cortex, then disruptions to a subset of connections can lead to altered semantic tuning, even though most of the knowledge representations distributed across the cortex would remain intact. Much work will be needed to relate the properties of white matter tissue organization to human knowledge representation, and this avenue of investigation is an exciting and important one.

Acknowledgments

We thank S. Vinci-Booher, E. Rosdeitcher, D. Bullock, and B. McPherson for comments on early versions of this manuscript.