Loss of Olfactory Receptor Function in Hominin Evolution

The mammalian sense of smell is governed by the largest gene family, which encodes the olfactory receptors (ORs). The gain and loss of OR genes is typically correlated with adaptations to various ecological niches. Modern humans have 853 OR genes but 55% of these have lost their function. Here we show evidence of additional OR loss of function in the Neanderthal and Denisovan hominin genomes using comparative genomic methodologies. Ten Neanderthal and 8 Denisovan ORs show evidence of loss of function that differ from the reference modern human OR genome. Some of these losses are also present in a subset of modern humans, while some are unique to each lineage. Morphological changes in the cranium of Neanderthals suggest different sensory arrangements to that of modern humans. We identify differences in functional olfactory receptor genes among modern humans, Neanderthals and Denisovans, suggesting varied loss of function across all three taxa and we highlight the utility of using genomic information to elucidate the sensory niches of extinct species.


Introduction
Neanderthals, whose fossils have been found throughout Eurasia, are extinct relatives of modern humans. Neanderthals disappeared from the fossil record approximately 30,000 years ago and a number of hypotheses exist to explain why [1]. An analysis of the 3D structure of the Neanderthal cranium shows a significant size difference between the olfactory bulb of modern humans and Neanderthals, with human olfactory bulbs being larger, suggesting a potential difference in olfactory capabilites between Neanderthals and modern humans [2]. However, a recent study concluded that although there are differences in the size of the olfactory bulb among living humans (smokers versus non-smokers), there was little to no difference detected in the ability to differentiate odors, suggesting that odor detection thresholds may be more important than bulb size [3]. Therefore, although the difference in bulb volume between modern humans and Neanderthals may represent varying olfaction capabilities in the two lineages, morphology alone may not be enough to determine this. Isotope analysis of Neanderthal fossilised teeth suggests a diet high in protein with some plant consumption [4]. Fossils of another hominin, the extinct Denisovan hominin -a sister taxon to Neanderthals, have been found in the Denisova cave in Siberia [5]. Almost no phenotypic information exists for this species, however both Neanderthal and Denisovan genomes have recently been sequenced allowing further exploration of their genomes [1,6].
Olfactory receptors (ORs) are G protein-coupled receptors located in the olfactory epithelium of the nasal passage, encoded by the largest mammalian gene family [7]. ORs are a highly plastic gene family showing frequent gene birth/loss events [7]. Two OR classes have been described, Class I and II which bind water and air borne odours respectively [8]. ORs are grouped into subfamilies based on sequence similarity [8]. A 'many -to -many' binding relationship exists between odorant molecules and ORs, with a single odorant molecule binding to many receptors and a single receptor binding many odorant molecules [21]. The 1000 Genomes project revealed that SNPs can lead to differences in functional and non-functional ORs between modern humans, including instances of 'rescue' from pseudogene to functional gene [9]. Loss and gain of OR function correlates with ecological niche adaptation as observed by the high numbers of pseudogenes in cetaceans, assumed driven by adaptation to aquatic environments [8].
Given the physical changes observed between the Neanderthal and modern human olfactory bulb [2] we examined whether OR gene losses vary across modern humans, Neanderthals and Denisovans representing differences in the evolution of olfaction in hominin species.

Materials and Methods
Olfactory Receptor Data 853 annotated modern human ORs were downloaded from the HORDE (Build #43) online database [10]. Of these, 466 are annotated pseudogenes. 813 chimpanzee ORs, 433 of which are pseudogenes, have been previously annotated by Go and Niimura [11]. We compared non-functional ORs between the two to identify conserved stop codons present in both species. A stop codon was considered conserved if it was present in the same position in modern human and chimpanzee. Chimpanzee pseudogenes were assigned to one of the 13 OR subfamily clusters, OR 1/3/7, OR 2/13, OR 4, OR5/8/9, OR 6, OR 10, OR 11, OR 12, OR 14, OR 51, OR 52, OR 55 and OR 56, using methods described in Hayden et al. 2010 [8]. OR pseudogenes shared between the chimpanzee and modern human genomes were also assumed to be non-functional in the Neanderthal and Denisovan genomes despite limited or no read coverage [11].

Hominin Genome Alignments
Alignments of the 16 Neanderthal genome, the recently sequenced 506 Altai Neanderthal (http://www.eva.mpg.de/ neandertal/index.html), and both the 16 and 306 Denisovan genomes with the modern human genome were initially in BAM format and were subsequently converted to SAM format via SAMtools [1,12]. Neanderthal reads were parsed and mapped to the database of modern human ORs using Bowtie [13]. Allowing a difference of n = 3 nucleotides per read, a total of 59,695 and 3,737,906 reads were mapped for the 16 and 506 Neanderthal genomes respectively. Denisovan reads were also mapped to the modern human OR dataset with Bowtie, allowing n = 3 mismatches. For the Denisovan genomes, 84,867 reads (16) and 8,110,291 reads (306) were mapped to the reference modern human ORs. A gene was considered non-functional if it contained an 'in-frame' stop codon [14].
Putative Neanderthal and Denisovan OR reads were aligned to the modern human OR gene family using Clustal Omega [15]. These reads were translated into amino acids using the open reading frame with respect to the target modern human OR. Due to the possibility of sequencing error leading to stop codons in the ancient genomic data, a stop codon needed to be present in at least 2 reads mapping to the same position to be considered a potential indicator of OR loss of function. This is depicted in Figure 1. Alignments between modern human and extinct hominins were grouped into the following 4 categories: (1) A stop codon present in at least 2 or more ancient hominin reads, that mapped to the same position in a functional modern human OR (functional in modern human lineages); (2) A stop codon present in at least 2 reads that maps to the same position in a non-functional modern human OR (non-functional in both); (3) Stop codons present in reference modern human OR not found in ancient hominin reads mapping to the same locus (non-functional in the modern human lineage); (4) functional in both taxa.
To further confirm mutations that lead to the gain or loss of function in the Neanderthal and Denisovan, both the modern human reference OR and hominin reads were realigned using the Genome Analysis ToolKit (GATK) worfklow and variant calling tools [16,17]. For each potential mutation identified using the above methodology, GATK also identified said loss of function. To account for potential deamination in the ancient DNA samples that could result in a false functionality assumption we compared the ORs in the high coverage genomes to the low coverage genomes, looking for the same mutation at the same position indicating that the mutation was real.

Multi-locus Read Mapping
Due to the high degree of conserved sequence similarity between ORs in the same subfamilies, there was a possibility of reads mapping to multiple loci for both the Denisovan and Neanderthal genomes, inflating the number of genes unique to each lineage. To overcome this issue when identifying non-sense mutations, we only used reads that shared a distinct unique sequence with a single target OR. In cases where an amino acid sequence was not phylogenetically divergent enough due to redundancy in the genetic code, a nucleotide sequence was used. The sequences used for each resulting OR of interest are described in Table 1.
The mutation leading to non-functionality for OR2L8, OR8I2 and OR5M11 are shared between the Denisovan and Neanderthal, with the same mutation existing in some modern humans. This may indicate that the ancestral state for these ORs in the Neanderthal-Denisovan-modern human most recent common ancestor is non-functional, with modern humans regaining function post-divergence. The mutation that retains function for OR1P1P, OR2J1P and OR9H1P is shared between Neanderthals, Denisovans and some modern humans. This may indicate an ancestral functional state, with a loss of function happening along the modern human lineage. However, without a larger population sampling of extinct genomes for each hominin species, it is difficult to discriminate among alternative evolutionary trajectories.
The deamination of cytosine-to-thymine and guanine-toadenine can cause a large number of sequencing errors in studies using ancient DNA [18]. Three Neanderthal (OR5AR1, OR6C74, OR10X1) and 3 Denisovan apparent loss of function mutations (OR4C12, OR5AC2, OR51Q1) potentially could have resulted from such deamination (Table 1). While it may be difficult to differentiate between real loss of function and deaminationinduced damage, 2 of these ORs (OR6C74, OR10X1) show the same loss in the 16 genome for Neanderthals, suggesting that it is a real mutation event as opposed to deamination. None of the potential deamination events in the 306 Denisovan genome have coverage in the 16 genome version [19], however one of these mutations can be found as a variant of its modern human homolog (OR51Q1).
There are 466 and 433 non-functional ORs in the modern human and chimpanzee genomes respectively. Of these, 99 modern human non-functional ORs have conserved stop codons also present in the chimpanzee, suggesting they were inherited as pseudogenes from the human-chimpanzee common ancestor and are therefore most likely present as inherited pseudogenes in the Neantherthal and Denisovan lineages. These data suggest at least 473 (466 modern human pseudogenes with an additional 10 pseudogenes, minus 3 functional Neanderthal ORs that are pseudogenes in modern humans) and 469 ORs are non-functional within the Neanderthal and Denisovan genomes respectively (Figure 3).  Based on DNA sequence analysis, the divergence time between Denisovans and Neanderthals is 640,000 years [5]. The divergence of Denisovans, Neanderthals and modern humans from their most recent common ancestor is estimated at 804,000 years ago [5]. The relatively recent divergence time between modern humans, Neanderthals and Denisovans suggests a rapid pseudogenization for these ORs. Periods of cold climatic conditions such as those of the middle-late Pleistocene [4] may have caused environmental pressures that could potentially play a role in loss of OR function. Considering that a minimum odorant detection threshold must be met to detect a smell in modern humans [3], in conjuncture with the effect of cold temperatures limiting odor volatility [20], it is possible that the loss of 10 Neanderthal and 8 Denisovan ORs may have had an affect on odorant perception for these hominin species. The loss of 10 ORs may be related to the decreased size of the olfactory bulb in Neanderthals, however inferring how gene loss may affect phenotype in extinct species remains challenging. Further research into how different odors bind to different receptors [21] may shed further light on this issue. This is an ongoing challenge in comparative sensory genomics.
This analysis of OR genes suggests that modern humans, Neanderthals and Denisovans were subject to a number of changes in their respective OR repertoires. As more ancient genomes are sequenced, such analyses may provide a better picture of how sensory perception has evolved, particularly in species for which little phenotypic information is known.