Fig 1.
A section of chromosome 1 showing the estimated Neanderthal frequency (pn, black line) for the EUR sample from [13] and the expected frequency (pt, red line) predicted by our best fitting model.
The midpoints of exons are shown as blue bars. Note that the estimated frequency is expected to have much greater variance along the genome than our prediction due to genetic drift. Our prediction refers to the mean around which the deviation due to genetic drift is centered (S2 Text).
Fig 2.
The scaled RSS surface (RSSmin − RSS) as a function of s and μ for EUR and ASN autosomal chromosomes.
Each value of the RSS is minimized over p0, making this a profile RSS surface. Regions in darker shades of orange represent parameter values of lower scaled RSS. Black circles show bootstrap results of 1000 blockwise bootstrap reestimates, with darker circles corresponding to more common bootstrap estimates.
Fig 3.
The scaled RSS surface (RSSmin − RSS) of autosomal chromosomes as a function of the initial admixture proportion p0.
Results are shown for a model where only the nearest-neighboring exonic site under selection is considered, and for t = 2000 generations after the Neanderthal admixture event into the ancestors of EUR (grey) and ASN (pink) populations. Dots and horizontal lines show the value of p0 that minimizes the RSS and the respective 95% block-bootstrap confidence intervals. The RSS surfaces are shown for values of the selection coefficient (s) and exonic density of selection (μ) given in Table 1.
Table 1.
Point estimates and 95% bootstrap confidence intervals for the focal parameters.
Estimates are based on a minimization of the residual sum of squared deviations (RSS) between observations and a model in which, for each neutral site, only the nearest-neighboring exonic site under selection is considered. Introgression is assumed to have happened t = 2000 generations ago.
Fig 4.
The contrast between the inferred parameters for the East Asian (ASN) and European (EUR) samples for the autosomes (A) and both the X and the autosomes (B).
Plots show bootstrap estimates of the initial admixture proportion p0 against the estimated exonic density of selection μs, with the empty symbols denoting our minimum RSS estimates. The clear separation of the point clouds for autosomes and the X for both EUR and ASN modern humans suggests that the combination of selection and initial admixture level are likely the reason why the present-day frequency of Neanderthal alleles differs between autosomal and X chromosomes. Note the different scales of the axes in (A) and (B).
Fig 5.
Genomic regions with lower exonic density contain higher average Neanderthal allele frequency in both in Europeans (grey circles) and Asians (pink circles).
We find a good fit to this pattern under our model (black and red triangles). Ranks are obtained by splitting the genome into 1 cM segments, calculating the number of exonic sites for each segment and sorting the segments into ten bins of equal size. Dashed lines represent 95% blockwise bootstrap confidence intervals. Plots created for different segment sizes look similar (S2 Text).
Fig 6.
Simulations showing that the Neanderthal population is predicted to harbor an excess of weakly deleterious fixed alleles compared to humans.
(A) A two-dimensional histogram of the difference in allele frequency between the Neanderthal and human population, and the deleterious selection coefficient over all simulated sites. (B) The fraction of sites in the simulations where there is a human- or Neanderthal-specific fixed difference, binned by selection coefficient. Dotted lines indicate the nearly-neutral selection coefficient (i.e. the inverse of the effective population size) for Neanderthal (right) and Human (left) populations. Solid lines show the 95% CI of s for ASN (the larger of the two CI) that we inferred. Note that monomorphic sites are not shown, but are included in the denominator of the fraction of sites.