^{1}

^{2}

^{3}

^{3}

^{4}

^{3}

^{5}

^{1}

^{*}

† Deceased.

The authors have declared that no competing interests exist.

Conceived and designed the experiments: JMB HJC. Performed the experiments: JEa ENM MF HJC. Analyzed the data: JEa ENM MF HJC. Contributed reagents/materials/analysis tools: SMBJ JMB. Wrote the paper: JEa MF JMB HJC.

Approaches based on linear mixed models (LMMs) have recently gained popularity for modelling population substructure and relatedness in genome-wide association studies. In the last few years, a bewildering variety of different LMM methods/software packages have been developed, but it is not always clear how (or indeed whether) any newly-proposed method differs from previously-proposed implementations. Here we compare the performance of several LMM approaches (and software implementations, including EMMAX, GenABEL, FaST-LMM, Mendel, GEMMA and MMM) via their application to a genome-wide association study of visceral leishmaniasis in 348 Brazilian families comprising 3626 individuals (1972 genotyped). The implementations differ in precise details of methodology implemented and through various user-chosen options such as the method and number of SNPs used to estimate the kinship (relatedness) matrix. We investigate sensitivity to these choices and the success (or otherwise) of the approaches in controlling the overall genome-wide error-rate for both real and simulated phenotypes. We compare the LMM results to those obtained using traditional family-based association tests (based on transmission of alleles within pedigrees) and to alternative approaches implemented in the software packages MQLS, ROADTRIPS and MASTOR. We find strong concordance between the results from different LMM approaches, and all are successful in controlling the genome-wide error rate (except for some approaches when applied naively to longitudinal data with many repeated measures). We also find high correlation between LMMs and alternative approaches (apart from transmission-based approaches when applied to SNPs with small or non-existent effects). We conclude that LMM approaches perform well in comparison to competing approaches. Given their strong concordance, in most applications, the choice of precise LMM implementation cannot be based on power/type I error considerations but must instead be based on considerations such as speed and ease-of-use.

Recently, statistical approaches known as linear mixed models (LMMs) have become popular for analysing data from genome-wide association studies. In the last few years, a bewildering variety of different LMM methods/software packages have been developed, but it has not always been clear how (or indeed whether) any newly-proposed method differs from previously-proposed implementations. Here we compare the performance of several different LMM approaches (and software implementations) via their application to a genome-wide association study of visceral leishmaniasis in 348 Brazilian families comprising 3626 individuals. We also compare the LMM results to those obtained using alternative analysis methods. Overall, we find strong concordance between the results from the different LMM approaches and high correlation between the results from LMMs and most alternative approaches. We conclude that LMM approaches perform well in comparison to competing approaches and, in most applications, the precise LMM implementation will not be too important, and can be chosen on the basis of speed or convenience.

Recently, linear mixed models based approaches have been proposed as appealing alternatives to principal component based approaches when adjusting for population substructure in genome-wide association studies of apparently unrelated individuals

Fitting a full linear mixed model for each SNP in turn across the genome is computationally challenging. These computational considerations have led to the development of several faster approximations for constructing tests of the fixed SNP effects of interest in the linear mixed model

A limited comparison of several LMM implementations, via application to real and simulated data from Genetic Analysis Workshop 18 (GAW18)

Here we expand the investigation of Eu-ahsunthornwattana et al.

Package/method and version | Approach | Kinship estimation method | Reference(s) |

EMMAX emmax-intel‐binary-20120210.tar.gz | LMM (approximate) | Kinship matrix estimated internally using user-supplied set of SNPs, or set to theoretical/estimated values calculated externally | |

FaST-LMM v2.04 | LMM (approximate or exact) | Kinship matrix estimated internally using user- supplied set of SNPs, using SNPs selected through FaST-LMM-Select procedure, or set to theoretical/estimated values calculated externally | |

GEMMA v0.91 | LMM (exact) | Kinship matrix estimated internally using user-supplied set of SNPs, or set to theoretical/estimated values calculated externally | |

GenABEL v1.7-6 (FASTA) | LMM (approximate) | Kinship matrix estimated internally using user-supplied set of SNPs, or set to theoretical/estimated values calculated externally | |

GenABEL v1.7-6 (Grammar-Gamma) | LMM (approximate) | ||

GTAM (implemented in MASTOR v0.3) | LMM (approximate) | Kinship matrix calculated externally (assumed to reflect ‘known’ (theoretical) pedigree relationships) | |

Mendel v13.2 | LMM (approximate or exact) | Kinship matrix estimated internally using theoretical pedigree relationships, estimated within estimated pedigree clusters (using all SNPs), or fully estimated (using all SNPs) | |

MMM v1.01 | LMM (approximate or exact) | ||

FBAT v2.0.4 | Transmission of alleles within pedigrees | Method by definition uses ‘known’ (theoretical) pedigree relationships | |

MASTOR v0.3 | Retrospective quantitative trait version of MQLS | Kinship matrix calculated externally (assumed to reflect ‘known’ (theoretical) pedigree relationships) | |

MQLS v1.5 | Adjusted version of retrospective case/control test | Kinship matrix calculated externally (assumed to reflect ‘known’ (theoretical) pedigree relationships) | |

ROADTRIPS v1.2 (RM test) | Adjusted version of retrospective case/control test | Kinship matrix calculated externally (assumed to reflect ‘known’ (theoretical) pedigree relationships). Further correction based on genome-wide set of SNPs applied internally. |

The approaches are compared via application to real and simulated data derived from a genome-wide association study of visceral leishmaniasis (VL) in 348 Brazilian families comprising 3636 individuals (1970 with both genotype and phenotype data). This Brazilian family data set was used (together with a larger Indian case/control data set) by Fakiola et al.

Before embarking on a detailed comparison of different methods, we explored the use of different SNP sets (containing different numbers of SNPs) for estimating pairwise kinship measures, in order to identify a robust set of SNPs that could be used for subsequent comparisons. We considered using either the full genome-wide set of SNPs (545,433 SNPs), a ‘pruned’ set of 50,129 SNPs selected to have minor allele frequencies

A comparison of the kinship estimates output by different software packages based on the pruned set of SNPs is shown in

Plots above the diagonal show a comparison of kinship measures, with correlations between the kinship measures indicated below the diagonal. EM_BN = EMMAX (Balding-Nichols), EM_IBS = EMMAX (IBS method), FLMM_C = FaST-LMM using covariance matrix, FLMM_R = FaST-LMM using realised relationship matrix, GA = GenABEL, GMA_C = GEMMA using centred genotypes, GMA_S = GEMMA using standardised genotypes, KING_H = KING with homogeneous population assumption, KING_R = KING with robust estimation.

Within any given method, we found the kinship measures (for each pair of individuals) and p-values obtained (in the real data set) based on the full SNP set to be very similar to those based on the pruned set, whereas those calculated based on the thinned set were less similar (see

PLINK = analysis in PLINK with no adjustment made for relatedness. Other methods/software packages are listed in

Listgarten et al.

The automated version of FaST-LMM-Select available as an option within the current version of the FaST-LMM package uses a slightly different strategy involving

We compared the performance of the different LMM and alternative approaches listed in

The success of the various approaches in controlling the overall genome-wide type 1 error rate (i.e. controlling the genomic inflation factor

Trait analysed | ||||||

Method | Description | Kinships used | Real disease (VL) | Simulated strong (sim-D1) | Simulated weak (sim-D2) | Simulated quantitative (sim-Q) |

Unadjusted | Standard linear or logistic regression | None | 1.23 | 1.12 | 1.04 | 1.43 |

EM_BN | EMMAX (Balding-Nichols kinships) | Estimated | 0.99 | 0.99 | 1.00 | 0.99 |

EM_IBS | EMMAX (IBS kinships) | Estimated | 0.99 | 0.99 | 1.00 | 1.00 |

FLMM_A | FaST-LMM (approximate calculation) | Estimated | 0.99 | 0.99 | 1.00 | 1.00 |

FLMM_E | FaST-LMM (exact calculation) | Estimated | 1.00 | 0.99 | 1.01 | 1.00 |

GA_FA | GenABEL (FASTA) | Estimated | 0.99 | 0.99 | 1.00 | 0.99 |

GA_GRG | GenABEL (GRAMMAR-Gamma) | Estimated | 0.99 | 0.99 | 1.00 | 1.00 |

GMA_C | GEMMA using centred genotypes | Estimated | 1.00 | 0.99 | 1.01 | 1.00 |

GMA_S | GEMMA using standardised genotypes | Estimated | 1.00 | 0.99 | 1.01 | 1.00 |

GTAM | GTAM (implemented in MASTOR) | Pedigree | 1.20 | 1.00 | 0.99 | 0.99 |

Mendel_T | Mendel with theoretical kinships | Pedigree | 1.11 | 1.00 | 0.99 | 0.99 |

Mendel_P | Mendel with kinships estimated within estimated pedigree clusters | Estimated | 1.10 | 1.00 | 0.99 | 0.99 |

Mendel | Mendel with fully estimated kinships | Estimated | 1.03 | 0.99 | 1.00 | 1.00 |

MMM_E | MMM (exact calculation) | Estimated | 1.00 | 0.99 | 1.01 | 1.00 |

MMM_G | MMM (GLS approximation) | Estimated | 0.99 | 0.99 | 1.00 | 0.99 |

FBATaff |
FBAT (transmissions to affecteds only) | Pedigree | 1.02 | 1.01 | 1.00 | – |

FBATboth | FBAT (transmissions to all individuals) | Pedigree | 1.01 | 1.00 | 1.01 | 1.00 |

MASTOR | MASTOR (implemented in MASTOR) | Pedigree | 1.15 | 1.00 | 0.99 | 0.99 |

MQLS1972 |
MQLS (using 1972 genotyped individuals) | Pedigree | 1.15 | 1.01 | 0.99 | – |

MQLS3626^{,} |
^{,} |
Pedigree | 1.16 | – | – | – |

MQLS1972_E | MQLS using 1972 genotyped individuals and estimated kinships | Estimated | 0.94 | 0.90 | 0.91 | – |

RT1972 |
ROADTRIPS (using 1972 genotyped individuals) | Pedigree & estimated | 1.00 | 1.00 | 0.99 | – |

RT3626^{,} |
ROADTRIPS (using all 3626 individuals with or without genotype data) | Pedigree & estimated | 1.00 | – | – | – |

FBATaff, MQLS and ROADTRIPS are only applicable to binary traits and so do not have results in the ‘Simulated quantitative’ column.

In the simulated data sets, MQLS and RT could only be based on the 1972 individuals with simulated phenotypes, and so no simulated trait results are displayed in the MQLS3626 and RT3626 rows.

The Brazilian populations studied here are believed to be long-term (

We also used as covariates in a logistic regression analysis the first nine coordinates obtained from a multidimensional scaling (MDS) analysis of the pruned SNPs in PLINK (having considered between one and ten coordinates, nine was the number that minimised the genomic control inflation factor). The resulting genomic control inflation factor was 1.08, considerably smaller than the unadjusted inflation factor of 1.23, but still not perfectly controlled. Inclusion of MDS coordinates as covariates, similar to including principal components scores, might be expected to account for more subtle levels of population substructure than are accounted for by the use of the ADMIXTURE program (and may possibly also indirectly account for relatedness), which perhaps explains the greater success of this procedure. However the fact that LMM approaches based on estimated kinships still do better (with respect to controlling

An intuitive overview of the expected power provided by the different (real and simulated) data sets can be obtained from

Powers (left hand plots) are defined as the proportion of replicates (out of 1000) in which both simulated disease loci are detected, with ‘detection’ corresponding to any SNP within 40 kb of the simulated disease locus reaching the specified

The points marked in red denote the confirmed significant region from Fakiola et al. (2013). FLMM_E = FaST-LMM using exact calculation, MQLS1972 = MQLS using 1972 genotyped individuals, RT1972 = ROADTRIPS using 1972 genotyped individuals, FBATaff = FBAT using transmissions to affecteds only, FBATboth = FBAT using transmissions to both affecteds and unaffecteds. Results from all other LMM methods were indistinguishable from FLMM_E and so are not shown.

Although the LMM (and several alternative) approaches show similar overall levels of power, an interesting separate question is the degree of concordance between the different methods with respect to the association signals detected. In the real data set we found the p-values obtained at each SNP from the different LMM methods to be highly concordant (

The high concordance between the different LMM methods (and, to a slightly lesser extent, between LMM methods and all methods other than FBAT) is also seen for the simulated (weak disease) trait (

Mean (standard deviation) in 1000 replicates of proportion of top |
||||||

Trait | Method |
|||||

sim-D1 | Unadjusted | 0.991 (0.042) | 0.990 (0.030) | 0.981 (0.033) | 0.975 (0.032) | 0.973 (0.027) |

EM_IBS | 0.999 (0.017) | 0.999 (0.009) | 0.997 (0.015) | 0.997 (0.013) | 0.996 (0.012) | |

FLMM_A | 1.000 (0.009) | 1.000 (0.003) | 1.000 (0.007) | 1.000 (0.004) | 1.000 (0.003) | |

FLMM_E | 0.998 (0.021) | 1.000 (0.005) | 0.999 (0.008) | 0.999 (0.005) | 1.000 (0.004) | |

GA_FA | 0.998 (0.018) | 1.000 (0.005) | 0.999 (0.011) | 0.999 (0.008) | 0.998 (0.008) | |

GA_GRG | 0.998 (0.021) | 0.999 (0.011) | 0.996 (0.017) | 0.998 (0.010) | 0.998 (0.008) | |

GMA_C | 0.998 (0.021) | 1.000 (0.004) | 0.999 (0.009) | 0.999 (0.005) | 1.000 (0.004) | |

GMA_S | 0.998 (0.021) | 1.000 (0.005) | 0.999 (0.008) | 0.999 (0.005) | 1.000 (0.004) | |

GTAM | 0.998 (0.022) | 0.995 (0.022) | 0.990 (0.025) | 0.988 (0.022) | 0.987 (0.020) | |

Mendel | 0.997 (0.025) | 0.996 (0.019) | 0.991 (0.024) | 0.989 (0.021) | 0.989 (0.018) | |

MMM_E | 0.991 (0.041) | 1.000 (0.004) | 0.999 (0.009) | 0.999 (0.005) | 1.000 (0.004) | |

MMM_G | 0.993 (0.036) | 1.000 (0.003) | 1.000 (0.007) | 1.000 (0.005) | 0.999 (0.005) | |

FBATaff | 0.684 (0.253) | 0.790 (0.115) | 0.773 (0.090) | 0.771 (0.080) | 0.760 (0.072) | |

FBATboth | 0.859 (0.130) | 0.844 (0.084) | 0.811 (0.078) | 0.795 (0.075) | 0.777 (0.071) | |

MASTOR | 0.993 (0.038) | 0.994 (0.024) | 0.989 (0.027) | 0.985 (0.024) | 0.985 (0.022) | |

MQLS | 0.978 (0.062) | 0.981 (0.040) | 0.960 (0.043) | 0.951 (0.041) | 0.941 (0.038) | |

RT | 0.981 (0.059) | 0.984 (0.037) | 0.962 (0.042) | 0.952 (0.041) | 0.942 (0.038) | |

sim-D2 | Unadjusted | 0.982 (0.060) | 0.984 (0.041) | 0.979 (0.039) | 0.974 (0.040) | 0.973 (0.036) |

EM_IBS | 0.997 (0.029) | 0.997 (0.024) | 0.995 (0.025) | 0.994 (0.028) | 0.994 (0.024) | |

FLMM_A | 0.998 (0.027) | 0.998 (0.024) | 0.997 (0.025) | 0.997 (0.029) | 0.997 (0.026) | |

FLMM_E | 0.995 (0.035) | 0.997 (0.025) | 0.997 (0.025) | 0.996 (0.030) | 0.997 (0.026) | |

GA_FA | 0.992 (0.044) | 0.998 (0.024) | 0.997 (0.026) | 0.996 (0.030) | 0.996 (0.026) | |

GA_GRG | 0.994 (0.038) | 0.997 (0.026) | 0.996 (0.027) | 0.995 (0.030) | 0.996 (0.026) | |

GMA_C | 0.995 (0.035) | 0.997 (0.025) | 0.997 (0.025) | 0.996 (0.030) | 0.997 (0.026) | |

GMA_S | 0.995 (0.035) | 0.997 (0.025) | 0.997 (0.025) | 0.996 (0.030) | 0.997 (0.026) | |

GTAM | 0.988 (0.050) | 0.990 (0.036) | 0.983 (0.037) | 0.982 (0.036) | 0.982 (0.032) | |

Mendel | 0.988 (0.051) | 0.992 (0.033) | 0.986 (0.035) | 0.984 (0.036) | 0.987 (0.031) | |

MMM_E | 0.995 (0.037) | 0.997 (0.025) | 0.997 (0.025) | 0.996 (0.030) | 0.997 (0.026) | |

MMM_G | 0.998 (0.028) | 0.998 (0.024) | 0.997 (0.025) | 0.997 (0.029) | 0.997 (0.026) | |

FBATaff | 0.413 (0.255) | 0.571 (0.201) | 0.614 (0.157) | 0.639 (0.128) | 0.651 (0.102) | |

FBATboth | 0.664 (0.246) | 0.718 (0.146) | 0.699 (0.111) | 0.691 (0.099) | 0.686 (0.088) | |

MASTOR | 0.971 (0.075) | 0.988 (0.038) | 0.981 (0.038) | 0.978 (0.039) | 0.979 (0.033) | |

MQLS | 0.934 (0.107) | 0.962 (0.056) | 0.942 (0.053) | 0.928 (0.051) | 0.917 (0.047) | |

RT | 0.943 (0.099) | 0.965 (0.055) | 0.943 (0.053) | 0.930 (0.052) | 0.919 (0.047) | |

sim-Q | Unadjusted | 0.987 (0.049) | 0.983 (0.038) | 0.962 (0.040) | 0.963 (0.034) | 0.954 (0.033) |

EM_IBS | 0.998 (0.020) | 0.998 (0.016) | 0.993 (0.020) | 0.994 (0.017) | 0.993 (0.015) | |

FLMM_A | 1.000 (0.000) | 1.000 (0.000) | 1.000 (0.004) | 1.000 (0.005) | 1.000 (0.004) | |

FLMM_E | 1.000 (0.009) | 0.999 (0.008) | 1.000 (0.005) | 1.000 (0.005) | 0.999 (0.005) | |

GA_FA | 1.000 (0.006) | 0.999 (0.010) | 0.998 (0.010) | 0.998 (0.010) | 0.996 (0.012) | |

GA_GRG | 0.994 (0.034) | 0.999 (0.010) | 0.995 (0.018) | 0.996 (0.014) | 0.996 (0.012) | |

GMA_C | 1.000 (0.009) | 1.000 (0.007) | 1.000 (0.004) | 1.000 (0.004) | 1.000 (0.004) | |

GMA_S | 1.000 (0.009) | 0.999 (0.008) | 1.000 (0.005) | 1.000 (0.005) | 0.999 (0.005) | |

GTAM | 0.995 (0.032) | 0.991 (0.028) | 0.984 (0.030) | 0.985 (0.024) | 0.984 (0.022) | |

Mendel | 0.998 (0.021) | 0.996 (0.020) | 0.987 (0.027) | 0.988 (0.022) | 0.988 (0.019) | |

MMM_E | 0.899 (0.100) | 0.999 (0.008) | 1.000 (0.004) | 1.000 (0.004) | 1.000 (0.004) | |

MMM_G | 0.903 (0.100) | 1.000 (0.003) | 1.000 (0.003) | 1.000 (0.004) | 1.000 (0.003) | |

FBAT | 0.906 (0.101) | 0.896 (0.067) | 0.869 (0.059) | 0.844 (0.067) | 0.814 (0.066) | |

MASTOR | 0.998 (0.020) | 0.992 (0.027) | 0.984 (0.030) | 0.984 (0.025) | 0.983 (0.023) |

See

Most LMM packages (although not Mendel) allow a separation between the ‘estimation of kinships’ step and the ‘association testing’ step. This is convenient as it allows the user to read in theoretical or estimated kinships as desired, and to consider using an alternative package for estimating kinships to the one used for the actual association testing. We investigated performing an analysis in FaST-LMM (exact calculation), but with the kinships estimated from various different software packages (see

Given that many of the software implementations we investigated (and in particular all the various LMM implementations) showed similar levels of power and type 1 error, and gave rather similar inference in terms of localisation of signals and

With respect to computational speed, as a rule of thumb we found Mendel (theoretical kinships), FaST-LMM (approximate) and GenABEL (GRAMMAR-Gamma) to be the fastest LMM implementations, taking between 3 minutes and a quarter of an hour on our system to analyse 545,433 SNPs in 1972 genotyped individuals. These were closely followed by EMMAX and MMM (approximate) which took around half an hour, GenABEL (FASTA), GEMMA, FaST-LMM (exact) and MMM (exact) which typically took 1–2 hours, Mendel (estimated kinships) which took around 2.5 hours, and GTAM which took around 4 hours. Of the non-LMM methods, FBAT, MQLS and MASTOR were the fastest, taking a few hours to perform the analysis, while ROADTRIPS was the slowest, taking several days. Inputting estimated (rather than theoretical) kinships into MQLS increased the time taken to around 4 days (and appeared to over-correct the genomic inflation, see

Eu-ahsunthornwattana et al.

Trait analysed | ||

Method |
Longitudinal (sim-L20) | Longitudinal polygenic (sim-P20) |

Unadjusted | 20.82 | 21.53 |

EM_BN | 1.01 | 1.01 |

EM_IBS | 0.99 | 0.97 |

FLMM_A | 1.01 | 1.01 |

FLMM_E | 1.01 | 1.01 |

GA_FA | 1.06 | 2.39 |

GA_GRG | 0.66 | 0.47 |

GMA_C | 1.01 | 1.01 |

GMA_S | 1.01 | 1.01 |

MMM_E | 1.01 | 3.52 |

MMM_G | 1.01 | 3.52 |

See

Analysing each repeated measure as if it comes from a different individual treats our data set as a larger ‘pseudo data set’ containing many apparent twins/triplets/quadruplets (actually, in this case, 20-tuplets). Although less satisfactory than a proper longitudinal analysis that takes into account correlations due to both relatedness between individuals and repeated measures within individuals

We also investigated a ‘proper’ longitudinal analysis implemented within the R software package longGWAS

Another program that can, in theory, implement a ‘proper’ longitudinal analysis is the lmekin function within the R package coxme. We found this function to be computationally infeasible for analysis of genome-wide data, but application to a selected set of 2423 SNPs (of different effect sizes) in the sim-L20 data suggested that the results were very similar to those obtained from GenABEL (FASTA), EMMAX, FaST-LMM, GEMMA and MMM. However, we were unable to get lmekin to give meaningful results (most results were “NA”) when applied to the sim-P20 data. We also speculated that a ‘proper’ longitudinal analysis should, in theory, be implementable in the package Mendel

Here we have demonstrated, through simulations and application to real data, that linear mixed model approaches such as those implemented in the packages GenABEL, EMMAX, FAST-LMM, GEMMA and MMM offer a convenient and robust approach for family-based GWAS of quantitative or binary traits, are successful in controlling the overall genomic inflation factor to an appropriate level, and offer higher power than traditional family-based association analysis approaches such as those implemented in FBAT. Similar inference is also provided by related and alternative approaches implemented in the software packages Mendel, ROADTRIPS, MQLS and MASTOR, although our results from analysis of the real data suggest that, for Mendel, MQLS and MASTOR, care may need to be taken to use estimated kinships based on SNP data rather than known pedigree relationships, if one is to avoid any inflation in the test statistics.

Our current study focused mostly on family data in which genuine close relationships between many individuals exist. Nevertheless we found similar results with respect to the LMM methods investigated (adequate control of type 1 error and extremely similar performance in terms of power and concordance between top findings) when applied to a subset of 462 founder individuals from our pedigrees, selected to be approximately unrelated to one another (see

Traditional methods for family-based association analysis make use of pedigree relationships either (e.g. FBAT) through direct use of known pedigree structure or else (e.g. MQLS, ROADTRIPS and all LMM methods) through use of a covariance matrix that involves the known kinship between each pair of individuals (the probability that a randomly chosen allele at a locus in each individual is identical by descent i.e. is a copy of a common ancestral allele, under the assumption that the pedigrees are correctly specified and all founders in a pedigree are completely unrelated i.e. share no alleles identical by descent). The assumption that all founders in a pedigree share no alleles identical by descent is clearly a fiction, given human population history, while the assumption that all pedigrees are correctly specified and unrelated to one another is also likely to be violated in most real studies. The use of estimated kinships based on SNP data rather than theoretical kinships based on known pedigree relationships removes the reliance on these untenable assumptions, and allows essentially the same analysis approaches to be applied to apparently unrelated individuals (who may nevertheless display distant levels of shared ancestry). The question then arises as to what exactly these estimated kinships (or related measures) are actually measuring? We consider a detailed discussion of this issue to be beyond the scope of the current manuscript, but we refer the reader to the more detailed expositions given in

The recent popularity of LMM approaches for the analysis of apparently unrelated individuals

An interesting finding of our current study was the fact that longitudinal traits (repeated measures) could be successfully analysed in an LMM framework simply by treating each measurement as if it came from a separate person and expanding out the genetic data set accordingly (resulting in an expanded data set containing many apparent twins, triplets, quadruplets etc.). From a practical point of view this is useful as analysis of an expanded data set in standard LMM software is computationally convenient; we found a ‘proper’ analysis using software such as longGWAS

A caveat to all the results presented here is that they relate to genotypes derived from a single data set, our Brazilian family study of visceral leishmaniasis

On this note, we point out that each package has its own particular advantages (and disadvantages). These include the fact that EMMAX, GEMMA and MMM allow the input of dosages derived from imputed (in addition to real) genotypes; MMM has the advantage of allowing the output of regression coefficients and standard errors for the SNP effects on the (log) odds ratio scale, making it convenient to compare or combine the results with results from traditional case/control studies analysed via logistic regression; GenABEL (GRAMMAR-Gamma) has the advantage of scaling linearly with sample size, which makes it attractive for the analysis of very large data sets; FaST-LMM has the advantage, along with EMMAX and Mendel, of internally imputing missing data at any (genetic or non-genetic) covariates, which can make it convenient for implementing stepwise conditional analyses; and, unlike most LMM implementations, ROADTRIPS, MQLS and MASTOR have the advantage of using all phenotype information, including that for individuals that have not been genotyped, which can in theory generate a small increase in power.

One of the main differences between the different software implementations we investigated was the time taken to perform the analysis (not including the time required to re-format data into an appropriate format for a given package). We were unable to do a strict head-to-head comparison as the precise timings depend on a number of factors including the computer architecture available (in particular the ability of the system and of any given program to allow multi-threading and/or parallel processing), however our rough comparison (

In conclusion, we recommend linear mixed model approaches as a convenient and powerful approach for family-based GWAS of quantitative or binary traits. We find these approaches to be successful in controlling the overall genome-wide error rate and to perform well in comparison to competing approaches.

Ethical approval for the Belem Family Study was obtained originally from the local ethics committee at the Instituto Evandro Chagas, Belém, Para, Brazil. Approval for continued use of the Belem Family Study samples, and for collection and use of the samples from Natal, has been granted from the local Institutional Review Board at the Universidade Federal do Rio Grande do Norte (CEP-UFRN 94–2004), nationally from the Comissão Nacional de Ética em Pesquisa (CONEP: 11019), and from the Ministerios Cencia e Tecnologia for approval to ship samples out of Brazil (portaria 617; 28 September 2005). Informed written consent for sample collection was obtained from adults, and from parents of children

Sample collection and genotyping of the Brazilian subjects used here is described in detail in

For the majority of analyses considered here, we used either the 1972 genotyped individuals or else the entire set of 3626 individuals (with or without genotype data) that are required to define the ‘known’ (theoretical) pedigree relationships. For power comparisons between LMM methods, we also investigated use of a subset of 462 ‘founder’ individuals, chosen on the basis of theoretical relationships and estimated kinships to be approximately unrelated to one another.

We generated simulated phenotypes for the 1972 individuals that had genome-wide SNP data available. We used two different models for generating binary (disease) traits, one corresponding to ‘strong’ genetic effects (sim-D1) and one corresponding to ‘weak’ genetic effects (sim-D2), with the trait in both cases governed by two SNPs (rs9271252 and rs233722) located on chromosomes 6 and 12 respectively. In addition to modelling genetic effects at rs9271252 and rs233722, we allowed for 22 weaker ‘polygenic’ effects caused by genotype at the 100th SNP on each autosomal chromosome. Each effect contributed multiplicatively to the probability of developing disease. Thus, the mathematical model for generating the simulated phenotype was _{j}

We also simulated a model (sim-Q) for quantitative traits, again governed by rs9271252 and rs233722 on chromosomes 6 and 12. The traits were generated as a linear combination of the effect from each of the strong and polygenic effect SNPs, with a normally distributed error component, thus: _{ij}

We simulated a model (sim-L20) for longitudinal quantitative traits (with

The baseline trait

To make the analyses feasible whilst still maintaining the overall degree of relatedness, the longitudinal data set was constructed based on a subset of 498 individuals selected through stratified sampling from the original data set, with number of individuals randomly selected from each extended family approximately proportional to their family size while also ensuring that every family is represented by at least one individual. Phenotypes for these 498 individuals were then generated 20 times to create the final longitudinal data set.

In addition we simulated a purely polygenic longitudinal model (sim-P20) in which the strong effects

We generated 1000 replicates of each simulated data set, apart from the longitudinal and polygenic longitudinal data sets for which we only simulated a single replicate. For visualisation of results from a whole genome scan, we analysed only a single replicate (replicate 1). For investigation of power, type 1 error and concordance, to reduce computation time we analysed all 1000 replicates but only generated test statistics at 40 SNPs that lay within 40 kb of the simulated disease loci (for evaluation of power) and 20 SNPs that lay well outside the region of any simulated disease loci (for evaluation of type 1 error). By default, the programs Mendel and ROADTRIPS require all SNPs that are being used to estimate genome-wide relatedness to also be read in and tested for association; to perform the analysis of all 1000 replicates in reasonable time we therefore included the 50,129 ‘pruned’ SNPs rather than the full genome-wide set of SNPs that would normally be used by these programs.

All the LMM implementations evaluated here attempt to fit either an exact or an approximate version of the standard linear mixed model:

The

Rather than fitting the full linear mixed model

The

In GRAMMAR _{i}

In the original GRAMMAR publication

The GRAMMAR-Gamma method _{GRG} = _{new}/γ that can be shown to be approximately equivalent to the FASTA statistic

Kang et al.

In the approach of Kang et al.

In the approach of Kang et al.

Although not entirely clear from the description in Kang et al.

The method of Kang et al.

A similar approach to

Lippert et al.

FaST-LMM uses either maximum likelihood (ML) or restricted maximum likelihood (REML). In early versions of FaST-LMM the default was ML but in later versions the default became REML. After some experimentation, we deemed ML to be the most reliable and have used that for all results presented here.

Zhou and Stephens

Pirinen et al.

An approximate (score test) LMM implementation, suitable for analysis of GWAS data, has also been implemented in the software package Mendel

Traditional approaches for family-based association analysis focus on the transmission of high-risk alleles through pedigrees, in an approach that is closely related to traditional linkage analysis. Indeed, the well-known transmission disequilibrium test (TDT)

The TDT was originally designed for the analysis of case/parent trios (i.e. units consisting of an affected child together with their parents) but has been extended to allow analysis of nuclear families and larger pedigrees _{ij}_{ij}_{ij}_{ij}

Although, for binary traits, contrasting transmissions to affecteds with transmissions to unaffecteds seems an attractive idea, in practice this results in comparing the probability of transmission of high-risk alleles to affected individuals (which is expected, under the alternative hypothesis, to exceed 0.5) with an

By default, FBAT divides larger pedigrees into nuclear families and constructs a test that corresponds to testing ‘linkage in the presence of association’

Thornton and McPeek

The ROADTRIPS test statistic takes the form:

Thornton and McPeek note that many commonly-used case/control statistics can be coerced into this form. Here

Recently, Jakobsdottir and McPeek

The LMM approaches considered here, as well as methods such as MQLS, ROADTRIPS, MASTOR and GTAM, all involve modelling the relatedness between individuals through one or more kinship matrices, constructed either on the basis of known (hypothesized) pedigree relationships between individuals, or through estimating kinships on the basis of genome-wide SNP data (or from a subset of available genome-wide SNPs). The precise algorithms used to estimate kinships on the basis of genome-wide SNP data vary

We also explored the use of a smaller set of 1900 ‘thinned’ SNPs to estimate kinships. This number was chosen to capitalise on the speed-up that can be achieved in FaST-LMM by restricting the number of SNPs used to construct the kinship matrix

Several alternative packages exist for estimating genetic relationships from genome-wide SNP data, either for subsequent use in LMM type analyses

Comparison of estimated kinship measures and −log10(p-values) obtained based on full, pruned and thinned SNPs. (A) Estimated kinship measures (B)

(TIF)

QQ plots of real VL phenotype GWAS results, using different LMM software packages and different SNP sets for kinship estimation. The black diagonal lines represent the line of equality. The “theoretical” set used pedigree structure to derive theoretical kinship coefficients. EM_BN = EMMAX (Balding-Nichols), EM_IBS = EMMAX (IBS method), FLMM_C = FaST-LMM using covariance matrix, FLMM_R = FaST-LMM using realised relationship matrix, GA_FA = GenABEL (FASTA), GA_GRG = GenABEL (GRAMMAR-Gamma), GMA_C = GEMMA using centred genotypes, GMA_S = GEMMA using standardised genotypes, MMM_E = MMM using full mixed model (exact) calculation, MMM_G = MMM using GLS approximation, Unadj = unadjusted analysis. For methods with two ways to estimate the kinships, the same “theoretical” results were plotted twice. Unadjusted analysis results were plotted once in each column only for comparison, and did not use the kinship estimates for adjustment.

(TIF)

Performance of FaST-LMM-Select. Genomic control factor (

(TIF)

Manhattan plots for real and simulated data sets using FaST-LMM. The points marked in red denote either the confirmed significant region from Fakiola et al. (2013) (real phenotype), or the regions close to the simulated strong/weak effect SNPs (simulated phenotypes). real = real VL phenotype, sim-D1 = simulated strong binary (disease) trait, sim-D2 = simulated weak binary (disease) trait, sim-Q = simulated quantitative trait, sim-L20 = simulated longitudinal quantitative trait with 20 observations, sim-P20 = simulated polygenic longitudinal quantitative trait with 20 observations.

(TIF)

Manhattan plots for the simulated weak binary (disease) phenotype using FaST-LMM exact and alternative software packages. The points marked in red denote the regions close to the simulated weak effect SNPs. FLMM_E = FaST-LMM using exact calculation, RT = ROADTRIPS, FBATaff = FBAT using transmissions to affecteds only, FBATboth = FBAT using transmissions to both affecteds and unaffecteds. Results from all other LMM methods were indistinguishable from FLMM_E and so are not shown. MQLS and RT gave identical results with either 1972 or 3626 individuals, as phenotypes could only be simulated for the 1972 genotyped individuals.

(TIF)

Manhattan plots for the simulated strong binary (disease) phenotype using FaST-LMM exact and alternative software packages. The points marked in red denote the regions close to the simulated weak effect SNPs. FLMM_E = FaST-LMM using exact calculation, RT = ROADTRIPS, FBATaff = FBAT using transmissions to affecteds only, FBATboth = FBAT using transmissions to both affecteds and unaffecteds. Results from all other LMM methods were indistinguishable from FLMM_E and so are not shown. MQLS and RT gave identical results with either 1972 or 3626 individuals, as phenotypes could only be simulated for the 1972 genotyped individuals.

(TIF)

Comparison of −log10(p-values) using different LMM software packages, real disease phenotypes. Plots above the diagonal show a comparison of −log10(p-values), with correlations between the -log10(p-values) indicated below the diagonal. The grey solid lines represents the line of equality; the black dashed lines the linear regression line of the variable on the y axis on the variable on the x axis. EM_BN = EMMAX (Balding-Nichols), EM_IBS = EMMAX (IBS method), FLMM_A = FaST-LMM using approximate calculation, FLMM_E = FaST-LMM using exact calculation, GA_FA = GenABEL (FASTA), GA_GRG = GenABEL (GRAMMAR-Gamma), GMA_C = GEMMA using centred genotypes, GMA_S = GEMMA using standardised genotypes, MMM_E = MMM using full mixed model (exact) calculation, MMM_G = MMM using GLS approximation, Unadj = unadjusted analysis.

(TIF)

Comparison of −log(p-values) using LMM and alternative software packages, real disease phenotypes. Plots above the diagonal show a comparison of −log10(p-values), with correlations between the −log10(p-values) indicated below the diagonal. The grey solid lines represent the line of equality; the black dashed lines the linear regression line of the variable on the y axis on the variable on the x axis. FLMM_E = FaST-LMM using exact calculation, MQLS1972 = MQLS using 1972 genotyped individuals, MQLS3626 = MQLS using all 3626 individuals with or without genotype data, RT1972 = ROADTRIPS using 1972 genotyped individuals, RT3626 = ROADTRIPS using all 3626 individuals with or without genotype data, FBATaff = FBAT using transmissions to affecteds only, FBATboth = FBAT using transmissions to both affecteds and unaffecteds, MQLS_E = MQLS using estimated (rather than theoretical) kinships.

(TIF)

Comparison of −log(p-values) using LMM and alternative software packages, simulated weak binary (disease) phenotype. Plots above the diagonal show a comparison of –log10(p-values), with correlations between the –log10(p-values) indicated below the diagonal. The grey solid lines represent the line of equality; the black dashed lines the linear regression line of the variable on the y axis on the variable on the x axis. The colours denote: red = the two weak effect SNPs, magenta = SNPs within 500 kb of the weak effect SNPs, blue = 22 polygenic SNPs, green = SNPs within 500 kb of the polygenic SNPs, black = all other SNPs. Because the black/green/blue SNPs were plotted before the magenta/red SNPs, they may be obscured by the latter. FLMM_E = FaST-LMM using exact calculation, MQLS = MQLS using 1972 or 3626 individuals, RT = ROADTRIPS using 1972 or 3626 individuals, FBATaff = FBAT using transmissions to affecteds only, FBATboth = FBAT using transmissions to both affecteds and unaffecteds. MQLS and RT gave identical results with either 1972 or 3626 individuals, as phenotypes could only be simulated for the 1972 genotyped individuals.

(TIF)

Comparison of −log10(p-values) obtained from FaST-LMM using alternative kinship estimates, real disease phenotypes. Plots above the diagonal show a comparison of –log10(p-values), with correlations between the –log10(p-values) indicated below the diagonal. The grey solid lines represents the line of equality; the black dashed lines the linear regression line of the variable on the y axis on the variable on the x axis. KING_H = KING homogeneous method, KING_R = KING robust method, Ped = theoretical kinship estimates based on pedigree information, FLMM_R = FaST-LMM's own realised relationship matrix, Unadj = unadjusted, Wrong = misspecified kinships, chosen to be inversely related to the true kinship value.

(TIF)

Power and type 1 error of different LMM methods applied to 462 Brazilian founders. Powers (left hand plots) are defined as the proportion of replicates (out of 1000) in which both simulated disease loci are detected, with ‘detection’ corresponding to any SNP within 40 kb of the simulated disease locus reaching the specified

(TIF)

Genomic control factors achieved in analysis of the real data, or a single replicate of the simulated data, when feeding externally estimated kinships into FaST-LMM.

(PDF)

Computational speed and ease of use of various packages.

(PDF)

Concordance between top SNPs identified by different LMM methods when using 462 founder individuals.

(PDF)

Membership of Wellcome Trust Case Control Consortium 2.

(DOC)