As the scope of microbial surveys expands with the parallel growth in sequencing capacity, a significant bottleneck in data analysis is the ability to generate a biologically meaningful multiple sequence alignment. The most commonly used aligners have varying alignment quality and speed, tend to depend on a specific reference alignment, or lack a complete description of the underlying algorithm. The purpose of this study was to create and validate an aligner with the goal of quickly generating a high quality alignment and having the flexibility to use any reference alignment. Using the simple nearest alignment space termination algorithm, the resulting aligner operates in linear time, requires a small memory footprint, and generates a high quality alignment. In addition, the alignments generated for variable regions were of as high a quality as the alignment of full-length sequences. As implemented, the method was able to align 18 full-length 16S rRNA gene sequences and 58 V2 region sequences per second to the 50,000-column SILVA reference alignment. Most importantly, the resulting alignments were of a quality equal to SILVA-generated alignments. The aligner described in this study will enable scientists to rapidly generate robust multiple sequences alignments that are implicitly based upon the predicted secondary structure of the 16S rRNA molecule. Furthermore, because the implementation is not connected to a specific database it is easy to generalize the method to reference alignments for any DNA sequence.
Citation: Schloss PD (2009) A High-Throughput DNA Sequence Aligner for Microbial Ecology Studies. PLoS ONE 4(12): e8230. https://doi.org/10.1371/journal.pone.0008230
Editor: John Quackenbush, Dana-Farber Cancer Institute, United States of America
Received: July 8, 2009; Accepted: November 12, 2009; Published: December 14, 2009
Copyright: © 2009 Schloss. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This project was funded by the College of Natural Resources and the Environment at the University of Massachusetts, the School of Medicine at the University of Michigan, a grant from the Sloan Foundation, and a grant from the National Science Foundation (award #0743432). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The author has declared that no competing interests exist.
Recent advances in traditional Sanger sequencing and pyrosequencing technologies have facilitated the ability to design studies where 102−107 16S rRNA gene sequences ranging in length between 60 and 1500 bp are generated to address interesting ecological questions –. This data gush has forced computational microbial ecologists to re-factor software tools to make the analysis of these datasets feasible. A significant bottleneck in the analysis of these sequences is the generation of a robust multiple sequence alignment (MSA). An MSA is critical to generating phylogenies and calculating meaningful pairwise genetic distances that can be used to assign sequences to operationally-defined taxonomic units [OTUs, 5]. Because of the difficulty inherent in MSA calculations, investigators have bypassed OTU-based approaches in preference for phylotype-based approaches , . In such approaches, sequences are assigned to bins based on similarity to a curated database. This has the limitation that sequences in the same phylotype may be only marginally similar to each other or unknown sequences may not affiliate to a pre-existing taxonomy. Therefore, there is a significant need to reassess alignment techniques with regard to their speed, memory requirements, and accuracy.
For generic sequencing alignments, popular aligners have included ClustalW , MAFFT , and MUSCLE . Several recent pyrosequencing studies of the V6 16S rRNA region (ca. 60 bp long) have used MUSCLE to generate MSAs for up to 20,000 sequences , , . These techniques scale at least quadratically in space and time for sequence length and quadratically in space and to the third power in time for the number of sequences. Thus, as the number of sequences in a dataset surpasses their length, the memory required to double the number of sequences in an alignment increases at least four-fold and the time required increases at least eight-fold. Because these limitations are compounded in typical implementations by storing all of the data in RAM, it is not possible to align more than 5,000 full-length sequences on a typical desktop computer. Alternatively, some have proposed calculating genetic distances using only pairwise alignments . The time requirements of such an approach scale quadratically with the number of sequences and makes it impossible to insure positional homology. An additional limitation of the generic sequence aligners is that the alignments do not incorporate the predicted secondary structure of the 16S rRNA molecule and therefore it is impossible to compare datasets without re-aligning all of the sequences.
The secondary structure is an important feature to consider in generating the alignment because it increases the likelihood that the alignment conserves positional homology between sequences . Without such a consideration, the alignment is more sensitive to user-supplied parameters such as match and mismatch scores, and gap opening and extension penalties. There are currently four profile-based aligners that are used to generate 16S rRNA-specific alignments that each at least implicitly considers the secondary structure. Each of these methods is associated with well-established 16S rRNA gene databases and reference MSAs, which each have strengths and weaknesses. A general advantage of each of these methods is that rather than generating alignments de novo, they perform profile-based alignments and their complexity scales linearly in time and have a minimal memory footprint. In deciding upon an aligner it is important to consider the alignment quality, ability to align large datasets, speed, dependence on a specific database, cost, and openness of the algorithm.
The RDP (http://rdp.cme.msu.edu) aligner uses 16S rRNA secondary structure models to generate and apply hidden Markov models within Infernal , . Although this has the strength of directly incorporating the secondary structure of the 16S rRNA molecule, the number of available models is limited, which causes variable regions to not be aligned. Furthermore, while the aligner scales linearly in time (i.e. doubling the number of sequences doubles the time required to construct the MSA), the alignment process is relatively slow compared to other methods. Finally, the length and structure of the RDP alignment changes as new reference models are included. This requires users to re-align their data each time they acquire new sequences. The strengths of the RDP aligner are that it is free, open source, and can be run on a user's local computer.
The popular software package, ARB , has a built-in aligner that has yet to be fully described in the literature. Although specific details are lacking, the aligner uses a suffix tree to find related reference sequences and that the actual alignment step uses multiple reference sequences and secondary structure information to carryout the alignment. Perhaps the most significant limitation in the ARB implementation is that suffix tree server has become practically unusable to most users as the number of full-length sequences has increased. As an alternative, the ARB developers have spun-off the SILVA database project (http://www.arb-silva.de). SILVA serves as a repository for aligned rRNA sequences and the SINA aligner . This implementation of the ARB aligner is more convenient, but also has yet to be described in the literature. Complicating matters is that the website limits users to aligning 300 sequences at a time; aligning additional sequences is available on a pay-for-use basis. Although the MSA length and structure is stable, it is an unwieldy 50,000 columns long.
The aligner available through the greengenes website (http://greengenes.lbl.gov) is not explicitly dependent on secondary structure models; however, the generation of the reference database alignment does take into account the secondary structure (Fig. 1) , . The reference alignment is considerably shorter than the SILVA alignment (7,682 columns). Although the source code for the greengenes aligner is not open, the algorithm has been published . The original implementation used kmer searching with 7-mers to identify the closest template sequence in the reference database. The current implementation uses blastn  to identify the longest template sequence among the top-ten matches (TZ DeSantis, personal communication). In the second step, the algorithm uses blastn to generate pairwise alignments between the unaligned candidate and template sequences; in the current greengenes implementation the same blastn alignment generated in the first step is used for the second step. Finally, gap positions are introduced to the candidate sequence so that the final alignment is the same length as the reference database and positional homology is maintained using the nearest alignment space termination (NAST) algorithm. The speed of the aligner scales linearly with the number of candidate sequences so that doubling the number of candidate sequences would double the time requirement. The only significant memory requirement is what is required to store the reference alignment. A challenge in each of these profile-based methods is the creation of a high-quality reference alignment because the alignments that are generated will only be as good as the reference. Assuming that one has a good reference alignment, the greengenes aligner appears to have the most potential for quickly generating a high quality sequence alignment.
The published and current greengenes aligner algorithm is shown in black and the modifications that were tested in this study are shown in blue.
In the present study I used several simulations to assess how various permutations of the alignment algorithm effect alignment quality and speed. Specifically, I tested various methods of identifying the best template sequence and completing the pairwise alignments (Fig. 1). I was also interested in determining how well these results generalized to various regions within the 16S rRNA gene sequence commonly used in recently published surveys. This study enabled me to produce an aligner that rapidly produced high-quality alignments, robust to analysis of sequence fragments, could be generalized to genes other than 16S rRNA, and made independent of a specific database.
Comparison of Database Quality
I identified 200,433 unique, high-quality, aligned 16S rRNA gene sequences that were shared by the SILVA, greengenes, and RDP databases. ARB databases provided by SILVA and greengenes include helical mapping data that indicate which bases pair with each other within the predicted 16S rRNA secondary structure. For example, using the SILVA alignment, the 1,542 bp E. coli 16S rRNA secondary structure consists of 1,028 bases that are expected to form pairs. Of these, 808 form normal Watson-Crick base pairs (i.e. AT, GC; marked with a ‘∼’ in ARB), 202 form weak pairs (i.e. GA, TT, GT; marked ‘−’, ‘ = ’, or ‘+’), and 18 do not pair (i.e. AA, AC, CC, CT, GG; marked ‘#’). Although sequences naturally have non-pairing bases within the paring regions of the secondary structure, an excessive number of these is an indicator of poor sequence alignment (e.g. Fig. 2). With this in mind, I compared the number of non-paring bases in each sequence from the SILVA and greengenes databases. On average, each sequence alignment had 18.0 more non-paring bases (sd = 16.0) in the greengenes alignment than in the SILVA alignment; only 5.8% of the greengenes aligned sequences had fewer non-pairs than the SILVA aligned sequences. It was not possible to perform a similar analysis with the RDP alignment because similar helical mapping data are not available. Rather, I counted the number of unaligned bases in each sequence, which they indicate in a lower case typeface to characterize the RDP alignment (Fig. 2). On average, 7.6% of each sequence's bases were unaligned (sd = 1.5%) and within the variable regions targeted by pyrosequencing the percentage was higher. Based on these analyses, I decided to use the SILVA MSA to evaluate the new aligner.
Alignments were taken between positions 60 and 113 of the E. coli 16S rRNA gene sequence for E. coli and four Enteroccocus spp. The alignment generated for these sequences within this region using 8-mers and the Needleman-Wunsch algorithm was identical to that found in the SILVA alignment. The lower-case bases in the RDP alignment indicate unaligned positions. For the greengenes and SILVA alignments, yellow-highlighting represents bases that are predicted to form traditional Watson-Crick base-pairs in the secondary structure, gray-highlighting represents weak base-pairs, black-highlighting represents bases that will not form base-pairs, and a lack of highlighting represents bases that are predicted to be in loop structures.
Database Searching Methods
The first step in the alignment algorithm is to find the most similar template sequence for each unaligned candidate sequence. The ARB aligner uses a suffix tree search method and the greengenes aligner has used either a kmer searching method or blastn. Suffix tree and kmer searching have the advantage that they are alignment independent techniques, which could make them considerably faster than using the alignment-based blastn approach. The extent of the speed-up and the effect on accuracy were unknown. To assess the accuracy, I calculated the similarity between each SILVA-aligned candidate sequence and each of the template sequences. These similarities were used to assess how well each method identified the best match and resulting alignment quality.
Regardless of the region within the 16S rRNA gene, kmer searching outperformed both blastn and suffix tree searching in its ability to find the best template (Tables 1 and S1). When searching against full-length templates, 7 to 9-mers provided the closest matches. Across the 10 regions that I tested, between 68 and 77% of the candidates found their true best template match. With the exception of the V6 region, the average candidate sequence found a template sequence that was between 3.7 and 7.5% different from the optimal template when using kmer searching; the V6 candidates averaged a difference of 12.5% from their optimal template. A difference of 10% between V6 fragments is comparable to a 3% difference over the full length of the gene (unpublished data); therefore a 12.5% difference is not out of line. The suffix tree searches only found the best template sequence for 25–62% of the candidate sequences. With the exception of the V6 region, the average candidate sequence found a template sequence that was between 7 and 17% different from the optimal template when using suffix tree searching; the V6 candidates averaged a difference of 37% from their optimal template. Using blastn, I found that for every region except the V14 and V19, many candidate sequences could not find a significant match to the template database (Table 2). For example, 106 (0.06%) of the V2 and 64,389 (35%) of the V6 candidates could not find a significant match. Although this may be due to the use of a large blastn word size, more sensitive searches with smaller word sizes became to slow to be practical. The blastn approach found the best template for between 17 and 63% of the candidate sequences in each region.
With respect to the time required for each approach, kmer searching outperformed the other methods regardless of the region being investigated (Table 1 and S1). In general kmer sizes of 7 or 8 were typically the fastest. Also, speed scaled with the candidate sequence length. For example, approximately 1,490 V6, 192 V23, and 52 V19 candidates sequences could be searched per second. Suffix tree searching required between 25 and 65 times longer than kmer searching. Blastn searches required between 25 and 70 times longer than kmer searching. While these speeds are implementation-dependent, the stark differences in speed and quality indicate that kmer searching was superior to other methods. While slight improvements are possible by tailoring the kmer size to the region or specific gene of interest, in general, 8-mers provided the best and fastest alignments for 16S rRNA gene fragment sequences.
I also investigated the effect of using template sequences that corresponded to the region being aligned on whether there was an improvement in accuracy or speed (Table S2). There were no significant improvements in accuracy for any of the methods when using the customized template sequences. The most noticeable effects of shorter templates were the optimal kmer size and speed. For all regions the optimal kmer size decreased from 7 to 9-mers to 5 to 7-mers. Using region-specific template sequences increased the search speed by up to 56%.
Pairwise Alignment Methods
I investigated the accuracy and speed of the various alignment methods when the true best template sequence was selected for each candidate sequence (Tables 3 and S3). I tested the Needleman-Wunsch and Gotoh global alignment algorithms, which only differ in the number of parameters they use to penalize gaps. It was expected that Gotoh would be slower, but more accurate than the Needleman-Wunch algorithm because it uses an extra parameter. I also tested blastn, a local alignment method that approximates the Smith-Waterman algorithm . The greengenes aligner currently implements blastn to carryout pairwise alignments. This approach was expected to be the fastest, but perhaps least accurate of the three methods. Furthermore, because it is a local alignment approach, it was expected to trim the ends of sequences that were sufficiently different from their template.
I rewarded matches and penalized mismatches with one point each and varied the gap opening and extension penalties to identify the best alignment conditions. Surprisingly, there was little variation in alignment quality regardless of the algorithm or conditions selected (Tables 3 and S3). With the exception of the V6 region, the realigned candidate sequences were, on average, 0.04–0.66 percentage points more similar to the template sequence than the SILVA-aligned candidate sequences regardless of the algorithm or penalties employed. For the V6 region, blastn produced alignments that were 1.64–1.85 percentage points worse; however the Needleman-Wunsch and Gotoh algorithms produced alignments that were 0.30 to 0.63 percentage points better. Although the effects of the different penalty schemes were minimal, the best alignments were produced using the Needleman-Wunsch algorithm with a gap penalty of 2 for all regions except the V6, which had an optimal gap penalty of 1. As expected, the blastn algorithm tended to truncate the candidate sequence alignment more frequently than the global alignment algorithms. For example, the optimal alignment conditions for the V6 region aligned each of the candidate sequences completely. In contrast, the optimal blastn condition aligned an average of 95.4% of each sequence (s.d. = 18.7%).
Universally, the Needleman-Wunsch algorithm was the fastest of the three methods followed by the Gotoh and blastn approaches. My implementation of the Needleman-Wunsch algorithm aligned between 21 and 73 sequences per second. Similar to the kmer searching implementation, these speeds were affected by candidate sequence length. Using region specific template sequences had no significant effect on alignment quality but increased the pairwise alignment speed by up to 116% (Table S4).
Assessing the Overall Algorithm
The data in Tables 1, S1, and S2 indicated that the various search methods did not necessarily identify the best template sequence. Yet in the second set of simulations, I used the true best template to investigate the various alignment options and found that the alignments were as good as the SILVA alignments (Tables 2, S3, and S4). In the next set of simulations, I selected the best kmer size and alignment parameters for each region to evaluate the overall process (Table 4). On average the resulting alignments were between 0.61% worse and 0.34% better than the alignments generated by SILVA when using full-length template sequences. Although speed was affected by candidate sequence length, between 18 and 78 sequences were aligned per second. When using the region-specific template sequences, the alignments were between 0.07% worse and 0.40% better than the alignments generated by SILVA. The region-specific templates increased the overall alignment speed so that between 37 and 145 sequences were aligned per second.
Effect of Full-Length Alignment Length
This study used a 50,000-column MSA alignment for a gene that is approximately 1,500-bp long. In comparison, the greengenes alignment has 7,682 columns. The extended SILVA alignment has been justified by a desire to include archaeal 16S rRNA and eukaryotic 18S rRNA gene sequences as well as to provide sufficient padding for new sequences that may have long insertions. To simulate a greengenes-like alignment, I removed any column from the SILVA alignment only contained gap characters. This produced a 6,806-column alignment. Using the transformed template alignment sequences, I found that the alignment quality did not vary considerably from the results obtained with the 50,000-column MSA (Table 4). Furthermore, although the same number of bases were represented in each sequence, the speeds observed using vertical gap-filtered MSA were up to 160% faster than using the full 50,000 character alignment.
Comparison to the Greengenes Implementations
I implemented the approaches that have been used by greengenes to assess the sensitivity of the overall algorithm. Using the original approach of employing 7-mer searching combined with blast alignments I found that with the exception of the V6 region, the alignments were between 0.19% worse and 0.36% better than the SILVA alignments (Table 5). These values are somewhat deceiving as I found that, with the exception of the V6 region, between 0.8 and 4.8% of the sequence in each region had less than 95% of its bases aligned. This version of the algorithm aligned between 15 and 25 sequences per second. Although this is considerably slower than what I observed using with the optimal conditions it is considerably faster than the 10 sequences per minute (i.e. 0.17 sequences per second) that was described previously . To mimic the current greengenes implementation, I used blastn to find the closest match. This slightly improved the overall alignments over the original greengenes implementation so that, with the exception of the V6 region, the average alignment was between 0.27% worse and 0.69% better than the SILVA alignments. Again, these values are deceiving, as between 1.1 and 5.4% of the sequences had fewer than 95% of their bases aligned in each sequence. The greengenes aligner approach suffered when aligning the V6 region because of is dependence on blastn. When using 7-mers and blastn the candidate sequences averaged 4.6% (s.d. = 19.5%) worse than the SILVA candidate sequences and more than 18.0% of the sequences had less than 95% of its bases aligned. Considering 35% of the V6 candidate sequences could not be matched to a template via blastn, the current greengenes implementation was ineffective.
A critical step in analyzing DNA sequences generated from community surveys is generating a MSA. Here I described and validated a variation of the greengenes and SILVA aligners and showed that this aligner quickly generates a high-quality alignment. Also, although investigators are encouraged to perform similar types of experiments to optimize the alignment conditions for their region of interest, the kmer search and Needlema-Wunsch alignment approach was robust to perturbations in their settings. Interestingly, whereas other aligners appear to use multiple template sequences to align one candidate sequence, this alignment algorithm only requires one reference sequence per candidate sequence and does require explicit knowledge of the 16S rRNA secondary structure.
An important consideration in selecting a reference alignment is its underlying quality, yet manually curating a reference alignment is a tedious and painstaking process. It is a common practice to mask hypervariable regions when generating a deep-level phylogeny . Such a practice enables one to ignore the alignment of the masked out regions, which is typically where the problematic areas are located. However, when assigning sequences to OTUs or using phylogenies for community-based hypothesis tests, the fine level of detail contained within these variable regions is significant and should not be removed. The original Lane mask removes, on average, 14% of the bases from a full-length sequence alignment of E. coli's full-length 16S rRNA gene . Considering the typical pyrosequencing-based study focuses on these variable regions and the sequence reads generated by pyrosequencing tend to be shorter than 250 bp, it is important that these regions not be discarded. While some have mistakenly used such masks prior to performing other analyses , , –, a better practice would be to manually curate the reference alignment in these regions and to include all of the data. Ultimately, the quality of an alignment will only be as good as reference alignment, regardless of the algorithm.
Because this aligner is not tied to a particular database alignment, this aligner can be used with any reference aligner whether the DNA represent rRNA genes or protein coding genes [e.g. 26]. The first step in such an analysis is to generate a de novo multiple sequence alignment using software such as Clustal, MUSCLE, or MAFFT. Second, the new reference alignment should be curated to insure positional homology across the alignment. Third, unaligned sequences could then be aligned to the reference alignment using a variety of parameters. Finally, manual inspection of the newly aligned sequences should reveal the optimal parameters. Considering the general lack of sensitivity of the method to variations in the various parameters it is likely that the parameters described here will work for other genes.
As microbial ecologists continue to generate massive data sets, it is important to continually refine and validate every step in the analysis pipeline. Central to this ideal is the availability of open source software and peer-reviewed methods. The combined generation of a fast, parallelized, open source, and flexible aligner with the simulations performed in this study demonstrate that this tool will be a valuable contribution to future investigations.
Selection of Sequences Used in Analysis
I obtained the SSURef (Release 96) alignment from the SILVA database on October 14, 2008, which contained 271,543 bacterial 16S rRNA gene sequences longer than 1,200 bp. Aligned sequences were downloaded from the greengene and RDP databases on November 16, 2008. The SILVA database was selected because it was the longest and visual inspection of the alignment suggested that it had the highest overall quality (Fig. 2). Using ARB, I removed any sequence that affiliated with mitochondria or chloroplasts or were flagged as being of poor alignments, having more than five ambiguous bases, or appearing chimeric. This screen resulted in a collection of 243,472 high-quality aligned sequences. I dereplicated these sequences to obtain 222,086 unique sequences. Finally, I cross-matched accessions between the SILVA, greengenes, and RDP databases to obtain a collection of 200,433 sequences.
Because the SILVA reference alignment (i.e. the SEED) is not publicly available, I attempted to replicate the SEED database. First, I parsed the 200,433 sequences to identify those sequences that had an alignment quality score (i.e. ARB database field ‘align_quality_slv’) of 100. Next, I identified those sequences in this pool that started by E. coli position 28 and ended after position 1491. Sequences beginning before or ending after these coordinates were trimmed. The resulting collection of 14,227 sequences represented my full-length template database. The other 186,206 sequences represented the candidate sequence collection.
Generation of Region Specific Datasets
I selected 10 regions within the 16S rRNA gene for my simulations. While maintaining the overall 50,000-character alignment, I excised regions V19 (E. coli positions 28–1491), V12 (28–337), V14 (28–784), V2 (100–337), V23 (100–514), V3 (357–514), V4 (578–784), V6 (986–1045), V89 (1100–1491), and V9 (1300–1491). There were 186,206 candidate sequences for analyzing the V19, V2, V23, V3, V4, and V6 datasets. Because not all sequences extended through the first and ninth variable regions, I further screened these sequences to generate a collection of 139,987 candidate sequences for analyzing the V12 and V14 datasets and a collection of 77,685 sequences for the V89 and V9 datasets. These regions were selected because they are tractable by Sanger (V19, V14), 454 GS-FLX (V2, V3, V4, V6, V9), 454 Titanium (V12, V23, V89), and Illumina (V3, V6) sequencing technologies. Many of these regions have also been used in published studies: V19 [e.g. 2], V14 [e.g. 1], V2 [e.g. 4], V3 [e.g. 27], V4 , V6 [e.g. 3], and V9 [e.g. 28]. Considering there are myriad permutations of these regions, these provided a generous coverage of the 16S rRNA gene.
Permutations of the Greengenes Search Step
I tested three search options: blastn, kmer searching, and suffix tree searching (Fig. 1). First, I used blastn as made available from NCBI with a word size of 28, match reward of 1, mismatch penalty of -1. These settings are comparable to those used in megablast and were needed to make the search times competitive with the other methods. Other parameters used for the blast included returning one result in the tabular format (–b 1 –m 8). Second, I used kmer searching with word sizes ranging between 5 and 10; smaller and larger words did not improve the searches and were considerably slower. The kmer-searching algorithm involved generating a lookup table where the keys in the table corresponded to all possible 4K kmers. These keys pointed to a list of template sequence identifiers. Next, the software identified all possible kmers for a candidate sequence and counts the number of kmers each template sequence shares with the candidate sequence. An analogous procedure is used in the initial steps of the MUSCLE algorithm . The template with the most kmers in common was then used for further analysis. Third, I generated a suffix tree for each template sequence . Using the string-to-string block-move algorithm I identified the template suffix tree that broke the candidate sequence into the fewest suffix sequences .
Permutations of the Greengenes Pairwise Alignment Step
I tested three pairwise alignment methods: blastn  and the Needleman-Wunsch  and Gotoh ,  global alignment algorithms (Fig. 1). While my implementation of the aligner permits changing the match and mismatch scores, I chose to use +1 and -1 in all simulations. I used combinations of gap opening penalties of 5, 4, 3, 2, and 1 with gap extension penalties of 2 and 1. These values were selected to overlap as much as possible with the combinations that are implemented in the nucleotide-based BLAST program. The bl2seq BLAST program was used to obtain pairwise alignments using BLAST with the default word size of 11 (-W 11). To improve the Needleman-Wunsch and Gotoh alignments at the ends of sequences that do not fully overlap, I followed the end-space free variant algorithm described by Gusfield .
Benchmarking of Methods
To assess the ability of each method to properly identify the correct reference sequence, I calculated the raw similarity between each candidate sequence and all template sequences. In these similarity calculations, the comparison of a gap with a base is counted as a mismatch and the comparison between a pair of gaps does not factor into the calculation. These similarity scores were calculated for each region under consideration because the template for a candidate fragment would not necessarily be the same as the template for the full-length candidate sequence. The template sequence that was most similar to the candidate sequence was considered the true best template sequence.
All simulations were run on a MacPro computer with 9 GB 667 MHz DDR2 RAM and 2×3 GHz Dual-Core Intel Xenon processors. To insure that each analysis was running at optimal speed, I only used 3 processors at a time and each analysis only used one processor. At no time did RAM utilization approach 9 GB. All source code was written in C++ and compiled using the –O3 compiler optimization flag.
Availability of Software
The aligner described here is freely available and provided within the mothur software package as source code or as a Windows executable (http://www.mothur.org) . The defaults within mothur include using kmer searching with a word size of 8 and using the Needleman-Wunsch algorithm for pairwise alignments with a gap penalty of −2; however, all of the methods can be selected by users with the ability to modify any of the match and mismatch scores and gap penalties. Although not used in this study, the mothur implementation enables users to use multiple processors to accelerate the alignment. The mothur implementation requires that the user input FASTA-formatted files containing their candidate sequences and template database. Example template databases, including the one used in this study, are available from the mothur website.
Comparison of search methods when using various regions extracted from candidate sequences and full-length template sequences.
(0.07 MB PDF)
Comparison of search methods when using various regions extracted from candidate sequences and region-specific template sequences.
(0.06 MB PDF)
Summary of alignment improvement for various regions extracted from candidate sequences using the blastn, Gotoh, or Needleman-Wunsch pairwise alignment algorithms when the best full-length template was selected for each candidate sequence.
(0.11 MB PDF)
Summary of alignment improvement for various regions extracted from candidate sequences using the blastn, Gotoh, or Needleman-Wunsch pairwise alignment algorithms when the best region-specific template was selected for each candidate sequence.
(0.10 MB PDF)
I thank Sarah Westcott for helping to incorporate my original source code for the aligner into mothur and the numerous mothur users who have helped to test the aligner software.
Conceived and designed the experiments: PS. Performed the experiments: PS. Analyzed the data: PS. Contributed reagents/materials/analysis tools: PS. Wrote the paper: PS.
- 1. Schloss PD, Handelsman J (2006) Toward a census of bacteria in soil. PLoS Comp Biol 2: e92.PD SchlossJ. Handelsman2006Toward a census of bacteria in soil.PLoS Comp Biol2e92
- 2. Eckburg PB, Bik EM, Bernstein CN, Purdom E, Dethlefsen L, et al. (2005) Diversity of the human intestinal microbial flora. Science 308: 1635–1638.PB EckburgEM BikCN BernsteinE. PurdomL. Dethlefsen2005Diversity of the human intestinal microbial flora.Science30816351638
- 3. Sogin ML, Morrison HG, Huber JA, Welch DM, Huse SM, et al. (2006) Microbial diversity in the deep sea and the underexplored “rare biosphere”. Proc Natl Acad Sci U S A 103: 12115–12120.ML SoginHG MorrisonJA HuberDM WelchSM Huse2006Microbial diversity in the deep sea and the underexplored “rare biosphere”.Proc Natl Acad Sci U S A1031211512120
- 4. Turnbaugh PJ, Hamady M, Yatsunenko T, Cantarel BL, Duncan A, et al. (2009) A core gut microbiome in obese and lean twins. Nature 457: 480–484.PJ TurnbaughM. HamadyT. YatsunenkoBL CantarelA. Duncan2009A core gut microbiome in obese and lean twins.Nature457480484
- 5. Schloss PD, Handelsman J (2005) Introducing DOTUR, a computer program for defining operational taxonomic units and estimating species richness. Appl Environ Microbiol 71: 1501–1506.PD SchlossJ. Handelsman2005Introducing DOTUR, a computer program for defining operational taxonomic units and estimating species richness.Appl Environ Microbiol7115011506
- 6. Fierer N, Hamady M, Lauber CL, Knight R (2008) The influence of sex, handedness, and washing on the diversity of hand surface bacteria. Proc Natl Acad Sci U S A 105: 17994–17999.N. FiererM. HamadyCL LauberR. Knight2008The influence of sex, handedness, and washing on the diversity of hand surface bacteria.Proc Natl Acad Sci U S A1051799417999
- 7. Chenna R, Sugawara H, Koike T, Lopez R, Gibson TJ, et al. (2003) Multiple sequence alignment with the Clustal series of programs. Nucleic Acids Res 31: 3497–3500.R. ChennaH. SugawaraT. KoikeR. LopezTJ Gibson2003Multiple sequence alignment with the Clustal series of programs.Nucleic Acids Res3134973500
- 8. Katoh K, Asimenos G, Toh H (2009) Multiple alignment of DNA sequences with MAFFT. Methods Mol Biol 537: 39–64.K. KatohG. AsimenosH. Toh2009Multiple alignment of DNA sequences with MAFFT.Methods Mol Biol5373964
- 9. Edgar RC (2004) MUSCLE: a multiple sequence alignment method with reduced time and space complexity. BMC Bioinformatics 5: 113.RC Edgar2004MUSCLE: a multiple sequence alignment method with reduced time and space complexity.BMC Bioinformatics5113
- 10. Huber JA, Mark Welch DB, Morrison HG, Huse SM, Neal PR, et al. (2007) Microbial population structures in the deep marine biosphere. Science 318: 97–100.JA HuberDB Mark WelchHG MorrisonSM HusePR Neal2007Microbial population structures in the deep marine biosphere.Science31897100
- 11. Antonopoulos DA, Huse SM, Morrison HG, Schmidt TM, Sogin ML, et al. (2009) Reproducible community dynamics of the gastrointestinal microbiota following antibiotic perturbation. Infect Immun 77: 2367–2375.DA AntonopoulosSM HuseHG MorrisonTM SchmidtML Sogin2009Reproducible community dynamics of the gastrointestinal microbiota following antibiotic perturbation.Infect Immun7723672375
- 12. Sun Y, Cai Y, Liu L, Yu F, Farrell ML, et al. (2009) ESPRIT: estimating species richness using large collections of 16S rRNA pyrosequences. Nucleic Acids Res 37: e76.Y. SunY. CaiL. LiuF. YuML Farrell2009ESPRIT: estimating species richness using large collections of 16S rRNA pyrosequences.Nucleic Acids Res37e76
- 13. Gardner PP, Wilm A, Washietl S (2005) A benchmark of multiple sequence alignment programs upon structural RNAs. Nucleic Acids Res 33: 2433–2439.PP GardnerA. WilmS. Washietl2005A benchmark of multiple sequence alignment programs upon structural RNAs.Nucleic Acids Res3324332439
- 14. Nawrocki EP, Kolbe DL, Eddy SR (2009) Infernal 1.0: inference of RNA alignments. Bioinformatics 25: 1335–1337.EP NawrockiDL KolbeSR Eddy2009Infernal 1.0: inference of RNA alignments.Bioinformatics2513351337
- 15. Cole JR, Wang Q, Cardenas E, Fish J, Chai B, et al. (2009) The Ribosomal Database Project: improved alignments and new tools for rRNA analysis. Nucleic Acids Res 37: D141–145.JR ColeQ. WangE. CardenasJ. FishB. Chai2009The Ribosomal Database Project: improved alignments and new tools for rRNA analysis.Nucleic Acids Res37D141145
- 16. Ludwig W, Strunk O, Westram R, Richter L, Meier H, et al. (2004) ARB: A software environment for sequence data. Nucleic Acids Res 32: 1363–1371.W. LudwigO. StrunkR. WestramL. RichterH. Meier2004ARB: A software environment for sequence data.Nucleic Acids Res3213631371
- 17. Pruesse E, Quast C, Knittel K, Fuchs BM, Ludwig W, et al. (2007) SILVA: a comprehensive online resource for quality checked and aligned ribosomal RNA sequence data compatible with ARB. Nucleic Acids Res 35: 7188–7196.E. PruesseC. QuastK. KnittelBM FuchsW. Ludwig2007SILVA: a comprehensive online resource for quality checked and aligned ribosomal RNA sequence data compatible with ARB.Nucleic Acids Res3571887196
- 18. DeSantis TZ, Hugenholtz P, Larsen N, Rojas M, Brodie EL, et al. (2006) Greengenes, a chimera-checked 16S rRNA gene database and workbench compatible with ARB. Appl Environ Microbiol 72: 5069–5072.TZ DeSantisP. HugenholtzN. LarsenM. RojasEL Brodie2006Greengenes, a chimera-checked 16S rRNA gene database and workbench compatible with ARB.Appl Environ Microbiol7250695072
- 19. DeSantis TZ Jr, Hugenholtz P, Keller K, Brodie EL, Larsen N, et al. (2006) NAST: a multiple sequence alignment server for comparative analysis of 16S rRNA genes. Nucleic Acids Res 34: W394–399.TZ DeSantis JrP. HugenholtzK. KellerEL BrodieN. Larsen2006NAST: a multiple sequence alignment server for comparative analysis of 16S rRNA genes.Nucleic Acids Res34W394399
- 20. Altschul SF, Madden TL, Schaffer AA, Zhang J, Zhang Z, et al. (1997) Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic Acids Res 25: 3389–3402.SF AltschulTL MaddenAA SchafferJ. ZhangZ. Zhang1997Gapped BLAST and PSI-BLAST: a new generation of protein database search programs.Nucleic Acids Res2533893402
- 21. Lane DJ (1991) 16S/23S rRNA sequencing. In: Stackebrandt E, Goodfellow M,, editors. Nucleic Acid Techniques in Bacterial Systematics. New York: Wiley. pp. 115–175.DJ Lane199116S/23S rRNA sequencing.E. StackebrandtM, GoodfellowNucleic Acid Techniques in Bacterial SystematicsNew YorkWiley115175
- 22. Jones RT, Robeson MS, Lauber CL, Hamady M, Knight R, et al. (2009) A comprehensive survey of soil acidobacterial diversity using pyrosequencing and clone library analyses. ISME J 3: 442–453.RT JonesMS RobesonCL LauberM. HamadyR. Knight2009A comprehensive survey of soil acidobacterial diversity using pyrosequencing and clone library analyses.ISME J3442453
- 23. Lauber CL, Hamady M, Knight R, Fierer N (2009) Pyrosequencing-based assessment of soil pH as a predictor of soil bacterial community structure at the continental scale. Appl Environ Microbiol 75: 5111–5120.CL LauberM. HamadyR. KnightN. Fierer2009Pyrosequencing-based assessment of soil pH as a predictor of soil bacterial community structure at the continental scale.Appl Environ Microbiol7551115120
- 24. Bowers RM, Lauber CL, Wiedinmyer C, Hamady M, Hallar AG, et al. (2009) Characterization of airborne microbial communities at a high elevation site and their potential to act as atmospheric ice nuclei. Appl Environ Microbiol 75: 5121–5130.RM BowersCL LauberC. WiedinmyerM. HamadyAG Hallar2009Characterization of airborne microbial communities at a high elevation site and their potential to act as atmospheric ice nuclei.Appl Environ Microbiol7551215130
- 25. Crawford PA, Crowley JR, Sambandam N, Muegge BD, Costello EK, et al. (2009) Regulation of myocardial ketone body metabolism by the gut microbiota during nutrient deprivation. Proc Natl Acad Sci U S A 106: 11276–11281.PA CrawfordJR CrowleyN. SambandamBD MueggeEK Costello2009Regulation of myocardial ketone body metabolism by the gut microbiota during nutrient deprivation.Proc Natl Acad Sci U S A1061127611281
- 26. Schellenberg J, Links MG, Hill JE, Dumonceaux TJ, Peters GA, et al. (2009) Pyrosequencing of the chaperonin-60 universal target as a tool for determining microbial community composition. Appl Environ Microbiol 75: 2889–2898.J. SchellenbergMG LinksJE HillTJ DumonceauxGA Peters2009Pyrosequencing of the chaperonin-60 universal target as a tool for determining microbial community composition.Appl Environ Microbiol7528892898
- 27. Dethlefsen L, Huse S, Sogin ML, Relman DA (2008) The pervasive effects of an antibiotic on the human gut microbiota, as revealed by deep 16S rRNA sequencing. PLoS Biol 6: e280.L. DethlefsenS. HuseML SoginDA Relman2008The pervasive effects of an antibiotic on the human gut microbiota, as revealed by deep 16S rRNA sequencing.PLoS Biol6e280
- 28. Roesch LFW, Fulthorpe RR, Riva A, Casella G, Hadwin AKM, et al. (2007) Pyrosequencing enumerates and contrasts soil microbial diversity. ISME J 1: 283–290.LFW RoeschRR FulthorpeA. RivaG. CasellaAKM Hadwin2007Pyrosequencing enumerates and contrasts soil microbial diversity.ISME J1283290
- 29. Ukkonen E (1995) Online construction of suffix trees. Algorithmica 14: 249–260.E. Ukkonen1995Online construction of suffix trees.Algorithmica14249260
- 30. Tichy WF (1984) The string-to-string correction problem with block moves. ACM Trans Comput Syst 2: 309–321.WF Tichy1984The string-to-string correction problem with block moves.ACM Trans Comput Syst2309321
- 31. Needleman SB, Wunsch CD (1970) A general method applicable to the search for similarities in the amino acid sequence of two proteins. J Mol Biol 48: 443–453.SB NeedlemanCD Wunsch1970A general method applicable to the search for similarities in the amino acid sequence of two proteins.J Mol Biol48443453
- 32. Myers EW, Miller W (1988) Optimal alignments in linear space. Comput Appl Biosci 4: 11–17.EW MyersW. Miller1988Optimal alignments in linear space.Comput Appl Biosci41117
- 33. Gotoh O (1982) An improved algorithm for matching biological sequences. J Mol Biol 162: 705–708.O. Gotoh1982An improved algorithm for matching biological sequences.J Mol Biol162705708
- 34. Gusfield D (1997) Algorithms on strings, trees, and sequences: computer science and computational biology. New York: Cambridge University Press. pp. 228–229.D. Gusfield1997Algorithms on strings, trees, and sequences: computer science and computational biology.New YorkCambridge University Press228229
- 35. Schloss PD, Westcott SL, Ryabin T, Hall JR, Hartmann M, et al. (2009) Introducing mothur: Open source, platform-independent, community-supported software for describing and comparing microbial communities. Appl Environ Microbiol. PD SchlossSL WestcottT. RyabinJR HallM. Hartmann2009Introducing mothur: Open source, platform-independent, community-supported software for describing and comparing microbial communities.Appl Environ Microbiol