2.1.1 Pre-clustering.

The pre-clustering step aims at grouping similar sequences to form initial clonal lineage groups that can be refined later. Note that MobiLLe takes only IGH sequences as input since they are more diverse than IGL chains, providing a reliable signature for immune repertoire studies [20]. First, BCR sequences are annotated to identify their IGHV and IGHJ genes (and alleles) and locate their CDR3 regions. For this purpose, we used IMGT/HighV-QUEST [21], but in principle, any V(D)J annotation software could be used since MobiLLe accepts input data in AIRR format [22]. Sequences with the same IGHV and IGHJ genes and the same CDR3 sequence length are then grouped. Finally, we separate sequences with less than s% of CDR3 amino acid identity (by default s is 70%), see the “pre-clustering” panel in Fig 1.

2.1.2 Clustering refinement.

In this step, we iteratively refine clonal lineage groups until we reach the minimum values for intra-clonal distances and the maximum values for inter-clonal distances. The algorithm described in 1 takes the set of initial groups C as input; generated during the pre-clustering step. For each group K ∈ C and each sequence i ∈ k it computes two distances: a_i (intra-clonal) and b_i (inter-clonal). Such distances measure the cohesion/separation within detected groups; they were initially introduced to compute the Silhouette [23]. a_i is the average distance between the sequence i and any other sequence in the same group K; b_i is the smallest average distance of i to all sequences in any other group L. In a well-separated cluster, a_i is smaller than b_i, thus, if for a given sequence a_i is higher than b_i, it indicates an inconsistency, showing that the clusters K and L should be merged. Note that a_i needs to be recomputed for all sequences in the merged cluster, and b_i for all sequences. Consequently, each cluster merging launches a new iteration of the algorithm, and it stops if no merging is observed in the previous iteration or after a predefined number of iterations.

Certainly, the distance metric d(i, j) (between sequences i and j) plays an important role when computing a_i and b_i. Distances based on sequence similarity of the whole sequences can be inaccurate since different IGHV, and IGHJ genes can present considerable similarities. Moreover, CDR3 regions are shorter than IGHV/IGHJ genes, and a normalized distance should be more appropriate. Therefore, we defined a composed distance that splits sequences into three parts, IGHV, IGHJ, and CDR3 region, and computed a distinct distance of each part separately. The final distance d(i, j) is the ponderate mean of these three distances and is defined by the equation: (1) where is the IGHV distance, is the CDR3 distance, is the IGHJ distance, and α, β, and λ are coefficients. Note that when α = β = λ the arithmetic mean is computed. There are several ways to compute , and distances; we have implemented four different distance types:

1. the binary distance is based on gene identifications; it is 0 if i and j were annotated with the same IGHV/IGHJ gene or 1, otherwise. For CDR3 is 0 when i and j CDR3 amino acid sequences are identical, and 1, otherwise,
2. the normalized Levenshtein distance [24] computes the minimum number of single-character editions (insertions, deletions, or substitutions) required to transform i into j, to normalize it, we divided by the length of the longer sequence;
3. the k-mer based distance [25] is based on the fractional of common k-mers present on both sequences; it is zero if i and j are identical, and one if they do not share any k-mer;
4. the GIANA [26] is an isometric distance that encodes amino acid sequences into numeric vectors through a series of unitary transformations. After encoding, sequences are projected to high-dimensional space, allowing a rapid distance calculation between points in the Euclidean space.

Algorithm 1: Clustering refinement

Require: C {initial groups}

 repeat

  stop ← true

  for all k ∈ C do

   if |k| > 1 then

    for all i ∈ k do

     b_i ← min_l≠k∀_j∈l d(i, j)

     l = argmin_l∀_j∈l d(i, j)

     if a_i > b_i then

      merge clusters k and l

      stop ← false

     end if

    end for

   end if

  end for

 until not stop

The last step of the refinement tries to reduce the number of singletons. This is an important step because once singletons are formed, they cannot be merged to higher density clusters, just using Algorithm 1, since their intra-clonal distance a_i is zero, and it is smaller than any other inter-clonal distance. Thus, we defined the Algorithm 2 that considers every singleton k, and tries to merge it to its closest neighbor l if the merging does not substantially decrease cluster uniformity [27]. More precisely, we merge k and l if |unif(l + k) − unif(l)| < δ. The cluster uniformity measures the variability of a cluster, and smaller values represent a more uniform data distribution. The following equations compute cluster uniformity: (2) (3) (4) where n is the total number of sequences in the cluster k, and d(i,j) is the composed distance defined above.

Algorithm 2: Merging singletons

Require: C {set of singletons}

 for all k ∈ C do

  i ← k[0] {Take the single sequence of k}

  l = argmin_l≠k∀_j∈l d(i, j) {Find the closest neighbor of k}

  if |unif(l + k) − unif(l)| < δ then

   merge clusters k and l

  end if

 end for

2.2.1 Simulated repertoires.

IGH simulated sequences are largely used to evaluate clonal lineage grouping methods [13, 14, 28]. Some repertoire simulators have been proposed as part of such tools, but an independent B cell repertoire simulator that could produce different types of IGH repertoires (clonal, non-clonal), to the best of our knowledge, does not exist. In order to create simulated repertoires, we adapted GCTree [29], a B cell lineage simulator. We ran GCTree several times to produce independent B cell lineages, which were then assembled in a single repertoire.

To produce a B cell lineage, GCTree randomly selects IGHV, IGHD, and IGHJ germline genes from the IMGT database [4], then nucleotide(s) can be added to or removed from the IGHV-IGHD and IGHD-IGHJ junction regions. Next, a branching process is performed, and point mutations are included in the descendants to simulate SHM. For the branching, GCTree uses an arbitrary offspring distribution that does not require an explicit bounding. Instead, it uses a Poisson distribution with parameter λ to estimate each node’s expected number of offspring. SHM are simulated by a sequence-dependent process, where mutations are preferentially introduced within certain hot-spot motifs. GCtree uses the 5-mer context model [30] to compute the mutability μ₁, …, μ_i…, μ_l for each residue i of a sequence of length l. The mutability of the whole sequence μ₀, is then computed by averaging the mutability of its residues: . To determine the number of mutations m to be introduced in each mutant offspring sequence, GCTree also uses a Poisson distribution with parameter λ₀, m is then computed as Pois(μ₀λ₀); note that more mutable sequences (higher μ₀) tend to accumulate more point mutations.

Basically, the GCTree simulator has two main parameters to be set: λ, to estimate the expected number of offspring of each node, and λ₀, to determine the number of point mutations in mutant offspring sequences. We kept λ as the default value (e.g., 2), but we varied λ₀ to produce simulations with different mutation rates. We experimented with four values {0.16, 0.26, 0.36, 0.46}, where 0.26 is the default value. Note that higher λ₀ values produce more divergent B cell lineages. For each λ₀ setting, we simulated three types of repertoires: monoclonal, oligoclonal, and polyclonal, obtaining 12 different benchmarks. Each repertoire type’s initial clonal size setting is shown in S1 Table. Since we only kept productive sequences (without stop codons), the simulated repertoires, Table 1, can contain fewer sequences than shown in S1 Table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Simulated repertoire properties.

https://doi.org/10.1371/journal.pcbi.1010411.t001

2.2.2 Artificial monoclonal repertoires.

Gold standard experimental data, where truly clonal relationships are known with certainty, are challenging to obtain. In order to create experimental-based benchmarks, we constructed artificial monoclonal repertoires by combining sequences from a pure B cell lineage and a polyclonal repertoire. Our goal was to determine if clonal lineage grouping methods could separate sequences from these two sources. To form a benchmark, we considered 10000 sequences, where 10% of them were sampled from the pure lineage and 90% from the polyclonal background. Since we know the truly clonally-related sequences in each benchmark, we could compare the different tools for determining their grouping differences. We created three artificial monoclonal repertoires with three different pure B cell lineages, each having a specific V(D)J rearrangement. The pure B cell lineage of the artificial monoclonal benchmark AMR1 was annotated with IGHV1–69*01/IGHJ6*03 genes, AMR2 with IGHV3–48*02/IGHJ4*02 rearrangements, and the AMR3 with the IGHV3–15*01/IGHJ6*02 rearrangements. We used IMGT/HighV-QUEST [21] as V(D)J assignment tool. AMR1, AMR2, and AMR3 pure B cell lineage were sampled from 22747, 20371, and 23665 sequences, while polyclonal backgrounds were from 136977 sequences. S2–S4 Figs show IGHV/J gene usage distribution of the polyclonal background compared to AMR1, AMR2, and AMR3 sequences, respectively.

The pure B cell lineages and polyclonal backgrounds are human peripheral blood mononuclear cells obtained during routine diagnostic procedures at Pitié-Salpêtrière hospital (Paris-France). DNA sequences were obtained by polymerase chain amplification of IGH-VDJ rearrangements followed by paired-end sequencing on an Illumina MiSeq platform. We obtained one “Read 1” and “Read 2” FASTQ files for each sample, which were then merged by the PEAR software [31]. The merged FASTQ files were converted to FASTA format with seqtk (https://github.com/lh3/seqtk).

2.2.3 Experimental repertoires.

In order to evaluate our approach on realistically-sized repertoires, we used a dataset produced at the Pitié-Salpêtrière hospital (Paris-France), and two public datasets from IReceptor repository [32], totalizing 26 experimental repertoires.

The first dataset contains nine samples of human peripheral blood mononuclear cells collected during routine diagnostic procedures at Pitié-Salpêtrière hospital. We picked three samples from these repertoires to systematically compare MobiLLe’s results and existing clonal lineage grouping methods. DNA sequences were obtained and assembled as described in Section 2.2.2 Table 2 shows the number of reads (sequences), the number of unique sequences (clonotypes), and the clonality status defined by standard PCR amplification and capillary electrophoresis of amplicons (Genescan analysis) [33]. S5 Fig shows the output plot of Genescan profiles for each experimental repertoire.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Properties of nine experimental repertoires.

For each sample, we show clonality status, the individual label, the total number of sequences and unique sequences (clonotypes).

https://doi.org/10.1371/journal.pcbi.1010411.t002

The second dataset contains five experimental repertoires selected from a previous work that performed high-throughput sequencing to characterize the B cell populations in several lymphoproliferative diseases [34]. Samples come from different tissues such as blood, lymph nodes, liver, and bone marrow. From this study, we selected one healthy donor (HD) and four patients diagnosed with different lymphoproliferative diseases. Sequence data were obtained through the sequencing platform 454 GS-FLX. Table 3 summarizes the main properties of selected repertoires, for more details report to [34].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Properties of patients and healthy donor repertoires.

https://doi.org/10.1371/journal.pcbi.1010411.t003

The third dataset contains 12 repertoires selected from a previous work that conducted immune profiling studies of several individuals with different trajectories of SARS-CoV-2 infection and COVID-19 (moderate and severe) and compared them to healthy donors [35]. Samples were collected from peripheral blood mononuclear cells, and sequencing was performed with the Illumina MiSeq platform; Table 4 summarizes the properties of 12 considered repertoires; for more details, report to the original publication.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Clinical and repertoire characteristics of healthy donors and patients with moderate/severe COVID-19.

https://doi.org/10.1371/journal.pcbi.1010411.t004

2.3.1 Clustering accuracy.

When clonal assignments are known, one can quantitatively assess clonal lineage grouping algorithms’ performance by measuring their ability to identify clonally-related sequences. For that, we applied classical measures such as precision and recall for comparing the inferred clusters (clonal lineages) to the true ones. Consistently, we also computed the F-score (FS), the harmonic mean of precision and recall; it is an aggregate measure of the inferred cluster’s quality. Precision and recall require three disjoint sets, which are: true positive (TP), false-positive (FP), and false-negative (FN). From these, we compute precision , recall , and . The values of these three metrics are in the interval [0, 1], being one the best and zero the worst performance. Certainly, the way TP, FP, and FN are computed will affect the accuracy measures. Depending on the grouping level, there are at least two ways to compute these values: pairwise and closeness.

The pairwise procedure considers the binary clustering task and focuses on the relationship between each pair of sequences. A pair of sequences is counted as: TP if the sequences are found together in both ‘true’ and ‘inferred’ clusters; FP if the sequences are found separated in the true but together in the inferred cluster; FN if the pair is found together in the true but separated in the inferred cluster, see an illustration in S6(A) Fig.

The closeness procedure evaluates clonal compositions and the repertoire structure. For that, we first identified the best correspondence between inferred clonal lineages and correct clonal assignments. Then, we associated clonal lineage pairs sharing the maximum of common sequences. For each associated pair i, considering I_i inferred and T_i the correct clonal assignment, we computed TP_i as the intersection between the two sets (I_i ∩ T_i), FP_i as the difference between inferred and the correct clonal assignment (I_i\T_i), and FN_i as the difference between the correct clonal assignment and inferred group (T_i\I_i). Finally, we computed , and , where n is the number of associated clonal lineage pairs; see an illustration in S6(B) Fig.

2.3.2 Comparison of clonal distributions.

In order to compare clonal lineages obtained by different tools, we have defined five “events” that describe the differences between each pair of clonal distributions. For this, we labelled clusters in a clonal distribution d₂ by comparing them with clusters in a distribution d₁. These events are represented in Fig 2, and can be interpreted as follows:

1. identical: clusters in both distributions are identical, they contain the same set of sequences (Fig 2A),
2. join: when sequences of different clusters in d₁ were joined in the same cluster in d₂ (Fig 2B),
3. split: when sequences of a cluster in d₁ were divided into several clusters in d₂ (Fig 2C),
4. Mix: when a mixture of the three above events occurs. For instance, in Fig 2D, we observed two events, “split” (C₈ and C₉) and “join” (C₁₀, C₁₁ and C₁₂),
5. Not found: when a cluster in d₂ was not found among clusters in d₁. For instance, in Fig 2E, cluster C₁₃ was not present in d₁.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Clonal distribution comparisons.

Five “events” describe the differences between two clonal distributions (d₁ and d₂). The identical event counts the number of identical clonal lineages found in both distributions (a). The “join” event reports the number of clonal lineages in d₁ found merged in d₂ (b), while the split counts the number of clonal lineages in d₁ found separated in d₂ (c). The “mix” event is a mixture of these two later events (d), while “not found” reports the number of clonal lineages in d₂ not found in d₁ (e).

https://doi.org/10.1371/journal.pcbi.1010411.g002

2.4.1 BRILIA.

B cell Repertoire Inductive Lineage and Immunosequence Annotator (BRILIA) builds up lineage tree reconstruction, clonal grouping, and V(D)J annotation into a single algorithm [18]. From a collection of IGH sequences, BRILIA first provides initial V(D)J gene identification according to the IMGT database [4]. Then, it groups sequences with the same IGHV and IGHJ gene subgroups and the same CDR3 sequence length. It proceeds by reconstructing lineage trees that will determine groups of clonally-related sequences. For that, it determines parent-child sequence relationships within each group for further reconstructing lineage trees. Evolutionary relationships are based on an adjusted hamming distance that penalizes dissimilarities in the N regions. Next, BRILIA determines the root of each obtained tree as the sequence involved in a cyclic dependency having the smallest distance to all other sequences in that cluster. Finally, a clone is a group of sequences sharing a common root sequence.

2.4.2 Partis.

Partis [13] uses hidden Markov model (HMM) to represent V(D)J rearrangement events [36]. An HMM is a probabilistic model, where the modeled system is assumed to be a Markov process with hidden states and unknown parameters. Each hidden state emits a symbol representing an elementary unit of the modeled data; for example, in BCR sequences, the hidden states represent either gene positions or N-region (addition or deletion) nucleotides. Thus, the HMM states represent nucleotides of each IGHV, IGHD, and IGHJ gene. The emission probabilities incorporate the probability of SHM at each nucleotide, and transition probabilities represent the probability of moving from one state to another. The HMM parameters (emission and transition probabilities) are estimated from a large panel of available sequences. Once the model is trained, BCR sequences are annotated by computing the Viterbi path through the HMM and finding the maximum-likelihood annotation. After V(D)J assignment, Partis applies its clonal grouping strategy. First, it creates initial clonal lineages of sequences sharing the same IGHV and IGHJ genes and the same CDR3 length. Then, it applies an agglomerative clustering algorithm to merge clusters that maximize the likelihood ratio that could indicate that two clusters derive from the same rearrangement events.

2.4.3 SCOPe.

SCOPe requires V(D)J annotation before clonal grouping, and tools such as IMGT/HighV-QUEST [21], or IgBlast [37] can be used. To identify a clone, SCOPe applies a spectral clustering method with an adaptive threshold to determine the local sequence neighborhood; meaning that it does not require a fixed threshold for detecting clonally-related sequences. Given a set of IGH sequences, SCOPe first divides them into groups with the same IGHV gene, IGHJ gene, and junction length. Then, it computes the similarity matrix for each group considering the hamming distance between junction regions of each pair of sequences within the group. Next, it generates a fully connected graph from the data points and performs local scaling to determine the local neighborhood. Based on the graph, SCOPe builds an adjacency matrix and creates a Laplacian graph. The eigenvalues of such a graph can then be used to find the best number of clusters, and the eigenvectors can be used to find the actual cluster labels. Finally, SCOPe performs k-means clustering on the eigenvectors to get the labels (clone) for each node (sequence).

2.4.4 SONAR.

For SONAR (Software for the Ontogenic aNalysis of Antibody Repertoire) [17], a clonal group contains all IG reads that share a common ancestor. This tool focuses further on seeded lineage assignment, where the sequences of one or more known antibodies are used as seeds to find all sequences in the dataset from the same lineage while leaving the rest of the sequences unclassified. In addition, it can perform “unseeded lineage assignment,” which consists of classifying sequences into component lineages without any additional information. In order to perform an unseeded lineage assignment, SONAR separates sequences based on their assigned IGHV and IGHJ genes. The sequences in each group are then clustered based on their CDR3 nucleotide identity (by default, 90% of CDR3 sequence), using the UCLUST algorithm in USEARCH [38]. Eventually, each clone is identified as a distinct unseeded lineage.

3.2.1 Simulated BCR repertoires.

We first compared MobiLLe to the four selected methods on the 12 simulated repertoires described in Section 2.2.1. To quantify clustering accuracy, we used precision, recall and F-score computed in two ways: pairwise and closeness (Section 2.3.1).

Pairwise performances.

MobiLLe achieved the best pairwise performance across all simulated datasets, Fig 7 Top. SCOPe and Partis outperformed SONAR and BRILIA. All tools achieved a precision close to 1, with few false positives. However, most tools over-split clonal lineages, detecting many false negatives that considerably decreased their recall and F-score values.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Comparison of clustering accuracy on simulated repertoires.

Performance evaluation of five different BCR lineage grouping methods on 12 simulated repertoires.

https://doi.org/10.1371/journal.pcbi.1010411.g007

SCOPe achieved high recalls and F-scores for simulated benchmarks with lower mutation rates (λ₀ = {0.16, 0.26}), see S2–S7 Tables. Recall and F-score values were above 0.94 for these six simulated repertoires. We observed lower recalls and F-scores for the remaining benchmarks, produced with higher mutation rates λ₀ = {0.36, 0.46} (see S8–S13 Tables). On the other hand, Partis obtained a good pairwise performance across all simulated benchmarks independently of the mutation rate. The only exception was the monoclonal repertoire M36 (S8 Table). Partis detected 44 groups for this benchmark when the expected was 35, decreasing its recall considerably. Interestingly, for lower mutation rates, SCOPe outperformed Partis, but we observed the reverse for higher mutation rates for most simulated repertoires. Thus, Partis seems more accurate when analyzing clonally-related sequences with higher divergence. It had difficulties separating clusters with highly similar naive sequences, as reported in the original publication.

The different mutation rates impacted SONAR performances for oligoclonal and polyclonal repertoires. Recalls and F-scores decreased as long as mutation rates increased, especially for the oligoclonal repertoires. For the monoclonal samples with λ₀ = 0.26 (S5 Table), SONAR obtained lower recall and F-score than with λ₀ = 0.36 (S8 Table). We observed that SONAR over-split the largest clonal group of the first repertoire (λ₀ = 0.26), grouping only 37% of sequences. On the other hand, it splits less the most abundant clonal lineage of monoclonal repertoire generated with λ₀ = 0.36, grouping 62% of sequences. As splits in large clonal lineages contribute more to accuracy decreasing, it could explain the lower performance of SONAR on the monoclonal repertoire (λ₀ = 0.26). For the monoclonal repertoire with λ₀ = 0.46, SONAR detected four times more clonal groups than expected, obtaining its lowest recall and F-score, 0.03 and 0.06, respectively.

Most of the time, BRILIA achieved the lowest pairwise performances across all simulated repertoires generated with different mutation rates. BRILIA removes sequences that it cannot annotate, thus reducing the original dataset, which impacts the accuracy calculation. We also observed that BRILIA over-split repertoires, producing the highest number of clonal lineages for most simulated benchmarks. The best performance was obtained on polyclonal repertoires generated with lower mutation rates (λ₀ = {0.16, 0.26}, S4–S7 Tables) and the lowest performance on monoclonal repertoires with higher mutation rates (λ₀ = {0.36, 0.46}, S8 and S11 Tables).

Closeness performances.

MobiLLe achieved the best closeness performance across all simulated benchmarks independently of mutation rates; see Fig 7. The closeness evaluation approach is more challenging than the pairwise approach since clonal lineage properties such as size and distribution are also evaluated rather than pairwise relationships. Conversely, MobiLLe performance was not impacted by higher mutation rates or unbalanced clonal distributions, as observed for other tools. MobiLLe reconstructed all repertoire structures precisely, showing good stability and high accuracy. On the other hand, the four evaluated BCR lineage grouping tools obtained high precision values but much lower recalls and F-scores. As observed for pairwise measures, SCOPe and Partis outperformed SONAR and BRILIA.

SCOPe outperformed Partis for most monoclonal repertoires, but Partis surpassed SCOPe for the majority of oligoclonal and polyclonal samples. SCOPe achieved higher F-score values (> 0.73) on monoclonal repertoires generated with lower mutation rates (λ₀ = {0.16, 0.26, 0.36}, see S2, S5 and S8 Tables). However, its performance sharply decreased on the monoclonal repertoire with the highest mutation rate, achieving 0.16 and 0.28 for recall and F-score (S11 Table). This was particularly the case on oligoclonal and polyclonal samples, where we observed a significant difference between repertoires generated with λ₀ = {0.16, 0.26} and those generated with λ₀ = {0.36, 0.46}; Fig 7D and 7F. Higher mutation rates did not impact the performance of Partis on simulated repertoires. Its accuracy was stable on polyclonal repertoires and presented slight fluctuations on oligoclonal samples. Interesting, on the monoclonal benchmarks, Partis achieved better performance for highly mutated repertoires, achieving its best F-score on the sample generated with λ₀ = {0.46}; Fig 7B and S11 Table.

SONAR performance was greatly affected by high mutation rates. We systematically observed lower recalls/F-scores as mutation rates increased (Fig 7D and 7F and S2–S13 Tables). Independently of mutation rates, SONAR achieved low performance on monoclonal repertoires; F-scores were smaller than 0.2 (S2, S5, S8 and S11 Tables). SONAR over-split the most abundant clonal group of monoclonal repertoires that greatly decreased closeness performances. We observed better results on oligoclonal and polyclonal repertoires, especially on samples generated with lower mutation rates (λ₀ = {0.16, 0.26} (S3 and S6 Tables). On the other hand, for all repertoires generated with higher mutation rates, SONAR achieved an F-score inferior to 0.4 (S8–S13 Tables). BRILIA achieved the lowest performance for most of the analyzed repertoires. The only exception was the oligoclonal sample generated with λ₀ = 0.46, where it outperformed SONAR (S12 Table). For the remaining benchmarks, SONAR overpassed BRILIA; we observed a notable difference mainly on repertoires generated with lower mutation rates, where BRILIA achieved lower F-scores systematically (S3, S4, S6 and S7 Tables). BRILIA and SONAR achieved an equivalent performance in monoclonal repertoires, with shallow values for recall and F-score.

3.2.2 Artificial experimental repertoires.

To investigate the tools’ performance on experimental benchmarks, we created artificial repertoires from BCR high throughput sequencing data as described in Section 2.2.2. We generated three artificial monoclonal repertoires (AMR1, AMR2, and AMR3) by mixing sequences from a pure B cell lineage (10%) and a polyclonal background (90%). Each benchmark contains 10k sequences, and the performance measures clonal grouping tools’ ability to identify memberships within the most abundant group (the pure B cell lineage). Accurate tools might group sequences from the most abundant group and separate those from the polyclonal background in different clusters. Thus, we counted the number of split clusters (SC) and false positives (FP) of the most abundant group to measure the tools’ accuracy. We also used the alluvial diagrams for visualizing clustering results; it represents flows between expected clonal lineages (left) and inferred ones (right).

Fig 8 shows the performance of different tools for AMR1, AMR2, and AMR3. In each alluvial diagram, blue blocks (on the left) represent the pure B cell lineage, and pink or orange (on the right) inferred clonal lineages. Pink blocks contain only sequences of the pure B cell lineage (true positives), while the orange blocks sequences from the polyclonal background (false positives). Block height symbolizes the size of a clonal lineage, that is, the number of sequences. For AMR1, MobiLLe obtained the best clonal separation with just one split (SC = 1) and no false positive (FP = 0). Partis and SCOPe obtained few false positives but higher SC, 4 and 5, respectively. SONAR and BRILIA did not yield any false positives, but both tools produced a significant number of splits, 90 and 64, respectively. Note that the only sequence that MobiLLe left out had a different CDR3 length and less than 70% of amino acid sequence identity. All other tools also placed this sequence outside of the major clonal lineage.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Performance comparison on artificial monoclonal repertoires.

We generated three artificial monoclonal repertoires (AMR1, AMR2, and AMR3) by sampling sequences from a pure B cell lineage (10%) and a polyclonal background (90%). Each benchmark contained 10000 sequences. Accurate tools might group sequences from the pure B cell lineage and separate those from the polyclonal background. We measured the performance of BCR lineage grouping methods by computing the number of splits (SC) and false positives (FP) of the most abundant group. To better visualize and compare clustering results, we show alluvial diagrams for AMR1 (a), AMR2 (b), and AMR3 (c), where blue blocks represent the pure B cell lineage and pink or orange inferred groups. Pink blocks contain only sequences belonging to the pure B cell lineage (true positives), while the orange blocks contain sequences from the polyclonal background (false positives). SONAR and BRILIA did not produce results for the AMR3 benchmarks since they do not deal with non-productive sequences.

https://doi.org/10.1371/journal.pcbi.1010411.g008

MobiLLe did not split the AMR2 B cell lineage sequences into different groups (SC = 0). It also obtained the lowest FP value compared to Partis and SCOPe, which similarly did not divide the most abundant clonal lineage. SONAR and BRILIA still over-split the pure B cell lineage sequences into several groups, achieving an SC of 90 and 55, respectively. SONAR detected more groups than BRILIA but obtained fewer FPs. Notably, the pure B cell lineage of AMR3 contains non-productive sequences that SONAR and BRILIA did not consider. MobiLLe recovered all sequences of the most abundant clonal lineage (SC = 0), while SCOPe and Partis performed 2 and 3 splits, respectively, see Fig 8C. For all artificial monoclonal repertoires, we observed the same behavior in the results of each tool. We noted that BCR lineage grouping methods clustered sequences differently. Partis, SCOPe, and MobiLLe grouped most sequences from pure B cell lineages. SONAR and BRILIA over-split clonally-related sequences but detected fewer FPs than MobiLLe on benchmarks AMR2, and fewer FPs than Partis and SCOPe on all benchmarks. MobiLLe presented the best performance, achieving minimum splits and false positives.

3.2.3 Clonal distribution comparisons.

In order to understand the differences in clonal distributions of experimental repertoires, we compared MobiLLe cluster composition to each considered BCR lineage grouping tool. For comparing the methods, we selected three repertoires with different clonality status (Section 2.2.3 and S5 Fig). I₁ is a monoclonal repertoire where the most abundant clonal group contains 98% of all sequences, I₆ is oligoclonal, and I₈ is a polyclonal repertoire; see Table 2 and S5(A), S5(F) and S5(H) Fig. We compared the inferred clonal lineages of existing grouping tools with MobiLLe’s clustering results. For that, we defined five events: identical, join, split, mix, and “not found,” which represent the (dis)similarities between two clonal distributions, see Section 2.3.2 and Fig 2. Fig 9 shows the occurrences of such events when comparing MobiLLe against Partis, SCOPe, SONAR, and BRILIA. Note that we also quantified the number of MobiLLe’s missing clonal lineages as a “Not found” event since MobiLLe removes sequences with no V(D)J annotations. To better understand event results, we also computed pairwise performance between MobiLLe and each evaluated tool (Section 2.3.1) by comparing the inferred groups—obtained by other tools—to the MobiLLe groups.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Clonal distribution comparisons on three experimental repertoires.

We compared the inferred clonal lineages of each BCR lineage grouping tool with MobiLLe’s clustering results. For that, we defined five events: identical, join, split, mix, and not found, representing the (dis)similarities between two clonal distributions: d₁ (MobiLLe) and d₂ (another tool). The “identical” event accounts for the percentage of identical clonal lineages found in both distributions; the “join” event reports the percentage of d₁ clonal lineages found merged in d₂ while “split” the percentage of d₁ clonal lineages found separated in d₂. The “mix” event accounts for a mixture of “join” and “split” events while “not found” reports the percentage of clonal lineages in d₂ not found in d₁; see an illustration in Fig 2.

https://doi.org/10.1371/journal.pcbi.1010411.g009

When analyzing the repertoire I₁, we observed that BRILIA presented the maximum number of identical clusters, followed by Partis and SCOPe. These three tools obtained a similar number of clusters in the order of magnitude of the MobiLLe output. On the other hand, SONAR presented a significantly higher number of clusters, performing many splits. Interestingly, BRILIA and Partis achieved very high pairwise performances (S14 Table), indicating that most pairs of clonally-related sequences were identically clustered. SCOPe inferred more clonal lineages than MobiLLe, achieving a slightly lower pairwise performance but around 0.98 (S14 Table). As expected, the recalls and F-scores of SONAR were very low since it performed a considerable number of splits. All compared tools produced fewer joins, indicating that they did not group sequences separated by MobiLLe.

For the oligoclonal repertoire I₆, BRILIA inferred the highest number of identical clonal lineages, approximating the number of clonal lineages predicted by MobiLLe. For Partis and SCOPe, the predominant event was also “identical,” but both tools inferred fewer clonal lineages than MobiLLe. For SCOPe, we also observed that the “mix” event was frequent, accounting for 29% of clonal groups. Further analyses showed that “mix” events were associated with minor differences involving singletons. It was confirmed by pairwise performance values (S14 Table) showing very high F-scores for SCOPe and Partis. SCOPe generated more splits than BRILIA and Partis, but it did not impact its pairwise measures since most splits were not in the most abundant clonal group as observed for SONAR. SONAR over-split, detecting three times more clonal lineages than MobiLLe. Consequently, its pairwise recall 0.18 was very low (S14 Table). Its precision was also very low, showing many differences compared to MobiLLe.

We observed more identical inferred clonal lineages in the polyclonal repertoire I₈ for BRILIA, Partis and SCOPe. BRILIA presented the highest value, accounting for more than 93%, followed by SCOPe and Partis. BRILIA and Partis obtained the best pairwise values. There was a good agreement between Partis and MobiLLe, but the former produced more splits than BRILIA and SCOPe, resulting in a slightly decreased pairwise recall. In general, the clonal composition of the three tools was similar to MobiLLe (S15 Table). On the other hand, SONAR over-split the clonal lineages inferred by MobiLLe, detecting the largest number of groups. The clonal composition was also very different, resulting in low precision, (S14 Table).

3.2.4 Runtime.

Computational efficiency is an important factor in BCR lineage grouping. Efficient methods need to process a large number of BCR sequences within a reasonable time without compromising clustering quality. In order to compare the computational requirements of MobiLLe with the four selected BCR lineage grouping tools, we measured the running time on the I₁, I₆ and I₈ repertoires. Fig 10 shows the time in seconds required by each tool to process the three repertoires, using a 3.4 GHz Octa-Core processor with 32 GB of memory; see the exact time in S16 Table. MobiLLe took less than 20s to analyse the monoclonal repertoire I₁ that contains more than 30000 sequences. SONAR was also quick, but we observed an over-split that could explain its faster performance. BRILIA and SCOPe analysed the I₁ repertoire in a comparable time, around 200s and 100s, respectively. Partis was the most time-consuming tool, taking more than 2000 seconds to process the I₁ repertoire.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Comparing running times of clonal grouping tools.

The running times for MobiLLe and other tools were measured for three experimental repertoires with different clonal compositions. To a better visualisation, we used log scale, S16 Table shows the time in second for each considered tool.

https://doi.org/10.1371/journal.pcbi.1010411.g010

For I₆, an oligoclonal repertoire containing more than 140K sequences, MobiLLe also exhibited the fastest performance, but this repertoire required the longest processing time. Although I₆ presents an oligoclonal structure, it has a clonal lineage that groups more than 14% of sequences. The I₆ repertoire structure slowed down the clustering task since more comparisons among higher density clusters were required. Note that all evaluated tools spent more time analysing the I₆ repertoire. The polyclonal repertoire I₈, containing more than 70000, appeared slightly less challenging than I₆ with fewer sequences. Again, MobiLLe achieved the best time performance, clustering I₈ in 135s. SONAR and SCOPe were also fast, taking 187s. Partis achieved the lowest performance, taking more than 1500s.

We found that the clonal distribution significantly influences the running time of MobiLLe rather than the number of sequences. Calculating intra-clonal distances in repertoires with high abundant clonal lineages was more time-consuming. Accordingly, for repertoires with similar sizes but different clonal distributions, MobiLLe can present different running times.

3.3.1 Comparing standard clonality clinical assays with MobiLLe.

In order to compare MobiLLe results to standard clonality assays, we selected nine samples of human peripheral blood mononuclear cells collected during routine diagnostic procedures at the Pitié-Salpêtrière hospital. Three were leukemic samples (chronic lymphocytic leukemia, CLL), and six of them were considered non-clonal, originating from patients devoid of malignancy (Section 2.2.3). Their clonality status had been previously established by conventional methods, including PCR amplification of IGH-VDJ rearrangements followed by Genescan analysis [33] (S5 Fig).

Fig 11 shows the clonal distribution for each analyzed sample; see also S7 Fig. To measure the disequilibrium of a repertoire, we computed the Gini coefficient [41], which reflects the inequalities among values of a frequency distribution; zero indicates perfect equality, while one corresponds to maximal inequality, see panel C of S8–S16 Figs. Clonal repertoires presented the highest Gini indices, close to 1 for individuals 1 to 3 (compare panels C of S8–S10 Figs). Repertoires 1 and 3 presented similar clonal distributions, with the presence of a major clonal group representing the quasi-totality of the repertoire and a small number of minor clonal lineages having a low number of sequences (compare I1 and I3 in Fig 11 and S8 and S10 Figs). MobiLLe results were concordant with Genescan analysis, where a single pick was observed, indicating a monoclonal repertoire profile (S5(A) and S5(C) Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Clonal distribution/density of nine experimental repertoires.

We plotted the 20 most abundant clonal lineages for each repertoire; Circles represent clonal groups, while their areas are proportional to the clonal group abundance.

https://doi.org/10.1371/journal.pcbi.1010411.g011

Individual 2 presented a different clonal distribution with two major clonal lineages, each one accounting for more than 40% of the repertoire, see I2 in Fig 11 and S9 Fig). Detailed sequence analysis revealed that the two major clonal lineages were composed of a productive and an unproductive IGH-VDJ rearrangement, corresponding to a leukemic cell population with biallelic IGH rearrangements. This was also evident in Genescan analysis; S5(B) Fig, two peaks are observed, each corresponding to one IGH allele.

To better analyze non-clonal cases, we split them into two groups: (i) those with the predominance of several clonal lineages of moderate abundance: I₄, I₅ and I₆ (considered as minor clonal or oligoclonal repertoires), and (ii) the others with a more equilibrated (balanced) sequence distribution: I₇, I₈ and I₉, representing polyclonal repertoires. In the first group, repertoires 4 and 5 displayed similar clonal distributions, with Gini indices around 0.76 and 0.84, respectively, see Fig 11, S11 and S12 Figs. GeneScan analyses showed only minor peaks emerging above the polyclonal background, S5(D) and S5(E) Fig. Individual 6 presented a slightly different configuration from other repertoires in the same group, with the presence of a clonal lineage representing around 13% of the repertoire, Fig 11 and S13 Fig. Unsurprisingly, the repertoire I₆ displayed the most biased distribution among non-clonal repertoires with a Gini index of 0.91. We also observed a more irregular profile in the Genescan analysis, (S5(F) Fig).

In the second group, we found more homogeneous and less biased repertoires. Sample 7 and 8 had similar clonal distributions (Fig 11 and S14 and S15 Figs), while repertoire 9 was more irregular (Gini index around 0.83), see Fig 11 and S16 Fig. We also observed a slight difference when comparing distributions generated by Genescan analysis, where 9 displayed a more irregular polyclonal profile, compare S5(G)–S5(I) Fig. In these three cases, the size of the detected clonal lineages was small, each of them accounting for less than 1% of the total sequences; see S14–S16 Figs.

3.3.2 Characterizing B cell populations from different lymphoproliferative diseases.

To further evaluate MobiLLe, we selected an independent series of lymphoproliferative disorder repertoires from a previously published study [34] with available sequence data. Samples were collected from distinct tissues, see Table 3. Five cases were selected, including one patient diagnosed with follicular lymphoma/small lymphocytic lymphoma (P3-FL-SLL), one with chronic lymphocytic leukemia/small lymphocytic lymphoma (P4-C/SLL), and two samples from the same patient diagnosed with a post-transplant lymphoproliferative disorder: P5-L was sampled from the liver, while P5-BM from bone marrow. The fifth sample was from a healthy donor (HD).

Fig 12 compares the clonal distribution of each repertoire’s 20 most abundant clonal lineages; see also S17 Fig. As expected, we did not observe any clonal expansions in the HD repertoire. On the other hand, we observed a considerable clonal expansion in the repertoires of all patients. For P3-FL-SLL, MobiLLe detected two distinct V(D)J rearrangements; the most abundant clonal lineage grouped 74% of IGH sequences and used gene segments: IGHV1–18*01 and IGHJ4*02, while the second clonal B cell population grouped 18% of sequences and used IGHV3–30*02 and IGHJ4*02 genes. MobiLLe clonal distributions for these five repertoires agreed with the original paper [34]; their results were supported by morphological and immunophenotypic evidence, showing two different B cell lymphomas (follicular lymphoma and small lymphocytic lymphoma) in the tissue.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Clonal distributions of a healthy donor and individuals with different lymphoproliferative diseases.

We plotted the 20 most abundant clonal lineages for each repertoire; Circles represent clonal groups, while their areas are proportional to the clonal group abundance. Report to Table 3 for repertoire properties and individuals’ labels.

https://doi.org/10.1371/journal.pcbi.1010411.g012

The analysis of the P4-C/SLL repertoire showed an oligoclonal population with several clonal lineages with an abundance superior to 1%. These results agreed with the original publication, where they observed several groups represented by large points. The two most abundant clonal lineages grouped 37% and 35% of IGH sequences and used very different segment genes, IGHV4–61*02/IGHJ6*03 and IGHV1-NL1*01/IGHJ5*02, respectively. The other five most abundant clonal lineages used different genes with very different CDR3 regions.

Patient 5 was a challenging case, as reported in the original publication. The patient had undergone a liver transplant and later developed a large B cell lymphoma in the liver, a manifestation of a post-transplant lymphoproliferative disorder, a condition in which immunosuppression leads to B cell lymphomas, usually associated with Epstein-Barr virus infection. MobiLLe analyzed the two samples of patient 5 collected from bone marrow (P5-BM) and the liver (P5-L). We observed clonal expansions in both, being more abundant in P5-L than P5-BM, as shown in Fig 12 and S17(D) and S7(E) Fig. The original publication showed that capillary electrophoresis sizing of V(D)J rearrangements confirmed the presence of a clonal population in both samples. However, only sequencing data uncovered no relationship between the clonal lineages found in the liver and those in the bone marrow B cells. The most abundant clonal lineage of P5-BM grouped 22% of the repertoire sequences and used: IGHV3–15*01 and IGHJ6*02 genes, while P5-L grouped 50% of IGH sequences and used IGHV1–69*06 and IGHJ4*02 genes. Of note, it is not unusual to observe independent EBV-driven clonal B-cell proliferations in the context of post-transplantation immune suppression.