Grouped neutralization learning framework

One of the principal challenges of predicting virus-antibody neutralization is sparse data. Neutralization experiments are generally low throughput, and the space of possible virus and antibody sequences is enormous. We assume that our data consists of a set of antibody-virus neutralization values and corresponding virus sequences. The neutralization values can be arranged into an -dimensional matrix, U, where N_A is the number of antibodies and N_V the number of viruses, with the average neutralization value for antibody and virus given by (for an example, see Fig 1a). Here, we will primarily use the half-maximal inhibitory concentration (IC50)—a standard summary statistic obtained by fitting antibody titration curves to estimate the antibody concentration required to achieve 50% neutralization—as our metric of virus “sensitivity” to antibodies. This value need not be defined for all antibody-virus pairs, and viruses may be included even if there are no neutralization values measured for them for any of the antibodies in the panel. Our goal is then to use the observed and virus sequences to predict the unobserved . Here, we will assume that the neutralization values are quantified by IC50s on a logarithmic scale, but other metrics could also be used.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. GNL improves virus-antibody neutralization predictions.

(a) Visualization of HIV-1 neutralization data retrieved from CATNAP [23]. (b) Antibody similarity visualized by projection onto the top principal components of the similarity matrix . Antibodies are colored according to the part of the HIV-1 surface protein, Env, that they bind to. (c) Comparison of neutralization values predicted by GNL with true, withheld values. The rank of the low-rank approximation is selected as the smallest value for which the cumulative fraction of variance explained by the retained eigenmodes exceeds 95% (see S1 Text). Correlations between true and predicted neutralization values are highly significant, exhibiting a p-value of less than across all conditions. (d) For both GNL and the Einav et al. approach, predictive power (measured by Pearson’s R between true and predicted neutralization values) improves along with the amount of data used for training. As in Fig 1a, the rank values are selected based on our optimization criteria. (e) Predictive power depends nonlinearly on the rank used in the low-rank matrix approximation for both GNL and Einav et al. for 80% of fraction of the data observed. However, our approach exhibits stable and generally improved performance over a wide range of values. Error bars represent 95% confidence intervals, obtained by randomly partitioning neutralization values into training or testing subsets.

https://doi.org/10.1371/journal.pcbi.1014095.g001

One of the main ideas of our approach is to leverage similarities between antibodies to improve predictive power. We also associate viral sequence features with antibody sensitivity or resistance, enabling predictions for novel sequences. Finally, we employ a low-rank approximation for the antibody-virus neutralization matrix, which makes use of common patterns in the matrix to refine the results. Below, we describe each of these steps (S1 Fig) and their rationale.

Grouping similar antibodies

To identify antibodies with similar neutralization profiles, we calculate a similarity matrix between all antibody pairs based on the observed neutralization values (see Fig 1b for a visualization). The similarity between antibodies and is defined as the angular similarity of their neutralization vectors against the same viruses (S1 Text). Thus, pairs of antibodies that have highly similar neutralization profiles against the same viruses will receive higher similarity scores.

We then apply a spectral clustering algorithm to group antibodies into a set of K clusters [24]. Antibodies that exhibit similar neutralization activities across different viruses are grouped into the same groups. For the CATNAP data set, we used K = 100, though our results are robust to a range of values for K between 50 and 200. Most clusters contain one or a few antibodies, 10–20 clusters contain 10–20 antibodies, and only a few contain around 100 antibodies.

Extracting sequence features and learning neutralization values

Here, we used a simple approach to embed viral sequence data into a lower-dimensional space. We computed the eigenvalues and eigenvectors of the viral sequence covariance matrix and then projected each viral sequence onto the eigenvectors corresponding to the d largest eigenvalues (S1 Text). Specifically, given a binary (one-hot) encoding of a viral sequence , , we generate a projected sequence . Here is a projection matrix, with . This approach is statistically efficient, and it takes advantage of correlated patterns of mutational variation observed in the data, which can affect virus sensitivity to antibodies. For example, in the broad HIV-1 CATNAP data set, we observed that different viral subtypes tend to occupy distinct regions in the eigenspace due to shared, subtype-defining mutations (S2 Fig). While our main approach simply uses principal components, we also explored alternative dimensional reduction methods, including non-negative matrix factorization (NMF). However, the accuracy of our predictions changed little across different low-dimensional spaces as long as they provided statistically important features to distinguish viral sequences.

We used linear regression to learn the relationship between sequence features and neutralization values, inferring a separate family of regression models for each cluster of antibodies (S1 Text). For each cluster, we set d to be proportional to the number of unique viruses with measured neutralization values for antibodies in the cluster, up to a maximum of 100. We used the antibody similarity measures defined above, , to penalize differences between the linear regression parameters and for each pair of antibodies and within the same cluster. In this way, similar regression parameters will be learned for antibodies with similar neutralization profiles (i.e., ). In the limit that , the regression parameters for antibodies and become independent. Overall, this approach allows the information from virus-antibody neutralization data to be shared among similar antibodies.

Low-rank approximation for the neutralization matrix

Einav and collaborators found that typical virus-antibody neutralization matrices have a “low-rank” structure [20, 22, 21]. In other words, the neutralization profiles for each virus and antibody are not completely unique; virus-antibody neutralization scores tend to be highly correlated across antibodies (for closely-related viruses) and across viruses (for antibodies that bind similar epitopes in a similar way). Thus, the neutralization matrix can be well-characterized by a number of modes much smaller than its naive dimensions. Our approach makes use of the same insight.

After inferring linear regression parameters for each antibody , we used the regression model to predict the neutralization value for each virus-antibody pair that was not measured in the data set. We then refined this matrix by applying a low-rank approximation. Specifically, we used singular value decomposition (SVD) to factorize the preliminary U matrix, consisting of the measured with the unobserved entries filled by the regression model. We then selected the top singular values and basis vectors to approximate the matrix. As in prior work [20], we selected the rank to be the smallest rank such that the cumulative variance explained (equal to the sum of squared singular values) is greater than 95%. Our final predictions for the unobserved are then given by the entries of this low-rank approximation of the completed U matrix after regression.

In the previous work, Einav et al. used robust principal component analysis or nuclear norm minimization to develop a low-rank approximation of the neutralization matrix [20]. We also tested these methods, but in our analyses, we found that there was little difference between them and the simpler SVD factorization. Consistent with Einav et al., we also observed that the spectrum of singular values could reveal information about the neutralization matrix. We found that the fraction of variance explained scales roughly as a power law for intermediate eigenmodes, ranging from around rank 20–100 (S3 Fig).

Neutralization and virus genetic data

In this study, we consider two types of neutralization matrices derived from monoclonal anti-HIV-1 antibodies and HIV-1 strains. The first is a neutralization matrix obtained from cross-sectional studies in the CATNAP database [23] provided by Los Alamos National Laboratory (LANL) [25]. To ensure data quality, we first removed viruses and antibodies that had few associated neutralization values (<10 entries for viruses), and then antibodies that do not have any entries. We also removed viruses that had not been sequenced.

After filtering, we obtained a data set with N_A = 806 antibodies and N_V = 1,147 viruses, including 81,193 observed neutralization values. As indicated above, neutralization data in large data sets such as CATNAP are sparse: the measured neutralization values correspond to 4.5% of the total matrix entries.

The second data set comes from an HIV-1 intrahost longitudinal study, which tracked the development of bnAbs (CH235 and CH103) in one donor, CH505 (ref. [26,27]). We obtained virus-antibody neutralization values from Gao et al. [27] and corresponding HIV-1 Env sequences from LANL’s HIV sequence database [25]. This matrix includes N_A = 22 antibodies and N_V = 109 virus sequences, with 1,723 measured neutralization values. In contrast to the CATNAP database, 71.9% of the possible neutralization matrix entries were measured in this data set. More details on the formatting of the neutralization matrices, the processing of genetic sequences, data sources, and the parameters of the models are provided in S1 Text, S1 Table and S2 Table.

Overall GNL performance on CATNAP data

To test the performance of the GNL method, we masked a random fraction of virus-antibody neutralization values in the CATNAP data set. We then used GNL to predict the masked neutralization values and compared our predictions against the true measured values. First, we masked entries of U uniformly at random (i.e., each observed value was equally likely to be masked). Fig 1c shows a typical comparison between the true and predicted neutralization (log IC50) values when 20% of the data is masked, which are strongly correlated. In this example, there are 780 antibodies and 1,097 viruses in the withheld data.

We also compared our approach versus that of Einav et al. (ref. [20]) as we varied the fraction of the data used for training and validation, and the rank used in the low-rank matrix approximation. To measure typical performance, we repeated each test 10 times for each data fraction and value. Overall, we observed that the correlation between true and predicted log IC50 values was typically higher for GNL than for the alternative (Fig 1d, e). Comparatively, the results for GNL were especially better when the neutralization data were sparser. We also observed that GNL performance was more robust to changes in the rank , with a wider plateau near peak performance. This robustness is especially important for practical applications, where the true neutralization values are unknown, because it implies that strong predictions can be obtained even without fine-tuning parameters.

Determinants of prediction accuracy

Next, we aimed to identify conditions that make predicting virus-antibody neutralization values more or less difficult. We focused on two key factors in the neutralization data: the diversity of neutralization values and the amount of data available (Fig 2a-b; see also S4 Fig). Intuitively, antibodies with more observed neutralization values provide more information for training, potentially enabling higher accuracy. Antibodies with a broad range of neutralization values (i.e., ones with substantially different IC50 values for different virus strains) may also be more difficult to model than those with flatter neutralization profiles.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. GNL improves predictions when neutralization measurements are widely divergent or when data is sparse.

Overall distribution of the standard deviation of the neutralization measurements per antibody (a), and their number of neutralization measurements (b). As the fraction of observed data decreases, accuracy drops (i.e., MSE increases); however, GNL consistently shows lower MSE, especially at smaller data fractions (c). GNL also performs better for antibodies with higher diversity, as measured by standard deviation (d), or with fewer observed measurements used for training (e). Greater diversity in neutralization measurements increases MSE across all withholding levels for both INL and GNL (f-g). However, GNL better mitigates this accuracy drop (h). Similarly, we observed that fewer training measurements lead to higher MSE across all withholding levels (i-j). However, the GNL method maintains lower MSE with fewer observations (k). S4 Fig presents the same analysis including the Einav et al. method.

https://doi.org/10.1371/journal.pcbi.1014095.g002

Because some subsets of the data contained only a small number of virus-antibody pairs, we used the mean squared error (MSE) between predicted and observed neutralization values to quantify accuracy in these comparisons. With few data points, correlation metrics such as Pearson’s R may not fully capture the correspondence between predictions and data. This is especially true for neutralization values with little variation, where predictions may be quite accurate on an absolute scale even if they are not strictly ordered correctly. For example, antibodies VRC34-UCA, VRC34.05, vFP5.01, A12V163-a.02, DH650.8, and m66 have more than 200 neutralization observations, but the variance in measured log IC50 values for each is less than 0.1.

For reference, we compared our results against a simple version of our model where we treat each antibody independently. This independent neutralization learning (INL) model is equivalent to simple linear regression to predict neutralization values for each antibody. Overall, we found that MSE values tend to decrease as the amount of training data increases and as the variance in the training data decreases (Fig 2c-e). Errors for GNL were always smaller than those for INL, demonstrating the value of shared information across antibodies with similar neutralization profiles.

We grouped antibodies into five classes according to the standard deviations of their neutralization values. Fig 2f-g shows that MSE increases as the diversity of log IC50 values increases. Consistent with analyses in Fig 1, we set the fraction of data observed as 80%. More generally, when large fractions of the observed data are used for training, the difference in MSE values between the GNL and INL methods becomes more pronounced when the diversity of neutralization measurements is higher (Fig 2h), a setting in which learning neutralization values is more challenging.

We further partitioned antibodies based on the number of neutralization measurements used for training (Fig 2i-k). Naturally, MSE values tend to be larger for antibodies with fewer training data. However, the advantage of GNL over INL is especially prominent in the cases where data is most restricted, including a small fraction of the data observed, smaller training data sets, and less variation in the training data (Fig 2h, k). This suggests that the grouping mechanism in GNL, which allows predictive information to be shared across similar antibodies, is especially helpful in learning genotype-to-neutralization mappings with sparse data.

Integration of alignment-free sequence features

HIV-1 Env hypervariable regions involve frequent mutations, insertions, and deletions, which can make sequence alignment challenging in these regions. A prior study incorporated alignment-free features to characterize patterns associated with viral sensitivity to bnAbs [28]. In this study, we also explored the predictive power of models that combine alignment-free features along with the sequence-based features described above.

We retrieved alignment-free features, such as loop length, net charge, and the number of potential N-linked glycosylation sites (PNGSs), within the five variable regions (V1–V5), totaling about 30 additional features, from the LANL database. In addition to these alignment-free features, we used sequence alignments that include an additional letter ‘O’ to represent PNGSs, alongside the standard 20 amino acid letters. These additional features are incorporated into the GNL framework (see S1 Text for details). To evaluate neutralization prediction accuracy from multiple perspectives, we considered several statistical metrics, including Pearson’s R, Spearman’s , regression slope, concordance correlation coefficient, and MSE.

Here, we found that the effect of including alignment-free features and the PNGS symbol in the model was minor, and we observed no significant difference in performance with or without these additional features in this setting (Fig 3). As discussed in the next section, we also examined the influence of these additional features on predicting other viral sensitivity measures such as IC80 and Hill slope. While their overall effect remains small, we observed slight improvements in accuracy for certain cases, such as the Hill slope (S5 Fig, S6 Fig and S7 Fig). Therefore, alignment-free features could help identify patterns in viral sensitivity and improve interpretability. However, alignment-based features were sufficient to learn IC50 values in this example.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Accuracy of IC50 predictions assessed by multiple statistical metrics.

We compared the performance of models that learn the relationship between neutralization and genetic sequences using only sequence alignments with the standard 20 amino acids and gaps (dashed lines) to models that incorporate both sequence alignments and alignment-free features (AFFs). Specifically, we used loop length, net charge, and the number of PNGSs in variable loop regions (solid lines) for the AFFs. To assess accuracy, we employed multiple metrics: (a) Pearson’s R, (b) Spearman’s , (c) regression slope, (d) concordance correlation coefficient, and (e) MSE. Error bars represent 95% confidence intervals, estimated by randomly resampling the training and evaluation sets ten times. The concordance correlation coefficient (panel d) is computed as   where r, , and denote the Pearson correlation coefficient, mean, and variance of one dataset (x-axis), respectively. Overall, models that include alignment-free features achieve virtually the same prediction accuracy across a wide range of matrix ranks and across all evaluated metrics.

https://doi.org/10.1371/journal.pcbi.1014095.g003

Predicting viral sensitivity metrics beyond IC50

IC50 is just one of multiple possible metrics that can be used to assess viral sensitivity to antibodies. IC80 (the 80% inhibitory concentration) estimates the antibody dose required to achieve a greater reduction in viral replication than IC50. The Hill slope h describes how steeply neutralization changes as antibody concentration varies; by definition, it quantifies how sensitive neutralization titers are to changes in dosage. We characterized the Hill slope by IC50 and IC80 values, such that

(1)

Another important metric is the instantaneous inhibitory potential (IIP) [29,30]. Mathematically, the IIP at antibody concentration c is defined as

(2)

By definition, IIP quantifies the order of magnitude of the viral population that is neutralized (e.g., IIP = 1 corresponds to 90% neutralization, IIP = 2 corresponds to 99% neutralization, and so on).

To assess the robustness of the proposed prediction method, we explored predictions of IC80, Hill slope, and IIP values. Although IC50 has the most extensive collection of observations in the CATNAP database, IC80 values are also available, and a subset of measurements includes both IC50 and IC80. From these data, we estimated Hill slope h and IIP values using the expressions above. We evaluated IIP at a fixed antibody concentration of throughout this study. Because IC50 and IC80 values are often very close, accurately estimating the Hill slope can be challenging. Therefore, we considered only Hill slope values below a threshold, specifically setting a maximum Hill slope of (corresponding to a very steep sigmoidal curve), and treated only values below this threshold as reliable observations.

Overall, we observed consistent trends in the relationship between observed fraction and accuracy (Fig 4), the relationship between prediction accuracy and matrix rank, with reduced matrix rank generally improving prediction accuracy (S5 Fig). Our method is robust to the choice of matrix rank, and Pearson’s R exceeds 0.8 across a wide range of rank values when 80% of the data are observed. However, we note that prediction accuracy for Hill slope and IIP values is relatively insensitive to matrix rank (S5 Fig, S6 Fig and S7 Fig), and that GNL’s Pearson and Spearman correlation coefficients are comparable to or slightly lower than those obtained using the method of Einav et al. In contrast, for the regression slope and concordance correlation coefficient, the GNL method achieves higher performance than the other methods across all scenarios (S8 Fig and S9 Fig). Therefore, these results indicate that learning the relationship between genotypes and Hill slope is more challenging than learning IC50 or IC80 independently. This may be due to statistical uncertainty in estimating the Hill slope from IC50 and IC80 values, which can also propagate into IIP estimates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Prediction of viral sensitivity measures beyond IC50. Pearson’s R values as a function of the fraction of observed data are shown for (a) IC50, (b) IC80, (c) Hill slope, and (d) instantaneous inhibitory potential (IIP), using the optimized rank values (S1 Text).

Overall, increasing the fraction of observed data improves prediction accuracy across all metrics. However, for Hill slope and IIP, Pearson’s R values obtained using the GNL method are lower than those from the method of Einav et al., whereas GNL achieves higher regression slope, concordance correlation coefficients, and MSE values (S8 Fig and S9 Fig). This suggests that while GNL captures the overall scale and agreement of predictions, rank-based correlation is more difficult to achieve for these measures. We speculate that predicting Hill slope involves a more complex relationship with genotypes, likely because Hill slope is estimated from a limited number of IC50 and IC80 pairs, which introduces additional statistical uncertainty.

https://doi.org/10.1371/journal.pcbi.1014095.g004

Interpretability of model parameters

HIV viral sensitivity to antibodies is strongly influenced by amino acid variation in the Env protein, and certain mutations, such as those leading to glycan loss or shifts (e.g., at N332), can substantially impact sensitivity. To gain a better understanding of the genotype–sensitivity relationship, we analyzed which mutations most strongly contribute to the predicted neutralization values.

We first obtained the trained model parameters learned from all observed IC50 values using the GNL method, which we consider to provide the most accurate model. We then calculated effective mutation-specific weight parameters, derived from the trained model parameters and the transformation mapping one-hot encoded sequences to projected variables (S1 Text). These effective weights correspond to the contributions of individual mutations to the predicted values. Using these weights, we analyzed mutation contributions for specific classes of antibodies.

This analysis reveals the effective contributions of individual mutations across sites and amino acids, averaged over multiple antibodies within each antibody class (Fig 5a). V3-glycan antibodies are known to depend on the N332 glycan; therefore, the presence of a PNGS (‘O’) at position 332 is expected to decrease IC50, while loss or alteration of this glycan can lead to viral escape and increased IC50 values. Consistent with this expectation, for V3-glycan antibodies we observed that the contribution of mutation O332 is significantly negative; this mutation ranks 9.5 on average, corresponding to the top 0.1% of contribution values across all 110 V3-glycan antibodies (). Mutations N332 and T332 are significantly positive contributions, ranking 411.5 (5.0%) and 259.0 (3.1%), respectively when averaged over antibodies ( and , respectively; Fig 5b). Here, p-values are based on Fisher’s exact test, and the top 1% of mutations are selected according to the absolute values of their effective weights (S1 Text). Similarly, mutations S334 and O334 exhibit significant positive and negative contributions, respectively, ranking 8.5 (top 0.1%) and 42.0 (0.5%) in average contriubtion rank across all V3-glycan antibodies ( and ; Fig 5c). We also examined effective weight values for other antibody classes. For example, for V2 apex antibodies, negatively charged mutations are known to confer resistance. Consistent with this observation, our analysis shows that the positively charged mutation K171 decreases IC50 values and ranks 83 (top 1.0%), with a corresponding p-value of . (S10 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. GNL’s model parameters reveal key mutations that impact viral sensitivity.

We show the contributions of individual mutations to the predicted values, which indicate how much the predicted changes in the presence of each mutation. These estimated contribution values are averaged over all V3-glycan antibodies, comprising 110 antibodies. The selected sites include residues that can directly contact or indirectly influence binding affinity with V3-glycan antibodies [31–36]. Overall, mutations involving ‘O’, ‘N’, ‘T’, and ‘S’ tend to contribute significantly to the predicted IC50 values (a). Here ‘O’ indicates PNGS. We further show the contributions to for individual mutations at sites 332 and 334, which are frequently associated with glycan involvement (b, c). Mean contribution values across all antibodies are indicated by open circles.

https://doi.org/10.1371/journal.pcbi.1014095.g005

Beyond these specific mutations, we systematically examined mutations that exhibit significantly large contributions to the predicted log IC50 values and are shared across antibodies within the same class. Using these criteria, we selected 7, 5, 11, 8, and 6 mutations for the V3-glycan, CD4bs, V2-apex, MPER, and Interface/FP antibody classes, respectively (S3 Table). Among the 37 selected mutations, 23 are associated with altered viral sensitivity, and all mutations identified in the V3-glycan class affect glycan-related features (S3 Table). The majority of these mutations exhibit consistently positive or negative contributions to log IC50, corresponding to increased resistance or sensitivity to antibody neutralization. In contrast, mutations associated with the V2-apex class often display context-dependent, or dual, effects (S11 Fig).

Comparison against alternative approaches for virus-antibody neutralization prediction

Alongside low-rank approximation methods, there are other families of neutralization prediction approaches, including ones based on machine learning [11–13,16,18]. One widely used tool is SLAPNAP (Super Learner Prediction of NAb Panels) [13]. SLAPNAP has been applied to several studies and predicts neutralization values against bnAbs using viral envelope protein sequence features [14,15]. The underlying models include random forests and boosted regression trees with L₁ regularization. For combinations of bnAbs (a setting that we do not consider in this work), SLAPNAP uses the Bliss-Hill model [37], which estimates single bnAb neutralization curves via a Hill function and combines their effects based on Bliss independence.

To compare SLAPNAP’s performance with the GNL method as well as the Einav et al. method [20], we first replicated their training settings. In particular, we trained each model using V-fold cross-validation. In this setup, the data is divided into V parts: are used for training, and the remaining part is used for validation. This is repeated V times. We set V = 5, following prior work [13]. To ensure a fair comparison, we used the same neutralization data set (from CATNAP, as of July 23, 2021) previously tested for SLAPNAP. Following ref. [13], we used a subset of bnAbs that bind to different Env epitopes to compare the performance of GNL, SLAPNAP, and the Einav et al. method. Compared to the full CATNAP data set, the selected antibodies often had a large number of neutralization measurements (>200, for comparison see Fig 1b).

Fig 6 reports CV-R values for GNL, SLAPNAP, and Einav et al. methods on this data set, showing that GNL performs well across a wide range of antibodies. Across all antibodies, GNL achieved an average CV-R of 0.60, compared to 0.51 for SLAPNAP and 0.43 for the Einav et al. approach (S4 Table). For antibodies DH270.5, DH270.6, and VRC38.01, SLAPNAP was unable to produce a CV-R value. These antibodies have fewer neutralization measurements than other antibodies in this set, and thus the lack of a CV-R result may be due to a data restriction in SLAPNAP. To ensure a fair comparison across methods, we thus excluded antibodies for which SLAPNAP did not produce CV-R values. We also reported the Spearman’s correlation value based on cross-validation in the Supplementary section (S12 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Performance comparison of GNL and existing methods on CATNAP data.

Cross-validation Pearson’s R (CV-R) for neutralization predictions of bnAbs across a spectrum of antibodies with GNL, SLAPNAP [13], and Einav et al. For a few antibodies (DH270.5/6 and VRC38.01), SLAPNAP produced no output. The average CV-R of GNL across all antibodies (0.60) exceeded SLAPNAP (0.51) and Einav et al. (0.46) (S4 Table). GNL performance was especially strong for antibodies binding the variable loops (V1/V2 and V3) and the CD4 binding site. Error bars represent 95% confidence intervals based on a random selection of viruses used for cross-validation.

https://doi.org/10.1371/journal.pcbi.1014095.g006

Zero-shot prediction of neutralization values for novel viral sequences

Next, we tested the ability of GNL to perform “zero-shot” predictions of virus-antibody neutralization values for virus sequences that never appeared in training data. Predicting neutralization values for novel sequences is challenging because it requires learning generalizable relationships between virus properties (e.g., sequence, structure) and resistance, rather than simply relying on observed neutralization patterns. Matrix completion methods that depend exclusively on observed virus-antibody pairs cannot make predictions for entirely new viral sequences. To evaluate GNL’s zero-shot capabilities, we considered two experimental scenarios using longitudinal data from a single HIV-1 infected donor, CH505, and cross-data set predictions between independent neutralization studies.

As a first example, we considered longitudinal neutralization data from donor CH505, who was monitored over five years and developed bnAb lineages, CH103 and CH235, binding to CD4bs (Refs. [26,27]; see also S5 Table). The CH505 data set includes neutralization values for N_A = 22 antibodies tested against N_V = 109 viruses over six time points [26,27]. To validate our approach, we first confirmed that GNL could accurately predict randomly withheld neutralization values, achieving strong correlation (R = 0.89) when 80% of the data was observed (S13b Fig, see also S1 Text for the rank optimization scheme). We then examined two temporal prediction scenarios: predicting neutralization values for past viral sequences using models trained on later time points, and predicting future viral sequences using models trained on earlier data.

For temporal predictions, we split the neutralization data based on collection time, using early weeks post-infection (28, 96, and 140 weeks) and later weeks (210, 371, and 546 weeks) as separate training and validation sets. When predicting antibody-virus neutralization values for early viruses based on late infection data, GNL achieved Pearson’s R-values of approximately 0.8 across all time points (Fig 7c). For the more challenging task of predicting neutralization values for future viral sequences based on prior (early infection) data, GNL maintained good predictive accuracy (Pearson’s R ranging from 0.7 to 0.55) despite the continued accumulation of mutations (Fig 7b). This reflects the ability of GNL to learn generalizable maps from viral sequence to neutralization.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. GNL accurately predicts neutralization values for viral sequences across evolutionary time.

a, Visualization of the neutralization matrix, consisting of N_A = 22 antibodies and over N_V = 109 viral sequences sampled across six time points. b, Prediction of the neutralization values for past viral sequences. Neutralization values were predicted for sequences sampled at or before 140 weeks, using a model trained only on data collected after 140 weeks. c, Prediction of the neutralization activities for future viral sequences. The model was trained on values observed up to 140 weeks, and predictions were tested on sequences sampled after 210 weeks. GNL maintains accuracy even as viral sequences diverge from training data over time.

https://doi.org/10.1371/journal.pcbi.1014095.g007

In a second test, we evaluated whether GNL could predict neutralization values for an individual host using only data from the broader CATNAP database. We trained models exclusively on CATNAP data and predicted neutralization values for the CH505 individual host data set, focusing on the two antibodies that are present in both the CATNAP and CH505 data sets (Fig 8a). Since the CATNAP database and individual host data are largely disjoint, with none of the CH505 virus sequences present in CATNAP, this represents a stringent test of zero-shot predictive power. Here, GNL achieved meaningful predictive accuracy with a Pearson’s correlation of R = 0.55 between predicted and true neutralization values (Fig 8c). In comparison, predictions based on matrix completion are highly limited due to the absence of the specific test virus sequences in the training data (Fig 8b). These results show that GNL can successfully transfer neutralization patterns from cross-sectional studies to predict virus-antibody interactions in independent data sets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Neutralization values can be predicted for novel viruses using cross-data set transfer learning.

a, Schematic showing a subset of the CATNAP database (left) and the individual host (CH505) neutralization matrix to be predicted (right). CATNAP contains N_A = 806 antibodies and N_V = 1,147 viruses, while the CH505 matrix includes 22 antibodies and 109 viral strains. Only two antibodies are shared between the data sets. b, Comparison of true neutralization values from CH505 and predicted values using the Einav et al. method [20], which can only predict mean values from the training data. c, GNL achieves meaningful predictive accuracy for the same task. GNL’s (Einav et al.’s) Pearson’s R, Spearman’s , linear regression slope, MSE, values are and , respectively. As indicated by the regression slope, the predicted values are systematically smaller than the true ones in this example.

https://doi.org/10.1371/journal.pcbi.1014095.g008