Single-view versus multiview learning

The advancement of high-throughput technologies, which has resulted in tremendous amounts of biological data, has transformed biology from a descriptive science into a predictive science in which machine learning plays an important role. Although biological data are different from visual or speech data, all machine learning algorithms share a common mathematical background that can be described as the ERM principle [6]. In the following sections, we provide the formal descriptions of supervised and unsupervised learning alongside their corresponding ERM estimators.

Supervised learning.

In supervised setting, we have n labeled examples where (the label set) and (the domain), sampled from an unknown underlying joint distribution over . The goal is to find a function in a hypothesis space that predicts the output associated to any new pattern by f(x), as measured with respect to a known loss function . Note that function f stands for any transformation, ranging from linear projection to deep neural network and kernel function. For a candidate function , its empirical risk [6] is (1) and the regularizer controlling its smoothness is Ω(f) where is a penalty function ( is the set of nonnegative real numbers). The penalized ERM estimator is (2) where λ is regularization parameter. For example, support vector machine (SVM) is the combination of hinge loss and -regularizer, whereas ordinary least squares makes use of squared loss, etc. The basic idea of supervised learning is demonstrated in Fig 2A.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. MV-ERM.

(A) ERM for single-view learning. It demonstrates a general single-view learning algorithm (based on ERM estimator) that takes one data set as input, adopts a hypothesis space and a loss function , and outputs a function that predicts the label associated with any new datapoint x as f⁽¹⁾(x). (B) MV-ERM demonstrates a general multiview learning algorithm (based on MV-ERM estimator) that takes v datasets as v views, adopts v hypothesis spaces associated with v views, and outputs v functions that reveals the interactions within and between each pair of datasets (via the terms Ω_co(f⁽ⁱ⁾,f^(j)). The consensus and complementary principles are implemented by the term Ω(f⁽ⁱ⁾) and Ω_co(f⁽ⁱ⁾,f^(j)) respectively. Note that in MV-ERM estimator, the loss function is optional because multiview learning can be unsupervised. ERM, empirical risk minimization; MV-ERM, multiview empirical risk minimization.

https://doi.org/10.1371/journal.pcbi.1007677.g002

Unsupervised learning

In unsupervised setting [7], we have n unlabeled examples where sampled from an unknown underlying distribution over . The goal is to find a latent structure (low-dimensional or clustering representation, latent factors, etc.) from encoded by function in hypothesis space and decoded by function in hypothesis space , as measured with respect to a reconstruction error . For a pair of candidate functions , its empirical risk is (3) The ERM estimator is (4) Using a reconstruction error, this framework is general enough to encompass a variety of algorithms, such as principal component analysis (PCA), k-means, nonnegative Matrix factorization (NMF), autoencoder, etc. In fact, the equivalence of NMF and spectral and k-means clustering was investigated [8]. Both k-means and PCA methods can be considered as special cases of autoencoders [9]. Several studies have been explored the additional constraints for an autoencoder to perform NMF [10, 11, 12, 13]. Also, NMF has a good interpretability because, for example, it factorizes a gene expression profiles into 2 matrices, one of which describes the structure between genes while the other describes the structure between samples [14]. It also has good performances, especially in single-cell studies [15]. Because of the equivalence of NMF and other unsupervised methods, we represent here the formal settings of NMF as a typical case of unsupervised learning without loss of generality:

Given a data set represented by a nonnegative matrix X, NMF decomposes X into the production of nonnegative matrices G and F, i.e., X≈GF^T. The ERM estimator of NMF can be formulated as follows: (5)

Note that the objective function (5) takes the matrix form of unsupervised ERM estimator (4) where all x_i from S are column vectors of matrix X, f(X) = F, and G is the matrix representation of the linear operator g(⋅).

Pitfalls of single-view learning

Through the formal definitions of machine learning identified previously, its applications in biological domains can be regarded as abstracting out a representation of a single data type , where can be, for example, a gene expression profile. This representation captures the interactions of elements (i.e., genes) within and the phenotypic manifestation (e.g., cancer) resulting from those interactions. However, to understand complex traits in which the genotype—phenotype interactions manifest over multiple levels of information flows, relying on merely one single omics data type is limited and prohibits our knowledge from uncovering the comprehensive mechanism that underlies complex biological processes. Even with the availability of more than one data type (e.g., for gene expression, for methylation level), because they originate from different distributions, applying machine learning algorithms on these data independently may solely assemble some pieces of the puzzle of a complex phenomenon; many additional pieces associated with the interactions across different data types remain unknown. Therefore, a learning method that exploits not only information captured in each omic but also that infers from the associations between different omics is needed to understand complex traits. Multiview machine learning is such a method.

Multiview learning

Many real-life datasets comprise diverse views or modalities; for example, a website can contain both images and texts that refer to the same content. In multimedia applications, for instance, both the speech and lip motions of a character are often simultaneously accessible. These views or modalities are often compatible, which helps the learning model be more robust as well as complementary and thereby reveal further information that cannot be fully uncovered when depending on only one view.

In biology, the need for multiview data is quite trivial; on one hand, we have homogeneous biological data assayed from the same molecular level (e.g., gene or protein expression), yet these homogeneous data may be measured across different conditions, phenotypes, or species. In this context, comparative analysis is important to, for example, find a conserved gene set that functions in the same pathway between 2 different species. On the other hand, multiomics is heterogeneous data in which we have different omics (e.g., genomics, proteomics, epigenomics) assayed from the same tissue or cell. These various omics are encoded by different data views, such as for transcriptomic abundance or for protein concentration. The goal of multiview learning is to exploit multiple representations of the input data and improve the learning performance. Herein, we set up 2 formal frameworks of multiview learning: one based on the ERM principle of supervised learning (Eq 2) and the other based on the ERM principle of unsupervised learning (Eqs 4 and 5). We also analyze the following 2 characteristics of multiview data that underlie those frameworks: consensus and complementary.

Consensus and complementary principles

The consensus principle seeks to maximize the agreement among multiple distinctive representations of the data. In short, given an example being seen in 2 views with its label y∈Y, the goal is to maximize the probability: (6)

The complementary principle demonstrates that in a multiview learning problem, each representation or view may contain information that does not exist in other views. Therefore, combining different views makes the predictor more accurate, or in other words, improves the learning performance [16, 17]. The 2 principles are demonstrated in Fig 2B.

MV-ERM

In multiview setting, we have n labeled examples and m unlabeled examples , where and each example x = (x⁽¹⁾,x⁽²⁾,…,x^(v)) is seen in v views with for i = {1,2,…,v}. S and U are both sampled from an unknown underlying joint distribution over . The goal is to find v functions in v hypothesis spaces where , predicting the output associated to any new pattern by f⁽ⁱ⁾(x), as measured with respect to a known loss function . For a candidate function , its empirical risk is (7) And the regularizer controlling its smoothness is Ω(f⁽ⁱ⁾), where is a penalty function. Also, to impose a penalty on the complexity of each pair (f⁽ⁱ⁾,f^(j)) in a cross-product of 2 hypotheses in order to utilize unlabeled data in different views, we define the co-regularizer (8) The penalized MV-ERM estimator is (9) In Eq 9, the last term, co-regularizer Ω_co(⋅), preserves the consensus principle for multiview learning. Note that if λ_co = 0, this problem reduces to solving 2 independent problems, meaning only the complementary principle is used. In the following, we present 2 frameworks, i.e., alignment-based and factorization-based, for multiview learning that covers most of the recent methods. The basic idea of multiview learning is demonstrated in Fig 2B.

Alignment-based framework. In the Eq (9) of MV-ERM estimator, if there is no labeled data, i.e., n = 0 (or S = ∅), the problem is then (10) which can be seen as an alignment problem finding a set of embeddings (f⁽¹⁾,…,f^(v)) that transform the original multiview data into a new common space by identifying an alignment strategy denoted by the co-regularizer Ω_co(⋅). This co-regularizer serves as a pairwise symmetric alignment function across all different views to coordinate the information among them. This multiview framework is based on supervised setting of single-view machine learning in Eq (2) where the loss function is optional, so the learning algorithm will try to uncover the v functions not by comparing with a ground truth (i.e., y_i) but by comparing to each other (in a pairwise fashion), depicted by the co-regularization term Ω_co(f⁽ⁱ⁾,f^(j)). The co-regularization term can be correlation-based, in which the 2 embeddings f⁽ⁱ⁾(x⁽ⁱ⁾),f^(j)(x^(j)) are maximally correlated, or distance-based, in which the Euclidean distance between the 2 embeddings is minimized. This kind of multiview learning can be regarded as self-supervision in which the v learners try to learn from each other’s data.

Factorization-based framework. The second framework for multiview learning is based on single-view unsupervised learning (Eq 4), trying to seek a common latent representation for multiple different views. In terms of NMF (Eq 5), given a multiview nonnegative data set consisting of v different views as , for each view , multiview NMF factorizes X⁽ⁱ⁾≈G⁽ⁱ⁾F^(i)T, where F⁽ⁱ⁾ = f⁽ⁱ⁾(X⁽ⁱ⁾), and learns a common latent representation across all the views via the following MV-ERM estimator: (11) where λ is the regularization parameter, trying to balance the importance of different views and the reconstruction error. The latent representations F⁽ⁱ⁾ in different views are forced to be close to the consensus one [18]. In any deep learning architecture, the joint latent representation can be achieved by a joint layer preceded by separated layers corresponding to separated multiple view inputs.

Both 2 aforementioned frameworks can be regarded as representation learning approaches. Whereas in the alignment-based framework, data representations from each pairs of views are forced to be coordinated, the representation of all views in the factorization-based framework are forced to be the same. The consensus principle is demonstrated by the co-regularizer Ω_co(f⁽ⁱ⁾,f^(j)) in the alignment method and by common latent representation (or sometimes called dictionary) in the factorization method; the complementary principle is demonstrated by the regularizer Ω(f⁽ⁱ⁾) in alignment methods and by expansion coefficients G⁽ⁱ⁾ in factorization methods.

Multiomics interpretation of multiview learning

In terms of functional omics, each view of multiview data X⁽ⁱ⁾ can be a gene expression profile, DNA methylation level, or protein abundance. Multiview learning algorithms applied to these data aim to infer the interactions within each omic, represented by f⁽ⁱ⁾(X⁽ⁱ⁾) or G⁽ⁱ⁾, as well as the interactions across all omics, represented by Ω_co(f⁽ⁱ⁾(X⁽ⁱ⁾),f^(j)(X^(j))) or . In other words, multiview machine learning attempts to recover a common abstract space wherein the several types of omics data are comparable such that the cross-talk patterns may easily be revealed. For example, in single-view machine learning, gene expression profile clustering is one method of revealing functional modules in which a group of genes collaborate to deliver a biological function. However, the insights of many complex biological processes cannot be understood in terms of these functional modules at the transciptomic level. On the contrary, multiview learning can find a way to represent both gene expression X⁽ⁱ⁾ and protein expression X^(j) together such that the interactions of genes as well as the interactions between genes and gene products (e.g., proteins) can be captured for a holistic understanding of complex biological phenomena. For example, if gene expression, chromatin accessibility, and protein expression are represented in a common space, they can be simultaneously clustered not only such that a group of genes or a group of proteins that function together can be identified but also—and more importantly—such that the functional linkage between genes, regulatory elements, and proteins can be revealed (e.g., protein α binds to region β to regulate the expression of gene γ). Fig 3 illustrates this example using a factorization method. With a closely related machine learning technique called transfer learning, we can even infer information of an omic level from another omic level. As for homogeneous data across different species, multiview learning can be applied to infer and transfer knowledge from one species to another species [19].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Factorization-based versus alignment-based methods.

(A) Factorization-based single-view learning methods. They typically factorize a data matrix X from single view (e.g., gene expression matrix of samples by genes) into a product of matrix G (coefficient matrix) and matrix (dictionary matrix or pattern matrix). Because matrix factorization has an intrinsic clustering property [8], the matrix can represent a clustering structure of the view (i.e., the soft clustering assignments or indicators). For example, reveals 3 different gene clusters, a, b, and c, as denoted in the figure. (B) Factorization-based multiview learning methods. They factorize different matrices from multiomics, e.g., gene expression X⁽¹⁾ (i.e., green matrix), protein expression X⁽²⁾ (i.e., blue matrix), and chromatin accessibility X⁽³⁾ (i.e., orange matrix), into a product of different coefficient matrices G^(k)(k = 1,2,3) and the common dictionary matrix . This common representation enables revealing of cross-talk patterns among genes, proteins (more precisely, TFs), and regulatory elements (i.e., enhancers); e.g., a TF binds to a region to regulate a gene's expression. (C) Alignment-based multiview learning methods. The 3 input omic matrices are projected via functions f^(k)(k = 1,2,3) onto spaces where their internal relationships are revealed. These representations of different omics are pairwise coordinated to each other via the term Ω_co. For example, the figure demonstrates the pairwise alignments between X⁽¹⁾, X⁽²⁾ and between X⁽²⁾, X⁽³⁾ to reveal cross-talk patterns between TFs and enhancers, and between enhancers and gene expressions. (Alignment between X⁽¹⁾ and X⁽³⁾ is not shown for making the figure concise.) TF, transcription factor.

https://doi.org/10.1371/journal.pcbi.1007677.g003

Multiview learning methods

We categorized all recent state-of-the-art methods of multiview learning into 2 groups according to the frameworks explained in the previous sections: the alignment-based framework, which seeks a pairwise alignment among views, and a factorization-based framework, which seeks a common representation across all views. Each framework contains elements of either consensus principle or complementary principle or both. All the methods described in this review are summarized in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Multiview learning methods.

https://doi.org/10.1371/journal.pcbi.1007677.t001

Cancers

Cancer is a complex disease whose phenotypic manifestation might be related to many different levels of molecular signatures, such as gene expression and DNA methylation. In other words, cancer types and subtypes can be defined based on, for example, both genetic mutations and epigenetic landscapes. Therefore, any causal analysis based on solely one aspect or single omics will be a causal reductionism that might lead to insufficient results. Multiomics approaches in oncology research is thriving [63], and many applications of such approaches have been recently pursued.

Rappoport and Shamir [5] performed an extensive review and benchmark that compare 9 multiview methods on 10 cancer types using cancer datasets from The Cancer Genome Atlas (TCGA) spanning 3 omics—that is, gene expression, microRNA (miRNA) expression, and DNA methylation. Among the 9 algorithms chosen, LRAcluster [64], k-means, and spectral clustering are clustering methods that performs on the concatenation of various omics into a single matrix, which is a method often referred to in the literature as early integration [16]. The other 7 methods are similarity network fusion (SNF), regularized multiple kernel learning-locality preserving projections (rMKL-LPP), multiway canonical correlation analysis (MCCA), multiview NMF (MultiNMF), iClusterBayes, and PINS. These methods are chosen to reflect their categorization which, as stated therein, is overlapped; nevertheless, these practically available tools are widely used. The k-means, spectral clustering, and multiNMF methods are mentioned in the previous section. iClusterBayes [51] is also a factorization method belonging to the subspace clustering family that shares an objective function similar to that of NMF and k-means. MCCA [28] is a generalization of CCA wherein pairwise correlations between embeddings are maximally summed up. The rMKL-LPP method [40] is a case of MKL specifically developed for multiomics data. SNF [41, 42] is a graph-based approach that aims to fuse different graphs that represent various omics and may also be regarded as an MKL method on graph structure. The PINS method [65], belonging to the approach sometimes called late integration [16], is the averaging of all clustering results from different omics. In their paper, Rappoport and Shamir [5] demonstrate that rMKL-LPP performs most efficiently in terms of clinical enrichment, whereas MCCA and MultiNMF perform most efficiently with respect to survival. It is worth noting that rMKL-LPP is a specialized multiomic method, whereas MCCA and MultiNMF are general multiview learning methods.

Rappoport and Shamir [43] also developed a new method for multiomics clustering, neighborhood-based multi-omics clustering (NEMO), which can be applied to partial datasets in which patients' omic data are missing without performing data imputation. The authors also compared NEMO with the benchmark on a previous study [5] and demonstrated an improvement of their method on partial data. The idea behind NEMO is similar to those behind SNF and rMKL-LPP; all 3 methods are MKL approaches. Firstly, the similarity matrix of each omic is built based on a radial basis function kernel. Secondly, theses matrices are integrated into one average relative similarity matrix. Finally, spectral clustering is applied on this unified matrix wherein a modified eigenmap method is employed. The ability of NEMO to handle a partial data set is based on a local neighborhood approach. Experiements revealed that NEMO is faster and simpler than existing multiomics clustering algorithms.

Multiview NMF is also used for the selection of common codifferential genes [46]. In this paper, the authors implemented a graph-regularized version of multiview NMF (GMvNMF) to encode the data manifold of genomic data. Manifold regularization for multiview learning was first introduced in the form of manifold alignment. This kind of geometric information embedding can be applied to any multiview machine learning method, including GMvNMF. This method's validity was tested in 4 cancer multiomics data from TCGA, and each of these cancer types comprises 3 omics (i.e., gene expression, copy number variation, and DNA methylation). GMvNMF was demonstrated to perform more efficiently than other NMF methods, including plain multiview NMF.

Also using graph regularization, Zhang and Ma [52] proposed a regularized multiview subspace clustering (rMV-spc) method to discover common co-expressed modules. These modules serve as biomarkers across various cancer stages that might lead to the revealing of mechanisms that underlie the development of cancers. The graph regularization employed therein is the protein—protein interaction (PPI) network; their optimization procedure is based on interior point algorithm. They performed their method on breast cancer data from TCGA and reached a more favorable result compared with that of an artificial network benchmark. Yet while claiming the method's extensibility to heterogeneous multiomics data, in this study, the authors exclusively included gene expression data. Although the PPI network was also used, it is regarded as prior information in the form of regularization rather than a different source of data view.

Yu and colleagues [66] proposed a method for simultaneous clustering of multiview cancer data using a multiview spectral clustering method. However, their computational method substantially differs from other spectral clustering approaches in that, rather than calculating eigenvectors, the optimization procedure therein involves of a line-search algorithm on Stiefel manifold. In this gradient descent method, the gradient calculated from a Euclidean space in each iteration is projected onto an embedded matrix manifold. The authors applied this method to both simulation and real data, which also originate from TCGA. In both cases, the method performed favorably. In the real data set composed of gene expression, miRNA expression, and DNA methylation across 12 cancer subtypes, their method identified more clusters that are enriched by gene ontology and KEGG pathways, which could be used to explain the different mechanisms for each subtype of cancer development.

Brain diseases

The nature of mental disorders and neurodegenerative diseases, of which Alzheimer’s disease (AD) is the most common one, remains a puzzle. Although many psychiatry diagnoses are currently based on neuroimaging and, hence, multiview learning from different types of neuroimages (e.g., MRI, fMRI, positron emission tomography [PET], computerized tomography [CT]), a study of AD [67, 68] suggests that memory impairment and dementia are the results of nonlinear interactions involving multiple brain cell types (e.g., neurons, microglia), pathogenic forms of τ proteins and amyloid-β as the brain ages. Whereas these interactions are preserved in a healthy brain, they are dysfunctional in an unhealthy brain, thereby leading to mild cognitive impairment (MCI) and giving rise to AD. Also, each cell type is affected by unhealthy aging at multiple levels, such as transcriptomic, epigenomic, proteomic, metabolomic, and lipidomic. Therefore, a holistic approach that combines these omic data on blood, cerebrospinal fluid (CSF), brain samples, and also neuroimages as phenotypic traits is essential for revealing the complex mechanism underlying the disease. The combination of omics studies and medical imaging (sometimes called radiogenomics) advances our understanding of AD and neurodegenerative disorders in general at multiple levels through the identification of biomarkers for diagnosis and through association studies that reveal the interaction mechanism among genetic and phenotypic data.

Xu and colleagues [69] developed a Bayesian multiview learning method for association studies and diagnosis of AD through 2 data views: genetic variations (i.e., single-nucleotide polymorphism [SNP]) (discrete ordinal data) and MRI features (continuous data). By using sparse linear projections to factorize common latent features, the method aims to (1) simultaneously discover the interactions between genetic variations and MRI features and (2) select biomarkers associated with the disease. The authors also incorporated the linkage disequilibrium as a prior knowledge for the SNP data.

As also a radiogenomics approach, Zhou and colleagues [54] utilized both MRI and PET images as well as SNP to identify AD’s prodromal status—MCI—and to classify MCI subjects into 2 groups of either progressive MCI (pMCI) or stable MCI (sMCI); these groups are categorized based on who will develop AD and who will remain stable. For diagnostic predicting and for dealing with the heterogeneity and high-dimension, low-sample-size problem of the multiview data, the authors developed a deep multiview network that slowly fuses 3 different datasets into a common representation after a stage-wise training using the “maximum number of available samples”; specifically, the architecture makes use of a “three-stage deep feature learning and fusion framework.” In the first stage, latent representations of each view (MRI, PET, SNP) are learned separately, whereas in the second stage, the joint pairwise representations are learned by using the features in the first stage. In the third stage, diagnostic labels are learned by integrating all features from the second stage. The analysis is made on the AD neuroimaging initiative (ADNI) data set and achieves a favorable performance. The multiview deep learning approach taken therein can be regarded as a slow fusion architecture [70]. Another approach of discriminating MCI subgroups is presented by Young and colleagues [71], wherein the authors made use of Gaussian process as an MKL method to integrate volumetric MRI, fluorodeoxyglucose positron emission tomography (FDG-PET), CSF, and apolipoprotein E (APOE) genotype for a binary classification. This combination of neuroimaging, genomic, and metabolomic data delivered an accuracy of 74%, higher than any results achieved using a single modality. The combination of structural MRI, FDG-PET, and CSF are also used in an experimental study [21] in which 3 different multiview learning methods—namely CCA, MKL, and matrix factorization—are compared on the ADNI data.

For late-onset AD (LOAD), Mukherjee and colleagues [72] proposed a general multiview framework for feature learning of 27 previously known driver genes of LOAD, which may then be used to identify other potential driver genes. The authors also proposed a ranking method for these genes by aggregating the predictions associated with each feature set with genome-wide association study (GWAS) statistics. While claiming the framework's generality for any data modalities, the authors demonstrated the analysis via 3 modes of data: differentially expressed genes between AD cases and controls, global gene co-expression network features, and 42 tissue-specific co-expression modules. These transcriptomic data are collected from postmortem brain tissue across 3 different studies that are assumed to possess independently predictive information. To tackle sparsely labeled data (of only 27 known genes), they developed a multiview classification based on a co-training scheme. For feature learning, they indicated that topological features (e.g., node degree) are more predictive than are differential expression features; for ranking, they identified previously known and also potentially new LOAD driver genes that are significantly enriched for both SNPs and pathways associated with AD.

Another degenerative genetic disease is spinocerebellar ataxia (SCA), which is responsible for severe movement disorders. This complex disease, which possesses more than 40 genetically different types, must also be studied from an integrative approach that makes use of omics data, neuroimaging data, and clinical data, among others. Garali and colleagues [29] analyzed 4 subtypes of SCA, which are SCA1, SCA2, SCA3—the 3 most common subtypes—and SCA7, by employing component-based methods known as regularized generalized CCA (RGCCA) and sparse generalized CCA (SGCCA). These methods generalize CCA to analyze data sets structured in blocks, each of which represents a unique view of data. This kind of analysis aims to reveal information between and within blocks. Because SCA is characterized by the volume of a brain region called the pons, the authors performed RGCCA and SGCCA as block-based multimodal biomarkers approaches to discover the relationships between the pons volume, metabolomics, lipidomics, and metabolic imaging resulting from magnetic resonance spectroscopy.

Single-cell omics

Cellular populations are heterogeneous in nature. Although cells in a particular tissue are of the same type (e.g., neuron, muscle), they are nevertheless varied in terms of their states (e.g., mitotic, migratory) and behaviors according to the transcriptomic, proteomic, and other measurement levels in a spatiotemporal pattern. Research based on bulk sample of cells from a specific tissue undermines these variations across cells so that the emerged single-cell technologies flourish and thereby enable the exploration of cellular heterogeneity in complex diseases and stem cell differentiation. Single-cell multiomics provide diverse views for each individual cell (e.g., genomic, epigenomic, transcriptomic) that suggest how these different molecular levels interact to result in a phenotypic heterogeneity of cellular types, states, and fates (e.g., the effects of DNA methylation in the cell population on gene expression [73, 74, 75]). Integrating these multiomics remains a challenge, especially when sparseness and high dimensionality are the 2 pervasive characteristics of single-cell multiomics data [76, 77]. Sparseness is caused by dropout events wherein the gene expression is very high in some cells but very low or almost zero in other cells due to the stochastic nature of gene expression at the resolution of single-cell. The mixing of these false zeros with true zeros of nonexpressed genes makes the analysis difficult. Thus, to impute missing values from an omic, we need information from other omics. Also, the high dimensionality caused by the large number of genes in each cell would make any approach for discrimination between cells be very hard because, in this high-dimensional space, the distance between cells is indistinguishable.

Few imputation methods in single-cell analysis make use of multiview learning from multiomics data. Lin and colleagues [78] developed an ensemble regression imputation method that combines the self-imputation from a single omics (e.g., miRNA) as well as cross-imputation from other correlated omics (e.g., mRNA, DNA methylation). When comparing 5 other single-view imputation methods, the method presented therein was demonstrated to be advanced and efficient in terms of imputation accuracy and the recovery of mRNA-miRNA interactions. Multiomics factor analysis (MOFA) [50] is another multiomics integrative method that can efficiently identify outlier samples and accurately impute missing values. The method learns a set of hidden factors responsible for biological and technical variability from different omics and clearly identifies the consensus information shared across multiple omics as well as the diversity of specific information that belongs to individual omics. The inferred factors enable the identification of sample subgroups, data imputation, and sample outliers. When applied to a data set of chronic lymphocytic leukaemia on 200 patient samples, including somatic mutations, RNA expression, DNA methylation, and ex vivo drug responses, MOFA identified many sources of disease heterogeneity, such as immunoglobulin heavy-chain variable region status, trisomy of chromosome 12, and response to oxidative stress. When applied to single-cell multiomics data, MOFA identified coordinated transcriptional and epigenetic changes along cell differentiation. The ensemble method used by Lin and colleagues [78] can be regarded as an alignment-based method because it makes use of correlations between omics, whereas MOFA is a factorization-based method that attempts to reveal common latent factors.

The high dimensionality of single-cell multiomics requires any integration method to consider dimension reduction one of its goals. This requirement is naturally implemented in factorization-based methods because they attempt to reveal a common latent structure that often resides in a low-dimensional space. The desired result may also be accomplished if the embeddings f⁽ⁱ⁾ in alignment-based methods transform the original data to a linear or nonlinear manifold. Welch and colleagues [37] developed MATCHER to perform manifold alignment between transcriptomic and epigenomic levels from different cells. The method firstly uses a Gaussian process latent variable model to obtain pseudotime values for every cell by independently clustering them in every omics and secondly aligns the quantiles of the pseudotime distribution and those of a uniform distribution to make them directly comparable. As far as we understand, very few computational methods in bioinformatics—especially in multiomic integration—utilize the method of manifold alignment even though manifold structure is a suitable representation for gene regulatory networks because it preserves the locality of regulons [79]. Similar to MATCHER, ManiNetCluster [19] is another method for multiview learning that attempts to identify conserved or specific gene modules across species via manifold alignment. Although the data used in their studies are merely transcriptomic profiled from bulk samples, the method is general sufficient to apply to single-cell multiomics data to identify cell types, cell states, and even the functional linkage between various omics. It is worth noting that not all NMF methods used in single-cell multiomic integration are factorization-based approaches. Duren and colleagues [44] developed a method called coupleNMF to cluster cells using both gene expressions (scRNA-seq) and chromatin accessibility (scATAC-seq). This method does not recover a common dictionary matrix that captures the consensus across different views, as is the case in other multiview NMF methods; rather, it identifies the association between genes and regulatory elements and is thus a co-regularized method.

Plants

Multiomics and machine learning may be applied in plant science, especially to understand the mechanisms of photosynthesis and hydrogen metabolism that are valuable for biofuel research. Chlamydomonas reinhardtii is a microalgae often used as a premier reference organism to study biohydrogen production because of its high hydrogenase activity [80]. For example, a study applied both transcriptomic and proteomic levels to reveal a majority of the algal genomes being differentially expressed over the course of the light condition and the timing of specific genes being determined by their biological functions [81]. Another study implemented of genomics, transcriptomic, proteomics, and metabolomics to identify critical genes in hydrogen metabolism [80]. The combination of transcriptomic, proteomic, metabolite, and lipid profiling was also used to investigate the the regulation of photosynthetic process during nitrogen deprivation in C. reinhardtii [82]. However, these studies exclusively applied basic statistical techniques in their analysis and as such, lacked a systematic method for integrating and inferring from different types of omics.

As far as we understand, ManiNetCluster [19] is the only multiview learning method that has been used in plant science. In general settings, the method takes 2 different data sets as inputs, transforms them into a common latent subspace where they can be aligned with each other, then simultaneously clusters the aligned network for the discovery of conserved modules and functional linkages between 2 data types. In their study [19], a gene expression profile of C. reinhardtii between light and dark conditions was employed, and the 2 conditions were treated as 2 views of a multiview data set. ManiNetCluster was subsequently applied on these 2 inputs, which led to the discovery of conserved modules in which a group of genes retain their functions during both the daytime and nighttime. Some critical genes that serve as functional linkages to bridge and regulate daytime and nighttime functions were specifically identified.