The cPCA hyperparameter α compensates for bias toward high-variance dimensions in noisy, finitely-sampled datasets

To explain the need for the hyperparameter α in cPCA and how we avoid it in gcPCA, we will describe the objective function of each method and show how they perform in generated synthetic data. For illustration purposes, we generated synthetic data for two experimental conditions containing two-dimensional manifolds on a background of high-variance shared dimensions. The generated data consisted of condition A, with a manifold (additional variance) in the 71st and 72nd dimensions (ranked in order of descending variance), and condition B, with a manifold in the 81st and 82nd dimensions (Fig 1A). The manifold dimensions contained less total variance than most of the other dimensions in the dataset, but their variance is two-fold higher in one condition relative to the other (i.e., the 81st and 82nd dimensions have twice as much variance in condition B than condition A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Generalized contrastive PCA avoids cPCA’s need for the hyperparameter α in noisy, finitely-sampled data.

(A) We generated two conditions in noisy synthetic data that each contain low-variance manifolds that are not present in the other. These manifolds have lower overall variance than many other dimensions and are not trivially discoverable. (B) Eigenvalue spectra for each condition estimated from finite (dark line) or infinite (light line) data. Note the sampling error in the finite date case. (C) With infinite data, eigendecomposition of suffices to extract the correct answers (dimensions 71-72 and 81-82, light lines). However, with finite data, these peaks are smaller than the sampling error in high-variance dimensions, creating a bias toward high-variance dimensions being selected. D cPCA uses the hyperparameter α to adjust how much influence has on . As α increases, the bias toward high-variance dimensions decreases until it becomes negative with α > 1, eventually exposing the differences in lower-variance dimensions. Importantly, there is no way to know which value of α yields the correct solution. (E) Using gcPCA, the dimensions most changed in each condition are identified correctly, even with finitely-sampled data. (F) cPCA with the optimal choice of α does not extract the correct dimensions in B. (G) gcPCA identifies the enriched dimensions in each condition and correctly returns the low-variance manifolds. Because gcPCA is symmetric, it extracts the correct dimensions in both A and B.

https://doi.org/10.1371/journal.pcbi.1012747.g001

A key property of real-world biological datasets is that they are noisy and finitely sampled. We aimed to model the finite data regime by comparing samples (finite data) to samples, which approximates infinite data. To inspect the effects of finite sampling on estimated variance, we projected the "finite" and "infinite" data onto the ground truth dimensions and calculated the variance explained by each (see methods). Our results (Fig 1B) reveal that finite sampling yields noisy estimates of the true variance, with greater noise in high-variance dimensions.

To understand the practical consequences of this, it is helpful to start by reviewing traditional PCA. With PCA, the principal components of a data matrix D, of size n × p (samples  ×  features), are the dimensions explaining the most variance. These can be identified by estimating the covariance matrix , then solving the following quadratic optimization problem in equation 1:(1)

This problem can be solved by eigendecomposition of C, yielding the matrix of eigenvectors X known as principal components (PCs).

To extend this to two datasets, a logical strategy involves formulating an objective function to describe the difference in variances between the two conditions, enabling us to extract dimensions that show the greatest increase in variance in A relative to B. We now have two covariance matrices, and , and the contrastive PCs (cPCs) are the vectors that maximize the objective function 2:(2)

Analogous to traditional PCA, this problem can be solved by eigendecomposition of . This yields cPCs that account for more variance in either condition A or B, corresponding to the eigenvectors with the largest or smallest eigenvalues, respectively. With infinite data, the precise estimates of the eigenspectrum correctly find the dimensions enriched in conditions A and B (Fig 1C, light line), but with finite data, it fails to do so because the sampling error in higher-variance dimensions is larger than the true signal in the lower-variance dimensions (Fig 1C, dark line). In other words, it has a systematic bias toward high-variance dimensions. To compensate for this effect, cPCA [14] introduces the hyperparameter α, changing the following objective function to 3:(3)

As discussed in [14], α represents a trade-off determining the extent to which influences the identification of enriched vectors in . In our synthetic data, we can visually appreciate the effect of different values of α in the resulting cPCA objective function value (Fig 1D). In effect, α tunes the amount of bias toward high-variance dimensions in the cPCA calculation, with α < 1 biasing toward high-variance dimensions and α > 1 biasing against them. If the correct value of α is selected, the sampling error in the high-variance dimensions of A and B will cancel out, yielding the correct answer. In this example, α = 2 yields the correct solution that dimensions 71-72 are enriched in A (Fig 1F), but other values of α yield equally plausible, but incorrect, solutions. Importantly, we can only determine which solution is correct because we knew the answer in advance, which is not typically the case for experimental data. Further, negative values in cPCA are generally interpreted as dimensions enriched in condition B [14,15], but our simulation shows that values of α larger than 1 bias the highest-variance dimensions to be negative (Fig 1D). This creates the illusion that these dimensions are enriched in condition B, even though the correct answer is that only dimensions 81-82 are enriched in B. Similar to the situation of finding dimensions enriched in A, the results depend on the choice of α, with no way to determine which solution is correct (Fig 1F). The range of α’s yielding the correct solution can also be incredibly narrow, as in S2 Fig, where α = 2.6 yields the correct solution, but 2.2 or 3.0 do not.

gcPCA avoids hyperparameters by including a normalization factor to penalize high-variance dimensions

Our goal for gcPCA was to eliminate the need for hyperparameters and provide unique, correct solutions. To mitigate the bias toward high-variance dimensions, we therefore introduced a normalization factor to penalize high-variance dimensions. gcPCA can use several different normalization factors depending on the task at hand (Table 1); here we will use the total variance in both conditions, which can be calculated by summing the covariance matrices (). The objective function then becomes (eq. 4):(4)

This problem can be solved by eigendecomposition of , where M is the matrix square root of the denominator (see Methods). The resulting generalized contrastive PCs (gcPCs) maximize relative, rather than absolute, changes in variance between conditions A and B. Because the penalty and the sampling error both scale with the variance, this effectively handles the bias toward high-variance dimensions and successfully extracts the ground truth dimensions in our synthetic data, even with finite sampling (Fig 1E, 1G), and even when the range of acceptable α is narrow (S2 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. gcPCA variants in the gcPCA toolbox.

https://doi.org/10.1371/journal.pone.0313772.t001

This creates two minor complications to be aware of: first, unlike PCs or cPCs, gcPCs are not orthogonal in the original feature space. Instead, they are orthogonal in the normalized feature space in which high-variance dimensions have already been penalized by multiplication with . For the purpose of contrasting A and B, this is usually the best approach. However, if orthogonality in the original feature space is important for a particular application, we have implemented versions of gcPCA with an orthogonality constraint (S3A-S3D Fig, see Methods). Second, the normalization factor can create numerical instability if the data contain dimensions with zero or near-zero variance in the denominator, in effect leading to division by zero. This occurs if there are fewer samples than features or if the features are strongly correlated. Our implementation of gcPCA solves this problem by detecting and excluding zero-variance dimensions before performing the calculation (see Methods).

The open-source gcPCA toolbox contains multiple gcPCA variants enabling optimal handling of diverse use cases

We have developed an open-source gcPCA toolbox with implementations in Python and MATLAB of several different variants of gcPCA. This toolbox is freely available at: https://github.com/SjulsonLab/generalized_contrastive_PCA. Here we will present the different variants of gcPCA and their use cases.

Unsupervised extraction of facial expression features with gcPCA

To illustrate the utility of gcPCA with real-world datasets, we first use the Chicago Face Dataset [17], which contains faces with different facial expressions. Here we used happy and angry expressions as condition A and neutral faces as condition B (Fig 2A). This dataset is useful for two reasons: 1) the categorical separation of facial expressions allows an easy evaluation of the contrastive methods, and 2) the dimensions can be visually inspected for features that are being discovered by the method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. gcPCA correctly extracts contrastive facial features in the Chicago Face Dataset.

(A) For this test, contrastive methods were applied to a set of happy and angry face images (Condition A) versus neutral face images from the same subjects (Condition B). Facial expression changes along the happy-angry axis were therefore the low-dimensional pattern that is enriched in condition A. (B) The first α identified by cPCA’s algorithm, α = 0.2, yields loadings similar to the first PCA dimensions, i.e. eigenfaces. (C) Projecting the faces onto the cPCs reveals clusters unrelated to the happy and angry facial expressions in condition A, indicating an incorrect solution. (D) cPCA with α = 1.7 correctly reveals features associated with the expected facial expressions in condition A, such as furrowed eyebrows and the region around the mouth and nose. (E) Projecting the faces onto these cPCs reveals the separation of happy and angry faces along the first cPC. Importantly, it was only possible to determine this answer was correct because we knew the labels in advance. (F) Dimensions identified by gcPCA correctly reflect features related to the facial expressions in condition A. (G) Projecting the faces onto the first two gcPCs also reveals the separation of happy and angry faces along the first gcPC.

https://doi.org/10.1371/journal.pcbi.1012747.g002

We first applied cPCA to the two conditions and used its automatic α selection algorithm to pick two different α values. The automatic α selection algorithm developed by [14] finds representative α’s that yield different cPC embeddings so that the investigator can choose the appropriate α value. The algorithm is explained fully in the original paper (see supplementary methods - algorithm 2 [14]). Briefly, it calculates cPCA for an array of different α’s (default: 40 different α values ranging from 0.01 to 1000 spaced on a log scale), defining a subspace with the top k cPCs (default: 2 dimensions), and calculating an affinity matrix between subspaces of different α’s. This affinity matrix is then clustered to find p clusters, and the medoid of each cluster is a candidate α.

The first value returned is α = 0 . 2, and we can see that the first two cPC loadings resemble "eigenfaces" [18], the largest principal components of facial images (Fig 2B). This suggests that this α is too small, leading cPCA to extract the highest-variance components in condition A. Looking at the individual faces projected on these cPCs, two clusters can be identified that separate cleanly along cPC1 (Fig 2C). However, this solution is incorrect because these clusters do not reflect the two facial expressions comprising condition A. Instead, they represent skin color, which should account for equal variance in both datasets because images of the same subjects are present in both conditions. For the second α value returned (α = 1 . 7), the cPC loadings exhibit features specific to condition A (Fig 2D), and data projected onto these cPCs recovers the different expressions (Fig 2E). It is important to note that if we did not have the class labels a priori, we could easily believe α = 0 . 2 was the correct answer because it produces better clustering than for α = 1 . 7 (Fig 2C, 2E). Using gcPCA v3 (asymmetric, non-orthogonal) also reveals features specific to condition A (Fig 2F), and the two expressions in the dataset can be distinguished by their projection onto (Fig 2G), without the requirement of fine-tuning a hyperparameter. gcPCA accomplishes this by extracting axes of increased variance in unsupervised fashion (S1C Fig), not by clustering or classifying the faces in a supervised manner as with LDA (S1B Fig).

Applying gcPCA to neurophysiological recordings reveals hippocampal replay without a priori knowledge of replay content

A key application for gcPCA is neuronal recordings, which frequently contain activity from hundreds of individual neurons [19,20]. A well-studied neurophysiological phenomenon is hippocampal replay, in which hippocampal neurons encoding spatial trajectories traversed during a behavioral task "replay" the same activity patterns in post-task periods [21]. Note that spatial location is a continuous variable, so we are not expecting gcPCA or cPCA to find dimensions that cluster the activity into different groups, as with the facial expression data. Instead, we are hoping to see replay of the firing patterns that encode the linear track the animal recently explored [21,22]. We analyzed a previously published dataset recorded from hippocampal CA1 [23] where rats learn the location of an aversive air puff on a linear track. The air puff is only delivered when the rat is running in one direction, called the danger run, and the other direction is the safe run (Fig 3A). In this dataset, [23] used traditional methods to demonstrate that hippocampal neurons exhibit reactivation of task-related activity patterns in post-task activity. These methods use activity patterns during task performance to define templates, then test whether activity patterns matching those templates are more common in post-task activity than pre-task activity [21, 22, 24].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. gcPCA v4 correctly identifies hippocampal replay in neurophysiological data without prior knowledge of replay content.

(A) In [23], rats were trained to traverse a linear track where one direction has an aversive air puff and the other does not. Rats learned the location of the air puff and which direction was dangerous or safe. (B) Girardeau et al. (2017) recorded hippocampal neurons in pre- and post-task periods and used activity recorded during the task as a template to determine that that activity was "replayed" during post-task. We reanalyzed their published data using post-task activity as condition A and pre-task activity as condition B to identify the dimensions most enriched in post-task activity without taking task-related activity into account. (C) To visualize task-related activity, we first performed PCA on task data, then projected activity during runs onto the first two PCs. This revealed mostly separate representations of the start/puff/stop locations, connected by bundles of trajectories. (D) We next applied gcPCA to post- and pre-task data, then projected activity during runs onto the first two gcPCs, which are enriched in post-task. This readily identifies a structure similar to that seen by the task PCA (panel C), suggesting that gcPCA is extracting replay during the post-task period. (E) PCA on the pre-task (left) or post-task (right) activity yields representations that are similar to each other but not activity during the task. This suggests that PCA is extracting shared components rather than replay. (F) cPCA requires hyperparameter searches yielding multiple solutions, with no indication of which is correct. The automatic α selection algorithm from cPCA returns various α values depending on the number of cPCs requested (parameter k). Left Two Columns Representative α values returned with k=48 cPCs. cPC_1-2 are the dimensions most enriched in post-task. No discernible spatial structure is identified, indicating that replay was not detected. Right Two Columns With k=2, the α values returned are of different magnitudes, and one of them (α = 1.43, green boxes) reveals spatial structure related to the task, suggestive of hippocampal replay.

https://doi.org/10.1371/journal.pcbi.1012747.g003

We tested whether gcPCA could extract hippocampal replay in template-free fashion by contrasting post-task activity (condition A) with pre-task activity (condition B) (Fig 3B). We then compared the results of gcPCA to a template derived from activity during the task. To make this template, we performed traditional PCA on neuronal activity during running on the track. In these plots, the representations of the start, puff, and stop locations occupy distinct regions of PCA space, and the activity on the track forms bundles of trajectories connecting these points (Fig 3C). Since the PCA used data from all trials, danger and safe runs both exhibit this structure in PCA space. Strikingly, projecting the same runs onto the top gcPCs yielded qualitatively similar results (Fig 3D), suggesting that gcPCA (v4, see Methods) is extracting replay of task-related activity. Using PCA on either the pre- or post-task activity failed to extract this structure (Fig 3E), indicating it can only be found by contrasting POST with PRE. gcPCA was thus able to extract signatures of replay without prior knowledge of the task-related activity, which may prove useful in many analogous situations in which the experimenter does not have prior knowledge of the pattern they are searching for. Orthogonal and sparse gcPCA also yielded similar results (S3 Fig).

Using cPCA, it was not straightforward to detect replay. When we requested the cPCA algorithm evaluate all the dimensions (k = 48), the α values identified by the automatic selection did not reveal any obvious task-related spatial structure (Fig 3F – left, α values 0.4 and 942.7). When we requested a smaller set of dimensions (k = 2), the automatic α selection returned several different values, with one of them (α = 1.4) revealing task-related structure (Fig 3F - right). This reveals that although the only explicit hyperparameter for cPCA is α, the automatic α selection algorithm depends on k, the number of components requested, thus constituting a second de facto hyperparameter.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Sparse gcPCA v4 reveals possible disease heterogeneity in type II diabetes.

To test sparse gcPCA, we compared it to traditional sparse PCA in the analysis of published scRNA-seq data from isolated pancreatic cells from type II diabetes (T2D) patients (red) and healthy controls (gray) [25]. (A) We first performed sparse PCA separately on the T2D and Control cells. In the T2D PCA space (left), T2D beta cells from the same subjects (indicated by color) tended to cluster together. This structure was largely retained in the Control PCA space (right), suggesting that sparse PCA extracts sources of variance that are common to both groups. (B) This same pattern was observed in Control cells, which also clustered together by subject identity in both PCA spaces. (C) Linear Discriminant Analysis (LDA) was unable to distinguish T2D cells from Control cells, indicating that the average gene expression profile does not differ between groups. (D) PCA loadings from Control or T2D cells were strongly correlated (r = 0.41), suggesting that sparse PCA on either group alone extracts sources of variability that are shared in both groups. (E) The top 20 genes identified by sparse PCA on T2D cells contain several known diabetes-related genes (bold). (F) We next performed sparse gcPCA 4 contrasting the T2D and Control cells. In the sparse space (overrepresented in T2D cells, left), T2D cells from different subjects separated into distinct subgroups, possibly revealing disease heterogeneity. This structure was not observed in the sparse space (overrepresented in Control cells, right), indicating that gcPCA is extracting sources of variability that differ between groups. (G) Control cells in the sparse space (right) did not exhibit the heterogeneity found in T2D cells, and clustering by subject identity was not preserved in the sparse space (left). (H) The correlation between loadings for the top and bottom gcPCs was weak (r = 0.10), indicating that gcPCA is extracting sources of variance that differ between the T2D and Control cells. (I) Loadings for the top T2D PCs and gcPCs were less correlated (r = 0.10) than the T2D PC and Control PC loadings (r = 0.41, panel D), suggesting that PCA is extracting shared sources of variance, but gcPCA is extracting different sources. (J) The top 20 T2D genes extracted by gcPCA contained several genes previously implicated in T2D (bold), including TMEM176A/B, that were previously demonstrated as functionally important for T2D by Martínez-López et al. (2023).

https://doi.org/10.1371/journal.pcbi.1012747.g004

Applying sparse gcPCA to scRNA-seq data prioritizes disease genes and reveals disease heterogeneity in type II diabetes

Another key application of gcPCA is high-dimensional omics datasets, which often compare two conditions (e.g. disease vs. healthy) to identify a list of features (e.g. genes) that differ between conditions. Sparse gcPCA is well-suited for this task because the sparsification narrows down the list of genes to the most important contributors. We therefore compared the performance of sparse gcPCA v4 to traditional sparse PCA [3,16] in published single-cell RNA sequencing data from pancreatic beta cells biopsied from patients with type II diabetes (T2D) or healthy controls [25]. We first performed sparse PCA separately on each condition and plotted the cells from both conditions in both PCA spaces (Fig 4A-4B). Cells from different subjects tended to aggregate together, and this pattern was largely preserved in either PCA space. This is because T2D and control cells exhibited similar covariance structure, with the loadings on the top PCs highly correlated (Fig 4D). In other words, sparse PCA extracts patterns that are shared between the conditions, finding little difference between them. Notably, LDA also failed to detect any differences between T2D and control conditions (Fig 4C), indicating that the average cellular transcriptional profile is the same in both conditions. Despite the fact that the sparse PCs from each condition are correlated, sparse PCA of the T2D condition alone still extracts several known T2D-linked genes (Fig 4E, bold): MPV17 [26], SMCO4 [27], DDIT3 [28,29], NFE2L2 [30], G6PC2##1 [31], and SPPL2A [32].

We next used sparse gcPCA v4 on the same datasets with T2D beta cells as condition A and control beta cells as condition B. and therefore represented axes along which T2D cells vary more, and and were axes along which control cells vary more. We found that T2D cells from different subjects formed two distinct clusters in the subspace, possibly reflecting disease heterogeneity [33] (Fig 4F, left), but no clustering was evident for T2D cells in the subspace (Fig 4F, right). Control cells did not exhibit clustering in either the subspace (Fig 4G, left) or the subspace (Fig 4G, right). This is because the gcPCs overrepresented in the T2D and control cells are largely uncorrelated (Fig 4H), indicating that gcPCA is successfully extracting patterns that differ between conditions. As expected, the top sparse gcPCs for T2D cells were weakly correlated with the top sparse PCs for T2D cells (Fig 4I), with only one of the top 20 T2D genes found with sparse gcPCA also found with sparse PCA (Fig 4J, DDIT3). A greater proportion of the genes in the sparse gene list have been previously implicated in T2D (Fig 4J), including TNFRSF12A [34], TFF3 [35], IMMP2L [36], DDIT3 [28,29], PPP1R1A [37]), and TMEM163 [38]. Notably, the TMEM176A/B genes were only recently discovered to be involved in T2D by [25], who demonstrated experimentally that these genes are functionally important for T2D-related beta-cell function.

Generalized contrastive PCA

Our motivation for the following method stems from eliminating the necessity of the free parameter α in the contrastive PCA method. To accomplish this, we introduce a normalization factor to mitigate the bias toward high-variance dimensions. We will summarize the process of calculating the gcPCs using gcPCA v4 as an example, but v2 and v3 are analogous. gcPCA v4 has the following objective function, as shown in equation (eq. 8):(8)

A potential problem of this objective function is the denominator creating numerical instability if the denominator covariance matrix has eigenvectors with eigenvalues equal to (or sufficiently close to) zero. This can occur either if the features are strongly correlated or if the data is rank-deficient, e.g. scRNA-seq data containing fewer cells than genes or two-photon imaging data containing fewer frames than pixels. To address this, we consider only the principal subspace of the covariances matrices, i.e. the subspace spanned by eigenvectors with non-zero eigenvalues. To project the matrices into this subspace, we first make a projection matrix J composed of the first k principal components of the row-wise concatenated datasets A and B, where k is chosen so that the principal components that are effectively zero due to limited numerical precision are excluded from the analysis. We then substitute x using x = Jγ, yielding:(9)

J guarantees that in the denominator is positive definite, allowing us to find a symmetric matrix M that is its square root, yielding equation (eq. 10):(10)

Let y = Mγ, yielding:(11)

This optimization problem can be solved with the eigendecomposition of the numerator matrix to find Y. The vectors in X are then calculated using equation (eq. 12):(12)

The column vectors in X are referred to as gcPCs, following the term cPCs used in [14]. The other versions of gcPCA can be solved in the exact same way, by replacing with (v2) or replacing with (v2 and v3).

Orthogonality constraint

The core idea of gcPCA, as described above, is to use to map the numerator from the original feature space into a weighted inner product space that is weighted by the inverse of the denominator. This warps the space so that the variance in is equal in all directions. This is directly analogous to the whitening transformation used to calculate Mahalanobis distances, but instead of calculating distances, we diagonalize to find the gcPCs. Consequently, the gcPCs are orthogonal in this normalized feature space, not the original feature space. To calculate gcPCs that are orthogonal in the original feature space, we first take the largest eigenvector of X as the first gcPC. We then apply an orthogonality constraint by iteratively deflating matrix J to remove the subspace spanned by the gcPCs (i.e. columns of X) that have already been found. At each step, we compute the eigenvector x of equation 11 with either the smallest or largest eigenvalue (alternating between them), then project it into the feature space with equation 12 and concatenate it column-wise into the growing matrix X. To deflate J, we first regress out the x from J, as shown in equation 13:(13)

We then use SVD to get the left singular vectors of , and we define as the first n–i of these (on the i-th iteration). serves as an orthonormal basis for the subspace of J that is orthogonal to the columns of X that have already been calculated, and we use as the new J in eq. 9 for the next iteration. This process continues until k gcPCs are found, which can be the number of features in the dataset, the minimum rank of the conditions, or a number specified by the user.

Sparse gcPCA

We developed an extension for sparse gcPCA by using a similar approach to sparse PCA [3] and sparse cPCA [15]. We will first review the sparse PCA framework, then explain how we adapt it for gcPCA.

Synthetic data generation

In the synthetic data, we generated two conditions with samples and 100 dimensions. The dimensions were sampled from a Gaussian distribution (mu = 0 and sigma = 1) and then orthogonalized using singular value decomposition and picking the left singular vectors. In each condition, we created a pattern in the samples that was to be discovered. In condition A, we took dimensions 71 and 72 and drew the samples from a uniform distribution ( [ 0 , 1 ] ). In dimension 71 we replaced any value from 0.3 and 0.7 with a different uniform distribution ( [ 0 , 0 . 4 ] ). In dimension 72 all the values between 0.4 and 0.6 were replaced with another uniform distribution ( [ 0 , 0 . 4 ] ). This created the square with a square hole in the middle in Fig 1A. The values were then offset by 0.5, the samples were sorted by the angle they formed in each dimension, calculated by the inverse tangent (). The samples in each dimension were normalized by their -norm. In condition B, we generated the samples of dimensions 81 and 82 from a uniform distribution ( [ 0 , 1 ] ), sorted the sample values based on dimension 81, and rotated both dimensions by 45 degrees. This created the diamond shape seen in Fig 1A. The sample values were later normalized by their -norm. The samples for all the other dimensions were drawn from a Gaussian distribution (μ = 0 and σ = 1), and then normalized by their -norm. The magnitude of each dimension was established as a line with a negative slope, starting at value 10 in the 1st dimension and ending at 0.001 in the 100th dimension. For condition A, we doubled the magnitude in dimensions 71 and 72, while in condition B we doubled the magnitude in dimensions 81 and 82. Identifying these changes in magnitude is the goal of contrastive methods. Because the samples were drawn from a normal distribution, the dimensions will display correlations among them. To estimate the total variance explained by each dimension, we use a QR decomposition approach described in [3]. In brief, let Z be a matrix containing scores of each dimension generated, the variance is usually calculated through , where tr is the trace of the matrix. However, in correlated scores, this estimate is too optimistic. Using regression projection, it is possible to find the linear relationships of the dimensions and correct to find the adjusted total variance. Zou et al. (2006) [3] shows that this is equivalent to using QR decomposition in Z, such that Z = QR where Q is orthonormal and R is upper triangular, and calculating the adjusted variance as follows:(23)

Face dataset

For the facial expression analysis, we used the Chicago Face Database [17]. This database consists of neutral and emotional expression faces, and for the analysis we used a subset of samples that had happy and angry faces alongside neutral ones. We used only male faces to reduce variability in feature positioning. The images used were cropped by an ellipse (length of 75 pixels and width of 45 pixels) centered in the face to focus on the facial expression rather than other features such as hairstyle or shoulders. Each sample image was flattened from a two-dimensional matrix to a vector, and all the flattened samples were then concatenated, resulting in a matrix of samples x features. Each feature was z-scored and normalized by its -norm. Condition A consisted of all the samples of happy and angry facial expressions, while condition B samples were neutral expressions.

Hippocampal electrophysiology data

We used a previously published hippocampal electrophysiology dataset, with the experimental details listed in the original publication [23,39]. In brief, Long-Evans rats were implanted with silicon probes in the dorsal hippocampus CA1 region (either left or right hemisphere), and neuronal activity was isolated through automatic spike sorting and manually curated. Animals were trained to collect water rewards at the end of a linear track, and an air puff was introduced at a fixed location for every lap, in only one of the directions. Recordings consisted of task, where the animal learned the air puff location, and periods of pre- and post-task activity. For testing the contrastive methods, we used pre-task recordings as condition B and post-task recordings as condition A. For our analysis, we only used neurons that had a minimum firing rate of 0.01 spikes/s during the task. We binned the neural data using a bin size of 10 ms and smoothed using a rolling average with a Gaussian window of size 5 bins. The data was then z-scored and normalized by the norm before applying the contrastive methods. The task data was then projected on the contrastive dimensions for evaluation. The same normalized data was used for the PCA analysis.

Pancreatic single-cell RNA sequencing

For the single-cell RNA sequencing data analysis, we used a previously published dataset [25], available at GEO accession GSE153855, consisting of scRNA-seq data from human pancreatic islet cells from patients with type II diabetes and healthy controls. We used the annotated dataset to identify the beta cells, which were identified previously by the authors [25]. For condition A we used the beta cells from subjects that had type II diabetes, and for condition B we used the beta cells from healthy patients. We used the expression values in reads per kilobase of the gene model and million mappable reads (RPKMs). The values were log-transformed, and all the features were centered before the analysis. We used the same set of genes used by Martínez-López et al. (2023) [25]. To determine the top genes found by each method (sparse PCA or sparse gcPCA), we multiply the loadings on each dimension by their eigenvalue and calculate the -norm, returning a loading magnitude for each gene.