Estimation accuracy of sample-level cell frequencies

CIBERSORTx comes equipped with an inbuilt leukocyte gene signature matrix LM22, but also allows the creation of custom gene signatures. We created a custom signature using our sorted cell expression in the training set (Fig 2B). We deconvoluted CD4, CD8, CD14, and CD19 cell fractions from PBMC mixed cells based on different scenarios: CIBERSORTx with inbuilt (CIBX-inbuilt) and custom (CIBX-custom) matrices, bMIND using the custom matrix (bMIND-custom), debCAM using cell-type specific markers derived from expression in our sorted cell populations (debCAM-custom) and compared estimates of cell fractions to measures of ground truth derived from flow cytometry (Fig 2B).

Correlations were highest in CD14, followed by CD19, CD8, and CD4 regardless of methods and signature matrices (Fig 3). Generally speaking, CIBX-custom performed less well across all four cell types than the other three approaches, while CIBX-inbuilt and debCAM performed the best, although the exact ordering did vary between cell types. This difference between CIBX-custom and CIBX-inbuilt emphasises the importance of a well trained gene expression signature matrix. We note that CD14 predicted fractions were overestimated regardless of approach and CD4 generally under-estimated (Fig 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Prediction accuracy of cell fractions by cell type (column) and approaches (row).

Pearson correlation (R) and root mean square errors (RMSE) were calculated between estimated fractions (y-axis) and flow cytometry measures (x-axis). Each point is a testing sample and dashed blue lines indicate y = x. CIBX-inbuilt: CIBERSORTx fraction deconvolution using the inbuilt signature matrix; CIBX-custom: CIBERSORTx fraction deconvolution using the custom signature matrix; bMIND-custom: bMIND fraction estimation using the custom signature matrix; debCAM-custom: debCAM fraction estimation using cell-type specific genes.

https://doi.org/10.1371/journal.pcbi.1012859.g003

Estimation accuracy of sample-level cell type gene expression

Our main goal was to compare accuracy in imputing sample-level cell-type gene expression from PBMCs for the four sorted cell populations: CD4 T cells, CD8 T cells, CD14 monocytes and CD19 B cells. In addition to predicted expression by CIBERSORTx using inbuilt and custom signature matrices, we derived cell-type expression profiles using true cell fractions (estimated by flow cytometry) with bMIND and swCAM algorithms (Fig 2B). Finally, we trained regularised multivariate LASSO and ridge models to predict cell-type expression using all genes. While CIBERSORTx predicts only a subset of the most confident genes (5779-11794 depending on cell type), all other methods were able to predict expression for all or almost all 18871 genes across cell types (S1 Fig).

Despite the differing number of imputed genes among methods (S1 Fig), initial analysis comparing observed and predicted expression of genes in the same subjects suggested that all methods could predict cell-specific expression well, as judged by high correlation between observed and imputed expression (median r > 0.85) in the test data, although correlations were generally higher and root mean square error (RMSE) lower in LASSO and ridge than the other approaches (Fig 4A and 4B). To better interpret the correlations, we calculated “baseline” correlations. These were estimated either between observed expression in one individual and estimated expression in the same cell type from a different individual, or between the observed expression across different subjects within the matched cell types. These were also high, and only marginally different from baseline correlations, limiting the utility of this measure to discriminate among methods (S2 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Prediction accuracy of sample-level cell-type expression by approach.

(A) Pearson correlation and (B) log root mean square error (RMSE) comparing observed to predicted cell-type expression of genes from the same subjects, one estimate per subject. (C) Pearson correlation and (D) log RMSE between observed and predicted cell-type expression across testing samples for each gene, one estimate per gene. RMSE was standardised by the average observed expression per gene. CIBX-inbuilt: CIBERSORTx expression deconvolution with the inbuilt signature matrix; CIBX-custom: CIBERSORTx expression deconvolution with a custom signature matrix derived from sorted cell-type expression in training samples; bMIND: bMIND expression deconvolution with flow fractions; swCAM: swCAM deconvolution with flow fractions; LASSO/ridge: expression predicted from regularised multi-response Gaussian models.

https://doi.org/10.1371/journal.pcbi.1012859.g004

We therefore complemented this with a comparison of the observed and predicted expression across subjects for each gene. Correlation varied considerably between genes, irrespective of approaches (Fig 4C). All methods had comparable correlation per gene for each cell type, except for swCAM, which exhibited suboptimal performance for CD8, CD14, and CD19 (Fig 4C). Similar RMSE per gene was seen for each cell type across methods, although LASSO and ridge had slightly lower median values than other approaches (Fig 4D). Because different methods imputed values for different genes, we also compared methods restricting to the common gene sets predicted by all methods and found broadly the same ordering of methods by performance (S3 Fig).

Recovery of significant genes in association testing

Despite correlation and RMSE being commonly used to assess predictive accuracy, they do not necessarily capture performance of predictions in intended downstream analyses. We therefore defined a new measure of performance, differential gene expression (DGE) recovery, which used simulated phenotypes that deliberately correlated with observed expression across a subset of genes and conducted DGE analysis in parallel using predicted and observed cell-type data. DGE recovery measured the degree to which significant and non-significant signals in the observed data could be correctly identified in the imputed data (S4 and S5 Figs). According to this measure, LASSO and ridge exhibited higher median values of the area under the receiver operating characteristic curve (AUCs) across cell types than the AUCs achieved by CIBERSORTx with inbuilt and custom, bMIND and swCAM. This pattern was consistent across four simulated scenarios: either dichotomous or continuous phenotypes with or without covariate adjustment (Fig 5). Different methods can predict expression for different numbers of genes (S1 Fig), and we confirmed the broad pattern was also consistent when restricting to the common genes predicted by all approaches (S6 Fig). More detailed examination in the dichotomous phenotype setting showed that, generally, LASSO and ridge had higher sensitivity than CIBERSORTx, bMIND and swCAM, but also lower specificity (S7A Fig). In all cases, imputed estimates of log₂ fold changes were attenuated in the imputed data, with average slopes of 0.60-0.70 in CIBERSORTx inbuilt, 0.64-0.76 in CIBERSORTx custom, 0.66-0.85 in bMIND, 0.55-0.90 in swCAM, 0.69-0.76 in LASSO and 0.68-0.76 in ridge (S7C Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Differential gene expression (DGE) recovery based on CLUSTER data.

Area under curve (AUC) distributions estimated in held out test data by approach and cell type (columns) for each scenario (rows). Scenarios differed in simulated dichotomous (dichPheno)/ continuous (contPheno) phenotypes, with/without sex as a covariate in the DGE models. dichPheno: dichotomous phenotype; dichPheno+sex: simulated dichotomous phenotype and sex; contPheno: continuous phenotype; contPheno+sex: continuous phenotype and sex. Each point is a simulated phenotype, and there are ten simulated phenotypes. For each simulated phenotype, the receiver operating characteristic curve and AUC were estimated by FDR fixed at 0.05 in the observed data and varied FDRs from 0 to 1 by 0.05 in the imputed data. Box plots showed the AUC distributions, with horizontal lines from the bottom to the top for 25%, 50% and 75% quantiles, respectively. CIBX-inbuilt: CIBERSORTx with the inbuilt signature matrix; CIBX-custom: CIBERSORTx with a custom signature matrix; bMIND: bMIND with flow fractions; swCAM: swCAM with flow fractions; LASSO/ridge: regularised multi-response Gaussian models.

https://doi.org/10.1371/journal.pcbi.1012859.g005

To provide a broader range of scenarios to compare these methods, we constructed cell-specific bulk RNA-seq from single cell data from eQTLgen [27] and performed a similar analysis. We compared expression between cells stimulated with C. albincas for 3 hours to untreated cells, in three settings: either unadjusted for batch, adjusted using batch as a covariate, or adjusted using Combat-seq This allowed us to consider different models including covariates or different ways to account for batch effects. We varied training sample size from 20 (25%) to 80 (100%) samples. As the training sample size increased, AUCs generally increased for all approaches. LASSO and ridge regression consistently outperformed other methods across all cell types, regardless of whether raw or batch-corrected DGE results, in which batch effects were accounted for as a covariate or batch-corrected read counts by Combat-seq were used (Fig 6). This dominance persisted in further analysis of common gene sets (S8 Fig). All our results, taken together, suggest that regularised multivariate models performed better than the other three deconvolution-based methods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Differential gene expression (DGE) recovery based on pseudobulk data.

We varied training sample size from 20 (25%) to 80 (100%) (x-axis) and quantified area under curve (AUC) in held out test data for DGE recovery (y-axis) by cell type (columns) for each scenario (rows). cond: raw aggregated read counts were used for DGE analysis of in vitro stimulation with C. albicans after 3 hours (3hCA) vs untreated (UT). batch+cond: same as cond, and with batch (V2 & V3 chemistry) as a covariate in the 3hCA vs UT DGE model. Combat-seq cond: Combat-seq batch-corrected read counts were used in DGE analysis of 3hCA vs UT. Each point is the result of one analysis, and three replicates were conducted for each training sample size. For each pseudobulk data, the receiver operating characteristic curve and AUC were estimated by FDR fixed at 0.05 in the observed DGE results and varied FDRs from 0 to 1 by 0.05 in the DGE results using imputed expression. Local polynomial regression fitting (loess) were plotted for each approach. Note that CIBX-inbuilt is not shown for CD19/Combat-Seq cond. This is because it was able to impute < 60 genes, compared to ∼ 1,000 for CIBX-custom and > 11,000 for other methods, so estimates of AUC are very noisy. CIBX-inbuilt: CIBERSORTx with the inbuilt signature matrix; CIBX-custom: CIBERSORTx with a custom signature matrix based on pure cell expression in the training samples; bMIND: bMIND with true fractions; swCAM: swCAM with true fractions; LASSO/ridge: regularised multi-response Gaussian models.

https://doi.org/10.1371/journal.pcbi.1012859.g006

Computational performance

We compared CPU time and memory usage across methods in both datasets (S2 and S3 Tables). We found that typically CIBX was fastest, and bMIND required the least memory. swCAM and machine learning methods took longer and machine learning methods used more memory. For all methods, CPU time increased approximately exponentially with respect to training sample size but memory usage remained the same (S9 Fig).

Ethics statement

Data utilised in the study came from 158 subjects recruited in the CLUSTER consortium. Around 80% of subjects (N=126) have juvenile idiopathic arthritis (JIA); the rest are healthy controls from adults and children, and about 58% are female (N=91). Peripheral blood samples were obtained in accordance with the ethics approved by the London-Bloomsbury Research Ethics Committee (REC 05/Q0508/95, 95RU04, and 11/LO/0330). Formal written consent was obtained from the parent/guardian and the adult participants. The diagnosis for JIA followed the internationally agreed classification as described in [28].

Blood was collected in a heparinised tube and peripheral blood mononuclear cells (PBMC) were isolated by density gradient centrifugation with Lymphoprep (Stem Cell Technologies). The blood samples were collected from the JIA patients at different time points of treatment.

Cell sorting

Isolated PBMC were sorted by cell sorter (BD FACSAria III, BD Biosciences) into different cell populations (S11 Fig) with CD4-BV711 (clone OKT4, Biolegend 317440), CD8-APC (clone SK1, Biolegend 344722), CD14- FITC (clone 61D3, eBioscience 11-0149-42), and CD19-PE-Cy7 (clone HIB19, Biolegend 302216). CD3-BV605 (clone OKT3, Biolegend 317322) was used to differentiate between T cell and non-T cell populations. Dead cells were excluded before sorting using 4,6-diamidino-2-phenylindole (DAPI; Sigma). Sorted cell purity was accessed and on average was>90%. For each subject, we divided CD4, CD8, CD14, and CD19 cell counts by the sum of CD4, CD8, CD14 and CD19 cells to obtain cell-type fractions.

RNA sequencing and data processing

Unsorted PBMC and sorted immune cells were extracted with PicoPure RNA Isolation Kit (Applied Biosystems, KIT0204). The extracted RNA samples were sent to UCL Genomics for library preparation and sequencing.

RNA sequencing was carried out in four batches using Illumina NovaSeq6000. PBMC and sorted cell RNA-seq data were processed using the RSSnextflow (Resources), an RNA-seq pipeline customised for unique molecular identifiers (UMIs) tagged RNA-seq data built under the Nextflow framework [29]. Briefly, sequencing reads (2x100bp) were mapped to the reference genome GRCh38 using STAR aligner [30]. Two-passing mapping mode was used, with gene annotated features (Homo_sapiens.GRCh38.103.gtf) and the options of --twopassMode Basic and --sjdbOverhang 99. Default parameters were used unless otherwise specified. Read PCR duplicates were identified based on alignment coordinates and up to 1 mismatched UMI sequence using the je suite tool [31]. After deduplication, aligned reads were summarised over the gene features using the featureCounts programme [32], and a read count table was generated for each batch.

We selected RNA samples that have RIN ≥ 5.0 and library concentration ≥ 4.5nM (~0.7ng/µL) for RNA sequencing. Illumina TruSeq mRNA stranded v2 and Roche Kapa mRNA HyperPrep were used to create libraries. Samples from the same subjects were sequenced in the same batch, and we employed Combat-seq [33] to minimise batch effects (S12 Fig). Read counts from across the four batches were analysed together. A total of 723 RNA-seq samples from 158 subjects with data on PBMC and at least one cell type were used in the downstream analysis. We filtered out genes with counts-per-million <0.836, equivalent to 10 read counts in our median library size of 11.95 million reads, in less than 96 samples. Also, genes were excluded if their total read counts across samples were less than 15. These filtering steps were conducted using the edgeR filterByExpr function [34,35], with cell type information as the group argument. After filtering out low expressed genes, transcripts per million (TPM), as recommended by the authors of CIBERSORT [36], was estimated and utilised as observed expression.

Training/testing set

Of 158 subjects with PBMC gene expression data, 80 had complete RNA-seq data on all four sorted cells and PBMCs and formed the training set. The remaining 78 had data on PBMCs and partially complete gene expression data in sorted cells and were used as test samples, with numbers for each cell type of CD4: 71, CD8: 65, CD14: 52, CD19: 57 (Fig 2A). Sequencing batch did not differ significantly between training and testing sets (Chi-squared test p > 0.05, S13 Fig).

eQTLgen pseudobulk data

Pseudobulk data were generated using scRNA-seq data downloaded from the eQTLgen consortium (https://eqtlgen.org/sc/datasets/1m-scbloodnl-dataset.html) as of 26 September 2024. This dataset includes PBMC samples from 120 healthy Europeans, sequenced under seven conditions: an unstimulated condition (UA) and three in vitro stimulation conditions with C. albicans (CA), M. tuberculosis (MTB), and P. aeruginosa (PA) after 3 hours and 24 hours. Two sequencing chemistries (V2 and V3) were employed [27].

We specifically utilised QC-ed read counts from V2 and V3, along with cell type information and conditions, to create the pseudobulk data. The number of genes differed between V2 and V3; therefore, 19927 genes common to both chemistries were retained. Additional genes were excluded due to having different chromosome annotations (N=35) or not being found (N=4762) in the gene annotation data (Homo_sapiens.GRCh38.103), resulting in 15165 genes remaining.

We utilised Jaakkola and Elo’s method to generate pseudobulk data. Their approach first generates the cell-type fractions randomly based on a normal distribution, using the same mean and standard deviation (SD) observed in the original public dataset [21]. So, we estimated the relative fractions of CD4, CD8, CD14, and CD19 in the eQTLgen single cells for each condition and chemistry from cell barcode annotation files provided. We then averaged these fractions and calculated the corresponding SD across conditions and chemistries for each cell type. The average percentages (SD) were 42.74% (4.30) for CD4, 28.12% (2.40) for CD8, 26.29% (6.21) for CD14, and 2.85% (0.36) for CD19. We used these estimates to generate four cell fractions each sampled individual by generating random normal samples, and standardising these to sum to 1.

We simulated three datasets, each with a sample size of 160. Conditions (UA and 3hCA) and chemistry types (V2 and V3) were randomly assigned to the samples. For each sample, cell-type read count expression was summed over 5000 cells randomly selected from the corresponding cell-type pool at a given condition and chemistry. Bulk expression was calculated by summing the counts from CD4, CD8, CD14, and CD19 cells, all drawn from the cell-type expressions obtained in the previous step. The number of cells selected for each type was proportional to the fractions generated above.

We processed these three pseudobulk datasets using Combat-seq, incorporating the chemistry information as a batch factor. This approach mimicked a scenario where the batch effect was minimised during the data processing step. Both three original pseudobulk data and their Combat-seq counterparts were utilised to evaluate the performance of predicting sample-level cell type expression using different sample sizes of the training samples. For each of the six simulation datasets, we designated the first half of samples as the training set and the rest as the testing set. We then varied the sample sizes of training samples, from 25% (N=20) to 100% (N=80) by 25%, while keeping the same testing samples. All cells used to generate data for any sample were either from cells stimulated with C. albicans or untreated. We evaluated every trained model’s ability to predict the outcome of in vitro stimulation in the held out test data, quantifying this by AUC. In this case, there were six pseudobulk datasets at each sample size: three from the original pseudobulk and three from Combat-seq processed datasets, resulting in a total of 24 pseudobulk datasets. For each pseudobulk data, low expressed genes were filtered out using the edgeR filterByExpr function on cell type information prior to TPM expression calculation. No. genes in these pseudobulk data ranged from 11964 to 12464.

Signature gene matrices & cell-type specific genes

Two signature gene matrixes were used. CIBERSORTx in-built LM22 was derived from microarray gene expression in 22 purified leukocyte subsets [10]. We constructed a custom signature based on CD4, CD8, CD14, and CD19 TPM in the training subjects using the CIBERSORTxFractions module, with the default settings of “--G.min 300 --G.max 500 --q.value 0.01 --QN FALSE --single_cell FALSE” for sorted RNA-seq. For CLUSTER data there were fewer signature genes in the inbuilt matrix (N=547) than the custom one (N=1589), which we attributed to the dynamic ranges of gene expression measured by two different platforms, microarray for inbuilt and RNA-seq for custom (S10A and S10B Fig).Sorted cell expression in training samples was also used in debCAM to identify cell-type specific genes for fraction deconvolution. debCAM OVE.FC (one versus everyone - fold change) criteria of 1, 2, 5 and 10 were used (S10C Fig), and we selected 1247 cell-type specific genes from OVE.FC of 10 because this number was comparable to custom signature genes. Despite differences inbuilt/custom signature and debCAM cell-type genes separated testing samples well based on their cell types (S10D–S10F Fig).

For the eQTLgen data, a custom signature matrix was derived from synthesised expression in pure cells from the training samples using CIBERSORTx and was utilised later in CIBERSORTx for predicting sample-level cell-type expression.

CIBERSORTx analysis

We ran CIBERSORTx locally with a token requested from the CIBERSORTx website (Resources). We ran the CIBERSORTxHiRes module for deconvolution, with RNA-seq default settings and “--variableonly TRUE” for only outputting genes with variation in expression across subjects. When the LM22 signature was applied, two additional arguments of “--classes” and “--rmbatchBmode” were specified to aggregate cell type expression for 11 major leukocytes based on the shared lineage of 22 leukocytes [10] and minimise measuring variations in gene expression introduced by platforms, respectively.

CIBERSORTx fractions of CD4, CD8, CD14, and CD19 for LM22 were derived from the sum of proportions in their shared-lineage leukocytes (S10A and S14 Figs):

• CD4 as the sum of the proportions of T cells CD4 naive, T cells CD4 memory resting, T cells CD4 memory activated, T cells follicular helper, and T cells regulatory (Tregs). T cells
• CD8 was used as CD8 fraction
• CD14 fraction as the sum of Monocytes, Macrophages M0, Macrophages M1, and Macrophages M2
• CD19 as the sum of B cells naive and B cells memory

We then scaled the resultant proportions to the sum of 1. Imputed cell-type expression was log2 transformed for downstream evaluation.

bMIND and debCAM/swCAM prediction

bMIND cell fraction deconvolution was carried out using the authors’ bMIND function with our custom signature. Cell-type specific genes that were 10-fold over-expressed in one cell type compared to others, as previously described, were specified in debCAM AfromMarkers function for estimating cell fractions from PBMC mixture.

We followed the authors’ instructions to run bMIND and swCAM in a supervised mode on true cell fractions, as measured by flow cytometry in the CLUSTER data or designated ones in eQTLgen data, in the same samples to predict cell type gene expression for samples. More specifically, bMIND predicted expression profiles were obtained from the bMIND function of the MIND package [20], which took log2(TPM+1) transformed PBMC expression and cell fractions as inputs.

swCAM consisted of two steps. In the fine-tuning step, we conducted 10-fold cross-validation, randomly removing one-tenth of gene expressions from the sample and gene expression matrix followed by imputing back expression missingness, to determine the optimal lambda with the minimum of RMSE between missing and imputed expression. The R script, script-swCAM-cv.R, was used. In the predicting step, lambda of 800, PBMC TPM expression, true cell fractions, and grouped cell expressions (cell types * gene matrix) derived from NNLS were used in the sCAMfastNonNeg function for imputing sample-wise cell type expression. All the R scripts and functions related to swCAM were obtained from the authors’ GitHub repository https://github.com/Lululuella/swCAM [23]. We then log2 transformed the estimates of cell type expression for downstream analysis.

LASSO/ridge training and prediction

Standard penalised regression approaches aim to predict a single variable given observations of multiple predictors. Ordinary least squares aims to identify β that minimises for predictors and outcomes y measured in subjects . In our application, y is the expression of a target gene in a specific cell type, is the expression of gene j in PBMCs, is an intercept term and is the coefficient to predict y from . Penalised regression approaches minimise this quantity subject to constraints: (LASSO) or (ridge). We could train a separate LASSO model using all genes in the mixed cell data to predict each gene in a sorted cell in turn. However, given the correlations between expressions of different genes, we hypothesised that multi-response models may be more accurate given they allow sharing of information between the genes to be predicted. In a multi-response model, all genes in the mixed cell data are still used but now prediction is made for a set of genes. Information is shared as coefficients for any predictor are either zero for all target genes or non-zero for all target genes. However, note that non-zero coefficients have distinct values for each target gene. Thus multi-response models are appropriate to predict subsets of correlated genes (Fig 1).

We first used cell-type specific expression in training samples to define these subsets of correlated genes. We clustered genes into chunks on the basis of their expression profiles (in log2 TPM), with a size of up to 500 genes. Specifically, we performed a hierarchical clustering analysis on Euclidean distances between genes for each cell type in the training data. We grouped clustering dendrograms into the initial number of chunks, the minimum multiple of 500 to include all genes, using the R cutree function. For each chunk exceeding the size of 500 genes, we repeated clustering analysis on genes in a given chunk, followed by dendrogram grouping using the cutreeDynamic function [37] specified with “minClusterSize” of 250, if necessary, reducing by steps of 5, until all the resultant chunks met the desired size of < 500. The numbers of chunks (sizes) in the CLUSTER data were 74 (49-481) for CD4, 71 (71-493) for CD8, 76 (76-496) for CD14, and 83 (41-486) for CD19, respectively. In 24 pseudo-bulk datasets, the total numbers of gene chunks (respective size ranges) were 1358 (4-500) for CD4, 1306 (8-498) for CD8, 1533 (3-499) for CD14 and 1180 (2-497) chunks for CD19.

For each chunk, we fitted a penalised multi-response linear regression (LASSO/ridge) performed using PBMC expression in all genes to predict cell specific expression in the chunk genes, on the log2(TPM+1) scale. To ease the computational burden, the fine-tuned LASSO/ridge model from a 5-fold cross-validation based on the mean squared error (MSE) criterion was used to impute cell-type expression in the testing samples. Penalised modelling was carried out using the R glmnet package [38], for which“family=mgaussian”, “type.measure =mse”, and alpha=1 for LASSO/alpha=0 for ridge were specified in the cv.glmnet function for model training, and predict function was used for imputation.

Performance measurement

Predicted cell fractions were compared to the flow cytometry estimates described above, and Pearson r correlations and root mean square errors (RMSE) were calculated for measuring performance. We calculated correlations and RMSE per gene between imputed and observed log2 transformed expression to evaluate and benchmark the predicted accuracy of expression among CIBERSORTx using custom and inbuilt, bMIND using custom, swCAM with cell-type specific markers, LASSO and ridge.

We simulated ten phenotypes that correlated with gene expression in the observed cell type data, including testing samples, using principal component analysis (PCA). First we conducted PCA of the fully observed gene expression data in the target cell type. From each of these we derived a continuous and a binary phenotype. The continuous phenotypes were defined as the PC value plus a standard normal error. The binary phenotypes were by dichotomising each PC with values >0 designated as 1 for pseudo-cases; otherwise, 0 for pseudo-controls. For each simulated phenotype, DGE analysis was carried out twice in parallel under the limma framework [39]:

1. in all observed data (test + training)
2. In observed data from the training samples + imputed data from the test samples which had observed data for comparison

We treated (1) as the ground truth, and (2) as the likely analysis adopted in any real world study. The expression matrix in (1) and (2) were used in the same limma DGE analysis. We utilised log2 transformed expression data and a predefined design matrix that incorporated simulated phenotypes (with and without sex for CLUSTER data) or treatment conditions (with and without batch for the simulation data) in the limma::lmFit function to fit the linear model, followed by empirical Bayes moderation of the standard errors using the limma::eBayes function. A false discovery rate (FDR) of 0.05 was set as the significance level for true signals in the observed data. We varied FDRs in the imputed data from 0 to 1 by steps of 0.05. Receiver operating characteristic analysis was carried out by combination of cell type, method and scenario using the pROC package [40].

All analyses were performed with R-4.3.3 [41] unless otherwise stated

Resources

1. CLUSTER consortium-Childhood arthritis and its associated uveitis: stratification through endotypes and mechanism to deliver benefit, https://www.clusterconsortium.org.uk/
2. RSSnextflow workflow for processing RNA-seq FASTQ files to generate analysis-ready read counts https://gitlab.com/b8307038/rssnextflow
3. CIBERSORTx, https://cibersortx.stanford.edu/
4. bMIND, https://github.com/randel/MIND
5. swCAM, https://github.com/Lululuella/swCAM

Code availability

Nextflow workflow, R scripts, and markdown files to run the analyses, to generate and to summarise results in this work presented here, https://gitlab.com/b8307038/sceqtlgen