Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Validation of two multiplex platforms to quantify circulating markers of inflammation and endothelial injury in severe infection

  • Aleksandra Leligdowicz,

    Affiliations Department of Medicine, University of Toronto, Toronto, Canada, Sandra A. Rotman Laboratories, Sandra Rotman Centre for Global Health, University Health Network, Toronto, Canada

  • Andrea L. Conroy,

    Affiliation Department of Pediatrics, Indiana University School of Medicine, Indianapolis, United States of America

  • Michael Hawkes,

    Affiliation Division of Pediatric Infectious Diseases, University of Alberta, Edmonton, Canada

  • Kathleen Zhong,

    Affiliation Department of Medicine, University of Toronto, Toronto, Canada

  • Gerald Lebovic,

    Affiliation Applied Health Research Centre, The HUB, Li Ka Shing Knowledge Institute, University of Toronto, Toronto, Canada

  • Michael A. Matthay,

    Affiliations Departments of Medicine and Anesthesia, University of California, San Francisco, United States of America, Cardiovascular Research Institute, University of California, San Francisco, United States of America

  • Kevin C. Kain

    kevin.kain@uhn.ca

    Affiliations Department of Medicine, University of Toronto, Toronto, Canada, Sandra A. Rotman Laboratories, Sandra Rotman Centre for Global Health, University Health Network, Toronto, Canada

Validation of two multiplex platforms to quantify circulating markers of inflammation and endothelial injury in severe infection

  • Aleksandra Leligdowicz, 
  • Andrea L. Conroy, 
  • Michael Hawkes, 
  • Kathleen Zhong, 
  • Gerald Lebovic, 
  • Michael A. Matthay, 
  • Kevin C. Kain
PLOS
x

Abstract

Biomarkers can prognosticate outcome and enable risk-stratification. In severe infection, focusing on multiple markers reflecting pathophysiological mechanisms of organ injury could enhance management and pathway-directed therapeutics. Limited data exist on the performance of multiplex biomarker platforms. Our goal was to compare endothelial and immune activation biomarkers in severe pediatric infections using two multiplex platforms. Frozen plasma from 410 children presenting to the Jinja Regional Hospital in Uganda with suspected infection was used to measure biomarkers of endothelial (Angiopoietin-2, sFlt-1, sVCAM-1, sICAM-1) and immune (IL-6, IP-10, sTNFR-1, CHI3L1) activation. Two multiplex platforms (Luminex®, EllaTM) based on monoclonal antibody sandwich immunoassays using biotin-streptavidin conjugate chemistry were selected with reagents from R&D Systems. The two platforms differed in ease and time of completion, number of samples per assay, and dynamic concentration range. Intra-assay variability assessed using a coefficient of variation (CV%) was 2.2–3.4 for Luminex® and 1.2–2.9 for EllaTM. Correlations for biomarker concentrations within dynamic range of both platforms were best for IL-6 (ρ = 0.96, p<0.0001), IP-10 (ρ = 0.94, p<0.0001) and sFlt-1 (ρ = 0.94, p<0.0001). Agreement between concentrations obtained by both methods assessed by the Bland-Altman test varied, with best agreement for CHI3L1. Our data suggest that biomarkers of endothelial and immune activation can be readily measured with multiplex platforms. Luminex® and EllaTM produced reliable results with excellent CV% values. The EllaTM platform was more automated and completed in 75 minutes, potentially compatible with near-patient use. Trends in concentrations obtained by these methods were highly correlated, although absolute values varied, suggesting caution is required when comparing data from different multiplex platforms.

Introduction

The search for novel biological markers to predict response to therapies, prognosticate outcome, or assist in patient enrollment in clinical therapeutic trials is quickly evolving [1]. In the context of life-threatening infection, many biomarkers have been proposed to improve the discriminatory ability to achieve these goals [2, 3]. Emphasizing markers of pathophysiological pathways involved in severe infections and focusing on multiplex platforms with near-patient or point-of-care potential, could accelerate the development of precision medicine tools for life-threatening infections [4, 5].

The third iteration of the international consensus definitions for sepsis and septic shock [6] acknowledged that while multiple candidate biomarkers have been evaluated, robust validation is required prior to incorporating them into a clinical definition of sepsis. However, many challenges exist with the validation of biomarkers, including inconsistency in the biological reagents used, the detection platforms utilized, the combinations and permutations of markers tested, the diversity of patient cohorts from which samples are derived, and the statistical tests used to analyze the results.

Of the many pathophysiologic pathways that may contribute to the high morbidity and mortality of severe infections, endothelial [7, 8] and immune activation [9, 10] have been studied extensively. Key biomarkers of endothelial activation include the Angiopoietin-Tie2 axis [11], the soluble variant of the vascular endothelial growth factor, a receptor known as soluble fms-like tyrosine kinase-1 (sFlt-1) [12, 13], soluble vascular cell adhesion molecule-1 (sVCAM-1) and soluble intercellular adhesion molecule-1 (sICAM-1) [14, 15]. Multiple markers of inflammation have been identified and among these interleukin 6 (IL-6) [16], interferon-gamma-inducible protein-10 (IP-10, CXCL10) [17], chitinase-3-like-1 protein (CHI3L1) [18] and soluble tumor necrosis factor receptor-1 (sTNFR-1) [15, 19] have been correlated with severity of illness and clinical outcome in sepsis as well as other critical care illnesses such as the acute respiratory distress syndrome, usually caused by severe infection [20, 21].

The quantification of plasma proteins still largely relies on the use of enzyme-linked immunosorbent (ELISA)-based assays [22]. In the past decade, the simultaneous detection of multiple distinct proteins was enabled by using highly specific capture and detection monoclonal antibodies [2326]. The detection antibodies can be conjugated to different indicators, allowing for the quantification of properties such as optical density, electrochemiluminescence, chemiluminescence, or fluorescence intensity. The property of the indicator as well as the instrument used to detect it lead to variation in the sensitivity and dynamic range of the assay, the number of analytes that can be simultaneously analyzed, and reagent cost. Based on its established performance over the past 20 years, we selected the magnetic microsphere-based Luminex® platform [2629] and compared it to a novel, fully automated microfluidics-based platform, EllaTM [3032].

The main goal of this study was to use reagents prepared by a single manufacturer (R&D Systems) to compare concentrations of previously identified biomarkers of endothelial and immune activation in patients with severe infection measured by two different multiplex platforms. Secondly, we address the challenge of using appropriate statistical tests to compare the performance characteristics of two platforms, with an analytic focus on the ease of use, assay dynamic range, intra-assay variability, and agreement between biomarker concentrations computed by each platform. Our findings have important implications for future near-patient and point-of-care biomarker quantification in life-threatening infection.

Materials and methods

Patient sample selection

A previously described prospective cohort of 2,085 consecutive febrile children aged 2 months to 5 years old who presented to the Jinja Regional Referral Hospital in Uganda, 4.7% of whom died during hospital admission, [33] was used to generate a subcohort of patients for this study. Subjects for this nested case cohort design were selected by randomly sampling 18% of the whole cohort and adding all non-sampled deaths that occurred during hospital admission. The subcohort included 410 children, 99 of whom died during hospital admission and 301 who survived to hospital discharge.

Ethical approval was obtained from Uganda National Council for Science and Technology, Makerere University Research Ethics Committee in Uganda, and the University Health Network. Written informed consent for all study participants was provided by the parent or caregiver.

Plasma sample preparation

Up to 1 ml of whole blood was collected by venipuncture and anticoagulated using ethylenediaminetetraacetic acid (EDTA). Blood was centrifuged within 4 hours of sample collection and plasma was frozen at -80°C without freeze-thaw until analyzed. Samples were thawed overnight at 4°C and aliquoted at room temperature immediately prior to assay performance.

Luminex® platform protocol

Reagents for Luminex® assays were custom developed by R&D Systems. Thirteen biomarkers were selected and divided into 2 panels based on relative plasma abundance and assay dynamic range. Panel 1 included the following 5 high-abundance biomarkers tested at a dilution of 1:30: sVCAM-1, sICAM-1, sTNFR-1, CHI3L1, Cystatin C. Panel 2 included the following 8 low-abundance biomarkers tested at a dilution of 1:3: Angiopoeitin-2 (Ang-2), sFlt-1, IL-6, IP-10, Ang-1, IL-8, sTREM-1, Granzyme B. The 5 biomarkers that did not overlap between the platforms are presented in S1 Table. Unfiltered plasma was diluted using diluents supplied by the manufacturer. Each 96-well plate included 7-fold serial dilutions of standards tested in duplicate and 72 patient samples, 8 of which were tested in duplicate. A total of 6 batches of Luminex Panel 1 and 2 were necessary to complete analysis of 410 samples. Assays were performed according to manufacturer’s magnetic Luminex® screening assay protocol [34]. Briefly, a microparticle cocktail, diluted plasma, and biomarker standards were added to a 96-well plate. Following a 2-hour incubation, plates were washed and a biotin antibody cocktail was added. After a 1-hour incubation, plates were washed and streptavidin-Phycoerythrin (PE) was added for 30-min, followed by a final wash and resuspension in wash buffer. All incubations were done at room temperature on a microplate shaker at 800 rpm. Plates were read immediately on the MAGPIX® instrument and raw data were analyzed using the xPONENT® software. Values outside the lower limit of quantification were assigned a value of 1/3 of the lower limit of the standard curve.

EllaTM platform protocol

Reagents for the Simple PlexTM Ella microfluidic platform (Protein Simple, CA, USA) were custom developed. Eight biomarkers that overlapped with the Luminex® platform were selected and divided into 2 panels based on relative plasma abundance and assay dynamic range. Panel 1 included the following 4 high-abundance biomarkers tested at a dilution of 1:100: sVCAM-1, sICAM-1, sTNFR1, CHI3L1. Panel 2 included the following 4 low-abundance biomarkers tested at a dilution of 1:10: Ang-2, sFlt-1, IL-6, IP-10. Plasma samples were analyzed on the same day and after a single thaw, the same as the corresponding samples analyzed using the Luminex® platform. Unfiltered plasma samples were diluted using diluents supplied by the manufacturer and assays were performed according to manufacturer’s protocol. Briefly, 50 μl of diluted plasma was added to the appropriate cartridge, followed by placement in to the Ella instrument, requiring no further user intervention. Each cartridge included a built-in lot-specific standard curve and samples were run as internal triplicates. This was accomplished by the presence of three nanorods present inside each individual channel corresponding to a single biomarker and each nanorod was coated with biomarker-specific capture monoclonal antibodies [30]. Detection monoclonal antibodies and streptavidin-DyLight650 conjugate as well as all washing steps were automatically performed by the instrument. Data were manually screened from each run for clogged nanorods or >20% discrepancy in readings between one of the three nanorods. Sixteen samples were accommodated by each cartridge, 5–6 cartridges were performed per day, and all assays were completed within 6 days. Raw data were analyzed using the SimplePlex Explorer software.

Statistical analysis

Intra-assay variability was calculated using coefficient of variation (CV%) values [35] with the following formula: [standard deviation (σ)/mean (μ)]*100. The CV% values for the Luminex® platform were calculated from concertation values of duplicate samples from subjects who were part of the main cohort. 156 samples were included, spread across 20 plates. The CV% values for the EllaTM platform were calculated from internal triplicate values of samples from all patients who were part of the subcohort. 410 samples were included, spread across 26 cartridges.

Platform comparisons were performed for values within dynamic range of both assays using three statistical tests [36, 37]. Spearman’s rank correlation was used to compute relationships of biomarker concentrations obtained by each platform. The paired Wilcoxon signed rank sum test was used to compare the median values obtained by each platform. The Bland-Altman method [38] was used to measure agreement between of Loge-transformed biomarker concentrations obtained by each platform. The variance of differences in concentration ([Loge Luminex]-[Loge Ella]) across the mean of concentration values (([Loge Luminex]+[Loge Ella])/2) was calculated using the Pitman's test [39]. Linear mixed effects models were used to calculate the 95% confidence intervals and p-values for the difference between Loge-transformed biomarker concentrations (bias), correcting for random Luminex batch effects for values within dynamic range of both assays.

Statistical analyses were performed using STATA v14.1 (StataCorp 2015). Graphical presentation was done using Excel (Microsoft 2016) and Prism v5.0a (GraphPad 2007).

Results

Biomarker assay performance

The Luminex® and EllaTM platforms differed in ease of assay performance, number of samples analyzed per run, and time to perform each assay. Both platforms required optimization of plasma dilutions to ensure biomarker concentrations fell within the dynamic range of the assay.

The time to complete a Luminex assay was approximately 5 hours, accommodating up to 80 samples and up to 49 analytes per plate. The assay required manual preparation of serial dilutions of analyte standards, addition of pre-diluted samples, micro-particles, biotin-labeled antibodies, streptavidin substrate, three triplicate wash steps, and loading the plate into a MAGPIX instrument.

The time to complete an Ella assay was 1.25 hours, accommodating up to 16 samples and up to 4 analytes per plate. The EllaTM platform requires loading the pre-diluted samples into a cartridge and inserting it into the instrument. No further user interface is required since the Ella instrument is fully automated, performs all assays in triplicate and uses internal microfluidics for completion of all immunoassay wash and detection steps.

Biomarker assay limits of quantification and dynamic range

The Luminex® and EllaTM platforms differed in lower limit of quantification (assay sensitivity) and biomarker dynamic range (Table 1). For samples measured using the Luminex® platform at an appropriate plasma dilution, 63.7–99.5% were within assay dynamic range. Majority of values outside of dynamic range were below the lower limit of quantification, suggesting lower assay sensitivity. For samples measured using the Ella platform at an appropriate plasma dilution, 94.6–100% were within assay dynamic range. The EllaTM dynamic range was superior compared to Luminex® platform using reagents from R&D System especially, especially at the lower limits of quantification, and the only biomarker with values below the lower limit of assay quantification was IL-6. sICAM-1 values were not included in analyses as the sample dilution was inappropriate for reliable concentration quantification of values above the upper limit of the standard curve.

thumbnail
Table 1. Luminex® and EllaTM multiplex platforms: dynamic range and limits of quantification for biomarkers of severe infection.

https://doi.org/10.1371/journal.pone.0175130.t001

Intra-assay variability

Intra-assay performance was calculated using coefficient of variation (CV%) values, to determine the precision of the concentration values obtained by both platforms [35]. The results are shown in Fig 1.

thumbnail
Fig 1. Intra-assay variability for biomarker concentrations determined using the Luminex® and EllaTM platforms.

Coefficient of variance (CV%) values for 156 sample assayed in duplicate by the Luminex® platform (shaded circles) and for 406–410 samples assayed by internal triplicate by the EllaTM platform (open triangles). Analysis for sICAM-1 was excluded due to inappropriate dilution range. Graphs depict point estimates of the mean CV% and the corresponding 95% confidence intervals.

https://doi.org/10.1371/journal.pone.0175130.g001

For the Luminex® platform, 156 random samples were assayed in duplicate. The CV% for the 8 biomarkers varied between 2.2–3.4. For the EllaTM platform, all samples were run as internal triplicates and 2.7–12.3% of biomarkers were analyzed in duplicate due to an occlusion in one of three glass nanorods. For the 7 biomarkers analyzed at appropriate dilutions, the CV% varied between 1.2–2.9.

Inter-assay variability was determined by quantifying the same healthy control sample on each day of analysis (total of 6 days). The same sample was frozen in multiple aliquots and each aliquot was thawed once on the day of the assay. The quantified biomarkers for the healthy control sample were at the lower end or below the limit of quantification for both multiplex platforms. Therefore, inter-assay variability was greater than if a sample with biomarker concentration within the mid-zone of the standard curve was selected (S2 Table).

Comparisons of concentration correlations

Biomarker concentrations within dynamic range of both assays were compared using the Spearman correlation and paired Wilcoxon signed rank sum test. The strength of association for biomarker concentrations obtained using both platforms was excellent. For biomarkers quantified at appropriate dilutions, the strength of association varied between Spearman’s rho correlation coefficient (ρ) values of 0.79–0.97 (p-values <0.0001 for all analytes, Fig 2). The absolute median values for individual biomarkers quantified using the Luminex® and EllaTM platforms varied (Table 2). For biomarker concentrations within the dynamic range of both multiplex platforms, there was no difference between the median CHI3L1 concentrations (p value = 0.99) but other biomarker concentrations varied significantly between the two multiplex platforms.

thumbnail
Fig 2. Relationship between raw untransformed concentration values obtained by the Luminex® compared to the EllaTM platform.

Spearman’s rho correlation coefficient (ρ) and p-values obtained using Spearman rank correlation for values (n) within assay dynamic range of both platforms.

https://doi.org/10.1371/journal.pone.0175130.g002

thumbnail
Table 2. Correlation between Luminex® and EllaTM platform concentrations in pg/mL.

https://doi.org/10.1371/journal.pone.0175130.t002

Agreement between platforms

The Bland-Altman method [38] was used to assess agreement between Loge-transformed biomarker concentrations generated using the Luminex® and EllaTM platforms. This graphical tool allows for depicting systematic bias between two methods of measurement and significant outlier values across the range of assay values.

As shown in Fig 3, the Loge-transformed concentrations of the 7 biomarkers showed varying patterns of agreement. These results were consistent with significant differences in median values demonstrated by paired Wilcoxon signed rank sum test shown in Table 2. A positive bias was indicative of higher concentration values measured by the Luminex® platform, while a negative bias higher concentrations obtained by the EllaTM platform, A value close to zero meant that there was no difference between the two platforms for concentrations within the dynamic range of the two platforms. A significant bias towards one of the platforms was identified for all biomarkers, with the exception of CHI3L1 (Table 3). Using the Pitman's test of difference in variance, the bias for most biomarkers was consistent across the range of concentrations of both platforms, (p-values displayed in Fig 3).

thumbnail
Fig 3. Bland-Altman plots comparing agreement between biomarker concentrations determined using the Luminex® and EllaTM platforms.

Upper and lower limits of agreement (dotted black lines) correspond to 2 standard deviations (SD) away from the mean difference (dashed black line). P-values represent Pitman's test of difference in variance, with a non-significant value indicating no variance across the range of mean values determined by both platforms. Analyses were performed on Loge-transformed biomarker concentrations that were within assay dynamic range of both platforms (summarized in Table 2).

https://doi.org/10.1371/journal.pone.0175130.g003

thumbnail
Table 3. Bland-Altman bias representing difference in Loge-transformed biomarker concentrations computed by Luminex® compared to the EllaTM platform for values within dynamic range of both assays.

https://doi.org/10.1371/journal.pone.0175130.t003

Discussion

In the past decade, the use of multiplex immunoassays has made the quantification of multiple analytes obtained from clinical samples more feasible. However the variety of potential platforms and reagents has created challenges in selecting optimal assays for large-scale projects that demand consistent results and ultimately has hindered the use of these platforms in clinical research and their translation into clinical practice [25, 27, 40]. The lack of available data comparing multiplex platforms using consistent reagents makes it difficult to compare previous studies examining host biomarkers of severe infections. In this study, we evaluated two different multiplex platforms (Luminex® and EllaTM) using reagents from a single manufacturer (R&D Systems) to test over 400 well-annotated plasma samples from febrile children presenting with suspected severe infections. We found that the correlations between values for biomarkers of endothelial and immune activation tested within assay dynamic range were excellent. Moreover, using two different statistical methods, we identified consistent differences between absolute biomarker concentrations obtained using the two different multiplex platforms.

The selection of a multiplex immunoassay platform for a specific clinical project is based on many factors. When considering the optimal platform, we evaluated the availability of reagents for analytes of interest, the dilution factors to enable inclusion of multiple biomarkers in a single assay, the ease of assay performance, the time required to complete the assay with potential for translation to near-patient-care settings, as well as cost. Based on these criteria, the Luminex® [27] and EllaTM [30] platforms were selected. Both platforms were reliable, with CV% for both platforms less than 5%, which is below the acceptable Food and Drug Administration (FDA) recommendations [41]. The EllaTM platform has the advantage of being fully automated with internal triplicate sample testing, decreasing the sample and reagent quantity, and time required for assay performance. Although the per analyte cost was 5.8-fold greater for the Ella relative to the Luminex platform, the expense related to staff time for assay performance and data analysis could decrease this difference.

All of the biomarkers included in this study have been associated with severity of illness in severe infection [719]. Therefore, results of this study provide valuable data to guide future research in predictive and prognostic markers for life-threatening infections. In addition, several biomarkers evaluated here are noteworthy for their roles in infection-induced endothelial injury and as therapeutic targets for future clinical trials [4244]. Validating a platform with minimal user interface and fast turn-around times would enable its use at near-patient-care to facilitate patient recruitment into novel clinical therapeutic trials.

Of note, there were significant differences observed in absolute biomarker concentrations between these two platforms. The bias towards higher concentrations with either the Luminex® or EllaTM platforms, however, was uniform across the concentration ranges for most biomarkers as identified by the non-significant Pitman test p-values. This suggests that values of the biomarker concentrations were not different between the platforms at a different concentration range, making the relative values between the two tests comparable. Also worth noting is that although the absolute biomarker concentration values were statistically significantly different, the visual representation of the Loge-transformed biomarker concentrations displayed on Bland-Altman plots suggest that these concentrations were in a similar pg/mL range. The differences in absolute concentrations could be attributed to differences in the capture and/or detection monoclonal antibody specificity and/or affinity as described in S3 Table. Alternatively, competition between multiple monoclonal antibodies for antigen detection in the multiplex Luminex® assay could have been overcome with the EllaTM platform multiplex design which uses independent microfluidic channels for each analyte. Finally, although both methods used reagents from R&D Systems and fluorescence was used as the indicator, the fluorochrome conjugated to streptavidin differed (Phycoerythrin (PE) for Luminex® and DyLight650 conjugate for EllaTM), affecting how the fluorochrome was excited, and what emission wavelength was detected by each platform, possibly contributing to a difference in concentration ranges on both lower and upper limits of quantification.

Selecting a platform to analyze large number of samples in a short time frame is best performed with an assay that can be done in multi-well format, enabling high throughput. Of the two platforms we tested, the Luminex® platform was better suited to this purpose. However, when near-patient or point-of-care data is required, with fast turn-around and with the capacity to test fewer samples, the EllaTM platform offers the required characteristics. Our data provide evidence that concentrations obtained by both platforms are highly correlated. Therefore, it is possible to use either platform for clinical research examining a relationship between biomarker level and clinical end-point. However, given the statistically significant differences in absolute values, direct comparison of concentration values between platforms is problematic. These observations have implications for future point-of-care novel biomarker research as direct comparisons of data will likely only be possible if assays are performed using the same methodology with reagents obtained from the same manufacturer.

The strengths of this study include inclusion of biomarkers of current clinical interest in the field of critical illness and life-threatening infections. The large sample size of more than 400 patients provides this study with higher analytical power. Additionally, the selection of the Luminex® microsphere-based system, an established multiplex platform [27], as well as the novel EllaTM microfluidics-based system [30] allowed the validation a novel platform with the potential to be used in future studies requiring real-time results. One of the clearly demonstrated strengths of the EllaTM platform was a superior dynamic range compared to Luminex using reagents from R&D Systems.

This study had some limitations. First, frozen versus fresh plasma was used. Future studies are planned to validate biomarker concentrations quantified in fresh blood samples. Second, we could not analyze data obtained from the EllaTM platform for sICAM-1 due to inappropriate sample dilution. This limitation highlights the limitation of multiplex assays which require all analytes tested to be at dilution suitable for the assay dynamic range. Also, up to 34% of concentration values quantified by the Luminex platform were below the limit of quantification (CHI3L1, sFlt-1, and IL-6). This limited the ability to compare the complete sample set, likely diminishing the accuracy of the comparison between the two platforms. This also limits the generalizability of the results to more critically ill patients, with higher biomarker levels.

Future clinical studies evaluating the potential of plasma markers as predictive and prognostic tools will require reliable, robust, easy to use multiplex platforms, providing data in real time at the point-of-care. Ultimately this approach could enable precision medicine via rapid triage of febrile patients and stratification for pathway-directed therapeutics for severe infections that may be applicable in critically ill patients identified in an emergency department or an intensive care unit.

In conclusion, plasma biomarkers of endothelial and immune activation can be reliably quantified using multiplex immunoassay-based platforms. The dynamic range of biomarker concentrations is greater for the EllaTM platform. The intra-assay variability is excellent for both multiplex platforms. The trends in concentrations obtained by the Luminex and Ella instruments are highly correlated but the absolute value of analyte concentrations vary between these platforms. Novel multiplex assays that produce rapid and reliable results, with minimal user interface and enable near-patient application, could transform prediction of response to therapies, prognostication of clinical outcome, and risk-stratification for enrollment in clinical trials.

Supporting information

S1 Table. Dynamic range for biomarkers included in the Luminex® platform that did not overlap with EllaTM multiplex platform.

https://doi.org/10.1371/journal.pone.0175130.s001

(DOCX)

S2 Table. Inter-assay variability for a single healthy control quantified across 6 Luminex® assay plates and 6 EllaTM cartridges.

https://doi.org/10.1371/journal.pone.0175130.s002

(DOCX)

S3 Table. Comparison of reagents included in the Luminex® and EllaTM platform.

https://doi.org/10.1371/journal.pone.0175130.s003

(DOCX)

Acknowledgments

We would like to thank the children and their care givers from the Jinja Regional Referral Hospital for participation in the prospective observational study to investigate the relationship between biomarkers of endothelial and immune activation and mortality. We also thank the biostatistics faculty members of the Training in Clinical Research program at the University of California, San Francisco, for their guidance in statistical analysis selection. This work was supported by the Canadian Institutes of Health Research (CIHR) grants MOP-13721, MOP-115160, MOP-136813, a CIHR Foundation grant FDN-148439 (K.C.K.), the Canada Research Chair Program (K.C.K), a CIHR Banting fellowship (A.L.), and donations from Kim Kertland and the Tesari Foundation. M.A.M. has received grants from Amgen for experimental studies on acute lung injury and from GlaxoSmithKline for studies of sepsis in patients; consultancy agreements for ARDS or lung diseases with Boehringer-Ingelheim, Bayer Inc, Cerus Therapeutics, Biomarck Pharmaceuticals, Quark Pharmaceuticals, Thesan Pharmaceuticals, Incardia Inc, Biogen, and GlaxoSmithKline; DSMB Chair for clinical trials for Roche-Genentec. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Author Contributions

  1. Conceptualization: AL ALC MH KCK.
  2. Data curation: AL ALC MH GL KZ MAM KCK.
  3. Formal analysis: AL ALC MH GL MAM KCK.
  4. Funding acquisition: KCK.
  5. Investigation: AL ALC MH KZ.
  6. Methodology: AL ALC MH GL KCK.
  7. Project administration: AL ALC MH KCK.
  8. Resources: AL ALC MH KZ KCK.
  9. Software: AL GL.
  10. Supervision: AL ALC MH KCK.
  11. Validation: AL ALC MH GL MAM KCK.
  12. Visualization: AL ALC MH MAM KCK.
  13. Writing – original draft: AL ALC MH GL MAM KCK.
  14. Writing – review & editing: AL ALC MH GL MAM KCK.

References

  1. 1. Pene F, Courtine E, Cariou A, Mira JP. Toward theragnostics. Crit Care Med. 2009;37(1 Suppl):S50–8.
  2. 2. Kojic D, Siegler BH, Uhle F, Lichtenstern C, Nawroth PP, Weigand MA, et al. Are there new approaches for diagnosis, therapy guidance and outcome prediction of sepsis? World J Exp Med. 2015;5(2):50–63. PubMed Central PMCID: PMCPMC4436940. pmid:25992320
  3. 3. Reinhart K, Bauer M, Riedemann NC, Hartog CS. New approaches to sepsis: molecular diagnostics and biomarkers. Clin Microbiol Rev. 2012;25(4):609–34. PubMed Central PMCID: PMCPMC3485751. pmid:23034322
  4. 4. Cohen J, Vincent JL, Adhikari NK, Machado FR, Angus DC, Calandra T, et al. Sepsis: a roadmap for future research. The Lancet infectious diseases. 2015;15(5):581–614. pmid:25932591
  5. 5. Matthay MA, Liu KD. New Strategies for Effective Therapeutics in Critically Ill Patients. JAMA. 2016;315(8):747–8. pmid:26903328
  6. 6. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801–10. pmid:26903338
  7. 7. Xing K, Murthy S, Liles WC, Singh JM. Clinical utility of biomarkers of endothelial activation in sepsis—a systematic review. Crit Care. 2012;16(1):R7. PubMed Central PMCID: PMCPMC3396237. pmid:22248019
  8. 8. Lee WL, Slutsky AS. Sepsis and endothelial permeability. N Engl J Med. 2010;363(7):689–91. Epub 2010/09/08. pmid:20818861
  9. 9. Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–74. PubMed Central PMCID: PMCPMC4077177. pmid:24232462
  10. 10. Delano MJ, Ward PA. Sepsis-induced immune dysfunction: can immune therapies reduce mortality? J Clin Invest. 2016;126(1):23–31. PubMed Central PMCID: PMCPMC4701539. pmid:26727230
  11. 11. Ricciuto DR, dos Santos CC, Hawkes M, Toltl LJ, Conroy AL, Rajwans N, et al. Angiopoietin-1 and angiopoietin-2 as clinically informative prognostic biomarkers of morbidity and mortality in severe sepsis. Crit Care Med. 2011;39(4):702–10. pmid:21242795
  12. 12. Shapiro NI, Yano K, Okada H, Fischer C, Howell M, Spokes KC, et al. A prospective, observational study of soluble FLT-1 and vascular endothelial growth factor in sepsis. Shock. 2008;29(4):452–7. pmid:18598002
  13. 13. Skibsted S, Jones AE, Puskarich MA, Arnold R, Sherwin R, Trzeciak S, et al. Biomarkers of endothelial cell activation in early sepsis. Shock. 2013;39(5):427–32. PubMed Central PMCID: PMCPMC3670087. pmid:23524845
  14. 14. de Pablo R, Monserrat J, Reyes E, Diaz D, Rodriguez-Zapata M, de la Hera A, et al. Circulating sICAM-1 and sE-Selectin as biomarker of infection and prognosis in patients with systemic inflammatory response syndrome. Eur J Intern Med. 2013;24(2):132–8. pmid:23352000
  15. 15. Mikacenic C, Hahn WO, Price BL, Harju-Baker S, Katz R, Kain KC, et al. Biomarkers of Endothelial Activation Are Associated with Poor Outcome in Critical Illness. PloS one. 2015;10(10):e0141251. PubMed Central PMCID: PMCPMC4619633. pmid:26492036
  16. 16. Hack CE, De Groot ER, Felt-Bersma RJ, Nuijens JH, Strack Van Schijndel RJ, Eerenberg-Belmer AJ, et al. Increased plasma levels of interleukin-6 in sepsis. Blood. 1989;74(5):1704–10. pmid:2790194
  17. 17. Ng PC, Li K, Chui KM, Leung TF, Wong RP, Chu WC, et al. IP-10 is an early diagnostic marker for identification of late-onset bacterial infection in preterm infants. Pediatr Res. 2007;61(1):93–8. pmid:17211148
  18. 18. Kornblit B, Hellemann D, Munthe-Fog L, Bonde J, Strom JJ, Madsen HO, et al. Plasma YKL-40 and CHI3L1 in systemic inflammation and sepsis-experience from two prospective cohorts. Immunobiology. 2013;218(10):1227–34. pmid:23706599
  19. 19. de Pablo R, Monserrat J, Reyes E, Diaz-Martin D, Rodriguez Zapata M, Carballo F, et al. Mortality in patients with septic shock correlates with anti-inflammatory but not proinflammatory immunomodulatory molecules. Journal of intensive care medicine. 2011;26(2):125–32. pmid:21464065
  20. 20. Calfee CS, Delucchi K, Parsons PE, Thompson BT, Ware LB, Matthay MA, et al. Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. Lancet Respir Med. 2014;2(8):611–20. PubMed Central PMCID: PMCPMC4154544. pmid:24853585
  21. 21. Matthay MA, Ware LB, Zimmerman GA. The acute respiratory distress syndrome. J Clin Invest. 2012;122(8):2731–40. PubMed Central PMCID: PMCPMC3408735. pmid:22850883
  22. 22. Engvall E, Perlmann P. Enzyme-linked immunosorbent assay (ELISA). Quantitative assay of immunoglobulin G. Immunochemistry. 1971;8(9):871–4. pmid:5135623
  23. 23. Nolan JP, Mandy F. Multiplexed and microparticle-based analyses: quantitative tools for the large-scale analysis of biological systems. Cytometry A. 2006;69(5):318–25. PubMed Central PMCID: PMCPMC2200865. pmid:16604537
  24. 24. Elshal MF, McCoy JP. Multiplex bead array assays: performance evaluation and comparison of sensitivity to ELISA. Methods. 2006;38(4):317–23. PubMed Central PMCID: PMCPMC1534009. pmid:16481199
  25. 25. Tighe PJ, Ryder RR, Todd I, Fairclough LC. ELISA in the multiplex era: potentials and pitfalls. Proteomics Clin Appl. 2015;9(3–4):406–22. pmid:25644123
  26. 26. Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP Jr. Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004;61(1):35–9. pmid:15351980
  27. 27. Fulton RJ, McDade RL, Smith PL, Kienker LJ, Kettman JR, Jr. Advanced multiplexed analysis with the FlowMetrix system. Clin Chem. 1997;43(9):1749–56. pmid:9299971
  28. 28. Baker HN, Murphy R, Lopez E, Garcia C. Conversion of a capture ELISA to a Luminex xMAP assay using a multiplex antibody screening method. J Vis Exp. 2012;(65). PubMed Central PMCID: PMCPMC3471270.
  29. 29. Carrick DM, Chaturvedi AK, Shiels MS, Divi RL, Filipski KK, Hebert EF, et al. Using Immune Marker Panels to Evaluate the Role of Inflammation in Cancer: Summary of an NCI-sponsored Workshop. Cancer Epidemiol Biomarkers Prev. 2015. PubMed Central PMCID: PMCPMC4690808.
  30. 30. ProteinSimple. Ella [21.September.2016]. Available from: http://www.proteinsimple.com/ella.html.
  31. 31. Aldo P, Marusov G, Svancara D, David J, Mor G. Simple PlexTM: A Novel Multi-Analyte, Automated Microfluidic Immunoassay Platform for the Detection of Human and Mouse Cytokines and Chemokines. Am J Reprod Immunol. 2016;75(6):678–93. PubMed Central PMCID: PMCPMC5084752. pmid:27170460
  32. 32. Cao J, Seegmiller J, Hanson NQ, Zaun C, Li D. A microfluidic multiplex proteomic immunoassay device for translational research. Clin Proteomics. 2015;12:28. PubMed Central PMCID: PMCPMC4676148. pmid:26692826
  33. 33. Conroy AL, Hawkes M, Hayford K, Namasopo S, Opoka RO, John CC, et al. Prospective validation of pediatric disease severity scores to predict mortality in Ugandan children presenting with malaria and non-malaria febrile illness. Crit Care. 2015;19:47. PubMed Central PMCID: PMCPMC4339236. pmid:25879892
  34. 34. R&DSystems. Magnetic Luminex® Assay Human Premixed Multi-Analyte Kit [3.October.2016]. Available from: https://resources.rndsystems.com/pdfs/datasheets/lxsahm.pdf.
  35. 35. Reed GF, Lynn F, Meade BD. Use of coefficient of variation in assessing variability of quantitative assays. Clin Diagn Lab Immunol. 2002;9(6):1235–9. PubMed Central PMCID: PMCPMC130103. pmid:12414755
  36. 36. Zaki R, Bulgiba A, Ismail R, Ismail NA. Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review. PloS one. 2012;7(5):e37908. PubMed Central PMCID: PMCPMC3360667. pmid:22662248
  37. 37. Luiz RR, Szklo M. More than one statistical strategy to assess agreement of quantitative measurements may usefully be reported. J Clin Epidemiol. 2005;58(3):215–6. pmid:15718108
  38. 38. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10. pmid:2868172
  39. 39. Pitman EJG. A note on normal correlation. Biometrika. 1939;31(1–2):9–12.
  40. 40. Ellington AA, Kullo IJ, Bailey KR, Klee GG. Antibody-based protein multiplex platforms: technical and operational challenges. Clin Chem. 2010;56(2):186–93. PubMed Central PMCID: PMCPMC2901849. pmid:19959625
  41. 41. FDA. Guidance for Industry Bioanalytical Method Validation 2013. Available from: http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm368107.pdf.
  42. 42. Higgins SJ, Purcell LA, Silver KL, Tran V, Crowley V, Hawkes M, et al. Dysregulation of angiopoietin-1 plays a mechanistic role in the pathogenesis of cerebral malaria. Science translational medicine. 2016;8(358):358ra128. pmid:27683553
  43. 43. van der Heijden M, van Nieuw Amerongen GP, Chedamni S, van Hinsbergh VW, Johan Groeneveld AB. The angiopoietin-Tie2 system as a therapeutic target in sepsis and acute lung injury. Expert Opin Ther Targets. 2009;13(1):39–53. pmid:19063705
  44. 44. Tsao PN, Chan FT, Wei SC, Hsieh WS, Chou HC, Su YN, et al. Soluble vascular endothelial growth factor receptor-1 protects mice in sepsis. Crit Care Med. 2007;35(8):1955–60. pmid:17568329