https://orcid.org/0000-0001-7800-7614
Figures
Abstract
Introduction
Commercial pulse oximeters may not be well calibrated for oxygen saturations below 70%, conditions that may be met in high altitude aviation and mountaineering. We therefore examined the bias and variability of heart rate (HR) and blood oxygen saturation (SpO2) of four different pulse oximeters (PO) at arterial blood oxygen saturation (SaO2) between 55 and 100%.
Methods
Seventeen healthy participants (age 33 ± 11 (mean ± standard deviation (SD)) yr) were exposed to controlled desaturation at rest by stepwise reduction of the oxygen fraction in the breathing air between 20.9 and 8%. Parallel measurements of HR (n = 383) and blood oxygen saturation (n = 304) from four pulse oximeters (RAD-97, PM100N, M7500 and Nell1-SR) and from reference instruments (by hemoximetry and electrocardiography (ECG)) were taken during the exposure. The validity was assessed in intervals of 55–70%, 70–85% and 85–100% SaO2 using the Bland-Altman method (bias and 95% limits of agreement (LoA)) and the root mean square error for variability. The demarcation criteria for agreement between methods were ±6 percentage points O2Hb saturation and ±6 bpm HR.
Results
At the 85–100% SaO2 interval, all POs but the Nell1-SR were in agreement with the reference for O2 saturation. Only M7500 agreed with the reference for the 70–85% interval and none of the POs were in agreement with the reference for the 55–70% interval. The pulse oximeters and ECG were not in agreement for HR at neither the 55–70% nor the 85–100% interval but agreed at the 70–85% interval except the RAD-97. All pulse oximeters increased the bias or the variability for SpO2 significantly by reductions in oxygen saturation, while no systematic differences were found for HR.
Citation: Vikne H, Kjeserud JA, Westgaard W, Baalsrud Westlie R, Kåsin JI, Medbø JI, et al. (2025) Validity of pulse oximetry measures for heart rate and oxygen saturation during profound hypoxia in normobaric simulated extreme altitudes. PLoS One 20(6): e0326674. https://doi.org/10.1371/journal.pone.0326674
Editor: Agnese Sbrollini, Polytechnic University of Marche: Universita Politecnica delle Marche, ITALY
Received: July 10, 2024; Accepted: June 3, 2025; Published: June 23, 2025
Copyright: © 2025 Vikne et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data and materials necessary to reproduce the findings reported in this manuscript are available at Open Science Framework: https://osf.io/fmdk8/.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Oxygen is vital for the normal physiological functions of the human body, including cellular respiration and energy production. Generalized acute hypoxia – reduced oxygen availability at the whole-body level – is therefore a health- and incapacitation risk factor when otherwise healthy persons ascend in the atmosphere such as during mountaineering or aviation and this is known as altitude or hypobaric hypoxia. Severe acute hypoxia will generally lead to a temporary decrease in cognitive speed and function [1,2], may lead to loss of consciousness [3] and origin life-threatening situations. The atmospheric O2 pressure decreases almost exponentially by increasing altitude [4], and without supplemental oxygen, hypobaric hypoxia in healthy individuals is a normal consequence of the reduced environmental oxygen pressure and availability at increasing altitude [5]. In addition to symptoms, generalized hypoxia may be revealed by reductions in the arterial blood oxygen saturation – that is the relative amount of oxyhemoglobin to unbound hemoglobin, using direct measurement in arterial blood by hemoximetry (SaO2). Although hemoximetry yields highly precise measurements of an individual’s hematological status, the procedure necessitates arterial puncture, rendering it invasive and painful, with associated risks of tissue damage and infection [6]. Moreover, the requisite sampling procedures and equipment leaving continuous monitoring of SaO2 challenging and less practical outside specialized facilities.
A non-invasive alternative to the measurement of SaO2, pulse oximetry, was invented early in the 1970s [7] and is a photoplethysmographic (PPG) method for indirect, continuous measurement of arterial blood oxygen saturation (SpO2) and heart rate (HR) through the repetitive arterial blood pulses [8]. Pulse oximetry uses light of different wavelengths to illuminate blood as a method to indirectly estimate the percentage of oxygenated versus deoxygenated hemoglobin in real-time. Oxygen saturation is estimated by measuring the change in the ratio of red to infrared light absorbed by oxyhemoglobin and deoxygenated hemoglobin and received by a photodetector throughout a blood pulse, based on the assumption that the venous saturation component does not change. The ratio of absorbed red to infrared light is compared with an internal standard curve for oxygen saturation, based on empirical data of the specific pulse oximeter manufacturer [9,10]. Pulse oximetry has gained a wide range of uses from hospital care to altitude, exercise and sports science [11–14] due to its non-invasive nature, ease of use, and reasonably robust measurements. Manufacturers of pulse oximeters continuously improve the hardware, algorithms and performance of their instruments that are validated according to ISO standards for medical approvement of the manufacturer or a specialized test laboratory. The measures of SpO2 and HR of the pulse oximeters and sensors are validated at an SaO2 interval between 70 and 100% by using hemoximetry and electrocardiography (ECG) as reference standards, respectively. The accuracy and precision are assessed by use of Bland & Altman’s bias (mean difference between the reference and the pulse oximeter) and 95% limits of Agreement (1.96 times the SD of the bias) and the root mean square error of the difference (RMSE). Typically, medically approved pulse oximeters and sensors report an overall RMSE of ≈ 2–3 percentage points (pp) or a corresponding 95% limit of agreement (LoA) of 4–6% across the 70–100% saturation interval when compared with a reference [15–17]. There are, however, only few independent comparative examinations of the performance of pulse oximeters [18].
Pulse oximetry is generally subject to various threats to validity [9,10], including skin color [19], use of nail polish [20], motion [18,21], perfusion at the assessment site [21,22], probe types and sites of measurement [23,24], and fractions of the dyshemoglobins COHb and MetHb [25–27]. All these factors can affect the accuracy and precision of pulse oximeter performance. Differences between the pulse oximeter manufacturers in hardware, software and algorithms may also introduce variation in measurement errors. Previous examinations have revealed that the uncertainty of measurement of pulse oximeters generally increases by decreasing oxygen saturation [18,24,28,29]. While the accuracy and precision of pulse oximeters typically are high in humans (i.e., display minor bias, LoAs, and RMSE values) at high saturation intervals (90–100% SaO2), the variability is increased in the intervals of 70–80% saturation [30]. This increased variability by desaturation seems to be a general finding for pulse oximetry and is also a challenge in other mammals as well [31,32]. In several situations humans may experience Hb oxygen saturation below the limits used for medical approval of pulse oximeters. For example, may persons with severe obstructive sleep apnea drop towards the 70% O2 saturation validation limits at a moderate altitude of 2600 m [33]. During exposures in environmental conditions that may be physiologically challenging, such as during extreme altitude mountaineering and during acute and prolonged simulated altitudes, healthy individuals may experience arterial oxygen saturations towards and well below 70% [34–37]. At acute decompression to about 7600 m (25 000 ft) of simulated altitude in hypobaric chambers during hypoxia recognition training (HRT), some participants may experience profound hypoxia within 3–4 min, and the concurrent arterial oxygen saturation may even fall below 60% SpO2 [34,38]. There is consequently a gap between the range of measurement used in standard experiments for medical approbation and those that may be achieved during exposures to decreased atmospheric pressure. At saturation levels below 70%, the hemoglobin–oxygen dissociation curve is at its steepest [39]. If measurement variability increases even further below 70% saturation, there is a significant risk of failing to detect extreme desaturation, potentially leading to severe health deterioration and incapacitation.
The overall purpose of this study was to investigate and compare the validity of four medically approved commercial finger pulse oximeters on oxygen saturation and heart rate between 55 and 70% SaO2 using reference measurement of hemoximetry and electrocardiogram in healthy participants. Here we provide empirical data on the validity of four pulse oximeters at arterial oxygen saturation less than 70%, and the results of the study may therefore be useful for researchers, healthcare professionals in the treatment of people with very low oxygen saturation, mountaineers and in hypoxia recognition training of aircrews.
Methods
Approach to the problem
To examine the validity of four different pulse oximeters at severe hypoxic conditions, 17 healthy participants were stepwise desaturated using reduced fractions of inspired O2 towards 55% O2 saturation, and parallel measurements of pulse oximetry and reference instruments (hemoximeter and electrocardiography) were taken. Four finger pulse oximeters were examined simultaneously on each participant. The participants were exposed to normobaric hypoxia using breathing air mixtures containing reduced oxygen concentration supplied by a gas blender. Five to seven different air mixtures were given in a stepwise manner to introduce stable oxygen saturation plateaus between 55 and 100%. At each oxygen saturation plateau, 3–5 simultaneous measurements from the reference apparatuses and the pulse oximeters were taken, separated by ~30 s. To examine and compare the validity of the pulse oximeters below and above the 70% saturation demarcation level, the data were partitioned in three equally wide saturation intervals of 15 percentage points (pp), consisting of one interval below 70% (55–70%) and two intervals above (70.1–85% and 85.1–100%). The validity was examined within each saturation interval using the Bland & Altman’s limits of agreement method [40] adjusted for repeated measurements [41,42], and further by the root mean square error (RMSE) and then compared to the others. See Fig 1 for overview of the study. The study followed the ethical guidelines set by the Declaration of Helsinki, and pre-approval was given by the Regional Committee for Medical and Health Research Ethics, region South-East A (ID NO. 229473). The General Data Protection Regulation (GDPR) was followed, and the local Data Protection Authority was consulted a priori. All volunteers gave their informed written consent prospectively. The study protocol was a priory registered at the Open Science Framework Registries (Registration DOI: https://doi.org/10.17605/OSF.IO/UDX7G).
Participants: Recruitment, health examination and anthropometric information
Nineteen healthy adult women (n = 8) and men (n = 11) from the greater Oslo area volunteered to participate after promoting the study through flyers and information meetings at local aero clubs and universities. The data collection period lasted from the 25th of November 2022 to the 10th of March 2023. All participants received oral and written information about the study, about possible risks and disadvantages in participation. Potential participants were initially examined by a medical doctor, and descriptive variables were measured. Persons suffering from dizziness or fainting tendencies, anemia, circulatory or respiratory disorders or from kidney or liver disease, were excluded. Persons using anticoagulants, being allergic to lidocaine, prilocaine or other similar local anesthetics, had bleeding or coagulation disorders, were pregnant or breastfeeding, obese (BMI > 29.9 kg/m2), or using Continuous Positive Air Pressure (CPAP), feared injections (belonephobia/hemophobia/algophobia) or were smoking, were also excluded. The participants had to display normal levels of hemoglobin (men: > 13.4 g/100 ml, women > 11.7 g/100 ml), carboxyhemoglobin (COHb, < 3%), methemoglobin (MetHb, ≤ 1.1%) and speak Norwegian fluently. Two participants were excluded due to low Hb concentration or unsuccessful arterial line insertion. A total of 17 participants (7 women, 10 men; age range 20–54 years) were thus accepted as subjects, and all completed the exposure intervention. The anthropometric data are reported in Table 1. The sample size was calculated a priori in SciStat using the Lu et al. [43] method for Bland-Altman analyzes. Type I and type II errors were set to 5% and 20%, respectively. All input values were taken from the reported data of the reference monitor at the lowest oxygen saturation interval 68.5–80% (Table 2) in the recent study of Louie et al. 2018 [18]. The expected mean difference between the measurement methods (bias) was 0.9 pp (percentage points), and the expected standard deviation (SD) of the difference was 2.1 pp. The maximum allowable difference between the methods was set to 5.3 pp, slightly higher than the LoA, including the 95% CI reported by Louie et al. [18], to account for the fact that the planned saturation would be lower than that in the study of Louie et al. [18]. This resulted in a minimum sample size of 96 when taking into account a dropout rate of 25%. We thus needed about 20 participants with five observations per participant to be able to determine whether the measurements of the pulse oximeter and the reference methods are similar.
All potential participants were examined prior to the exposure by a medical doctor, who also was present during the intervention. Pulmonary function was assessed by measuring forced vital capacity (FVC, l), forced expiratory flow first second (FEV1, l) and peak expiratory flow (PEF, L s-1) (Spirare 3 spirometry, Diagnostica AS, Norway). The best of three consecutive measurements was used further. Body composition (absolute (kg) and relative (%) fat and muscle mass and fat-free mass (kg)) was measured once by the impedance method (InBody 720, InBody USA, CA, USA). To estimate body compositions, we used the internal software standard algorithms and the standardized procedures recommended by the manufacturer.
Resting 12-lead electrocardiography was measured once in the supine position (Spirare 3 ECG, Diagnostica AS, Norway). Systolic (SP) and diastolic (DP) blood pressure (mmHg) was measured thrice in the sitting position (M7 Intelli IT, Omron Healthcare Co., Japan), and the mean of the measurements was used. Mean arterial pressure (MAP) was calculated as MAP = DP + 1/3(SP – DP). Self-reported physical activity behavior (MET-minutes/week) was measured using the Norwegian version of the International Physical Activity Questionnaire, short form (IPAQ SF) [44] conducted as an interview. Hematological variables (total hemoglobin (g/100 ml), hematocrit (%), O2Hb (%), SaO2 (%), COHb (%), HHB (%), MetHb (%), pH, pCO2 (kPa), pO2 (kPa), p50 (kPa) and HCO3– (mmol/L)) were measured using ABL820 FLEX (Radiometer Medical ApS, Denmark)), and the mean of three samples were used further. Resting hematological data are displayed in Table 2 and hemodynamics and pulmonary function in Table 3.
Instrumentation
Pulse oximeter monitors and sensors.
Three factory new standalone, commercially available pulse oximeter monitors from three different manufacturers, RAD-97 (Masimo Corp., CA, USA), PM100N (Nellcor, Medtronic, MN, USA), Model 7500 (Nonin Medical, Inc., MN, USA) and in addition a Nell1-SR printed circuit board assembly (Nellcor, Medtronic, MN, US) within a gas mixer (ROBD2, see description below) were examined. The three finger transmittance sensors used by their respective standalone pulse oximeters were of a soft holster type (Masimo LNCS-DBI, Nellcor Flexmax and Nonin 8000SM), and the sensor for the Nell1-SR was a Nellcor Oximax compatible pinch type (S410-70P0, Cables & Sensors, FL, USA), which was the standard sensor setup for the ROBD2 gas blender. All sensors were new and obtained to the present study. The four finger sensors were randomly assigned to the second and fourth fingers of both hands for each participant using the RAND() function in Excel. The skin at the measuring areas was cleansed with 70% isopropanol wipes, and any nail polish was removed. The monitor sample averaging time intervals were adjusted to comparable levels for all pulse oximeters (RAD-97 (2–4 s averaging and FastSat algorithm weighing based on signal quality), PM100N (2–4 s), Model 7500 (PureSat averaging, using 4 beat exponential moving averaging 3 s) and the Nell1-SR (2–3 s). All pulse oximeters had a resolution of 1 pp for SpO2 in the measurement range 1–100% and further 1 beat per minute (bpm) for heart rate (Model 7500 (in the range 18–200 bpm), PM100N (20–250 bpm), RAD-97 (0–240 bpm). The updating frequency was reported to be 1 Hz for RAD-97, PM100N and Model 7500. For Nell1-SR, it was measured to 1 Hz using LabView software (National Instruments, TX, USA). Specified for the adult population and during non-motion conditions, all pulse oximeter monitor manufacturers reported the measurement accuracy of SpO2 to be 2% RMSE in the 70–100% measurement range. For HR the reported accuracy was 3 bpm RMSE for all monitors specified for the 25–240 bpm measurement range for RAD-97, 20–250 bpm for PM100N, 18–300 bpm for Model 7500 and 20–250 bpm for Nell1-SR. It is noted by the manufacturers that accuracy also varies by sensor type. Data from the pulse oximeters were sampled continuously during the intervention and transferred to a standalone PC for synchronization using LabView (see below for details).
Reference standards.
Arterial blood gas measurement by hemoximetry (ABL820 FLEX, Radiometer Medical ApS, Denmark) was used as the reference measure of functional arterial oxygen saturation (SaO2 (O2Hb/[Hb + O2Hb])). The hemoximeter was automatically calibrated, and quality control measurements were performed before and after experiments according to the manufacturer’s instructions. The hemoximeter had a measurement resolution of 0.1% pp for SaO2. An indwelling, 20-gauge artery catheter was placed in the radial artery of each participant after local anesthesia injections (Xylocain, 20 mg/ml) given in the skin to limit discomfort with the procedure. We mainly used the left hand of the participants, but for a few participants the right hand was used. Three to five blood samples (1.5 ml each) were taken at baseline (20.9% O2) and per reduced oxygen content gas (n = 4–7) for a total of 15–21 samples per participant. For most participants the blood sampling took about 3–5 s to complete, but for some participants the sampling took up to 15 s to obtain the necessary volume. The volume blood sampled was mixed thoroughly before analysis to ensure uniformity. Since blood collection takes a few seconds, the measured values are considered to represent an average over the sampling period. A 12-lead electrocardiography (CardioLaptop AT-110, Schiller AG, Switzerland) was used as a reference measure of heart rate (HR) and for health monitoring of the participants during the intervention. The electrodes were placed according to the manufacturer’s recommendations during the health examination and used further during the intervention. The ECG was continuously sampled at 1000 Hz, using a 50 Hz adaptive line frequency filter to suppress line interference. The sampled data were simultaneously transferred to a standalone PC at 20 Hz using LabView with a resolution of 1 bpm for synchronization. Beat for beat mean heart rate was subsequently calculated in windows of 10 beats.
Acquisition protocol
The intervention was completed in a laboratory (77 meters above sea level) without direct sunlight and with the participant sitting reclined (about 45°) in a dental chair with the forearms supported on armrests. The testing was completed at approximately the same time (starting between 08.00 and 09.00h) and the same room temperature (mean 24.2 °C (± 0.8 °C, SD)) for all participants. Participants wore regular, everyday clothing. If the participants’s fingers were cold, the hands were warmed by manual massaging, and knitted mittens were used. The mean atmospheric pressure was 101.5 ± 1.3 kPa. Hypoxia was introduced by decreasing the fraction of oxygen in the breathing air using a gas blender (Reduced Oxygen Breathing Device 2 (ROBD2, model 6202−1, Environics Inc, CT, USA). Two thermal mass controllers blended normobaric air and nitrogen (N2) in volumes resulting in gases containing reduced fractions of oxygen [45]. The ROBD2 is reported to deliver valid fractions of oxygen (O2) in the range between 5 and 21% of the total breathing gas content [46]. During the experiment the air flow rate was 50 L min–1 and delivered using a standard aircraft breathing mask (MBU-20/P, Gentex Corp, PA, USA) that sealed the mouth and nose from atmospheric air and was individually adjusted for each participant. In addition to data from the internal pulse oximeter, the ROBD2 reported the time, oxygen concentration in the breathing air and equivalent height.
A flowmeter (SFM3300, Sensiron AG, Switzerland) was connected in series with the air hose between the ROBD2 and the breathing mask to measure instantaneous airflow, with an update frequency of 2000 Hz. The instantaneous flow (L s–1) was then converted to tidal volume (TV, L), respiration rate (RR, breaths min–1) and pulmonary minute ventilation (L min–1). To do this, the volume and time data from four consecutive inspirations prior to the time marker were pooled, and from that, the relevant parameters were derived. Pulmonary minute ventilation was also normalized to fat-free mass (ml kg–1min–1). The participants were exposed to breathing gases with reduced fractions of inspired oxygen (FiO2) to introduce stable oxygen saturations in the participants in the following three target intervals: 55–70%, 70.1–85% and 85.1–100% SaO2. Since there can be considerable inter-individual variation in oxygen saturation response at a given oxygen fraction [47], a partly pragmatic exposure protocol was designed to account for this variability. The protocol consisted of eight different gas blends (20.9, 12, 11, 10, 9.5, 9, 8.75/8.5, and 8% FiO₂; see Table 4). All participants first inhaled 20.9% oxygen for approximately 10 min, followed by 12% and 9.5% FiO₂ for 3–5 min each, before being reoxygenated with 20.9% FiO₂ for 5–10 min. The subsequent FiO₂ levels were adjusted based on the participant’s oxygen saturation response at the 9.5% FiO₂ step. If SpO₂ remained relatively high (~70–75%), exposure continued with 10% and then 8.5% FiO₂. If SpO₂ was relatively low (~60–65%), exposure proceeded with 11% and then 9% FiO₂. In some cases, the exposures to the intermediate step of 10% FiO2 step were excluded. Finally, if the saturation remained above 65% at the last step, a further reduction to 9% or 8% FiO₂ was introduced. Thus, most participants were exposed to 5 (n = 8) or 6 (n = 7) combinations of breathing gases before reaching the end target saturation interval, while two participants were exposed to 7 combinations. The participants were asked to rest and minimize bodily movement during the intervention. Further to divert the attention from the ongoing measurements the participants watched a nature program on a separate monitor during the intervention. After the final exposure level, the participants were first given 100% oxygen for 5 min and then breathed normal room air without a mask during observation. The examination was also ended if the participant either wanted to abort or if the oxygen saturation fell below 55%. See Table 4 for an overview of the exposure and Fig 2 for the SpO2 and HR responses to a typical run.
The figure displays a typical run from one participant in the study chronologically exposed to FiO2s of 20.9, 12, 9.5, 20.9, 11, 9, 20.9, 10 and 8.5%. The X-axis is time (min), left Y-axis heart rate (BPM) and O2Hb saturation (SpO2, %); right Y-axis, fraction of inspired oxygen (%). Data are readings from one pulse-oximeter (SpO2, red line; heart rate, black line) and the FiO2 (blue line).
Data synchronization and missing data
A time marker (TM) was used to synchronize the data and was activated at the start of each blood sampling. Except for the point measurements for the reference SaO2 taken from arterial blood sampling, the data from pulse oximeters, ECG, flowmeter and the ROBD2 were sampled continuously and transferred to a separate, standalone PC (Dell, Optiplex 790) using LabView software. These data were subsequently synchronized with the time of the blood samples using the time markers. When stable values of the oxygen saturation were observed, defined as a variation of ≤ 3 pp [48], three to five separate blood samples per breathing gas level were taken successively with 20–30 s interval. In total, 15–21 repeated blood samples from each participant and a total of 304 blood samples were taken. At the TM for each blood sample the simultaneous measured SpO2 at that time point for each of the separate pulse oximeters were used in the further analysis. No time averaging was performed for either SpO2 or HR. The heart rate measurements used in the examination of validity were taken respectively at 15, 30, 45 and 60 s prior to the first blood sampling/TM at each separate breathing gas step to reduce any impact of the blood sampling procedures. As the participants were exposed to five to seven breathing gases, a total of 20–28 heart rate measurements per participant were taken. In total for all participants, 383 heart rate samples were recorded. For the description of heart rate response to the exposure of air with different oxygen contents, the heart rate synchronized with each blood sample was used. The collected data was reported at 20 Hz and transferred to and further treated in Excel.
Periodically, the ROBD2 software paused the transfer of data to LabView, seemingly at random, and we were not able to locate the origin of this error. Thus, some datapoints from the ROBD2 including the built-in pulse oximeter (Nell1-SR) were missing at random. Because of the minor loss of data relative to the total sampling time (ranging some 80–100 s of data of a total of about 3500 s), the number of missing observations from the ROBD2 at a given time marker was less than one per participant and reaching a total of 13 of 304 observations. All missing data were within ± 2 s of the nearest valid observation. Therefore, this value was imputed as a datapoint for blood oxygen saturation for the ROBD2. Since the data were taken under stable saturation and heart rate conditions, we expect this to have negligeable impact on the results. For the measurements of heart rate in the case of missing data from the ROBD, the measurement time-points were slightly shifted from the planned intervals to the time point of the first valid observation from ROBD2 for all pulse oximeters. As ECG was measured continuously, all heart rate data were therefore always kept synchronized. Because of a temporary connection fault, the ventilation data of one participant are missing. The descriptive resting data were collected and treated as described above. For the data describing the heart rate, hematological and ventilatory responses to the different exposure levels (Table 5), the mean of the 3–5 samples per individual per gas mix were used. All data are collected during the examination, and no data were obtained from sources other than those described.
Analysis and statistics
The following measures were used as descriptive variables of the participants in the study: age, sex, body height, body weight, body mass index, absolute and relative fat and muscle mass, forced vital capacity (FVC), forced expiratory volume in 1 s (FEV1), resting blood pressure, oxygen and carbon dioxide pressure (pO2 and pCO2), pH, hemoglobin (Hb), hematocrit (Hct), bicarbonate (HCO3–), oxyhemoglobin (O2Hb), deoxyhemoglobin (HHb), carboxyhemoglobin (COHb), methemoglobin (MetHb) and self-reported physical activity. For dichotomous variables, the number and relative values are reported. To statistically examine changes in hemoximetry, heart rate and ventilatory response to the reduced O2 exposures and to examine differences in response between the sexes, we used a linear mixed between-within model using the dependent variables and sex as factors. Multiple comparisons were Bonferroni-adjusted. If Mauchly’s test of sphericity was violated, the Greenhouse-Geisser correction was used to adjust for lack of sphericity. The data distributions were evaluated by graphical representations. As uneven number of participants were exposed to the different oxygen concentrations, 14 complete pairs at FiO2 levels 20.9, 12, 11, 9.5 and 9% were analyzed for hemoximetry and heart rate and 13 pairs for the ventilation data. The mean of the three to five measurements per participant at each FiO2 level was used as data points in the analyses, thus one participant was represented by one data point at each FiO2. We found no exposure times sex interactions, and the data were thus pooled for men and women.
To examine the validity of the pulse oximeters at different saturation levels, Bland & Altman’s bias and 95% limits of agreement (LoA), adjusted for repeated measurements [41,42] with exact confidence intervals [41] were computed and reported according to suggested standards [49]. Bias was taken as the value of the pulse oximeter minus the reference value (SaO2 or HR), and 95% LoA were taken as the standard deviation of the bias times 1.96 as described by Bland and Altman [40]. To limit the degree of desaturation below 70% SaO2, the exposures to reduced FiO2 were terminated at a saturation of 55% SaO2. This data interval below 70% (15 percentage points wide) was then examined and compared with the two partitioned, equally large intervals above 70% oxygen saturation (70.1–85% and 85.1–100%). The number of participants and observations in the three categories were as follows: 85.1–100%, 17 participants, 100 observations; 70.1–85%, 17 part., 102 obs.; 55–70%, 16 part., 102 obs. Since the measurements of heart rate were taken prior to the blood samples, we therefore used the mean SpO2 across the four examined pulse oximeters as indicator for oxygen saturation ranges when partitioning the heat rate data in the three saturation intervals. In the saturation interval 85.1–100% there were 133 observations from 17 participants; 70.1–85%, 147 observations from 16 participants, and in interval 55–70% there were 103 observations from 15 participants. The overall difference (± SD) in oxygen saturation between the mean value of all pulse oximeters compared with that of SaO2 was 0.8 pp (2.1), n = 304. Since the O2 saturation and HR are in principle changing due to the exposure and breathing pattern, immediate replicates were not taken, and thus repeatability of measures was not analyzed in this study. The data were consequently analyzed using the “Method where the true value varies” [42]. The root mean square error (RMSE) has been used as a compound indicator of both accuracy and precision for SpO2 and HR of pulse oximeter manufacturers and regulatory authorities [15–17]. We therefore calculated the RMSE (square root of the mean of the squared difference between the pulse oximeter and reference) values of the three saturation intervals (55–70%, 70.1–85% and 85.1–100%) using the equation, , where SpO2 is the oxygen saturation values of the pulse oximeter, and SaO2 is that of the hemoximeter. For heart rate, the values were beats per minute determined by the pulse oximeter and ECG, respectively.
As inference criteria for the Bland-Altman analyses, the maximum allowable difference in oxygen saturation and heart rate between the pulse oximeter and the reference methods was set to ± 6 pp and ± 6 bpm respectively. Those values were based upon the RMSE values reported by the pulse oximeter manufacturers for the interval 70–100% SaO2 (3%, [15–17]) and multiplied by 1.96, which corresponds to a 95% LoA of ≈ 6 pp and bpm. The two measurement methods (pulse oximetry and reference measurements) are considered to be in agreement if the higher and lower Limit of Agreement including the 95% CI for these limits, is respectively lower and higher than the maximum allowable difference [50].
To statistically examine the effect of reduced O2Hb saturation on the bias and RMSE of O2Hb saturation and HR within a pulse oximeter, the SpO2 and HR for the three saturation intervals were statistically compared using one-way repeated measures ANOVA with Bonferroni post hoc test adjustment for multiple comparisons. If Mauchly’s test of sphericity was violated, the Greenhouse-Geisser correction was used. For these comparisons, the mean of all observations for each participant within a saturation interval was used, ensuring that each participant was represented by a single observation per saturation interval. For these within-group analyses, there were 16 complete pairs of observations across all three saturation intervals and 15 complete pairs for HR. The data distributions were evaluated by graphical representations. Data are reported as mean ± SD, median (interquartile range) and number. P–values less than 0.05 were considered as being statistically significant. Data analysis was completed using either IBM SPSS version 29 or GraphPad Prism version 9.4.1. Adjusted Bland-Altman analyses was completed using the free web-app: https://sec.lumc.nl/method_agreement_analysis/ [41].
Results
17 healthy participants were examined, and a total of 304 oxygen saturation samples (range 15–21 per participant) and 383 heart rate samples (20–28 measurements per participant) were taken. Baseline data of the participants are given in Table 1 (anthropometrics), Table 2 (hematologic and blood gas values) and Table 3 (hemodynamics and pulmonary function).
Acute physiological response to reduced oxygen fraction
No statistically significant interactions between exposure times and sex were observed for any physiological response. The mean results of men and women have therefore been pooled (Table 5). SaO2 and dissolved O2 (pO2) fell gradually as a function of the reduced oxygen fraction in the breathing air from a mean of 97.9 (0.6)% and 12.9 (0.92) kPa at 20.9% O2, to 64.1 (8.5)% and 4.35 (0.95) kPa at 9% O2 for SaO2 and pO2 respectively. There was a large variation between subjects in the saturation response at the lower O2 concentrations; while the SD was 0.6 pp at baseline, it rose to 8.5 pp at 9% O2.
The pCO2 decreased from a baseline value of 5.02 (0.48) kPa to 4.72 (0.34) kPa at 11% O2 and further to 4.44 (0.38) kPa at 9% O2. The mean arterial blood pH was in the normal range across all exposures with a slight increase from 7.37 (0.05) at baseline to 7.41 (0.05) at the 9% FiO2 exposure. The concentration of plasma HCO3– was stable across all FiO2 exposures. P50 displayed a decrease from 3.59 (0.19) kPa at 20.9 FiO2 to 3.39 (0.12) kPa at 12% FiO2, after which no further changes were noted. The heart rate increased steadily by reductions in FiO2 from a baseline value of 67.4 (9.4)) to 92.6 (11.5) bpm at the 9% FiO2. There were no systematic changes in neither respiratory frequency nor tidal volume by changes in FiO2, but both the absolute and normalized pulmonary minute ventilation increased at the lower FiO2 exposures.
Agreement between pulse oximeters and reference measurements
O2 saturation.
The individual observations from the four different pulse oximeters versus the reference values are shown in Fig 3 for oxygen saturation. The data for Bland-Altman’s bias and LoA, variance and RMSE of oxygen saturation are given in Table 6 and Fig 4. The bias for all stand-alone pulse oximeters were close to zero for saturation levels 85–100 and 70–85%, while for the Nell1-SR PCBA the bias was some –2 and –3.8 pp, respectively. Using the demarcation limits of ± 6 pp, all pulse oximeters except for the Nell1-SR were in agreement with the reference value at oxygen saturation interval 85–100%, while only M7500 displayed LoAs ±95% CI within the demarcation criteria for the interval 70–85%. The pulse oximeters displayed RMSE values less than 3 pp and were thus within the self-reported performance limits for the 85–100% and 70–85% intervals, with one exception. The Nell1-SR displayed 4.7 pp RMSE in the 70–85% interval. In the 55–70% O2 saturation interval, the PM100N displayed less than 1 pp of bias, while the offset was –2.5, + 2.5 and –5 pp for RAD-97, M7500 and Nell1-SR, respectively. Also, none of the pulse oximeters displayed LoAs ± 95% CI within the demarcation limits and were not in agreement with the hemoximeter at the lowest levels of O2 saturations. Moreover, all pulse oximeters displayed RMSE values greater than 3 pp at this interval.
Scatter plots displaying the SpO2 of the four pulse-oximeters versus reference SaO2 of the hemoximeter (ABL820) for all measurements (n = 304). X-axes are SaO2 (%), Y-axes are SpO2 (%). A. RAD-97 vs. ABL820, B. PM100N vs. ABL820, C. Nell1-SR vs. ABL820, D. M7500 vs. ABL820. Line of identity which signifies SpO2 = SaO2 is marked in red.
Bland-Altman plots for Hb oxygen saturation at intervals SaO2 85–100% (A, D, G, J), SaO2 70–85% (B, E, H, K) and SaO2 55–70% (C, F, I, L). Data for RAD-97 are given in A, B, and C; PM100N in D, E and F; Nell1-SR in G, H and I, and M7500 are displayed in J, K and L. X-axis displays the average value of SaO2 and SpO2.Values on the Y-axis show the difference between SaO2 and SpO2 and the scaling is equal across plots. Bias (mean difference) is represented by black solid line, limits of agreement (–1.96 SD to + 1.96 SD) are given as black dashed lines, and shaded areas are the 95% CI of the bias and limits of agreement. Zero bias is indicated by dotted black lines, and the red lines signifies the demarcation criteria of 6 pp.
Heart rate.
Individual data of heart rate taken from the pulse oximeters versus the ECG reference values are shown in Fig 5. Data for bias and LoA, variance and RMSE of heart rate are given in Table 7 and Fig 6. The mean bias varied between –0.3 and –1.6 bpm for all pulse oximeters across the three saturation intervals. At saturation level 85–100% none of the pulse oximeters displayed LoAs ± 95% CI within the limit criteria (± 6 bpm), and the data were therefore not in agreement with the reference values. Similarly, all pulse oximeters displayed RMSE values larger than 3 bpm at this interval. At the 70–85% saturation, all pulse oximeters except RAD-97 were in agreement with the reference, and they similarly displayed RMSE values less than 3 bpm. At the 55–70% interval none of the pulse oximeters were in agreement with the reference. The RMSE was lower or equal to 3 bpm in the Nell1-SR and M7500 but higher than that value in the RAD-97 and PM100N.
Scatter plots displaying the heart rate of the four pulse-oximeters versus reference values from the ECG (n = 383). X-axes shows HR by ECG (BPM), Y-axes shows HR by pulse oximeter (BPM). A. RAD-97 vs. ECG, B. PM100N vs. ECG, C. Nell1-SR vs. ECG, D. M7500 vs. ECG. Line of identity signifying that HR pulse oximeter = HR ECG and is marked in red.
Bland-Altman plots for heart rate at intervals SaO2 85–100% (A, D, G, J), SaO2 70–85% (B, E, H, K) and SaO2 55–70% (C, F, I, L). Data for RAD-97 are given in A, B, and C; PM100N in D, E and F; Nell1-SR in G, H and I; and M7500 are displayed in J, K and L. X-axis displays the average value of HR from the pulse oximeter and the reference (ECG). Y-axis displays the difference in HR between the ECG and the pulse oximeter with equal scaling across plots. Bias (mean difference) is represented by black solid line, limits of agreement (–1.96 SD to + 1.96 SD) are given in black dashed lines, and shaded area are the 95% CI of the bias and limits of agreement. Zero bias is indicated by dotted black lines and the red lines signifies the demarcation criteria of 6 bpm.
Effect of desaturation on pulse oximeter performance
For three pulse oximeters the mean bias of the SpO2 at O2Hb saturation interval 55–70% was significantly different compared with the mean bias at the 70–85% and 85–100% intervals (Table 8). For RAD-97 and Nell1-SR, the offset was negatively increased as compared with the 70–85% and 85–100% interval, while for M7500 the O2Hb saturation was positively biased. The mean bias of PM100N did not change as a function of desaturation and was close to zero for all saturations examined. The variability of the difference between SpO2 and SaO2 was also affected by desaturation. The mean RMSE was significantly larger for all pulse oximeters at the 55–70% interval than at the 85–100% interval. For three of the pulse oximeters the variability of difference was also larger at the 70–85% compared with the 55–70% interval. For heart rate, neither bias nor RMSE changed as a function of desaturation for any pulse oximeter (Table 8).
Discussion
As far as we know, this is one of the first independent systematic examinations of pulse oximeter performance at oxygen saturation levels between 55 and 70% in SaO2 in healthy participants after the 2000s. We examined a total of 304 parallel samples of O2Hb saturation and 383 heart rate samples from 17 healthy female and male participants during profound experimental hypoxia. The pulse oximeters displayed a gradual decrement in SpO2 measurement performance with increases in bias and variability by decreased O2 saturation. At the 55–70% saturation interval, none of the examined pulse oximeters were in agreement with reference values for O2Hb interval using LoA of 6 percentage points (pp) ± 95% CI. Likewise, all RMSE values were above 3 pp for this interval. Desaturation had different effects on the reported heart rates. All pulse oximeters except the RAD-97 agreed with the reference ECG on the measurement of heart rate at the 70–85% saturation interval, but none agreed at the 55–70% nor the 85–100% saturation intervals. However, the observed biases in heart rate were less than 2 bpm, which we deem quite small.
Acute physiological changes during the hypoxic exposure
Despite data being sampled under stable conditions, we acknowledge that it cannot be ruled out that the physiological responses at a given FiO2 level may not have reached steady state due to the relatively brief exposure duration.
The participants displayed normal physiological hemostatic responses to the acute stepwise decreases in breathing air oxygen concentration in order to cope with the reduced oxygen availability [51]. As emphasized above, although the blood samples and heart rate were taken under stable conditions, we cannot assume that the data were recorded under truly steady state conditions for all physiological variables as the primary goal of the study was instrument validation and not physiological responses to hypoxia. During the exposure there was a gradual decrease in pO2 and SaO2 by reductions in FiO2. Although the mean pO2 fell substantially at all FiO2 exposures except baseline, there was no clear hypoxic ventilatory response (HVR) in the participants to the exposures. Neither tidal volume nor respiratory frequency increased systematically by decreased FiO2. When normalized for fat-free mass, there was a mean increase in pulmonary minute ventilation during the 9.5% FiO2 exposure only. This small increase in ventilation was followed by a small, but significant decrease in pCO2 during exposure. The baseline values for pCO2 were almost identical to that of reference values at sea level [52], so the moderate increase in ventilation could not be explained by an unusual high baseline value. It is possible that the known large variation in human HVR response [53] and onset of the hypoxic ventilatory decline [54] can have masked a systematic increase in ventilation in the present study. Also, there were minor changes in pH during the short exposures, and these were within the normal values for sea level [52]. The heart rate increased from a mean of 67 bpm at baseline to 93 bpm at the 9% FiO2, which is a normal, compensatory response to acute hypoxia [55], mainly as a function of increased sympathetic and reduced parasympathetic activity [56,57].
Bias and agreement of the pulse oximeters
For O2Hb saturation, the three standalone pulse oximeters (Masimo RAD-97, Nellcor PM100N and Nonin M7500) and the manufacturer specific finger sensors performed according to their self-reported accuracy and precision claims under no motion conditions, which generally is less than 3 pp RMSE across the intervals 70–100% (see results for details). For the partitioned intervals 70–85% and 85–100%, all standalone pulse oximeters also displayed RMSE values below 3 pp. In this study the demarcation criterion for considering the pulse oximeters to agree with the reference values were defined to ± 6 pp and ± 6 bpm for SpO2 and heart rate, respectively. The ± 6 pp or bpm corresponds to twice the RMSE performance claims of the manufacturers (< 3 pp or bpm) and which is consistent with a 95% LoA of 6 pp or bpm when the bias is zero [15–17]. For the interval 85–100% O2Hb saturation, all standalone pulse oximeters displayed biases close to zero, LoAs ± 95% CI within ± 6 pp and were therefore in agreement with the reference values. At the 70–85% O2Hb saturation interval, the biases of the three standalone pulse oximeters were also about zero, but at this interval only the Nonin M7500 was in agreement with the reference. The LoAs ± 95% CI of the RAD-97 and PM100N exceeded the 6 pp criteria. The seemingly contradictory conclusions drawn from the RMSE and LoA for the RAD-97 and the PM100N can be explained by a small increase in bias and the inclusion of the 95% CI of the LoA. While the error claims of the manufacturers are based upon the RMSE without any measure of uncertainty, the inclusion of the 95% CI for the LoA causes a widening of the error term that exceeded the 6 pp limits.
The embedded PCBA pulse oximeter Nell1-SR in the ROBD2 gas blender used for the hypoxic exposures was examined with a non-Nellcor finger sensor as it was set up by the gas blender manufacturer. This configuration demonstrated a systematic error with a bias of –2% and LoAs ± 95% CI that exceeded the ± 6 pp agreement limit criteria at the 85–100% SaO2 saturation interval. Thus, even under the most well-saturated conditions, this monitor and sensor combination was not in agreement with the reference. At the 70–85% O2Hb saturation interval, the bias decreased further to –3.8 pp. For the RMSE, the combination of the Nellcor PCBA and a non-Nellcor sensor displayed values within 3% for the 85–100% interval, but 4.7% at the 70–85% interval, resulting in a pooled RMSE of 3.8% for the 70–100% interval. The failure to operate within reported standard performance through the 70–100% interval for the Nellcor Nell1-SR may likely origin from the use of a non-Nellcor finger sensor. As the PCBA technology and algorithms within the standalone Nellcor PM100N and Nell1-SR are identical according to the manufacturer, this indicates that the increased errors origin from the use of the non-Nellcor sensor. It may be that the non-Nellcor sensor had emission spectra slightly different from that used by Nellcor and which therefore increased the measurement error [58]. Previous studies have shown a large difference in mean bias and variability across the 70–100 O2Hb saturation range between different finger sensors attached to the same pulse oximeter monitor [59], which supports this interpretation. Another factor that may have had impact on the errors was that while soft-holster type sensors was used in combination with the three standalone monitors, the non-Nellcor finger sensor was a clip-on type, and it is possible that this sensor is more subjected to noise than the holster type sensors. However, as the performance of the Nellcor PCBA was not control examined using a Nellcor holster type sensor, conclusions about possible effects of the monitor-sensor configuration cannot be drawn from this study.
Under the most severe hypoxic conditions, the 55–70% O2Hb saturation interval, none of the examined pulse oximeters were in agreement with the reference (± 6 pp), and neither displayed RMSE values less than 3 pp. Therefore, a clear and general increasing deterioration in performance by decreased saturation was seen for all four pulse oximeters examined. The mean measurement biases were significantly increased at the 55–70% interval compared with the 85–100% interval for three of the examined pulse oximeters. The RAD-97 and Nell1-SR displayed negatively increased biases of respectively –2.5 and –5 pp compared with the reference, that is an underestimation of the oxygen saturation, while the M7500 overestimated the O2 saturation by 2.5 pp. The bias of the Nellcor PM100N did not increase significantly at the 55–70% interval (0.9 pp). The mean variability as measured by the RMSE increased further for all pulse oximeters and were significantly larger than that recorded at the 85–100% saturation interval. Reduced oximeter performance by decreased SaO2 seems to be a consistent finding across saturation levels and have been shown in both controlled laboratory hypoxia studies in healthy participants [24,60] and in clinical, real-life studies in various patient and age groups [61,62]. At controlled oxygen saturations below 70% the pulse oximeter precision deteriorates even further compared with at higher saturation level [24,63–66]. A multitude of factors are known to affect the measurements of a pulse oximeter, including physical, technical, and physiological factors [9,10,30,67]. The manufacturers use specific algorithms to convert the optical signal of the pulse oximeter to an SpO2 value by comparing the signal with a reference SaO2 value based on empirical derived calibration curves [9,10,68]. As the calibration involves a set of human participants with a given set of individual characteristics, any errors that are introduced by the empirical calibration itself will therefore limit the inherent accuracy of the pulse oximeter [9,69]. During the calibration, errors cannot only arise from the pulse oximeter–participant interaction, but may also be introduced by the reference hemoximeter [30]. Because of the known difference in performance between reference hemoximeters [70], it is recommended to use two separate hemoximeters in validation studies [30]. In the present study, only one reference hemoximeter was available. A slight difference between the hemoximeter used in this study compared with the hemoximeters used for developing the original calibration curve may therefore have introduced parts of the variation in the measurements. In addition, since the empirical calibration of the pulse oximeters are completed in the 70–100% O2Hb saturation range, any estimation of saturation below this range is based on extrapolating from higher saturations [64,69]. It has therefore been advised that any readings below this limit should be taken cautiously [10,69]. The extrapolation to lower saturation values is probably a main cause of the deterioration in performance found at the 55–70% saturation interval in the present study.
Other possible sources of measurement error are related to physical differences between the participants involved in different calibration studies [9]. The differences between a study population and the calibration population in factors known to affect the optical signal by scattering, absorption or reflecting of the photons [71] can therefore have impact on the measurement error across all saturation values. In the present study we have attempted to control other factors known to affect the pulse oximetry measurements through the study inclusion criteria and the implementation of the study protocol. Possible effects of factors like anemia [72], COHb [25] and MetHb [26] were minimal as shown by direct measurements of these parameters showing normal values. The use of nail polish [20,73] was not permitted, and the fingers of all participants were washed using isopropanol before placing the sensors to minimize any external influence on the pulse oximeters light path. A possible limitation in the study is that we did not optically shield the finger sensors from each other during the experiments. It cannot be ruled out that this may have caused some interference between the sensors [74]. However, the sensors were spaced one finger apart (2nd and 4th finger), and three of the sensors were of a soft holster type that is likely more shielded from both receiving and sending interfering light. We therefore assume that a possible interference between the sensors have not been a major problem [74].
The hypoxic exposures were also completed in a “no-motion” approach to limit any effects of movement on the measurements [18,21]. The perfusion of the hands [18,22] were controlled by using standard room temperature in the laboratory, and if the subject felt cold, the hands were warmed, and knitted mittens were used during the examination. All participants were circulatory and respiratory healthy, and none were using vasoactive drugs. A well-known systematic difference in O2Hb bias exists between persons with different content in skin chromophores such as melanin [19,60]. In the present study no participants with dark skin pigmentation volunteered, leaving the group homogenous with light skin tone. The lack of variation in skin pigmentation in this study could therefore negatively affect the measurement bias if the calibration curves of the manufacturer was based on empirical data with participants of various degree of pigmentation. However, the variability within the study group may have been less by the same reason.
Pulse oximeters averaging times may have a large impact on the response-time and fluctuations as it functions as a filter of the data in dynamically varying recordings [75]. In the present study all pulse oximeters were adjusted to comparable averaging times of 2–4 s, and the samples were taken under stable saturation conditions. Thus, variations within a sampling period are not likely to have affected the results much. Also, the sampling of blood of the arterial line may take some seconds to complete, and since this blood is thoroughly mixed, the SaO2 may be regarded as a sampling time average across the sampling period. The placement of the arterial line in a. radials also minimized the time difference between the sampling of the arterial blood and the finger pulse oximeter [30]. For these reasons we believe that any differences in time of sampling had negligible impact on the results. The sensors were placed randomly on the second and forth finger of both hands. Previous studies have not found any differences in oxygen saturation between these fingers [76], and the randomization should provide a statistical control for any potential unknown systematic difference between fingers.
For heart rate the mean bias for all pulse oximeters was within ± 1.6 bpm across all saturation intervals, and it did therefor not display any clear systematic offset. However, the LoA ± 95 CI of all pulse oximeters were outside the demarcation criteria of 6 bpm at both the 85–100% and 55–70% O2Hb intervals. The readings were for that reason not in agreement with the ECG reference values. At the 70–85% interval on the other hand, three of the four pulse oximeters were in agreement with the electrocardiogram. No clear relationship between pulse oximeter performance and saturation level for the measurement of heart rate were therefor seen. Moreover, no statistical differences in bias nor variability were found between the saturation intervals. These results are similar to what others have found, i.e., pulse oximetry display little bias, but relatively large measurement variability as compared with ECG measurement of heart rate [77,78].
In the present study, we show that under controlled laboratory conditions, the performance of the examined pulse oximeters worsen by reduced oxygen saturations, in that the measurement difference range between the pulse oximeters and the reference gradually increase as saturation decrease. At the lowest saturation interval, the three standalone pulse oximeters display ± LoAs of 5–7 percentage points, indicating that for a measured pulse oximeter value displaying 55% saturation with zero bias, the true, unknown SaO2 could be some 49–61% for a given individual. In a non-laboratory situation, for example during high altitudes where there is less control of the factors known to affect the validity of the oximeters, it is expected that the differences between the measurement difference between a pulse oximeter and the reference would likely increase and thus the performance of the pulse oximeters to worsen. Indeed, large measurement differences between pulse oximetry and arterial blood gas have been reported previously during Mount Everest expedition [79].
Conclusions
For a healthy population while controlling for other factors that can affect pulse oximeter performance, the examined standalone pulse oximeters return reasonable performance for measuring O2 saturation at the 70–100% saturation interval on average. However, for a single individual, the measurements can vary by several percentage points compared to the true values. At severe arterial desaturation (<70% SaO2) none of the examined pulse oximeters were in agreement with the reference. The measurement of heart rate agreed with the electrocardiogram at the saturation interval 70–85% but not at the 55–70% or 85–100% saturation intervals.
Acknowledgments
We would like to thank the following persons that gave us valuable assistance in preliminary work and during the study: Sandra Helland, Annette Slaattun, Kamilla Skram Knudsen at the Institute of Aviation Medicine, Truls Tambs-Lyche at the Institute of Basic Medical Sciences, University of Oslo and Jostein Hallén, Department of Physical Performance, Norwegian School of Sports Sciences. Finally, we would like to thank Jarl Lyder Hansen and Anders Terland at Triolab AS, Norway, for their positive forthcomings and competent service of the hemoximeter throughout the study.
References
- 1. Wilson MH, Newman S, Imray CH. The cerebral effects of ascent to high altitudes. Lancet Neurol. 2009;8(2):175–91. pmid:19161909
- 2. McMorris T, Hale BJ, Barwood M, Costello J, Corbett J. Effect of acute hypoxia on cognition: A systematic review and meta-regression analysis. Neurosci Biobehav Rev. 2017;74(Pt A):225–32. pmid:28111267
- 3. Ottestad W, Hansen TA, Pradhan G, Stepanek J, Høiseth LØ, Kåsin JI. Acute hypoxia in a simulated high-altitude airdrop scenario due to oxygen system failure. J Appl Physiol (1985). 2017;123(6):1443–50. pmid:28839003
- 4.
ICAO. Manual of the ICAO standard atmosphere. Extended to 80 kilometres (262 500 feet). Third edition ed. Montreal, Canada: International Civil Aviation Organization; 1993.
- 5. Peacock AJ. ABC of oxygen: oxygen at high altitude. BMJ. 1998;317(7165):1063–6. pmid:9774298
- 6. Rowling SC, Fløjstrup M, Henriksen DP, Viberg B, Hallenberg C, Lindholt JS, et al. Arterial blood gas analysis: as safe as we think? A multicentre historical cohort study. ERJ Open Res. 2022;8(1):00535–2021. pmid:35237684
- 7. Aoyagi T. Pulse oximetry: its invention, theory, and future. J Anesth. 2003;17(4):259–66. pmid:14625714
- 8. Miyasaka K, Shelley K, Takahashi S, Kubota H, Ito K, Yoshiya I, et al. Tribute to Dr. Takuo Aoyagi, inventor of pulse oximetry. J Anesth. 2021;35(5):671–709. pmid:34338865
- 9. Nitzan M, Romem A, Koppel R. Pulse oximetry: fundamentals and technology update. Med Devices (Auckl). 2014;7:231–9.
- 10. Chan ED, Chan MM, Chan MM. Pulse oximetry: understanding its basic principles facilitates appreciation of its limitations. Respir Med. 2013;107(6):789–99. pmid:23490227
- 11. Ascha M, Bhattacharyya A, Ramos JA, Tonelli AR. Pulse oximetry and arterial oxygen saturation during cardiopulmonary exercise testing. Med Sci Sports Exerc. 2018;50(10):1992–7. pmid:29771822
- 12. Durand F, Raberin A. Exercise-induced hypoxemia in endurance athletes: consequences for altitude exposure. Front Sports Act Living. 2021;3:663674. pmid:33981992
- 13. Baumgartner RW, Keller S, Regard M, Bärtsch P. Flunarizine in prevention of headache, ataxia, and memory deficits during decompression to 4559 m. High Alt Med Biol. 2003;4(3):333–9. pmid:14561238
- 14. Dünnwald T, Kienast R, Niederseer D, Burtscher M. The use of pulse oximetry in the assessment of acclimatization to high altitude. Sensors (Basel). 2021;21(4):1263. pmid:33578839
- 15.
Masimo. Rad-97® Pulse CO-Oximeter. Operator’s Manual. Irvine, CA, USA: Masimo Corporation; 2020.
- 16.
Covidien. Bedside SpO2 Patient Monitoring System. Operators;s Manual. Mansfield, MA, USA: Covidien; 2018.
- 17.
Nonin. Operator’s Manual. Model 7500FO. Pulse Oximeter. North Plymouth, Minnesota, USA: Nonin Medical, Inc.; 2019.
- 18. Louie A, Feiner JR, Bickler PE, Rhodes L, Bernstein M, Lucero J. Four types of pulse oximeters accurately detect hypoxia during low perfusion and motion. Anesthesiology. 2018;128(3):520–30. pmid:29200008
- 19. Al-Halawani R, Charlton PH, Qassem M, Kyriacou PA. A review of the effect of skin pigmentation on pulse oximeter accuracy. Physiol Meas. 2023;44(5):05TR01. pmid:37172609
- 20. Aggarwal AN, Agarwal R, Dhooria S, Prasad KT, Sehgal IS, Muthu V. Impact of fingernail polish on pulse oximetry measurements: a systematic review. Respir Care. 2023;68(9):1271–80. pmid:37185113
- 21. Giuliano KK, Bilkovski RN, Beard J, Lamminmäki S. Comparative analysis of signal accuracy of three SpO2 monitors during motion and low perfusion conditions. J Clin Monit Comput. 2023;37(6):1451–61. pmid:37266709
- 22. Poorzargar K, Pham C, Ariaratnam J, Lee K, Parotto M, Englesakis M, et al. Accuracy of pulse oximeters in measuring oxygen saturation in patients with poor peripheral perfusion: a systematic review. J Clin Monit Comput. 2022;36(4):961–73. pmid:35119597
- 23. Seifi S, Khatony A, Moradi G, Abdi A, Najafi F. Accuracy of pulse oximetry in detection of oxygen saturation in patients admitted to the intensive care unit of heart surgery: comparison of finger, toe, forehead and earlobe probes. BMC Nurs. 2018;17:15. pmid:29692684
- 24. Feiner JR, Severinghaus JW, Bickler PE. Dark skin decreases the accuracy of pulse oximeters at low oxygen saturation: the effects of oximeter probe type and gender. Anesth Analg. 2007;105(6 Suppl):S18–23. pmid:18048893
- 25. Hampson NB. Pulse oximetry in severe carbon monoxide poisoning. Chest. 1998;114(4):1036–41. pmid:9792574
- 26. Feiner JR, Bickler PE, Mannheimer PD. Accuracy of methemoglobin detection by pulse CO-oximetry during hypoxia. Anesth Analg. 2010;111(1):143–8. pmid:20007731
- 27. Feiner JR, Rollins MD, Sall JW, Eilers H, Au P, Bickler PE. Accuracy of carboxyhemoglobin detection by pulse CO-oximetry during hypoxemia. Anesth Analg. 2013;117(4):847–58. pmid:23477959
- 28. Singh AK, Sahi MS, Mahawar B, Rajpurohit S. Comparative evaluation of accuracy of pulse oximeters and factors affecting their performance in a tertiary intensive care unit. J Clin Diagn Res. 2017;11(6):OC05–8. pmid:28764215
- 29. Wilson BJ, Cowan HJ, Lord JA, Zuege DJ, Zygun DA. The accuracy of pulse oximetry in emergency department patients with severe sepsis and septic shock: a retrospective cohort study. BMC Emerg Med. 2010;10:9. pmid:20444248
- 30. Batchelder PB, Raley DM. Maximizing the laboratory setting for testing devices and understanding statistical output in pulse oximetry. Anesth Analg. 2007;105(6 Suppl):S85–94. pmid:18048904
- 31. Mtetwa TK, Zeiler GE, Laubscher L, Pfitzer S, Meyer LCR. Evaluation of the reliability of pulse oximetry, at different attachment sites, to detect hypoxaemia in immobilized impala (Aepyceros melampus). Vet Anaesth Analg. 2020;47(3):323–33. pmid:32278648
- 32. Burns PM, Driessen B, Boston R, Gunther RA. Accuracy of a third (Dolphin Voyager) versus first generation pulse oximeter (Nellcor N-180) in predicting arterial oxygen saturation and pulse rate in the anesthetized dog. Vet Anaesth Analg. 2006;33(5):281–95. pmid:16916350
- 33. Bazurto Zapata MA, Dueñas Meza E, Jaramillo C, Maldonado Gomez D, Torres Duque C. Sleep apnea and oxygen saturation in adults at 2640 m above sea level. Sleep Sci. 2014;7(2):103–6.
- 34. Singh B, Cable GG, Hampson GV, Pascoe GD, Corbett M, Smith A. Hypoxia awareness training for aircrew: a comparison of two techniques. Aviat Space Environ Med. 2010;81(9):857–63. pmid:20824992
- 35. Wen D, Tu L, Wang G, Gu Z, Shi W, Liu X. Psychophysiological responses of pilots in hypoxia training at 7000 and 7500 m. Aerospace Med Human Perf. 2020;91(10):785–9.
- 36. Tannheimer M, van der Spek R, Brenner F, Lechner R, Steinacker JM, Treff G. Oxygen saturation increases over the course of the night in mountaineers at high altitude (3050-6354 m). J Travel Med. 2017;24(5).
- 37. Sutton JR, Reeves JT, Wagner PD, Groves BM, Cymerman A, Malconian MK, et al. Operation Everest II: oxygen transport during exercise at extreme simulated altitude. J Appl Physiol (1985). 1988;64(4):1309–21. pmid:3132445
- 38. Malle C, Bourrilhon C, Quinette P, Laisney M, Eustache F, Piérard C. Physiological and cognitive effects of acute normobaric hypoxia and modulations from oxygen breathing. Aerosp Med Hum Perform. 2016;87(1):3–12. pmid:26735227
- 39. Collins J-A, Rudenski A, Gibson J, Howard L, O’Driscoll R. Relating oxygen partial pressure, saturation and content: the haemoglobin-oxygen dissociation curve. Breathe (Sheff). 2015;11(3):194–201. pmid:26632351
- 40. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10. pmid:2868172
- 41. Olofsen E, Dahan A, Borsboom G, Drummond G. Improvements in the application and reporting of advanced Bland-Altman methods of comparison. J Clin Monit Comput. 2015;29(1):127–39. pmid:24806333
- 42. Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17(4):571–82. pmid:17613642
- 43. Lu MJ, Zhong WH, Liu YX, Miao HZ, Li YC, Ji MH. Sample size for assessing agreement between two methods of measurement by Bland-Altman method. Int J Biostat. 2016;12(2).
- 44. Kurtze N, Rangul V, Hustvedt B-E. Reliability and validity of the international physical activity questionnaire in the Nord-Trøndelag health study (HUNT) population of men. BMC Med Res Methodol. 2008;8:63. pmid:18844976
- 45.
Environics. ROBD2 - Reduced Oxygen Breathing Device 2. Programming and Technical Guide. Tolland, CT, USA; 2013.
- 46. Still DL, Temme LA. An independent, objective calibration check for the reduced oxygen breathing device. Aviat Space Environ Med. 2012;83(9):902–8. pmid:22946356
- 47. Näslund E, Lindberg L-G, Lund I, Näslund-Koch L, Larsson A, Frithiof R. Measuring arterial oxygen saturation from an intraosseous photoplethysmographic signal derived from the sternum. J Clin Monit Comput. 2020;34(1):55–62. pmid:30805761
- 48. Morey TE, Rice MJ, Vasilopoulos T, Dennis DM, Melker RJ. Feasibility and accuracy of nasal alar pulse oximetry. Br J Anaesth. 2014;112(6):1109–14. pmid:24736392
- 49. Abu-Arafeh A, Jordan H, Drummond G. Reporting of method comparison studies: a review of advice, an assessment of current practice, and specific suggestions for future reports. Br J Anaesth. 2016;117(5):569–75. pmid:27799171
- 50. Stöckl D, Rodríguez Cabaleiro D, Van Uytfanghe K, Thienpont LM. Interpreting method comparison studies by use of the bland-altman plot: reflecting the importance of sample size by incorporating confidence limits and predefined error limits in the graphic. Clin Chem. 2004;50(11):2216–8.
- 51. Winslow RM. The role of hemoglobin oxygen affinity in oxygen transport at high altitude. Respir Physiol Neurobiol. 2007;158(2–3):121–7. pmid:17449336
- 52. Crapo RO, Jensen RL, Hegewald M, Tashkin DP. Arterial blood gas reference values for sea level and an altitude of 1,400 meters. Am J Respir Crit Care Med. 1999;160(5 Pt 1):1525–31. pmid:10556115
- 53. Weil JV. Variation in human ventilatory control-genetic influence on the hypoxic ventilatory response. Respir Physiol Neurobiol. 2003;135(2–3):239–46. pmid:12809623
- 54. Teppema LJ, Dahan A. The ventilatory response to hypoxia in mammals: mechanisms, measurement, and analysis. Physiol Rev. 2010;90(2):675–754.
- 55. Vogel JA, Harris CW. Cardiopulmonary responses of resting man during early exposure to high altitude. J Appl Physiol. 1967;22(6):1124–8.
- 56. Joyce W, Wang T. Regulation of heart rate in vertebrates during hypoxia: A comparative overview. Acta Physiol (Oxf). 2022;234(3):e13779. pmid:34995393
- 57. Kane AD, Kothmann E, Giussani DA. Detection and response to acute systemic hypoxia. BJA Educ. 2020;20(2):58–64. pmid:33456931
- 58. Milner QJW, Mathews GR. An assessment of the accuracy of pulse oximeters. Anaesthesia. 2012;67(4):396–401. pmid:22324874
- 59. Choi B-M, Kang BJ, Yun H-Y, Jeon B, Bang J-Y, Noh G-J. Performance of the MP570T pulse oximeter in volunteers participating in the controlled desaturation study: a comparison of seven probes. Anesth Pain Med (Seoul). 2020;15(3):371–7. pmid:33329838
- 60. Leeb G, Auchus I, Law T, Bickler P, Feiner J, Hashi S, et al. The performance of 11 fingertip pulse oximeters during hypoxemia in healthy human participants with varied, quantified skin pigment. EBioMedicine. 2024;102:105051. pmid:38458110
- 61. Harris BU, Char DS, Feinstein JA, Verma A, Shiboski SC, Ramamoorthy C. Accuracy of pulse oximeters intended for hypoxemic pediatric patients. Pediatr Crit Care Med. 2016;17(4):315–20. pmid:26914626
- 62. Ruppel H, Makeneni S, Faerber JA, Lane-Fall MB, Foglia EE, O’Byrne ML, et al. Evaluating the accuracy of pulse oximetry in children according to race. JAMA Pediatr. 2023;177(5):540–3. pmid:36939727
- 63. Hannhart B, Haberer JP, Saunier C, Laxenaire MC. Accuracy and precision of fourteen pulse oximeters. Eur Respir J. 1991;4(1):115–9. pmid:2026231
- 64. Severinghaus JW, Naifeh KH. Accuracy of response of six pulse oximeters to profound hypoxia. Anesthesiology. 1987;67(4):551–8. pmid:3662082
- 65. Severinghaus JW, Naifeh KH, Koh SO. Errors in 14 pulse oximeters during profound hypoxia. J Clin Monit. 1989;5(2):72–81.
- 66. Bickler PE, Feiner JR, Severinghaus JW. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology. 2005;102(4):715–9. pmid:15791098
- 67. Tekin K, Karadogan M, Gunaydin S, Kismet K. Everything about pulse oximetry-part 1: history, principles, advantages, limitations, inaccuracies, cost analysis, the level of knowledge about pulse oximeter among clinicians, and pulse oximetry versus tissue oximetry. J Intensive Care Med. 2023;38(9):775–84. pmid:37437083
- 68. Nitzan M, Nitzan I, Arieli Y. The various oximetric techniques used for the evaluation of blood oxygenation. Sensors (Basel). 2020;20(17):4844. pmid:32867184
- 69. Sinex JE. Pulse oximetry: principles and limitations. Am J Emerg Med. 1999;17(1):59–67. pmid:9928703
- 70. Gehring H, Duembgen L, Peterlein M, Hagelberg S, Dibbelt L. Hemoximetry as the “gold standard”? Error assessment based on differences among identical blood gas analyzer devices of five manufacturers. Anesth Analg. 2007;105(6 Suppl):S24–30. pmid:18048894
- 71. Lister T, Wright PA, Chappell PH. Optical properties of human skin. J Biomed Opt. 2012;17(9):90901–1. pmid:23085902
- 72. Severinghaus JW, Koh SO. Effect of anemia on pulse oximeter accuracy at low saturation. J Clin Monit. 1990;6(2):85–8.
- 73. Hinkelbein J, Genzwuerker HV, Sogl R, Fiedler F. Effect of nail polish on oxygen saturation determined by pulse oximetry in critically ill patients. Resuscitation. 2007;72(1):82–91. pmid:17098347
- 74. Kyriacou PA. Optical crosstalk and other forms of light interference in pulse oximeter comparison studies. J Clin Monit Comput. 2023;37(6):1481–8. pmid:37610524
- 75. McClure C, Jang SY, Fairchild K. Alarms, oxygen saturations, and SpO2 averaging time in the NICU. J Neonatal Perinatal Med. 2016;9(4):357–62. pmid:27834782
- 76. Basaranoglu G, Bakan M, Umutoglu T, Zengin SU, Idin K, Salihoglu Z. Comparison of SpO2 values from different fingers of the hands. Springerplus. 2015;4:561. pmid:26543696
- 77. Opio MO, Kellett J, Kitovu Hospital Study Group. How well are pulses measured? Practice-based evidence from an observational study of acutely ill medical patients during hospital admission. Am J Med. 2017;130(7):863.e13-863.e16. pmid:28235461
- 78. Dawson JA, Saraswat A, Simionato L, Thio M, Kamlin COF, Owen LS, et al. Comparison of heart rate and oxygen saturation measurements from Masimo and Nellcor pulse oximeters in newly born term infants. Acta Paediatr. 2013;102(10):955–60. pmid:23800004
- 79. Prosperi P, Verratti V, Taverna A, Rua R, Bonan S, Rapacchiale G, et al. Ventilatory function and oxygen delivery at high altitude in the Himalayas. Respir Physiol Neurobiol. 2023;314:104086. pmid:37257573