A tolerability assessment of new respiratory protective devices developed for health care personnel: A randomized simulated clinical study

Background U.S. health care personnel (HCP) have reported that some respiratory protective devices (RPD) commonly used in health care have suboptimal tolerability. Between 2012 and 2016, the U.S. National Institute for Occupational Safety and Health, and the Veterans Health Administration collaborated with two respirator manufacturers, Company A and B, to bring new RPD with improved tolerability to the U.S. health care marketplace. The purpose of this study was to compare the tolerability of four new prototype RPD to two models commonly used in U.S. health care delivery. Methods A randomized, simulated workplace study was conducted to compare self-reported tolerability of four new prototype RPD (A1, A2, B1, and B2) worn by HCP and two N95 control respirators commonly used in U.S. health care delivery, the 1870 and 1860, manufactured by 3M Corporation. A new survey tool, the Respirator Comfort, Wearing Experience, and Function Instrument (R-COMFI), developed previously in part for the current study, was used as the primary outcome metric. With a maximum total score of 47, lower R-COMFI scores reflected better self-reported tolerability. Poisson regression analyses were used to estimate prototype relative risks compared to controls. Results Conducted between 2014 and 2015 in two inpatient care rooms at the North Florida/South Georgia Veterans Health System, among 383 participants who enrolled, 335 (87.5%) completed the study. Mean total R-COMFI scores for the 3M 1870, 3M 1860, and prototypes A1, A2, B1, and B2 were 8.26, 9.36, 5.79, 7.70, 6.09, and 5.71, respectively. Compared to the 3M 1870, total R-COMFI unadjusted relative risks (RR) and 95 percent confidence intervals (CI) were A1 (RR 0.70, CI 0.60, 0.82), A2 (RR 0.93, CI 0.82, 1.06), B1 (RR 0.74, CI 0.64, 0.85), and B2 (RR 0.69, CI 0.60, 0.80). Compared to the 3M 1860, prototype total R-COMFI unadjusted RR and 95 percent CI were A1 (RR 0.62, CI 0.53, 0.72), A2 (RR 0.82, CI 0.73, 0.93), B1 (RR 0.65, CI 0.57, 0.74), and B2 (RR 0.61, CI 0.53, 0.70). Similarly, models adjusted for demographic characteristics showed that prototypes A1, B1, and B2 significantly improved tolerability scores compared to both controls, while prototype A2 was significantly improved compared to the 3M 1860. Conclusions Compared to the 3M 1870 and 3M 1860, two RPDs commonly used in U.S. health care delivery, tolerability improved for three of four newly developed prototypes in this simulated workplace study. The R-COMFI tool, used in this study to assess tolerability, should be useful for future comparative studies of RPD.


Introduction
Health care personnel (HCP) have reported that many respiratory protective devices (RPD) used in U.S. health care delivery have suboptimal tolerability, possibly having a negative influence on the willingness of HCP to wear RPD during patient care [1][2][3][4][5][6]. To be fully effective, properly fitting RPD must be worn correctly for the duration of exposures, in accordance with Occupational Safety and Health Administration (OSHA) Respiratory Protection Standards [7]. Among the unfavorable characteristics of RPD reported by HCP have been discomfort [1,2,6,8,9], interference with occupational duties [2,6], interference with communication [2,10], heat accumulation behind the mask [1,6], facial irritation [1,6], and breathing resistance [2,6,9]. HCP have requested availability of new RPD on the U.S. market that are tailored to their specific workplace needs. Among the RPD characteristics sought by HCP for health care delivery include improved comfort, improved straps, less interference with breathing, and proper facial fit despite the presence of facial hair [2,6].
To shepherd to the U.S. marketplace new RPD that meet the needs of HCP, the National Institute for Occupational Safety and Health (NIOSH) and the Veterans Health Administration (VHA) co-led a Federal government interagency working group called the Better Respiratory Equipment using Advanced Technologies for Healthcare Employees (Project BREATHE) from 2008 to 2016. This working group previously published a list of 28 idealized and prioritized criteria to be considered for the next generation of RPD for health care delivery, including 10 criteria pertaining to comfort and tolerability: breathing resistance, facial irritation, allergenicity, facial pressure, facial heat, air exchange, moisture management, mass, odor, and prolonged tolerability [3].
In 2012, NIOSH and VHA partnered with two U.S. respirator manufacturers, Companies A and B, to facilitate the development of new RPD for HCP based on the Project BREATHE criteria, seeking improved cost-conscious models that would be sought by health care delivery organizations. In 2014, NIOSH evaluated the physiologic and subjective performance of several candidate devices in a laboratory setting [11]. After further development efforts, four prototype respirators were selected among numerous candidates for further evaluation by VHA. Between 2014 and 2015, the VHA evaluated the subjective tolerability of these four devices in a clinical workplace setting using as the primary outcome metric a new survey tool, the Respirator Comfort, Wearing, Experience, and Function Instrument (R-COMFI), developed and internally validated, in part, for Project BREATHE [12]. The purpose of the current study was to compare the tolerability of the four selected prototypes to two RPD models commonly worn in U.S. health care delivery, using the R-COMFI scores as the primary outcome metric.

Study design
Overview. Between September 8, 2014 and May 15, 2015, we conducted a randomized, simulated workplace study in which HCP performed clinically relevant activities in a fully functional inpatient hospital care room. Compared were comfort, general wearing experience, and function of four newly developed prototype RPD to two N95 respirators commonly used in U.S. health care delivery, the 1870 and 1860 N95 respirators, manufactured by the 3M Corporation (St. Paul, MN). The study was conducted at the North Florida/South Georgia Veterans Health System (Gainesville, FL).
Prior to participant enrollment, the study was approved by the University of Florida Institutional Review Board, Protocol #201300693 and the North Florida/South Georgia Veterans Health System Research and Development Committee, Protocol #201300693. Each participant signed IRB-approved VHA informed consent. Following study completion, a de-identified dataset of the results was provided to the National Personal Protective Technology Laboratory at NIOSH. Prior to conducting analyses, the study was exempted by the NIOSH Human Subjects Research Board, Protocol # 17-NPPTL-01XM.
Setting. This study was conducted in two single occupancy patient care rooms, both measuring 8'1" x 20'3/8", in a 240 bed tertiary care hospital. Performed in Room 1 ( Fig 1A) were informed consent, randomized RPD assignment, donning of respirators and other protective equipment, a five-minute acclimation period, fit testing, and two concentration activities. All other activities occurred in Room 2, a patient care room, equipped with a hospital bed, bedside table and monitor, computer, sofa, sink, and other patient-and study-related equipment ( Fig  1B).
Participants. Three hundred eighty-three participants, HCP employed by the North Florida/South Georgia Veterans Health System (Gainesville, FL) and/or Shands Hospital at the University of Florida (Gainesville, FL), were recruited (Table 1).
Inclusion criteria were (a) previously receiving medical clearance to wear a filtering facepiece respirator (FFR) commensurate with regulations issued by OSHA, and (b) passing an OSHA-accepted qualitative fit test while wearing the RPD to which the participant was randomized [7]. Participants were excluded from participation if they exhibited (a) a health condition that precluded wearing an RPD, (b) physical characteristics that may have interfered with the ability to obtain an adequate facial seal during fit testing, (c) pregnancy, or (d) any condition or issue placing the participant at undue risk of harm or interfering with data integrity, as determined by the principal investigator.

Procedures
Each eligible participant was randomized, using a random number generator, to wear one of six RPD: prototypes A1, A2; B1, B2, control 1, or control 2 ( Table 2). Each participant was permitted to wear only one respirator during one test session; to preclude habituation bias [13], participants did not engage in repeated test sessions (Fig 2).
Respirators. Four prototype RPDs developed by two private manufacturers (company A and B) in collaboration with NIOSH and VHA were utilized ( Table 2). The four concept-level prototypes met filtration test requirements of � 95% efficiency, none were equipped with an exhalation valve, and none represented final design lockdown nor the refinement that would be expected of a mass production sample. Three (A1, B1, B2) were filtering facepieces and one A filtering facepiece respirator, available in small and medium / large sizes, with a curved, horizontally positioned, oblong shaped, plastic frame that houses a central filter panel, to which is attached a chin panel and a less permeable nasal panel with adjustable aluminum nasal bar, and an adjustable single piece strap.

Company A Prototype Respirator 2 (A2)
A reusable filtering facepiece respirator/elastomeric respirator hybrid available in small, medium and large sizes, comprised of a pliable, opaque silicone facemask with a centrally located, vertically positioned, parabolic-shaped, replaceable filter and filter housing with a single piece elasticized harness.

Company B Prototype Respirator 1 (B1)
A disposable soft, cup shaped filtering facepiece respirator model available in small and standard sizes with two non-adjustable elasticized straps and a pliable metal nose bar.

Company B Prototype Respirator 2 (B2)
A disposable pliable, V-shaped, pleated filtering facepiece respirator model available in small and standard sizes with two non-adjustable elasticized straps and a pliable metal nose bar.  (A2) was an elastomeric half-mask respirator. Two N95 models commonly used in U.S. health care delivery [14], the 3M 1860 and 3M 1870, were used as controls. Fit testing. Following informed consent and intervention assignment, OSHA-accepted qualitative fit testing was conducted using Bitrex, except for one participant who was tested with saccharin due to a previous adverse reaction to Bitrex. A user seal check was performed, followed by a five minute acclimation period while wearing the respirator. Participants who failed fit testing on the first RPD to which they were initially randomized were randomized again and fit tested for a second RPD. Participants who failed both an initial and secondary fit tests were excluded from further participation (Fig 2).

Control Respirators
Simulated workplace test sessions. Participants performed simulated HCP workplace tasks designed to last approximately one hour. The research assistant played the role of a simulated patient when patient assessment activities were performed.
Test session initiation. After donning nitrile gloves (model XTRA, Kimberly-Clark, Irving, Texas), an isolation gown (model NON-27-SMS-2, Medline Industries, Northfield, IL) and safety glasses (model 11000-500, Fisher Scientific, Hampton, NH), if they were not already wearing corrective eyeglasses, participants were instructed to don their assigned RPD using manufacturers' written instructions as a guide. Following a user seal check 7 , the research assistant started timing the length of the test session.
Test session activities and tasks. Participants completed a series of clinically relevant simulated workplace activities (Table 3).
Test session completion and survey. After the research assistant recorded the time of test session completion, participants were instructed to be seated at the Table in Room 2 to complete the data collection instrument, the R-COMFI (Fig 3), that was previously developed, internally validated, and described [12]. In brief, participants answered a series of questions about discomfort, general wearing experience, and function using likert-type response options in which each response was weighted equally. Scores were obtained for the three subscales and summed for a total score. For the discomfort and general wearing experience subscales, possible responses included "none of the time" (zero points), "some of the time" (one point) or "all of the time" (two points). For the function subscale, possible responses included strongly disagree (zero points), disagree (one point), agree (two points), or strongly agree (three points). The range of possible scores was zero to 20, zero to 12, and zero to 15 for the discomfort, general wearing experience, and function subscales, respectively. The total maximum R-COMFI score was 47 in which lower scores reflected better tolerability.

Statistical design and analyses
Assuming a global significance level of 5% and power at least 80%, sample size was estimated to be 56 individuals per group (336 in total) to detect a minimum difference of one standard deviation on the total R-COMFI score between control and prototype. A six group block randomized design was used in which each participant wore only one of six respirators to avoid habituation bias and to develop statistical models that did not violate the assumption of independence between observations. Statistical analyses were performed using SAS Institute Inc., SAS 9.1.3 (Cary, NC). Mean R-COMFI subscales (discomfort, general wearing experience, and function) were summed to calculate mean total scores.
The resulting distribution of the collective individual R-COMFI scores deviated from normality (Shapiro-Wilk <0.05) and took the form of a non-negative, positively skewed, integer distribution. Using a regression analytical framework to estimate the mean difference in R-COMFI scores between the prototypes and controls, the fit of a Poisson distribution was examined in relation to the Normal distribution. Consistent with the visual appearance of the data and the results of the Shapiro-Wilk test, the Poisson distribution was found to provide better fit through Akaike's Information Criterion, Bayesian Information Criterion, and log likelihood values. Differences in R-COMFI scores were examined using univariate Poisson regression to compare the tolerability of each prototype to both controls. Multivariate Poisson regression was also used to examine differences in R-COMFI scores among the respirator types in relation to demographic characteristics, including the categorical variables gender (male, female), categorical age (�25, 26-49, �50), and job type (nurses and other healthcare assistants, primary care providers, respiratory therapists, and other) and the continuous variables, average weekly hours worked and average weekly hours of patient contact. R-COMFI scores were collected for each of the respirators and aligned with each of the prototypes and controls in the dataset using a categorical variable. This categorical variable was entered into the regression models with the controls as the reference group. This allowed for the comparison of R-COMFI scores for each of the prototypes with each of the control respirators. Differences between each of the prototypes and the each of the controls were examined individually for the total R-COMFI score, discomfort, general wearing experience, and function. Wald χ 2 p-values <0.05 were considered statistically significant. Tolerability was estimated by calculating relative risks (RR)  in which each prototype was compared to each control using total R-COMFI scores as the primary outcome.

Results
Three hundred eighty-three participants were recruited. Demographic characteristics of the participants who were eligible for randomization were similar across study arms (Table 1). Forty-four (11.5%) participants were unable to pass either of two fit tests and were excluded from further participation, including 14 (3.7%), four (1.0%), three (0.8%), seven (1.8%) and 16 (4.0%) assigned to prototype A1, A2, B1, B2, and controls respectively (Fig 2). Four participants declined secondary fit test and were excluded from further participation. Accordingly, the passing rates were 73.1% for prototype A1, 94.4% for prototype A2, 94.1% for prototype B1, 88.7% for prototype B2 and 85.4% for the combined control groups.
Three hundred thirty-five (87.5%) completed the study. Participants were mostly female (74%), aged 26 to 49 (62%), nurses and health care assistants (70%), and worked an average of 41.5 hours weekly, with an average of 32.1 hours of weekly patient contact (  Using Poisson regression to adjust for demographic characteristics, prototypes A1, B1, and B2 received improved total R-COMFI tolerability scores compared to both controls, while prototype A2 was improved compared to one of the controls, the 3M 1860 (Table 4). Adjusted and unadjusted subscale results were also similar ( Table 5); only two scores shifted from marginally significant to non-significant: prototype A1 compared to the 3M 1870 on the discomfort subscale and prototype B2 compared to the 3M 1860 on the wearing experience subscale. Considered together, the adjusted models showed that total R-COMFI scores were significantly predicted by occupation, in which respiratory therapists reported approximately 30 percent lower scores (better tolerability) than nurses; age, in which participants aged 26 to 64 reported 30 to 50 percent lower scores than those aged �25; and average weekly hours worked, in which each hour increase was associated with one percent decrease in scores. Participant gender and average weekly patient contact hours were not significant predictors of total R-COMFI scores. No study related serious adverse events were reported. One participant who experienced claustrophobia during fit testing was withdrawn.

Discussion
U.S. health care personnel have sought, and would benefit from, RPD that are better tolerated in the context of patient care and do not interfere with occupational duties [1,2,[4][5][6][8][9][10]. After working with two U.S. RPD manufacturers to develop four new prototype respirators designed to meet the needs of HCP, we evaluated the tolerability of each prototype by assessing comfort, general wearing experience, and function using a new instrument, the R-COMFI, in a randomized, simulated workplace study. Compared to the 3M 1870 and 3M 1860, two commonly used RPD in U.S. health care, three of four newly developed prototype RPD received improved total tolerability scores. The fourth prototype received improved total tolerability scores compared to the 3M 1860, but not the 3M 1870. No significant differences in total R-COMFI scores were found between the unadjusted and adjusted models.
On the discomfort subscale, all four prototypes showed improvement compared to the 3M 1860. Compared to the 3M 1870, prototypes B1 and B2 showed improvement; however, model A2 showed no improvement and prototype A1 shifted from a marginal pre-defined level of significance in the unadjusted model to non-significance in the adjusted model. While both companies were able to make substantial improvement in comfort with their FFR prototypes, the hybrid elastomeric prototype made by Company A showed little if any detectable improvement in this study, perhaps a reflection of a unique design that could be refined in the future.
Regarding the wearing experience subscale, prototypes A1 and B1 showed improvement in both unadjusted and adjusted models compared to the 3M 1870 and the 3M 1860, although A1 was marginally significant compared to the 3M 1870. The improvement in B1 may have been related, in part, to familiarity; cup shaped devices like prototype B1 are the most commonly used RPD in U.S. health care [14].
On the function subscale, prototypes A1, B1, and B2 showed improvement in both unadjusted and adjusted models compared to the 3M 1860. Compared to the 3M 1870, prototypes A1 and B2 demonstrated improvement in both models, although prototypes A2 and B1 showed no improvement. Lack of improvement with prototype A2, the hybrid elastomeric respirator, is not surprising given that elastomeric respirators, in general, interfere more with speech intelligibility than FFRs [10]. Interference with communication and occupational duties are among the most common reasons cited for lack of adherence to respiratory protection guidance [1,2,6]. Additional development efforts emphasizing improvement in this characteristic may be beneficial to worker safety [3].
While there is no widely accepted definition of clinical significance, using 25% improvement (relative risk � 0.75) in tolerability as a suitable threshold [15], prototypes A1 and B1 surpassed this level for total R-COMFI scores compared to both controls, and prototype B2 surpassed this level compared to the 3M 1860 control for the function subscale. The higher (less tolerable) scores for wearing experience and function of prototype A2, the hybrid elastomeric, primarily prevented it from surpassing this threshold.
The demographic factors significantly predictive of better tolerability included working as a respiratory therapist, higher age, and higher average weekly hours worked, and may suggest that familiarity with wearing respiratory protection is an important determinant of tolerability. For example, compared to a 2010 HCP tolerability study [1] in which participants worked in a variety of practice settings for eight hour work shifts, participants in a similar 2013 study [16] comprised of participants who worked in an intensive care unit setting for 12-hour work shifts, tended to report better RPD tolerability.
Although prototype A2, the hybrid elastomeric respirator, demonstrated minimal tolerability improvement in our study, it may still hold promise in health care settings, pending refinement. During a hypothetical influenza pandemic, several billion N95 respirators may be needed to protect HCP who serve on the front lines of patient care [17], orders of magnitude less than the number held by acute care hospitals in the U.S. [18]. To conserve N95 and ensure HCP have sufficient numbers of RPD to care for large surges of ill patients, reusable respirators may be necessary [19][20][21][22].
The R-COMFI is the first internally validated, comprehensive, and psychometrically sound survey instrument that measures comfort, wearing experience, and function among HCP wearing RPD while performing typical health care tasks. This study represents the first time the R-COMFI, developed for filtering facepiece respirators, was evaluated for external validity in a simulated workplace study. While the mean R-COMFI scores reported by participants were generally low, representing six to 25 percent of possible values on the R-COMFI measurement scales, the range of values reported by participants utilized 66 to 100 percent of the four scales. The precision of the R-COMFI may be improved by revising the 21 criteria to be more contextually sensitive, especially in settings where RPD are worn for relatively brief periods and routine health care tasks are performed. The R-COMFI validity has not yet been evaluated in settings where prolonged RPD wearing periods are necessary or in situations where more complex medical procedures are performed [23]. As a new tool, the R-COMFI should be helpful in future efforts to evaluate and develop RPD that meet the specific needs of HCP [24].
Our study has a number of limitations. Although it was sufficiently powered to detect predefined differences between the control respirators and prototypes, a small sample size in one geographic location limits the generalizability to larger, more diverse populations. Since the participants were aware they were being monitored, their duty performance may have been positively influenced, raising tolerability scores; however, we would expect such an effect to be balanced across randomized arms. [25]; although, the randomized block design should have helped balance this effect across arms. A simulated workplace setting, while faster and less expensive, does not fully reflect the many competing objectives and complexity of a functioning health care system. Similarly, a simulated clinical experience does not precisely reflect the daily occupational responsibilities of HCP who are caring for ill patients. Finally, there are additional forces that may affect adoptability of a new respirator type, such as cost and other market forces, which were not addressed by the outcomes metrics used in this study and should be considered in future studies.

Conclusion
Compared to the 3M 1870 and 1860, two RPDs commonly used in U.S. health care, tolerability improved for three of four newly developed prototypes in this simulated workplace study. The R-COMFI tool, used in this study to assess tolerability, should be useful for future comparative studies of RPD.