Figures
Abstract
Background
Settings where Emergency Medical Services (EMS) are provided to stabilize patients and transport them to locations better equipped to provide comprehensive care, “prehospital settings,” are not frequently considered when designing packaged products. Packaging design is an understudied area, potentially impacting both healthcare provider behavior and patient outcomes. Our objectives were to: 1) describe difficulties associated with packaging in prehospital settings 2) investigate the coping strategies used by paramedics when difficulties occurred, and 3) assess the potential impacts these difficulties had on patient care.
Methods
An online, cross-sectional survey was distributed via email using the National EMS Certification database maintained by the National Registry of Emergency Medical Technicians (NREMT) to a random sample of nationally-certified paramedics. Eligible respondents were aged 18 and older, employed as paramedics and had administered care in a prehospital setting within the previous 12 months. Survey items explored difficulties experienced and coping strategies used when difficulty was encountered identifying or opening medications and/or medical supplies. Descriptive statistics and logistic regression were calculated to analyse responses for trends.
Results
Of the 12,000 emails sent, 1,912 participants responded (response rate = 16%). After removing respondents who had not administered care within the past 12 months and partial surveys, data from 1,702 respondents were analysed. Nearly 20% of all respondents reported that they had experienced difficulties identifying (21.1%) or opening (20.5%) medications and identifying (17.0%) or opening (23.4%) medical supplies within the past year. Between 1.2% (identifying a medication) and 3.0% (opening supplies) of those included in the analysis indicated that reported difficulties had negatively impacted patient care. Common coping strategies reported to deal with difficulty opening included partner assistance, tool use (scissors, pens, and knives), and the use of teeth, all potential pathways for the transmission of microbes, conceivably further impacting outcomes.
Citation: Lee J, Cash RE, Crowe RP, Hong HG, Panchal AR, Silk K, et al. (2021) Paramedic interactions with the packaging of medications and medical supplies: Poor package design has the potential to impact patient outcomes. PLoS ONE 16(8): e0255099. https://doi.org/10.1371/journal.pone.0255099
Editor: Dylan A. Mordaunt, University of Adelaide, AUSTRALIA
Received: October 22, 2020; Accepted: July 11, 2021; Published: August 11, 2021
Copyright: © 2021 Lee et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its supporting information files.
Funding: The Author(s) received no specific funding for this work. A portion of Laura Bix's salary is funded through a NIFA Hatch Act Grant provided by the USDA under project MICL02263.
Competing interests: The authors have declared that no competing interests exist.
Background
Engineering safety into the overall health care system to reduce errors and improve patient outcomes is an important paradigm that is being actively embraced by the designers of healthcare products and systems, caregivers and policy makers [1]. But research investigating the relationship of the design of packaged products and their impact on the behaviour of healthcare providers and patient care has not been widely conducted. This is likely because, historically, the regulators of medications and medical supplies have required the submission of objective evidence that manufacturing processes produce packaging that preserves, enables, and maintains the product’s safety and efficacy throughout the expected shelf life. Although preserving, protecting and manufacturing products used for patient care is critically important, what has not been assessed is how the package design impacts the ability of the healthcare provider (or patient) to accomplish critical tasks (e.g. properly identifying, aseptically opening and transferring, and properly dosing and administering). Only recently have a host of standards and regulatory documents started to proliferate which require the objective evaluation of how packaging performs in the hands of users (both healthcare providers and patients) [2–6] (Table 1).
Even as standards and requirements for packaging change in an attempt to engineer safety into healthcare, little evidence exists describing the relationship of packaging design to provider behaviours, and ultimately, patient outcomes. This could be because the errors associated with packaging and labelling design are latent in nature, with the mistake occurring upstream from the presentation of the problem. Further, latent errors tend to be “systems related factors;” [16] that is, one factor among several that contribute to an event. Papers that review root-cause analysis of medical errors and adverse events suggest that similar packaging and labelling are fundamental reasons for medication errors [16–19], though other potential problems related to poor packaging design (e.g. difficulty opening or transferring cleanly) have yet to be thoroughly investigated.
And it isn’t just packaging that hasn’t been thoroughly researched, understood, and optimized; the healthcare environment itself, undoubtedly, impacts how care givers interact with products to deliver patient care. Of particular interest to us was how packaging performs within the prehospital setting. Emergency Medical Services (EMS) are administered during prehospital care to stabilize patients and transport them to a location better equipped to provide comprehensive care. Despite the extreme conditions that may be present (e.g., poor lighting, extreme heat or cold, noise, chaos, emergency vehicle movement, interloping friends and family), they are infrequently (if ever) considered by product and package designers.
As evidence of this, we offer data collected in healthcare transport environments, specifically intrahospital transports of critically ill patients, where the rates of adverse events have been noted to be as high as 70% [20–22]. Bergman et al. identified five broad categories of hazards contributing to transport-related adverse events and identified the most common to be errors associated with the category tools and technology, within this category a common problem was “products that were not designed in ways that consider the context/environment of care and support the needs [of heathcare providers] [23].”
Anecdotally, paramedics have been observed to employ flashlights to identify products, or use one hand, and scissors (or teeth) to open items required urgently. These coping strategies, used to deal with shortcomings of designs which don’t appropriately consider the myriad of contexts where paramedics deliver care, have the potential to play a role in patient outcomes. For example, tools and/or inappropriate opening techniques may serve as an indirect mechanism for the transfer of microbes [17,24], a source of healthcare associated infections (HAIs). Although precise sources of microbes affiliated with HAIs are difficult to identify, a surveillance study investigating prehospital infection rates suggests that patients treated by advanced life support (ALS) providers prior to being admitted to the hospital have greater rates of HAIs than those transported by other means [25]. Supporting the idea that the prehospital environment is a fertile place for systemic improvement is the work of Orellana et al [24]. Among the factors that their research team identified as significant risk factors associated with Methicillin-resistant Staphylococcus aureus (MRSA) detection on EMS personnel were infrequent hand hygiene after glove use and low frequency of hand washing. Researchers found a high prevalence of MRSA among Ohio EMS personnel, noting this was both an occupational hazard and a patient safety concern [24].
It is unclear whether conditions present in prehospital environments affect the use of packaged products in ways that provide a source of risk for patients. In light of this, we evaluate the notion that prehospital work creates unique challenges for paramedics as they use packaged products to deliver care. In doing so, our objectives were to:
- describe difficulties associated with packaging design in prehospital settings;
- investigate the coping strategies used by paramedics when difficulties occurred; and
- assess the potential impacts these difficulties had on patient care.
Methods
The study was conducted in accordance with methods approved by the Institutional Review Board at the American Institutes for Research (AIR (#EX00411)), the review board affiliated with the National Registry of Emergency Medical Technicians (NREMT). The transfer of de-identified data to MSU for analysis was determined to not meet the definition of human subjects as defined by the US Department of Health and Human Services) by the MSU Institutional Review Board Manager (application #x16-1412e).
Study design, population & setting
We conducted a cross-sectional survey of nationally certified paramedics included in the National EMS Certification database. Maintained by the National Registry of Emergency Medical Technicians (NREMT), the National EMS Certification database is the largest database of EMS professionals in the United States, with 406,939 EMS professionals entered as of 2018 [26]. Power calculations were performed to determine the number of respondents needed to make estimates with 95% confidence. The calculated sample size was inflated assuming a conservative 10% response rate, leading to a simple, random sample of 12,000 paramedics drawn from the aforementioned database. The survey was limited to paramedics because they can provide a higher level of patient care and, therefore, were more likely to interact with multiple packaging types in the course of care delivery.
Survey instrument and data collection
The questionnaire was developed as a collaborative process between researchers at Michigan State University (MSU) and the NREMT. A cognitive walk-through (See S1 File, “Cogntive walk-through protocol”) was conducted with six participants from the target population to ensure that survey items functioned as intended. Based on this testing, changes were made to enhance understanding by the target population (e.g., the term “medical devices” was changed to “medical supplies”).
The questionnaire (see S3 File “Study Questionnaire”) included items to characterize respondents (demographic questions and questions regarding their work history) as well as those which gathered information about their experiences with two categories of packaged products: medications and medical supplies. Within each product category, questions were organized to probe difficulties related to two critical tasks: identifying packaged products and opening packages. When respondents reported difficulty identifying or opening a packaged medication or medical supply, a cascading series of questions probed their experience to explore the reasons that they had encountered the difficulty (Objective 1), the way that they coped with the issue (Objective 2), and whether or not it had negatively impacted the patient (Objective 3).
Invitations to take part in the electronic questionnaire were sent to a random sample of 12,000 nationally certified paramedics with valid email addresses. Completion of the questionnaire had no bearing on the person’s EMS certification and no identifying information was requested. After the initial invitation, reminder emails were sent approximately one and two weeks later. Respondents who completed a questionnaire by December 1st of 2016, were put into a drawing, with 10 randomly selected to receive $100 Amazon gift cards.
Data analysis
To be included in the analysis, respondents had to be older than 18, be practicing as a paramedic, have provided medical care in a prehospital setting within the previous 12 months, and have completed the questionnaire in full.
Analyses were conducted using SPSS (IBM, Version 22). Descriptive analysis was performed on frequencies of respondent characteristics as well as their reported reasons for difficulties by critical task (identification and opening) for each of the products (medications and medical supplies). Descriptive variables related to participants were regrouped into meaningful categories to better understand the factors affecting difficulty. Specifically, race was dichotomized to white and all others; years of experience to 10 years or less, and more than 10 years of experience. Education level was categorized as non-college vs college; primary EMS role was categorized as patient care provider and all others (e.g., dispatcher, educator, etc.). Community size was regrouped into the three categories: small town (less than 25,000 residents), medium town (25,000–149,999 people) or city (more than 150,000 people).
We conducted multivariable logistic regression analysis to identify risk factors related to “difficulties associated with opening medication within the past year” and “difficulties associated with opening medical supplies the past year.” Univariable logistic regression was also performed for each independent variable to investigate the marginal relationship between each variable and the binary outcome (had/didn’t have difficulty by product category); univariable and multivariable logistic regression analysis were conducted separately for modelling the difficulty in opening medications and the difficulty in opening medical supplies using the variables selected with the previously described method. Odds ratios and 95% confidence intervals (95% CI) were calculated accordingly.
Results
Characteristics of study subjects
A total of 1,912 responses were received (response rate = 16%). Respondents were excluded because they had not provided care in the prehospital setting in the past 12 months (n = 193) or due to incomplete responses (n = 17). The majority of the survey respondents were male (79.3%), white (88.7%), had a more than 10 years’ experience with EMS (61.5%) and served the role of care provider (74.7%). Table 2 presents an overview of the results related to participant characterization for the 1,702 responses included for analysis.
Difficulties and coping strategies
Frequencies and proportions related to all three objectives are found in Table 3 (medications) and Table 4 (medical supplies). Nearly 20% of the paramedics indicated that they had encountered difficulty related to packaging within the previous 12 months for both tasks (identifying and opening) across product categories (medications and medical supplies); specifically: 359 respondents (21.1%) reported that they had difficulty identifying medications within the previous 12 months (Table 3); while 290 respondents (17.0%) reported difficulty identifying medical supplies (Table 4); 349 (20.5%) reported difficulty opening a medication and 399 (23.4%) had difficulty opening a medical supply during this same time frame.
The most commonly reported reason for difficulty identifying a medication was that “different medications have similar packaging” (n = 246, 68.5%, Table 3), while the most commonly reported reason leading to difficulty in identifying a medical supply was “crowded label made it difficult to read” (n = 189, 65.2%, Table 4).
The top reason for difficulty opening was the same across both product categories, “product required two hands to open” (n = 172, 49.3%; n = 270, 67.7%). Frequencies of reported, specific reasons associated with difficulty opening products are presented in Table 3 for medications and Table 4 for medical supplies.
“Flashlight use” was reported as the most common coping strategy employed when paramedics reported difficulty identifying both medications (n = 211/359, 58.8%) and medical supplies (n = 156/290, 53.8%). “Scissors” were the most commonly reported strategy employed when opening difficulties were encountered; this was the case for both medications (n = 189/349, 54.2%; Table 3) and medical supplies (n = 307/399, 76.9%; Table 4). Of additional interest, was the reported use of teeth to open both medications (n = 103/349, 29.5%) and medical supplies (n = 156/399, 39.1%) when difficulties were encountered.
We fit two multivariable logistic regression models with the two outcomes of interest: difficulty opening medication (Model 1, Table 5) and difficulty opening medical supplies (Model 2, Table 6). The following covariates were considered for both models: age, sex, race, years of experience in EMS, primary role, the level of education, and size of community. In addition, difficulty in identifying medication and difficulty in identifying medical supplies were included for Model 1 and Model 2, respectively.
Tables 4 and 5 present the logistic regression analysis for difficulties opening medications and difficulties opening medical supplies, respectively. The odds of difficulty opening medications for those who had experienced difficulty identifying medications within the past year was 2.91 times that of the odds of difficulty opening medications for those who had not experienced difficulty identifying medications within the past year (OR: 2.91, 95% CI: 2.21–3.83). Significant effects of sex and primary role on the odds of reporting difficulty opening medications were also observed (Table 5). Specifically, the odds of reporting a difficulty opening medication was reduced by approximately 49% for males when compared to that of female respondents (OR: 0.51, 95% CI: 0.38–0.68). Additionally, the odds of reporting difficulty with opening medications were reduced by 29% for those with other roles such as dispatcher, instructor and administrator, compared to patient care providers (OR: 0.71, 95% CI: 0.52–0.97).
When difficulties associated with opening medical supplies were examined (Table 6), results were similar to those reported for opening medications (Table 5). The odds of difficulty opening medical supplies for participants who reported a difficulty identifying medical supplies within the past 12 months were 3.67 times higher than the odds for those who had not reported difficulty identifying a medical supply within the past 12 months (OR: 3.67, 95% CI: 2.80–4.85). For every one-year increase in age, there was a 2% increase in the odds of difficulty opening medical supplies (OR: 1.017, 95% CI: 1.00–1.03).
Negative patient outcomes
The prevalence of a negative outcome to patient care outcomes affiliated with difficulties with packaging ranged from 1.2% of all participants (n = 20, or 5.6% of the 359 people reporting difficulty identifying a medication, Table 3) to as high as 3.0% of all participants (n = 51, or 12.8% of the 399 people indicating that they had encountered difficulty opening a medical supply, Table 4). Intermediate to these two task/product combinations were those related to identifying a medical supply (Table 4), with 31 paramedics reporting a negative outcome on patient care (1.8% of respondents or 10.7% of the 290 who reported experiencing this difficulty) and 32 negative outcomes associated with opening medications (1.9% of the respondent population and 9.2% of 349 reporting difficulty with this task, Table 3).
Discussion
Although other researchers have indicated healthcare products not designed in ways which consider care context as among the most common hazard associated with medical errors [23], there is a dearth of work characterizing how packaged products perform in realistic contexts of use. The most oft-studied healthcare environment is the perioperative context, an environment that is arguably less demanding for product design than that of prehospital environment. Further, the performance of package design, a critical and ubiquitous component of all products, has, to our knowledge, never been investigated in prehospital environments, despite the fact that similar packaging and labelling have been identified as fundamental reasons for medication errors [27].
Our analysis provides some initial evidence that paramedics have difficulties both identifying and opening packaged products and that these difficulties have the potential to negatively impact patient outcomes (Tables 3 and 4). Several factors were found to be significantly associated with the reported difficulty with opening for both medications and medical supplies. For both categories of product (medications and medical supplies), respondents had significantly higher odds of reporting difficulty with opening if they had also reported difficulty with identifying within the past year (Tables 5 and 6). These results could indicate that respondents who were more comfortable reporting difficulties were more likely to report for each of the tasks studied (identifying and opening); however, it could also suggest that when manufacturers do not engage in a thoughtful design process, poor design affects multiple, critical tasks. If the latter is the case, paramedics could encounter serial difficulties among the tasks to be accomplished (first in identifying and then, subsequently, in opening needed products), potentially delaying patient care. The potential and impact for serialized difficulties associated with packaging in prehospital contexts was beyond the scope of the work but presents an interesting idea that needs study.
Results also advocate for the incorporation of users likely to have difficulties associated with packaging during the iterative design process. Findings indicate that females were significantly more likely to report difficulty opening medications than their male counterparts (Table 5), potentially because of documented differences in hand strength [28–31]. That said, this could also be indicative of a willingness to express vulnerability that men do not share [32]; if this is the case, it would support the idea that difficulties are actually underreported. We also found older paramedics to be significantly more likely to report difficulties opening medical supplies than their younger counterparts (Table 6); declines in the strength of the hand are well-documented as a natural process that occurs with aging [29,33] and offer an explanation of this result.
The limited work that has been done to understand how packaging performs in healthcare contexts has focused on perioperative environments [27,34,35]. Many of the published findings conducted in these more controlled contexts of care indicate improvements in packaging are needed, parallel to the findings we present here. Published studies using perioperative contexts have concluded that packages are: “hard to open” [35,36] and employ “crowded label contents” which make crucial information difficult to locate [27,34,35]. While our survey of paramedics echoed the difficulties voiced by perioperative personnel, they also expressed problems, and coping strategies, that were uniquely their own. The indication or implication that paramedics only have one hand available to physically manipulate packaging was apparent throughout the results. The need for designs which enable one-handed use was evident in this sample, with the “need for two hands [when opening the packages] [32]” reported as a common difficulty and “use of teeth” or the need for a “partner’s assistance” as common coping mechanisms when difficulties were encountered. It is not beyond fathom that the use of instruments or teeth have the potential to serve as an indirect mechanism for the transfer of microbes, one possible contributor among many which could help to explain the high rates of infection among patients who have received advanced life support (ALS) prior to hospital admission as compared to those transported by other means [25].
If this is the case, it would suggest that the rates of negative impact on patient outcome (Tables 3 and 4) are actually under-reported, further supporting this as an issue in need of careful consideration.
Limitations
One of the more interesting contributions of our study is the number of participants who reported having difficulties that resulted in negative impact on patient care (1.2% associated with identifying a medication; 1.9% associated with opening a medication;1.8% associated with identifying a medical supply; and 3% associated with opening a medical supply). Readers are urged to interpret results related to negative patient outcomes cautiously. The survey itself did not specifically define the term “negative outcome,” instead leaving the survey respondent to interpret its meaning (see Supplemental files for survey). The open-ended nature of this approach enables a wide array of potential outcomes to be considered as negatively impacting care (from delay of care to death of patient). A lack of questions that probed details surrounding the negative outcome precludes detailed interpretation of these results and implores the need for future research.
A further limitation of the survey is the binary nature of the question about difficulties affiliated with packaging use. Specifically, a single question asks respondents about difficulties (identifying or opening) by product type (medications or medical device) and is only presented once (for each combination). A “yes response” related to the difficulty question triggered a cascade of questions probing the difficulty and coping strategies that were employed. Although this provides us with an indication of the frequency across participants, the way that the survey was framed we cannot deduce the number of occurrences for each respondent (across the last year) or whether or not the coping mechanisms that they reported arose from multiple incidences or a single event.
Conclusions
Our findings suggest that paramedics encountered difficulties identifying and opening packaged products and provides preliminary evidence that suboptimal designs potentially negatively impact patient outcomes. In light of this, the designers of healthcare products should incorporate insights gathered in prehospital environments to enhance package designs in ways that enable the performance of tasks critical to care. This becomes an imperative as healthcare evolves to better integrate the complex components that must work in synergy toward common, patient-centered goals, and is requisite to improved outcomes [37].
Acknowledgments
The author would like to thank Hope Akaeze who assisted in her role with the MSU CSTAT organization. Ms. Akaeze provided useful resources for statistics concepts presented herein.
References
- 1.
The Joint Commission, Patient Safety Systems, in Comprehensive Accreditation Manual for Hospitals,. 2016.
- 2.
International Organization for Standardization [ISO] ISO 17049:2013, in Accessible design—Application of braille on signage, equipment and appliances. 2013.
- 3.
International Organization for Standardization [ISO] ISO 11607–1:2019, in Packaging for terminally sterilized medical devices -Part 1: Requirements for materials, sterile barrier systems and packaging systems. 2019.
- 4.
The european parliament and the council of the european union, DIRECTIVE 2001/83/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 6 November 2001 on the Community code relating to medicinal products for human use. 2001: Official Journal of the European Union.
- 5.
The european parliament and the council of the european union, Regulation (eu) 2017/745 of the european parliament and of the council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. 2017: Official Journal of the European Union.
- 6.
U.S Food & Drug Administration, U.S.F., Applying Human Factors and Usability Engineering to Medical Devices, in Guidance for Industry and Food and Drug Administration Staff. 2016: Center for Devices and Radiological Health.
- 7.
International Organization for Standardization, Packaging—Accessible design—General requirements, in ISO 11156:2011. 2011.
- 8.
International Organization for Standardization, Packaging—Accessible design—Ease of opening, in ISO/TC 122 Packaging 17480:2015. 2015.
- 9.
International Organization for Standardization Packaging—Accessible design—Information and marking, in ISO 19809:2017. 2017.
- 10.
International Organization for Standardization Packaging—Braille on packaging for medicinal products, in ISO 17351:2013. 2013.
- 11.
International Organization for Standardization Packaging—Accessible design—Handling and manipulation. 2019.
- 12.
U.S Food and Drug Administration, Applying Human Factors and Usability Engineering to Medical Devices- Guidance for Industry and Food and Drug Administration Staff. 2016.
- 13. U.S. Food and Drug Administration, TITLE 21—Food and Drugs, in Part 211 Current Good Manufacturing Practice For Finished Pharmaceuticals. 2019, Department of Health and Human Services.
- 14.
The european parliament and the council of the european union, Regulation (eu) 2017/745 of the european parliament and of the council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC 2017: Official Journal of the European Union.
- 15.
Sterile Barrier Association, S., Moving from the MDD to the MDR. 2017.
- 16.
Mhairi Jhugursing, Valerie Dimmock, and Haresh Mulchandani, Error and Root Cause Analysis. Bja Education, 2017.
- 17.
The Joint Commission, Root Cause Analysis in Health Care: Tools and Techniques, Parker J., Editor. 2015, Joint Commission Resources.
- 18.
Donaldson Molla S, Corrigan Janet M, and Kohn Linda T, To err is human: building a safer health system. Vol. 6. 2000: National Academies Press.
- 19. Knudsen P, et al., Preventing medication errors in community pharmacy: root-cause analysis of transcription errors. Qual Saf Health Care, 2007. 16(4): p. 285–90. pmid:17693677
- 20. Papson Jonathan P, Russell Kassandra L, and Taylor David M, Unexpected events during the intrahospital transport of critically ill patients. Academic Emergency Medicine, 2007. 14(6): p. 574–7. pmid:17535981
- 21. Erika Parmentier-Decrucq, et al., Adverse events during intrahospital transport of critically ill patients: incidence and risk factors. Annals of Intensive Care, 2013. 3(1): p. 10. pmid:23587445
- 22. Ira Smith, Suzanne Fleming, and Aurel Cernaianu, Mishaps during transport from the intensive care unit. Critical Care Medicine, 1990. 18(3): p. 278–81. pmid:2302952
- 23. Bergman Lina M, et al., Safety Hazards During Intrahospital Transport: A Prospective Observational Study. Critical Care Medicine, 2017. 45(10): p. e1043–e1049. pmid:28787292
- 24. Robert Orellana. C, et al., Methicillin-resistant Staphylococcus aureus in Ohio EMS Providers: A Statewide Cross-sectional Study. Prehospital Emergency Care, 2016. 20(2): p. 184–90. pmid:26516797
- 25.
Page Scott E, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies-New Edition. 2008: Princeton University Press.
- 26.
National Registry of Emergency Medical Technicians [NREMT], 2018 Annual Report. 2018.
- 27.
Jingzhe Cai, Perceptions of Medical Device Packaging Used by Operating Room Personnel, in School of Packaging. 2012, Michigan State University: East Lansing, Michigan.
- 28. Voorbij AIM and Steenbekkers LPA, The twisting force of aged consumers when opening a jar. Applied ergonomics, 2002. 33(1): p. 105–109. pmid:11831209
- 29. Yoxall Alaster and Janson R, Fact or friction: a model for understanding the openability of wide mouth closures. Packaging Technology and Science: An International Journal, 2008. 21(3): p. 137–147.
- 30. Sheik N Imrhan, Muscular strength in the elderly-implications for ergonomic design. International Journal of Industrial Ergonomics, 1994. 13(2): p. 125–138.
- 31. Mathiowetz Virgil, et al., Grip and pinch strength: normative data for adults. Archives of physical medicine and rehabilitation, 1985. 66(2): p. 69–74. pmid:3970660
- 32. Bord R.J. and O’Connor R.E., The gender gap in environmental attitudes: The case of perceived vulnerability to risk. Social science quarterly, 1997: p. 830–840.
- 33. Peebles Laura and Norris Beverley, Filling ‘gaps’ in strength data for design. Applied ergonomics, 2003. 34(1): p. 73–88. pmid:12523807
- 34. Bix Laura, et al., Evaluating Varied Label Designs for Use with Medical Devices: Optimized Labels Outperform Existing Labels in the Correct Selection of Devices and Time to Select. PloS one, 2016. 11(11): p. e0165002. pmid:27824873
- 35.
Perez Paula, Reducing Levels of Medical Device Contamination through Package Redesign, Seal Geometry and Opening Technique. 2018: Michigan State University.
- 36.
Jingzhe C., Perceptions of Medical Device Packaging Used by Operating Room Personnel, in School of Packaging. 2012, Michigan State University.: East Lansing.
- 37. Sturmberg Joachim P, O’Halloran Di M, and Martin Carmel M, Understanding health system reform–a complex adaptive systems perspective. Journal of evaluation in clinical practice, 2012. 18(1): p. 202–208. pmid:22221420