Evaluation and feedback of hand hygiene (HH) compliance are important elements of the WHO multimodal strategy for hospital infection control. Overt observation is recommended, but it may be confounded by Hawthorne effect. Covert observation offers the opportunity to decrease observer bias. In this study we conducted a one year hospital-wide HH promotion program that included medical students (MS) as covert observers.
HH compliance for the five WHO indications was determined by trained and validated observers. The overt observers consisted of eleven infection control nurses (ICNs) and two unit HH ambassadors (UAs) in each of 83 wards. The covert observers consisted of nine MS during their rotating clinical clerkships. Feedback was provided to department heads and staff each quarter.
Of the 23,333 HH observations 76.0% were by MS, 5.3% by ICNs and 18.7% by UAs. The annual compliance rates were MS 44.1%, ICNs 74.4% and UAs 94.1%; P<0.001. The MS found significantly lower annual compliance rates for 4/5 HH indications compared to ICNs and UAs; P<0.05. The ICNs reported significantly improvement from the first to the fourth quarter; P<0.001. This was associated with feedback from the MS of very poor compliance by nurses during the first quarter.
Based on these findings we recommend a two-pronged approach to HH programs. The role of ICNs and UAs is to educate, serve as role models, establish, sustain good HH practices and provide direct feedback. The role of the covert observers is to measure compliance and provide independent feedback.
Citation: Pan S-C, Tien K-L, Hung I-C, Lin Y-J, Sheng W-H, Wang M-J, et al. (2013) Compliance of Health Care Workers with Hand Hygiene Practices: Independent Advantages of Overt and Covert Observers. PLoS ONE 8(1): e53746. https://doi.org/10.1371/journal.pone.0053746
Editor: Jan Kluytmans, Amphia Ziekenhuis, The Netherlands
Received: September 17, 2012; Accepted: December 3, 2012; Published: January 14, 2013
Copyright: © 2013 Pan et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors are grateful to members of the NTUH Center for Infection Control and Center for Quality Control for their sustained support of the hand hygiene program and the hospital staff for their commitment to improve patient safety and reduce healthcare-associated infections. The authors are indebted to the Center for Diseases Control, Department of Health, Taiwan, for providing financial support for the Center for Excellence for Hand Hygiene (EU098040). Y. C. Chen also received a grant from Department of Health, Taiwan (DOH100-TD-B-111-001). The funding organizations were not involved in designing or conducting the study, the data collection, management, analysis, and interpretation, nor the preparation, and approval of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The World Health Organization (WHO) launched the global hand hygiene (HH) program in 2004 to reduce healthcare-associated infections (HAIs) and improve patient safety. Evaluation and feedback of HH performance are important elements of this program. The WHO Guidelines on Hand Hygiene in Health Care and the Centers for Disease Control and Prevention, USA recommend direct observation of compliance and measuring the consumption of HH products –. Direct observation helps to pinpoint areas of strength or weaknesses in HH behavior, identify the number of HH opportunities, their indications, assess technique and provide feedback to healthcare workers (HCWs) , . HH performance is usually improved when the HCWs know that they are under observation. These changes in behavior are often attributed to the well-known “Hawthorne effect” .
In a recent editorial commentary Daniels emphasized the need to reconsider our approach to monitoring hand washing . He pointed out that HCW compliance rates with HH practices remain unacceptably low, and that there are multiple problems with the current gold standard of direct observation. These include “ investment in human capital all but ensures that undersampling will occur”, direct observations are limited to work shifts, and “secret observers” who are not part of the health care team will eventually be detected. He was intrigued by the report of Hornbeck et?al.  in which the authors described the use of a mote-based sensor to record contacts among HCWs and their important observation that HCW contact patterns dramatically affects disease diffusion.
We also have been concerned that observation bias, cueing and peer pressure overestimates the rates of compliance. The current study was designed to evaluate the potential problems of observation bias by comparing the rates of HH compliance by covert overt as well as overt observers. It was conducted as part of a one year hospital-wide HH promotion program at a large teaching medical center in Taipei, Taiwan.
We invited medical students (MS) to serve as covert observers during their clinical clerkships. We reasoned that since MS are an integral part of the health care team they would have multiple opportunities to unobtrusively observe HH practices by HCWs during daily rounds. Infection control nurses (ICNs) and unit HH ambassadors (UAs) served as overt observers. All three groups received the same training and used the same data collection methods.
Materials and Methods
Hospital Setting and Hand Hygiene Program
National Taiwan University Hospital (NTUH) is a 2200-bed major teaching hospital in Taipei, Taiwan that provides both primary and tertiary medical care. Approximately one third of the hospital’s house staff are replaced each year. The distribution and time trends of HAIs and infection control programs during 1981 to 2007 have been described previously . An alcohol-based hand rub was introduced in 2004 and a hospital-wide hand hygiene (HH) program was implemented and promoted annually , . The program was reviewed, revised and promoted annually in accordance with plan-do-check-act cycle based on the performance in the preceding year. The WHO multimodal hand hygiene improvement strategy was adapted in 2009. A detailed description of the action plan and hand washing campaigns is provided in appendix S1. The five WHO indications for HH are listed at the bottom of Table 1 .
Overt and Covert Observers
The overt observers consisted of eleven experienced ICNs and two UAs on each ward. The UAs consisted of a physician and a nurse who were worked on the unit under study. They were responsible for education, communication, audit and immediate feedback of HH performance by the HCWs on their ward. The ICNs and UAs wore uniforms and representative badges on their chests. Some stated their purpose before making their observations. They observed HH practices of physicians, nurses and other HCWs on 83 in-patient units: 24 medical, 18 surgical, 11 pediatric, 4 gynecological wards, and 26 other wards including orthopedics and urology.
The covert observers consisted of nine 5th to 7th year volunteer MS who rotated on the medical, surgical, gynecological and pediatric services and made ward rounds with the assigned medical team. The staff of the units under study were not aware of their identify as covert observers.
Training and Validation of Observers
The observers were required to take the same basic course as other HCWs in HH practices. In addition they participated in a 2-hour workshop, conducted by the same trainer, using WHO standard training materials. A WHO training film included in the WHO Implementation Toolkit  was used to provide case scenarios of the five moments for HH. A standardized hand hygiene observation test, modified from “Hand-hygiene Observation Tool” test developed in UK, was then used to evaluate the participants . The performance of students, ICN, and UAs were 100±0, 70.4±9.7, and 72.0±10.4, respectively (total score 100). The answers to each question were provided following the tests. The trainer facilitated discussions by the participants to clarify the concepts provided in slide presentation that accompanied the film.
Evaluation and Feedback
A standard data form was used to record compliance by HCWs with the five WHO HH indications. The data included hospital unit, professional category (physicians, nurses and other HCWs) and clinical department. The HCWs were not identified to protect their privacy. Unit and department-specific HH compliance rates were evaluated each month by the investigators. The data were provided to the chief physician and head nurse of the unit and department directors each quarter. In addition to periodic feedback, we re-educated the HH UAs in March 2011 to emphasize weaknesses identified during their audits.
Hand Hygiene Product
The hand hygiene products included an alcohol based hand rub (Clean Anti-bacterial Hand Sanitizer, Panion & BF Biotech Inc., Taiwan), chlorhexidine (Hibiscrub, Molnlycke Healthcare, UK), and liquid soap (Luis Pearly Liquid Soap, Sa-Toun, Taiwan). To improve the accessibility of the hand hygiene products, the location of the alcohol-based hand rub was increased from the first quarter from one per multi-bed room to at least one for every 2 beds.
The consumption of hand hygiene products was monitored by the amount sent to each ward by the Department of General Affairs.
Data Analysis and Statistics
A single individual entered all the data from hard copy onto a standardized spreadsheet. Compliance was defined as the ratio of the number of performed actions to the number of HH opportunities . The Chi square test was used to test the difference in compliance rates by type of observers, quarter, professional category, department and the HH indication.
This study was approved by the Institutional Ethics Review Board of the National Taiwan University Hospital (No. NTUH- 201109015RC) and the requirement for informed consent from each patient was waived due to the project was part of the hospital-wide quality improvement program and an observational study.
The distribution of the opportunities to observe HH performance, during the study year, by type of observer, HCW category and WHO indication are shown in Table 1. The MS made 76.0% of the 23,333 observations, followed by the UAs (18.7%) and ICNs (5.3%). The differences among the observers in frequency of opportunities to witness HH performances were highly significant; P<0.001. The most frequent indications for HH were before (38.6%) and after touching a patient (42.0%). These were followed by after touching the patients surroundings (16.1%), before a clean aseptic procedure (6.8%) and after exposure to body fluids (5.5%). The MS were less likely than ICNs and UAs to witness aseptic procedures and body fluid exposures, both p<0.001.
The overall HH compliance rates by type of observer for each quarter and the study year are shown in Figure 1. The quarterly and annual compliance rates observed by the MS were significantly lower than those of the ICNs and UAs (44.1% versus 74.4% and 94.1%, respectively; P<0.001). The MS found significantly lower annual compliance rates for 4/5 HH indications compared to ICNs and UAs; P<0.05. The ICNs noted significant improvement in quarterly compliance rates from the first to last quarter, p<0.001. This was associated with feedback from the MS of poor HH compliance by the nurses during the first quarter (see Feedback and improvement).
The compliance rates observed by medical students (MS) were significantly lower than those by infection control nurses (ICNs) and unit HH ambassadors (UAs) in each quarter (all the P value <0.001). The numbers in parenthesis represented hand hygiene opportunities observed. T-bar represented one standard deviation.
The HH compliance rates, during the study year, according to professional category and type of observer are shown in Figure 2. The MS reported significantly lower compliance rates than ICNs and UAs for all professional categories; P<0.001. Both the students and ICNs reported lower compliance rates for doctors than nurses; both P<0.001.
Compliance observed by medical students (MS) was significantly lower as than by infection control nurses (ICNs) and unit HH ambassadors (UAs) (all P<0.001). The numbers in parenthesis are the hand hygiene opportunities. T-bar represents one standard deviation.
Feedback and Improvement
Feedback of the HH practices was provided to staff by ICNs and UAs each quarter with special emphasis on HH for clean procedures. The quarterly HH compliance rates, by type of observer, according to professional category, department and HH indication are shown in Figure 3. The MS reported significantly improved compliance by the nurses; P<0.001 (see above), but poorer compliance by other HCW; P<0.001. The ICNs reported significant improvement by the pediatric department; P = 0.004 and hand indication 2, P<0.001. The UAs consistently reported compliance rates of 80 to 100% for all quarters.
The average consumption of hand hygiene products per 1000 patient-days is shown in Figure S1. These consisted of 34.9 L of alcohol-based hand rub, 13.6 L of disinfectant, and 10.3 L of liquid soap. Consumption for any of these products did not significantly change during the study period (P = 0.72, 0.24, and 0.55, respectively).
The key findings of this study were the ability of well-trained, volunteer MS to covertly observe three quarters of the 23,333 opportunities for HH compliance and to detect significantly lower rates of compliance than full time overt ICNs and UAs over a one-year period at a major medical center in Taiwan. In addition, the MS finding of poor compliance by the nursing staff, during the first quarter, led to feedback that significantly improved their compliance in subsequent quarters. The MS also found that the quarterly HH compliance rates did not improve despite intense educational efforts during the study year. The ability of MS to serve as covert observers to assess simple HH procedures was confirmed during their training and verified in the first quarter to be concordant with experienced ICNs. The ICNs appear to have overestimated compliance by 30.3% and UAs by 50.0% when compared to MS. These large differences between covert and overt observers question the ability of ICNs and UAs to adequately assess HH compliance.
The MS noted improved HH compliance by nurses, but not physicians despite repeated feedback during the study period. This might be explained by the cultural characteristics of busy physicians and surgeons, decreased time on duty of house staff, and changes in residency training programs.
The characteristics and pros and cons of covert and overt HH observers are summarized in Table 2. The advantages of using MS as covert observers during their clinical clerkships are: far more opportunities to witness HH practices by HCWs on day, night and weekend shifts, avoiding selection bias and cues that might lead to the Hawthorne effect, while maintaining patient privacy. It is hoped that this experience will make them more critical observers and foster the development of a cadre of physicians concerned with hospital infection control.
Several investigators have reported significant improvement in infection control programs when front line ward HCWs were recruited as participants –. The advantages of using ICNs and UAs are their ability to promote the HH program, educate and stimulate compliance with good HH practices and provide immediate feedback . In this respect the so-called Hawthorne effect should enhance the opportunity to reinforce good HH practices. The disadvantages of using ICNs to assess compliance are: fewer opportunities to witness HH by HCWs, short periods of observation and ready identification by HCWs. The disadvantage of using UAs to assess compliance is their apparent inability or unwillingness to detect poor compliance. This is most likely accounted for by peer pressure and the need to attend to more compelling duties and responsibilities.
The first hospital-wide study was conducted in 1994 by Pittet et?al. at a teaching hospital in Switzerland . Infection control nurses conducted 2,834 20-minute observations on 48 wards distributed randomly during the day and night over 14 days. The average compliance was 48%. They found that noncompliance was higher among physicians and other HCWs than among nurses and was lowest on weekends. Noncompliance was higher in intensive care than internal medicine units and during procedures that carried a high risk for contamination. Our findings, using MS as covert observers, were virtually identical to those of Pittet et?al. The annual compliance rates were 44.1% and physicians were found to be less compliant than nurses.
We are aware of only two studies that used staff volunteers or research nurses as covert observers. Both were conducted in the ICU setting , . ICUs are structured quite differently from regular hospital rooms. They tend to be relatively open areas or small self-contained units. ICUs provide numerous opportunities for patient contact with multiple personnel of different levels of training and experience as well as exposure to contaminated instruments and fluids . The ICU architecture allows research nurses to make observations at a distance without revealing their identities . In contrast, most hospital wards are structured as separate rooms containing one or more patients. It is much more difficult for observers to explain their presence. MS need not identify themselves as covert observers because they are considered as part of the medical team at the point of care. MS have previously been shown to be effective overt observers in HH programs . They reduced labor costs, sustained surveillance and provided immediate feedback, but their role was similar to those of ICNs.
We believe that use of MS as covert observers, as reported in the current study, addresses several of the criticisms of overt observers in Daniels’ editorial commentary  and confirms the work of Pittet et?al . The advantages of using MS as covert observers are: First, the investment in human capital was greatly reduced. Second, the MS were able to make a far greater number of observations than ICNs, including all work shifts and days of the week. Third, the MS were able to provide feedback to the infection control team about instances of poor compliance that was missed by the ICNs. Electronic monitoring systems add to our understanding of the consequences of exposing patients to peripatetic personnel within the complex hospital environment . It remains to be seen whether electronic systems can be adapted to monitor effective hand washing practices in hospitals. Even if effective monitoring devices can be developed considerable effort would still be needed to assure compliance. In the meantime, we believe that the combination of overt and covert observers combined with effective feedback offers the most effective approach to improve hand washing practices.
This study has several strengths and limitations. The strengths include: it was conducted at a major teaching hospital experienced in promoting good HH practices; it included a very large number observations; confounding by seasonal variations was minimized by including all quarters during the study year; the MS observations were in accord with no significant change in use of HH hygiene products during the study year; and we were able to compare covert and overt observers, while maintaining HCW privacy. The limitations include: the observers were not required to record a fixed proportion of observations according to professional categories, HH indications or department. This might have resulted in selection bias in comparing quarters and among observers. To deal with this issue we used stratification analysis to show that the compliance rates observed by students were still lower than those of the overt observers, irrespective of professional category, department or HH indication. We were unable to directly compare the covert and overt observers on individual wards because of the small number of observations made by ICNs and UAs on each unit.
Future strategies include earlier incorporation of hand hygiene training in the curriculum of health care professionals and more vigorous enforcement.
In conclusion, education, observation of HH performance and feedback are key elements of an HH program. We recommend a two-pronged approach to HH programs. The role of ICNs and UAs is to educate, serve as role models, establish and sustain good HH practices and to provide direct feedback. The “Hawthorne effect” probably strengthens these efforts. The role of the covert observers is to establish efficacy independent of observer bias and the “Hawthorne effect”.
The monthly consumption of hand hygiene products during 2010 and 2011 at the National Taiwan University Hospital.
Conceived and designed the experiments: YCC SCC. Performed the experiments: KLT ICH WHS MJW. Analyzed the data: SCP YJL. Wrote the paper: SCP CMK.
- 1. Boyce JM, Pittet D (2002) Guideline for hand hygiene in healthcare settings. Recommendations of the Healthcare Infection Control Practices Advisory Committee and the HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force. MMMR 51: 1–45.
- 2. Pittet D, Hugonnet S, Harbarth S, Mourouga P, Sauvan V, et al. (2000) Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Lancet 356: 1307–1312.
- 3. Van de Mortel T, Murgo M (2006) An examination of covert observation and solution audit as tools to measure the success of hand hygiene interventions. Am J Infect Control 34: 95–99.
- 4. World Health Organization (2009) WHO guidelines on hand hygiene in health care. Geneva.
- 5. Haas JP, Larson EL (2007) Measurement of compliance with hand hygiene. J Hosp Infect 66: 6–14.
- 6. Eckmanns T, Bessert J, Behnke M, Gastmeier P, Rudenet H (2006) Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol 27: 931–934.
- 7. Daniels TL (2012) Reconsidering hand hygiene monitoring. J Infect Dis 206: 1488–1490.
- 8. Hornbeck T, Naylor D, Segre AM, Thomas G, Herman T, et al. (2012) Using Sensor Networks to Study the Effect of Peripatetic Healthcare Workers on the Spread of Hospital-Associated Infections. J Infect Dis 206: 1549–1557.
- 9. Chuang YC, Chen YC, Chang SC, Sun CC, Chang YY, et al. (2010) Secular trends of healthcare-associated infections at a teaching hospital in Taiwan, 1981–2007. J Hosp Infect 76: 143–149.
- 10. Lin HC, Tien KL, Sun CC, Wang SH, Chen YC, et al.. (2010) Promotion and achievement of a hospital-wide hand hygiene program implemented during 2004–2007 at a teaching hospital in Taiwan. Infect Control J 20, 146–62.
- 11. Chen YC, Sheng WH, Wang JT, Chang SC, Lin HC, et al. (2011) Effectiveness and limitations of hand hygiene promotion on decreasing healthcare-associated infections. PLoS ONE 6: e27163.
- 12. WHO Training Film. Implementation Toolkit. Geneva. World Health Organization. Available: http://www.who.int/gpsc/en/. Accessed 2011 May 17.
- 13. McAteer J, Stone S, Fuller C, Charlett A, Cookson B, et al. (2008) Development of an observational measure of healthcare worker hand-hygiene behaviour: the hand-hygiene observation tool (HHOT). J Hosp Infect 68: 222–229.
- 14. Seto WH, Ching TY, Yuen KY, Chu YB, Seto WL (1991) The enhancement of infection control in-service education by ward opinion leaders. Am J Infect Control 19: 86–91.
- 15. Miyachi H, Furuya H, Umezawa K, Itoh Y, Ohshima T, et al. (2007) Controlling methicillin-resistant Staphylococcus aureus by stepwise implementation of preventive strategies in a university hospital: impact of a link-nurse system on the basis of multidisciplinary approaches. Am J Infect Control 35: 115–121.
- 16. Pittet D, Moourouga P, Perneger TV (1999) Compliance with handwashing in a teaching hospital. Ann Intern Med 130: 126–130.
- 17. Son C, Chuck T, Childers T, Usiak S, Dowling M, et al. (2011) Practically speaking: rethinking hand hygiene improvement programs in healthcare settings. Am J Infect Control 39: 716–24.
- 18. Rosenthal T, Erbeznik M, Padilla T, Zaroda T, Nguyen DH, et al. (2009) Observation and measurement of hand hygiene and patient identification improve compliance with patient safety practices. Academic Medicine 84: 1705–1712.