Figures
Abstract
The United States federal animal welfare regulations and the Public Health Service Policy on Humane Care and Use of Laboratory Animals require that institutional animal care and use committees (IACUCs) conduct continuing reviews of all animal research activities. However, little is known about the lapse rate of IACUC continuing reviews, and how frequently investigators continue research activities during the lapse. It is also not clear what factors may contribute to an institution’s lapse in IACUC continuing reviews. As part of the quality assurance program, the Department of Veterans Affairs (VA) has collected performance metric data for animal care and use programs since 2011. We analyzed IACUC continuing review performance data at 74–75 VA research facilities from 2011 through 2015. The IACUC continuing review lapse rates improved from 5.6% in 2011 to 2.7% in 2015. The rate of investigators continuing research activities during the lapse also decreased from 47.2% in 2012 to 7.4% in 2015. The type of IACUCs used and the size of animal research programs appeared to have no effect in facility’s rates of lapse in IACUC continuing reviews. While approximately 80% of facilities reported no lapse in IACUC continuing reviews, approximately 14% of facilities had lapse rates of >10% each year. Some facilities appeared to be repeat offenders. Four facilities had IACUC lapse rates of >10% in at least 3 out of 5 years, suggesting a system problem in these facilities requiring remedial actions to improve their IACUC continuing review processes.
Citation: Tsan M-F, Grabenbauer M, Nguyen Y (2016) Lapse in Institutional Animal Care and Use Committee Continuing Reviews. PLoS ONE 11(9): e0162141. https://doi.org/10.1371/journal.pone.0162141
Editor: Chang-Qing Gao, Central South University, CHINA
Received: June 17, 2016; Accepted: July 18, 2016; Published: September 8, 2016
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data is within the paper and its Supporting Information file.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no completing interest exist.
Introduction
The United States federal animal welfare regulations require that the institutional animal care and use committee (IACUC) conducts continuing reviews of animal research activities at appropriate intervals as determined by the IACUC, but not less than once per year [1]. The intent of this requirement is to provide information to the research facilities regarding all on-going animal research activities to ensure compliance. On the other hand, the United States Public Health Service (PHS) Policy on Humane Care and Use of Laboratory Animals requires that the IACUC continuing reviews of animal research activities be conducted not less than once every three years [2]. The Office of Laboratory Animal Welfare (OLAW) has interpreted this PHS Policy provision for triennial continuing reviews as a requirement for a “de novo” review, requiring a comprehensive review that satisfies all criteria for the approval of initial protocol reviews [3]. Thus, the purpose of continuing reviews appears to be threefold, namely, to inform the IACUC of the current status of the project, to ensure continued compliance with federal animal welfare regulations, PHS Policy, and institutional requirements, and to provide for re-evaluation of the animal research activities at appropriate intervals [3].
To comply with both the federal animal welfare regulations and the PHS Policy continuing review requirements, the Department of Veterans Affairs (VA) requires that IACUCs must review the conduct of all animal protocols annually. At the first and second anniversaries, the IACUC may review a standard form giving current basic information about the protocols. However, prior to the third anniversary, the IACUC must conduct a complete “de novo” review of the protocol [4].
Failure to conduct timely IACUC continuing reviews may result in suspension or termination of protocols, causing considerable interruption and disruption of animal research activities. However, despite the importance of timely IACUC continuing reviews, little is known about the lapse rate of IACUC continuing reviews and how frequently investigators continue research activities during the lapse. It is also not clear what factors may contribute to an institution’s rate of lapse in IACUC continuing reviews.
The VA Health Care System is the largest integrated health care system in the United States with 75 facilities conducting research involving laboratory animals in 2015. As part of the quality assurance program, VA has been collecting performance metric data, including IACUC continuing reviews, for animal care and use programs since 2011 [5].
In the current study, we analyzed VA IACUC continuing review data from 2011 to 2015. We report here the lapse rates in IACUC continuing reviews over a 5-year period and whether the size of animal research programs or the types of IACUCs used, has any effects on lapses in IACUC continuing reviews.
Methods
Data collection
As part of the VA quality assurance program, VA facilities conducting research involving laboratory animals were required to conduct regulatory audits of all active animal research protocols once every 3 years [6]. Audit tools were developed and research compliance officers were trained to use these tools to conduct audits throughout the year [7]. Approximately one third of all active animal research protocols were audited each year.
Results of the protocol regulatory audits conducted between June 1 and May 31 of each year were collected through a web-based system from all VA research facilities. Information collected included: IACUC and Research and Development Committee initial approval of animal research protocols; for-cause suspension or termination of animal research protocols; compliance with IACUC continuing review requirements; research personnel scopes of practice; and investigator animal research protection training requirements [5].
As this was a VA quality assurance project and it did not involve the use of laboratory animals or human subjects including collection of individually identified private information, no IACUC or Institutional Review Board review and approval was required [8].
Data analysis
All data collected were entered into a computerized database for analysis. We used the analysis of ordered categorical data to determine the trend of changes from 2011 through 2015 [9]. This was performed using JavaStat ordinal contingency table analysis available at www.statpages.info. For the comparison of two means, the Student’s t test was used to determine the level of significance. A p value of < 0.05 was considered to be statistically significant. When multiple comparisons were required, post hoc analysis using Bonnferoni correction for multiple comparisons was performed [10].
Results
Lapse in IACUC continuing reviews
Table 1 summarizes the data on IACUC continuing reviews. The lapse rate of all protocols audited was 5.6% in 2011. It decreased in subsequent years and was 2.7% in 2015. Despite a spike of lapse rate in 2014 to 4.3%, there was a statistically significant trend of improvement from 2011 through 2015 (p = 0.0013 using analysis of ordered categorical data).
In nearly a half of lapsed protocols, investigators continued animal research activities during the lapse in 2012 (2011 data not collected). As shown in Table 1, the rate of investigators who continued research activities during the lapse also significantly improved from 47.2% in 2012 to 7.4% in 2015 (p = 0.0069)
Effect of the types of IACUC used
Based on the types of IACUC used, VA research facilities can be categorized into two groups, i.e., those using their own VA IACUCs and those using affiliate university IACUCs as their IACUCs of record. We analyzed our data to determine whether the type of IACUCs used had any effect on lapses in IACUC continuing reviews.
As shown in Table 2, on the average, approximately 60 facilities each year used their own VA IACUCs and 14 facilities each year used affiliated university IACUCs as their IACUCs of record. However, the type of IACUCs used had no effect on the facility’s lapse rate in IACUC continuing reviews (p = 0.5607, Student’s t test).
Effect of the size of animal research programs
We also analyzed our data according to the sizes of facility’s animal research programs. We defined a small research program as those with less than 20 active animal research protocols, a medium research program as having 20–50 active animal research protocols, and a large research program as having more than 50 animal active research protocols.
As shown in Table 3, on the average, approximately 22 facilities each year had a small size research program (of which 16 facilities used VA IACUCs and 6 facilities used affiliated university IACUCs); 31 facilities each year had a medium size research program (of which 25 facilities used VA IACUCs and 6 facilities used affiliated university IACUCs); and 21 facilities each year had a large size research program (of which 18 facilities used VA IACUCs and 3 facilities used affiliated university IACUCs). Facilities with a small size research program had the highest lapse rate in IACUC continuing reviews, namely, 5.98%, while facilities with a large size research program had the lowest lapse rate of 3.08%. However, there were no statistically significant differences among the three groups using the Student t test with Bonferroni correction for multiple comparisons (after Bonferroni correction for multiple comparison (n = 3), in order to be considered statistical significant, p value needs to be < 0.017). Thus, the size of animal research programs had no correlation with the facility’s IACUC continuing review lapse rates.
Facilities with high rates of lapse in IACUC continuing reviews
Since neither the type of IACUCs used, nor the size of animal research programs had significant effects on the lapse rates of IACUC continuing review, we then focused on those facilities with a high lapse rate, namely, more than 10%. As shown in Table 4, approximately 80% of facilities each year reported no lapse in IACUC continuing reviews, approximately 6% and 14% of facilities each year reported IACUC continuing review lapse rates of >0%–10% and >10%, respectively.
Analysis of facilities with a lapse rate of >10% from 2011 through 2015 revealed that 19 facilities had a lapse rate of >10% once in 5 years; 8 facilities had a lapse rate of >10% twice; 1 facilities had a lapse rate of >10% thrice; and 3 facilities had a lapse rate of >10% in 4 out of 5 years.
Discussion
The data presented in this report demonstrate that the lapse rates in IACUC continuing reviews at VA research facilities improved from 5.6% in 2011 to 2.7% in 2015. The rate of investigators continuing research activities during the lapse also decreased from 47.2% in 2012 to 7.4% in 2015. The type of IACUCs used and the size of animal research programs appeared to have no effect in facility’s rates of lapse in IACUC continuing reviews. While approximately 80% of facilities reported no lapse in IACUC continuing reviews, approximately 14% of facilities had lapse rates of >10% each year. Some facilities appeared to be repeat offenders. For example, 4 facilities had IACUC lapse rates of >10% in at least 3 out of 5 years, suggesting a system problem in these facilities requiring remedial actions to improve their IACUC continuing reviews processes.
The major purpose of collecting performance metric data is to promote quality improvement. Each year VA research facilities were provided with their own performance metric data, including IACUC continuing review data, along with the VA national and network averages, so that each facility knows where it stands at the national and network levels (VA facilities are geographically grouped into 21 Veterans Integrated Service Networks) [11]. Thus, facilities can identify their strengths and weaknesses, and carry out quality improvement measures accordingly, as described in detail previously [11]. This might be in part responsible for the observed improvement in lapse in IACUC continuing reviews as reported here.
Performance measurement has been well recognized as an important tool for improving the quality of health care [12]. Health care providers and payers devote substantial resources to collect, analyze, and report data on providers’ performance. Our observation that lapse rates in IACUC continuing reviews improved from 5.6% in 2011 when VA started to collect IACUC performance data, to 2.7% in 2015, supports the utility of performance measurement as a potentially important tool for improving the quality of animal care and use programs. Considerable evidence suggests that postapproval monitoring can lead to improved compliance [13]. However, postapproval monitoring is a labor intensive and costly program. We believe that performance measurement of animal care and use programs can provide important information to guide facility administrators regarding where postapproval monitoring efforts should be directed.
Little is known about factors affecting the IACUC continuing review lapse rates. In the current study, we demonstrated that neither the type of IACUCs used nor the size of animal research programs had any effect on the IACUC continuing review lapse rates. On the other hand, approximately 14% of facilities each year had lapses of >10% and were largely responsible for the observed lapses in IACUC continuing reviews. In addition, some facilities were repeat offenders, suggesting that remedial actions should be directed toward these facilities. Unfortunately, our study was not designed to determine the cause(s) leading to high lapse rates, nor designed to prevent or improve lapse in IACUC continuing reviews. Therefore, we are unable to provide guidance on specific strategies to prevent or improve lapse in IACUC continuing reviews at this point. Future studies should be directed toward strategies for preventing or improving lapse in IACUC continuing reviews.
In the current study, our data collection did not make a distinction between lapse in annual continuing reviews and lapse in triennial de novo reviews. Therefore, it is not clear whether the observed lapse in IACUC continuing reviews was primarily due to annual continuing reviews or triennial de novo reviews. It is also not clear whether the observed improvement in lapse rates from 2011 through 2015 was due to improvement in annual continuing reviews or triennial de novo reviews, or both. Further studies are necessary to clarify these questions.
Supporting Information
S1 File. Lapse in IACUC continuing reviews original database.
1. Protocols lapsed in IACUC continuing reviews; and 2. Protocols audited.
https://doi.org/10.1371/journal.pone.0162141.s001
(XLSX)
Acknowledgments
The authors wish to thank J. Thomas Puglisi, Ph.D., Executive Director, Office of Research Oversight, for his support of this project and critical review of the manuscript, as well as all VA research compliance officers for their contributions in conducting audits and collecting data presented in this report.
Author Contributions
- Conceived and designed the experiments: MFT.
- Performed the experiments: MFT MG YN.
- Analyzed the data: MFT MG YN.
- Wrote the paper: MFT MG YN.
References
- 1.
Animal Welfare Regulations. 9 Code of Federal Regulations (CFR) §1–4. 2008.
- 2.
Public Health Service. Policy on Humane Care and Use of Laboratory Animals. 2002.
- 3. Oki GSF, Prentice ED, Garnett NL, Schwindaman DF, Wigglesworth CY. Model for performing institutional animal care and use committee: Continuing review of animal research. Contemp Topics 1996; 35(5): 53–56.
- 4.
United States Department of Veterans Affairs. Use of Animals in Research. VHA Handbook 1200.07. 2011. Available: http://www1.va.gov/vhapublications/. Accessed 1 June 2016.
- 5. Tsan MF, Bannerman DD, Lakshman R, McVicker B, Gao B, Nguyen Y, et al. Assessing the quality of VA animal care and use programs. Fed Prac 2015; 32(9): 58–63.
- 6.
United States Department of Veterans Affairs. Research Compliance Reporting Requirements. VHA Handbook 1058.01. 2015. Available: http://www1.va.gov/vhapublications/. Accessed 1 June 2016.
- 7.
United States Department of Veterans Affairs Office of Research Oversight. Research Compliance Officer Audit Tools. 2015. Available: http://www1.va.gov/ORO/Research-Compliance-Education.asp. Accessed 1June 2016.
- 8. Tsan MF, Puglisi JT. Health care operations activities that may constitute research—The Department of Veterans Affairs’ perspective. IRB: Ethics & Human Research, 2014; 36(1): 9–11.
- 9. Liu I, Agresti A. The analysis of ordered categorical data: an overview and a survey of recent developments. Test, 2005; 14(1): 1–73.
- 10.
Matthews DE, Farewell VT. Using and understanding Medical Statistics. 2nd ed. S. Karger AG, Basel, Switzerland. 1988. pp. 178–181.
- 11. Tsan MF, Tsan LW. Assessing the quality of human research protection programs to improve protection of human subjects participating in clinical trials. Clin Trials 2015; 12(3): 224–231. pmid:25631384
- 12. Cassel CK, Conway PH, Delbanco SF, Jha AK, Saunders RS, Lee TH. Getting more performance from performance measurement. NEJM 2014; 371:2145–2147. pmid:25470691
- 13. Vanderford DA, Doss S, Banks RE. A retrospective review of postapproval monitoring at a large academic institution. Lab Anim (NY) 2015; 44(10): 395–401.