Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Effects of embedded acceptance and commitment training in veterinary medical and technology programs as students transition into the workforce: A study protocol

  • Mary Beth Spitznagel ,

    Contributed equally to this work with: Mary Beth Spitznagel

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing

    mspitzna@kent.edu

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

  • John Martin ,

    Roles Data curation, Investigation, Methodology, Project administration, Software, Validation, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

  • Nikki E. Bennett ,

    Roles Data curation, Methodology, Project administration, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

  • Christopher Was ,

    Roles Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

  • Christopher M. Fulkerson ,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation College of Veterinary Medicine, Purdue University, West LaFayette, Indianna, United States of America

  • Jim Weisman ,

    Roles Data curation, Investigation, Project administration, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation College of Veterinary Medicine, Purdue University, West LaFayette, Indianna, United States of America

  • Luca Giori ,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation College of Veterinary Medicine, University of Tennessee - Knoxville, Knoxville, Tennessee, United States of America

  • Erin Kelly-Snyder ,

    Roles Data curation, Investigation, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Veterinary Technology Program, Columbus State Community College, Columbus, Ohio, United States of America

  • Alanna S. G. Updegraff ,

    Roles Investigation, Methodology, Software, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychological Sciences, Kent State University, Kent, Ohio, United States of America

  • Michael P. Twohig ,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Software, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychology, Utah State University, Logan, Utah, United States of America

  • Michael E. Levin

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Software, Writing – review & editing

    ‡ These authors also contributed equally to this work.

    Affiliation Department of Psychology, Utah State University, Logan, Utah, United States of America

Abstract

Psychological distress is elevated in the field of veterinary medicine. Recent evidence demonstrates that stress and burnout in veterinary workers can be reduced through an Acceptance and Commitment Training intervention targeting “burden transfer,” or reactivity to challenging interactions with veterinary clients (“Unburdened”). Exposing students to Unburdened could optimize mental health outcomes as they transition into the veterinary workforce. Unburdened will be adapted for student use in a self-guided digital format and embedded into 5 veterinary medical or technology programs. Participants will be 200 advanced students recruited into a parallel-arms design: assessment-only Control versus Intervention (<2 hour Unburdened intervention embedded into final semester requirements) conditions. Online assessments at baseline, program completion (1 month), and follow-up (3, 6, 9, 12 months) will measure Kirkpatrick outcomes: Reaction (engagement, perception of the program), Learning (knowledge test performance), Behavior (skill use frequency), and Results (burden transfer, stress, burnout, anxiety, and depression). Latent growth curve modeling will compare conditions throughout the year following graduation. Institutional Review Board approval has been obtained. Informed consent is electronically granted by students electing to participate. A safety officer unconnected to the study will monitor study progress and safety. Pending positive outcomes, this ready-to-share program will be made available to training programs. If successful, its routine adoption into training curricula could decrease burden transfer, stress, burnout, anxiety, and depression in the field. Ultimately, the proposed work could provide a scalable and sustainable strategy to enhance mental health and well-being across the veterinary workforce.

Study registration: The trial to evaluate Unburdened’s potential impact on mental health is considered observational, as the involved educational institutions chose to implement the program for its educational value. Details can be found in NIH RePORTER: https://reporter.nih.gov/search/FhAMx6slrEautC8YKesk0Q/project-details/10980628. This study is registered with the Open Science Framework, where data will ultimately be made available: DOI 10.17605/OSF.IO/QEAKM.

Introduction

While highly rewarding in many ways, occupational hazards of veterinary medicine can make it an emotionally challenging work setting [15]. Despite the urgent need to ensure wellbeing in this field, reviews of wellness interventions in veterinary healthcare providers note a lack of rigorous research [6] and call for evidence-based approaches [7]. Focusing on mechanisms of distress associated with specific and systemic problems in the field may be valuable.

One driver of distress in the field is a high frequency of difficult client interactions [8,9]. “Burden transfer” occurs when the distress of veterinary clients about their companion animal leads to challenging interactions with veterinary providers, shifting distress to the healthcare team [9]. This transfer of burden has been recently identified as a potential mechanism of occupational distress in the field of veterinary medicine [9,10]. Notably, evidence suggests that while frequency of difficult client interactions is related to stress and burnout, reaction to the situation is a far stronger predictor of these negative outcomes [9]. Therein lies a mechanism for change, and strong potential for improvement.

Recent research demonstrates the usefulness of “Unburdened,” an Acceptance and Commitment Training (ACT)-based intervention, in helping veterinary healthcare teams alter their response to challenging client interactions [1012]. ACT teaches self-reflection through mindfulness and acceptance interventions via experiential exercises, while providing techniques for behavior change [13]. Meta-analysis of ACT in the workplace shows medium effect size reductions in stress [14], with recent work demonstrating significant reductions of distress and burnout in healthcare staff [15]. Moreover, this modality is effective even when delivered in a digital self-guided format [16], which provides a highly scalable and accessible format for disseminating ACT. Unburdened, with its focus on difficult client interactions in the veterinary healthcare setting (see Methods for greater detail), has exhibited effectiveness across various delivery methods (e.g., small group, telehealth, remote asynchronous) in enhancing mental health outcomes for persons currently working in this field [1012].

Used in a training setting, Unburdened is likely to facilitate important competencies identified in the veterinary educational frameworks [17,18], such as skills for reflecting on personal actions, promoting attention to sources of workplace stress, and taking action to remedy adverse situations. Additionally, students preparing for a career in veterinary medicine are already likely to be experiencing emotional distress: 49% report at least moderate stress, 66% report mild to moderate depressive symptoms, and 30% are at high risk for burnout [1921]. A tailored-for-students version of Unburdened, developed to be broadly adopted into curricula of veterinary doctorate (DVM) and veterinary technology (RVT) training programs, has the potential to enhance well-being and reduce the likelihood of future stress and burnout for future generations of providers.

The overall aim of this project is to study whether Unburdened training, adapted for student use (“Unburdened – Student Edition”) in a digital self-guided format and embedded into DVM and RVT education, can reduce stress, burnout, and symptoms of depression and anxiety as trainees transition into the workforce. Pending successful outcomes, the easily shareable program will be made broadly available for DVM and RVT programs.

Methods

Study design

This project is a parallel arms study recruiting participants from 5 training programs agreeing to embed the intervention into final semester requirements on an alternating basis. Site alternation was chosen over individual randomization to prevent risk of students within tight-knit cohorts sharing Unburdened – Student Edition techniques with Control students. Unburdened – Student Edition will be completed at a pace of 1 session per week (approximately 30 minutes per session) over the course of 3 weeks. During years in which the program is embedded at an institution (Intervention), online research assessments will be conducted at baseline, completion (1 month), and follow-up (3, 6, 9, 12 months). When the intervention is not embedded at an institution, participants will complete measures at the same intervals (i.e., “Training as Usual” Control). The intervention will be made freely available to controls after their participation in the study is completed. See Table 1. Enrollment began March 17, 2025. Assuming the study enrolls as expected, the anticipated end of recruitment will be May of 2027, with an anticipated end of data collection in May of 2028. Peer review of the protocol was conducted as part of the federal review process of the National Institute of Safety and Occupational Health (Safety and Occupational Health study section), as well as by an independent safety officer consulting for the study. The primary institutional review board reviewed and approved the protocol; participating institutions reviewed the protocol and deemed reliance upon the primary IRB appropriate.

thumbnail
Table 1. Schedule of enrollment, intervention/control, and assessments for each cohort.

https://doi.org/10.1371/journal.pone.0334898.t001

Study setting

Study procedures all take place through the Kent State University’s Department of Psychological Sciences. Programs utilizing the Unburdened – Student Edition program and allowing student recruitment for research participation include Purdue College of Veterinary Medicine DVM and RVT programs, University of Tennessee – Knoxville DVM program, Columbus State Community College RVT program, and Kent State University RVT program.

Ethical considerations

The protocol described here was approved by the Kent State University Institutional Review Board #1530. The project will be completed in accordance with the Declaration of Helsinki. The informed consent document is presented online prior to the online research survey opening; consent is documented through the participant clicking to agree to study procedures. Students are explicitly informed that participation in research is voluntary, can be discontinued at any time, will be kept confidential from their own educational institution, and that their decision to participate (or not) will not affect their grades or status in their program.

Recruitment

The above institutions will embed Unburdened – Student Edition into final semester requirements as applicable (e.g., coursework, competency checklist) on an alternating basis, allowing for collection of both Intervention and Control data from all sites. Because the program is embedded into training, participation in research involves only the commitment to complete brief online questionnaires at 6 time points and allowing researchers to examine program engagement. Control participants will be recruited using pre-approved language sent by an instructor at their institution as a mass email containing the study link and informing students of the opportunity to participate in optional online survey-based research examining the “wellbeing of students as they transition into the workforce after graduation.” Interested students clicking the link will be directed to the Informed Consent, where they may opt in or out of research. Intervention sites will similarly inform students of an upcoming research opportunity via mass email; however, Intervention participants will be passively recruited through Unburdened – Student Edition. Upon enrollment, they will be informed that there is an optional research opportunity related to their use of the program. If the student clicks to “learn more,” they are directed to the Informed Consent, where they may opt in or out of research. Additionally, separate Control and Intervention site flyers have been created utilizing similar verbiage to the respective mass email messages; these will be placed in student mailboxes or distributed in class as applicable. Recruitment is expected to continue through April of 2027.

Participants

Inclusion and exclusion criteria were chosen to maximize generalizability to student makeup of DVM and RVT programs, and ultimately to the workforce in veterinary medicine. Participants will be neither screened for nor excluded on the basis of common psychological disorders including depression and anxiety [22], as their exclusion could limit generalizability of findings; however, data collection and analyses will examine presence of common psychological symptoms including depression and anxiety. Inclusion criteria will include DVM and RVT students age 18 or older who are proficient in English and are planning to be in the veterinary workforce within 1 month of graduation. Exclusion criteria will include having less than 10 hours per week of direct interaction with veterinary clients.

Intervention

The ACT framework is a well-established modality that relies on six interrelated components [13,2326]. 1) “Being Present” involves consciously experiencing internal and external events in the moment. 2) “Acceptance” is the active embracing of inner struggles or difficult processes (e.g., thoughts, feelings) while they occur. 3) “Defusion” involves reducing excessive entanglement in thoughts by responding to thoughts as just thoughts. 4) Understanding of “Self-as Context” involves observation of inner experiences from an outside perspective. 5) “Values” are areas of importance to an individual that can be identified and embraced to guide action. Finally, taking 6) “Committed Action” means behaving with intention, in a manner that serves a chosen value.

The Unburdened program is based on these six processes, tailoring education and use of techniques to common challenging client interactions in the veterinary healthcare setting. Participants of the Unburdened program individually identify areas of burden transfer with which they struggle and learn these skills within a personalized framework. For example, a participant who experiences frustration associated with clients who repeatedly call for reassurance could learn to openly disentangle and observe their stress, connect with personal values of helpfulness and self-care, and commit to working with this client in a functional way. The protocol for Unburdened has been developed through a series of feasibility studies, randomized pilot trials, and randomized controlled trials examining it in various delivery formats (e.g., in-person group format, teleconferencing delivery, and asynchronous delivery). Through this iterative approach, the program has been refined into a digital self-guided format; this format has been adapted for student use (“Unburdened – Student Edition;” see Table 2).

All components of this program are integrated into a single, Qualtrics-housed program that is easily shared and embedded into a learning system via a single link. Unburdened – Student Edition includes 3 main modules lasting approximately 30–45 minutes, each demarcated with a “halfway point” to maximize use in the classroom (i.e., program could ultimately be used during 3 classes at 1 session per class, or over 6 classes at ½ session per day to accommodate group discussion). Unburdened – Student Edition is a highly interactive tutorial involving brief passages presenting new material, student thought/responses, and experiential exercises, ultimately teaching ACT skills through constant input that integrates the individual’s personal experience. Accompanying the tutorial are frequent videos of recent DVM and RVT graduates sharing real-world workplace experiences involving burden transfer to help students, who may be less familiar with how client interactions impact the veterinary setting, authentically understand these dynamics.

Assessments

Outcomes will be examined through objective and subjective measures utilizing the Kirkpatrick framework [27] to assess the degree to which participants respond favorably to the intervention (“Reaction”), knowledge demonstrated from it (“Learning”), extent to which their behavior reflects application of the new knowledge (“Behavior”), and degree to which the program results in targeted outcomes (“Results”). Measures will be assessed at each study timepoint (excepting basic demographic information which will only be collected at baseline, and Reaction variables, which can only be assessed following program use). Responses will be collected via Qualtrics and automatically scored to reduce potential error or bias in scoring.

‘Reaction’.

Objective tracking of program use in Qualtrics will be conducted by examining percent of each Module that participants complete. Total percent will be examined to assess engagement. Subjective assessment of usefulness and relevance will be assessed via items presented on a Likert-type scale as in prior work [11,12], with assessment of internal consistency prior to analysis.

‘Learning’.

Objective assessment of information learned in the intervention. Prior work [12] shows good internal consistency (α = 0.73) of this performance-based measure.

‘Behavior’.

A frequency count will be taken of intervention skills used by participants estimating the frequency of use of learned techniques over the 2 weeks prior to assessment. Number of hours worked will be accounted for to produce average workday skill use. Past work [11,12] using this rating scheme shows excellent internal consistency (α = 0.92).

‘Results’.

Previously validated self-report instruments will be used to examine outcomes, including the Burden Transfer Inventory (BTI) [9], Perceived Stress Scale (PSS) [28], Copenhagen Burnout Inventory (CBI) [29], Center for Epidemiology Studies Depression scale (CES-D) [30], and Generalized Anxiety Disorder 7-Item Scale (GAD-7) [31].

On the BTI, participants rate Frequency of and Reaction to 5 domains of DANCE interactions: Daily Hassles, Affect, Non-Adherent/ Inconsiderate, Confrontation, and Excess Communication. This measure demonstrates excellent internal consistency (α = 0.92-.94), respectively. The Reaction subscale will be the primary scale of interest, as it correlates strongly with stress and burnout (r = 0.39–0.70) [9]. Because Frequency may change in the transition from training to the workforce, this subscale will be used as a control variable in analyses if needed. The PSS [28] is a widely-used measure of stress perception that demonstrates good internal consistency (α = 0.68 to 0.78) and convergent validity including prediction of anxiety and anger (β = 0.54–0.68) [32]. Total PSS will be the variable analyzed from this measure. The CBI is a measure of burnout with separate personal, work, and client-related burnout subscales [29]. Psychometric properties include high internal consistency (Cronbach’s α = 0.85 to 0.87) and construct validity (negative correlations with measures of mental health and vitality; r = 0.39–0.75) [30]. Work- and client-related burnout will be the primary variables of focus. The CES-D is a common tool for assessing depressive symptoms31 which shows convergence validity with clinical diagnosis of depression at r = .45, internal consistency of α = 0.82, and test-retest reliability of r = 0.52 [33]. Total CES-D will be the primary analytic variable. The GAD-7 is a common tool for screening general anxiety symptoms33 which shows sensitivity and specificity for clinical diagnosis of anxiety above.80. Internal consistency is α = 0.92 and intraclass correlation is r = 0.83 [31]. Total GAD-7 will be the primary analytic variable. Data will be collected directly from participants via Qualtrics, with automatic calculation to reduce potential for error.

Statistical analysis, sample size, and power estimates.

Medium-to-large effect sizes [34] are expected based upon prior work using many of the same outcome assessments work [11,12]. The anticipated sample size of 200 students accounts for expected attrition of approximately 25%. To examine initial response to Unburdened –Student Edition, descriptive statistics will be utilized for Reaction variables, while other Learning, Behavior, and Results will be examined in condition over time using repeated measures analysis of variance. Extended follow-up through 12 months will be examined using latent growth curve models, an analytic technique within the structural equation modeling framework to examine change over time. Additionally, several potential predictors of outcomes will be explored via path analysis, including contribution of demographics (e.g., age, gender, etc.), program status (i.e., DVM versus RVT), and employment status/number of hours worked (e.g., part- versus full-time). Mode of analysis will be intent-to-treat (primary) and per protocol (secondary). We will use multiple imputation to address attrition in the intent-to-treat objective.

Data and safety monitoring plan.

A licensed Psychologist (PhD) who is knowledgeable about human subjects research and study methodology, and free from conflicts of interest has agreed to serve as an independent safety officer for this study. This individual has reviewed the study protocol and will monitor study enrollment, any adverse events, and IRB compliance. Reports outlining study progress with recruitment, retention, and adverse events will be submitted to the safety officer at 6-month intervals throughout the trial (or, in the case of a serious adverse event, as they occur).

Dissemination.

The plan for dissemination of this work considers sharing of study data, results, and materials. The trial is registered with Open Science Framework, where deidentified data and protocols will be uploaded following study completion. Results will be shared in peer-reviewed scientific journals and through team-led science blog sites. Finally, the intervention created for this work will be integrated into a single program that is accessible at the click of a link, and made available for use by DVM and RVT programs pending successful outcomes.

Discussion

The significance of this study lies within the potential for Unburdened – Student Edition as a standardized way to help students preparing for the veterinary field learn tangible strategies for essential competencies such as reflection on personal actions (e.g., practicing self-reflection) and attention to personal well-being (e.g., recognizing sources of workplace stress and acting to remedy adverse situations). These skills will optimally translate to safeguard the emerging workforce in veterinary medicine against psychological distress related to a pervasive occupational hazard in the field of veterinary medicine (i.e., difficult client interactions). If successful, the program will be made available for DVM and RVT programs to embed into student curricula.

Acknowledgments

The authors wish to thank Ms. Maria Burkholder, Ms. Amy Butcher, Ms. Jessica Chastain, and Ms. Sydney Vernadakis for their efforts in facilitating this research.

References

  1. 1. Ouedraogo FB, Lefebvre SL, Hansen CR, Brorsen BW. Compassion satisfaction, burnout, and secondary traumatic stress among full-time veterinarians in the United States (2016-2018). J Am Vet Med Assoc. 2021;258(11):1259–70. pmid:33978434
  2. 2. Hayes GM, LaLonde-Paul DF, Perret JL, Steele A, McConkey M, Lane WG, et al. Investigation of burnout syndrome and job-related risk factors in veterinary technicians in specialty teaching hospitals: a multicenter cross-sectional study. J Vet Emerg Crit Care (San Antonio). 2020;30(1):18–27. pmid:31840933
  3. 3. Cohen SP. Compassion fatigue and the veterinary health team. Vet Clin North Am Small Anim Pract. 2007;37(1):123–34; abstract ix. pmid:17162116
  4. 4. Moses L, Malowney MJ, Wesley Boyd J. Ethical conflict and moral distress in veterinary practice: A survey of North American veterinarians. J Vet Intern Med. 2018;32(6):2115–22. pmid:30320478
  5. 5. Vande Griek OH, Clark MA, Witte TK, Nett RJ, Moeller AN, Stabler ME. Development of a taxonomy of practice-related stressors experienced by veterinarians in the United States. J Am Vet Med Assoc. 2018;252(2):227–33. pmid:29319445
  6. 6. Kim RW, Patterson G, Nahar VK, Sharma M. Toward an evidence-based approach to stress management for veterinarians and veterinary students. J Am Vet Med Assoc. 2017;251(9):1002–4. pmid:29035658
  7. 7. Moir FM, Van den Brink A. Current insights in veterinarians’ psychological wellbeing. N Z Vet J. 2020;68(1):3–12. pmid:31607215
  8. 8. Spitznagel MB, Cox MD, Jacobson DM, Albers AL, Carlson MD. Assessment of caregiver burden and associations with psychosocial function, veterinary service use, and factors related to treatment plan adherence among owners of dogs and cats. J Am Vet Med Assoc. 2019;254(1):124–32. pmid:30668290
  9. 9. Spitznagel MB, Ben-Porath YS, Rishniw M, Kogan LR, Carlson MD. Development and validation of a Burden Transfer Inventory for predicting veterinarian stress related to client behavior. J Am Vet Med Assoc. 2019;254(1):133–44. pmid:30668296
  10. 10. Spitznagel MB, Updegraff A, Twohig MP, Carlson MD, Fulkerson CM. Reducing occupational distress in veterinary medicine personnel with acceptance and commitment training: a pilot study. N Z Vet J. 2022;70(6):319–25. pmid:34082645
  11. 11. Beth Spitznagel M, Updegraff ASG, Was C, Martin JT, Sislak M, Wiborg L, et al. An acceptance and commitment training program reduces burden transfer, stress, and burnout among veterinary healthcare teams. J Am Vet Med Assoc. 2022;260(12):1554–61. pmid:35943946
  12. 12. Spitznagel MB, Martin JT, Was C, Updegraff ASG, Sislak M, Wiborg L, et al. Self-paced acceptance and commitment training reduces burden transfer, stress, and burnout in veterinary healthcare teams. J Am Vet Med Assoc. 2023;261(6):907–16. pmid:36965475
  13. 13. Hayes SC. Acceptance and Commitment Therapy and the New Behavior Therapies: Mindfulness, Acceptance and Relationship. New York: Guilford Press. 2004.
  14. 14. Ost L-G. The efficacy of Acceptance and Commitment Therapy: an updated systematic review and meta-analysis. Behav Res Ther. 2014;61:105–21. pmid:25193001
  15. 15. Prudenzi A, Graham CD, Flaxman PE, Wilding S, Day F, O’Connor DB. A workplace Acceptance and Commitment Therapy (ACT) intervention for improving healthcare staff psychological distress: A randomised controlled trial. PLoS One. 2022;17(4):e0266357. pmid:35442963
  16. 16. Klimczak KS, San Miguel GG, Mukasa MN, Twohig MP, Levin ME. A systematic review and meta-analysis of self-guided online acceptance and commitment therapy as a transdiagnostic self-help intervention. Cogn Behav Ther. 2023;52(3):269–94. pmid:36847182
  17. 17. Association of American Veterinary Medical Colleges. AAVMC’s veterinary education competency framework. https://cbve.org
  18. 18. American Veterinary Medical Association, Committee on Veterinary Technician Education and Activities. n.d. https://www.avma.org/education/center-for-veterinary-accreditation/committee-veterinary-technician-education-activities/cvtea-accreditation-policies-and-procedures-appendix-g
  19. 19. Killinger SL, Flanagan S, Castine E, Howard KAS. Stress and Depression among Veterinary Medical Students. J Vet Med Educ. 2017;44(1):3–8. pmid:28206849
  20. 20. McArthur ML, Andrews JR, Brand C, Hazel SJ. The Prevalence of Compassion Fatigue among Veterinary Students in Australia and the Associated Psychological Factors. J Vet Med Educ. 2017;44(1):9–21. pmid:28206848
  21. 21. Liu AR, van Gelderen IF. A Systematic Review of Mental Health-Improving Interventions in Veterinary Students. J Vet Med Educ. 2020;47(6):745–58. pmid:32027214
  22. 22. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington (VA): American Psychiatric Association. 2013.
  23. 23. Hayes SC, Strosahl K, Wilson KG. Acceptance and Commitment Therapy: An Experiential Approach to Behavior Change. New York: Guilford Press. 1999.
  24. 24. Luoma JB, Hayes SC, Twohig MP, Roget N, Fisher G, Padilla M, et al. Augmenting continuing education with psychologically focused group consultation: Effects on adoption of group drug counseling. Psychotherapy (Chic). 2007;44(4):463–9. pmid:22122324
  25. 25. Twohig MP, Hayes SC. ACT verbatim for depression and anxiety: annotated transcripts for learning Acceptance and Commitment Therapy. Oakland (CA): New Harbinger. 2008.
  26. 26. Bach P, Moran DJ. ACT in practice: Case conceptualization in acceptance and commitment therapy. Oakland (CA): New Harbinger Press. 2008.
  27. 27. Kirkpatrick D. Evaluating training programs. San Francisco (CA): Berrett-Koehler. 1994.
  28. 28. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–96. pmid:6668417
  29. 29. Kristensen TS, Borritz M, Villadsen E, Christensen KB. The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work & Stress. 2005;19(3):192–207.
  30. 30. Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1:385–401.
  31. 31. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7. pmid:16717171
  32. 32. Perera MJ, Brintz CE, Birnbaum-Weitzman O, Penedo FJ, Gallo LC, Gonzalez P, et al. Factor structure of the Perceived Stress Scale-10 (PSS) across English and Spanish language responders in the HCHS/SOL Sociocultural Ancillary Study. Psychol Assess. 2017;29(3):320–8. pmid:27280744
  33. 33. Lewinsohn PM, Seeley JR, Roberts RE, Allen NB. Center for Epidemiologic Studies Depression Scale (CES-D) as a screening instrument for depression among community-residing older adults. Psychol Aging. 1997;12(2):277–87. pmid:9189988
  34. 34. Funder DC, Ozer DJ. Evaluating Effect Size in Psychological Research: Sense and Nonsense. Advances in Methods and Practices in Psychological Science. 2019;2(2):156–68.