Skip to main content
Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

An apprenticeship model in the training of psychotherapy students. Study protocol for a randomized controlled trial and qualitative investigation

  • Heidi Brattland ,

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Writing – original draft

    Affiliation Nidelv Community Mental Health Center, St. Olav’s University Hospital, Trondheim, Norway

  • Katrine Høyer Holgersen,

    Roles Conceptualization, Investigation, Methodology, Project administration, Writing – review & editing

    Affiliations Nidelv Community Mental Health Center, St. Olav’s University Hospital, Trondheim, Norway, Department of Psychology, the Norwegian University of Science and Technology (NTNU), Trondheim, Norway

  • Patrick A. Vogel,

    Roles Conceptualization, Investigation, Methodology, Writing – review & editing

    Affiliation Department of Psychology, the Norwegian University of Science and Technology (NTNU), Trondheim, Norway

  • Timothy Anderson,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Department of Psychology, Ohio University, Athens, Ohio, United States of America

  • Truls Ryum

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Writing – review & editing

    Affiliation Department of Psychology, the Norwegian University of Science and Technology (NTNU), Trondheim, Norway



One approach towards advancing the quality of mental health care is to improve psychotherapists’ skills through education and training. Recently, psychotherapy training has benefitted from adapting training methods from other professions (e.g., deliberate practice). The apprenticeship model has a long history in skill trades and medicine, but has yet to be adopted in training mental health professionals. This study aims to investigate the impact of apprenticeship training on clinical psychology students’ skills.


In a pragmatic mixed-methods trial, 120 first year students in a Master’s degree clinical psychology program will be randomized to either training-as-usual or training-as-usual plus psychotherapy apprenticeship. In the intervention group, students will participate, over a period of 10 weeks, in weekly treatment sessions together with licensed therapists at outpatient mental health and substance use treatment clinics. Outcomes are assessed post-intervention and at two-year follow-up. The main outcome measure is the Facilitative Interpersonal Skills (FIS) performance test. Additional self-report measures tap self-efficacy, self-compassion, worry, rumination, and stress. Weekly reflection log entries written by the students will be qualitatively analyzed in order to gain an in-depth understanding of the learning process. Students’ and therapists’ experiences with the intervention will be explored in focus group interviews.


To the best of our knowledge, this is the first controlled study to investigate the impact of apprenticeship as an isolated training component in the education of clinical psychologists. The study is designed so as to yield a comprehensive understanding of an approach which could prove to be a valuable supplement to the existing educational methods in this field and ultimately, contribute to improve the quality of mental health care.


Psychotherapists vary considerably in their abilities to help their patients [1, 2]. As argued by Callahan and Watkins Jr [3, p. 211], high-quality psychotherapist training is “perhaps the most impactful systems-level intervention available to our field”: Any single therapist will treat thousands of patients in the course of their career and the effective improvement of each patient’s mental health yields multiplicative societal effects. Applying a training principle common in other educational fields such as medicine [4], but largely overlooked in the training of clinical psychologists, this study protocol describes an exploratory randomized controlled trial (RCT) and qualitative study into the impact of apprenticeship training for clinical psychology students.

The education of psychotherapists typically consists of a combination of didactic methods (e.g., coursework, reading treatment manuals, discussions of cases and treatment approaches) and clinical practice under supervision. Does training-as-usual facilitate the development of therapist expertise, defined, as recommended by Tracey and colleagues [5], as a demonstrable and progressive improvement of performance and/or client outcomes?

Regarding didactic methods, the overall conclusion from several literature reviews [69] is that while activities such as coursework and reading might increase students’ perceived and declarative knowledge, they are not alone sufficient to change their behaviors or improve client outcomes. Likewise, supervision may be more facilitative of declarative knowledge than of practical skills. Consistent with this notion, a recent review and meta-analysis of supervision research [10] reported consistently small effects of supervision on client-rated treatment outcomes [see also 11, 12]. It is likely that the addition of experiential and interactive methods is necessary for declarative knowledge to translate into improved performance, i.e. procedural knowledge. Perhaps not surprisingly, therapists typically describe practical experience working with clients as essential to their professional development [e.g., 13]. However a series of investigations of large naturalistic data sets found little or no evidence of an improvement of client outcomes with increasing clinical experience for trainees [1417] or licensed therapists [18]. This body of research indicates that the combination of didactic methods and increased clinical experience under supervision might be less conductive to the progressive improvement of trainees’ patient outcomes than their subjective experience might perhaps suggest.

Beyond didactic methods and supervision, a new development in psychotherapist training is the application of training methods found to characterize expert performers in other fields, such as sports or music. Deliberate practice (DP) [19] involves practicing isolated skills or behaviors repeatedly outside of the performance situation. In the context of psychotherapy training, this involves, for instance, performing role plays where the trainee practices isolated skills (e.g., expressing empathy) or specific psychotherapy techniques. For further suggestions of how these principles can be applied to psychotherapy training, see for instance Miller and colleagues, [20], Rousmaniere and colleagues [21] and the Theravue electronic platform, The notion that DP might help improve trainees’ therapy skills has received some experimental support [2227] but the evidence for an impact of DP on client outcomes is, to date, indirect [e.g., 28]. Indeed, meta-analyzing DP in diverse fields, Macnamara and colleagues [29, 30] found that DP had a limited influence on performance in low-predictability environments, that is, performance settings in which the range of possible actions is high and circumstances can change while the performer is planning and executing an action–such as is the case in psychotherapy.

In sum, evidence does not unequivocally support the combination of didactic methods and practice seeing patients under supervision with which most therapists are trained. While the application of DP techniques gives cause for optimism, their feasibility and impact on treatment outcomes in the low-predictable context of psychotherapy remain to be demonstrated. The exploration of supplementary training methods is warranted, and perhaps especially experiential methods that address the complexity of psychotherapy as it is practiced in real-world treatment settings.

Performing a task together with someone who knows it better is, arguably, a basic mechanism for human knowledge and skill transmission. We propose that the benefits of training-as-usual might be augmented by a simplified, time-limited psychotherapy apprenticeship intervention in which students of psychotherapy regularly sit in on therapy sessions conducted by licensed therapists. Apprenticeship training addresses aspects of a craft which cannot be learned by verbal communication or in simulated settings alone. As defined by social anthropologist Gowlland [31, p. 760],

apprenticeship can be characterized as a mostly non-didactic way of teaching and learning, grounded in a local context and dependent on participation of the learning in work related-activities; the acquisition of skills during an apprenticeship involves, among others, social participation and interaction, observation and imitation, and engagement through the senses with tools and context.

Drawing upon Albert Bandura’s theoretical framework of self-efficacy [32, 33], we expect psychotherapy apprenticeships to increase students’ beliefs in their ability to perform psychotherapy successfully in the future. According to this theory, knowing how to do a task (i.e., declarative knowledge) is not all it takes to perform well at it. Individuals with higher self-efficacy perform better because their positive self-appraisals and expectations help them approach tasks in a flexible manner, utilize adaptive coping mechanisms, and persist when faced with obstacles.

Bandura [34] theorized that one’s self-efficacy in a given area is influenced by four sources: Mastery experiences of being able to succeed in performing behaviors or tasks in that area, vicarious experiences of watching social role models similar to oneself perform such behaviors or tasks successfully, verbal persuasion in the form of positive feedback and encouragement from others, and one’s own level of emotional arousal while performing the task. Psychotherapy apprenticeships might work to improve psychotherapy students’ self-efficacy through all four sources. Most notably, the exposure to relatable therapist role models performing their craft in a real-life setting is likely to be a powerful vicarious experience. Moreover, in therapy sessions students can perform simple psychotherapy tasks (e.g., expressing warmth and positive regard for the patient, sharing their understanding of what the patient is experiencing, history-taking), thereby gaining mastery experiences. Any encouragement or positive feedback that therapists give to students following such experiences is a source of verbal persuasion that is directly linked that performance. Finally, repeated exposures to real-life therapy in the presence of a more experienced other may habituate psychotherapy students to the emotional challenges involved in working with individuals in distress, thereby decreasing their arousal and increasing their self-efficacy. We find this aspect particularly important in light of psychotherapy students’ high degree of worry about their abilities to perform therapy [35, 36] and novice therapists’ elevated anxiety levels in treatment sessions [13, 37], which is likely to draw their attention away from the unfolding interaction with the patient [35].

From the experienced therapists’ perspective, apprenticeship offers a way to transmit experience-based knowledge to a new generation of practitioners. It also provides an extra set of eyes and ears in the therapy room. Our own experience working with students is that their “naïve stance” can provide new, and sometimes surprising, perspectives on our own work. Moreover, we find students’ positive energy and enthusiasm for our craft refreshing and energizing.

Our literature search identified few attempts to apply the principle of apprenticeship to psychotherapy training, and then only as one of several components in comprehensive training programs. Examples include the Cognitive Apprenticeship Model of psychotherapy training and supervision [38, 39] and the Progress Cascading Model [40]. In a non-controlled study, the latter model was applied to the training in exposure therapy for obsessive-compulsive disorder [41]. Here, an improvement in trainees’ therapy delivery relative to before training was demonstrated. To the best of our knowledge however, apprenticeship has never been tested as an isolated component and in a controlled design in the context of psychotherapist training.

The main objective of the current randomized trial and qualitative study is to explore the potential benefits of supplementing the existing curriculum with a psychotherapy apprenticeship for first-year Master’s program students in a clinical psychology program. In the intervention group students will observe or co-treat patients together with licensed therapists at outpatient mental health and substance abuse treatment clinics. As a proxy to client outcome data, the main outcome will be Facilitative Interpersonal Skills (FIS); a series of studies [4245] have demonstrated that this practical skills test is a meaningful indicator of overall therapist effectiveness. Additional quantitative outcomes as well as qualitative data from both students and therapists will be collected to yield a deeper understanding of the impact of apprenticeship experiences.

The following research questions will be investigated:

  1. Does the addition of apprenticeship to training-as-usual increase psychology students’ Facilitative Interpersonal Skills (FIS) more than training-as-usual alone?
  2. Is the hypothesized impact of apprenticeship on FIS associated with an increase in self-efficacy and a decrease in stress over the course of the practicum?
  3. Is apprenticeship training beneficial to students’ levels of worry, rumination, and self-compassion?
  4. Are students’ FIS scores associated to their personality traits, interpersonal functioning, social skills, attachment security, capacities for mentalization, and the group climate of their training-as-usual practicum course, and do these variables moderate the hypothesized impact of apprenticeship on FIS?
  5. Are the hypothesized gains obtained through apprenticeship training maintained over a two-year period?
  6. What are the salient aspects of therapy sessions for first-year students, and what characterizes the learning process in apprenticeships?
  7. How does working together with student apprentices impact licensed therapists, and is apprenticeship a feasible training model in mental health care?

Materials and methods

The protocol is reported according to the SPIRIT guidelines [46] and a SPIRIT checklist can be found in the Supporting Information, S1 Table.

Study design

A pragmatic mixed-methods explorative study, consisting of (1) a randomized controlled, parallel group, two-arm, superiority trial comparing training-as-usual to training-as-usual plus apprenticeship, and (2) a qualitative investigation of students’ and therapists’ experiences of the apprenticeship intervention. The study design is illustrated in Fig 1.

Study setting

Participants will be recruited from the clinical psychology program at the Norwegian University of Science and Technology (NTNU) in Trondheim, Norway. A clinical psychologist license in Norway is earned through a six-year master’s level clinical psychology track. The first three years of the study program are mainly dedicated to the acquisition of theoretical knowledge on psychology, psychopathology, and psychotherapy, with limited and only indirect practical experience with psychotherapeutic work. From year four onwards, students have internship practicums with child and adult patients followed by a full-time, six-month externship practicum.

The apprenticeship training will take place at three clinics at the St. Olavs University Hospital, Norway, two of which are community health centers and one, a substance use treatment clinic. As part of the specialized mental health care system, these treatment facilities serve a population of adult individuals (18 years of age or older) with intermediate to severe mental health or substance use disorders. In their apprenticeships, students will rotate between different outpatient teams. While some of these teams are general psychiatric teams (i.e., treat mental health problems of all diagnostic categories) that cover different geographical areas, others are dedicated to specific patient populations such as patients with eating disorders, young adults or individuals experiencing mental health crises, or to specific tasks such as the early assessment of newly referred individuals for whom the suitability for specialized mental health care is unclear.

Participants and eligibility criteria

Student participants will be first-year students at the clinical psychology program (N = 120), randomized 1:1 to training-as-usual or training-as-usual plus apprenticeship. All first-year students are eligible to participate in this study (no exclusion criteria). The apprenticeship intervention will be performed by therapist employees (licensed clinical psychologists, psychiatrists, psychiatric nurses, or from other professions) of the three participating clinics. All therapist employed outpatient teams in these clinics are eligible to participate.


Training-as-usual (control group).

The apprenticeship intervention will be evaluated against the standard education for psychotherapists in Norway. At the NTNU, subjects covered during the first year of the clinical psychology program are the history and methodology of psychology, mental health disorders, and introduction to cognitive, developmental, and personality psychology. In addition to these theoretical subjects, students attend a clinical practicum course with the aim of gaining a basic understanding of clinical psychological work and in particular, the importance of relational competency and communicative skills for the practice of psychotherapy. This course is workshop-based and conducted in groups of approximately 10 students, one instructor, and one student assistant. The main teaching methods are lectures, exercises and discussions. As part of this course, all first-year students attend a one-day observational practice with a clinical psychologist.

Apprenticeship training (intervention group).

In addition to training-as-usual, students will attend weekly treatment sessions with licensed therapists at the mental health and substance use clinics over a period of 10 weeks, each week with a different therapist and patient. A set weekday and time of day for each clinic will be determined for the apprenticeship intervention, and therapist participants will make appointments with patients they view as suitable at that time. Students will rotate between different therapists so as to gain the broadest possible experience base, effectively meeting a new therapist and patient for each apprenticeship session.

In sessions, students will either observe passively or assume the more active role of a co-therapist. Participating therapists are encouraged to include the student in sessions as much as they see fit. Respecting individual preferences and the organic nature of therapeutic conversations, the study protocol does not dictate the degree of student in-session participation but instead, leaves the decision up to each particular therapist, student, and patient in that specific session. Students will likely be increasingly able over their apprenticeship period to participate actively in sessions, especially if paired with therapists who are comfortable working with co-therapists and with patients who draw students into the conversation. Because of the rotation of students between therapists, variations in degree of participation will largely be due to chance and the students’ own preferences.

Following each treatment session students will have a 15–20 minutes reflective conversation with the therapist. This discussion will be unstructured so as to allow for any questions or topics that the student might find important or interesting. The learning process will then be further facilitated by the writing of reflective log entries focusing on the students’ experiences of participating in each treatment session. Members of the research team meet with student participants before and after each apprenticeship training session in order to administer outcome measures and continually monitor the students’ adherence to the protocol. Therapists will be invited to a meeting with the research group during the intervention period to exchange experiences and practical tips regarding their work with the students, and facilitate their adherence to the protocol.

The apprenticeship intervention is based on results from a pilot trial (unpublished) in which quantitative and qualitative data from both therapists and students were collected, analyzed, and used to refine the intervention and study procedure.

Measures and outcomes

See Table 1 for an overview of all measures and outcomes.

Table 1. Schedule of enrolment, interventions, and assessments.

Primary outcome.

The main outcome change from baseline in the Facilitative Interpersonal Skills (FIS) test [43], performed at baseline, post-intervention and two-year follow-up. It is a practical skills test in which a test person responds to video stimuli of actors portraying clients in various difficult or challenging moments in therapy. The situations are designed to represent behavior in the various areas of the Interpersonal Circumplex model [47], such as an angry and confrontational client or a quiet and withdrawn client. The test person is instructed to respond verbally to each particular situation, as if they themselves were that client’s therapist, in a way they believe might be helpful. Responses are video recorded and subsequently rated in accordance with the FIS manual [48] for the degree of interpersonal skills on a scale from 1 (not characteristic) to 5 (extremely characteristic). The FIS assesses eight interpersonal skills, namely 1. Verbal Fluency, 2. Hope and Positive Expectations, 3. Persuasiveness, 4. Emotional Expression, 5. Warmth, Acceptance, and Understanding, 6. Empathy, 7. Alliance Bond Capacity, and 8. Alliance Rupture-Repair Responsiveness. The overall mean score will be utilized in analyses.

The original FIS stimuli clips were translated into Norwegian language and re-recorded with student actors by authors of this study protocol. A preliminary validation study [49] found interrater agreement and internal consistency on the Norwegian FIS test to be excellent. In this trial, students in both conditions will be tested on six clips each at pre- and post-interventions and at two-year follow-up. At each measurement occasion clips are selected to represent a balanced and varied range of challenging clinical situations. Responses will be rated by psychology students trained in the FIS methodology and blind to treatment condition and time of assessment.

Secondary outcomes.

All secondary outcomes are pen-and-paper self-report and administered at baseline, post-intervention, and at two-year follow-up. For all measures, the main parameter of interest is change from baseline in overall mean score.

Self-efficacy will be assessed with an adaptation of the Counselor Activity Self-Efficacy Scales, CASES [50] to suit first-year students with limited experience and knowledge of psychotherapy. CASES is a 59-item measure of self-perceived capability to perform basic helping skills, manage session tasks, and negotiate challenging counseling situations and challenging issues. Items are scored on a 10-point Likert scale from 0 (no confidence) to 9 (complete confidence). The scale has been shown to be highly reliable and sensitive to change [50]. For the purpose of this study we selected 11 items that we thought our participants would be able to respond to, and rephrased the instructions from “Indicate how confident you are in your ability to use each of the following helping skills effectively, over the next week, in counseling most clients” to “In an imagined therapy situation, how confident are you in your ability to…”. One item was selected from the basic helping skills dimension (i.e., “capture and understand the messages that clients communicate”), four from the session management dimension (e.g., “help your client to understand his or her thoughts, feelings, and actions”), and six from the challenging client issues dimension (e.g., “client is extremely nervous”).

Students’ degree of worry will be assessed with the Penn State Worry Questionnaire, PSWQ [51]. A total of 16 items are scored on a 5-point Likert scale, from 1 (not at all typical) to 5 (very typical). A sample item is “I know I shouldn’t worry about things, but I just can’t help it”. The measure is widely used and has been shown to possess good psychometric properties and to tap a construct independent of other indicators of anxiety or depression [51].

The Rumination Response Scale, RRS [52], will measure rumination as a method of coping with negative mood. A total of 22 items are scored on a 4-point Likert scale, from 1 (almost never) to 4 (almost always). A sample items is “go away by yourself and think about why you feel this way”. The scale has been demonstrated to be reliable and valid, and to measure trait-like coping styles that are not confounded by state effects of depressed mood [53, 54].

Self-compassion will be assessed with the Forms of Self-Criticising/Attacking & Self-Reassuring Scale, FSCRS [55], a 22-item measure of self-criticism and self-reassurance. Items are scored on a 5-point Likert scale from 0 (not at all like me) to 4 (extremely like me). A sample item is “I find it easy to forgive myself”. An analysis of data from 12 individual studies utilizing this instrument [56] found it to be robust and reliable in both clinical and non-clinical samples, with established normative data for each population.

Additional measures.

Participants in the intervention group will be administered the following self-report measures at each apprenticeship occasion in order to examine changes in self-efficacy and stress over the course of the apprenticeship: A generic one-item, 10-point Likert scale of subjective stress immediately before attending treatment sessions (Instruction: “Notice how you’re feeling right now. Indicate on the scale below how much stress you’re experiencing in this moment”); four items from the session management dimension of the Counselor Activity Self-Efficacy Scales, CASES [50] (see description of the measure above), and a generic one-item, 10-point Likert scale of confidence in the therapist role (Instruction: “Indicate the alternative that you feel represents you best right now in your development as a therapist: How confident do you feel in the role as a therapist?”).

At baseline, a battery of questionnaires will be administered to participants in both conditions in order to examine predictors of FIS across conditions and potential moderators to the impact of the apprenticeship training. Personality will be assessed with the NEO Five-Factor Inventory, NEO-FFI [57], a 60-item measure of the five basic personality factors Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness. Items are scored on a 5-point Likert scale, from 1 (strongly disagree) to 5 (strongly agree). The NEO-FFI is a reliable, valid, and widely used personality measure [e.g., 58].

Self-assessed social skills will be assessed with the Social Skills Inventory, SSI [59], a 90-item measure of three types of verbal and non-verbal communication skills: expressivity, sensitivity, and control over communication. Items are scored on a 5-point Likert scale from 1 (not at all like me) to 5 (exactly like me). A sample item is “I am often told that I am a sensitive, understanding person”. The instrument has been reported to have good psychometric properties including test-retest reliability and convergent and discriminant validity [60].

Interpersonal functioning will be assessed with the Inventory of Interpersonal Problems, IIP-64 [47, 61], a 64-item measure of maladaptive relationship behavior. Items are scored on a 5-point Likert scale, from 0 (not at all) to 4 (extremely). Eight subscales correspond to octants in the Interpersonal Circumplex (domineering/controlling, vindictive/self-centered, cold/distant, socially inhibited, non-assertive, overly accommodating, self-sacrificing, and intrusive/needy). The inventory is widely used both in treatment and research and its validity and reliability are well documented [e.g., 62, 63].

Adult attachment quality will be assessed with the Experiences in Close Relationships–Short form, ECR-S [64]. A total of 12 items are scored on a 7-point Likert scale, from 1 (disagree strongly) to 7 (agree strongly). A sample item is “I find it difficult to allow myself to depend on romantic partners”. Like the original, 36-item version of ECR, the psychometric properties of ECR-S have been shown to be sound [64].

Capacity for mentalization will be assessed with the Mentalization Scale, MentS [65]. A total of 24 items are scored on a 5-point Likert scale, from 1 (completely incorrect) to 5 (completely correct). Three dimensions are assessed: self-related mentalization, other-related mentalization, and motivation to mentalize. A sample item is “Often I cannot explain, even to myself, why I did something”. The MentS is a relatively new inventory whose consistency, reliability, and validity has been reported to be adequate to good [65, 66].

Psychosocial functioning will be assessed with the Identity and Intimacy subscales (28 items) of the Measures of Psychosocial Development MPD [67]. The full, 112-item instrument assesses the positive and negative ends of each of Erikson’s eight psychosocial stages as well as the resolution of these stages. Items are scored on a 5-point Likert scale from 1 (not at all like me) to 5 (very much like me). Adequate to strong construct validity, test-retest and internal consistency was reported in the MPD manual [67].

The quality of the group climate in students’ practicum groups, which all students attend as part of training-as-usual, will be assessed with the Group Climate Questionnaire—Short form, GCQ [68]. A total of 12 items are scored on a 7-point Likert scale, from 1 (not at all) to 7 (extremely). Three subscales are assessed: Engagement, Avoidance, and Conflict. The GCQ is the most frequently used group process measure in the group psychotherapy literature and satisfactory reliability of the measure has been reported in several studies [e.g., 69, 70].

To examine and control for the potential impact of stress/heightened activation on performance on the FIS, students will be asked to complete a generic one-item measure of subjective stress (similar to the one administered at each apprenticeship occasion, see Secondary outcomes), immediately before the FIS test. Following the FIS test participants are asked to rate, on 10-point Likert scales, how much they feel that what they said would be helpful to the patient and to what degree they feel their responses represent their skills at a therapist, considering their current educational stage.

Qualitative data.

Written reflective logs will be completed by students in the intervention group immediately following each treatment session they attend. The logs serve the dual purpose of facilitating the learning process by encouraging students to reflect upon salient aspects of their apprenticeship experience, and of providing a session-by-session insight into the learning process. In the logs, students will be asked to describe and reflect upon their immediate experiences of participating in each particular treatment session. Students in the intervention group as well as therapist participants will be invited to participate in focus group interviews focusing on the overall experience and impact of the apprenticeship training. Each focus group will consist of four to seven students or therapists and a moderator. All interviews will be audio-recorded and transcribed verbatim.



Prospective student participants will be informed about the study and encouraged to volunteer in lectures at the university. Written, informed consent will be required for student participants and will be obtained by members of the research team prior to randomization. Therapist participants will be recruited by presenting the study to each of the various outpatient teams at the three clinics.

To facilitate recruitment, student participants in both conditions will be compensated with bus passes for the duration of the apprenticeship intervention. Control condition participants will be offered a workshop on interpersonal skills following their completion of the post-intervention measures. No data will be collected from this workshop; instead, it is intended as a compensation for the students’ participation.


Following informed consent and baseline measures, participants will be randomized 1:1 to the experimental or control condition. The randomization will be conducted by one of the members of the research team using a web-based randomization program for medical research ( Participants will be informed immediately about what condition they are assigned to. Blinding of participants or researchers will not be practically feasible.


Baseline data (-t1) will be collected prior to randomization and consists of the FIS skills test, administered by members of the research team, as well as several self-report measures of secondary outcomes and potential correlates of the FIS. See above for details about measures and outcomes. Session-by-session self-reported stress- and self-efficacy measures as well as reflective log entries will be completed by students in the intervention group at each clinic and immediately before and after each treatment session they attend (t1 –t10). Following the 10-week apprenticeship period (t11), students in both conditions will complete the FIS-test at the university, using different stimulus clips from those that were used pre-intervention, as well as self-report measures of secondary outcomes. At this point in time, students in the intervention group as well as therapists will be invited to focus group interviews. Follow-up data (t12) will be collected from students in both conditions two years after the completion of the apprenticeship condition, and consist of the FIS test as well as self-reported secondary outcomes.

Data management.

Participants will be given a numeric study ID which is identifiable by a coding list only accessible to the principal investigator. Data will be manually double-entered by research assistants who are blind to allocation.

Sample size

Sample size was determined based on parameters on the main outcome measure FIS from three samples [42, 43, 71]. A sample size of 100 yields an 80% probability for detecting a medium intervention effect (d = 0.5) with a one-sided alpha level of .05. Allowing for 20% attrition, we aim to include 120 student participants.

Statistics and data analyses

Quantitative outcomes will be analyzed using regression models in which post-intervention and follow-up scores will be predicted by condition, controlled for pre-intervention scores and other possible covariates. Longitudinal models will examine the development over time on the session-by-session stress and self-efficacy measures. Missing data will be handled with multiple implementation. Qualitative data will be thematically analyzed.

Data monitoring

There are no anticipated harmful consequences of participation in this study and therefore, no stopping guidelines are developed and no interim analyses are planned.

Ethics approval and consent to participate

The study will be conducted in accordance with the Declaration of Helsinki. It was approved by the Regional Ethics Committee in Mid-Norway (REK) as an extension of a larger research project (2012/1498). Participation is voluntary and written, informed consent is obtained. No patient data will be collected in the project; in interviews and written log entries, students and therapists will be instructed not to share any identifiable client information. Students in the intervention group will sign a confidentiality agreement prior to the apprenticeship intervention. Therapists will obtain verbal consent to have a student co-therapist present in sessions from those of their patients that they consider to be suitable. The therapists will be responsible for the treatment delivered.

Status and timeline

This publication is based on protocol version 1 (05-04-2021). Recruitment for the trial began on 09-20-2021, and is expected to be completed by October 2023.


This randomized controlled trial and qualitative investigation will explore the potential impact of applying an educational method widely practiced in other fields, the apprenticeship, to the training of future psychotherapists. We consider it to be a promising supplementary element to the traditional (e.g., coursework, clinical practice under supervision) as well as the newer (e.g., deliberate practice) methods of psychotherapist training. In their apprenticeships, students will be exposed to the complexity—and sometimes messiness and confusion—of psychotherapy as it is practiced in actual clinical settings. A variety of ways to respond to challenging interactions in therapy will be demonstrated by relatable role models. Importantly, students will have the opportunity to discuss and gain insight into what considerations lie behind these role models’ actions with their clients, as well as to try out their own therapeutic skills under close guidance. We expect this to be facilitative of students’ interpersonal skills, to increase their self-efficacy, decrease their performance anxiety and stress, and to advance their understanding of the practical psychological work. This would demonstrate a proof-of-concept by this study. We also expect it to be engaging and helpful for the experienced therapists to welcome student co-therapists into their therapy room.

Strengths and limitations

The study has several important strengths. The experimental intervention is apprenticeship in a very simple form. The randomized controlled design will allow us to examine any additive effects of apprenticeship as a single element to training-as-usual while decreasing the influence of systematic biases on results. The qualitative data from several sources will help provide a more nuanced understanding of both students’ and therapists’ apprenticeship experiences. Interviews and reflective logs will supplement the quantitative measures, allow for unexpected findings to emerge, and be vital in generating hypotheses for future studies. We consider a sample of first-year psychology students to be ideal in testing the impact of apprenticeship on interpersonal skills; they are untainted by previous training and may have fewer preconceptions and be less preoccupied with the technical aspects of the specific treatment models than more advanced students perhaps would. Possibly, this demonstration and practical experience in real-world psychotherapy at the very beginning of their careers will provide them with a strong starting point from which to absorb and assimilate all subsequent learning experiences more effectively. The main outcome measure, the FIS performance test, assesses skills in standardized situations in which client variability is controlled for, and has been demonstrated to be highly predictive of client outcomes [72]. It is designed to tap therapist behaviors that are likely to be of importance regardless of what specific treatment model therapists adhere to and as such, we expect results to be generalizable to therapists of all theoretical orientations.

The main limitation of apprenticeship as a training method is perhaps the unpredictability of clinical work. In this study, there will be no control over what clients the students meet or what happens in sessions with those clients. Students’ exposure to challenging interpersonal interactions will be completely at random. It is also very possible that the therapists’ demonstration of interpersonal skills will be less than optimal. Arguably however, examples of what not to do can also be very educational. Moreover, observing therapists performing less than optimally can be a healthy antidote for students who may have perfectionistic tendencies. It is our hope that this early exposure to the complexity of psychotherapy, in conjunction with other educational methods, will help prepare them to the unpredictability that characterizes real-life clinical work [73].

Some of the design choices we have made inevitably result in certain limitations. The rotation of students between different therapists is intended to counteract therapist effects and ensure that students are exposed to a variety of different clients, therapists, and ways of conducting therapy. Disadvantages of therapist rotation is that students will be less able to form a secure relationship to one role model or to follow the full course of treatment with one client. Moreover, the apprenticeship intervention is shorter in duration and less intensive than the full-time apprenticeships that are perhaps more commonly practiced in other fields. The rationale behind this is both feasibility–the intervention needs to fit into the ongoing study program as well as the day-to-day practice at busy treatment clinics–and the acknowledgement of the importance of the other educational methods that this intervention is meant to supplement, but not replace.


The aim of this study is to gain preliminary knowledge on a relatively simple and, hopefully, feasible training method intended to improve the treatment of mental health problems through the improved education of future therapists. Given the high prevalence and costs of mental health problems for individuals and society at large, we find the endeavor to educate more efficient psychotherapists of utmost importance.

Dissemination plans

Upon completion, results will be published in peer-reviewed journals and presented at research conferences and other relevant arenas for research dissemination. Authorship to all future publications will be granted in accordance with the Vancouver recommendations. Any modifications to this protocol will be reported in subsequent publications.

Supporting information


We are grateful for the enthusiasm and support for this project from students Department for Psychology, NTNU, and management and employees at the three participating clinics: Nidelv Community Mental Health Center, the Nidaros Community Mental Health Center, and the Clinic of Substance Use and Addiction Medicine, St. Olavs University Hospital. In particular, we wish to thank the reference group of experienced and highly esteemed therapists, Øyvind Watne, Per Knutsen, Jon Morten Landrø, and Anne Elisabeth Skjervold, for sharing their insights and advice in the planning stages of this study.


  1. 1. Baldwin SA, Imel Z. Therapist effects: Findings and methods. In: Lambert MJ, editor. Bergin and Garfield’s handbook of psychotherapy and behavior change. 6. Hoboken, NJ: John Wiley & Sons; 2013. p. 258–97.
  2. 2. Johns RG, Barkham M, Kellett S, Saxon D. A systematic review of therapist effects: A critical narrative update and refinement to review. Clinical Psychology Review. 2019;67:78–93. pmid:30442478
  3. 3. Callahan JL, Watkins CE Jr. Evidence-based training: The time has come. Training and education in Professional Psychology. 2018;4(12):211–8.
  4. 4. Sinclair S. Making doctors: an institutional apprenticeship. Berg. Oxford and New York: Berg; 1997.
  5. 5. Tracey TJ, Wampold BE, Lichtenberg JW, Goodyear RK. Expertise in psychotherapy: An elusive goal? American Psychologist. 2014;69(3):218–29. pmid:24393136
  6. 6. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations. Clinical psychology review. 2010;30(4):448–66. pmid:20304542
  7. 7. Beidas RS, Kendall PC. Training therapists in evidence‐based practice: A critical review of studies from a systems‐contextual perspective. Clinical Psychology: Science and Practice. 2010;17(1):1–30. pmid:20877441
  8. 8. Rakovshik SG, McManus F. Establishing evidence-based training in cognitive behavioral therapy: A review of current empirical findings and theoretical guidance. Clinical Psychology Review. 2010;30(5):496–516. pmid:20488599
  9. 9. Frank HE, Becker‐Haimes EM, Kendall PC. Therapist training in evidence‐based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice. 2020;27(3):e12330. pmid:34092941
  10. 10. Keum BT, Wang L. Supervision and psychotherapy process and outcome: A meta-analytic review. Translational Issues in Psychological Science. 2021;7(1):89–108.
  11. 11. Rousmaniere TG, Swift JK, Babins-Wagner R, Whipple JL, Berzins S. Supervisor variance in psychotherapy outcome in routine practice. Psychotherapy Research. 2016;26(2):196–205. pmid:25274037
  12. 12. Whipple J, Hoyt T, Rousmaniere T, Swift J, Pedersen T, Worthen V. Supervisor Variance in Psychotherapy Outcome in Routine Practice: A Replication. SAGE Open. 2020;10(1).
  13. 13. Rønnestad MH, Skovholt TM. The journey of the counselor and therapist: Research findings and perspectives on professional development. Journal of career development. 2003;30(1):5–44.
  14. 14. Okiishi JC, Lambert MJ, Eggett D, Nielsen L, Dayton DD, Vermeersch DA. An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome. Journal of clinical psychology. 2006;62(9):1157–72. pmid:16688682
  15. 15. Erekson DM, Janis R, Bailey RJ, Cattani K, Pedersen TR. A longitudinal investigation of the impact of psychotherapist training: Does training improve client outcomes? Journal of Counseling Psychology. 2017;64(5):514–24. pmid:29048197
  16. 16. Owen J, Wampold BE, Kopta M, Rousmaniere T, Miller SD. As good as it gets? Therapy outcomes of trainees over time. Journal of counseling psychology. 2016;63(1):12–9. pmid:26751153
  17. 17. Hill CE, Baumann E, Shafran N, Gupta S, Morrison A, Rojas AEP, et al. Is training effective? A study of counseling psychology doctoral trainees in a psychodynamic/interpersonal training clinic. Journal of Counseling Psychology. 2015;62(2):184–201. pmid:25602603
  18. 18. Goldberg SB, Rousmaniere T, Miller SD, Whipple J, Nielsen SL, Hoyt WT, et al. Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of counseling psychology. 2016;63(1):1–11. pmid:26751152
  19. 19. Ericsson KA. The differential influence of experience, practice, and deliberate practice on the development of superior individual performance of experts. In: Ericsson KA, Hoffman RR, Kozbelt A, Williams AM, editors. The Cambridge handbook of expertise and expert performance, 2nd ed. Cambridge handbooks in psychology. 2 ed. New York, NY, US: Cambridge University Press; 2018. p. 745–69.
  20. 20. Miller SD, Hubble MA, Chow D, Seidel J. Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy. 2015;52(4):449–57. pmid:26641375
  21. 21. Rousmaniere T, Goodyear RK, Miller SD, Wampold BE. The cycle of excellence: Using deliberate practice to improve supervision and training. New York, NY: John Wiley & Sons; 2017.
  22. 22. Di Bartolomeo AA, Shukla S, Westra HA, Shekarak Ghashghaei N, Olson DA. Rolling with resistance: A client language analysis of deliberate practice in continuing education for psychotherapists. Counselling and Psychotherapy Research. 2021;21(2):433–41.
  23. 23. Shukla S, Di Bartolomeo AA, Westra HA, Olson DA, Shekarak Ghashghaei N. The impact of a deliberate practice workshop on therapist demand and support behavior with community volunteers and simulators. Psychotherapy. 2021;58(2):186–95. pmid:32915009
  24. 24. Westra HA, Norouzian N, Poulin L, Coyne A, Constantino MJ, Hara K, et al. Testing a deliberate practice workshop for developing appropriate responsivity to resistance markers. Psychotherapy. 2021;58(2):175–85. pmid:32673002
  25. 25. Perlman MR, Anderson T, Foley VK, Mimnaugh S, Safran JD. The impact of alliance-focused and facilitative interpersonal relationship training on therapist skills: An RCT of brief training. Psychotherapy Research. 2020;30(7):871–84. pmid:32028859
  26. 26. Barrett‐Naylor R, Malins S, Levene J, Biswas S, Mays C, Main G. Brief training in psychological assessment and interventions skills for cancer care staff: a mixed methods evaluation of deliberate practice techniques. Psycho‐Oncology. 2020;29(11):1786–93. pmid:32281250
  27. 27. Hill CE, Kivlighan DM III, Rousmaniere T, Kivlighan DM Jr, Gerstenblith JA, Hillman JW. Deliberate practice for the skill of immediacy: A multiple case study of doctoral student therapists and clients. Psychotherapy. 2019;57(4):587–897. pmid:31448935
  28. 28. Chow DL, Miller SD, Seidel JA, Kane RT, Thornton JA, Andrews WP. The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy. 2015;52(3):337–45. pmid:26301425
  29. 29. Macnamara BN, Hambrick DZ, Oswald FL. Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis. Psychological science. 2014;25(8):1608–18. pmid:24986855
  30. 30. Corrigendum: Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis. Psychological Science. 2018;29(7):1202–4. pmid:29733772
  31. 31. Gowlland G. Apprenticeship as a model for learning in and through professional practice. In: Billett S, Harteis C, Gruber H, editors. International handbook of research in professional and practice-based learning. Berlin, Germany: Springer; 2014. p. 759–79.
  32. 32. Bandura A. Self-efficacy mechanism in human agency. American psychologist. 1982;37(2):122–47.
  33. 33. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychological review. 1977;84(2):191–215. pmid:847061
  34. 34. Bandura A. Self-efficacy: The exercise of control. New York, NY: W. H. Freeman; 1997.
  35. 35. Skovholt TM, Rønnestad MH. Struggles of the novice counselor and therapist. Journal of Career Development. 2003;30(1):45–58.
  36. 36. Hansen TI, Svendsen B, Hagen R. Worry related to clincial training of Norwegian psychology students. Tidsskrift for norsk psykologforening. 2010;47(6):505–10.
  37. 37. Rønnestad MH, Skovholt TM. The Developing Practicioner. Growth and Stagnation of Therapists and Counselors. New York, NY: Routledge; 2013.
  38. 38. Feinstein RE, Huhn R, Yager J. Apprenticeship model of psychotherapy training and supervision: utilizing six tools of experiential learning. Academic Psychiatry. 2015;39(5):585–9. pmid:25663305
  39. 39. Feinstein RE. Descriptions and Reflections on the Cognitive Apprenticeship Model of Psychotherapy Training & Supervision. Journal of contemporary psychotherapy. 2021;51(2):155–64. pmid:33235394
  40. 40. Balkhi AM, Reid AM, Guzick AG, Geffken GR, McNamara JP. The progress cascading model: A scalable model for teaching and mentoring graduate trainees in exposure therapy. Journal of Obsessive-Compulsive and Related Disorders. 2016;9:36–42.
  41. 41. Reid AM, Guzick AG, Balkhi AM, McBride M, Geffken GR, McNamara JP. The progressive cascading model improves exposure delivery in trainee therapists learning exposure therapy for obsessive-compulsive disorder. Training and Education in Professional Psychology. 2017;11(4):260–5.
  42. 42. Anderson T, Crowley MEJ, Himawan L, Holmberg JK, Uhlin BD. Therapist facilitative interpersonal skills and training status: A randomized clinical trial on alliance and outcome. Psychotherapy Research. 2016;26(5):511–29. pmid:26344392
  43. 43. Anderson T, Ogles BM, Patterson CL, Lambert MJ, Vermeersch DA. Therapist effects: Facilitative interpersonal skills as a predictor of therapist success. Journal of Clinical Psychology. 2009;65(7):755–68. pmid:19437509
  44. 44. Schöttke H, Flückiger C, Goldberg SB, Eversmann J, Lange J. Predicting psychotherapy outcome based on therapist interpersonal skills: A five-year longitudinal study of a therapist assessment protocol. Psychotherapy Research. 2017;27(6):642–52. pmid:28277042
  45. 45. Anderson T, McClintock AS, Himawan L, Song X, Patterson CL. A prospective study of therapist facilitative interpersonal skills as a predictor of treatment outcome. Journal of Consulting and Clinical Psychology. 2016;84(1):57–66. pmid:26594945
  46. 46. Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ: British Medical Journal. 2013;346:e7586. pmid:23303884
  47. 47. Alden LE, Wiggins JS, Pincus AL. Construction of circumplex scales for the Inventory of Interpersonal Problems. Journal of personality assessment. 1990;55(3–4):521–36. pmid:2280321
  48. 48. Anderson T, Patterson C, McClintock AS, McCarrick SM, Song X. Facilitative Interpersonal Skills Task and Rating Manual. Athens, Ohio: Ohio University; 2019.
  49. 49. Ryum T, Nordnes AM, Brattland H, Holgersen KH, Anderson T, Vogel PA. A Norwegian pilot study of the Facilitative Interpersonal Skills performance test. Tidsskrift for norsk psykologforening. 2021;58(4):256–65.
  50. 50. Lent RW, Hill CE, Hoffman MA. Development and validation of the Counselor Activity Self-Efficacy Scales. Journal of Counseling Psychology. 2003;50(1):97–108.
  51. 51. Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behaviour research and therapy. 1990;28(6):487–95. pmid:2076086
  52. 52. Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: A psychometric analysis. Cognitive therapy and research. 2003;27(3):247–59.
  53. 53. Nolen-Hoeksema S, Morrow J. Effects of rumination and distraction on naturally occurring depressed mood. Cognition & emotion. 1993;7(6):561–70.
  54. 54. Just N, Alloy LB. The response styles theory of depression: tests and an extension of the theory. Journal of abnormal psychology. 1997;106(2):221–9. pmid:9131842
  55. 55. Gilbert P, Clarke M, Hempel S, Miles JNV, Irons C. Criticizing and reassuring oneself: An exploration of forms, styles and reasons in female students. British Journal of Clinical Psychology. 2004;43(1):31–50. pmid:15005905
  56. 56. Baião R, Gilbert P, McEwan K, Carvalho S. Forms of self‐criticising/attacking & self‐reassuring scale: Psychometric properties and normative study. Psychology and Psychotherapy: Theory, Research and Practice. 2015;88(4):438–52.
  57. 57. Costa PT, McCrae RR. Professional manual: revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources; 1992.
  58. 58. McCrae RR, Costa PT Jr. A contemplated revision of the NEO Five-Factor Inventory. Personality and individual differences. 2004;36(3):587–96.
  59. 59. Riggio RE. Assessment of basic social skills. Journal of Personality and Social Psychology. 1986;51(3):649–60.
  60. 60. Riggio RE. The Social Skills Inventory (SSI): measuring nonverbal and social skills. The sourcebook of nonverbal measures: Going beyond words. 2005:25–33.
  61. 61. Horowitz L, Alden L, Wiggins J, Pincus A. Inventory of Interpersonal Problems. London, UK: Psychological Corporation; 2000.
  62. 62. Acton GS, Revelle W. Interpersonal personality measures show circumplex structure based on new psychometric criteria. Journal of personality assessment. 2002;79(3):446–71. pmid:12511015
  63. 63. Tracey TJ, Rounds J, Gurtman M. Examination of the general factor with the interpersonal circumplex structure: Application to the Inventory of Interpersonal Problems. Multivariate Behavioral Research. 1996;31(4):441–66. pmid:26788593
  64. 64. Wei M, Russell DW, Mallinckrodt B, Vogel DL. The Experiences in Close Relationship Scale (ECR)-short form: Reliability, validity, and factor structure. In: Manusov V, editor. Journal of personality assessment. 88. Mahwah, NJ: Routledge; 2005. p. 187–204.
  65. 65. Dimitrijević A, Hanak N, Altaras Dimitrijević A, Jolić Marjanović Z. The Mentalization Scale (MentS): A self-report measure for the assessment of mentalizing capacity. Journal of personality assessment. 2018;100(3):268–80. pmid:28436689
  66. 66. Richter F, Steinmair D, Löffler-Stastka H. Construct Validity of the Mentalization Scale (MentS) within a mixed psychiatric sample. Frontiers in Psychology. 2021;12:1–9. pmid:34149501
  67. 67. Hawley G. MPD: Measures of Psychosocial Development. Professional Manual. Odessa, Florida: Psychological Assessment Resources. Inc; 1988.
  68. 68. MacKenzie K. The clinical application of a group climate measure. In: Dies R, MacKenzie K, editors. Advances in group psychotherapy: Integrating research and practice. New York, NY: International Universities Press; 1983.
  69. 69. Tasca GA, Balfour L, Ritchie K, Bissada H. Developmental changes in group climate in two types of group therapy for binge-eating disorder: A growth curve analysis. Psychotherapy Research. 2006;16(4):499–514.
  70. 70. Kivlighan DM Jr, Lilly RL. Developmental changes in group climate as they relate to therapeutic gain. Group Dynamics: Theory, Research, and Practice. 1997;1(3):208–21.
  71. 71. Ryum T, Nordnes AM, Brattland H, Holgersen KH, Anderson T, Vogel PA. Fasiliterende interpersonlige ferdigheter. Pilot utprøving av en klinisk ferdighetstest i et norsk utvalg. Tidsskrift for Norsk Psykologforening. 2021;58(4):256–65.
  72. 72. Anderson T, Finkelstein JD, Horvath SA. The facilitative interpersonal skills method: Difficult psychotherapy moments and appropriate therapist responsiveness. Counselling and Psychotherapy Research. 2020;20(3):463–9.
  73. 73. Stiles WB, Honos‐Webb L, Surko M. Responsiveness in psychotherapy. Clinical psychology: Science and practice. 1998;5(4):439–58.