Conceptual framework

The conceptual framework guiding this study draws from work on professional development, teacher professional knowledge, and cognitive demand. Many factors converge to influence instructor teaching practices. Teaching practices influence students’ practices within the classroom, affecting student outcomes (Fig 1) [9–11]. In this study, institutional and course context does not vary, so this box has been greyed out in Fig 1. Instructors’ teaching practices are critical in directing students’ interactions and cognition in the classroom, and are the focal point of this framework [10].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Conceptual framework.

Adapted from Stein et al., [10] and Reeves et al., [12].

https://doi.org/10.1371/journal.pone.0262841.g001

Instructors’ teaching practices consist of the actions implemented to facilitate instruction, including lesson planning, in-person instruction, and assessment [12]. Instructors’ practices can be influenced by instructors’ professional knowledge, attitudes, and beliefs, as well as the curricular tasks. Curricular tasks provide the initial plan for the content, structure, and cognitive demand of the lesson [13] (Fig 1). The instructor’s professional knowledge consists of their understanding of pedagogical techniques, students’ ideas about the subject matter, and the concepts within the task [12, 14, 15]. Instructors’ professional knowledge, attitudes, and beliefs may influence how they plan and implement curriculum [12]. PD is designed to shift elements of instructors’ professional knowledge, attitudes, and beliefs to affect instructors’ teaching practices (Fig 1). However, changes in instructors’ knowledge, attitudes, and beliefs may not always lead to changes in teaching practices [7, 14, 16]. This study investigates the impact of PD and curriculum on TAs’ teaching practices following an instructional intervention that manipulates features of the curriculum and PD.

Literature review

PD has been well-studied in the context of K-12 teachers, and elements of effective PD for K-12 teachers have been identified, including: PD should be linked to classroom lessons and focus on subject matter content, PD should model teaching strategies that teachers are asked to use with their students, and PD should engage teachers in reflection [17–22]. The elements of effective PD identified for K-12 teachers are effective despite myriad school contexts, suggesting that these elements may also be effective in structuring TA PD [18, 20, 21].

In contrast to the large body of literature on K-12 teacher PD, fewer studies have investigated PD for TAs [12]. Most of these studies have examined changes in TAs’ attitudes and beliefs, or their self-perceptions of their knowledge and skills. These studies have found that PD can increase TAs’ teaching self-efficacy, knowledge of pedagogical strategies, and beliefs about how people learn [23–32]. Few studies have examined the effects of PD on TAs’ instructional practices [33–36], which is critical because changes in beliefs and knowledge are not always translated into changes in practices [7, 14, 16].

Many graduate students report that they do receive PD for teaching, but state that these PD programs do not provide adequate support for developing pedagogical skills [26, 37]. One-time, pre-semester workshops are the predominant mode of PD for TAs [37], despite studies suggesting that one-time workshops are often ineffective in changing instructors’ teaching practices [38]. Sustained, multi-session PD programs can be successful in shifting TAs’ ideas about teaching, and/or TAs’ teaching practices in the classroom [24, 25, 29, 35, 36, 39–41]. For example, multi-session PD programs have resulted in increased use of student-centered teaching practices [36], changes in TAs’ attitudes and ideas about teaching [29], and TAs reporting increased knowledge of topics covered during PD [39]. Many of these programs run for a full semester, requiring double-digit hours of contact time between TA participants and PD facilitators. While effective, this lengthy time commitment may pose a barrier for implementation of similar programs at many institutions [42]. It is necessary to identify effective PD strategies that can be implemented within time constraints that are a reality for participants and facilitators.

Multiple reports of sustained, multi-session PD for TAs describe programs that are directly linked to the curriculum for a specific course [31, 34–36, 43]. Most of these programs exceeded 20 hours (average 37 hours). Three studies investigated the effect of course-linked PD on TAs’ teaching practices, and all found evidence that TAs incorporated student-centered practices learned during PD into their teaching [34–36]. These studies did not examine whether TAs implemented these student-centered teaching practices in lessons that were not related to PD [34–36]. Situating PD within the instructors’ teaching context is also an evidence-supported strategy for structuring effective PD for K-12 teachers [18, 44]. However, few studies across the K-16 spaces have investigated to what extent curriculum-linked PD effectively prepares instructor participants to transfer newly developed ideas and practices to teaching curriculum that was not linked to PD.

Student-centered curricula push students to engage with core concepts and practices of a discipline through scientific inquiry [1, 2, 45]. This style of instruction asks students to construct explanations of phenomena and use evidence to support claims, increasing the cognitive demand on students [10, 13, 45, 46]. Curricular tasks exist on a spectrum of cognitive demand. To compare cognitive demand across curricular tasks, the Task Analysis Guide in Science (TAGS) framework breaks this spectrum of cognitive demand into five levels [45].

An example of a high-cognitive-demand task (level 5 of 5) is one where students tackle an open scientific question with very little prescribed structure or directives from the instructor (described as Doing Science by Tekkumru-Kisa et al. [45]). This type of task requires students to use discipline-relevant practices to deepen their understanding of a scientific phenomenon [10, 45, 47, 48]. An example of a low-cognitive-demand (level 1 of 5) task is a memorization activity, where students are required to remember previously introduced definitions of scientific content or practices. Memorization does not require students to have a deep understanding of the content or practices, and can provide opportunities for students to practice new skills or remember basic facts [10, 45]. However, low-cognitive-demand tasks are less effective in helping students generate and retain knowledge [49]. These examples represent the highest and lowest ends of the cognitive demand spectrum, respectively. Additional gradations of cognitive demand exist between these endpoints, with less prescriptive structure and more space for inquiry occurring as tasks progress from lower- to higher-cognitive demand (Table 1) [45].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Descriptions and examples of TAGS levels.

https://doi.org/10.1371/journal.pone.0262841.t001

There may be an association between cognitive demand on students and teaching practices [11, 50]. Cognitively demanding tasks can result in frustration while students grapple with unfamiliar concepts or contexts. Some instructors respond to this frustration by taking over challenging elements of tasks, thereby reducing cognitive demand and subsequent opportunities for student learning [11]. Implementing and maintaining student-centered pedagogical practices is associated with maintaining a high level of cognitive demand [10, 11]. Instructors’ teaching practices have a clear impact on the enacted cognitive demand of a task [10, 46, 51]. However, the relationship between planned cognitive demand (as written in curricular materials) and instructors’ enacted teaching practices is underexplored.

Study context and objective

In this study, we implement a multi-session curriculum-linked PD program that is relatively short in duration (4 hours). Elements shown to be effective for K-12 teachers and TAs (modelling of teaching practices, structured opportunities for TAs to reflect on the modelled teaching practices, and a direct link to classroom content) were included in this PD intervention [18, 21, 31, 36]. This PD intervention is explicitly linked to a companion curriculum intervention consisting of five short (30-minute) tasks that supplement regular laboratory exercises in an undergraduate introductory biology laboratory course [52]. These tasks are designed to guide student inquiry and promote TAs’ use of student-centered teaching practices to facilitate the activities. Part of this curriculum focuses on statistics, a difficult topic for both students and experts [53, 54], further necessitating PD that incorporates both the implementation of teaching strategies and the curriculum subject matter [55, 56]. We first investigate the effectiveness of this reduced-length, multi-session Intervention PD on TAs’ teaching practices in two instructional contexts: when teaching PD-linked Intervention curriculum, and when teaching Traditional curriculum that was not linked to PD. We then investigate the relative contributions of the PD intervention and cognitive demand of curricular tasks (which varies within the Intervention and Traditional curriculum) on TAs’ teaching practices. Specifically, this study aims to answer these research questions:

1. To what extent do the intervention curricular activities and PD prepare TAs to implement student-centered teaching practices in the classroom?
2. To what extent do TAs that participated in the curriculum/PD intervention implement student-centered teaching practices when teaching curriculum that was not linked to PD?
3. To what extent do PD and the cognitive demand of the curricular tasks impact teaching practices of TAs when controlling for curriculum type?

Study design

This study was conducted within a larger study on the impact of a curriculum intervention within an undergraduate introductory biology laboratory course across three semesters at a large, American, Ph.D.-granting university. This study was approved by the University of Minnesota Institutional Review Board ID#: STUDY00003137. Consent followed the process approved by the IRB. TAs were provided with information about the study on the BioVEDA survey and an introductory email. They were asked on the BioVEDA survey whether they agreed to have their observation and survey data included in the study. TAs were also informed that they could opt out of the study at any time by contacting one of the researchers.

The course was divided into laboratory sections each containing approximately 20 students. Half the sections were randomly designated as ‘Traditional’ instruction sections, and the remaining sections were designated as ‘Intervention’ sections. TAs in Intervention sections (N = 21; hereafter, ‘Intervention TAs’) implemented five short (~30 minute) Intervention curricular tasks in addition to the typical laboratory exercises. TAs in Traditional sections (N = 21; hereafter, ‘Traditional TAs’) only implemented the typical laboratory exercises. All TAs participated in course preparation meetings designed to support procedural implementation of the typical laboratory exercises. Intervention TAs also participated in four additional hours of PD that was designed to support implementation of Intervention tasks, while Traditional TAs participated in time-matched PD that focused on inclusive teaching (Intervention and Traditional curriculum and PD are described below). Fig 2 presents an overview of the study.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Study design.

All TAs participated in course preparation meetings and taught the traditional laboratory exercise curriculum. TAs participated in Traditional or Intervention supplemental PD. Intervention TAs taught supplemental Intervention curriculum activities. Cognitive demand level is measured by the Task Analysis Guide in Science (TAGS; see the ‘Cognitive Demand’ section for detail). All participants were observed, and audio recorded while teaching. Recordings were subsequently analyzed using the Reformed Teaching Observation Protocol (RTOP; see the ‘Teaching Practices’ section for detail).

https://doi.org/10.1371/journal.pone.0262841.g002

This study was a collaboration between the course coordinators and the research team. The research team designed the Intervention curriculum and PD. The course coordinators designed the Traditional curriculum and PD. The research team had no day-to-day involvement in hiring TAs, the Traditional course curriculum or facilitation of laboratory preparation meetings.

Participants

Study participants included one teaching specialist and 41 graduate students who were paid TAs for an introductory biology laboratory course. TAs were randomly assigned to Traditional or Intervention sections by a course coordinator who was not directly involved in this study. There were no statistically significant differences between Traditional and Intervention TAs in the number of years in graduate school, TA experience, or gender (Table 2). Intervention TAs scored higher than Traditional TAs on a pre-semester content knowledge assessment (BioVEDA, described below).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Participant demographic information.

https://doi.org/10.1371/journal.pone.0262841.t002

Curriculum

The Traditional core curriculum consisted of largely scripted laboratory exercises based on a lab manual detailing procedures that students completed in class (hereafter, the ‘Traditional curriculum’). Content covered by the Traditional curriculum included laboratory techniques, experimental design, and data analysis. All TAs (Traditional and Intervention) taught the Traditional curriculum. Instructional style was not prescribed for TAs.

The Intervention curriculum consisted of five short tasks that supplemented the Traditional curriculum during the first third of the semester in Intervention laboratory sections. The supplemental Intervention tasks comprised approximately 3% of the total instructional contact hours across the semester, and the remaining 97% of instructional time was spent on Traditional tasks. The content covered by the Intervention curriculum complemented the laboratory exercises specified in the Traditional curriculum. A detailed explanation of the Intervention curriculum is published elsewhere [52]. Table 3 summarizes features of the Traditional and Intervention curricula.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Features of traditional and intervention curricula.

https://doi.org/10.1371/journal.pone.0262841.t003

For Intervention curricular tasks, Intervention TAs were provided with instructor supports modelled after educative curricular materials (ECM) used for K-12 teachers [57]. The ECM consisted of an overview detailing the learning objectives and design rationale for the task; lesson plans describing how to implement each task with callout boxes highlighting information about pedagogical techniques, student thinking, or specific learning goals; and copies of student worksheets (see example ECM in the S1 File).

Professional development

All TAs attended weekly preparation meetings (~25 hours per semester) where course coordinators lectured on logistics, procedures, and content. All TAs also participated in 4 hours of Traditional or Intervention PD that occurred over 4 sessions during the first third of the semester.

Traditional and Intervention PD differed in topic and structure. Traditional PD focused on shifting the attitudes and interest of TAs and their students about course content to make laboratory exercises more inclusive and accessible. Targeted course content varied by semester and was chosen by the course coordinator. In these sessions, TAs learned about teaching techniques and then designed ways to implement them in the classroom.

Intervention PD focused on preparing TAs to teach the Intervention curricular tasks and aimed to elevate TAs’ knowledge of subject matter within the tasks, and knowledge of teaching practices that should be used during task enactment. Facilitators presented each task’s learning goals. Trainings were structured as modelled versions of curricular tasks, with facilitators acting as TAs, and TAs acting as students. Facilitators modelled teaching strategies that TAs could use to enact the tasks. TAs were asked to reflect on their thought process as they participated in the tasks, note any difficulties they had, anticipate areas of difficulty for students, and reflect on teaching practices that facilitators implemented.

The inclusion of modelled teaching practices linked to specific tasks and opportunities for reflection are evidence-supported practices in K-12 PD and represent key differences between Intervention and Traditional PD [18, 21]. While both groups received instruction on teaching practices, Traditional TAs did not experience these practices as students or see an example of how one might implement these practices.

Intervention PD facilitators were members of the research team and part of the curriculum development team. These facilitators consisted of a Ph.D. student in a STEM Education program and a postdoctoral associate with a biology Ph.D. and one year of biology education research experience. Traditional PD facilitators consisted of a course coordinator (teaching faculty) and a member of the research team (education research faculty, who was also involved in development of the Intervention curriculum).

Data collection

To examine how PD and curriculum impacted classroom practices, TAs were audio recorded teaching Traditional and Intervention curricular tasks (a maximum of two times per semester). TAs were assured that the research team would not share any identifiable information with persons involved in managing or hiring for TA appointments. Traditional TAs were recorded teaching Traditional tasks and Intervention TAs were recorded teaching both a Traditional and an Intervention task.

Measures

The Reformed Teaching Observation Protocol (RTOP) was used to measure teaching practices that TAs enacted in the classroom [58]. The RTOP was chosen because it accounts for instructor and student behaviors in a detailed way that goes beyond broad categorization of actions. The RTOP has been extensively validated and used to analyze the degree of student-centered teaching used by K-12 teachers, university faculty, and teaching assistants [14, 36, 58, 59]. The RTOP is a holistic measurement tool containing 25 items that are rated on a 0-4-point scale: 0 indicates the item never occurred, and 4 indicates the item was very characteristic of the lesson. Total RTOP scores (summed across all 25 items) were used to measure TAs’ instructional practices.

A research assistant who was not involved in the design or implementation of the curriculum or PD was the primary coder of the audio recordings of classroom lessons. A second coder independently rated 45% of the audio recordings of classroom lessons. The second coder was involved in the design and implementation of the curriculum and PD. Both coders were blinded to the identity of the TAs, their Traditional/Intervention group status, and Traditional/Intervention designation of the recorded curricular task. The intra-class correlation coefficient for the two coders is 0.75 (95% CI = [0.68, 0.81]), indicating excellent inter-rater agreement [60].

This study assessed the cognitive demand of the tasks that were taught by TAs during the recorded lessons. Cognitive demand was determined by using the Task Analysis Guide in Science (TAGS) framework [45] to analyze the curricular materials for that task (e.g. lesson plans, student laboratory manuals and/or worksheets). Tasks were subdivided into activities which were categorized using TAGS by two independent coders (67% agreement). An overall cognitive demand score for each task was calculated by averaging the cognitive demand levels across individual activities within the task and categorizations were averaged across two independent coders. On a five-point scale, the planned cognitive demand of the tasks in the curricular materials ranged from an average of 2 (scripted tasks) to 4 (guided tasks that integrate scientific content and scientific practices). Table 1 provides an example of a task at each of these TAGS levels. Most tasks were either a level 2 or 3.

The Biological Variation in Experimental Design and Analysis (BioVEDA) assessment was used to measure TA content knowledge because it can be used to evaluate respondents’ understanding of variation in experimental design and data analysis, which is a learning goal for this course [61]. All TAs completed the BioVEDA assessment before the start of the semester.

Analyses

Differences in RTOP scores across PD and curriculum type were examined via one-way ANOVA. Assumptions of ANOVA were met (independence and normality of samples, homogeneity of variance). Post-hoc analyses were conducted via Tukey’s HSD. Effect sizes have been provided for comparisons between groups (Cohen’s d).

To investigate the relative impact of the supplemental PD (Traditional or Intervention) and cognitive demand of the task (measured by TAGS) on classroom teaching practices (measured by RTOP) while controlling for curriculum type (Traditional or Intervention), linear models were fitted to the data using forward and backward stepwise regression. Assumptions of linear regression (linear relationship of residuals, normally distributed residuals, homoscedasticity of residuals, absence of highly influential cases) were considered satisfied after graphical examination. The assumption of multicollinearity was met (Variance Inflation Factor (VIF) is 3.6) [62].

An alpha level of .05 was used to determine statistical significance for all analyses. Data used in these analyses has been provided in a CSV file in (S1 File).

Implications

These results suggest ways to prioritize resource investment when designing TA PD. While pre-semester orientations are a popular, low resource-investment form of PD [37], studies indicate that one-time workshops are less effective than PD that takes place in multiple sessions over a longer time period [18, 38]. Multi-session, curriculum-linked PD programs have been effective in shifting TAs’ teaching practices for the curriculum covered during PD [34–36]. However, multi-session, curriculum-linked PD in published studies typically requires more time investment and is tailored to a specific group of instructors [31, 34–36] Thus, there are concerns about investing resources in preparing TAs for a narrow range of teaching experiences. Curriculum-linked PD may be necessary though when the content is unfamiliar to instructors [55, 56]. The Intervention PD described here focused exclusively on teaching Intervention curricular tasks, yet Intervention TAs still showed gains in teaching practices when teaching Traditional curricular tasks compared to Traditional TAs. These results show that curriculum-linked PD can be short and still be effective. Moreover, TAs can transfer teaching practices learned in curriculum-linked PD to teach other curricular tasks that were not covered during PD suggesting that curriculum-linked PD can be an effective investment of resources.

Student-centered teaching practices facilitate the implementation of a higher-cognitive demand curriculum [11] and this study shows that the cognitive demand of the curriculum is associated with greater use of student-centered teaching practices. Combined, these studies suggest that to be most effective, PD should be accompanied by redesign of materials so that they are cognitively demanding. This may be one way to support both instructors and TAs in changing to a more student-centered pedagogy, which has been associated with improved student performance [70].

Study limitations and future directions

This study presents evidence indicating that Intervention PD was a significant predictor of TAs’ implementation of more student-centered teaching practices. However, several features of the Intervention PD differed from features of Traditional PD. Therefore, teasing apart the effects of individual features of the Intervention PD awaits further research. Moreover, the study reported here was for one curriculum context (statistics in a laboratory setting) at one institution. Additional studies are needed to see if the findings generalize across contexts and institutions.

Individual TAs have differing experiences, motivations, and abilities regarding teaching [10]. Any of these factors could have affected their teaching. This study showed that content knowledge did not have an impact on TA teaching practices, in agreement with other studies [55–62]. Future studies are needed to explore the impact of the other factors (including PCK) on TAs’ teaching practices after PD. Additionally, individual TAs may experience different motivations and/or comfort with the content on different days and this could impact their teaching. One limitation of this study is that the claims are based on one to two samples of each individual’s teaching. However, claims are based on observations of many TAs mitigating against individual variation. Examining the extent of variation in individual TAs teaching and the factors responsible for that variation is an important avenue for future research.

This study suggests that TAs are transferring knowledge learned during Intervention PD to activities that are not related to the PD sessions. This inference is made based on observations of teaching analyzed using the RTOP, which limits the ability to draw stronger conclusions about how TAs may be applying knowledge about teaching gained from PD to novel activity contexts. Future studies should include qualitative analysis of interviews with TA participants to gain greater insight into TAs’ choices about whether and how to apply student-centered teaching practices to activities that were not part of PD.