https://orcid.org/0000-0003-4845-5170
Figures
Abstract
Myriad studies support the claim that active learning improves student academic performance in STEM, yet lecture remains the dominant form of instruction. Many faculty offer multiple reasons for not using active learning with many expressing confusion as to what active learning is. In an effort to better understand how specific elements of active learning might contribute to enhanced academic performance, we used a more fine-grained classroom observation tool, PORTAAL, to observe teaching practices across 146 STEM courses. We used a principal component analysis to identify specific sets of teaching practices and we found a continuum of implementation of teaching practices that ranged from instruction-centered to learner-centered. The instruction-centered practices included high Bloom’s level questions, students working alone, instructors answering and explaining questions, and providing alternative answers. The learner-centered practices were volunteer or randomly-called students explaining answers, instructors prompting students’ logic, and instructors giving positive feedback. Using a linear mixed model to analyze the data from all courses, we found medium and medium-high levels of learner-centered teaching practices correlated with higher student exam performance compared to instruction-centered practices. We also analyzed interactions of both binary gender and first generation status with learner-centered teaching on exam scores. We propose that when instructors use these learner-centered practices, they shift the responsibility of the intellectual work of the problem—the learning—to the students. As even moderate levels of these learner-centered practices have a positive correlation with exam performance, faculty may incorporate these few key elements of learner-centered practices into their own teaching to enhance student learning.
Citation: Jackson MA, Liu H, Moon S, Doherty JH, Wenderoth MP (2026) Use of a specific set of learner-centered evidence-based teaching practices correlates with higher exam performance across seven STEM departments. PLoS One 21(3): e0327269. https://doi.org/10.1371/journal.pone.0327269
Editor: Mohammed Saqr, University of Eastern Finland: Ita-Suomen yliopisto, FINLAND
Received: June 23, 2025; Accepted: February 17, 2026; Published: March 20, 2026
Copyright: © 2026 Jackson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: All support for this study was provided to authors M.P.W. and J.H.D. by the National Science Foundation (https://www.nsf.gov/) DUE 1725149. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of themanuscript. There was no additional external funding received for this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
There are myriad studies that support the claim that active learning improves student academic performance in Science, Technology, Engineering and Mathematics (STEM) [1–9] and non-STEM courses [10–14]. Though there is strong evidence to support the benefits of implementing active learning in the STEM classroom, lecture remains the dominant form of instruction in STEM [15–17]. Faculty provide many reasons for why they have not implemented active learning in their classrooms. Some state that these new teaching methods consume too much class time at the expense of covering course content [18]. Others cite a lack of incentive to invest the time needed to redesign their courses and to learn a new teaching method. Finally, there is a large cohort who express confusion as to what active learning is and what it looks like in practice [19–23].
“What is active learning?” is, in fact, an important question. Recent articles by Driessen et al., [24] and Lombardi et al., [25] acknowledge that active learning is often poorly defined by education researchers and can encompass a wide variety of teaching methods. Two of the most highly cited meta-analyses on the impact of active learning on student academic performance acknowledge that active learning was often minimally described in the papers in their study [1,2]. Studies were coded as active learning if they meet the following criteria: “Active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert. It emphasizes higher-order thinking and often involves group work” [1]. Given these broad criteria and that many of the included studies offered vague descriptions of the type of teaching in the study, the actual type of active learning used in each study was not analyzed. A follow up meta-analysis of 41 studies published between 2010 and 2016 found that a high intensity of active learning (use of active learning > 66% of class time) improved student exam performance more than medium or low levels of active learning [2]. This second study collected and analyzed different types of named active learning methods such as flipped classes, in-class group work, and worksheets. However, the authors urged caution in interpreting results as the active learning types reported in the studies were “author-defined and represent general characteristics’‘ [2].
In their 1991 book, Active Learning: Creating Excitement in the Classroom, Bonwell and Eison defined active learning as “a method of learning in which students are actively or experientially involved in the learning process” [26]. The philosophical foundation of active learning is socio-cultural constructivism which posits that learners acquire new knowledge by working with others to construct new understanding through social discourse and integrating that understanding into their existing bodies of knowledge [27,28]. Basic characteristics of active learning that Bonwell and Eison noted are that students need to be involved in doing things and thinking about what they are doing. That “doing” can be reading, writing, discussing or solving problems and students need to be engaged in the challenging tasks of analysis, synthesis and evaluation [26]. Since Bonwell and Eison’s seminal work in 1991, a multitude of named active learning methods have been created. These methods include but are not limited to Cooperative Learning [29], Collaborative Learning [30], Team Based Learning [31], Problem Based Learning [32,33], Peer Instruction [34], Jigsaws [29], think-pair-share [35], in-class clicker questions [36], Process Oriented Guided Inquiry Learning (POGIL) [37], high structure classes [38], and flipped classrooms [39].
As researchers seldom fully describe the active learning methods they are assessing in their research, it is challenging to determine if one method of active learning is more effective at enhancing student learning than another. As a result of this lack of clarity, there have been calls to begin second-generation research on active learning practices in undergraduate classrooms [1,40]. This new area of research would not only identify key elements of active learning teaching practices but also determine the dosage of those elements that correlate with increased student exam performance across demographic groups of students.
In an effort to better understand how certain elements of active learning might contribute to enhanced student academic performance, we created and validated the Practical Observation Rubric To Assess Active Learning (PORTAAL) [41]. To develop PORTAAL, we surveyed the education research literature and identified elements of active learning that when tested individually against more traditional passive learning methods had evidence of significant improvement in student academic performance. PORTAAL identifies the elements of active learning that can be observed in the classroom that are research-supported teaching methods. Therefore, throughout this paper we will use the term “PORTAAL practices” to denote teaching practices that are associated with active learning and that are evidence-based.
Previously, we used PORTAAL to collect data from a large set of introductory biology courses at an R1 institution to determine which individual PORTAAL practices correlated with improved student exam performance [42]. We found that student exam scores correlated with the use of four individual PORTAAL practices: small group work, randomly calling on students for answers, providing alternative answers to questions, and the total time students engaged in class. As the use of these practices increased, so did student exam scores. We were aware that instructors were implementing more than one of the PORTAAL practices during their class sessions. However, given our sample size, we were unable to analyze if instructors preferentially used different sets of PORTAAL practices and if those sets of practices had a differential impact on student academic performance.
To expand on our previous research, we conducted an analysis to determine if we could identify sets of PORTAAL practices used in 146 upper- and lower-division courses across seven STEM departments. We reasoned, if we could identify sets of PORTAAL practices, we could investigate how these sets and the intensity of use of these sets of PORTAAL practices correlate with student exam performance. Results from this type of study could better inform faculty as to which teaching practices they might implement and how often to use them to potentially enhance student learning. The research questions for this study are as follows:
- RQ1. Can we identify sets of correlated PORTAAL practices based on their use in courses across seven STEM departments?
- RQ1a: Does the use of these sets of PORTAAL practices vary by STEM department?
- RQ2. Does the use of these sets of PORTAAL practices correlate with student exam performance?
- RQ2a. Do the correlations vary by student demographic groups?
Methods
Data collection
Classroom observation data.
Human subjects research was conducted with the approval of the Institutional Review Board at the institution where the study took place (STUDY00002830). Participants were recruited between July 30, 2017 and March 20, 2020. Faculty participation in the study was voluntary and documentation of consent was waived for faculty. Consent was waived for students. Students had an opt-out procedure as described in the IRB protocol if they did not want their data included in the study.
A subset of faculty from seven STEM departments (biology, chemistry, computer science and engineering [CSE], mathematics, physics, psychology, and public health) volunteered to have their fully in-person courses observed as a part of this study. A total of 146 lower- and upper-division courses, spanning from Winter Quarter 2017 to Winter Quarter 2020, were included in the study. Lower-division courses ranged from approximately 20–700 students per course with a median of 184 students, whereas upper-division courses ranged from approximately 20–240 students per course with a median of 71 students.
Classroom observations of the 146 courses were conducted in the same manner as in Jackson et al. [43]. Video recordings of class sessions were collected using lecture capture technology. On average, four videos were randomly selected to be coded from each course, totaling approximately 600 individual video recordings. We randomly selected the videos by dividing the ten-week quarter into four equal time periods excluding the first and last week of the quarter and randomly selecting one video from each time period. Pairs of trained researchers coded classroom videos using the PORTAAL classroom observation tool [41]. The PORTAAL coding rubric and manual can be found in the supporting information (S1 & S2 Appendices). Two coders individually coded the entire video, then met to discuss any disagreements in coding and came to complete consensus on all codes. When coding using the fine-grained PORTAAL rubric, every second of a class session is given at least one code and there is no single PORTAAL score for each class session. Therefore, we were not able to calculate inter-rater reliability between coders. After coding, we calculated the average value for each of the PORTAAL practices in each course by totaling the number or duration of each practice and dividing by the number of videos coded for that course. To standardize comparisons, these averages were adjusted to a 50-minute class session, aligning with the institution’s standard class length.
Given the presence of over 30 individual PORTAAL practices in the standardized dataset, we aggregated closely related practices and removed those that were infrequently observed to ensure the power and interpretability of the analysis (S1 Table). First, we aggregated some of the practices as they measured similar classroom behaviors. For example, students’ volunteered responses (“Vol_Ans_Exp”) represents the combined number of times two similar events occurred in class: students volunteered answers or students volunteered an explanation for their answers (see S1 Table for further explanation of PORTAAL practices). Second, we removed variables that were rarely observed in the sample of 146 courses. For example, instructors providing negative feedback to students was seldom observed across all of the courses, so it was not included in the final set of practices. The final set of ten PORTAAL practices used for analysis is found in Table 1.
Exam data.
We collected student exam scores from each of the courses in the study. The number of exams per academic quarter ranged from one to ten, though the majority of courses had three to five exams per quarter. Prior to the analysis, we calculated an average standardized exam score for each student per course by totaling the number of points each student earned on all exams in the course, dividing by the total possible exam points in the course, and multiplying by 100 to standardize the score. Boxplots of standardized exam score distributions for each course by STEM department are shown in Fig 1.
Some students enrolled in multiple courses and therefore had multiple course records and associated PORTAAL values in our dataset. We will refer to each instance of student data and the associated PORTAAL values from that course as a student data point. Student data points with any missing exam scores were removed prior to analysis. There was a total of 27,114 student data points in the 146 courses included in the analysis. These student data points represented 15,990 distinct students; 43% of students took more than one course. Information concerning the number of student data points, the total number of courses and introductory courses among them, and the number of instructors for each department can be found in S2 Table. The number and percentage of upper- and lower-division courses can be found in S3 Table.
Student demographic information.
Student demographic information and GPA at the start of the quarter were obtained from the university registrar. Demographic factors included self-identified binary gender, if the student was the first in their family to graduate from college (first generation status or FGN), if their race/ethnicity is underrepresented in STEM (underrepresented minority or URM: e.g., Hispanic/Latino, Black/African-American, Pacific Islander, Native Hawaiian, Native American), and if they qualified for the Educational Opportunity Program for students identified by the university as economically or educationally disadvantaged (EOP). Descriptive statistics of students in each group are reported in the supporting information (S4 Table).
Data analysis
Principal component analysis.
Prior to conducting a principal component analysis (PCA), we first computed a pairwise correlation matrix to assess collinearity among PORTAAL practices (S1 Fig). Although most practices were positively correlated, no correlation exceeded 0.8, suggesting no severe multicollinearity. Consequently, all selected PORTAAL practices were included in the PCA.
To identify potential underlying sets of correlated PORTAAL practices, we conducted principal component analysis (PCA) as an exploratory analysis. PCA is a dimension reduction approach that summarizes the variation in the original PORTAAL practices using a small number of new variables called principal components (PCs). Each PC is a linear combination of the original variables, and the PCs are uncorrelated with each other. The first few PCs capture the majority of variation within the original data. In the context of our analysis, these PCs could be interpreted as teaching patterns that are characterized by a set of PORTAAL practices. We conducted the PCA analysis using the stats package in R (version 4.0.2).
We performed the PCA on the PORTAAL practices data at the course level. Each data point in the PCA analysis corresponded to a distinct course, with 146 courses in total. We examined the contributions of the PORTAAL practices to the top PCs by inspecting the loading matrix, and further examined how the courses clustered by department based on the PC values. We used a variance cutoff of 60% to decide the number of PCs to retain in our analysis. Based on this cutoff value, we used the first two PCs (PC1 and PC2) from the PCA analysis of PORTAAL practices as variables in subsequent regression models.
PC1 and PC2 are a continuum of values representing variation in the number of times PORTAAL practices were used in the courses. PC1 represents the number of times each PORTAAL practice was used (i.e., intensity of use), while PC2 represents a constellation of practices that were more often used together (i.e., teaching mode). As we were interested in how broad, categorical levels that could represent low, medium, medium-high, and high use of the sets of teaching practices correlated with exam scores, we binned the observed values of each PC into quartiles to be used in the subsequent analyses. For both PC1 and PC2, quartile 1 (Q1) represents the lowest values, quartile 2 (Q2) represents medium values, quartile 3 (Q3) represents medium-high values, and quartile 4 (Q4) represents the highest values.
Regression modeling.
We performed regression modeling using linear mixed models (LMM) to determine how PC1 and PC2 correlated with student exam scores. We conducted the analyses with LMM models in R (version 4.0.2) using the lme4 and lmerTest packages. The PC1 and PC2 quartiles (tp_PC1_quartile and tp_PC2_quartile in the formula below) were included as predictors of interest and treated as fixed effects, and the average exam score (Average) was the outcome. We included four intercept random effects at different clustering levels to account for the semi-nested structure of the data, where student exam scores within the same cluster could be correlated with each other (Fig 2). First, to account for the correlation among exam scores from students taught by the same instructor, we included the instructor ID (Instr_ID) as a random effect. Secondly, to account for the correlation among exam scores from the same course, we included the course code (Course_Short) as a random effect. Furthermore, the same course could be offered in multiple academic quarters/terms, so we included a random effect for the unique offering of a course during a quarter (Course_Short_Quarter) to account for the correlation among exam scores from the same course in the same term. Finally, to account for the correlation among exam scores from the same student, we included the student ID (ID) as a random effect.
In several cases, a single instructor taught more than one course, and some courses were taught by multiple instructors, either within the same term or in different terms. Additionally, students could be enrolled in more than one course. As a result, the data had a semi-nested structure rather than a strictly nested one. We included the following covariates as fixed effects in the regression model: students’ incoming GPA (GPA_Start) for each course, gender (Gender), EOP status (EOP), URM status (URM), first generation status (FGN), and course level (upper- or lower-division; Course_Level). As different departments could adopt systematically different teaching approaches and have distinct exam difficulties, the department was likely a confounding factor in the association between PORTAAL practices and exam scores. We also observed a large variation in average exam scores by department as shown in Fig 1. Therefore, we also included the department (Department) as a fixed effect in the regression model. The full regression model for the primary analysis is as below:
The variance explained by fixed effects was calculated according to Nakagawa and Shielzeth’s method [44], and the variances explained by random effects and the residual variance were obtained from the variance component estimates of the LMMs.
We conducted a subgroup regression analysis for introductory courses to determine if the correlation of PC1 and PC2 quartiles with exam scores was different for these courses than in the overall analysis. Courses were coded as introductory if they were lower-division courses required to complete STEM majors (e.g., introductory biology, introductory chemistry, algebra-based physics, introductory computer programming). For this analysis, the regression model remained the same as above but the data set was reduced to introductory courses only.
We conducted additional analyses to examine the interaction between demographic variables of interest (binary gender and FGN status) and PC1 and PC2 quartiles on exam scores. The general formulas for the regression models with interaction effects are as below:
Results
RQ1. Can we identify sets of correlated PORTAAL practices based on their use in courses across seven STEM departments?
A scree plot of the percent of variance explained by each PC showed that the first two PCs explained 61.2% of the total variance in the use of the 10 PORTAAL practices (S2 Fig). In reviewing the results of PCA, it is important to note that only the relative PC magnitudes and relative sign patterns of the loadings, rather than the absolute signs, are meaningful. Positive and negative signs are only meant to represent opposite directions of loading and do not imply that one set of factors is better than or worse than another—they are merely different from each other, not unlike the positive and negative poles on a battery.
Based on the heatmap of the loading matrix (Fig 3), PC1 was positively correlated with all practices, meaning we can interpret PC1 as positively correlated with a teaching pattern that incorporates all 10 practices at varying levels of use (i.e., intensity). PC2 correlated positively with one group of practices and correlated negatively with another group of practices. The group that was positively correlated with PC2 consisted of the following four practices: volunteers giving/explaining answers, randomly called students giving/explaining answers, the instructor prompting logic, and the instructor giving positive feedback. The group that was negatively correlated with PC2 consisted of five practices: instructors explaining, instructors answering, students working alone, students or the instructor explaining alternative answers, and use of high Bloom’s level in-class questions. The positively correlated PORTAAL practices (red in the heatmap of PC2) represents a more learner-centered teaching pattern, as students rather than the instructor were more actively engaged in the classroom learning environment. The negatively correlated practices (blue in the heatmap of PC2) represents a more instruction-centered teaching pattern.
Each entry represents how much a particular PORTAAL practice contributes to, or correlates with, a particular PC, where darker colors indicate larger contributions. Red represents a positive correlation and blue represents a negative correlation.
RQ1a: Does the use of these sets of PORTAAL practices vary by STEM department?
To visualize how the sets of PORTAAL practices were used in each course in each department, we plotted the PC1 values against the PC2 values, colored by department (Fig 4). The PC1 values on the x-axis represent a positive linear combination of the use of all 10 PORTAAL practices. A higher PC1 value is associated with a higher overall use of the 10 practices (i.e., intensity of use). The PC2 values on the y-axis represent the variation in use of learner-centered versus instruction-centered sets of PORTAAL practices (i.e., teaching mode). Higher PC2 values represent higher use of the set of four more learner-centered practices. Smaller PC2 values represent higher use of the set of five more instruction-centered practices. For example, a course with a PC2 value of −3 used more instruction-centered practices on average than a course with a PC2 value of −1.
PC1 (x-axis) represents the overall use of all 10 PORTAAL practices, from low use (−2) to high use (4). PC2 (y-axis) represents the use of instruction-centered and learner-centered sets of practices, with higher use of instruction-centered practices less than zero, and higher use of learner-centered practices greater than zero.
Courses tended to cluster by department, indicating that there are similar teaching styles within departments and different patterns of teaching across departments. Courses in the biology department seem to have a vastly different teaching pattern compared to other departments: they have a higher intensity of use of more teaching practices overall (based on the larger PC1 values on the x-axis compared to other departments) and have large variations in learner-centered versus instruction-centered teaching styles based on the wide range of PC2 values on the y-axis. Courses in psychology had lower average PC1 values indicating less overall use of PORTAAL practices, but also had a large range in PC2 values. Physics courses had more variation in PC1 values, which indicates a range of total use of PORTAAL practices, but most of these practices were instruction-centered, as indicated by the lower PC2 values. CSE, public health, and chemistry courses had lower average use of all PORTAAL practices, and CSE and public health courses used more learner-centered practices on average than in chemistry.
RQ2. Does the use of these sets of PORTAAL practices correlate with student exam performance?
We conducted a linear regression analysis using data from all the courses to determine how PC1 (intensity of use) and PC2 (teaching mode) correlated with student exam scores. Next, we conducted a subgroup analysis using only the data from the introductory courses. We conducted follow-up linear regression analyses to determine if there was an interaction between demographic variables, binary gender and first generation status of the students, and PC1 and PC2 quartiles on exam scores.
Analysis of all courses.
We fitted the LMM model as described in the methods to the full data set with all courses to determine the correlation between PC1 and PC2 quartiles and exam scores. Table 2 shows the percent variance explained by the fixed effects and each of the random effects, out of the total variance in student exam score. The fixed effects, including the teaching practices and covariates, accounted for 47.3% of the total variation in student score. Among the random effects, variation across students accounted for the most variation (11.1%), followed by variation across instructors, courses, and unique course offerings (Table 2). Even after including fixed effects and random effects, there was still 25.4% residual variation in student exam scores that was not accounted for.
The results of the LMM for the correlation between PC1 or PC2 quartiles and student exam scores from all the observed courses are reported in Table 3. Quartile 1, representing the lowest PC values for each PC, was used as the reference group in the analysis. The overall results indicate no significant correlation between the PC1 quartiles and student exam scores when controlling for the PC2 quartiles, covariates, and random effects. This suggests that the overall usage of the 10 PORTAAL practices at any level or intensity represented by the four quartiles do not contribute significantly to the variation in student exam scores across all STEM departments.
However, the PC2 (teaching mode) quartiles were correlated with student exam scores. Compared to a low level of learner-centered and a high level of instruction-centered practices (Q1 of PC2), a medium level of learner-centered practices (Q2 of PC2) is associated with an increase of 3.08 exam points (p < 0.001) on average. The same holds for a medium-high level of learner-centered practices (Q3 of PC2), with an increase of 3.85 exam points (p < 0.01) on average compared to a low level of learner-centered and a high level of instruction-centered practices. The highest intensity of learner-centered and lowest level of instruction-centered practices, represented by quartile 4 of PC2, was not significantly correlated with exam scores (1.29 points; p = 0.3897).
Analysis of introductory courses.
We conducted a subgroup regression analysis to determine how the PC values correlated with exam scores in introductory STEM courses. The results of the LMM for introductory courses are reported in the supporting information (S5 Table). We found that, in general, the results followed the pattern of the overall results where a medium level of learner-centered activity (Q2 of PC2) was correlated with higher exam scores (3.24 points; p < 0.001). However, the association between a medium-high level of learner-centered activity (Q3 of PC2) and exam scores was not significant (2.19 points; p = 0.0899), unlike in the overall analysis.
RQ2a. Do the correlations vary by student demographic groups?
We conducted follow-up regression analyses to determine if the association between teaching patterns, as measured by PC1 and PC2 quartiles, and exam scores was different for female students compared to male students, or for students who were the first in their family to graduate from college compared to non-first generation students.
Interaction analysis for gender.
There was no significant interaction between gender and PC1 quartiles in terms of association with student exam scores. In other words, there was no evidence that the overall usage of 10 PORTAAL practices at varying intensities (quartiles) was associated with differential exam performance for female and male students. There was a significant interaction between gender and the quartiles of learner-centered vs. instruction-centered activity (PC2) on student exam scores (p < 0.001 from likelihood ratio test). Table 4 shows the results of the interaction of PC2 and gender on student exam scores. In particular, the average increase in exam score when comparing a medium level of learner-centered activity (Q2 of PC2) to a low level of learner-centered activity (Q1 of PC2) was significantly larger in male students than in female students (3.78 versus 2.59 exam points; p < 0.001). The difference between male students and female students’ exam scores was not significant when comparing a medium-high or high level of learner-centered activity (Q3 or Q4 of PC2) to a low level of learner-centered activity (Q1 of PC2).
Interaction analysis for first generation status.
There was no significant interaction between first generation status and PC1 quartiles in terms of association with student exam scores. Compared to a low level of learner-centered activity (Q1 of PC2), both a medium level of learner-centered activity (Q2 of PC2) and a medium-high level of learner-centered activity (Q3 of PC2) were associated with higher exam scores in both FGN and non-FGN students (Table 5). The overall association between the intensity of learner-centered vs. instruction-centered activity (PC2) and student exam score differed between FGN and non-FGN students (p-value < 0.001 from likelihood ratio test). The average score increase comparing a medium-high level of learner-centered activity (Q3 of PC2) to a low level of learner-centered activity (Q1 of PC2) was significantly larger in non-FGN students than in FGN students (4.03 exam points versus 3.21 exam points; p < 0.05). In contrast, the average score increase comparing a high level of learner-centered activity (Q4 of PC2) to a low level of learner-centered activity (Q1 of PC2) was significantly larger in FGN students than in non-FGN students (1.85 exam points versus 1.03 exam points; p < 0.05), although the score increase by itself was not significantly different from zero in either non-FGN or FGN students.
Discussion
Ours is one of the first studies to contribute to the new area of second-generation active learning research. This new area of education research calls for not only identifying key elements of active learning teaching practices, but also for determining the dosage of those elements that correlate with increased student exam performance across demographic groups of students [1,40]. Our research has demonstrated a significant correlation between a specific set of PORTAAL teaching practices (i.e., learner-centered practices), the intensity of use of those practices, and increased student exam performance. These results should provide faculty with more clarity as to which teaching practices they should emphasize in their classrooms and how frequently they could implement those teaching practices if their goal is to increase student exam scores.
One of the major challenges of active learning research is that active learning teaching methods encompass a wide variety and combination of classroom activities, called by many different names and implemented in a variety of ways. The PORTAAL classroom observation tool identifies specific teaching practices that when used alone are documented in the literature to improve student academic performance [41]. The PORTAAL tool therefore provides a more granular analysis of teaching methods and their correlation with student academic performance.
Our previous work quantified how individual PORTAAL practices correlate with student exam performance across 33 biology courses at an R1 institution [42]. Our current work extends this research in three ways; we have determined 1) sets of PORTAAL practices used in courses 2) how the sets and intensity of use of these sets of PORTAAL practices correlate with student exam scores, and 3) we have extended our findings beyond undergraduate biology courses to include six other undergraduate STEM disciplines.
RQ1. Can we identify sets of correlated PORTAAL practices based on their use in courses across seven STEM departments?
Intensity of use (PC1).
We found that instructors used the 10 PORTAAL practices at varying levels while teaching their courses, and some practices were used at a higher intensity than others. This set of 10 PORTAAL practices are relatively common teaching practices used in some combination in many of the named active learning techniques that include but are not limited to think-pair-share [35], peer instruction [45], traditional clicker question format [36,46], and high structure classes [38]. Given the widespread calls to move away from traditional lecture-based teaching and create more interactive STEM classrooms that fully engage students in the learning process [47–50], as well as the growing efforts to train and support faculty in making these changes [51–56], we were pleasantly surprised to observe that many instructors who volunteered to be a part of this study were using at least low levels of these 10 teaching practices in their courses that ranged in size from 20 to 700 students. These results may suggest that active learning is becoming a part of the pedagogy of STEM courses at this university.
Teaching mode (PC2).
Our analysis also found two different sets of teaching practices. One set of practices more often included PORTAAL practices we have designated as instruction-centered while the other set included practices that we have designated as learner-centered. Individually, each of the 10 PORTAAL practices in this study has evidence in the literature that the practice supports improved student learning, as compared to traditional lecture-based teaching [41]. However, our results indicate that use of a specific set of these practices correlates with an increase in academic performance as compared to the other set of practices.
To illustrate how these sets of teaching practices appeared in the classrooms we observed, we constructed representative teaching scenarios based on common patterns seen across the classroom videos. These scenarios highlight the typical sequences of instructor and student actions associated with the learner-centered and instruction-centered sets of practices. In the learner-centered classroom, the practices of students volunteering their answers, randomly calling on students for their answers, the instructor prompting logic, and providing positive feedback to students were used more often. Courses that used more of these practices had higher PC2 scores. A possible teaching scenario in this course would be that the instructor poses a question to the class, prompts the students to use logic when solving the problem, allows time for students to arrive at an answer, and then requests for students to answer and explain their logic by either asking for a student to volunteer their answer or using random call to call on a student or group of students. The instructor closes the activity by providing positive feedback to the class or individual students. We see these practices as actively centering the classroom on the learning experience of the students, which is why we have termed this set of teaching practices, learner-centered.
The teaching practices found in the instruction-centered set include the instructor posing high Bloom’s level activities, students working alone, the instructor answering the problem they posed, the instructor providing the explanation for the answer, and instructors or students providing alternative answers to questions. Courses that implemented more of these practices in their classes had lower PC2 scores. A classroom scenario for the instruction-centered practices could be the instructor asking students to answer a multiple-choice question that involves a higher Bloom’s level analysis question. The students are instructed to first work alone and then use an online response system to select the correct answer from multiple choices. After a period of time, the instructor provides the correct answer with an explanation. The instructor would then proceed to explain why each of the incorrect answers is incorrect.
RQ1a: Does the use of these sets of PORTAAL practices vary by STEM department?
There was a continuum of use of all 10 PORTAAL practices, and a range of practices from more instruction-centered to more learner-centered across the seven STEM departments. The biology department that showed the greatest variation in PC values for courses. Instructors in biology courses used the PORTAAL practices more often and ranged from instruction-centered to learner-centered, though the majority of the courses were learner-centered. Many of the faculty in this department have been involved over the past 15 years in faculty development efforts to implement evidence-based teaching practices in their courses. This department has also had an active discipline-based education research group, which has created a community of support for implementing active learning. Andrews et al. [57] documented the positive influence discipline-based education research faculty have on encouraging colleagues within a department to implement more evidence-based teaching practices.
Of the remaining six departments, physics and mathematics showed a broad range of frequency of use of the practices. However, physics courses appeared to be more instruction-centered while mathematics courses tended toward more learner-centered practices. Nationally, physics has a long tradition of implementing peer instruction, as this teaching practice originated with Eric Mazur, a Harvard physics professor [45]. Therefore, we were not surprised to see some physics courses have a high frequency of use of practices. Peer Instruction often involves thinking and answering alone before discussing with a peer, re-submitting an answer, and then having the instructor provide the answer and explanation as well as discuss alternate answers. Given the method of Peer Instruction, we expected to see many of these courses fall more in the instruction-centered range. As many of the topics in physics involve complex problem solving, it is reasonable to assume that many in-class questions would be coded as high Bloom’s level. The PORTAAL data from the mathematics courses indicated that instructors included in this study often structured their class sessions around complex, multi-step problems. After assigning the problem to the class, students would often work in groups to solve the problem and would then be called on to offer answers to each step of the problem. Our finding that the mathematics courses were, on average, more learner-centered than the physics courses may suggest that students in mathematics courses offered more answers and explanations to questions, while in physics, instructors more often provided answers and explanations to in-class questions.
The four remaining departments each showed lower levels of implementation of PORTAAL practices in their courses. However, in public health and CSE courses, the practices that were implemented were more learner-centered, while for chemistry and psychology the practices were more instruction-centered. Based on the classroom observations of public health courses, we noted that there is a strong emphasis on group work and projects which could reflect the emphasis on building community that is promoted in this major on this campus. As group work and projects involve considerable student input, they are by their nature very learner-centered activities. However, group work and projects are more time consuming which could explain the lower number of times these practices were observed.
CSE courses also had low implementation of learner-centered practices, and also the least variation in the type and intensity of learner-centered practices. This tight clustering of learner-centered courses could be due to the discipline-specific pedagogy instructors use. Many of the CSE courses involved live coding as a teaching tool and often called on students to explain the outcomes of the code, which could have resulted in many of these courses clustering on the lower end of learner-centered teaching. However, many of these CSE courses are also prerequisites for upper-division courses in this highly competitive major and faculty may feel time pressures to cover content required in subsequent courses. The pressure faculty feel to cover content is a well-documented barrier to implementation of evidence-based teaching practices [18,21,23,58–61] and could account for the overall lower levels of learner-centered practices.
Similar time constraints may be felt by faculty in the chemistry department [62]. Instructors often feel time pressures because they must cover a large amount of material in each class, as this content is often foundational for the courses that follow. The pressure to cover content was stated explicitly by faculty that we interviewed after we observed their courses, and is explored in depth in our previous work [63]. As many of the teaching practices (such as having students answer and explain concepts) may require more class time, faculty may be reluctant to use them and may prefer to focus on explaining the answers themselves which shifts the class into a more instruction-centered format.
Though many psychology courses had a lower frequency of use of the PORTAAL practices, they did show a range of instruction- to learner-centered practices. Again, it is possible that the content of the course in this major may influence the teaching methods. Some of the psychology courses in this study were for non-majors. It is possible that when faculty are relieved of the pressure to cover content and have an interest in generating curiosity in learners they may shift to more learner-centered practices. However, some of the courses were for majors and the need to cover content could once again be shifting the teaching practices toward instruction-centered.
RQ2. Does the use of these sets of PORTAAL practices correlate with student exam performance?
Intensity of use (PC1).
To extend our previous research that focused on how individual PORTAAL practices correlate with exam performance, we used PCA as a dimension reduction approach to identify how the intensity of use of sets of PORTAAL practices correlate with student exam performance. Our analysis of all courses did not show a significant correlation between intensity of use of the 10 PORTAAL practices and student exam scores across all STEM departments. This could suggest that when these 10 PORTAAL practices are used in concert, they do not contribute significantly to the variation seen in students’ exam scores. These results could indicate there are other factors that play a more significant role in explaining students’ exam performance than this particular set of teaching practices. In our original paper describing the development of the PORTAAL tool [41], we acknowledged that there are many factors that contribute to student learning and academic success including but not limited to pre- and post-class assignments, student motivation, and self-efficacy.
It is also possible that in courses that had increased use of these practices, the instructor may also have increased the cognitive challenge of exams. This is what Freeman et al. found when they analyzed weighted Bloom’s scores of exams across six terms of transforming an introductory biology course from a low level of active learning to a much higher level of active learning [38]. As they added more active learning activities in each subsequent term, they found that the cognitive challenge of their exams also increased. Increasing the cognitive challenge level of exams may diminish the statistical impact of active learning on exam scores.
As we consider it unethical to use traditional lecture as a control condition, we used PCA and regression modeling to explore how different components of active learning correlate to student exam scores and the relative effectiveness of the components. When comparing the learner-centered set of PORTAAL practices to the instruction-centered set of practices, we found as the intensity of learner-centered practices increased, so did the student exam scores. Others have also reported a correlation between increased intensity of use of broadly-defined active learning practices and increased academic performance [38,64–66]. In each of these studies, various types of active learning or course structure such as pre-reading quizzes, practice exams, in-class clicker questions and group work, and use of worksheets were added to courses over time to determine the association with student academic performance. Our study differs from the previous research as we examined how the increased use of a specific set of the PORTAAL practices correlates with student exam scores.
Teaching mode (PC2): Learner-centered.
Each of the 10 PORTAAL practices when used alone is documented in the literature to support student learning [41]. However, based on our findings, we propose that when faculty use the set of learner-centered practices (students volunteer their answers, randomly call on students for their answers, prompt students to use their logic, and provide positive feedback to students after the students have answered), the instructor is setting up the problem and the format for student interaction and shifts the responsibility of the intellectual work of the problem—the learning—to the students. The instructor is facilitating the students’ opportunity to practice the scientific skill of explaining, using a combination of volunteer and random call to allow a greater variety of student voices to be heard, and is offering timely positive feedback to encourage the continued intellectual work of learning by the students.
We propose that a key aspect of the set of learner-centered PORTAAL practices is their strong alignment with a sociocultural constructivist theory of learning. Sociocultural constructivism posits that learning occurs as learners interact and collaborate to test and build their understanding [27]. A similar emphasis on the interaction of learners in the learning process is a key component of the ICAP Framework [67]. Chi and Wylie hypothesized and provided evidence that as students become more engaged with the learning process and move from passive to active to constructive and finally interactive engagement, their learning increases. By having students or groups of students answer and explain answers to in-class questions, the instructor has the potential to facilitate a whole class interaction with the learning process.
We were surprised that there was no evidence to show a positive or negative correlation for the use of the highest intensity quartile of the learner-centered practices with exam scores. One possible explanation for this result could be that in our sample, there were more upper-division courses in the highest intensity quartile than in the other three quartiles. To enroll in upper-division science courses at this university students must successfully advance through the introductory level prerequisite courses. It is possible that students in these upper-division courses have become more autonomous learners and are not as dependent on the teaching methods used in the courses.
Teaching mode (PC2): Instruction-centered.
Each of the PORTAAL practices found in the instruction-centered teaching profile (use of high Bloom’s level questions, students working alone, instructor answers and explains the question) is also documented in the literature to support student learning. For example, Jensen et al. found that using high-level Bloom’s questions during formative assessments resulted in students acquiring deep conceptual understanding of the course material and higher grades [68]. Similarly, based on interviews of students who had engaged in peer instruction that involved first working alone, Nielsen et al. found that students valued the alone time as a crucial opportunity to construct their arguments and explanations [69]. This allowed the students to engage in more fruitful discussion with their colleagues during the peer discussion. Smith et al. compared student performance on a second isomorphic in-class concept question after either peer discussion or instructor explanation or a combination of the techniques [70]. They found the combination approach improved average student performance considerably when compared with either technique alone, indicating the value of instructor explanations in addition to group work.
Why then was this grouping of instruction-centered practices not correlated with improved student exam performance as compared to the learner-centered practices? Though using high Bloom’s level questions on in-class assignments has been shown to be beneficial for many students, these questions may cause anxiety for other students and hinder productive thinking during class. Work by Wright et al. found that female students and students with lower socioeconomic status had lower performance on constructed response exams as the Bloom’s level of the exam increased [71]. Wright et al. proposed that these groups of students may have experienced stereotype threat or had a more fixed mindset. Therefore, instructors may need to better scaffold high Bloom’s questions and support students to counteract these influences.
We propose that though Nielsen et al. found students valued working alone [69], this practice also can minimize student interactions. Driessen et al. found that implementation of group work during class increased student performance by 1.00 standard deviations [72] and Weir et al. found student performance on diagnostic tests were highest in courses that added worksheets during in-class group activities [65]. A key aspect of the socio-constructivist framework of learning is an emphasis on the role of social interaction in the development of reasoning in the learning process [27]. Therefore, it is possible that group work may be more beneficial than alone time for student academic performance.
The instruction-centered activity of the instructor answering and explaining answers to in-class questions also may deny students the opportunity to engage in the scientific process of explaining and interpreting results. Having students practice hypothesizing and explaining has been shown to positively predict students’ felt recognition as a scientist, which in turn increased student motivation and predicted course grades [73]. Hearing student voices when answering in-class questions also has an impact on how students view the academic ability of their peers. A study by Grunspan et al., found that males underestimated the academic ability of their female classmates in courses where males were more outspoken than females during class [74]. To ensure that all student voices are heard, instructors can use random or warm calling when calling on students [75,76].
Introductory STEM courses.
Students in introductory STEM courses are usually earlier in their academic careers and may require more scaffolding and support in the course. Indeed, we found that students in these courses also benefited from the medium intensity of learner-centered practices as their exam scores also increased. This result could indicate that these practices provide the level of support younger students need to perform well on exams. Interestingly, students in introductory courses did not benefit from the medium-high or high levels of learner-centered practices. It’s possible that these students didn’t have enough prior experience with these new teaching methods, which may have prevented them from fully benefiting from their impact on learning. These results could be encouraging to faculty of introductory courses who are just learning how to implement learner-centered teaching practices. Our results indicate that even a medium level of implementation may benefit students.
RQ2a. Do the correlations vary by student demographic groups?
Gender.
Another aspect of second-generation active learning research is to explore the impact of the teaching intervention on the disaggregated student population. Many studies have reported gender gaps in academic performance in STEM courses [71,75,77–80]. We were surprised to see a gender difference at medium levels of implementing learner-centered teaching practices. Our results indicate that male students’ exam performance benefited more than female students’ in courses taught at a medium level. In more recent years, there has been a growing concern that boys and young men are being left behind by the educational system as fewer young men are graduating from high school and fewer men are earning college degrees as compared to women [81]. Men may be entering college more underprepared for learning than their peers. Learner-centered teaching practices are designed to scaffold and model the learning process for students. Therefore, underprepared students may benefit most from these teaching practices in small doses.
We were encouraged to see that there was no significant difference between exam performance of men and women across courses taught at a medium-high or high intensity of learner-centered practices compared to low levels of learner-centered practices. Therefore, we can conclude that learner-centered teaching practices are equally effective at enhancing exam performance for both groups of students at these two levels of use.
First generation status.
Numerous studies have documented the academic gap that many now refer to as an opportunity gap that exists for first generation students [2,82,83]. Many explanations have been offered to explain this gap, from poor preparation in high schools [84], to a decreased sense of belonging in STEM courses [83], to systemic factors of course structure in higher education [82]. To ameliorate some of these factors, many have proposed incorporating a higher intensity of active learning teaching techniques in higher education [2,64,85]. We were pleased to see that both a medium and a medium-high level of learner-centered teaching are associated with higher exam scores in both first generation and non-first generation students. Just as with gender, the learner-centered teaching methods were equally effective with these two groups of students. It was encouraging to see that a high level of learner-centered had a significantly greater correlation with exam scores for first generation students than for non-first generation students, though the exam score increase by itself is not significantly different from zero.
Limitations
We were not able to determine the cognitive challenge of the exams used in the courses in this study given the diversity of content across the seven STEM departments and the lack of knowledge of the specific topics that were taught in each course. Most exams in STEM across institutions in the United States are a lower Bloom’s level [86,87], which are often much easier in terms of difficulty than higher Bloom’s level questions. It is possible the higher exam scores in some courses are due to a lower cognitive challenge level of the exam, but we have no reason to believe that easier exams are more correlated with more learner-centered teaching practices.
Another possible limitation of our study is the classroom observation tool used in the study. The PORTAAL tool was developed in biology classrooms, though the practices included in PORTAAL are not limited to practices used in biology. We did however find that it was more difficult to categorize some of the teaching practices that occurred in CSE and mathematics courses because of the course content and the types of active learning practices those disciplines tend to use. We found that PORTAAL could still be adapted for these classrooms, but it is important to have conversations as a research team and come to a consensus about how to code these discipline-specific types of activities.
To investigate the association between teaching practices and exam score, ideally we would want the courses to only differ in teaching practices but be otherwise similar. However, in practice, many aspects of the courses could be different, such as the amount of homework assigned, the availability of teaching or learning assistants, problem solving sessions, and office hours. Most of these aspects would contribute to student exam performance, while being correlated with in-class teaching practices at the same time. Though the random effects included in our model could partially account for the intrinsic differences between courses, it appears that certain important course-level confounders such as exam difficulty or work completed outside of class time would still need to be adjusted in order to achieve a more comprehensive conclusion. Therefore, we need to be cautious when interpreting the significant results from our current models and acknowledge the possibility of spurious associations caused by unmeasured confounders.
Conclusions
Our work is a significant contribution to the new field of second-generation active learning research. We have extended our previous research in this field by analyzing sets of evidence-based teaching practices (PORTAAL practices) rather than isolated practices used in the classroom. We have shown that the use of the set of practices termed “learner-centered” correlates with higher exam scores as compared to “instruction-centered” practices. Furthermore, we have evidence that moderate and moderately-high intensities of use of learner-centered practices are significantly correlated with higher student exam performance. We have also expanded our previous findings on the positive correlation of the use of evidence-based teaching practices with student exam performance from one STEM department, biology, to six other STEM departments: chemistry, CSE, mathematics, physics, psychology, and public health. In addition, our findings have expanded from primarily lower-division courses in biology to upper- and lower-division courses across these seven STEM disciplines. Our results support the claim that learner-centered practices are associated with higher academic performance across many STEM disciplines, across academic levels, and in class sizes ranging from around 20 students to classes of 700 students.
By using the PORTAAL classroom observation tool to analyze the courses in our study, we have been able to identify at a more granular level the characteristics of teaching methods that are effective at improving exam performance. As most named active learning teaching practices require fidelity of implementation, often having multiple steps or tasks that should be completed in a specific order, this can be daunting for instructors to execute. The results from our study may support faculty as they mix and match the use of PORTAAL practices based on their own comfort level with the practices and implement practices that are appropriate for the content area of their course and the abilities of their students.
As the evidence accumulates that increasing student engagement in the learning process enhances student academic performance and narrows opportunity gaps for underrepresented students [2,64,85] it may be time to evaluate a “teaching gap”, as there is now significant evidence in the literature that teaching practices have a demonstrable impact on student academic performance.
Supporting information
S1 Table. PORTAAL practice code names and long descriptions, intensity or duration.
https://doi.org/10.1371/journal.pone.0327269.s003
(PDF)
S2 Table. Number of courses, instructors, and student data points for each STEM department.
https://doi.org/10.1371/journal.pone.0327269.s004
(PDF)
S3 Table. The number (percentage) of lower- and upper-division courses in each quartile of PC2.
https://doi.org/10.1371/journal.pone.0327269.s005
(PDF)
S4 Table. Descriptive statistics of student data points in all courses by demographic variables.
https://doi.org/10.1371/journal.pone.0327269.s006
(PDF)
S5 Table. LMM results for introductory courses.
https://doi.org/10.1371/journal.pone.0327269.s007
(PDF)
S1 Fig. Correlation matrix of the 10 PORTAAL practices used for analysis.
https://doi.org/10.1371/journal.pone.0327269.s008
(PNG)
S2 Fig. Scree plot of the percent variance explained by each of the 10 principal components (PCs) included in the principal component analysis.
https://doi.org/10.1371/journal.pone.0327269.s009
(PNG)
S1 File. Full de-identified dataset for analysis.
Includes average student exam scores, demographic information, course information, and mean PORTAAL practice values for all student data points.
https://doi.org/10.1371/journal.pone.0327269.s010
(XLSX)
Acknowledgments
We thank the undergraduate researchers who used PORTAAL to code the classroom recordings: E. Swanberg, P.J. Kambhiranond, A. Immel, C. Meng, and J. McAleer.
References
- 1. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci. 2014;111(23):8410–5.
- 2. Theobald EJ, Hill MJ, Tran E, Agrawal S, Arroyo EN, Behling S. Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proc Natl Acad Sci. 2020;117(12):6476–83.
- 3. Aji CA, Khan MJ. The Impact of Active Learning on Students’ Academic Performance. Open Journal of Social Sciences. 2019;7(3):204–11.
- 4. Deslauriers L, McCarty LS, Miller K, Callaghan K, Kestin G. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proc Natl Acad Sci. 2019;116(39):19251–7.
- 5. Lugosi E, Uribe G. Active learning strategies with positive effects on students’ achievements in undergraduate mathematics education. Int J Math Educ Sci Technol. 2022;53(2):403–24.
- 6. Wilton M, Gonzalez-Niño E, McPartlan P, Terner Z, Christoffersen RE, Rothman JH. Improving academic performance, belonging, and retention through increasing structure of an introductory biology course. CBE—Life Sci Educ. 2019;18(4):ar53.
- 7. Prince M. Does active learning work? A review of the research. J Eng Educ. 2004;93(3):223–31.
- 8.
Mintzes JJ, Walter EM. Active Learning in College Science: The Case for Evidence-Based Practice. Springer Nature. 2020.
- 9. Hake RR. Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics. 1998;66(1):64–74.
- 10. Strelan P, Osborn A, Palmer E. The flipped classroom: A meta-analysis of effects on student performance across disciplines and education levels. Educ Res Rev. 2020;30:100314.
- 11. McCarthy JP, Anderson L. Active learning techniques versus traditional teaching styles: Two experiments from history and political science. Innov High Educ. 2000;24(4):279–94.
- 12. Archer CC, Miller MK. Prioritizing active learning: An exploration of gateway courses in political science. PS Polit Sci Polit. 2011;44(2):429–34.
- 13. Oliván Blázquez B, Masluk B, Gascon S, Fueyo Díaz R, Aguilar-Latorre A, Artola Magallón I, et al. The use of flipped classroom as an active learning approach improves academic performance in social work: A randomized trial in a university. PLoS One. 2019;14(4):e0214623. pmid:30947270
- 14. Albert M, Beatty BJ. Flipping the classroom applications to curriculum redesign for an introduction to management course: Impact on grades. J Educ Bus. 2014;89(8):419–24.
- 15. Benabentos R, Hazari Z, Stanford JS, Potvin G, Marsteller P, Thompson KV. Measuring the implementation of student-centered teaching strategies in lower- and upper-division STEM courses. J Geosci Educ. 2020;:1–15.
- 16. Drinkwater MJ, Matthews KE, Seiler J. How Is Science Being Taught? Measuring Evidence-Based Teaching Practices across Undergraduate Science Departments. CBE Life Sci Educ. 2017;16(1):ar18. pmid:28232589
- 17. Yik BJ, Raker JR, Apkarian N, Stains M, Henderson C, Dancy MH. Evaluating the impact of malleable factors on percent time lecturing in gateway chemistry, mathematics, and physics courses. Int J STEM Educ. 2022;9(1):15.
- 18. Petersen CI, Baepler P, Beitz A, Ching P, Gorman KS, Neudauer CL, et al. The Tyranny of Content: “Content Coverage” as a barrier to evidence-based teaching approaches and ways to overcome it. CBE Life Sci Educ. 2020;19(2):ar17. pmid:32412836
- 19. Brownell SE, Tanner KD. Barriers to faculty pedagogical change: Lack of training, time, incentives, and…tensions with professional identity?. CBE—Life Sci Educ. 2012;11(4):339–46.
- 20. Carbone A, Drew S, Ross B, Ye J, Phelan L, Lindsay K, et al. A collegial quality development process for identifying and addressing barriers to improving teaching. High Educ Res Dev. 2019;38(7):1356–70.
- 21. Henderson C, Dancy MH. Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Phys Rev Spec Top - Phys Educ Res. 2007;3(2):020102.
- 22. Kim AM, Speed CJ, Macaulay JO. Barriers and strategies: Implementing active learning in biomedical science lectures. Biochem Mol Biol Educ. 2019;47(1):29–40. pmid:30548154
- 23. Shadle SE, Marker A, Earl B. Faculty drivers and barriers: Laying the groundwork for undergraduate STEM education reform in academic departments. Int J STEM Educ. 2017;4(1):8.
- 24. Driessen EP, Knight JK, Smith MK, Ballen CJ. Demystifying the meaning of active learning in postsecondary biology education. CBE Life Sci Educ. 2020;19(4):ar52. pmid:33001767
- 25. Lombardi D, Shipley TF, Astronomy Team BT. The curious construct of active learning. Psychol Sci Public Interest. 2021.
- 26.
Bonwell CC, Eison JA. Active Learning: Creating Excitement in the Classroom. https://eric.ed.gov/?id=ED336049. 1991. Accessed 2020 July 21.
- 27.
Vygotsky LS, Cole M, John-Steiner V, Scribner S, Souberman E. Mind in society: The development of higher psychological processes. 1978.
- 28.
Brooks JG, Brooks MG. In search of understanding: The case for constructivist classrooms. 2nd ed. Pearson College Div. 2001.
- 29. Morgan BM, Rodriguez AD, Rosenberg GP. Cooperative learning, jigsaw strategies, and reflections of graduate and undergraduate education students. Coll Teach Methods Styles J. 2011;4(2):1.
- 30.
O’Donnell AM, Hmelo-Silver CE. Introduction: What is collaborative learning?: An overview. The International Handbook of Collaborative Learning. Routledge. 2013.
- 31. Michaelsen LK, Sweet M. The essential elements of team‐based learning. New Drctns for Teach & Learn. 2008;2008(116):7–27.
- 32. Hmelo-Silver CE. Problem-Based Learning: What and How Do Students Learn?. Educ Psychol Rev. 2004;16(3):235–66.
- 33. Barrows HS. Problem‐based learning in medicine and beyond: A brief overview. New Drctns for Teach & Learn. 1996;1996(68):3–12.
- 34. Mazur E. Peer instruction: A user’s manual. Am J Phys. 1999;67(4):359–60.
- 35. Kaddoura M. Think Pair Share: A Teaching Learning Strategy to Enhance Students’ Critical Thinking. Educ Res Q. 2013;36(4):3–24.
- 36.
Smith MK, Knight JK. Clickers in the biology classroom: Strategies for writing and effectively implementing clicker questions that maximize student learning. Mintzes JJ, Walter EM. Active Learning in College Science: The Case for Evidence-Based Practice. Cham: Springer International Publishing. 2020. 141–58.
- 37.
Moog RS. Origins of POGIL: Process Oriented Guided Inquiry Learning. POGIL. Routledge. 2019.
- 38. Freeman S, Haak D, Wenderoth MP. Increased course structure improves performance in introductory biology. CBE Life Sci Educ. 2011;10(2):175–86. pmid:21633066
- 39. O’Flaherty J, Phillips C. The use of flipped classrooms in higher education: A scoping review. Internet High Educ. 2015;25:85–95.
- 40. Bernstein DA. Does active learning work? A good question, but not the right one. Scholarsh Teach Learn Psychol. 2018;4(4):290–307.
- 41. Eddy SL, Converse M, Wenderoth MP. PORTAAL: A Classroom Observation Tool Assessing Evidence-Based Teaching Practices for Active Learning in Large Science, Technology, Engineering, and Mathematics Classes. CBE—Life Sci Educ. 2015;14(2):23.
- 42. Moon S, Jackson MA, Doherty JH, Wenderoth MP. Evidence-based teaching practices correlate with increased exam performance in biology. PLoS One. 2021;16(11):e0260789. pmid:34847190
- 43. Jackson MA, Moon S, Doherty JH, Wenderoth MP. Which evidence-based teaching practices change over time? Results from a university-wide STEM faculty development program. Int J STEM Educ. 2022;9(1):22.
- 44. Nakagawa S, Schielzeth H. A general and simple method for obtaining R 2 from generalized linear mixed‐effects models. Methods Ecol Evol. 2012;4(2):133–42.
- 45. Crouch CH, Mazur E. Peer instruction: Ten years of experience and results. Am J Phys. 2001;69(9):970–7.
- 46. Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, et al. Why peer discussion improves student performance on in-class concept questions. Science. 2009;323(5910):122–4. pmid:19119232
- 47. Cooper MM, Caballero MD, Ebert-May D, Fata-Hartley CL, Jardeleza SE, Krajcik JS. Challenge faculty to transform STEM learning. Science. 2015;350(6258):281–2.
- 48. Handelsman J, Ebert-May D, Beichner R, Bruns P, Chang A, DeHaan R, et al. Scientific Teaching. Science. 2004;304(5670):521–2.
- 49. Alberts B. Considering science education. Science. 2008;319(5870):1589. pmid:18356492
- 50. Anderson WA, Banerjee U, Drennan CL, Elgin SCR, Epstein IR, Handelsman J, et al. Changing the Culture of Science Education at Research Universities. Science. 2011;331(6014):152–3.
- 51. Borda E, Schumacher E, Hanley D, Geary E, Warren S, Ipsen C. Initial implementation of active learning strategies in large, lecture STEM courses: Lessons learned from a multi-institutional, interdisciplinary STEM faculty development program. Int J STEM Educ. 2020;7(1).
- 52. Callens MV, Kelter P, Motschenbacher J, Nyachwaya J, Ladbury JL, Semanko AM. Developing and implementing a campus-wide professional development program: Successes and challenges. J Coll Sci Teach Wash. 2019;49(2):68–75.
- 53. Derting TL, Ebert-May D, Henkel TP, Maher JM, Arnold B, Passmore HA. Assessing faculty professional development in STEM higher education: Sustainability of outcomes. Sci Adv. 2016;2(3):e1501422. pmid:27034985
- 54. Herman GL, Greene JC, Hahn LD, Mestre JP, Tomkin JH, West M. Changing the teaching culture in introductory STEM courses at a large research university. J Coll Sci Teach. 2018;47(6):32–8.
- 55. McConnell M, Montplaisir L, Offerdahl E. Meeting the conditions for diffusion of teaching innovations in a university STEM department. J STEM Educ Res. 2020;3(1):43–68.
- 56. Viskupic K, Ryker K, Teasdale R, Manduca C, Iverson E, Farthing D. Classroom observations indicate the positive impacts of discipline-based professional development. J STEM Educ Res. 2019;2(2):201–28.
- 57. Andrews TC, Conaway EP, Zhao J, Dolan EL. Colleagues as Change Agents: How Department Networks and Opinion Leaders Influence Teaching at a Single Research University. CBE—Life Sci Educ. 2016;15(2):ar15.
- 58. Michael J. Faculty perceptions about barriers to active learning. Coll Teach. 2007;55(2):42–7.
- 59. Andrews TC, Lemons PP. It’s personal: biology instructors prioritize personal evidence over empirical evidence in teaching decisions. CBE Life Sci Educ. 2015;14(1):ar7. pmid:25713092
- 60. Turpen C, Dancy M, Henderson C. Perceived affordances and constraints regarding instructors’ use of Peer Instruction: Implications for promoting instructional change. Phys Rev Phys Educ Res. 2016;12(1):010116.
- 61. Sturtevant H, Wheeler L. The STEM Faculty Instructional Barriers and Identity Survey (FIBIS): Development and Exploratory Results. Int J STEM Educ. 2019;6(1):35.
- 62. Kraft A, Popova M, Erdmann RM, Harshman J, Stains M. Tensions between depth and breadth: An exploratory investigation of chemistry assistant professors’ perspectives on content coverage. Chem Educ Res Pract. 2023;24(2):567–76.
- 63. Jackson MA, Wenderoth MP. A thematic analysis of interviews reveals how a STEM faculty development program supported the complexity of implementing evidence-based teaching. Improve Acad J Educ Dev. 2024;43(1).
- 64. Eddy SL, Hogan KA. Getting under the hood: How and for whom does increasing course structure work?. CBE Life Sci Educ. 2014;13(3):453–68. pmid:25185229
- 65. Weir LK, Barker MK, McDonnell LM, Schimpf NG, Rodela TM, Schulte PM. Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLoS One. 2019;14(8):e0220900. pmid:31461447
- 66. Connell GL, Donovan DA, Chambers TG. Increasing the Use of Student-Centered Pedagogies from Moderate to High Improves Student Learning and Attitudes about Biology. CBE Life Sci Educ. 2016;15(1):ar3. pmid:26865643
- 67. Chi MTH, Wylie R. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educ Psychol. 2014;49(4):219–43.
- 68. Jensen JL, McDaniel MA, Woodard SM, Kummer TA. Teaching to the test…or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. Educ Psychol Rev. 2014;26(2):307–29.
- 69. Nielsen KL, Hansen-Nygård G, Stav JB. Investigating peer instruction: How the Initial Voting Session Affects Students’ Experiences of Group Discussion. ISRN Education. 2012;2012:1–8.
- 70. Smith MK, Wood WB, Krauter K, Knight JK. Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE Life Sci Educ. 2011;10(1):55–63. pmid:21364100
- 71. Wright CD, Eddy SL, Wenderoth MP, Abshire E, Blankenbiller M, Brownell SE. Cognitive Difficulty and Format of Exams Predicts Gender and Socioeconomic Gaps in Exam Performance of Students in Introductory Biology Courses. CBE Life Sci Educ. 2016;15(2):ar23. pmid:27252299
- 72. Driessen EP, Wilson AE, Hall I, Brewer P, Odom S, Ramsey SB, et al. Group work enhances student performance in biology: A meta-analysis. BioScience. 2024;74(3):207–17.
- 73. Starr CR, Hunter L, Dunkin R, Honig S, Palomino R, Leaper C. Engaging in science practices in classrooms predicts increases in undergraduates’ STEM motivation, identity, and achievement: A short-term longitudinal study. J Res Sci Teach. 2020;57(7):1093–118.
- 74. Grunspan DZ, Eddy SL, Brownell SE, Wiggins BL, Crowe AJ, Goodreau SM. Males Under-Estimate Academic Performance of Their Female Peers in Undergraduate Biology Classrooms. PLoS One. 2016;11(2):e0148405. pmid:26863320
- 75. Eddy SL, Brownell SE, Wenderoth MP. Gender gaps in achievement and participation in multiple introductory biology classrooms. CBE Life Sci Educ. 2014;13(3):478–92. pmid:25185231
- 76. Metzger KJ, Via Z. Warming up the cold call: Encouraging classroom inclusion by considering warm- & cold-calling techniques. Am Biol Teach. 2022;84(6):342–6.
- 77. Matz RL, Koester BP, Fiorini S, Grom G, Shepard L, Stangor CG, et al. Patterns of gendered performance differences in large introductory courses at five research universities. AERA Open. 2017;3(4).
- 78. Farrar VS, Aguayo BYC, Caporale N. Gendered performance gaps in an upper-division biology course: Academic, demographic, environmental, and affective factors. CBE—Life Sci Educ. 2023;22(4):ar52.
- 79. Cotner S, Jeno LM, Walker JD, Jørgensen C, Vandvik V. Gender gaps in the performance of Norwegian biology students: The roles of test anxiety and science confidence. Int J STEM Educ. 2020;7(1):55.
- 80. Odom S, Boso H, Bowling S, Brownell S, Cotner S, Creech C, et al. Meta-analysis of Gender Performance Gaps in Undergraduate Natural Science Courses. CBE Life Sci Educ. 2021;20(3):ar40. pmid:34283633
- 81.
Reeves RV. Of boys and men: Why the modern male is struggling, why it matters, and what to do about it. Brookings Institution Press. 2022.
- 82. Denaro K, Dennin K, Dennin M, Sato B. Identifying systemic inequity in higher education and opportunities for improvement. PLoS One. 2022;17(4):e0264059. pmid:35395005
- 83. Jantzer J, Kirkman T, Furniss KL. Understanding differences in underrepresented minorities and first-generation student perceptions in the introductory biology classroom. J Microbiol Biol Educ. 2021;22(3):e00176-21. pmid:34804325
- 84.
Holmes DL, Slate JR. Differences in GPA by gender and ethnicity/race as a function of first-generation status for community college students. 2017.
- 85. Dewsbury BM, Swanson HJ, Moseman-Valtierra S, Caulkins J. Inclusive and active pedagogies reduce academic outcome gaps and improve long-term performance. PLoS One. 2022;17(6):e0268620. pmid:35704639
- 86. Momsen JL, Long TM, Wyse SA, Ebert-May D. Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. CBE Life Sci Educ. 2010;9(4):435–40. pmid:21123690
- 87. Uminski C, Burbach SM, Couch BA. Undergraduate biology lecture courses predominantly test facts about science rather than scientific practices. CBE—Life Sci Educ. 2024;23(2):ar19.