The Effectiveness of the Problem-Based Learning Teaching Model for Use in Introductory Chinese Undergraduate Medical Courses: A Systematic Review and Meta-Analysis

Background Although the problem-based learning (PBL) emerged in 1969 and was soon widely applied internationally, the rapid development in China only occurred in the last 10 years. This study aims to compare the effect of PBL and lecture-based learning (LBL) on student course examination results for introductory Chinese undergraduate medical courses. Methods Randomized and nonrandomized controlled trial studies on PBL use in Chinese undergraduate medical education were retrieved through PubMed, the Excerpta Medica Database (EMBASE), Chinese National Knowledge Infrastructure (CNKI) and VIP China Science and Technology Journal Database (VIP-CSTJ) with publication dates from 1st January 1966 till 31 August 2014. The pass rate, excellence rate and examination scores of course examination were collected. Methodological quality was evaluated based on the modified Jadad scale. The I-square statistic and Chi-square test of heterogeneity were used to assess the statistical heterogeneity. Overall RRs or SMDs with their 95% CIs were calculated in meta-analysis. Meta-regression and subgroup meta-analyses were also performed based on comparators and other confounding factors. Funnel plots and Egger’s tests were performed to assess degrees of publication bias. Results The meta-analysis included 31studies and 4,699 subjects. Fourteen studies were of high quality with modified Jadad scores of 4 to 6, and 17 studies were of low quality with scores of 1 to 3. Relative to the LBL model, the PBL model yielded higher course examination pass rates [RR = 1.09, 95%CI (1.03, 1.17)], excellence rates [RR = 1.66, 95%CI (1.33, 2.06)] and examination scores [SMD = 0.82, 95%CI (0.63, 1.01)]. The meta-regression results show that course type was the significant confounding factor that caused heterogeneity in the examination-score meta-analysis (t = 0.410, P<0.001). The examination score SMD in “laboratory course” subgroup [SMD = 2.01, 95% CI: (1.50, 2.52)] was higher than that in “theory course” subgroup [SMD = 0.72, 95% CI: (0.56, 0.89)]. Conclusions PBL teaching model application in introductory undergraduate medical courses can increase course examination excellence rates and scores in Chinese medical education system. It is more effective when applied to laboratory courses than to theory-based courses.


Conclusions
PBL teaching model application in introductory undergraduate medical courses can increase course examination excellence rates and scores in Chinese medical education system. It is more effective when applied to laboratory courses than to theory-based courses.

Background
The problem-based learning (PBL) teaching model was first developed in 1969, and the approach has since become a popular education model internationally [1,2]. According to World Health Organization data, the PBL teaching model has been used in more than 1,700 medical schools globally, and this number continues to grow [3]. The PBL teaching model was first used in higher medical education settings in China in the 1980s. Due to current shifts in approaches to medical education prevalent in China, this model has been extensively applied as an experimental teaching method in Chinese medical schools. The annual number of published Chinese studies focusing the application of PBL teaching methods has increased exponentially from 14 in 2000 to 474 in 2011.
The PBL teaching model is still controversial [4,5]. Numerous studies have found that in medical education settings, relative to traditional, lecture-based learning (LBL) models, the PBL model presents certain advantages with respect to improving student abilities in inactive learning, two-way communication, clinical thinking, and teamwork [6][7][8][9]. A study by Abraham et al. suggested that physiology teaching outcomes could be improved through the use of the PBL teaching model [6]. A study by Mehadizadeh et al. demonstrated that anatomy students that had been instructed via PBL teaching methods not only achieved higher examination scores, but were also highly satisfied with this teaching method [8]. Furthermore, a study by the University of Missouri School of Medicine revealed that overtime, the PBL teaching model may improve the passing rate of the United States Medical Licensing Examination [9]. However, other researchers do not consider the PBL teaching model to be superior to the LBL teaching model with respect to the acquisition of theoretical and fundamental knowledge [10][11][12][13][14][15].
PBL teaching reforms in China have largely been applied to clinical courses, and these reforms have affected levels of teaching effectiveness in similar ways as they have in other countries [16][17][18]. However, a systematic, quantitative assessment of the outcomes of PBL teaching model application during learning stages of introductory medical courses has not yet been conducted. For this study, meta-analysis methods were applied to compare the effects of PBL and LBL teaching models on course examination results of introductory undergraduate medical courses in China, thereby providing a scientific basis for evaluating the necessity and feasibility of PBL application in such courses.

Inclusion and exclusion criteria
For this study, we used the following definition of PBL provided by Kinkade [19]: a curriculum of carefully selected activities that test the learner's critical knowledge acquisition, problemsolving, self-directed learning, and team-participation capacities. Students work in small groups, generate hypotheses about the given case and learning objectives, work outside of class hours to fulfill learning objectives, and then reconvene and solve the problem.
Studies included in this review met the following inclusion criteria: 1) examination of PBL use as a teaching method for five-year undergraduate medical curricula applied in Chinese medical schools; and 2) use of randomized or nonrandomized controlled trials (RCTs), in which experimental groups were instructed using either the PBL teaching model alone or using the PBL teaching model in combination with the traditional LBL teaching model while control groups were instructed strictly based on the LBL teaching model. Courses for which PBL was applied were introductory medical courses in physiology, biochemistry, pharmacological, anatomy, medical statistics, etc. Course examinations were used to assess study populations, and data on examination results were reported.
We excluded studies that did not include a control group; that examined postgraduate or other non-undergraduate courses; that involved non-introductory postgraduate medical courses in internal medicine, surgery, diagnostics and clinical practice; and that did not cite objective course examination data and republished studies.

Search strategy
To identify relevant studies, we searched for publications using the following databases from the earliest available date through 31 August of 2014: PubMed (1st January 1966), the Excerpta Medica Database (EMBASE) (1st January 1966), the China Knowledge Resource Integrated Database (China National Knowledge Infrastructure, CNKI http://www.cnki.net/, 1st January 1979) and the VIP China Science and Technology Journal Database (VIP-CSTJ, http:// oldweb.cqvip.com, 1st January 1979). The search terms "PBL," "problem-based learning," "based on problems," "active learning," and "learner centered" were used to identify PBL studies, and these were combined with other key terms such as "medical," "undergraduate," "Chinese," and "China." We also manually searched through the reference lists of retrieved articles to trace potentially relevant papers.

Data extraction and quality assessment
Literature screening was independently performed by two reviewers (L.Z. and X.L.) in accordance with the inclusion and exclusion criteria; the data were then extracted and crosschecked. Data extraction in consistencies were resolved through discussion, and secondary calculations found during data extraction were resolved in consultation with a third reviewer (Y. Z.).The extracted data included general study information(the title, author name, publication year and literature resources); basic study characteristics (the number of experimental and control groups, participant characteristics, course name and type, study type, intervention process, literature quality assessment characteristics; etc.)and outcomes (the number of "excellent," "pass" and "fail" scores, or experimental and control group examination sores).On a 100-point scale, "excellent" denotes a score of80 points, "pass" denotes a score of60 points and "fail" denotes a score of<60 points.
Methodological quality assessments of the included studies were independently performed by two researchers using the modified Jadad scale [20]. The scale included eight items: randomization, blinding, withdrawals, dropouts, inclusion/exclusion criteria, adverse effects and statistical analysis ( Table 1). The total score for each article ranged from 0 to 8 and was computed by summing the score of each item. Low quality studies wielded scores of 0 to 3, and high quality studies achieved scores of 4 to 8.

Statistical methods
The outcome measures of this study were course examination results, which were given two expression forms. The first was a dichotomous outcome ("excellent", "pass" or "fail" evaluation), and the other was a continuous outcome (i.e., examination scores).
RevMan version 5.3 (Cochrane Collaboration, Copenhagen, Denmark) and the meta-analysis module included in Stata 11.0 (College Station, Texas 77845 USA)were utilized for the meta-analysis. The analytical statistics of relative risk (RR) and standardized mean difference (SMD) at 95% confidence intervals (95% CIs)were used to determine the teaching effectiveness of the PBL model for dichotomous and continuous outcomes, respectively. Before the study results were combined, the I-square statistic and Chi-square test of heterogeneity were used to assess the statistical heterogeneity of the included studies. Values of I 2 >50% or P<0.10 were considered to exhibit significant heterogeneity across studies. The total RR or SMD score at 95% CI was calculated using a random-effects model when heterogeneous results appeared. Otherwise, a fixed-effects model was used.
Meta-regression was used to examine the confounding factors' effect. Confounding factors included the following: degree major, teaching pattern, course type, PBL group tutor scale, study type and modified Jadad score. We also performed subgroup meta-analyses based on these confounding factors. For the subgroup analysis based on teaching patterns, two subgroups based on whether the PBL teaching model was used independently for the experimental group were used. For one subgroup, the comparator was PBL vs. LBL, and the PBL teaching model was used independently for the experimental group. In the other subgroup, the comparator was PBL+LBL vs. LBL, and both PBL and LBL teaching models were used for the experimental group. We utilized funnel plots and Egger's tests to assess the degree of publication bias both graphically and statistically. A sensitivity analysis was performed by exchanging the combined model (fixed effects model and random effects model).

Literature search results
Using the literature search method, a total of 3,915 relevant studies were initially retrieved. After reviewing the titles, abstracts, and full texts of these studies, 3,884 studies were excluded, and 31 studies were used for the qualitative synthesis and meta-analysis . The literature screening process and results are depicted in Fig 1.
A meta-regression was performed because a relatively high degree of heterogeneity between the included studies was found. The following confounding factors that were considered: degree major, teaching pattern, course type, PBL group tutor scale, study type and modified Jadad score. Among the three outcomes, we discovered that course type is the significant confounding factor that causes examination-score meta-regression heterogeneity (t = 0.410, P<0.001) ( Table 5).To further examine heterogeneity from confounding factors, we performed a subgroup meta-analysis based on the following factors: degree major, course type, PBL group tutor scale, study type and modified Jadad score. For most of the subgroups, heterogeneity has not been eliminated. However, for the "laboratory course" subgroup, we did not detect heterogeneity through our meta-analysis of pass rates (I 2 = 0%, P = 0.950).As well, heterogeneity was not identified through our meta-analysis of "laboratory course" subgroup examination scores (I 2 = 49%, P = 0.160)or of the substantial heterogeneity group(I 2 = 18%, P = 0.300). The subgroup examination-score meta-analysis also revealed that the difference in SMD levels between the "theory course"[SMD: 0.72, 95% CI: (0.56, 0.89)] and "laboratory course" subgroups [SMD: 2.01, 95% CI: (1.50, 2.52)]is statistically significance (P<0.001).The examination score SMD value between experimental and control groups for the "laboratory course" subgroup was found to be higher than that of the "theory course" subgroup (Table 6).

Publication bias
The pass rate, excellence rate and examination score funnel plots do not reveal a significant degree of publication bias between the included studies (Figs 5-7). However, publication bias Egger's test results reveal a minor degree of publication bias among pass rate (t = 2.310, P = 0.050) and examination score (t = 2.130, P = 0.045) results (Table 7).

Sensitivity analysis
A sensitivity analysis was performed by changing the combined model from a random effects model to a fixed effects model. The results of the fixed effects model were consistent with those of the random effects model (Figs 8-10).

Discussion
Since its first application in Canada in the late 1960s, PBL has been widely adopted innumerous universities internationally. As in China, education systems in various regions and countries Table 5. Meta-regression of the effects of confounding factors on pass rate, excellent rate and examination score.

Outcome
Factor Coefficient (95%CI) std. error t P  differ considerably from those of the U.S. Furthermore, cultural differences influence the effectiveness of PBL methods [52]. Therefore, the target population must be limited to Chinese students of the Chinese education system in order to examine the potential effectiveness of the PBL teaching model in China. Our research method will serve as a valuable reference for researchers based in countries of differing cultural backgrounds who wish to evaluate the effectiveness of the PBL teaching model. Our research conclusions will promote PBL teaching model application in countries of differing cultural characteristics. The results of our meta-analysis indicate that the PBL teaching model can yield significantly positive results relative to the LBL teaching model, particularly in excellence rates and examination scores, which is inconsistent with the previous finding that PBL students either perform no differently or slightly worse than students in conventional on measures of knowledge such as basic sciences examinations [53][54][55]. Considering the differences in higher medical education between China and the West [56], we speculate that the results may be due to the following reasons. In the PBL teaching model, students play a major role in the process of teaching, and teachers facilitate the student learning and support that learning through experimentation, clinical cases, and seminars. The PBL teaching model is rather different from the traditional LBL teaching model in which students often passively accept their teachers' knowledge. Chinese students have accepted the LBL teaching model for more than 10 years, starting with their primary education, and the PBL teaching model is a novelty that has greatly stimulated students' interest in learning. [56] Most Chinese medical universities prefer using uniform textbooks for all students, which is quite different from the United States and other Western countries where no uniform textbooks or standard formats of lectures for medical universities exist. [56] In our study, students in the PBL group used the same textbook as students in the LBL group, and they appeared to be better at active learning, which led them to earn more positive examination results. Conversely, the use of the PBL teaching method in Chinese medical higher education is still in its infancy, and the evaluation of the effectiveness of the PBL teaching method is still relatively unsophisticated, particularly for basic medical education. Both teachers and students emphasize course exams [56]; Chinese students' keen pursuit of positive test scores helps them excel on exams.

Major
The "course type" subgroup analysis shows that the PBL teaching model is more effective when applied in laboratory class settings than in theory-based class settings. Generally speaking, laboratory class exams focus more on execution and analytical skills [23,26,31,36,42,50,53]. The PBL teaching model can inspire students to engage in proactive learning and thinking initiatives, facilitating a stronger grasp of experimental processes and logic. Consequently, students may acquire a deeper understanding of experiments that they conduct, thus enabling them to produce higher quality experimental reports [23]. Therefore, the PBL teaching model would be best applied for laboratory course examinations. On the other hand, while theory courses in several schools also utilize the PBL teaching model, due to limitations on teaching conditions, students are divided into groups but are remain in one classroom, and the number of students in a single classroom can exceed 100 [24,29,37,44,46,49,51]. In contrast, laboratory courses are typically conducted in small groups, which is more suitable for PBL teaching model adoption [23,26,31,36,42,50,57]. Hence, the advantages of the PBL teaching model relative to the LBL teaching model are more evident when considering laboratory courses.
A number of researchers believe that utilizing a combination of PBL and LBL teaching models for introductory medical courses may improve teaching effectiveness because while the PBL teaching model boosts student initiative and improves proactive learning abilities, the LBL teaching model improves student comprehension of structural knowledge systems and student

Study limitations
Because our study only focuses on undergraduate Chinese medicine higher education, the conclusions may be most applicable to circumstances in China and Asia. Furthermore, this study only evaluated test results-objective outcomes-and it did not assess student attitudes about the PBL and LBL models because these subjective outcomes were not "objectively" measured in the original studies. We also decided to omit a description of subjective outcomes and include only objective outcomes. The overall quality of the included studies was not high. The mean modified Jadad score for the included studies was only 3.6, and 54.8% of the studies showed modified Jadad scores of less than 4. This study on the effectiveness of the PBL teaching model was also not completely randomized or conducted through double-blind trials. Among the 10 studies on RCT, only two described processes used for randomization sequence generation. These issues may have resulted in low modified Jadad scores and information bias. Higher quality studies on RCT must be examined to better assess the effect of PBL teaching methods.
The three indicators analyzed in this meta-analysis all exhibited marked degrees of heterogeneity. The high heterogeneity may be attributable to the variations in PBL implementation procedures, varying degrees of difficulty in examinations and varying levels of teaching quality among the included studies. Although meta-regression and subgroup meta-analyses were performed, much of the heterogeneity in the subgroup was not eliminated. Heterogeneity among PBL methods is a challenge inherent of all PBL research [13]. Understandings of PBL differ considerably between researchers [58] (e.g., PBL and LBL teaching model teaching hours, course examination methods, etc.). Unfortunately, most of the studies examined did not include detailed information on these factors. Therefore, a random-effect model was applied as the meta-regression model in this study. The heterogeneity maybe affected the reliability of the conclusions of this meta-analysis to some extent. This study also presents a slight degree of publication bias. Though we conducted a search for literature through the PubMed and EMBASE databases, no non-Chinese studies listed on these databases meet the inclusion criteria. We did not perform a grey literature search, which may have generated information on publication bias. The fixed-effect and random-effect model analysis results of the sensitivity analysis are consistent. This indicates that the analysis results of this research are robust and reliable to a certain degree.

Conclusions
PBL teaching model application in introductory undergraduate medical courses can increase course examination excellence rates and scores in Chinese medical education system,. The PBL teaching model is more effective when applied in laboratory course settings than when applied in theory-based course settings.