https://orcid.org/0000-0001-6647-9497
Correction
9 May 2024: Manuaba IBAP, -No Y, Wu CC (2024) Correction: The effectiveness of problem based learning in improving critical thinking, problem-solving and self-directed learning in first-year medical students: A meta-analysis. PLOS ONE 19(5): e0303724. https://doi.org/10.1371/journal.pone.0303724 View correction
Figures
Abstract
Background
The adaptation process for first-year medical students is an important problem because it significantly affects educational activities. The previous study showed that 63% of students had difficulties adapting to the learning process in their first year at medical school. Therefore, students need the most suitable learning style to support the educational process, such as Problem-based learning (PBL). This method can improve critical thinking skills, problem-solving and self-directed learning. Although PBL has been adopted in medical education, the effectiveness of PBL in first-year medical students is still not yet clear. The purpose of this meta-analysis is to verify whether the PBL approach has a positive effect in improving knowledge, problem-solving and self-directed learning in first-year medical students compared with the conventional method.
Methods
We searched PubMed, ScienceDirect, Cochrane, and Google Scholar databases until June 5, 2021. Search terms included problem-based learning, effectiveness, effectivity, and medical student. We excluded studies with the final-year medical student populations. All analyses in our study were carried out using Review Manager version 5.3 (RevMan Cochrane, London, UK).
Citation: Manuaba IBAP, -No Y, Wu C-C (2022) The effectiveness of problem based learning in improving critical thinking, problem-solving and self-directed learning in first-year medical students: A meta-analysis. PLoS ONE 17(11): e0277339. https://doi.org/10.1371/journal.pone.0277339
Editor: Huijuan Cao, Beijing University of Chinese Medicine, CHINA
Received: July 14, 2021; Accepted: October 25, 2022; Published: November 22, 2022
Copyright: © 2022 Manuaba et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The adaptation process for first-year medical students is an important problem because it is one of the factors that significantly affect educational outcomes [1]. Struggling can occur at any time, but first-year students are particularly susceptible as they adapt to new learning methods at university [2]. A study on the adaptation process of first-year medical students involving 200 participants showed that 63% of students had problems adapting to the learning process [3]. Consequently, students need to know the most suitable learning style to support the educational process. In addition, the appropriate learning approach can also help the adaptation process of first-year medical students and maximize their study outcomes. Therefore, educational institutions need to ensure that applied learning methods improve the learning atmosphere for first-year medical students [4].
Problem-based learning (PBL) encourages students to identify their knowledge and skills to achieve specific goals [5]. Many studies have evaluated the effectiveness of PBL in the medical curriculum and found that PBL can improve understanding, team performance, learning motivation, student satisfaction, and critical thinking [5, 6]. The PBL method not only helps students to understand in-depth, but it also encourages independent learning in students because they have to formulate their own learning goals after understanding PBL scenarios, solve their problems via literatures and internet, compare scenarios with theories from various sources and actively participate in group discussions [7]. PBL has three main learning objectives, namely (1) to apply deep content learning, (2) to apply problem analysis skills and develop solutions to solve problems, and (3) to apply self-directed learning as an approach to adapt learning styles [8]. Therefore, this teaching model has been highly praised in medical education courses in the past two decades [9]. In conventional lecture methods, students are passively exposed to the material and less likely to learn or apply concepts actively. Meanwhile, in PBL, students will learn actively using case-based peer-to-peer teaching, stimulating students to learn based on lecture materials and independent learning to solve cases under the guidance of a facilitator. The PBL approach aims to promote the integration of learned knowledge, rather than simply implanting knowledge and skills compared with the conventional teaching model [8] and also has been design to emphasizes active participation, problem-solving, and critical thinking skills compared to conventional medical education practices [6].
Several reports have showed the effectiveness of PBL for the first-year medical students in improving the final score with the help of map concept compared to PBL only group. The average score was improved significantly, namely 10.07±3.49 versus 5.97±2.09, p<0.001 [10]. Another study compared the final score between the PBL method and the conventional method accompanied by a workshop for first-year medical students. The final results were also statistically significant, namely 8.25±0.79 versus 5.46±0.96, p<0.01 [11]. However, due to the limitation of the studies, the effect of PBL for first-year medical students is yet to be concluded. Also, there is still no meta-analysis that evaluates this topic to date. Therefore, we conducted a systematic review and meta-analysis to verify whether the PBL approach has a positive effect in improving knowledge/critical thinking, problem-solving and self-directed learning in first-year medical students compared with the conventional method.
Methods
Study design
A Meta-analysis was performed from March to June 2021 to assess the effectiveness of PBL in improving knowledge/critical thinking, problem-solving and self-directed learning in first-year medical students. To attain our goal, potentially relevant papers were identified and collected from PubMed, Cochrane, ScienceDirect, and Google Scholar to calculate the mean difference and 95% confidence interval (95%CI) using a random and fixed-effect model. We used meta-analysis protocols as a guide in our present study [12].
Search strategy
We conducted a systematic search in PubMed, Cochrane, ScienceDirect and Google Scholar for search strategy up to June 5, 2021. The search strategy conformed to medical subjects heading (MeSH), involving the use of a combination of the following keywords: (Problem-based Learning [MeSH Major Topic]) AND (effectiveness OR effectivity AND medical student AND first-year). Language constraints were applied in our quest policy. We only used the bigger sample size analysis, which was up to date when we saw the same results in the experiments. We also scanned the possible papers of the appropriate or qualifying studies reference list by searching "Articles linked”. Two independent inspectors found potentially vital records (I.B.A.P.M, Y.N). Disagreements between two independent researchers related to the article were settled by a debate and/or consultation with the senior investigator for finding the third opinion (C.C.W).
Eligibility criteria and data extraction
The inclusion criteria for this study included: (1) research subjects were medical students at the first year (first or second semester), (2) study that evaluated the knowledge/critical thinking, problem-solving and self-directed learning of the student, (3) study that provided sufficient data for calculation of mean difference and 95%CI, p-value, and study heterogeneity. Meanwhile, the exclusion criteria were as follows: (1) studies with insufficient data, (2) samples size less than 50, (3) intervention duration less than one year, (4) review, letter to the editor, and comments articles. Data extraction was conducted by two authors (I.BA.P.M, Y.N). Both of those authors independently screened the collected article’s title, abstract, and full text. Two reviewers extracted the data, which was then extracted to Google Spreadsheet by two reviewers (I.BA.P.M, Y.N). Information was derived from each article included in this study as follows: (1) first author’s name and year of release, (2) age of the participant, (3) interventional and control method, (4) sample cases and control sizes, (5) country of study, (6) study program, (7) duration of PBL intervention, (8) score of PBL and control group. Two independent authors carried out data extraction to prevent human mistakes. If there were a disagreement, a discussion would be held to discuss the solution.
Quality assessment
Two independent authors (I.BA.P.M, Y.N) assessed the quality of the studies to ensure each sample’s validity and prevent the possible exaggeration of each study. The authors use major and minor criteria in assessing the risk of bias for quality assessment. There were four major and four minor criteria. The authors assigned 2 points each to the major criteria and 1 point each to the minor criteria so that the total score would be 12 points. If the article got 9–12 points, then it assigned as “low-risk bias,” if the article got 6–8 points, then it assigned as medium risk bias”, and if the article got < 5 points, then it assigned as “high risk of bias”. When there was a disagreement between the two authors, a discussion was held. If the conflict has not been settled, the two authors discuss it with the third author (C.C.W).
Statistical analysis
Assessment of Methodological Quality of Individual Trials in each article was assessed at the risk of bias before enrolling in meta-analysis. The Z-test was used to assess the effectivity learning method from self-directed learning and its sub-group analysis, critical thinking/knowledge, and problem-solving. Forest plots defined the group measurement and impact estimate. Heterogeneity was provided by using several parameters that we provide, such as Chi2, Tau2, and I2. In the beginning, Comprehensive Meta-Analysis (CMA, New Jersey, US) version 2.1. was used to assess effect models. If the p-value was less than 0.10, the random-effect model was used to evaluate heterogeneity. In contrast, a fixed-effect model was used if the P-value > 0.10. Our study’s analyses were carried out using Review Manager version 5.3 (RevMan Cochrane, London, UK) and Comprehensive Meta-Analysis (CMA, New Jersey, US) version 2.1.
Results
Literature searching
This systematic review and meta-analysis extracted articles from four databases: PubMed, Cochrane, ScienceDirect, and Google Scholar. We found 5536 articles for identification. There was 11 article record removed before screening due to duplication. In the first step screening, there were 5407 articles excluded due to a mismatch of the titles and abstracts. Thus, 120 articles were recorded and continued to the next screening. From 120 articles, the full text was not available for 39 articles. Then, 81 articles were assessed for eligibility according to the inclusion and exclusion criteria and bias quality. There were several articles excluded as follows: no information about duration intervention (n = 16), low sample size (<50 samples) (n = 17), not appropriate study method (n = 12), and insufficient data (n = 29). Finally, seven articles were enrolled in this review (Fig 1).
From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta- Analyses: The PRISMA Statement PLoS Med 6(7): e1000097. doi: 10.1371/journal.pmed1000097 For more information, visit www.prisma-statement.org.
Baseline characteristic involves the study and quality assessment
All of the studies were published within the last 20 years, and most were in Asia. The sample sizes of the seven studies ranged from 56 to 131 participants, and the pooled sample size was classified into two groups (PBL vs. conventional learning methods). The participants were in the first year of medical student major (three articles), dentist major (one article), nurse major (two articles), and midwife major (one article). The length of the intervention varied from several months to one year. No specific gender was evaluated (Table 1). All of enrolling studies were in various study types. According to our assessment, two studies had a low risk of bias (score range 9–12 points), and the remaining articles had a medium risk of bias (score range 6–8 points).
The effectiveness comparison of PBL and conventional learning method
The critical thinking/knowledge evaluation.
In our finding conventional learning method consist of the conventional method (two articles), LBL (lecture base learning) (two articles), tutorial learning group (one article), and theory-based discussion (one article). Three studies show a higher PBL pre-test score, and three studies show a higher conventional method pre-test score. Meanwhile, there was not much difference in the mean score of each group in the pre-test. Evaluation post-test score after the intervention was found to be improved in each group. Post-test scores among PBL groups were mostly higher than the conventional group, except Choi et al., study. It is also in line with Choi et al.’s conclusion that stated no significant finding. In addition, Lohman et al. study also found that different teaching methods did not significantly influence students’ knowledge (Table 2).
This table provides pre/post-test results in each group and the authors’ interpretation of their funding.
The problem-solving evaluation.
Two articles investigated the critical thinking or knowledge aspect and problem-solving. There was not much difference in the average value of the pre-test and post-test result; meanwhile, Lohman et al.; found a significant association between the learning method and problem-solving aspect. Meanwhile, Choi et al. did not find a significant difference in each learning method in the problem-solving aspect (Table 3).
The self-directed learning evaluation.
Three articles evaluated self-directed learning. In the Lohman et al. study, the course instructor assessed the student and scored self-directed learning. The higher score obtained, the better level of self-directed learning. Unfortunately, no significant difference was found in comparing learning methods to enhance self-directed learning in all the included studies (Table 4).
Meta-analysis assessment
Our meta-analysis assessment classified three groups: critical thinking/knowledge, problem-solving, and self-directed learning.
Critical thinking/knowledge assessment.
Six articles evaluated the critical thinking/knowledge in conventional and PBL groups. This analysis used random effect due to p-value of heterogeneity <0,10. The heterogeneity of these articles was evaluated by using the I2 parameter. According to ReVMan analysis, we established I2 was 93%. It belonged to 75% to 100% classification that had good heterogeneity. We found that for developing critical thinking, PBL was a better program. Unfortunately, there is no significant difference between PBL and conventional learning methods (p = 0.29) (Fig 2). This section had a sub-group analysis according to duration intervention and Asia’s critical thinking aspect (Fig 3).
(A) Fixed effect models. (B) Random effect models.
Moreover, the critical thinking studies were regrouped according to the duration of the intervention (≤ 6 months vs > 6 months) and countries (Asia vs. western). The analysis of the duration intervention found no significant difference between PBL and conventional learning methods. Subgroup analysis was assessed by using random effect and fixed-effect models. The studies with the duration intervention at more than six months and learning method comparison in western countries subgroup were found to have low heterogeneity ((I2 = 0% (might not be important)). However, high heterogeneity scores were found in the studies with duration intervention less than six months (I2 = 96%) and learning method comparison in Asian countries subgroup (I2 = 95%). We discovered no statistical difference between PBL and conventional learning methods in each group even though the test for each subgroup analysis’s overall effect from the forest plot graph (diamond) is more inclined to the PBL (Fig 3).
Problem-solving.
We found two studies that discussed the problem-solving aspect between PBL and conventional learning methods. Both studies had good heterogeneity (I2 = 86% (considerable heterogeneity). The overall results were analyzed by using random effects. It is more toward the conventional teaching for enhancing problem-solving skills, but it was not statistically significant (Fig 4).
Self-directed learning.
Self-directed learning was evaluated by using a fixed-effect model. The heterogeneity by using the I2 parameter has shown no heterogeneity (0% = might be unimportant)—the overall effect was more inclined toward the conventional method for enhancing self-directed learning. However, there was no statistical difference (p = 0.34) (Fig 5).
Discussion
Problem Based Learning is a learning method developed to be used as a solution to conventional learning methods that have been used in various disciplines, one of which is health science. Problem Based Learning is a learning method that emphasizes the active participation of students in solving and solving a given problem, both in group and individual settings, so that it can improve students’ skills in analyzing and solving problems [5, 6].
Various studies have been conducted regarding the effectiveness of PBL to be applied in the teaching and learning process [13, 16, 18]. Several factors may influence the implementation of PBL, such as the number of years of study from students, the material taught, and the field of knowledge pursued by students. According to the critical thinking/ knowledge aspect, we found no significant difference between the conventional learning method group and the PBL group (p = 0.29). This finding likely resulted from the lack of association between PBL in enhancing critical thinking/knowledge in the majority of the study. Three studies showed insignificant results from six studies analyzed, and only Tripathi’s (2015) [18] has a linear result with our hypothesis. Accordingly, Choi et al. stated that their insignificant (p = 0.7) finding was due to a short amount of time of the intervention to produce any meaningful effects [17]. Therefore, intervention duration might not be an absolute factor of PBL effectiveness, as found by Tripathi [18]. Likewise, this study also had the shortest intervention duration but still found significant results. Moreover, research conducted by Li et al. related to critical thinking showed a significant difference between the experimental and control groups (p < 0.001) [20]. Then, Tseng et al., also reported a significant difference in critical thinking scores between the experimental and control groups, where the experimental group had a higher score (p < 0.0001) [21].
The research sample characteristics can also affect the PBL results. In this meta-analysis, we analyzed the medical students’ data in their first year. First-year students often experience obstacles in adapting to lecture methods that are different from high school teaching methods [1]. This problem is influenced by various factors, one of which is the difference in lecture methods in each institution. Adaptation to new environments and habits is also a challenge for medical students in the first year. Adaptation to learning methods is a process of response in terms of mental and individual behavior to a demand from the individual or a formal task related to academic work. Therefore, students familiar with the teacher-centered method tend to face difficulty applying the student-centered with PBL method in higher education. They also tend to experience challenges in accepting the study materials, which impact the teaching and learning process in the first semester of lectures for medical students [22]. Those factors explained above could also affect problem-solving and self-directed learning.
Other aspects besides critical thinking/knowledge of the PBL are problem-solving and self-directed learning. We found that PBL is not superior to conventional learning in enhancing problem-solving (p = 0.47). It might be due to the limited studies that assessed this issue and included in this study. The problem-solving aspect was only analyzed in two studies, and they have different results. Choi et al. [17] had a higher total sample, and the study also had a higher weight analysis (56.9%) compared to Lohman et al. [13]. Therefore the results will tend to follow Choi et al. (insignificant finding) [17], besides several aspects as explained above.
Similar results with problem-solving aspect, PBL also failed to show any superiority in increasing self-directed learning compared to the conventional learning method. Two studies in this aspect had shown insignificant results, such as Lohman et al. (2002) [13] and Choi et al., [17]. However, different findings were reported Hayashi et al. (2013) [16]. According to the baseline characteristic of the study, Hayashi’s study had a longer duration of intervention than Lohman et al. (2002) [13] and Choi’s [17] studies. Thus, it might impact the results because the study subjects were exposed to the intervention much longer, so the desired effect was seen [16]. The PBL learning system that focuses on increasing the active participation of students is expected to be able to improve those aspects compared to using the conventional approach. Research by Tseng et al., 2011 involving 120 nursing students (51 in the experimental group, 69 in the control group) showed a significant difference in self-directed learning scores, where the experimental group had a higher mean value than the control group (p < 0.0001) [21]. Three aspects of PBL were evaluated in this meta-analysis, and none were significant. Unfortunately, the specific aspect that might impact the result did not mention or explained in each study in detail.
The problem-based learning method has been used widely, and to the best of our knowledge, further investigation about this learning method is needed. The strength of this study was that our meta-analysis evaluated the specific outcome of PBL such as critical thinking/knowledge assessment, problem-solving, and self-directed learning. Several studies discuss the PBL effect on general learning outcomes and specific backgrounds [9, 14, 18, 19, 23]. Our meta-analysis not only provided pre-test and post-test scores in each group, but we also explained the outcome in each study. Furthermore, we noted that high levels of heterogeneity across studies were found in this meta-analysis. Factors that may cause heterogeneity include the sample from different countries with different backgrounds. Second, the instrument used to evaluate the PBL progression in each study was different. Third, the duration of intervention was also varied, bringing different outcomes. All of these factors may contribute to our meta-analysis heterogeneity. Subgroup analysis has been conducted to minimize the heterogeneity. This method can only reduce the heterogeneity in terms of the critical thinking/acknowledgment aspect, especially when the duration of intervention was more than six months and when the learning method was compared in the Western country sub-group. Meanwhile, no effect was found in terms of heterogeneity when duration of intervention was less than six months, and the learning method was conducted in the Asian countries sub-group. It might be due to several factors that have been pointed out above. Unfortunately, we cannot run subgroup analyses due to limited studies discussing this topic.
Additionally, we believe that further primary study is needed to evaluate the effectiveness of PBL. A multicenter approach is suggested as the most appropriate method to identify the cumulative effect and the difference between geographic areas or races. Moreover, researchers can also compare between educational centers as well as the impact of culture and technological progress of the local area in the implementation of PBL due to the rarity of the study regarding these topics. Psychological aspects also need to be discussed because medical students in the first year may still have the learning method from high school, potentially affecting the PBL.
Conclusion
In conclusion, according to our analysis, PBL is not superior to conventional learning in improving critical thinking/knowledge, problem-solving and self-directed learning in first-year medical students. In addition, our meta-analysis had several limitations, such as only evaluating the learning outcomes in the first year, and no studies were found with multiyear approach. We could not equate the instruments used in PBL and did not evaluate specifically based on the study program. We also could not assess the socio-demography that might contribute to their learning process, particularly their social culture. Therefore, a multicenter approach is suggested as the most appropriate method to identify the cumulative effect and the difference between geographic areas or races.
References
- 1. Ocheretnyuk A., Lysenko D, Palamarchuk OV. The problem of adaptation of first-year medical students to the educational process. Reports Vinnytsia Natl Med Univ. 2018;22(3):543–7.
- 2. Picton A, Greenfield S, Parry J. Why do students struggle in their first year of medical school? A qualitative study of student voices. BMC Med Educ. 2022;22(1):100. pmid:35172820
- 3. Rahayu M, Arianti R. Penyesuaian Mahasiswa Tahun Pertama Di Perguruan Tinggi: Studi Pada Mahasiswa Fakultas Psikologi UKSW. J Psikol Sains dan Profesi. 2002;4(2):73–84.
- 4. Hernández-Torrano D, Ali S, Chan CK. First year medical students’ learning style preferences and their correlation with performance in different subjects within the medical course. BMC Med Educ. 2017;17(1):1–7.
- 5. Shin IS, Kim JH. The effect of problem-based learning in nursing education: A meta-analysis. Adv Heal Sci Educ. 2013;18(5):1103–20. pmid:23283571
- 6. Zhou J, Zhou S, Huang C, Xu R, Zhang Z, Zeng S, et al. Effectiveness of problem-based learning in Chinese pharmacy education: A meta-analysis Approaches to teaching and learning. BMC Med Educ. 2016;16(1).
- 7. Sultana A, Riaz R, Tehseen I. Comparison of problem based learning with traditional teaching as perceived by the students of Rawalpindi Medical College. Rawal Med J. 2010;35(2):238–41.
- 8. Veronese C, Richards JB, Pernar L, Sullivan AM, Schwartzstein RM. A randomized pilot study of the use of concept maps to enhance problem-based learning among first-year medical students. Med Teach. 2013;35(9). pmid:23617466
- 9. Gao J, Yang L, Zhao J, Wang L, Zou J, Wang C, et al. Comparison of problem-based learning and traditional teaching methods in medical psychology education in China: A systematic review and meta-analysis. PLoS One. 2020 Dec;15(12):e0243897. pmid:33315939
- 10. Azer SA. Learning surface anatomy: Which learning approach is effective in an integrated PBL curriculum. Med Teach. 2011;33(1):78–80. pmid:21182390
- 11. Pourshanazari AA, Roohbakhsh A, Khazaei M, Tajadini H. Comparing the long-term retention of a physiology course for medical students with the traditional and problem-based learning. Adv Heal Sci Educ. 2013;18(1):91–7. pmid:22361894
- 12. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul;6(7):e1000097. pmid:19621072
- 13.
Lohman MC, Finkelstein M. to foster problem-solving skill. 2002;121–7.
- 14. Tiwari A, Lai P, So M, Yuen K. A comparison of the effects of problem-based learning and lecturing on the development of students’ critical thinking. Med Educ. 2006;40(6):547–54. pmid:16700770
- 15. Sangestani G, Khatiban M. Nurse Education Today Comparison of problem-based learning and lecture-based learning in midwifery. YNEDT. 2013;33(8):791–5.
- 16. Hayashi S, Tsunekawa K, Inoue C, Fukuzawa Y. Comparison of tutored group with tutorless group in problem-based mixed learning sessions: a randomized cross-matched study. BMC Med Educ. 2013 Dec;13:158. pmid:24289490
- 17. Choi E, Lindquist R, Song Y. Effects of problem-based learning vs. traditional lecture on Korean nursing students’ critical thinking, problem-solving, and self-directed learning. Nurse Educ Today. 2013 Mar;34.
- 18. Tripathi RK, Sarkate PV., Jalgaonkar SV., Rege NN. Development of active learning modules in pharmacology for small group teaching. Educ Heal Chang Learn Pract. 2015;28(1):46–51. pmid:26261114
- 19. Friedrich O, Hemmerling K, Kuehlmeyer K, Nörtemann S, Fischer M, Marckmann G. Principle-based structured case discussions: do they foster moral competence in medical students?—A pilot study. BMC Med Ethics. 2017 Mar;18(1):21. pmid:28253882
- 20. Khatiban M, Sangestani G. The effects of using problem-based learning in the clinical nursing education on the students’ outcomes in Iran: A quasi-experimental study. Nurse Educ Pract. 2014;14(6):698–703. pmid:25458229
- 21. Tseng H-C, Chou F-H, Wang H-H, Ko H-K, Jian S-Y, Weng W-C. The effectiveness of problem-based learning and concept mapping among Taiwanese registered nursing students. Nurse Educ Today. 2011;31(8):e41–6. pmid:21159411
- 22. Elviani P. Vol. 1, No. 6, Desember 2020. Jurnalt Heal Sains. 2020;1(6):401–6.
- 23. Luy-Montejo C. Problem Based Learning (PBL) in the Development of Emotional Intelligence of University Students. Propósitos y Represent. 2019;7(2):353–83.