Effectiveness of blended learning in pharmacy education: An experimental study using clinical research modules

Background &objectives Though there are studies to evaluate the effectiveness of blended learning in pharmacy education, most of them originate from USA and have used previous year students’ scores as control. Also there is less research in comparing use of self -regulated learning strategies between blended and other learning strategies. Primary aim was to evaluate the effectiveness of blended learning on knowledge score using clinical research modules. Secondary objective was designed to compare the use of self-regulated learning strategies between blended learning, web-based e-learning and didactic teaching. Materials and methods A prospective cluster randomized trial was conducted with didactic teaching as control and web-based e-learning and blended learning as interventions. The target population was final year Pharm D students. Outcome was assessed using a validated knowledge questionnaire, a motivated strategies for learning questionnaire and a feedback form. All statistical analyses were carried out using Statistical Package for Social Science (SPSS) Version 20. Results A total of 241 students from 12 colleges completed the study. Mean knowledge score of students in blended learning group was higher than those in the didactic teaching and web- based e- learning program (64.26±18.19 Vs 56.65±8.73 Vs 52.11±22.06,p<0.001).Frequency of use of learning strategies namely rehearsal, elaboration, organization and critical thinking was statistically significantly higher in the blended learning group compared to those of didactic and web-based e-learning group (p<0.05) But there were no statistically significant difference of motivational orientations between didactic and blended learning group except strategies of extrinsic goal orientation and self-efficacy. Students preferred blended learning (86.5%) over didactic and web-based e-learning. Conclusion Blended learning approach is an effective way to teach clinical research module. Students of blended learning group employed all motivational and learning strategies more often than students of the didactic and web- based e-learning groups except strategies of intrinsic goal orientation, task value, control of learning belief and help seeking.


Materials and methods
A prospective cluster randomized trial was conducted with didactic teaching as control and web-based e-learning and blended learning as interventions. The target population was final year Pharm D students. Outcome was assessed using a validated knowledge questionnaire, a motivated strategies for learning questionnaire and a feedback form. All statistical analyses were carried out using Statistical Package for Social Science (SPSS) Version 20.

Results
A total of 241 students from 12 colleges completed the study. Mean knowledge score of students in blended learning group was higher than those in the didactic teaching and webbased e-learning program (64.26±18. 19 Vs 56.65±8.73 Vs 52.11±22.06,p<0.001).Frequency of use of learning strategies namely rehearsal, elaboration, organization and critical thinking was statistically significantly higher in the blended learning group compared to those of didactic and web-based e-learning group (p<0.05) But there were no statistically significant difference of motivational orientations between didactic and blended learning

Introduction
Blended learning (BL) or the integration of face-to-face and online instruction is widely adopted across the globe for tertiary education with the increasing use of information technology and electronic learning [1]. There is considerable growth in research analyzing the effectiveness of BL and understanding students' preference and satisfaction towards BL in pharmacy education [2][3][4][5][6]. Some studies have concluded that BL is as effective as traditional teaching [2,3] while a few have reported that BL is more effective than conventional teaching [4][5][6]. However, most of these studies originate from US, and have employed previous year students' scores as control. Moreover, only few studies come from developing countries. Currently lacking is, prospective controlled trials to analyze the effectiveness of BL in pharmacy education. A systematic review and meta-analysis on this topic is published by same authors [7]. Accordingly, we designed our study to evaluate the effectiveness of BL in final year Pharm D (Doctor of Pharmacy) students employing a 9 hour learning module entitled "Fundamentals of clinical research". "Clinical research" was chosen because India is the preferred destination for multinational companies for clinical trials which raises the demand for skilled qualified personnel. Academia is unable to keep pace with industry because of quick regulatory reforms and technology upgrades thereby making timely syllabus revisions difficult. Training in these areas will help graduates gain more updated knowledge [8,9]. Self-regulated learning (SRL) involves cognitive, metacognitive, motivational, emotional, and behavioural aspects of learning and is currently a subject of extensive research and discussion in educational psychology. Student learning strategies are dynamic and developed to achieve learning objectives in a limited time span [10]. However, there is a little or no study comparing SRL strategies between BL and conventional methods [11]. In this study we have used Pintrich' motivated strategies for learning questionnaire (MSLQ) [12,13] comprised of 15 variables and measures cognitive, metacognitive, motivational, and emotional aspects of learning procedures (Details of MSLQ is provided in S1 File). MSLQ successfully measures motivation and learning strategies in higher education, irrespective of discipline. Accordingly, we designed our secondary objective to compare the use of motivation and learning strategies between BL, web-based e-learning (WEL) and didactic teaching (DT). Blended learning refers to combination of online and face-to face, whereas web-based e-learning refers to all education that takes place online, while didactic teaching refers to all learning experiences that take place face-to-face in the classroom [14][15][16][17].

Design and implementation of web-based e-learning platform
Description of the learning module. This learning module was intended to teach sitebased clinical trial activities to final year Pharm D students. The learning module comprised of five chapters, which included how to start a clinical trial, randomization, informed consent procedure, electronic case report forms, and adverse drug reaction reporting. Each chapter described the theory with the aid of case studies. Details of each chapter are outlined in Table 1. Experts from relevant areas verified contents in each chapter.

Trial design
A prospective cluster randomized trial, having three arms with DT as control and WEL and BL as interventions 1 & 2 respectively, was conducted between October 2019 to February 2020.

Ethical consideration
Ethical approval for the study was obtained from Institutional Ethics Committee of Kasturba Hospital Manipal (25/2018), Karnataka, India. This study was registered under clinical trial registry of India (CTRI/2019/11/022021).

Inclusion criteria for colleges and participants
Colleges affiliated with All India Council for Technical Education (AICTE)/Pharmacy Council of India (PCI) were included. The target population was final year Pharm D students. Students who were interested in participating in the research study were included.
would be allocated to control/interventions based on randomization. Consent was obtained from institutional heads for colleges' participation.
Once consent obtained from institutional heads, investigators visited respective colleges and explained the study to participants. Participant information sheet(PIS) was provided to participants and all interested candidates willfully signed written informed consent form.

Sample size calculation
Sample size was calculated based on the primary outcome, total knowledge score. Effect size was calculated based on a pilot study, involving 22 students from Manipal College of Pharmaceutical Sciences, Manipal (S2 Appendix for pilot study details). Difference between the largest and smallest means divided by the square root of the mean square error, was used to compute the effect size [18]. A similar effect size was obtained by our meta-analysis of effectiveness of blended learning in pharmacy education [7]. Effect size (σ) of 0.6, at 5% level of significance, 80% power and design effect (d) of 2, the required sample size was 90 in each method of learning strategies. The required number of colleges in each geographical area was 3, assuming approx. 30 students in each class to each method of learning strategies. Students who participated in pilot study were excluded from the main study.

Randomization
Colleges that granted permission to conduct the study were grouped based on the geographical areas (GA) namely, Southern Karnataka (GA1), South East Karnataka (GA2), and Kerala (GA3) First, these geographical areas were randomly allocated to three methods of learning strategies and within each geographical area, 4 colleges were selected. There was no random selection of institutions in GA1 as there was only 4 institutions agreed to participate. Eight institutions out of ten agreed to participate were chosen at random in GA2 and GA3 (To reduce the contamination bias, we chose geographical areas as clusters. If students from surrounding colleges enrolled for different arms, there is a chance that they will share their login id & password).

Study procedure
DT, WEL and BL had the same course contents. All face-to-face sessions in DT & BL were conducted in respective colleges by same experts. Chapters were presented in the format of Microsoft PowerPoint with synchronous voice over slides in website. Script of voice was also attached in platform. Same PowerPoint slides were used to take classes in DT. Website contained 5 chapters, each lasting one hour as well as 4 hours of additional case studies and simulated forms, totaling 9 hours. Each chapter contained theoretical explanation, followed by reinforcing the concept with one or two case studies. Additional case studies and simulated forms were attached to the platform, which was available only to students of WEL. There was a case scenario, based on which students were asked to fill the simulated forms to provide realistic experiences. Simulated form included feasibility questionnaire, ADR reporting form, electronic case report form, and informed consent form. Students of BL were unable to access additional case studies and simulated forms on the website which was taught by face-to-face instruction. Students were required to sign up for first time and log in to the web page to access the learning materials. Students were enrolled for a study period of 8 weeks (details of each learning strategy is depicted in S3 Appendix).

Web-based e-learning platform for Fundamentals of clinical research
CLINIC E-LEARNING (http://clinilearn.in/my/): A new learning management system, CLIN-ILC E-LEARNING, was developed to implement training program for students. This platform was developed using Moodle platform Version 3.2. PHP 7.0x was used to code the contents. CLINIC E-LEARNING contained five chapters as described above, and each chapter had preand post-test, case study questions, case study for references, important links and files, simulated forms, assignments and feedback forms. Only eligible participants were permitted to access the learning management system through login id and password. (See S4 Appendix for details of Website).

Experimental and control group
DT-(control). Students attended the 9 hour face-to-face lecture delivered by experts with the aid of PowerPoint slides (Same slides of website). Face-to face session was conducted via workshops in each selected college.
WEL-(Experimental group 1). WEL group were provided with URL and were asked to sign up with the option to access the website anytime, anywhere. They could clarify doubts by posting their queries to experts in the learner management system. Students were told to complete 9 hour learning module (5 chapters within 6 weeks and remaining two weeks for case study analysis and practicing simulated forms) within 2 months.
BL-(Experimental group 2). BL students were provided with 5-hour e-learning and 4 hour face-to-face instruction, totaling 9hours. Students of BL studied the same audiovisual slides (in the same website) during the same period. Followed by, 4 hour discussion by experts (face-to-face instruction provided by the DT team) on additional case studies and important forms used in clinical trials.

Outcome assessment
We employed the same tests to assess outcomes in all three groups. Outcomes were measured immediately after the completion of the learning module. Outcome assessment consisted of 3 parts (see S5 Appendix for questionnaires). All questionnaires were validated by 5 experts for content validity. 3. Evaluation of student's experience and satisfaction: Feedback collected in two sessions with 12 questions.

Statistical analysis
All statistical analyses were carried out using Statistical Package for Social Science (SPSS) Version 20. Continuous variables were reported in terms of mean and standard deviation whereas categorical variables were expressed as percentages. Shapiro Wilks test was performed to ensure the normal distribution of variables. One way Anova or Kruskal Wallis test was conducted to find significant differences in test scores between the groups. Pairwise comparison performed by Dunn's post hoc test, and Games Howell test. Relationships between variables were examined by multiple linear regressions. Spearman correlation was conducted to find out the correlation between variables. Mann Whitney U test performed to analyze difference in opinions of students of WEL and BL groups.

Results
A total of 317 students enrolled from 12 colleges, of them 241 students completed the learning module on Fundamentals of clinical research. Only students who finished both pre-test and post-test were included for the analysis.

Comparison of knowledge score
Pretest scores were compared by one way ANOVA and found that there was no significant difference between groups, p>0.05. As shown in Table 2, mean difference between post-test and pre-test, case studies score, and total score in BL was higher than DT and WEL. There was statistically significant difference in mean difference score (Post to pretest) in knowledge (p<0.001), post-case study score (p = 0.034), and total score (p<0.001) between learning strategies. BL showed statistically significant highest total score than didactic teaching (p<0.001) and webbased e-learning program (p<0.001) by Dunn's post hoc test. Results are presented in Table 2. Multiple linear regression model showed an R 2 value of 0.41, which means independent variable (pretest, gender and learning strategies keeping BL as reference category) can explain 41% of the variability of dependent variable (total score). The coefficient on Intervention (β3 = -5.32) means that, holding all other variables constant, the score on the final test of a student who was taught via the DT is expected to decrease 5.32 points compared to a student who received the BL intervention. Similarly, total score of WEL group students are expected to decrease -12.03 compared to BL group. The findings are depicted in Table 3.

Comparison of MSLQ scores
It was found that usage of SRL strategies were higher in BL group than DT and WEL except intrinsic goal orientation, task value, control of learning belief and help seeking. There was no statistically significant difference in test anxiety, time management and effort regulation between groups (p>0.05). SRL strategies are composed of 15 variables and were analyzed by Kruskal Wallis test and presented in Table 4. Further pairwise comparison between the groups showed that there was no statistically significant difference in usage of intrinsic goal orientation, task value and control of learning belief between DT and BL (p>0.05). Pairwise comparison of usage of rehearsal, elaboration, organization, and critical thinking indicated there was no statistically significant difference between the DT and WEL. Peer learning showed no significant difference between DT and BL, whereas significant difference exist between the DT and WEL in help seeking.
To determine the relationship between motivational and learning strategy use and final score, Spearman correlation was performed separately for DT, WEL and BL groups. Result showed final score had significant weak positive correlation with extrinsic goal orientation (r = 0.213), control of learning belief (r = 0.280) and rehearsal(r = 226) for students of BL. We did not find any significant correlation between final score and MSLQ strategies in DT and WEL.

Comparison of students' opinion about contents of web-based e-learning platform
It was found that there was no statistically significant difference(p>0.05) in opinions regarding design of module, clarity of explanation, comprehensive coverage of subject matter, and relevancy of hyperlinks between the WEL and BL groups who used the website for learning. Students of BL were asked their preference for learning strategies and found that 86.5%of students prefer BL, 2.7% and 9.5% preferred DT and WEL, respectively. Students responded that they were satisfied with the clinical research modules and accomplished various learning activities. Mann-Whitney test showed no significant difference between students' opinion (p>0.05).
A total of 92.2% students enrolled for didactic teaching responded that they were satisfied with workshops. More than three quarters of DT students agreed that they were satisfied with clarity of explanation and discussion of module lesson, comprehensive coverage of subject matter, consistency of content with subject objectives and syllabus Results are presented in Table 5.

Discussion
Published literature suggests that BL generates better performance in pharmacy education, but most of such studies used previous year students' score as control [19,20]. To the best of our knowledge, this report will be the first attempt to evaluate the effectiveness of BL in clinical research modules in pharmacy education. Despite the fact that the term "blended learning" is commonly used, there are some uncertainty about what it means. These approaches are still little understood in higher education, and their descriptions in the literature are inconsistent. In higher education, the term "hybrid" is often used synonymously with "blended" learning, "flipped," "online," or "technology-enhanced" learning [15]. A recent article by Kendra Gagnon distinguish blended and hybrid learning based on face-to-face time [16]. Blended learning does not replace face-to-face time but hybrid learning does. To be more specific, hybrid learning reduces seat time in class. As a result, our model will be better suited to hybrid learning. However, we used the term blended learning because literatures describe hybrid learning is a type of blended learning, and a precise definition remains an ongoing conversation [14,15]. In comparison to blended learning, the term hybrid learning might have been more extensively used in practice than research as there are less highly cited articles on hybrid learning. According to Stefan Hrastinski it is important that researchers and practitioners carefully describe what blended learning means to them. He also proposes, researchers should carefully consider while using a more precise, descriptive term as a supplement or replacement for blended learning [15]. So we believe we have adequately addressed this issue. Though Clinical trials appear to be concentrated in developed nations, highest percentage increase in registered human clinical research, particularly phase 3 trials are increasing in developing countries [21]. So training in these areas have the potential to enhance students' basic understanding of how the industry works. We have shown that BL is more effective than DT and WEL in teaching clinical research modules, which is in agreement with previous reports in similar settings [22,23]. Ample data have shown that case studies are a gratifying and motivational educational tool that extends declarative and procedural knowledge/expertise [24]. We incorporated case studies in all three learning strategies. Both DT and BL used case study discussions as an active learning strategy in the classroom and enhanced student tutor interaction. Though case studies were incorporated on website and students could post their queries in learning management system itself, there were no provision of face-to-face tutor to student interaction for students who participated in WEL. Students who experienced face-to-face discussion (DT & BL) demonstrated better performance than WEL group, underlining the importance of face-to-face interaction in improving the final score.
In blended learning model class time was effectively used for active learning strategies as students had already viewed recorded videos. This allowed more meaningful face-to face interaction, application of course content and also provided an equal educational value as didactic teaching. BL students spent less time in class compared to DT, which echoes the result of a similar study by the US Department of Education on Distance Education as well as another study by Philips JA [25,26]. BL employed motivational and learning strategies more frequently than DT, except in strategies of intrinsic goal orientation, task value, control of learning belief and help seeking. WEL students scored less in all scales of motivational and learning strategies compared to BL. However, DT students employed all motivational and learning strategies more often than WEL students, except in extrinsic goal orientation, self-efficacy and test anxiety. Interestingly, there is no significant difference between DT and BL in the use of motivational strategies, except in extrinsic goal orientation and self-efficacy; suggesting that face-to-face interaction with tutor considerably influences the use of motivational strategies. This result is in accordance with a study by Kassab et al [27]. DT scored highest for intrinsic goal orientation but least for extrinsic goal orientation, possibly because DT students were personally motivated by the tutor.
BL showed statistically significant highest score in all scales of cognitive strategies (rehearsal, elaboration, organization, and critical thinking) than DT and WEL. Possible reason could be, in BL, students are getting more meaningful interaction with tutors as they are familiar with the content before class. This finding is mostly in accord with a study by Ruchan Uz [28] who reported use of blended instruction is more effective than traditional instruction in terms of developing learning strategies. However our results are in contrast to those of Broadbent [11] who reported that online learning students reported more frequent usage of learning strategies than BL.
Another key finding of this study was that no significant difference was found in metacognitive self-regulation score between DT and BL. So we assume that face-to-face interaction with tutors can demonstrate what to learn, when to learn and how to learn which in turn affect on metacognitive approaches such as planning, monitoring, and regulating.
As expected peer learning and help seeking score was less in WEL group. Help seeking was highest in DT. Understandably, online students felt more isolated when deprived of a sense of community. This is in line with previous studies [11,29].
We have shown that only extrinsic goal orientation, control of learning belief and rehearsal strategies had a weak positive correlation with final score of clinical research examination in BL. On the other hand, SRL did not correlate with final score in DT and WEL.
To the best of our knowledge, there was only one study by Broadbent et al [11] who compared the use of SRL between online and blended environment in higher education. Our result contradicts Broadbent, possibly because, we employed the same learning module for all evaluation unlike much more broad based subject range by Broadbent et al. Moreover, online students had an opportunity for one-to-one interaction with peers and instructors in Broadbent's study. Since student's attitude towards course content also determine SRL, Broadbent's results may not apply to our situation.
In addition, the blended-learning approach used in this course was generally well received by the students. These findings align with other studies that report positive student perceptions of BL in pharmacy education [30,31].
There are some inherent limitations to this study. Firstly, as this study was focused on a single topic, our findings may not be extrapolated to other topics of pharmacy or any other healthcare disciplines. Second, this study accessed short term retention of material. Third, we did not assess how many times students logged in to the website and how long they spent to viewing the materials. As stated by previous reported study [26] we could overcome this by stating that students logging in does not necessarily mean they were actively studying the content. The time obtained may have provided time that the students may not have been physically seated in front of the screen watching the e-lectures because the lecture window would remain active as long as they were signed on to the web-based e-learning portal. Fourth, due to the anonymous nature of feedback form, we were not able to link students' satisfaction level and academic achievement.

Conclusion and future research lines
Blended learning approach, with online and face-to-face instruction is an effective way to teach clinical research module. Results showed students of blended learning group employed all motivational and learning strategies more often than students of didactic and web-based elearning groups except strategies of intrinsic goal orientation, task value, control of learning belief and help seeking. Also Students prefer blended learning program than didactic and pure online courses.
Blended learning will be the future of the education sector, since online learning cannot replace physical experience at universities. Given the rapid growth of online studies in last decades, and in the wake of COVID 19, it is paramount to evaluate the effectiveness of computer mediated instruction and to understand how students utilise SRL strategies to achieve academic success. There is an urgent need of fine grained studies with solid experimental design to evaluate the use of SRL between various pedagogies. More insights are needed to correlate the use of SRL and academic achievement. Most BL studies, including current research have assessed short-term retention of knowledge, so future research should address long-term retention of knowledge. Thunga.