https://orcid.org/0000-0002-6372-1476
Figures
Abstract
Background
Evidence-based medicine (EBM) is a widely accepted scientific advancement in clinical settings that helps achieve better, safer, and more cost-effective healthcare. However, presently, validated instruments to evaluate healthcare professionals’ attitude and practices toward implementing EBM are not widely available. Therefore, the present study aimed to determine the validity and reliability of a newly developed knowledge, attitude, and practice (KAP) questionnaire on EBM for use among healthcare professionals.
Methods
The Noor Evidence-Based Medicine Questionnaire was tested among physicians in a government hospital between July and August 2018. Exploratory factor analysis and internal consistency reliability-based Cronbach’s alpha statistic were conducted.
Results
The questionnaire was distributed among 94 physicians, and 90 responded (response rate of 95.7%). The initial number of items in the KAP domains of the Noor Evidence-Based Medicine Questionnaire were 15, 17, and 13, respectively; however, two items in the practice domain with communalities <0.25 and factor loadings <0.4 were removed. The factor structure accounted for 52.33%, 66.29%, and 55.39% of data variance in the KAP domains, respectively. Cronbach’s alpha values were 0.81, 0.81, and 0.84 for KAP domains, respectively, indicating high reliability.
Citation: Norhayati MN, Nawi ZM (2021) Validity and reliability of the Noor Evidence-Based Medicine Questionnaire: A cross-sectional study. PLoS ONE 16(4): e0249660. https://doi.org/10.1371/journal.pone.0249660
Editor: Yuka Kotozaki, Iwate Medical University, JAPAN
Received: August 30, 2020; Accepted: February 28, 2021; Published: April 22, 2021
Copyright: © 2021 Norhayati, Nawi. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information.
Funding: This research was funded by Tabung Insentif Pembangunan Pengajian Siswazah PPSP, Universiti Sains Malaysia. The funder provided support in the form of salaries for authors [NMN], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: EBM, evidence-based medicine; EFA, exploratory factor analysis; KMO, Kaiser–Meyer–Olkin
Background
Evidence-based medicine (EBM) is the deliberate, transparent, and judicious use of existing best evidence in decision making about individual patient treatment [1,2]. It relies on three pillars, namely, (i) individual clinical expertise (healthcare professionals), (ii) patient values and desires, and (iii) best research available [3]. Hence, EBM is an essential component of medical practice and constitutes explicitly stated practice [4]. Both knowledge and attitude of healthcare professionals play a vital role in determining EBM implementation [5]. However, a large gap in attitude, information, and practices related to EBM has been reported globally, resulting in incomplete EBM application in real-world scenarios [6,7].
In Malaysia, the implementation of evidence-based practice is low because of limited awareness, knowledge, attitude, and resources. Many healthcare practitioners use non-evidence-based sources and access the same through new approaches such as texting. These findings were derived from a qualitative study among medical officers and family medicine specialists that identified five themes related to EBM among healthcare professionals: (i) doctors predominantly regarded EBM as statistics, research, and guidelines; (ii) reactions to EBM were overwhelmingly negative; (iii) doctors relied on consultants, colleagues, and their guidance, and non-evidence-based internet sources for information; (iv) information sources were accessed using novel methods such as mobile applications; and (v) there were several barriers to evidence-based practice, including physician-, EBM-, patient- and system-related factors, such as inadequacies in knowledge, attitude, management support, time, and access to authentic information sources [8].
Several questionnaires are available for evaluating EBM awareness among healthcare professionals, such as Fresno, Berlin, Baum, McColl, and EBM Questionnaire [9–17]. The most commonly used questionnaire developed by McColl et al. [18] evaluates self-reported awareness, attitudes, and barriers toward EBM [16,19]. Baum’s Questionnaire also measures attitude toward EBM and self-reported EBM capabilities among healthcare professionals [15], whereas the Berlin questionnaire measures knowledge and skills related to EBM [13].
The Fresno test is considered the best questionnaire. It evaluates both competency and skills related to EBM and, thereby, serves as a reliable and accurate method for detecting instruction impact [9]. However, the assessment of inter-rater reliability, internal consistency, and discrimination were intimately dependent on the population which has taken the test and are all likely to be lower with a more homogenous group of evidence-based medicine learners [20]. While the adapted Fresno test is relevant to rehabilitation professionals and has removed some statistical questions [21]. The Assessing Competency in EBM (ACE) tool, in addition to the Fresno test, is also a reliable and validated instrument for evaluating EBM competency among healthcare practitioners. The ACE method offers a novel method of assessment as it evaluates user performance in four main steps; however, its application across various patient scenarios remains limited [10]. Notably, the use of multiple EBM tests has resulted in the incongruous assessment of available EBM domains in a sample because of the heterogeneity of items on these tools [12].
Despite these research tools’ availability, new questionnaires should be developed, adapted and tested. The purpose of the development of the current questionnaire is to assess the depth of knowledge, direction of attitude, and practice of EBM among healthcare professionals. This questionnaire’s scoring system will be based on each item to produce valid overall and cut-off scores irrespective of the population in which it is being tested. As it is necessary to have a tool to identify gaps and improve EBM implementation in clinical practice, the development, and validation of psychometrically robust instruments is crucial. Therefore, the present study aimed to determine the validity and reliability of a newly developed knowledge, attitude, and practice questionnaire on EBM for healthcare professionals.
Methods
EBM questionnaire
The development of the Noor EBM questionnaire began with a literature search on EBM to ensure content quality. A literature search was undertaken in Medline to ensure good content of the scale; keywords such as “evidence-based medicine”, “evidence-based practice”, “healthcare” “doctors”, “questionnaire” and “scale” were used.
Using a modified Delphi method, the questionnaires were structured to encourage research team members to participate in the final round to clarify the problems and present arguments that explain their views [22]. Four experts, including a public health physician, a family medicine specialist, an evidence-based medicine expert, and a biostatistician, were involved in the development process. Each item is dealt with in detail to ensure that it is appreciated in the same way by all respondents. It avoided unclear content in one item or a double-barreled item, complicated or ambiguous words, different thoughts or notions.
Two sets of questionnaires were prepared for each item to ask for a similar context but differently. With ten respondents, including experts in the field and healthcare professionals, the 45-item questionnaires underwent cognitive debriefing. For consistency, appropriateness, and significance, each item was evaluated. The wordings of several items have been updated accordingly.
The Noor EBM questionnaire originally comprised knowledge, attitude, and practice domains with 15, 17, and 13 items. All three domains required responses on a five-point Likert scale with the knowledge and attitude domains using the Strongly Agree = 5/Agree = 4/Neutral = 3/Disagree = 2/Strongly Disagree = 1 scale, and the practice domain using the Always = 5/Often = 4/Sometimes = 3/Seldom = 2/Never = 1 scale. Reverse scoring was used for negatively worded items.
Total scores were calculated for each domain, and each total raw score was transformed into “percent score” by dividing the raw score with the maximum score and multiplying it by 100. It was done to appreciate differences better, as the scale would then range from 0 to 100. The results of this questionnaire can be expressed in outcome scores and categories, and the percent score for each domain can be categorized based on Bloom’s cut-off point (60%–79%) [23]; this method has been used in other studies as well [24,25]. Scores within this range were defined as moderate knowledge, neutral attitude, and fair level of practice. In contrast, scores above this cut-off point were considered as a high level of knowledge, positive attitude, and a good level of practice. Scores below this cut-off point were equated with low knowledge, negative attitude, and poor practice level.
Participants
In this cross-sectional study, construct validity and reliability were evaluated among randomly selected physicians at the Hospital Sungai Buloh between July to August 2018. Physicians registered with the national medical council and working in a government hospital in Selangor, Malaysia, were recruited. House officers were excluded. Sample size calculation was based on the attitude domain’s responses (17 items), and a subject item ratio of 1 to 5 [26] was applied. A sample size of 94 subjects was obtained, assuming a 10% non-response rate.
Convenient sampling was applied wherein the researcher explained the present study and distributed an informed consent form. Once the participant understood and consented to join the study, they were given the self-administered questionnaire and were encouraged to self-complete the EBM scale. The questionnaire was written in English, as it is a global language and used to train doctors in most medical schools. Upon completion, questionnaires were checked for completeness of responses, and the participants were thanked for their co-operation. Participation in the present study was not expected to lead to any potential or foreseeable risk.
Ethics approval and consent to participate
The research proposal was approved by the Research and Ethics Committee of the Universiti Sains Malaysia (USM/JEPeM/18040195) and the National Research Medical Registration (NMRR-18-349-40727). All patients gave written informed consent before answering the questionnaire. Confidentially of the data were maintained through anonymity and presented as grouped data.
Statistical analyses
Exploratory factor analysis (EFA) and internal consistency reliability-based Cronbach’s alpha statistic were used to estimate the questionnaire’s construct validity and reliability. Certain items were carefully eliminated without compromising content validity based on the set parameters. The principal axis factoring extraction method with Promax rotation was applied to produce a solution with the best simple structure. Items were treated as continuous responses to allow evaluation of dimensionality (number of factors) of the items. To meet this assumption, the Kaiser–Meyer–Olkin (KMO) statistic was calculated, and a cut-off value of >0.7 was used to ensure data suitability for factor analysis. Furthermore, Bartlett’s test of sphericity with a p value of <0.05, indicating good correlation among items, was mandated. To determine the number of extracted factors, Eigenvalues of >1.0, parallel analysis, and scree plot inspection was conducted [27]. Communalities >0.25 and factor loadings >0.4 were considered acceptable [28]. For internal consistency reliability, a Cronbach’s alpha coefficient >0.7 was considered acceptable. All statistical analyses were conducted using the IBM SPSS Statistics version 24. A p value of <0.05 was considered statistically significant.
Results
In the present study, questionnaires were distributed to 94 clinicians, and a response rate of 95.7% was recorded. Participants included 32 men (35.6%), 23 specialized clinicians (25.6%), 65 Malays (72.2%), and 34 married respondents (37.8%). The average age (standard deviation, SD) of the cohort was 31.8 years (3.97; range 26–46 years), and median (interquartile range) experience in healthcare was two years (3.0; range 1–10).
Knowledge domain on EBM
The mean score for individual items in the knowledge domain ranged from 2.9 to 4.4 (SD range 0.59–1.07; Table 1). Item-level descriptive statistics of the 15 items in the knowledge domain showed that item K1, “Evidence-based medicine involves the process of critically appraising research findings as to the basis for clinical decisions” had the highest mean (SD) score at 4.4 (0.59).
EFA for the knowledge domain initially yielded a KMO of 0.76, and Bartlett’s Test of Sphericity was significant (P < 0.001). None of the items was removed due to low communalities or low factor loading, and all 15 items were retained with factor loading ranging from 0.42 to 0.78. Factor analysis of the 15 items yielded a three-factor structure that explained 52.33% of the data variance. However, factor 3 comprised only one item. The overall Cronbach’s alpha score for the knowledge domain was 0.81, and the mean (SD) total score for the 15 items was 55.6 (6.31).
Attitude domain on EBM
The mean scores for individual items in the attitude domain ranged from 2.8 to 4.4 (SD range 0.50–1.18; Table 2). Item-level descriptive statistics of the 17 items in the attitude domain showed that item A15, “I think it is mandatory for physicians to continuously update their knowledge to deliver efficient patient care” had the highest mean (SD) score at 4.4 (0.66).
EFA for the attitude domain had an initial KMO value of 0.74, and Bartlett’s Test of Sphericity was significant (P < 0.001). None of the items was removed due to low communalities or low factor loading, and all 17 items were retained with factor loading ranging from 0.44 to 0.92. The factor analysis of the 17 items yielded a five-factor structure that explained 66.29% of the variance in data. However, factors 3, 4, and 5 comprised only three or two items. The overall Cronbach’s alpha statistic for the attitude domain was 0.81, and the mean (SD) total score for the 17 items was 66.2 (6.06).
Practice domain on EBM
The mean score for individual items in the practice domain ranged from 1.6 to 3.7 (SD range 0.80–1.26; Table 3). Item-level descriptive statistics of the 13 items in the practice domain showed that item P6, “I consider patient’s wishes in the decision making process,” had the highest mean (SD) score at 3.7 (0.92).
During EFA, the initial KMO statistic for the practice domain was 0.77, and Bartlett’s Test of Sphericity was significant (P < 0.001). Two items, P6 (I consider the patient’s wishes in the decision making process) and P8 (I only use the computer provided at my workplace rather than a smartphone to search for online databases) with communalities <0.25 and factor loadings <0.4 were removed, and their removal was judged as not affecting content validity. Thus, the final number of items in the practice domain was 11, with factor loading ranging from 0.43 to 0.99. Factor analysis of the 11 items yielded a two-factor structure that explained 55.39% of the data variance, and factor 2 comprised only two items (P4, P5). The overall Cronbach’s alpha statistic for the practice domain was 0.84. The mean (SD) total score for the 11 items was 32.5 (7.09).
Discussion
The advent of EBM and the growing role of the evidence from these studies in the decision making process has necessitated the assessment of relevant competencies among healthcare providers. This Noor EBM questionnaire was intended for use among healthcare professionals, including medical officers and specialist physicians, and hence adequately represents the target population during the evaluation of psychometric parameters in the tool.
The results presented here show that the Noor EBM questionnaire performed well. The reliability of the individual domains was significantly greater than the suggested value for Cronbach’s alpha [29], and this high reliability implies a high degree of confidence in our findings. This is related to the small random error point, which did not exceed 10% to 20%, suggesting that random fluctuations in test results that are defined in classical psychometric test theory do not reduce the value of alpha in this case [30].
The EFA for the practice domain led to the removal of two items, P6 and P8. In P6, “I consider patient’s wishes in the decision making process,” the word “wishes” is rather vague and out of context, which may have resulted in participants rejecting it as a legitimate item. For item P8 (I only use the computer provided at my workplace rather than a smartphone to search for online databases), its rejection may be due to improper selection of search devices in the healthcare delivery setting.
Measurements obtained for the Noor EBM questionnaire’s individual items suggest a relatively high level of similarity among study participants in all three domains, i.e., knowledge, attitude, and practice. Furthermore, we found a minimal variation (low SD range), suggesting high uniformity in measured traits. Similar findings were seen with mean scores for individual items on the questionnaire, implying the absence of a “floor’ and “ceiling” effect, i.e., a narrow range between extreme mean values for all three domains (knowledge: 2.9 to 4.4; attitude: 2.81 to 4.36; and practice: 1.57 to 3.71).
In the present study, the average score of items in the knowledge and attitude domains ranged between 3 and 4, suggesting that the physicians were either “neutral to” or “agreed to” the statements. Although such a score may reflect “unsure” to “reasonably adequate” knowledge and neutral to positive attitudes, it also indicates that greater focus must be laid on increasing EBM knowledge and positive reinforcement of attitudes. The average score of items in the practice domain ranged from about 2 to 4, suggesting that EBM practice ranged from “seldom” to “practice.” This finding indicates that, in the future, special consideration should be paid to behaviour and competency in EBM awareness programs among healthcare professionals.
Although studies have shown positive attitudes toward EBM among healthcare professionals [5,19,31–36], there is evidence of insufficient experience [19,31,33]. One study reported that only a small percentage (<10%) of medical trainee specialists could correctly state the hierarchy of best types of evidence [7]. Many healthcare practitioners have also reported that they did not use it although they had heard about EBM [37].
The most significant impediments to EBM adoption have been listed as differences in human disease (61.0%), followed by a lack of hospital/department funding (39.8%) [5]. A review of 28 papers on barriers to EBM implementation found that the level of understanding, expertise, and evidence-based performance was less than 50.0%. This was due to lack of equipment, resources, and research methodology skills. Additionally, the degree of familiarity with EBM-related terminology was low (44.2%), and textbooks were regarded as the most important source of knowledge [38].
The Noor EBM questionnaire evaluated healthcare professionals’ knowledge, attitudes, and behaviour toward EBM. It has been tested in a survey among primary-care practitioners in Selangor, Malaysia; the results are published in a separate manuscript. The findings can be used to detect and correct obstacles in the workplace that hinder EBM practice. Thus, the questionnaire can also be used for developing training programs, as it can highlight areas that require particular emphasis or focus. The Noor EBM questionnaire’s modular nature is undoubtedly an advantage as individual domains can be used in isolation, based on the needs of the study. Future studies may translate the Noor EBM questionnaire’s English version into the Malay language or any other local languages.
As with any instrument, the Noor EBM questionnaire has some weaknesses. One of them is the inability to determine the use of other pieces of non-scientific information. Clinical decisions based on non-scientific facts, personal experience, practice, or intuitions contradict the principle of EBM. Future testing of the Noor EBM questionnaire in other groups of medical specialists will expand the applicability of this tool. Retesting the questionnaire’s robustness in other populations to produce an adapted shorter version and the degree of agreement with this original is suggested for future research.
Another limitation of the present study is the lack of convergent validity analysis; this psychometric property could not be evaluated because of the lack of a standard tool for evaluating EBM. Second, we developed the items without imposing certain subdomains to test for construct validity. However, the factors that emerged were too inadequate as individual subdomains due to the limited number of items. The separation of items into subdomains did not reveal a specific representational theme. Third, confirmatory factor analysis was not conducted on a different set of samples to confirm the unidimensional construct of the findings. However, scholars have argued that EFA is more appropriate when analyzing new scales [39].
Conclusions
Psychometric analysis of the Noor EBM questionnaire confirms that this tool is of high quality. The questionnaire may be used to evaluate competency, attitude, and behaviour toward EBM among healthcare professionals.
Supporting information
S1 File. Noor Evidence-Based Medicine Questionnaire.
https://doi.org/10.1371/journal.pone.0249660.s001
(PDF)
References
- 1. Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5(1):1. Epub 2005/01/07. pmid:15634359.
- 2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–2. pmid:8555924.
- 3. Szajewska H. Evidence-based medicine and clinical research: both are needed, neither is perfect. Ann Nutr Metab. 2018;72(suppl 3)(3):13–23. pmid:29631266
- 4. Sheridan DJ, Julian DG. Achievements and limitations of evidence-based medicine. J Am Coll Cardiol. 2016;68(2):204–13. pmid:27386775
- 5. Hong J, Chen J. Clinical physicians’ attitudes towards evidence-based medicine (EBM) and their evidence-based practice (EBP) in Wuhan, China. Int J Environ Res Public Health. 2019;16:3758. pmid:31591286
- 6. Rye M, Torres EM, Friborg O, Skre I, Aarons GA. The evidence-based practice attitude scale-36 (EBPAS-36): a brief and pragmatic measure of attitudes to evidence-based practice validated in US and Norwegian samples. Implement Sci. 2017;12(1):44. pmid:28372587
- 7. Capraş R-D, Bulboacă AE, Bolboacă SD. Evidence-based medicine self-assessment, knowledge, and integration into daily practice: a survey among Romanian physicians and comparison between trainees and specialists. BMC Med Educ. 2020;20(1):19. pmid:31948441
- 8. Hisham R, Liew SM, Ng CJ, Mohd Nor K, Osman IF, Ho GJ, et al. Rural doctors’ views on and experiences with evidence-based medicine: the freedom qualitative study. PLoS One. 2016;11(3):e0152649. pmid:27031700
- 9. Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. British Med J. 2003;326(7384):319–21. Epub 2003/02/08. pmid:12574047.
- 10. Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine. BMC Med Educ. 2014;14:114. Epub 2014/06/10. pmid:24909434.
- 11. Smith AB, Semler L, Rehman EA, Haddad ZG, Ahmadzadeh KL, Crellin SJ, et al. A cross-sectional study of medical student knowledge of evidence-based medicine as measured by the Fresno test of evidence-based medicine. J Emerg Med. 2016;50(5):759–64. pmid:26961178.
- 12. Buljan I, Jerončić A, Malički M, Marušić M, Marušić A. How to choose an evidence-based medicine knowledge test for medical students? Comparison of three knowledge measures. BMC Med Educ. 2018;18(1):290. pmid:30514288
- 13. Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. British Med J. 2002;325(7376):1338–41. Epub 2002/12/07. pmid:12468485.
- 14. Zwolsman SE, Wieringa-de Waard M, Hooft L, van Dijk N. Measuring evidence-based medicine knowledge and skills. The Dutch Berlin Questionnaire: translation and validation. J Clin Epidemiol. 2011;64(8):928–30. Epub 2011/06/07. pmid:21640555.
- 15. Lavizeh M, Kouhpayeh zadeh J. Validity and reliability of the attitudes towards and self-reported ability in evidence-based medicine questionnaire. Int J Med Educ. 2015;14(11):998–1006.
- 16. Zwolsman SE, van Dijk N, te Pas E, Wieringa-de Waard M. Barriers to the use of evidence-based medicine: knowledge and skills, attitude, and external factors. Perspect Med Educ. 2013;2(1):4–13. pmid:23670651
- 17. Hisham R, Ng CJ, Liew SM, Lai PSM, Chia YC, Khoo EM, et al. Development and validation of the Evidence Based Medicine Questionnaire (EBMQ) to assess doctors’ knowledge, practice and barriers regarding the implementation of evidence-based medicine in primary care. BMC Fam Pract. 2018;19(1):98. pmid:29935527.
- 18. McColl A, Smith H, White P, Field J. General practitioners; perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998;316(7128):361. pmid:9487174
- 19. Barzkar F, Baradaran H. Knowledge, attitude, and practice of evidence-based medicine among physicians: a systematic review. BMJ Open. 2017;7(Suppl 1).
- 20. McKinley RK. Rapid response. Validation of the Fresno test of competence in evidence based medicine. https://www.bmj.com/content/326/7384/319/rapid-responses. BMJ. 2003;326:319. pmid:12574047
- 21. McCluskey A, Bishop B. The Adapted Fresno Test of competence in evidence-based practice. J Contin Educ Health Prof. 2009;29(2):119–26. Epub 2009/06/17. pmid:19530195.
- 22. Eubank BH, Mohtadi NG, Lafave MR, Wiley JP, Bois AJ, Boorman RS, et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Med Res Methodol. 2016;16:56. Epub 2016/05/22. pmid:27206853.
- 23.
Bloom BS. Taxonomy education. New York: David McKay; 1956.
- 24. Abdullahi A, Hassan A, Kadarman N, Saleh A, Baraya YuSa, Lua PL. Food safety knowledge, attitude, and practice toward compliance with abattoir laws among the abattoir workers in Malaysia. Int J Gen Med. 2016;6(9):79–87.
- 25. Koo H-C, Poh BK, T RA. Assessment of knowledge, attitude and practice towards whole grains among children aged 10 and 11 years in Kuala Lumpur, Malaysia. Int J Food Sci Nutr Diet. 2015;4(1):171–7.
- 26. Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation. 2009;(12(2)):131–46.
- 27.
Brown TA. Confirmatory factor analysis for applied research. 2nd ed. New York: The Guilford Press; 2015.
- 28.
Stevens JP. Applied multivariate statistics for the social sciences. 5th ed. New York: Rouledge; 2009.
- 29.
Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1967.
- 30.
Guilford JP. Psychometric methods. New York: McGraw-Hill; 1954.
- 31. Albarqouni L, Elessi K. Awareness, attitudes, and knowledge of Palestinian doctors about evidence-based medicine: a cross-sectional survey. Lancet. 2017;390:S14.
- 32. Abeysena C, Jayawardana P, Wickremasinghe R, Wickramasinghe U. Evidence-based medicine knowledge, attitudes, and practices among doctors in Sri Lanka. J Evid Based Med. 2010;3(2):83–7. Epub 2011/02/26. pmid:21349049.
- 33. Baig M, Sayedalamin Z, Almouteri O, Algarni M, Allam H. Perceptions, perceived barriers, and practices of physicians’ towards evidence-based medicine. Pak J Med Sc. 2016;32(1):49–54. Epub 2016/03/30. pmid:27022344.
- 34. Barghouti F, Halaseh L, Said T, Mousa AH, Dabdoub A. Evidence-based medicine among Jordanian family physicians: Awareness, attitude, and knowledge. Can Fam Physician. 2009;55(7):e6–e13. PMC2718582. pmid:19602641
- 35. Csertő M, Berényi K, Decsi T, Lohner S. Self-reported attitudes, knowledge and skills of using evidence-based medicine in daily health care practice: A national survey among students of medicine and health sciences in Hungary. PLoS One. 2019;14(12). pmid:31881036
- 36. Abdel-Kareem A, Kabbash I, Saied S, Al-Deeb A. Knowledge, practices and attitudes of physicians towards evidencebased medicine in Egypt. East Mediterr Health J. 2019;25(2):82–9. pmid:30942471.
- 37. Lafuente-Lafuente C, Leitao C, Kilani I, Kacher Z, Engels C, Canouï-Poitrine F, et al. Knowledge and use of evidence-based medicine in daily practice by health professionals: a cross-sectional survey. BMJ Open. 2019;9(3):e025224. pmid:30928940
- 38. Ghojazadeh M, Azami-Aghdash S, Pournaghi Azar F, Fardid M, Mohseni M. A systematic review on barriers, facilities, knowledge and attitude toward evidence-based medicine in Iran. J Anal Res Clin Med. 2015;3(1): 1–11.
- 39.
McCoach DB, Gable RK, Madura JP. Instrument development in the affective domain. New York, NY: Springer 2013;10:978–1.