https://orcid.org/0000-0003-0031-5873
Figures
Abstract
Introduction and objective
Open science (OS) aims to make the dissemination of knowledge and the research process transparent and accessible to everyone. With the increasing popularity of complementary, alternative, and integrative medicine (CAIM), our goal was to explore what are CAIM researchers’ practices and perceived barriers related to OS.
Methods
We conducted an anonymous online survey of researchers who published in journals listed in Scopus containing the words “complementary”, “alternative”, or “integrative” medicine in their names. We emailed 6040 researchers our purpose-built electronic survey after extracting their email address from one of their publications in our sample of journals. We questioned their familiarity with different OS concepts, along with their experiences and challenges engaging in these practices over the last 12 months.
Results
The survey was completed by 392 researchers (6.5% response rate, 97.1% completion rate). Most respondents were CAIM researchers familiar with the overall concept of OS, indicated by those actively publishing open access (OA) (n = 244, 76.0%), registering a study protocol (n = 148, 48.0%), and using reporting guidelines (n = 181, 59.0%) in the past 12 months. Preprinting, sharing raw data, and sharing study materials were less popular. A lack of funding was reported as the greatest barrier to publishing OA by most respondents (n = 252, 79.0%), and that additional funding is the most significant incentive in applying more OS practices to their research (n = 229,72.2%). With respect to preprinting barriers, 36.3% (n = 110) participants believed there are potential harms in sharing non-peer-reviewed work and 37.0% (n = 112) feared preprinting would reduce the likelihood of their manuscript being accepted by a journal. Respondents were also concerned about intellectual property control regarding sharing data (n = 94, 31.7%) and research study materials (n = 80, 28.7%).
Conclusions
Although many participants were familiar with and practiced aspects of OS, many reported facing barriers relating to lack of funding to enable OS and perceived risks of revealing research ideas and data prior to publication. Future research should monitor the adoption and implementation of OS interventions in CAIM.
Citation: Ng JY, Santoro LJ, Cobey KD, Steel A, Cramer H, Moher D (2024) Complementary, alternative, and integrative medicine researchers’ practices and perceived barriers related to open science: An international, cross-sectional survey. PLoS ONE 19(5): e0301251. https://doi.org/10.1371/journal.pone.0301251
Editor: Vincent Antonio Traag, Leiden University: Universiteit Leiden, NETHERLANDS
Received: November 7, 2023; Accepted: March 13, 2024; Published: May 6, 2024
Copyright: © 2024 Ng et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are included in this manuscript or posted on Open Science Framework: https://doi.org/10.17605/OSF.IO/AX6ZE.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: ASJC, All Science Journal Classification; CAIM, complementary, alternative, and integrative medicine; CHERRIES, Checklist for Reporting Results of Internet E-Surveys; FORRT, Framework for Open and Reproducible Research Training; OS, open science; OSF, Open Science Framework; PMID, PubMed identifier
Introduction
The goal of open science (OS) is to make the dissemination of knowledge and the research process faster, transparent, and open to everyone. Open access publishing, data and code sharing, and the distribution of open research resources are all examples of OS practices. Several jurisdictions in the research and publishing communities have adopted regulations and roadmaps to promote effective implementation and adoption of OS, while there is also growing momentum worldwide for OS practices to become more deeply ingrained in the research ecosystem [1–6]. Up to 85% of research may be wasted according to prior studies6, with evidence indicating that the scientific system is exposed to problems such as publication bias, insufficient reporting standards, and lack of reproducibility [7–13]. By reducing avoidable study duplication, researchers and publishers can save both time and limited resources and funding. Moreover, making various stages of the research lifecycle accessible not only reduces bias, but also fosters creativity and ingenuity as anybody is free to use and build upon study data and resources. By limiting constraints on information access, the OS movement also supports equity. Ideally, this would lead to the removal of obstacles that the public and researchers encounter when it comes to accessing health information in the field of medicine, for example. However, no nation or research field has fully embraced OS despite the value of doing so on a global scale. There are a number of actual and perceived difficulties in deviating from the typical practice of ‘closed research’. A number of problems have been mentioned as potential obstacles, such as how to successfully alter behavior to promote OS initiatives, how to educate academics on the formal procedures associated with OS, and how to balance openness with intellectual property protection [14–16].
Complementary, alternative, and integrative medicine (CAIM) has been a difficult field of medicine to define due to its dynamic nature. The US National Center for Complementary and Integrative Health (NCCIH) has differentiated between complementary and alternative medicine, with the former described as non-mainstream approaches used together with conventional medicine, and the latter described as non-mainstream approaches used instead of conventional medicine [17,18]. The NCCIH further describes integrative health as the convergence of conventional and complementary approaches in a coordinated way [17,18]. The most recent operational definition of CAIM was informed by a systematic search of quality-assessed information resources and includes 604 unique therapies [19]. Despite the increasing popularity of CAIM therapies worldwide, many barriers remain with respect to the conduct and rigor of CAIM research [20]. To the best of our knowledge, no research has yet investigated what are CAIM researchers’ practices and perceived barriers related to OS. To assess the state of OS in their communities, researchers in fields such as social science [21], economics [22], and psychology [23–25] have participated in extensive surveys. These surveys serve as the foundation for developing new interventions that more successfully implement open research, which can also be used to track the progression of OS using a longitudinal design. By gathering and understanding the practices and perceived barriers that CAIM researchers face concerning OS, we can then identify the best ways to increase OS adoption in the CAIM field.
We used a cross-sectional online survey, sent to authors who have been selected based on their status as corresponding authors on publications in journals indexed in Scopus that contain the words “complementary”, “alternative” and/or “integrative” medicine in their journal titles. Our goal was to explore what are CAIM researchers’ practices and perceived barriers related to OS. This is a descriptive study, with no formal hypothesis.
Methods
Ethics approval and consent to participate
We sought and were granted ethics approval by the Ottawa Health Science Network Research Ethics Board prior to beginning this project (OHSN REB Number: 20220642-01H). Implied consent was collected from each participant; upon clicking the survey invitation link, participants were presented with the consent form, and were informed that “By completing the survey your consent to participate is implied.”.
Transparency statement
Approval from the Ottawa Health Sciences Research Ethics Board was received before beginning this project (REB Number: 20220642-01H). Implied consent was collected from each participant; upon clicking the survey invitation link, participants were presented with the consent form, and were informed that “By completing the survey your consent to participate is implied.”. Prior to participant recruitment, a study protocol was registered and made available on the Open Science Framework (OSF) [26]. The study materials, including survey, and de-identified raw data were shared using OSF at the time of the study’s preprint being posted: https://doi.org/10.17605/OSF.IO/AX6ZE. A preprint of the study is also available on medRxiv at: https://doi.org/10.1101/2023.10.24.23297458.
Study design
We conducted an anonymous, online, cross-sectional survey of a sample of authors who have published in CAIM journals indexed in Scopus from January 1, 2018 –December 31, 2022.
Sampling framework
Journals containing the words “complementary”, “alternative” or “integrative” in their names were chosen from Table 2 of Ng [27] which is based on the All Science Journal Classification (ASJC) pertaining to the Scopus category “complementary and alternative medicine” (code 2707) [28]. The full list of journals can be found in Table 1. All manuscripts published in these journals from January 1, 2018 –December 31, 2022, which had a corresponding author and associated PubMed identifier (PMID) were selected. The corresponding author’s name and email address were then extracted from each PMID [29]. Authors who have published manuscripts of any type were included in this study [29]. Please refer to Table 2 for a complete explanation of our method for retrieving author emails adapted from Cobey et al [29]. A sample size calculation was not conducted since this is a convenience sample with descriptive work and the absence of any inferential testing.
Participant recruitment
Only researchers who were identified using our sampling framework (Table 2) were contacted to take part in our study and complete the closed survey. SurveyMonkey was used to send emails to the authors captured in our sample. The prospective participants received an email containing an explanation of the study and its goals on February 12, 2023. This email also contained a link to an informed consent form which participants had to agree to before they could access the online survey. Reminder emails were sent to participants after the first, second, and third weeks after the original invitation email. The survey closed two weeks after the final reminder email on March 19, 2023. There was no financial compensation and no requirement to participate in this study. Any participant who did not wish to respond to a question could skip it.
Survey design
The complete survey adapted from Cobey et al [29] can be found on OSF. It contained 34 items in total, and was displayed across 12 pages (screens). The survey began by asking participants a screening question followed by six general demographic questions (e.g., location, age). They were then asked five questions regarding their role in research and their expertise. Using the Framework for Open and Reproducible Research Training (FORRT) Glossary [30], several definitions of OS and OS practices were presented to participants, of which they were asked to indicate how familiar they were with each concept. The remaining groups of questions asked participants about their experiences engaging in OS practices in the past 12 months and the barriers they encountered. There were 32 multiple-choice questions and 2 open-ended questions. All questions were optional and could be skipped to proceed through the survey. The survey was pilot tested by two independent CAIM researchers to integrate their feedback into the survey prior to distribution.
Data management, analysis, and reporting
The survey data that was gathered was exported to Microsoft Excel. Basic descriptive statistics including counts and percentages were generated based on the analysis of the quantitative data. With respect to qualitative data, a thematic content analysis was conducted by two researchers who analyzed repeated ideas from the open-ended text-based responses. The researchers first collected, categorized, and grouped the responses into themes separately, followed by a discussion to obtain a final consensus of the themes and codes for reporting. The Checklist for Reporting Results of Internet E-Surveys (CHERRIES) was used to report results [31].
Results
Search strategy
In total, 40 journals were eligible according to our search criteria. After searching these journals for publications between January 1, 2018 and December 31, 2022, 16 175 articles with an accompanying PMID were extracted. From those articles, 6040 unique email addresses were obtained as not all articles contained an extractable corresponding author email address. Our survey’s raw, deidentified data is available here: https://osf.io/ytxqd.
Demographics
Overall, 392 researchers participated in our survey out of the 6040 authors emailed (6.5% response rate, 97.1% completion rate); it should be noted that not all participants responded to all questions, hence we provide the total number of responses for each of the questions in parentheses. Responses were deemed incomplete when no questions were answered after the initial screening question. Moreover, we have reported our raw response rate, which is underestimated as we cannot determine how many of the 6040 authors who were emailed currently identify as a CAIM researcher. It should be noted that we also did not calculate email bounce back rate, although we assume this to be around 10%, hence the response rate of those who were invited is in fact slightly higher. The survey took approximately 24 minutes to complete. The majority of respondents (n = 375, 95.7%) answered “yes” when asked if they identify as a researcher of CAIM therapies. Respondents were primarily located in South-East Asia (n = 99, 27.2%), Americas (n = 73, 20.2%), or Europe (n = 72, 19.9%), and identified as a senior career researcher (n = 186, 53.1%) holding positions of faculty member/principal investigator (n = 235, 67.1%). Most respondents also indicated that CAIM was their primary area of research (n = 218, 62.6%), with 47.1% (n = 165) of participants focusing on clinical research. Complete participant demographics are available in Table 3. Furthermore, crosstabs by age, career stage, whether they are a caregiver, gender, whether they belong to a minority group, and WHO region are provided on OSF: https://osf.io/ax6ze/.
Open science practices and experiences
The respondents were “somewhat familiar” (n = 69, 21.7%) with the overall concept of OS, with more than half of respondents cumulatively being “very familiar” (n = 82, 25.8%) or “moderately familiar” (n = 132, 41.5%). Only 2.83% of respondents (n = 9) reported being “not at all familiar” with the concept of OS (Fig 1). After asking participants their familiarity with various OS practices, respondents were “very familiar” with open access publishing (n = 210, 64.2%), preprinting (n = 134, 41.7%), reporting guidelines (n = 141, 44.3%), and protocol registration (n = 122, 38.2%). Respondents also reported being “moderately familiar” with the practices of open data (n = 118, 37.0%) and open materials (n = 98, 31.0%). While almost 80% of respondents were at least “slightly familiar” with the practice of patient and public involvement in research, 10.6% (n = 34) reported being “not at all familiar”, the highest among the seven open science practices investigated (Fig 2).
Open science training, promotion, and implementation
When asked the source of their training or knowledge about OS, of the 317 respondents who answered, more than half reported learning or training on their own accord while conducting research (n = 188, 59.5%). Approximately one in three participants received no formal training with respect to OS (n = 104, 32.9%), followed by 21.8% (n = 69) receiving mentorship from their supervisor/peers, with 18.0% (n = 57) participating in formal coursework or workshops about OS (Fig 3). When probed about their preferred format of OS training, the majority of respondents ranked a website of resources (not including webinars) as their first choice (n = 128, 43.1%). In-person lectures (n = 136, 46.0%) and in-person workshops (n = 131, 44.1%) were the least preferred formats, being ranked by most respondents 6th and 7th respectively (Fig 4). Almost three quarters of respondents indicated that additional funding would be the greatest incentive to allow respondents to apply more OS practices to their research (n = 229, 72.2%). About half of respondents reported that practical support from their institution (e.g., a person to ask questions about the practicalities of performing OS) would also be helpful (n = 161, 50.8%), followed by clearer communication about the benefits and value that OS has to research (n = 145, 45.7%). Approximately one third of participants noted that additional training on applying OS practices would also incentivize their application (n = 105, 33.1%). Roughly a quarter of respondents (n = 83, 26.2%) expressed that being recognized for applying OS practices with respect to being hired, promoted and tenured would be valuable, with another quarter (n = 80, 25.2%) indicating that having additional staff trained on OS practices would allow them to apply more OS practices to their research (Fig 5). Respondents closely ranked funders (n = 100, 37.0%) and research institutions (n = 97, 35.9%) as the most significant stakeholders to create policies resulting in the successful uptake of OS. Scholarly journals and scholarly societies were ranked lower, with 60.9% of respondents (n = 165) ranking the latter as the least significant stakeholder (Fig 6). When 163 participants responded to the open-ended question asking about the best ways to promote open science, 205 codes were generated. The 205 codes were grouped into 27 subthemes, and then into 6 overarching themes. Some prevalent subthemes that emerged were, reducing or eliminating expensive article processing charges (APCs) when submitting manuscripts to open access journals, increasing financial support from funders and institutions and promoting OS: https://osf.io/ex7ky (Table 4). When 82 respondents shared any thoughts or opinions at the end of the survey, 26 codes were generated which were grouped into 16 subthemes, and then into 7 overarching themes. Prevalent subthemes that emerged were that the costs of OS are too high, and that funding is mandatory for OS: https://osf.io/xpgft (Table 5).
Open access
In the past 12 months, 76.0% of respondents (n = 244) indicated they have published an article ‘open access’ (Table 6). When asking respondents to identify the most prominent barriers preventing them from publishing open access are, roughly four out of five respondents (n = 252, 79.0%) reported insufficient financial capital to cover high APCs that are common with open access journals. A further 16.6% of respondents (n = 53) stated they believe their institution does not value the practice of publishing open access. Additionally, 10.3% of respondents (n = 33) reported not facing any of the perceived barriers mentioned in the survey (Fig 7).
Preprinting
With respect to preprinting in the past 12 months, more than half of respondents (n = 194, 61.0%) indicated they have not produced a preprint prior to publishing an article (Table 6). Respondents were asked about the barriers they have faced with respect to creating preprints. Approximately one-third of respondents (n = 112, 37.0%) stated that they worry preprinting their work will reduce their manuscript’s chance of being accepted by a peer reviewed journal, with another third of respondents indicating that they feel there are potential harms in sharing work that has not been peer reviewed (n = 110, 36.0%). Additionally, many respondents stated they do not see the benefit in making a preprint (n = 96, 31.7%), feel that their institution does not value creating preprints (n = 78, 25.7%), or that they do not know how to make a preprint (n = 71, 23.4%) (Fig 8).
Sharing data
When participants were asked if they have shared the raw data for a research study at the time of publication, only 22.9% (n = 72) reported that they did in the past 12 months (Table 6). Respondents were also asked to identify barriers they have encountered with respect to sharing the raw data from their research when publishing. Participants principally selected that they have concerns regarding the unintended use of secondary data (n = 110, 37.0%), concerns about the misinterpretation of the data (n = 95, 32.0%), concerns of intellectual property control (n = 94, 31.7%), and concerns about patient privacy when data is shared (n = 86, 29.0%). Roughly one-quarter of respondents (n = 77, 25.9%) indicated that they do not know how to prepare their data appropriately for sharing (Fig 9).
Sharing study materials
Approximately two-thirds of respondents (n = 209, 67.6%) stated that they have not shared the study materials underpinning a study at the time of publication (Table 6). When asked which barriers they have encountered with regards to sharing study materials when publishing, greater than a quarter of respondents (n = 80, 28.7%) indicated having concerns about intellectual property control. Respondents also indicated that they have concerns about the unintended use of their study materials (n = 73, 26.0%), they do not know how to prepare their study materials for sharing with the research community (n = 71, 25.5%), and they have concerns about the misinterpretation of their study materials (n = 64, 22.9%) (Fig 10).
Registering a study protocol
Respondents were relatively split when asked if they have registered a study protocol in the past 12 months. Roughly half of respondents (n = 148, 48.4%) indicated they have registered a protocol in the past 12 months (Table 6). When asked what barriers they have faced with respect to registering their study protocol prior to starting a research project, roughly one fifth of respondents (n = 47, 20.4%) stated they do not know how to create a study registration. Respondents also indicated that they feared they will be scooped if they share their study plan before publication (n = 45, 19.5%), while roughly another fifth (n = 44, 19.1%) stated that they do not have the time to register their studies (Fig 11).
Using reporting guidelines
When participants were asked if they have used and referenced any reporting guideline checklists in the past 12 months, more than half of respondents (n = 181, 59.2%) said that they have (Table 6). Additionally, participants were asked to identify the barriers they encountered with respect to using reporting guidelines when reporting their research. Of the 212 respondents, 58 (27.4%) stated they do not know where to find relevant reporting guidelines. Respondents also indicated that they do not have the time to use reporting guidelines (n = 37, 17.5%), and that they do not know how to use reporting guidelines (n = 35, 16.5%) (Fig 12).
Discussion
The goal of this study was to explore what are CAIM researchers’ practices and perceived barriers related to OS. Our findings demonstrate that CAIM researchers are familiar with and actively publish open access, register study protocols and use reporting guidelines. However, CAIM researchers identify a lack of funding as a prominent barrier preventing further implementation of OS practices in their work, particularly concerning APCs when publishing open access. Additionally, CAIM researchers have concerns regarding intellectual property control and how their research is used/interpreted when sharing data, study materials, and non-peer reviewed work prior to publication. Overall, respondents from our study as well tend to be most familiar with the practice of open access publishing (n = 210, 64.2%) as this may be the most popular OS practice. It is estimated that roughly a third to almost 50% of published scientific research is available as open access [32]. These results are consistent with previous studies investigating researchers in various disciplines such as social sciences and humanities, engineering, and natural sciences which found that participants were most knowledgeable about open access publishing [33–36].
In addition, the present study found that many researchers were self-taught regarding OS practices while conducting research of their own (n = 188, 59.5%). This coincides with the fact that many participants preferred to learn about OS using a website of resources (n = 128, 43.1%). As this intervention is primarily self-initiated, completed alone, and for low cost, this may explain why our sample of researchers preferred it over in-person lectures or workshops, which they ranked as their least preferred formats. Altogether, our sample of CAIM researchers appear to lack formal training of OS practices. This is troubling as roughly half of our respondents (n = 161, 50.8%) indicated that practical support from their institution, along with clearer communication regarding the value of OS (n = 145, 45.7%) would incentivize them to apply more OS practices to their research. While greater institutional support would likely benefit CAIM researchers, much of CAIM training has been reported to take place in private colleges rather than universities, leading to limited placements and opportunities for further post-graduate education [20]. Therefore, CAIM researchers may not obtain the same research experiences as those in the mainstream biomedical field, leading to poorer research practices such as not incorporating international research reporting standards in their work [20]. Strategies such as educational training programs, increased CAIM-related masters and doctoral programs, and OS training would likely improve research literacy and evidence-based CAIM [37,38].
Approximately three quarters of respondents (n = 229, 72.2%) indicated that additional funding would be the greatest incentive to incorporate more OS practices into their research. This point was reiterated in the analysis of open text responses which identified participant preferences for financially motivated incentives such as additional funding or financial support, and waivers or discounts to APCs. One of the largest financial burdens faced by researchers include APCs and open access charges that are often very high and unaffordable, especially for researchers from developing countries [39]. For example, BMC Complementary Medicine and Therapies charges £1990.00/$2690.00/€2290.00 for each article accepted for publication [40]. This may explain why when asked to identify barriers to publish open access, three quarters of respondents (n = 252, 79.0%) indicated they do not have the funding to afford the APCs that are common at open access journals, despite a similar proportion of respondents (n = 244, 76.0%) having published open access in the past 12 months. Comparative studies such as the survey of Molecular Biology Society of Japan members found that 76.6% (n = 478) respondents have published open access [35,41,42]. Additionally, 91.5% (n = 571) of respondents in the same survey reported wanting to publish open access [35]. Costs, however, remain a barrier to those wishing to publish open access, with some researchers having to use their personal funds towards APCs [35]. Ideally, OS practices such as open access publishing promote equity and reduce barriers that end-users and authors face. Tools such as Sherpa Romeo exist to help users understand individual journal policies regarding preprinting, post-printing, and open access publishing. Finding journals that lack APCs or imply charges should therefore be easier for readers and authors, which is especially important for those in lower-income countries.
A lack of understanding or knowledge regarding the OS practices of preprinting and data and materials sharing appears to be a theme among our sample of CAIM researchers. These were the least popular OS practices with similar barriers being reported between them. For example, 37.0% of participants (n = 112) report they worry preprinting will reduce the chance of their manuscript being accepted at a peer-reviewed journal despite greater than 60% of preprints posted before 2017 eventually being published [43]. With respect to sharing data and study materials, participants were concerned about intellectual property control, unintended use of their data, and data misinterpretation. However, researchers can quell these fears by actively publishing detailed metadata or descriptions of the data collection process/analysis, set permissions for data access and reuse, understandings the legal terms and conditions that exist to protect researcher’s rights, and preprinting to make a first claim [44]. Overall, CAIM researchers would likely benefit from further training and experience regarding the incentives and benefits of various OS practices.
Registering a study protocol and referencing a reporting guideline checklist were more popular OS practices among our cohort, used by 48.4% (n = 148) and 59.2% (n = 181) of participants respectively. Since roughly half of the participants (n = 165, 47,1%) stated that clinical research was their primary focus, this may explain why a similar proportion report preregistering their study protocols, which is a World Health Organization requirement to conduct clinical trials [45]. However, barriers such as not knowing how to create a study registration and not knowing where to find relevant reporting guidelines were indicated by participants, once again suggesting that further support, training, and knowledge of the OS practices may aid their implementation in our participants’ research.
Strengths and limitations
In this study, we leveraged a cross-sectional survey due to its efficient and cost-effective nature, enabling us to capture a snapshot of our target group without requiring long-term follow-up. This approach also enabled us to make broader generalizations across CAIM researchers worldwide, which is another notable strength. To achieve this, we surveyed a sample of CAIM researchers who had varying experiences and perceived barriers related to open science. We also achieved an excellent completion rate among those who responded. However, we acknowledge that our study has limitations such as our low response rate and offering the survey solely in English, likely making it more difficult for non-English speakers to participate and share their thoughts and opinions. As a result, our findings may not be applicable to researchers who engage in open science practices using languages other than English. Furthermore, our cross-sectional survey methodology is inherently limited by recall and non-response biases. We also underestimated the response rate of our survey since we did not account for invalid or non-functioning email addresses (e.g., bounce-backs after sending) or authors with autoreplies indicating vacation or sick leave.
Conclusion
In this study, our goal was to explore what are CAIM researchers’ practices and perceived barriers related to OS. The survey participants shared their experiences, thoughts, and attitudes, which can provide valuable insights for both the open science and CAIM communities. Although participants were familiar with and implemented open science practices such as publishing open access, registering a protocol, and using reporting guidelines, funding barriers and concerns about sharing their work before publication persisted. To the best of our knowledge, this is the first study to document how CAIM researchers view and adopt open science practices in their scholarly work. While previous literature has suggested solutions to increase the adoption of open science in other scientific disciplines, our study provides a solid foundation for implementing current strategies and developing new approaches to enhance the uptake of open science in CAIM. Future work can build upon our baseline study to monitor the implementation of open science interventions in the CAIM field. We hope that our findings and analysis can also benefit other disciplines and contribute to the global adoption of open science, while avoiding the drawbacks of “closed research”.
References
- 1.
https://plus.google.com/+UNESCO. UNESCO Recommendation on Open Science. UNESCO. Published March 2, 2020. Accessed September 28, 2022. https://en.unesco.org/science-sustainable-future/open-science/recommendation.
- 2. Open Science. European Commission—European Commission. Accessed September 28, 2022. https://ec.europa.eu/info/research-and-innovation/strategy/strategy-2020-2024/our-digital-future/open-science_en.
- 3. Government of Canada. Roadmap for Open Science—Science.gc.ca. Accessed September 28, 2022. http://science.gc.ca/eic/site/063.nsf/eng/h_97992.html.
- 4. Open Science—OECD. Accessed September 28, 2022. https://www.oecd.org/sti/inno/open-science.htm.
- 5.
G7 Expert Group on Open Science. Open Scholarship Policy Observatory. Published December 1, 2017. Accessed September 28, 2022. https://ospolicyobservatory.uvic.ca/g7-working-group-open-science/.
- 6. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of evidence. The Lancet. 2009;374(9692):786. pmid:19733780
- 7. Baker M. Is there a reproducibility crisis? Nature. 2016;533:452–454.
- 8. Sena ES, Bart van der Worp H, Bath PMW, Howells DW, Macleod MRPublication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol. 2010;8(3). pmid:20361022
- 9. Goldacre B, Drysdale H, Dale A, et al. COMPare: A prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):1–16. pmid:30760329
- 10. Baker D, Lidster K, Sottomayor A, Amor S. Two years later: journals are not yet enforcing the ARRIVE guidelines on reporting standards for pre-clinical animal studies. PLoS Biol. 2014;12(1):e1001756. pmid:24409096
- 11. Cobo E, Cortes J, Ribera JM, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;343(nov22 2):d6783–d6783. pmid:22108262
- 12. Reproducibilty Project: Cancer Biology. Accessed September 28, 2022. https://elifesciences.org/collections/9b1e83d1/reproducibility-project-cancer-biology.
- 13. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015;349(6251):4716. pmid:26315443
- 14.
Data Sharing in Psychology: A Survey on Barriers and Preconditions—Bobby Lee Houtkoop, Chris Chambers, Malcolm Macleod, Dorothy V. M. Bishop, Thomas E. Nichols, Eric-Jan Wagenmakers, 2018. Accessed September 28, 2022. https://journals.sagepub.com/doi/full/10.1177/2515245917751886.
- 15. Caso R, Ducato R. Intellectual Property, Open Science and Research Biobanks. Social Science Research Network; 2014.
- 16. Moher D, Cobey KD. Ensuring the success of data sharing in Canada. FACETS. 2021;(forthcoming).
- 17.
National Center for Complementary and Integrative Health (NCCIH). Complementary, Alternative, or Integrative Health: What’s In a Name? https://www.nccih.nih.gov/health/complementary‐alternative‐or‐integ rative‐health‐whats‐in‐a‐name. Accessed September 28, 2022.
- 18. Ng JY, Boon HS, Thompson AK, Whitehead CR. Making sense of "alternative", "complementary", "unconventional" and "integrative" medicine: exploring the terms and meanings through a textual analysis. BMC Complement altern med. 2016; 16:134. pmid:27206976
- 19. Ng JY, Dhawan T, Dogadova E, et al. Operational definition of complementary, alternative and integrative medicine derived from a systematic search. Accessed September 28, 2022. https://bmccomplementmedtherapies.biomedcentral.com/articles/10.1186/s12906-022-03556-7. pmid:35413882
- 20. Veziari Y, Leach MJ, Kumar S. Barriers to the conduct and application of research in complementary and alternative medicine: a systematic review. Published online March 23 2017. Accessed September 28, 2022. pmid:28335766
- 21. Christensen G, Wang Z, Levy Paluck E, et al. Open Science Practices are on the Rise: The State of Social Science (3S) Survey. Published online January 14, 2020. Accessed September 28, 2022. https://escholarship.org/uc/item/0hx0207r.
- 22. Scherp G, Siegfried D, Biesenbender K, Breuer C. Results report from an online survey among researchers in econo- mics at German higher education institutions in 2019. ZBW—Leibniz Information Center for Economics. 2020.
- 23. Houtkoop BL, Chambers C, Macleod M, Bishop DVM, Nichols TE, Wagenmakers EJ. Data Sharing in Psychology: A Survey on Barriers and Preconditions. Adv Methods Pract Psychol Sci. 2018;1(1):70–85.
- 24. Borghi JA, Gulick AEV. Data management and sharing: Practices and perceptions of psychology researchers. PLOS ONE. 2021;16(5):e0252047. pmid:34019600
- 25. Stürmer S, Oeberst A, Trötschel R, Decker O. Early-Career Researchers’ Perceptions of the Prevalence of Questionable Research Practices, Potential Causes, and Open Science. Soc Psychol. Published online November 23, 2017. Accessed September 28, 2022. https://econtent.hogrefe.com/doi/abs/10.1027/1864-9335/a000324.
- 26. Open Science Framework. Accessed September 28, 2022. https://osf.io/.
- 27. Ng JY. Insight into the characteristics of research published in traditional, complementary, alternative, and integrative medicine journals: a bibliometric analysis. BMC complement med ther. 2021; 21:185. pmid:34210316
- 28. Scopus Preview. Sources; 2021. Available from: https://www.scopus.com/sources.uri?zone=TopNavBar&origin=searchbasic.
- 29. Cobey KD, Saba S, Ebrahimzadeh S, Moher D. Cardiology researchers’ practices and perceived barriers related to open science: an international cross-sectional survey protocol. OSF; 2022. Available from osf.io/hjr5f.
- 30. Glossary. FORRT—Framework for Open and Reproducible Research Training. Accessed September 28, 2022. https://forrt.org/glossary/.
- 31. Eysenbach G. Improving the quality of web surveys: The checklist for reporting results of internet E-surveys (cherries). Journal of Medical Internet Research. 2004;6(3). pmid:15471760
- 32. Basson I, Simard M-A, Ouangré ZA, Sugimoto CR, Larivière V. The effect of data sources on the measurement of open access: A comparison of dimensions and the web of science. PLOS ONE. 2022;17(3). pmid:35358227
- 33. Pardo Martínez C, Poveda A. Knowledge and perceptions of open science among researchers—a case study for Colombia. Information. 2018Nov21;9(11):292.
- 34. Borrego Á. Article Processing Charges for Open Access Journal Publishing: A Review. Learned Publishing. 2023 Jul 3;36(3):359–78.
- 35. Ide K, Nakayama J. Researchers support preprints and Open Access Publishing, but with reservations: A questionnaire survey of MBSJ members. Genes to Cells. 2023;28(5):333–7. pmid:36876468
- 36. Toribio-Flórez D, Anneser L, deOliveira-Lopes FN, Pallandt M, Tunn I, Windel H. Where do early career researchers stand on open science practices? A survey within the Max Planck Society. Frontiers in Research Metrics and Analytics. 2021;5. pmid:33870051
- 37. Veziari Y, Kumar S, Leach M. Addressing barriers to the conduct and application of research in Complementary and Alternative Medicine: A scoping review. BMC Complementary Medicine and Therapies. 2021;21(1). pmid:34266441
- 38. Steel A. and McEwen B. (2014) ’The need for higher degrees by research for complementary medicine practitioners’, Australian Journal of Herbal Medicine, 26(4).
- 39. Chakravorty N, Sharma CS, Molla KA, Pattanaik JK. Open science: Challenges, possible solutions and the way forward. Proceedings of the Indian National Science Academy. 2022;88(3):456–71.
- 40. Basson I, Simard M-A, Ouangré ZA, Sugimoto CR, Larivière V. The effect of data sources on the measurement of open access: A comparison of dimensions and the web of science. PLOS ONE. 2022;17(3). pmid:35358227
- 41. Halevi G, Walsh S. Faculty attitudes towards article processing charges for Open Access Articles. Publishing Research Quarterly. 2021;37(3):384–98.
- 42. Lucraft M. APCs in the Wild: could monitoring and tracking of OA payments help speed up the transition to OA?. InSeptentrio Conference Series 2020 Sep 22 (No. 4).
- 43. Richard J Abdill, Ran Blekhman(2019) Meta-Research: Tracking the popularity and outcomes of all bioRxiv preprints eLife 8:e45133. https://doi.org/10.7554/eLife.45133.
- 44. Gomes DG, Pottier P, Crystal-Ornelas R, Hudgins EJ, Foroughirad V, Sánchez-Reyes LL, et al. Why don’t we share data and code? perceived barriers and benefits to public archiving practices. Proceedings of the Royal Society B: Biological Sciences. 2022;289(1987). pmid:36416041
- 45.
Joint statement on public disclosure of results from clinical trials [Internet]. World Health Organization; [cited 2023 Aug 27]. Available from: https://www.who.int/news/item/18-05-2017-joint-statement-on-registration.