http://orcid.org/0000-0003-3503-9610
Figures
Abstract
Objective
Place of birth is a known determinant of health care outcomes, interventions and costs. Many studies have examined the maternal and perinatal outcomes when women plan to give birth in hospitals compared with births in birth centres or at home. However, these studies vary substantially in rigour; assessing their quality is challenging. Existing research appraisal tools do not always capture important elements of study design that are critical when comparing outcomes by planned place of birth. To address this deficiency, we aimed to develop a reliable instrument to rate the quality of primary research on maternal and newborn outcomes by place of birth.
Study design
The instrument development process involved five phases: 1) generation of items and a weighted scoring system; 2) content validation via a quantitative survey and a modified Delphi process with an international, multi-disciplinary panel of experts; 3) inter-rater consistency; 4) alignment with established research appraisal tools; and 5) pilot-testing of instrument usability.
Results
A Birth Place Research Quality Index (ResQu Index) was developed comprising 27 scored items that are summed to generate a weighted composite score out of 100 for studies comparing planned place of birth. Scale content validation indices were .89 for clarity, .94 for relevance and .90 for importance. The Index demonstrated substantial inter-rater consistency; pilot-testing confirmed feasibility and user-friendliness.
Conclusion
The ResQu Index is a reliable instrument to evaluate the quality of design, methods and interpretation of reported outcomes from research about place of birth. Higher-scoring studies have greater potential to inform evidence-based selection of birth place by clinicians, policy makers, and women and their families. The Index can also guide the design of future research on place of birth.
Citation: Vedam S, Rossiter C, Homer CSE, Stoll K, Scarf VL (2017) The ResQu Index: A new instrument to appraise the quality of research on birth place. PLoS ONE 12(8): e0182991. https://doi.org/10.1371/journal.pone.0182991
Editor: Kelli K. Ryckman, Univesity of Iowa, UNITED STATES
Received: June 22, 2017; Accepted: July 27, 2017; Published: August 10, 2017
Copyright: © 2017 Vedam et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by the National Health and Medical Research Council (Australia), https://www.nhmrc.gov.au/, Award Number: 1022422, Birthplace in Australia.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The selection of birth place has important implications for the health and wellbeing of women and newborns, as well as for health services planning and healthcare costs [1–3]. Decisions about place of birth are largely based on the preferences of women and their families, but are also influenced by the availability of services, equipment and providers in their communities, and their understanding of the available scientific evidence [4, 5]. Women’s preferences about place of birth are most often based on provider or peer recommendations, or information gained via the Internet [6–11]. However, these sources do not always provide unbiased information about benefits and risks associated with different birth settings.
Evidence on the relative safety of different places of birth is complex and frequently controversial [12]. Several studies [13–19] have found reduced obstetric interventions and optimal outcomes among healthy women who planned to give birth at home or a birth centre under the care of midwives. In some high-income countries, where maternity care is integrated across birth settings, researchers have concluded that there are no significant differences between birth places in morbidity or mortality for newborns [15, 16, 19] and/or that the absolute risks of mortality are extremely low [13, 14]. Other investigators have reported a significant increase in adverse perinatal outcomes related to planned home births, especially where skilled birth attendants are not universally integrated into regional health systems, or in population-based studies that include at-risk pregnancies [20–22].
Policy makers and clinician leaders have responded to the research with similar discordance. Some clinical guidelines and policy statements in high-income countries support access to midwife-led care in birth centres and home births as cost-effective options for women with uncomplicated pregnancies [23–26]. Other professional organisations have issued statements questioning the evidence basis for support of women’s choice of birth place, and stating that hospital birth is the only setting that assures safe outcomes [27, 28]. In low- and middle-income countries, these latter views have led to policy initiatives and incentives towards universal institutional birth [29–31].
Inconsistency in the design and quality of research on place of birth underpins the difficulty of crafting universally acceptable recommendations for service provision across birth settings. Further, the interpretation and dissemination of research findings can be subject to publication and critical bias, and the influence of professional viewpoints and culture [12, 32].
Accurate information about relative and absolute risk is vital to inform women and their families, clinicians and health administrators as they work together to select optimal birth places within the context of a person-centred care plan and regional resources. However, without a reliable means to assess the quality of the evidence, it is not possible to interpret the best available data and its implications for the safety of women and infants. This study was designed to address the need for a standardised system for evaluating research on the safety of birth place. It aimed to develop and pilot test a novel instrument for appraising the quality and rigour of research that examines the impact of place of birth on maternal and perinatal outcomes.
Why develop an instrument specific to birth place?
While there are several valuable instruments for assessing the quality of randomised and non-randomised clinical studies [33–38], these are not always appropriate to studies of place of birth. Some rating tools are only applicable to randomised studies; others focus on the extent to which non-randomised studies emulate randomised trials (e.g. by reducing selection bias) [34, 38–41]. However, birth place represents an intervention that is difficult to evaluate through randomised trials. Women are not inclined to relinquish their choice of birth place and participate in trials that present major practical and ethical limitations [42]. Another limitation of conventional rating instruments in this context is that, unlike many medical conditions where the impact of interventions on recovery or amelioration can be studied, pregnancy, labour and birth are not diseases or injuries (although they may be accompanied by morbidity). A woman may have multiple pregnancies, but each birth is a unique and finite event. Therefore instruments focused on longer-term recovery may not be relevant to studies on labour and birth, e.g. items on proportion of participants followed-up over time to ascertain whether symptoms have recurred.
When comparing outcomes across birth settings, the use of consistent definitions and inclusion criteria across cohorts, and reliable outcome measures, is imperative. A significant error in some published research on birth place is amalgamating data from unplanned home births (without skilled birth attendants) with data from planned births at home or in birth centres within integrated systems.
Aims
Following revelations of critical flaws in some published studies [43–46], researchers developed key principles for appraising the quality of research on place of birth [45, 47–49]. Our team designed the Birth Place Research Quality Index (ResQu Index) as a quantitative scoring system based on these key principles, as well as best practices for critical appraisal of research [39, 50]. The objective of this paper is to describe the development and testing of the ResQu Index which aims to facilitate consistent assessment of the quality of research on place of birth.
Methods
Development of the ResQu Index involved five distinct phases: 1) generating items and a weighted scoring system; 2) conducting expert content validation via a quantitative survey and a modified Delphi process; 3) testing inter-rater consistency; 4) assuring compatibility with established research quality checklists and 5) piloting the ResQu Index in a large systematic review to assess instrument usability and feasibility.
We developed this instrument to conduct an extensive systematic review of the literature on outcomes related to place of birth, within a larger research project [blinded]. A protocol was lodged with PROSPERO (CRD42016042291). Ethics clearance was not required for the instrument development.
Phase 1: Item generation and scoring
Item generation was informed by a literature review and the principles proposed by Vedam [45], Hutton [51] and Nove [47]. Each item was selected to be consistent with other systems for appraisal of research quality [34, 35, 37, 38, 52] but adapted, when necessary, to focus on the unique aspects of comparing outcomes across birth settings.
Some items in the Index relate to rigour common to all research studies, addressing issues such as clarity of key terms, definitions of the ‘intervention’ (i.e. place of birth), integrity of data, appropriateness of sample size and selection, transparency of methods and comparability of cohorts. Other items relate specifically to studies of birth place as outlined by Vedam [49] and Nove [47], and to address issues identified as problematic in critiques of previous research in this field [43, 44, 46]. These items include identifying the timing of birth place decisions in relation to an intention-to-treat model and ensuring that studies of home birth clearly distinguish data from planned home births with skilled attendants, from data generated by unplanned home births or “free births” without professional support. The individual items need to be relatively broad to be applicable to studies of different birth settings available across various regions. However, they also need to be sufficiently focused to identify meaningful differences in the rigour of research methodology.
Each item incorporates a rubric that scores the level of quality of the study design and interpretation of findings. Item scores are summed to generate a total composite Index score for a study. Scoring options vary from a simple yes/no response to a range of potential criteria. The expert content validation process (Phase 2) evaluated both the wording of the scoring rubrics and their relative numerical value (weighting).
The first draft version of the ResQu Index contained 25 items, relating to five domains addressing: quality of design, sample definition, measurement of outcomes, comparability of cohorts, and accuracy of interpretation and reporting (domains indicated in Fig 1). The Index scores total to 100, with higher scores indicating higher quality. The wording and scoring scale of items in the first draft version are included under Results.
Phase 2: Expert content validation
Participants.
An international multidisciplinary expert panel was invited to examine the draft ResQu Index for face and content validity via a quantitative survey and a modified Delphi process involving multiple rounds of consultation. The panel constituted both content experts and potential users of the Index, both of whom should be included in a content validation [53, 54]. In line with best practice for expert panel review, the invited panel of 42 experts spanned a range of perspectives in terms of profession, expertise, country of residence/practice, and acknowledged attitudes about place of birth [53, 54]. In total, 21 experts completed the validation process.
The panel included 15 academics, 12 researchers, 8 midwives, 6 perinatal epidemiologists, 4 statisticians, 2 policy makers and one each of consumer, nurse and obstetrician (more than one response was possible). Their areas of expertise included: research on birth place (16 experts), planned home birth (13), research methodology/study design (11), midwifery practice (11), evidence-based practice (10), health services (10), appraisal of evidence (10), public health (9), hospital maternity care (7), birth centres (7), health systems (5), statistics (5), population-based services (4), patient-oriented outcomes (4), global health (3). Two experts indicated professional interest in ‘transfer’ and one each ‘law or policy’, ‘nursing practice’ and ‘medical practice’. Anonymous response was possible, but twelve of the 21 experts supplied their names. All supplied the country/region in which they were based, including North America (11), Australia (7), United Kingdom (2), and the Netherlands (1). Several indicated that they also worked in a range of other countries or regions.
Procedure.
Panel members were contacted using publicly-available contact details, and participation was voluntary. We emailed the panel of 42 experts explaining the background and purpose of the Index, and inviting them to participate in the expert review process. We outlined the conceptual basis and scope of the Index [53] and explained how the process differed from content validation for instruments assessing skills or attitudes.
The content validation took place during June 2016, online via Survey Gizmo. Experts considered each item on the first draft version for clarity, importance and relevance, rating each aspect on a four-point ordinal scale from ‘very clear/important/relevant’ (1) to ‘not clear/important/relevant’ (4). In addition to the quantitative rating process, they were invited to provide written feedback on the wording and scoring rubric for each item, to suggest non-essential items that might be omitted, and to comment on individual items and/or the overall purpose, utility and scope of the draft Index [55].
Following expert review, several items were amended in line with the experts’ quantitative responses and qualitative feedback (S1 and S2 Tables give examples of this process). Two team members used the updated version to assess 41 studies, and made small adjustments in wording to arrive at a second draft version. The second draft version was re-circulated for review and approval to the expert panel.
To assess utility and user-friendliness, we invited the same experts to beta-test the second draft ResQu Index, by supplying them with two published articles on birth place to assess. The articles were selected to reflect different settings, methodologies, and eras. They were from two different countries, one a relatively recent study comparing outcomes from freestanding midwifery units with hospital obstetric units and the other a pre-2000 study comparing home births and hospital births. Four completed this beta-testing process, including two who did not participate in the first stage of content validation. Responses and recommendations were incorporated into the final draft before pilot testing.
Data analysis.
Responses from the online survey were imported into MS Excel for simple descriptive analysis. An item content validation index (I-CVI) was calculated for the three aspects (clarity, importance and relevance) of each item, being the proportion of experts that gave it a positive response (1 or 2). Items that achieved a total score above .80 for relevance were retained (except for one which scored low in importance). Those items below .80 for clarity were rephrased or modified to reflect the expert comments (S1 and S2 Tables) or discarded. The overall scale content validity index (S-CVI) was calculated for each aspect, being the average (mean) of the I-CVIs of the retained items. Polit and Beck recommend that a level of .90 for the S-CVI be used as the standard for excellent content validity using this approach [56].
Phase 3: Inter-rater consistency and consensus
To test inter-rater reliability, two members of the research team (VS and CR) used the second draft version of the ResQu Index (revised but still 25 items) to independently rate 20 studies. Stemler [57] describes different purposes and methods for measuring inter-rater reliability. We used Spearman’s rho coefficient to explore the consistency of our total scores for the articles because the scores were continuous but not necessarily normally distributed [57]. Secondly, we converted each study’s score to a simple scale of research evidence: strong (scores of 75% and above), moderate (65–74%) and weak (less than 65%). We then examined inter-rater consensus on the three-tier scale using Cohen’s kappa statistic [58]. We used SPSS version 23 to analyse inter-rater consistency and consensus.
In addition to investigating inter-rater consistency and consensus, this process enabled us to review each study’s total Index scores, with particular attention to where the two raters diverged. In conjunction with further comments from expert reviewers, this contributed to the final version of the Index (Fig 1).
Phase 4: Research quality checklists
To ascertain comprehensiveness, the ResQu Index was compared to the taxonomy of domains developed by Deeks and colleagues [39] in their extensive evaluation of scales and checklists for assessing quality in non-randomised studies. We matched ResQu items to domains identified by these authors as critical to study quality, internal and external validity, and standard of reporting. We also compared the Index to the Cochrane tool for assessing Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) [38], which is widely used to assess research quality in systematic reviews [59].
Phase 5: Pilot testing
As a final stage of development, two authors (VS and CR) assessed usability and feasibility by using the ResQu Index in a systematic review on maternal and perinatal outcomes related to place of birth for women at low risk of obstetric complications in high-income countries. We used the updated ResQu Index to rate 68 articles, comparing and discussing their ratings when total scores diverged. This process was done iteratively, in conjunction with the search and screening stages of the review. Having identified studies that met the eligibility criteria for the systematic review, one or both authors applied the current version of the Index.
Results
Item generation and scoring
The final ResQu Index contains 27 items across five domains, each rated on a numerical scale where higher values indicated higher quality with respect to the aspect of quality being measured. The items and the final scoring rubrics are shown in Fig 1. The total possible composite score is 100, to guide categorisation of studies into strong, moderate, and weak quality.
Expert content validation
The experts’ quantitative responses about the clarity, importance and relevance of the items in the first draft version are summarised in Table 1. I-CVIs are the proportion of respondents who rated each item positively (1 or 2).
Table 2 summarises the experts’ quantitative responses about the scoring of items, showing the proportion of those who gave the proposed rubric for each item either 1 or 2.
In addition to responding to the online survey, several experts corresponded directly with the project team, providing comments on the draft Index or the validation process, or seeking clarification. This correspondence, together with the quantitative and qualitative survey responses contributed to further revision of the Index. Accordingly, the wording and/or the relative weighting of scoring rubrics was refined, specifically to enhance clarity: three items were removed as feedback indicated that they were not perceived as relevant as others (items 15, 17 and 22 in Table 1); two new items were added (on sample size calculation and ethical research); one item was split into two (item 24); and two were collapsed (items 7 and 8) into one. Another (item 19) was split into two, to address separately the reliable use of statistical methods and the clarity of presentation of comparisons between birth places, and to increase the relative weighting of items on statistical rigour. We also amended the wording or scoring language for some items (S1 and S2 Tables).
The S-CVIs were calculated following the removal of the three items (items 15, 17 and 22), as described above. The average (mean) of the I-CVIs for the retained items was 0.89 for clarity, 0.94 for relevance and 0.90 for importance (Table 1). These levels are close to or above the recommended level for excellent content validity of .90 [56].
In relation to the proposed scoring rubric for the retained items, the average S-CVI was 0.85. Of the 22 retained items, seven received a positive rating of 80% or less. Of these, five were substantially reworded and or reweighted. The other two (items 7 and 8) were merged into one with a revised scoring rubric (Table 2).
The four experts who used the Index to beta-test its applicability in scoring the two sample articles reported positive experiences in terms of usability and acceptability. Following their comments, the wording and scoring in the final version were fine-tuned.
Inter-rater reliability
Table 3 indicates results of inter-rater reliability testing for 20 studies. Four articles received identical scores. Another 11 were scored within five percentage points. There was a very strong positive correlation between the scores from the two raters (Spearman’s rho = .868, p<0.001), indicating a high degree of consistency in ratings. Comparing the two raters’ scores on the three-tier scale (strong, moderate, weak) also demonstrated substantial levels of consensus on the relative strength of the articles (kappa = .697, p<0.001) [57, 58].
Research quality checklist
The comparison of the ResQu Index with the domains identified by Deeks and colleagues [39] demonstrated considerable congruence between the Index and most of the Deeks domains (Table 4). The ResQu Index addresses some items in the Background/Context domain. However, the Blinding and Follow-up domains were not included because items related to randomisation, blinding or follow-up are not relevant to research into place of birth. Following this process of comparison, two items were added to the final version (items 1 and 9 in Fig 1).
Three ResQu items (items 10, 13 and 14 in Fig 1) are not compatible with any of the domains identified, as they constitute birth setting-specific items i.e. timing of decision about birth place and instances where women transfer from home or birth centre to give birth in hospital. Item 6 is also unique to birth place research, and was included in the Index because it identifies a potential source of bias.
We also compared the Index to the domains identified in the Cochrane ROBINS-I tool [38] devised to assess risk of bias (Table 5).
Pilot testing
The pilot testing phase was conducted concurrently with a systematic review of literature on outcomes by place of birth, where the ResQu Index scores were used to assess the research quality of studies that met the review’s inclusion criteria. This process confirmed its utility, feasibility and applicability in a systematic review process. We were able to generate scores for each reviewed study, with congruence between reviewers, and excluded studies scoring less than 75% in the sensitivity analyses.
Team members also recorded how long it took them to read and assess a sample of 24 articles using the ResQu Index. This ranged from 15 to 45 minutes per article depending on article length, clarity and detail, with a mean of 31.25 minutes per article. The scoring process became quicker over time, as raters became more familiar with the items and scoring rubrics.
The pilot testing phase highlighted a few limitations in the Index and suggested potential new items or, more commonly, new levels or examples within the scoring rubric. We discussed discrepancies in scores until these could be resolved by team consensus; we also used this experience to refine the Index by clarifying the wording of items and scoring rubrics. The final Birth Place ResQu Index appears in Fig 1.
Discussion
We undertook a formal, rigorous process to develop and validate a Birth Place Research Quality Index that provides a reliable means of assessing the quality of research on place of birth. The ResQu Index provides a consistent, transparent, and pragmatic solution to the dilemma presented by the ongoing scientific debate on the significance of results from studies on birth place [12]. The inclusion of 21 experts from various disciplines in the development and content validation process has added to the scope, acceptability, and utility of the Index [55]. Our collaborative approach involved ongoing correspondence and exchange with several experts, to ensure applicability for researchers and clinicians across the health professions. The S-CVI scores, close to or above .90, indicate excellent content validity as assessed by this large panel of international, multi-disciplinary experts.
The resulting ResQu Index is a more comprehensive, nuanced, and workable tool to assess the relative significance of studies than checklists which simply identify whether research adequately incorporates certain essential elements [40]. While the ResQu Index includes items fundamental to research integrity, such as clarity of objectives, appropriate sampling, treatment of confounders and use of statistical analysis [60], it also requires users to consider issues specific to childbirth and potentially attributable to place of birth. Moreover, the formal content validation process reflects expert consensus on wording and parameters of items to be included, as well as scoring and weighting of each aspect.
The compatibility of the ResQu items with most research quality domains [39] (Table 4) and with the ROBINS-I framework [38] (Table 5) indicates the comprehensiveness of the Index and its capacity to evaluate research rigour according to commonly-accepted standards. It excludes items not pertinent to birth place research, such as randomisation and blinding. Conversely, it does address critical items to assess quality of studies on birth place, such as maternal transfer, and ensuring that data on planned place of birth are not tainted by unintended home births or those without a qualified birth attendant [43–46].
The ResQu Index identifies birth place as the ‘intervention’; however, childbirth is a unique phenomenon within healthcare because birth place cannot be considered in terms of ‘dose’ or ‘exposure’. Issues of withdrawal or drop-out from the intervention have very distinct implications within a study on birth setting and must be treated differently from withdrawal in studies of other interventions. The process of developing the items required attention to the distinction between the quality of a study’s methods and the quality of reporting results [61]. This generated discussion within the project team during the development and pilot stages about the extent to which we were rating the research or the article.
The final stage of developing the ResQu Index was to use it in a systematic review to assess the quality of studies (published between 2000 and 2016) on the maternal and perinatal outcomes of different places of birth, for women with healthy, low-risk pregnancies in high-income countries. Two authors separately and jointly used the ResQu Index system to inform a rigorous process of inclusion and exclusion of identified studies. They also developed a user dictionary (available on request) to assist with using the Index to assess studies in practice. While several low-scoring studies were excluded by the parameters of our systematic review (e.g. because they did not follow an intention to treat approach, or they included births to women at different levels of risk), we were able to utilise the Index to examine the quality of studies to inform inclusion in our meta-analysis of data.
Place of birth research has been plagued with controversy over study design, variables to be measured, cohort definition and other factors specific to this context. A growing volume of academic literature and media attention has focused on research about birth place. The availability of a reliable method to assess quality of the evidence has the potential to create consensus about the quality of studies among clinicians and researchers who have been (often fiercely) divided on the issue [12, 62, 63]. Women and families are ill-served by inter-professional conflict, given that their choices are influenced by provider attitudes [11, 32, 64, 65]. In the age of increased consumer access to scientific findings, it is imperative that public health information and recommendations are based on the best available data. Studies that score higher on the Index thus have greater potential to reliably inform the evidence base for decisions about birth place by women and health professionals.
The ResQu Index could facilitate the development of clinical guidelines on provision of maternity services across places of birth. For example, following a review of the evidence, some professional clinical bulletins [28] have relied on conclusions from publications that are widely cited [22] but have been subject to critical reviews questioning their quality [44, 46]. Other investigators have published ethical practice recommendations based on findings from research that would not meet the ResQu Index scoring threshold to support a recommendation [62]. Consumers and clinicians will benefit from having a consistent, transparent and reliable method to distinguish and rate the relative importance of research conclusions and recommendations related to place of birth. In addition to guiding the design of studies, the ResQu Index could provide a framework for peer review for publication or serve as a quality assurance guide for research grant funding agencies. The Index could also be used to appraise research on birth place across population cohorts, taking adequate account of the characteristics of women in different settings, and applying the same principles of comparability between cohorts and transparency of research methods. Finally, the Index may help to contextualise results of research in jurisdictions where care is not integrated across settings.
Strengths and limitations
The development of a quality assessment instrument is a complex process, benefiting from the contributions of a multidisciplinary team. Despite careful measures to maximise content validity and consistency, it inevitably involves subjective judgements, both in the selection and weighting of items, and their application to appraise published studies of place of birth. Other researchers may debate its relative emphasis on some aspects of this literature at the expense of others.
The results of pilot-testing confirm that the Index is, like other such scales, subject to some personal interpretation. However, comparison of the inter-rater consistency and consensus of scores from two raters, including one non-clinician researcher, with a large selection of studies showed considerable similarity. One of the strengths of the Index is its capacity to facilitate multi-disciplinary, collaborative evaluation of research outputs, by using commonly-defined terms.
The ResQu Index was not designed to assess qualitative studies and would not, by virtue of its focus on specific aspects of methodology, rate them highly. We acknowledge that some important questions about quality and safety of birth place, or person-centred care, cannot be answered by quantitative methods alone.
Conclusions
The Birth Place Research Quality Index is a reliable, pragmatic tool to systematically appraise the quality of research on place of birth. The composite scoring system highlights the unique nature of childbirth and the specific characteristics of study design and analysis that must be considered when evaluating reported findings. We used a detailed, formal process to design and content validate the Index in consultation with an international panel of multi-disciplinary experts in this field. Pilot testing demonstrated usability, feasibility, and inter-rater consistency. The Index was well suited to systematic review of research on birth place. It could also be used to inform clinical guidelines on maternity service provision across birth settings, to inform the design of studies, or to provide a framework for peer review for publication.
Supporting information
S1 Table. Examples of how item wording changed following expert review.
https://doi.org/10.1371/journal.pone.0182991.s001
(DOCX)
S2 Table. Examples of how rating of items changed following expert review.
https://doi.org/10.1371/journal.pone.0182991.s002
(DOCX)
Acknowledgments
We sincerely thank the panel of international experts who contributed thoughtfully and enthusiastically to the content validation process, and our colleagues on the [blinded] research study.
References
- 1. Schroeder E, Petrou S, Patel N, Hollowell J. Puddicombe D. Redshaw M. et al. Cost effectiveness of alternative planned places of birth in woman at low risk of complications: evidence from the Birthplace in England national prospective cohort study. BMJ. 2012;344(7854):18.
- 2. Scarf V, Catling C, Viney R, Homer C. Costing Alternative Birth Settings for Women at Low Risk of Complications: A Systematic Review. PloS one. 2016;11(2):e0149463. pmid:26891444
- 3. Janssen PA, Mitton C, Aghajanian J. Costs of Planned Home vs. Hospital Birth in British Columbia Attended by Registered Midwives and Physicians. PLOS ONE. 2015. pmid:26186720
- 4. Gee RE, Corry MP. Patient engagement and shared decision making in maternity care. Obstetrics & Gynecology. 2012;120(5):995–7.
- 5. Nieuwenhuijze M, Low LK. Facilitating women's choice in maternity care. Journal of Clinical Ethics. 2013;24(3):276–82 7p. pmid:24282854. Language: English. Entry Date: 20140131. Revision Date: 20150712. Publication Type: Journal Article. Journal Subset: Biomedical.
- 6. Lagan BM, Sinclair M, George Kernohan W. Internet use in pregnancy informs women’s decision making: a web‐based survey. Birth. 2010;37(2):106–15. pmid:20557533
- 7. Vedam S, Stoll K, Martin K, Jolicoeur G. Women taking charge of changing childbirth in British Columbia: abstracts from research forums presented at the American College of Nurse-Midwives 60th annual meeting. Journal of Midwifery & Women’s Health. 2015;60(5):637.
- 8.
Public Health Agency of Canada. What mothers say: the Canadian maternity experiences survey. Data tables. Ottawa2009.
- 9.
Declercq E, Sakala C, Corry MP, Applebaum S, Herrlich A. Listening to mothers III: pregnancy and birth. New York: Childbirth Connection; 2103. https://doi.org/10.1111/birt.12135
- 10. Fairbrother N, Stoll K, Schummers L, Carty E. Obstetrician, family physician, or midwife: Preferences of the next generation of maternity care consumers. Canadian Journal of Midwifery Research and Practice-Revue Canadienne de la Recherche et de la Pratique Sage-femme. 2013;11(2):8–15.
- 11. Coxon K, Chisholm A, Malouf R, Rowe R, Hollowell J. What influences birth place preferences, choices and decision-making amongst healthy women with straightforward pregnancies in the UK? A qualitative evidence synthesis using a ‘best fit’ framework approach. BMC Pregnancy and Childbirth. 2017;17(1):103. pmid:28359258
- 12. de Vries RG, Paruchuri Y, Lorenz K, Vedam S. Moral science: ethical argument and the production of knowledge about place of birth. Journal of Clinical Ethics. 2013;24(3):225–38 14p. pmid:24282850. Language: English. Entry Date: 20140131. Revision Date: 20150712. Publication Type: Journal Article. Journal Subset: Biomedical.
- 13. Birthplace in England Collaborative Group, Brocklehurst P, Hardy P, Hollowell J, Linsell L, Macfarlane A, et al. Perinatal and maternal outcomes by planned place of birth for healthy women with low risk pregnancies: the Birthplace in England national prospective cohort study. BMJ. 2011;343:d7400. PubMed pmid:22117057; PubMed Central PMCID: PMCPMC3223531.
- 14. Olsen O, Clausen JA. Planned hospital birth versus planned home birth. Cochrane Database of Systematic Reviews. 2012;(9). pmid:105836416. Language: English. Entry Date: 20101029. Revision Date: 20150711. Publication Type: Journal Article.
- 15. de Jonge A, Geerts CC, van der Goes BY, Mol BW, Buitendijk SE, Nijhuis JG. Perinatal mortality and morbidity up to 28 days after birth among 743 070 low-risk planned home and hospital births: a cohort study based on three merged national perinatal databases. BJOG: An International Journal of Obstetrics and Gynaecology. 2015;122(5):720–8.
- 16. Hutton EK, Cappelletti A, Reitsma AH, Simioni J, Horne J, McGregor C, et al. Outcomes associated with planned place of birth among women with low-risk pregnancies. CMAJ: Canadian Medical Association Journal. 2016;188(5):E80–E90 11p. pmid:26696622. Language: English. Entry Date: In Process. Revision Date: 20160323. Publication Type: Article. Journal Subset: Biomedical.
- 17. Homer CS, Thornton C, Scarf VL, Ellwood DA, Oats JJ, Foureur MJ, et al. Birthplace in New South Wales, Australia: an analysis of perinatal outcomes using routinely collected data. BMC Pregnancy & Childbirth. 2014;14:206. PubMed pmid:24929250; PubMed Central PMCID: PMCPMC4067683.
- 18. Stapleton SR, Osborne C, Illuzzi J. Outcomes of care in birth centers: Demonstration of a durable model. Journal of Midwifery and Women's Health. 2013;58:3–14. pmid:23363029
- 19. Janssen PA, Saxell L, Page LA, Klein MC, Liston RM, Lee SK. Outcomes of planned home birth with registered midwife versus planned hospital birth with midwife or physician. Canadian Medical Association Journal (CMAJ). 2009;181(6–7):377–83.
- 20. Kennare RM, Keirse MJ, Tucker GR, Chan AC. Planned home and hospital births in South Australia, 1991–2006: differences in outcomes. Medical Journal of Australia. 2010;192(2):76–80. pmid:20078406.
- 21. Snowden JM, Tilden EL, Snyder J, Quigley B, Caughey AB, Cheng YW. Planned Out-of-Hospital Birth and Birth Outcomes. The New England Journal of Medicine. 2015.
- 22. Wax JR, Lucas FL, Lamont M, et al. Maternal and newborn outcomes in planned home birth vs planned hospital births: a metaanalysis. American Journal of Obstetrics and Gynecology. 2010;203(3):243–5. pmid:20598284
- 23.
American College of Nurse Midwives. Position Statement: Planned home birth. Silver Spring, Maryland: ACNM; 2011.
- 24.
Royal College of Obstetricians and Gynaecologists, Royal College of Midwives. Joint statement: home births. London2007.
- 25.
National Institute for Health and Care Excellence. Intrapartum care for healthy women and babies: clinical guideline CG190. London: NICE; 2014.
- 26.
College of Physicians and Surgeons of British Columbia. Professional Standards and Guidelines: Planned home birth. Vancouver2009.
- 27.
The Royal Australian and New Zealand College of Obstetricians and Gynaecologists. Home births statement. RANZCOG; 2014.
- 28.
American Congress of Obstetricians and Gynecologists. Committee opinion: planned home births. 2016.
- 29. Alkema L, Chou D, Hogan D, Sanqian Z, Moller A-B, Gemmill A, et al. Global, regional, and national levels and trends in maternal mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN Maternal Mortality Estimation Inter-Agency Group. Lancet. 2016;387 North American Edition(10017):462–74 13p. pmid:26584737. Corporate Author: United Nations Maternal Mortality Estimation Inter-Agency Group collaborators. Language: English. Entry Date: 20160220. Revision Date: 20160221. Publication Type: journal article. Journal Subset: Biomedical.
- 30. Koblinsky M, Matthews Z, Hussein J, Mavalankar D, Mridha MK, Anwar I, et al. Going to scale with professional skilled care. Lancet. 2006;368(9544):1377–86. PubMed pmid:17046470.
- 31. Prata N, Passano P, Rowen T, Bell S, Walsh J, Potts M. Where there are (few) skilled birth attendants. Journal of Health, Population and Nutrition. 2011;1:81–91.
- 32. Vedam S, Stoll K, Schummers L, Fairbrother N, Klein MC,Thordarson D, et al. The Canadian birth place study: examining maternity care provider attitudes and interprofessional conflict around planned home birth. BMC Pregnancy and Childbirth. 2014.
- 33. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008.
- 34. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of clinical epidemiology. 2011;64(4):401–6. pmid:21208779
- 35. Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, et al. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. American Family Physician. 2004;69(3):548–56. pmid:14971837.
- 36. Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. Journal of clinical epidemiology. 2013;66(7):719–25. pmid:23312392
- 37.
Howick J, Chalmers I, Glasziou P, Greenhalgh T, Heneghan C, Liberati A, et al. The Oxford 2011 levels of evidence. Oxford: Oxford Centre for Evidence-Based Medicine; 2011.
- 38.
Sterne J, Higgins J, Reeves B. on behalf of the development group for ACROBAT-NRSI. A Cochrane Risk Of Bias Assessment Tool: for non-randomized studies of interventions (ACROBAT-NRSI), Version 1.0. 0, 24 September 2014. sites google com/site/riskofbiastool/(accessed Apr 2015). 2015.
- 39. Deeks J, Dinnes J, D'amico R, Sowden A, Sakarovitch C, Song F, et al. Evaluating non-randomised intervention studies. Health technology assessment (Winchester, England). 2003;7(27):iii–x, 1–173.
- 40. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of epidemiology and community health. 1998;52(6):377–84. pmid:9764259
- 41. Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non‐randomized studies (MINORS): development and validation of a new instrument. ANZ journal of surgery. 2003;73(9):712–6. pmid:12956787
- 42. Hendrix M, van Horck M, Moreta D, Nieman F, Nieuwenhuijze M, Severens J, et al. Why women do not accept randomisation for place of birth: feasibility of a RCT in The Netherlands. BJOG: An International Journal of Obstetrics and Gynaecology. 2009;116(4):537–44.
- 43. Gyte G, Dodwell M, Newburn M, Sandall J, Macfarlane A, Bewley S. Estimating intrapartum-related perinatal mortality rates for booked home births: when the 'best' available data are not good enough. BJOG: An International Journal of Obstetrics and Gynaecology. 2009;116(7):933–42.
- 44. Sandall J, Bewley S, Newburn M. 'Home birth triples the neonatal death rate': public communication of bad science?. . .Am J Obstet Gynecol. 2010 Sep;203(3):243.e1–8. American Journal of Obstetrics & Gynecology. 2011;204(4):e17-8 1p. pmid:104860271. Language: English. Entry Date: 20110708. Revision Date: 20150711. Publication Type: Journal Article.
- 45. Vedam S. Home birth versus hospital birth: Questioning the quality of the evidence on safety. Birth. 2003;30(1):57–63. pmid:12581041
- 46. Michal CA, Janssen PA, Vedam S, Hutton EK, De Jonge A. Planned home vs hospital birth: a meta-analysis gone wrong. Medscape OB/GYN & Women’s Health. 2011.
- 47. Nove A, Berrington A, Matthews Z. The methodological challenges of attempting to compare the safety of home and hospital birth in terms of the risk of perinatal death. Midwifery. 2012;28(5):619–26. PubMed pmid:22951423.
- 48.
Vedam S, Schummers L, Fulton C. Home birth: an annotated guide to the literature. Vancouver: Division of Midwifery, University of British Columbia; 2013.
- 49.
Vedam S. Evidence-based home birth practice. In: Bailes A, editor. The Home Birth Practice Handbook. Silver Spring Maryland: American College of Nurse-Midwives; 2016.
- 50. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Rating quality of evidence and strength of recommendations: Grade: Going from evidence to recommendations. BMJ: British Medical Journal. 2008;336(7652):1049–51. pmid:18467413
- 51. Hutton EK, Reitsma A, Thorpe J, Brunton G, Kaufman K. Protocol: systematic review and meta-analyses of birth outcomes for women who intend at the onset of labour to give birth at home compared to women of low obstetrical risk who intend to give birth in hospital. Systematic Reviews. 2014;3(55).
- 52. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. Journal of clinical epidemiology. 2011;64(4):395–400. pmid:21194891
- 53. Grant JS, Davis LL. Selection and use of content experts for instrument development. Research in nursing & health. 1997;20(3):269–74.
- 54. Haynes SN, Richard D, Kubany ES. Content validity in psychological assessment: A functional approach to concepts and methods. Psychological assessment. 1995;7(3):238.
- 55. Lynn MR. Determination and quantification of content validity. Nursing Research. 1986;35(6):382–6. pmid:3640358
- 56. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health. 2006;29(5):489–97.
- 57. Stemler SE. A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation. 2004;9(4):1–19.
- 58. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–3. pmid:15883903
- 59. Seehra J, Pandis N, Koletsi D, Fleming PS. Use of quality assessment tools in systematic reviews was varied and inconsistent. Journal of clinical epidemiology. 2016;69:179–84. e5. pmid:26151664
- 60. Thomas B, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews on Evidence‐Based Nursing. 2004;1(3):176–84. pmid:17163895
- 61. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Controlled clinical trials. 1995;16(1):62–73. pmid:7743790
- 62. Cheyney M, Burcher P, Vedam S. A Crusade Against Home Birth. Birth: Issues in Perinatal Care. 2014;41(1):1–4 p. pmid:104049794. Language: English. Entry Date: 20140325. Revision Date: 20151008. Publication Type: Journal Article.
- 63. Zielinski R, Ackerson K, Kane Low L. Planned home birth: benefits, risks, and opportunities. International Journal of Women's Health. 2015;7:361–77. pmid:25914559
- 64. Finsen V, Storeheier AH, Aasland OG. Cesarean section: Norwegian women do as obstetricians do—not as obstetricians say. Birth. 2008;35(2):117–20. pmid:18507582
- 65. Allcock C, Griffiths A, Penketh R. The effects of the attending obstetrician's anxiety trait and the corresponding obstetric intervention rates. Journal of Obstetrics and Gynaecology. 2008;28(4):390–3. pmid:18604670