Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Diversity and inclusion for the All of Us research program: A scoping review

  • Brandy M. Mapes ,

    Roles Conceptualization, Methodology, Visualization, Writing – original draft, Writing – review & editing

    brandy.mapes@vumc.org (BMM); joni.rutter@nih.gov (JLR)

    Affiliation Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America

  • Christopher S. Foster,

    Roles Conceptualization, Formal analysis, Methodology, Project administration, Writing – review & editing

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Sheila V. Kusnoor,

    Roles Formal analysis, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Center for Knowledge Management, Vanderbilt University Medical Center, Nashville, TN, United States of American

  • Marcia I. Epelbaum,

    Roles Formal analysis, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Center for Knowledge Management, Vanderbilt University Medical Center, Nashville, TN, United States of American

  • Mona AuYoung,

    Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Scripps Whittier Diabetes Institute, Scripps Health, San Diego, California, United States of American

  • Gwynne Jenkins,

    Roles Project administration, Writing – review & editing

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Maria Lopez-Class,

    Roles Project administration, Writing – original draft

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Dara Richardson-Heron,

    Roles Project administration

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Ahmed Elmi,

    Roles Project administration

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Karl Surkan,

    Roles Conceptualization, Writing – review & editing

    Affiliation Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America and All of Us Research Program Participant Representative

  • Robert M. Cronin,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Department of Biomedical Informatics, Medicine, and Pediatrics, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America

  • Consuelo H. Wilkins,

    Roles Conceptualization, Writing – review & editing

    Affiliation Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America

  • Eliseo J. Pérez-Stable,

    Roles Conceptualization, Writing – review & editing

    Affiliation National Institute on Minority Health and Health Disparities, Bethesda, Maryland, United States of America

  • Eric Dishman,

    Roles Project administration

    Affiliation Office of the Director, National Institutes of Health, Bethesda, Maryland, United States of America

  • Joshua C. Denny,

    Roles Conceptualization, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Departments of Biomedical Informatics and Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America

  • Joni L. Rutter ,

    Roles Conceptualization, Methodology, Project administration, Writing – original draft, Writing – review & editing

    brandy.mapes@vumc.org (BMM); joni.rutter@nih.gov (JLR)

    Affiliation National Center for Advancing Translational Sciences, National Institutes of Health, Bethesda, Maryland, United States of America

  •  [ ... ],
  • the All of Us Research Program

    Membership of the the All of Us Research Program is provided in the Acknowledgments.

  • [ view all ]
  • [ view less ]

Diversity and inclusion for the All of Us research program: A scoping review

  • Brandy M. Mapes, 
  • Christopher S. Foster, 
  • Sheila V. Kusnoor, 
  • Marcia I. Epelbaum, 
  • Mona AuYoung, 
  • Gwynne Jenkins, 
  • Maria Lopez-Class, 
  • Dara Richardson-Heron, 
  • Ahmed Elmi, 
  • Karl Surkan
PLOS
x

Abstract

The All of Us Research Program (All of Us) is a national effort to accelerate health research by exploring the relationship between lifestyle, environment, and genetics. It is set to become one of the largest research efforts in U.S. history, aiming to build a national resource of data from at least one million participants. All of Us aims to address the need for more diversity in research and set the stage for that diversity to be leveraged in precision medicine research to come. This paper describes how the program assessed demographic characteristics of participants who have enrolled in other U.S. biomedical research cohorts to better understand which groups are traditionally represented or underrepresented in biomedical research. We 1) reviewed the enrollment characteristics of national cohort studies like All of Us, and 2) surveyed the literature, focusing on key diversity categories essential to the program’s enrollment aims. Based on these efforts, All of Us emphasizes enrollment of racial and ethnic minorities, and has formally designated the following additional groups as historically underrepresented: individuals—with inadequate access to medical care; under the age of 18 or over 65; with an annual household income at or below 200% of the federal poverty level; who have a cognitive or physical disability; have less than a high school education or equivalent; are intersex; identify as a sexual or gender minority; or live in rural or non-metropolitan areas. Research accounting for wider demographic variability is critical. Only by ensuring diversity and by addressing the very barriers that limit it, can we position All of Us to better understand and tackle health disparities.

Introduction

The All of Us Research Program (All of Us), funded by the National Institutes of Health (NIH), is a national effort to accelerate health research by exploring the relationship between lifestyle, environment, and genetics [1]. The program is set to become one of the largest health care research efforts in U.S. history, aiming to build a longitudinal resource of multiple data types and biosamples from at least one million participants.

A main goal of the NIH Precision Medicine Initiative (PMI) Working Group [2] was that All of Us “should broadly reflect the diversity of the U.S.” As stated in the 21st Century Cures Act, All of Us will “ensure inclusion of a broad range of participants, including consideration of biological, social, and other determinants of health that contribute to health disparities.” [3] The recognition that not all groups have benefited equally from research is a major step towards health equity. A key contribution that All of Us can make is setting the stage for more diverse participation in precision medicine research to come. We describe the efforts made to date to guarantee that All of Us reflects the broad diversity of the U.S. and enables precision medicine research for all.

To ensure that All of Us is rigorous in its efforts to enroll a diverse group of one million participants, we assessed the demographic characteristics of other U.S. biomedical research cohorts. To better understand which groups are represented or underrepresented in biomedical research, we 1) conducted a broad survey of the literature focused on ten demographic categories essential to the program’s diversity aims, and 2) reviewed the enrollment characteristics of national cohort studies similar to All of Us. By identifying these populations, we can understand the barriers and gaps in enrollment of key underrepresented groups. Based on previous evaluations of diversity and inclusion by race and ethnicity, non-White groups have already been established as underrepresented. [46] It is important to note here a distinction between minority health and underrepresentation of other groups, especially with regards to health disparities research and biomedical research representation. Minority health refers to “health characteristics and attributes of racial and/or ethnic minority groups (defined by Office of Management and Budget), who are socially disadvantaged due in part by being subject to potential discriminatory acts.” [7] Historically, women have been underrepresented as well, though their proportion now exceeds 50% in NIH-funded research. [8] In addition to racial and ethnic diversity in biomedical research, this work proposes that researchers examine other demographic and social determinants when considering inclusion of diverse participant populations. This review established a landscape of demographic characteristics and social determinants meant to guide All of Us towards measuring diversity among its participants.

A Guide for diversity inclusion

Defining underrepresentation for All of Us and stakeholder input

All of Us has relied on designated definitions of diversity guided by the leading authorities on health disparities. In addition, we have considered the inclusion of groups that may be underrepresented in biomedical research, specifically, through consultation with a variety of resources and stakeholders, as follows. The National Institute on Minority Health and Health Disparities (NIMHD), congressionally mandated to designate disparity groups, [9] has designated the following groups as most affected by health disparities in the U.S.: Blacks/African Americans, Hispanics/Latinos, American Indians/Alaska Natives, Asian Americans, Native Hawaiians and other Pacific Islanders, socioeconomically disadvantaged populations, underserved rural populations, and sexual and gender minorities. We also used the guidelines for including women and minorities in clinical research set forth by the NIH [10], as well as a review of the U.S. Census data (Fig 1) [11]. We sought input from NIH leadership and associated stakeholders, including the NIMHD; the Office of Research on Women’s Health; the Sexual and Gender Minority Research Office; the National Human Genome Research Institute; the National Institute of Child Health and Human Development; the National Institute on Aging; the Office of Extramural Research; and the All of Us Research Program. Lastly, input was obtained from the All of Us Participant Provided Information Committee (PPIC), an All of Us governance body who developed the program’s baseline demographic survey that uses existing measures from other national surveys. [1] A diverse team of PPIC experts, including All of Us participant representatives, and All of Us participant engagement teams vetted these measures to ensure appropriate terminology and phrasing and ensured consistency in terminology for guiding the diversity categories developed here, some of which are being implemented as of this writing. We sought to examine if these major groups have historically been underrepresented in the biomedical research literature, including clinical trial studies, and whether these categories would help measure the impact of the All of Us Research Program for these groups.

thumbnail
Fig 1.

(A-B). Projected Age, Race, & Ethnicity, and Sex Composition of the United States through 2040.

https://doi.org/10.1371/journal.pone.0234962.g001

Based on these major inputs, All of Us prioritized nine essential diversity categories to complement racial and ethnic minority inclusion. An important point to note is that, aside from racial/ethnic minorities and sex assigned at birth, the other categories have not been broken out demographically in most other studies. Therefore, it is not explicitly known whether they are truly underrepresented in biomedical research. Explicitly defining and evaluating all these demographic characteristics will be important for understanding their impact on social determinants of health. These include (listed alphabetically): access to care, age, annual household income, disability, educational attainment, gender identity, geography (i.e. rurality), sex assigned at birth, and sexual orientation. We anticipate that eventually 75% of the All of Us cohort will be from historically underrepresented groups in biomedical research and expect nearly half of this diversity to be from racial or ethnic minorities. [1] Of the nine categories, six are included in the NIMHD definitions of “disparity populations,” three are related to socioeconomic status, two to sexual and gender minorities, and one to rurality. For each category, we verified a designation of groups as either historically represented or underrepresented through a review of the literature.

Source selection

We established a team with pertinent expertise to plan and execute the review. The team included representatives from the NIH and participating All of Us institutions with expertise in library and information science, enrollment monitoring and reporting, underserved populations, health disparities research, and strategic planning for large-scale national consortia.

Searches of English language articles were conducted using PubMed and Google Scholar along with a mixture of Medical Subject Headings (MeSH) terms and keywords relevant to the nine diversity categories (S1 Table for search strategies and updated retrieval counts). Excluded from the results were duplicate articles, those not relevant to human subjects research, and those that did not contain data to either support or refute justification for considering designation of representation. Meta-analyses and systematic review articles published since 2012, specific to biomedical cohort studies conducted in the United States, and emphasizing results or conclusions specific to the categories of interest were prioritized for review by a subgroup of the authors. We also used handsearching and consulted the subject matter experts on our team to identify additional potentially relevant white papers and unpublished reports. Examples of ways evidence about representation in research was assessed included: evaluation of data from existing U.S. biomedical research studies and trials, evidence depicting how study eligibility criteria contributed to underrepresentation of these groups, and evidence outlining barriers to participation unique to these groups. Results were refined to include a small set of distinctive articles that emphasized findings specific to underrepresentation of the key groups (i.e., why they were considered underrepresented, whether the groups were included in study recruitment, data analysis, or conclusions). The literature was reviewed over the course of several months in 2017 through 2018 (S2 Table for final counts of screened and selected articles, S2 File for selected bibliography).

Complementary assessment of core diversity characteristics for similar cohort studies

Additionally, a selection of other national cohort studies was evaluated for demographic characteristics benchmarking a diversity baseline for All of Us. We included cohort studies with an enrollment of >75,000 and a study design or scope like All of Us. Those selected were U.S. based except for one international study, the U.K. Biobank. [12] Information was collected using publicly available sources including websites, datasets, and publications. When demographic information was not readily searchable, we made attempts to gather the information from study authors.

Of the national cohort studies evaluated, we identified fifteen as most relevant to informing All of Us based on the aims of the research and mixture of data types collected from participants. Five (33%) were still enrolling participants at the time of our review, and seven (47%) had begun enrollment in 2006 or later. We confirmed that thirteen (87%) collected questionnaire data, collected biospecimens, and linked to external data sources. Eight (53%) collected electronic health records. We also prioritized review of diversity categories including age, educational attainment, household income, U.S. geographic region, disability, race and ethnicity, sex assigned at birth, and sexual and gender minority status. See Fig 2 for a brief demographic snapshot of the cohorts we evaluated.

thumbnail
Fig 2. Core demographic characteristics of similar cohorts evaluated to support development of the All of Us guide for diversity and inclusion.

https://doi.org/10.1371/journal.pone.0234962.g002

Results

Roughly 645 articles were screened throughout the literature review and 170 were selected as significant for informing the All of Us Guide for Diversity and Inclusion (S2 Table for exact counts by search category). Based on the body of literature and consultation with stakeholders, All of Us emphasizes inclusion of racial and ethnic minorities and has prioritized the following additional groups as being possibly underrepresented in biomedical research: individuals with inadequate access to medical care; <18 or >65 years; with an annual household income at or below 200% of the federal poverty level (FPL); who have a cognitive or physical disability; or have less than a high school education or equivalent. Also included are those individuals who: are intersex; identify as a sexual or gender minority; or live in rural or non-metropolitan areas. Including the racial and ethnic groups, Table 1 shows historically represented and underrepresented characteristics of each category and how they are measured by the survey responses. [13] Researchers interested in studying specific subsets of these groups (for example, individuals < 3 or >80), are able to do so based on the way in which these demographic data have been collected.

These designations are supported by assessing the other national cohort studies. Historically underrepresented groups, where we were able to obtain data, were underrepresented across all fifteen studies evaluated. Annual household income was unknown, not collected, or not reported for thirteen (87%). Of those reporting income, one (7%) is U.S. based and only a small portion of the participants had an annual household income under $25,000. [15] Educational attainment data were obtained for seven (47%) cohort studies. Percentage of participants with less than a high school degree or GED ranged from 1% to 15% for those U.S. based studies. [1518] We confirmed inclusion of older adults in only five (33%) studies and their representation ranged from <17% to 37%. [16,1922] Participants under 18 were enrolled in two (13%) studies. [22,23] Only one (7%) study reported on representation of participants with a disability; these participants made up 18% of that total sample. [17]

Fourteen (93%) studies reported demographic representation of participants by sex. Males were not eligible for four of the studies. Ten (67%) were eligible to both males and females and reported on sex. Of those ten, females were overrepresented in five (50%). [12,18,21,22,24] None of the fifteen studies reported intersex. Five (33%) reported participants’ geographic region. Those mostly recruited from only one U.S. geographic region though two (13%) reported geographic dispersity stretching across at least four regions. [15,25] Data to specifically assess rurality were not readily available. Only two (13%) studies reported overrepresentation of racial and ethnic minorities compared to White participants. [23,24]

Lastly, most did not draw an intentional distinction between sex assigned at birth and gender. Data to permit evaluation of gender minority representation, however, were not collected from participants or not publicly reported for any of the fifteen studies. This was the case for sexual orientation data as well. See Fig 2 for aggregate findings from the cohort assessment and Table 1 for All of Us Guide for Diversity and Inclusion.

Discussion

Most of the articles sought to illustrate major health care and research barriers impacting racial and ethnic populations, along with other underrepresented groups. The literature demonstrated that racial/ethnic minorities and other groups are typically impacted by at least one of several barriers including financial (e.g., insurance coverage, income), cognitive (e.g., health literacy, technical proficiency), language (e.g., lack of translation resources), communication and cultural factors (e.g., cultural humility, lack of trust), structural (e.g., transportation, geographic accessibility), racial and ethnic discrimination, or study design (e.g., eligibility criteria limiting participation or inclusion, failure to reach diverse groups due to recruitment limitations). [7,2634] These barriers have significant effects on multiple groups. Access to research opportunities is critically impacted by structural barriers like transportation, but also by language, communication, and unnecessary restrictions in study design. For example, willingness to participate in research does not significantly differ between individuals who reside in rural areas and those who live in urban locations. Rather, the limiting factor is accessibility impacted by either structural barriers or recruitment limitations. [31,34,35] Older adults are also impacted by these barriers. The literature demonstrated that older adults have been marginalized in research due to a number of reasons including not meeting eligibility criteria (e.g., clinical trial restrictions on comorbid conditions due to risk of side effects), perceived burden on study staff (e.g., longer screening process or more “high touch” retention tactics needed), and cognitive or physical impairments that affect the ability to consent or fully participate. [3638] Similar barriers are faced by racial or ethnic minorities. [3941] Cultural barriers, specifically lack of trust, and not being trustworthy, are defining barriers across racial or ethnic minorities, particularly Black and African American individuals, [34,42] and also across many other groups including rural populations and sexual or gender minorities. Specifically in cancer-related clinical trials, studies have shown no racial or ethnic differences in participation rates [43], but this does not seem to be generally applicable to other types of clinical research settings. [44] Importantly, overcoming a lack of trust is through engagement and carefully designed and strategies [45] by which participants are invited to participate, in a respectful way, in research programs.

Representation of underrepresented groups is significantly affected by the lack of consistent demographic data collection and gaps in research domains specific to the health disparities that impact them. For example, unlike sex assigned at birth, which is a biological construct, gender is a construct that includes many dimensions. Domain experts recommend researchers include measures to capture both an individual’s assigned sex at birth and gender identity to fully represent one’s physical and internal sense of self. [46] The literature confirmed that gender identity as a social determinant of health is not only understudied but also unrecognized. [47,48] This body of literature is comprised of articles on evaluation of instruments that gather this information (e.g., questionnaires, electronic medical records, etc.), their limited use within healthcare and research [49,50] and the lack of knowledge and sensitivity of medical personnel about different gender identities. [51] Similarly, sexual orientation measures are not often collected from patients and research participants, especially noted for older adults. [52,53] These omissions were observed in our own review of national cohorts. The literature also noted that biomedical research has generally neglected exploration of sexual orientation, apart from research focused on understanding sexually transmitted disease. This implies a substantial gap in research that could lead to improvements in broader health outcomes for sexual and gender minorities. [54,55] The lack of data collection measures and demographic reporting does not allow us to confirm whether individuals that identify as sexual or gender minorities are truly underrepresented. Data collection is a mandatory first step, however, in addressing the obvious gaps in research results that directly impact health disparities faced by this population. For this reason, sexual and gender minorities are included as an underrepresented group.

We identified 12 articles that examined sex as a biological factor in biomedical research. Several of the articles call for more comparison of research findings by sex, noting that, like race and ethnicity, sex differences in many domains are still widely understudied. [5660] Regardless of actual representation of females in biomedical research, many health outcomes specific to them will remain unknown unless researchers prioritize exploring differences across sex assigned at birth, race, and ethnicity. In addition, sex is predominantly perceived as a binary biological construct and individuals born as intersex, or those with disorders of sexual development, have difficulties “fitting” into a male/female binary category. Including intersex as a biological construct is important for health care since individuals’ needs may not apply to either the male or female classification. [59] The literature indicates that this is also true for transgender individuals who have undergone gender affirming surgeries or who receive hormonal treatments as well. [61]

We identified literature that explained underrepresentation of select minority populations due to issues in how race and ethnicity data are traditionally collected and reported. [62,63] For example, there has been confusion among survey respondents over the categorization of Hispanic or Latino as an ethnicity rather than a race. [63] A lack of sufficient categories for race and ethnicity data, often in combination with underrepresentation within the research samples, means results are often reported in aggregated categories that can mask underlying disparities. For example, Middle Eastern and North African populations have often been classified as White in research questionnaires that do not specifically call out a category for Middle Eastern and North African, similarly Asian American, Native Hawaiian, and other Pacific Islanders have historically been grouped as Asian or Other. [62,63]

Limitations of this approach

The broad scope and depth of this review was limited since complex strategies were needed to canvass the literature, returning exceptionally large retrieval sizes. This work is meant as a survey of trends, relying mainly on reviews obtained from the literature to best inform the state of the science on groups historically underrepresented in biomedical research. Our evaluation of other large national cohort studies was also limited because we conducted most of this work using publicly available sources. In some instances, we were unable to confirm the accuracy or currency of the information. In addition, All of Us is a hypothesis and disease-agnostic research program with minimal inclusion/exclusion criteria. Most of the studies we identified in our evaluation differ from All of Us in both areas. For example, clinical trials-enrolled participants with specific medical conditions or those that may benefit from a therapeutic treatment. They may, by their very nature, exclude certain demographic groups. Finally, many studies do not collect comprehensive socio-demographic data, therefore, we do not know whether their populations include groups we assert to be underrepresented.

Commentary

There is a rich diversity within the U.S. that must be leveraged for understanding factors contributing to health and disease for the entire population. Historically, biomedical research has neglected this diversity to our detriment. A recent report [64] showed that, of the drugs removed from the U.S. market between 1997 and 2000, 80% were removed due to side effects or fatalities that occurred in women. Another report [65] found that racial factors, although important, do not solely account for the disparities in health care. Societal and environmental factors are equally important to account for context. [66] A widely-used antiplatelet drug, clopidogrel, needs to be converted to an active form to be effective. [67] However, due to a genetic variation that affects the body’s ability to metabolize it, only about 50% of individuals of Asian ancestry are able to benefit, along with other individuals who carry the variant. This is one example of a modern scientific advancement, in this case a life-saving medication, offering a high benefit for some individuals within populations, but significant risks for others. [68]

An early research opportunity for All of Us will be examining health disparities among these racial and ethnic populations intersected across other demographic groups and their impact on social determinants of health. For a summary of significant conclusions, see Table 2. The All of Us Research Program Guide for Diversity and Inclusion (Table 1) is an essential product for All of Us, serving as the program foundation for 1. future demographic enrollment monitoring, 2. gauging the success of new recruitment strategies as they are implemented and providing opportunity to revise when key enrollment gaps are identified, and 3. gauging the variability and strength of the resulting research data repository, and ensuring that it will support health disparity research for all historically underrepresented groups as it grows over time. This guide is a novel attempt to emphasize and define diversity and inclusion for All of Us based on a series of biologically and socially significant constructs. There are considerable nuances that impact diversity and inclusion monitoring and we respect that they should not be treated independently of one another or considered universally equal. For example, there is an enormous body of literature confirming the importance of racial and ethnic diversity in biomedical research. The All of Us guide emphasizes minority representation in biomedical research, and the additional demographic characteristics and social determinants highlighted are meant to complement, not dilute, that inclusion.

thumbnail
Table 2. Key points learned in developing the All of Us guide for diversity and inclusion.

https://doi.org/10.1371/journal.pone.0234962.t002

Conclusion

Research that accounts for wider demographic variability is critical. It is time that we move beyond relying on research generated from largely homogeneous populations of usually White, male, urban, and higher socio-economic status. [69] Only by incorporating more diverse factors as key variables, ensuring inclusion, and identifying and addressing barriers that limit research participation, can we position All of Us to better understand and adequately tackle health disparities.

Research is used to provide the basis by which we discover new pathways and identify new treatment approaches. As such, only by narrowing the gaps of health disparities and increasing our understanding of how social determinants of health impact health outcomes can we realize a future of research that is more precise. We note that discrimination is a standing issue in workforce diversity, [70] which impacts minority participation in research studies like ours. Based on the work presented, and in conjunction with community partners, All of Us has developed resources for engaging diverse communities (https://www.joinallofus.org/en/community/community-resources). These provide a springboard for researchers to more readily incorporate diversity and inclusion strategies to local settings [45,71] or disease specific contexts, or to use the https://www.researchallofus.org/ platform to build diverse cohorts. Within All of Us, the attention paid to these important factors can help surface the intersectionality of these barriers to research participation and help the research community to be better equipped to engage diverse populations.

The approach taken by All of Us was broad, methodical, and intentional in conceptualizing an inclusive guide. Despite the limitations, it encompasses a depth of inclusion that is unprecedented in other national cohorts. The emphasis on historically underrepresented minorities and other populations provides a reference for any future study that aims to ensure diversity and equity. The All of Us Research Program is well on its way to creating a national resource that reflects and supports the broad diversity of the U.S. necessary for advancing precision medicine for all.

Supporting information

S1 Table. Search strategies by diversity category and January 2020 retrieval counts.

https://doi.org/10.1371/journal.pone.0234962.s001

(DOCX)

S2 Table. Number of articles screened and selected in original literature search.

https://doi.org/10.1371/journal.pone.0234962.s002

(DOCX)

S1 File. Bibliography of selected references by search category.

https://doi.org/10.1371/journal.pone.0234962.s003

(DOCX)

S2 File. Bibliography of selected references by search category.

https://doi.org/10.1371/journal.pone.0234962.s004

(DOCX)

Acknowledgments

We thank our colleagues, David R. Williams, Scott Sutherland, James McClain, Holly Garriock, Ed Ramos, Aditi Reddy, Stephanie Devaney, and Kelly Gebo for their support and input on the Diversity and Inclusion guide. We thank Rebecca N. Jerome for providing input on the methodology used for the project and Carolyn Diehl for assistance collecting data for the cohort assessment. We also thank the PMI NIH Protocol Development group, the DRC Pilot Research Core, and the All of Us PPI Committee for their efforts developing, evaluating, and finalizing the demographic measures included in the All of Us Basics survey. The All of Us Research Program would not be possible without the partnership of contributions made by its participants. See the supplementary information (S1 File) for a roster of past and present All of Us principle investigators. To learn more about the All of Us Research Program’s research data repository, please visit https://www.researchallofus.org/.

References

  1. 1. The “All of Us” Research Program. New England Journal of Medicine. 2019 Aug 15;381(7):668–76. pmid:31412182
  2. 2. The Precision Medicine Initiative Cohort Program–Building a Research Foundation for 21st Century Medicine.: 108.
  3. 3. Bonamici S. H.R.34 - 114th Congress (2015–2016): 21st Century Cures Act [Internet]. 2016 [cited 2019 Feb 1]. Available from: https://www.congress.gov/bill/114th-congress/house-bill/34
  4. 4. Popejoy AB, Fullerton SM. Genomics is failing on diversity. Nature. 2016 Oct 12;538(7624):161–4. pmid:27734877
  5. 5. Knepper TC, McLeod HL. When will clinical trials finally reflect diversity? Nature. 2018 May;557(7704):157–9. pmid:29743700
  6. 6. Fisher ER, Pratt R, Esch R, Kocher M, Wilson K, Lee W, et al. The role of race and ethnicity in views toward and participation in genetic studies and precision medicine research in the United States: A systematic review of qualitative and quantitative studies. Mol Genet Genomic Med. 2019 Dec 23;e1099. pmid:31867882
  7. 7. Duran DG, Pérez-Stable EJ. Novel Approaches to Advance Minority Health and Health Disparities Research. Am J Public Health. 2019 Jan 1;109(S1):S8–10. pmid:30699017
  8. 8. Office of Research on Women’s Health and NIH Support for Research on Women’s Health. Report of the Advisory Committee on Research on Women’s Health Fiscal Years 2015–2016 [Internet]. National Institutes of Health; 2017. Available from: https://orwh.od.nih.gov/sites/orwh/files/docs/ORWH_Biennial_Report_WEB_508_FY-15-16.pdf
  9. 9. Legislative History [Internet]. NIMHD. [cited 2019 May 8]. Available from: https://www.nimhd.nih.gov/about/legislative-info/legislative-history.html
  10. 10. Inclusion of Women and Minorities as Participants in Research Involving Human Subjects—Policy Implementation Page | grants.nih.gov [Internet]. [cited 2019 Jan 25]. Available from: https://grants.nih.gov/grants/funding/women_min/women_min.htm
  11. 11. Bureau UC. 2017 National Population Projections Tables [Internet]. [cited 2019 May 8]. Available from: https://www.census.gov/data/tables/2017/demo/popproj/2017-summary-tables.html
  12. 12. Bycroft C, Freeman C, Petkova D, Band G, Elliott LT, Sharp K, et al. The UK Biobank resource with deep phenotyping and genomic data. Nature. 2018;562(7726):203–9. pmid:30305743
  13. 13. Cronin RM, Jerome RN, Mapes B, Andrade R, Johnston R, Ayala J, et al. Development of the Initial Surveys for the All of Us Research Program. Epidemiology. 2019 Apr 29;
  14. 14. Research C for DE and. E7 Studies in Support of Special Populations: Geriatrics [Internet]. U.S. Food and Drug Administration. 2019 [cited 2019 Jun 26]. Available from: http://www.fda.gov/regulatory-information/search-fda-guidance-documents/e7-studies-support-special-populations-geriatrics
  15. 15. Design of the Women’s Health Initiative clinical trial and observational study. The Women’s Health Initiative Study Group. Control Clin Trials. 1998 Feb;19(1):61–109. pmid:9492970
  16. 16. Participants’ Profile—NIH_AARP Diet and Health Study [Internet]. [cited 2018 Feb 13]. Available from: https://dietandhealth.cancer.gov/profile.html
  17. 17. Smith TC, Millennium Cohort Study Team. The US Department of Defense Millennium Cohort Study: career span and beyond longitudinal follow-up. J Occup Environ Med. 2009 Oct;51(10):1193–201. pmid:19786902
  18. 18. History of the Cancer Prevention Studies l American Cancer Society [Internet]. [cited 2018 Feb 13]. Available from: https://www.cancer.org/research/we-conduct-cancer-research/epidemiology/history-cancer-prevention-study.html
  19. 19. California Teachers Study [Internet]. California Teachers Study. [cited 2018 Feb 13]. Available from: https://www.calteachersstudy.org
  20. 20. Gaziano JM, Concato J, Brophy M, Fiore L, Pyarajan S, Breeling J, et al. Million Veteran Program: A mega-biobank to study genetic influences on health and disease. Journal of Clinical Epidemiology. 2016 Feb 1;70:214–23. pmid:26441289
  21. 21. Zhu CS, Pinsky PF, Kramer BS, Prorok PC, Purdue MP, Berg CD, et al. The prostate, lung, colorectal, and ovarian cancer screening trial and its associated research resource. J Natl Cancer Inst. 2013 Nov 20;105(22):1684–93. pmid:24115361
  22. 22. Roden DM, Pulley JM, Basford MA, Bernard GR, Clayton EW, Balser JR, et al. Development of a large-scale de-identified DNA biobank to enable personalized medicine. Clin Pharmacol Ther. 2008 Sep;84(3):362–9. pmid:18500243
  23. 23. Koebnick C, Coleman KJ, Black MH, Smith N, Der-Sarkissian JK, Jacobsen SJ, et al. Cohort Profile: The KPSC Children’s Health Study, a population-based study of 920 000 children and adolescents in southern California. Int J Epidemiol. 2012 Jun 1;41(3):627–33. pmid:21257603
  24. 24. Kolonel LN, Altshuler D, Henderson BE. The multiethnic cohort study: exploring genes, lifestyle and cancer risk. Nat Rev Cancer. 2004 Jul;4(7):519–27. pmid:15229477
  25. 25. Bao Y, Bertoia ML, Lenart EB, Stampfer MJ, Willett WC, Speizer FE, et al. Origin, Methods, and Evolution of the Three Nurses’ Health Studies. Am J Public Health. 2016 Sep;106(9):1573–81. pmid:27459450
  26. 26. Carrillo JE, Carrillo VA, Perez HR, Salas-Lopez D, Natale-Pereira A, Byron AT. Defining and targeting health care access barriers. J Health Care Poor Underserved. 2011 May;22(2):562–75. pmid:21551934
  27. 27. Neutens T. Accessibility, equity and health care: review and research directions for transport geographers. Journal of Transport Geography. 2015 Feb;43:14–27.
  28. 28. Campbell DJ, Manns BJ, Hemmelgarn BR, Sanmartin C, King-Shier KM. Development of a conceptual framework for understanding financial barriers to care among patients with cardiovascular-related chronic disease: a protocol for a qualitative (grounded theory) study. CMAJ Open. 2016 Jun;4(2):E304–8. pmid:27398378
  29. 29. Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014 Mar 25;14:42. pmid:24669751
  30. 30. George S, Duran N, Norris K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders. Am J Public Health. 2014 Feb;104(2):e16–31. pmid:24328648
  31. 31. Virani S, Burke L, Remick SC, Abraham J. Barriers to recruitment of rural patients in cancer clinical trials. J Oncol Pract. 2011 May;7(3):172–7. pmid:21886499
  32. 32. Woodall A, Morgan C, Sloan C, Howard L. Barriers to participation in mental health research: are there specific gender, ethnicity and age related barriers? BMC Psychiatry. 2010 Dec 2;10:103. pmid:21126334
  33. 33. Townsley CA, Selby R, Siu LL. Systematic review of barriers to the recruitment of older patients with cancer onto clinical trials. J Clin Oncol. 2005 May 1;23(13):3112–24. pmid:15860871
  34. 34. Kim SH, Tanner A, Friedman DB, Foster C, Bergeron CD. Barriers to clinical trial participation: a comparison of rural and urban communities in South Carolina. J Community Health. 2014 Jun;39(3):562–71. pmid:24310703
  35. 35. Friedman DB, Foster C, Bergeron CD, Tanner A, Kim SH. A qualitative study of recruitment barriers, motivators, and community-based strategies for increasing clinical trials participation among rural and urban populations. Am J Health Promot. 2015 Jun;29(5):332–8. pmid:24670073
  36. 36. Cherubini A, Oristrell J, Pla X, Ruggiero C, Ferretti R, Diestre G, et al. The persistent exclusion of older patients from ongoing clinical trials regarding heart failure. Arch Intern Med. 2011 Mar 28;171(6):550–6. pmid:21444844
  37. 37. Zulman DM, Sussman JB, Chen X, Cigolle CT, Blaum CS, Hayward RA. Examining the evidence: a systematic review of the inclusion and analysis of older adults in randomized controlled trials. J Gen Intern Med. 2011 Jul;26(7):783–90. pmid:21286840
  38. 38. Tahhan AS, Vaduganathan M, Greene SJ, Fonarow GC, Fiuzat M, Jessup M, et al. Enrollment of Older Patients, Women, and Racial and Ethnic Minorities in Contemporary Heart Failure Clinical Trials: A Systematic Review. JAMA Cardiol. 2018 Oct 1;3(10):1011–9. pmid:30140928
  39. 39. Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants. Annu Rev Public Health. 2006;27:1–28. pmid:16533107
  40. 40. Wendler D, Kington R, Madans J, Van Wye G, Christ-Schmidt H, Pratt LA, et al. Are racial and ethnic minorities less willing to participate in health research? PLoS Med. 2006 Feb;3(2):e19. pmid:16318411
  41. 41. Bolen S, Tilburt J, Baffi C, Gary TL, Powe N, Howerton M, et al. Defining “success” in recruitment of underrepresented populations to cancer clinical trials: moving toward a more consistent approach. Cancer. 2006 Mar 15;106(6):1197–204. pmid:16453333
  42. 42. Hamel LM, Penner LA, Albrecht TL, Heath E, Gwede CK, Eggly S. Barriers to Clinical Trial Enrollment in Racial and Ethnic Minority Patients With Cancer. Cancer Control. 2016 Oct;23(4):327–37. pmid:27842322
  43. 43. Langford AT, Resnicow K, Dimond EP, Denicoff AM, Germain DS, McCaskill‐Stevens W, et al. Racial/ethnic differences in clinical trial enrollment, refusal rates, ineligibility, and reasons for decline among patients at sites in the National Cancer Institute’s Community Cancer Centers Program. Cancer. 2014;120(6):877–84. pmid:24327389
  44. 44. Cunningham-Erves J, Villalta-Gil V, Wallston KA, Boyer AP, Wilkins CH. Racial differences in two measures of trust in biomedical research. J Clin Transl Sci. 2019 Jun 17;3(2–3):113–9. pmid:31660234
  45. 45. Wilkins CH. Effective Engagement Requires Trust and Being Trustworthy. Med Care. 2018 Oct;56(10 Suppl 1):S6–8.
  46. 46. Best Practices for Asking Questions to Identify Transgender and Other Gender Minority Respondents on Population-Based Surveys—Williams InstituteWilliams Institute [Internet]. [cited 2018 Jun 20]. Available from: https://williamsinstitute.law.ucla.edu/research/census-lgbt-demographics-studies/geniuss-report-sept-2014/
  47. 47. Reisner SL, Conron KJ, Tardiff LA, Jarvi S, Gordon AR, Austin SB. Monitoring the health of transgender and other gender minority populations: validity of natal sex and gender identity survey items in a U.S. national cohort of young adults. BMC Public Health. 2014 Nov 26;14:1224. pmid:25427573
  48. 48. Pega F, Veale JF. The case for the World Health Organization’s Commission on Social Determinants of Health to address gender identity. Am J Public Health. 2015 Mar;105(3):e58–62. pmid:25602894
  49. 49. Deutsch MB, Keatley J, Sevelius J, Shade SB. Collection of gender identity data using electronic medical records: survey of current end-user practices. J Assoc Nurses AIDS Care. 2014 Dec;25(6):657–63. pmid:24880490
  50. 50. Schneider C, Cerwenka S, Nieder TO, Briken P, Cohen-Kettenis PT, De Cuypere G, et al. Measuring Gender Dysphoria: A Multicenter Examination and Comparison of the Utrecht Gender Dysphoria Scale and the Gender Identity/Gender Dysphoria Questionnaire for Adolescents and Adults. Arch Sex Behav. 2016 Apr;45(3):551–8. pmid:26883025
  51. 51. Carabez R, Pellegrini M, Mankovitz A, Eliason M, Scott M. Does your organization use gender inclusive forms? Nurses’ confusion about trans* terminology. J Clin Nurs. 2015 Nov;24(21–22):3306–17. pmid:26263919
  52. 52. Fredriksen-Goldsen KI, Kim H-J. Count me in: response to sexual orientation measures among older adults. Res Aging. 2015 Jul;37(5):464–80. pmid:25651579
  53. 53. Sell RL, Becker JB. Sexual orientation data collection and progress toward Healthy People 2010. Am J Public Health. 2001 Jun;91(6):876–82. pmid:11392926
  54. 54. Boehmer U. Twenty years of public health research: inclusion of lesbian, gay, bisexual, and transgender populations. Am J Public Health. 2002 Jul;92(7):1125–30. pmid:12084696
  55. 55. Mitchell M, Howarth C, Kotecha M, Kreegan C. Sexual orientation research review 2008. Equality and Human Rights Commission; 2008.
  56. 56. Segarra I, Modamio P, Fernandez C, Marino EL. Sex-Divergent Clinical Outcomes and Precision Medicine: An Important New Role for Institutional Review Boards and Research Ethics Committees. Front Pharmacol. 2017;8:488. pmid:28785221
  57. 57. Powers MS, Smith PH, McKee SA, Ehringer MA. From sexless to sexy: Why it is time for human genetics to consider and report analyses of sex. Biol Sex Differ. 2017;8:15. pmid:28473910
  58. 58. Mansukhani NA, Yoon DY, Teter KA, Stubbs VC, Helenowski IB, Woodruff TK, et al. Determining If Sex Bias Exists in Human Surgical Clinical Research. JAMA Surg. 2016 Nov 1;151(11):1022–30. pmid:27551816
  59. 59. Nolan MR, Nguyen TL. Analysis and reporting of sex differences in phase III medical device clinical trials-how are we doing? J Womens Health (Larchmt). 2013 May;22(5):399–401.
  60. 60. Weinberger AH, Smith PH, Kaufman M, McKee SA. Consideration of sex in clinical trials of transdermal nicotine patch: a systematic review. Exp Clin Psychopharmacol. 2014 Oct;22(5):373–83. pmid:25133506
  61. 61. Deutsch MB, Buchholz D. Electronic health records and transgender patients—practical recommendations for the collection of gender identity data. J Gen Intern Med. 2015 Jun;30(6):843–7. pmid:25560316
  62. 62. NW 1615 L. St, Suite 800Washington, Inquiries D 20036USA202-419-4300 | M-857-8562 | F-419-4372 | M. Census may change questions on race, Hispanic origin for 2020 [Internet]. Pew Research Center. [cited 2019 Mar 26]. Available from: https://www.pewresearch.org/fact-tank/2017/04/20/seeking-better-data-on-hispanics-census-bureau-may-change-how-it-asks-about-race/
  63. 63. 2015 National Content Test: Race and Ethnicity Analysis Report [Internet]. [cited 2019 Mar 26]. Available from: https://www.census.gov/programs-surveys/decennial-census/2020-census/planning-management/final-analysis/2015nct-race-ethnicity-analysis.html
  64. 64. Nowogrodzki A. Clinical research: Inequality in medicine. Nature. 2017 Oct 4;550:S18–9. pmid:28976953
  65. 65. LaVeist T, Pollack K, Thorpe R, Fesahazion R, Gaskin D. Place, Not Race: Disparities Dissipate In Southwest Baltimore When Blacks And Whites Live Under Similar Conditions. Health Affairs. 2011 Oct 1;30(10):1880–7. pmid:21976330
  66. 66. Leong AB, Ramsey CD, Celedón JC. The Challenge of Asthma in Minority Populations. Clinic Rev Allerg Immunol. 2012 Aug 1;43(1):156–83.
  67. 67. Mega JL, Simon T, Collet J-P, Anderson JL, Antman EM, Bliden K, et al. Reduced-function CYP2C19 genotype and risk of adverse clinical outcomes among patients treated with clopidogrel predominantly for PCI: a meta-analysis. JAMA. 2010 Oct 27;304(16):1821–30. pmid:20978260
  68. 68. Reference GH. Clopidogrel resistance [Internet]. Genetics Home Reference. [cited 2019 Mar 25]. Available from: https://ghr.nlm.nih.gov/condition/clopidogrel-resistance
  69. 69. Clinical Trials Still Don’t Reflect The Diversity Of America [Internet]. NPR.org. 2015 [cited 2019 Jan 15]. Available from: https://www.npr.org/sections/health-shots/2015/12/16/459666750/clinical-trials-still-dont-reflect-the-diversity-of-america
  70. 70. Laurencin CT. The context of diversity. Science. 2019 Nov 22;366(6468):929–929. pmid:31753971
  71. 71. Friedman EA, Gostin LO. From local adaptation to activism and global solidarity: framing a research and innovation agenda towards true health equity. International Journal for Equity in Health. 2017 Feb 21;16(1):18. pmid:28219436