Revisiting OMB standards

In 1997, the Office of Management and Budget (OMB) established standardized questions on r/e that federal agencies and recipients of federal funds were obligated to report data on. This move was motivated by a need to enforce civil rights laws, especially by monitoring equal access to housing, education, and employment for “population groups that historically had experienced discrimination and differential treatment” [11]. In March 2024, after nearly two years of consultation and input, the OMB published revisions to Statistical Policy Directive No. 15 [12]. The new directive notes that “the race and ethnicity categories set forth are sociopolitical constructs and are not an attempt to define race and ethnicity biologically or genetically” [12]. It is beyond the scope of this article to examine the new OMB directive in detail, but we address it further below.

Prior to the 2024 revisions, the OMB standards had been comprised of two ethnicity categories (“Hispanic or Latino” and “Not Hispanic or Latino”) and five race categories. Section 3101 of the Public Health Service Act also requires any federally conducted or supported healthcare or public health program, activity, or survey to collect and report data using the Department of Health & Human Services (HHS) standards [13]. The HHS standards broke down the OMB’s five race categories into 13 subcategories, and the two ethnicity categories into four [14]. Individual hospitals can create more granular r/e categories, but these must roll up into the OMB standards. While hospitals not receiving federal funding will be exempt from these standards, professional associations like the American Hospital Association recommend the collection of r/e data [15].

Others have sought to standardize r/e data collection, while still conforming to the OMB standards. For example, the Observational Outcomes Partnership (OMOP) established the Common Data Model (CDM), “an open community data standard, designed to standardize the structure and content of observational data” [16]. Although there has been significant internal debate on how race and ethnicity should be standardized [17], the CDM currently follows OMB standards, with the “Race” field containing dozens of concepts, while the “Ethnicity” field contains two [18].

The new OMB standards emerged from years of debate. In June of 2022, the Chief Statistician of the U.S. announced that his office would be reviewing and revising current OMB standards on r/e data [19]. This move was in recognition of (1) increasing racial and ethnic diversity; (2) the growing number of people identifying as more than one race or ethnicity; and (3) changing immigration and migration patterns. Following OMB’s request for comment on some initial proposals, a debate about r/e data categories has resurfaced, with some stakeholders advocating for the r/e questions to be combined, given evidence that many respondents view race and ethnicity as similar or the same [20]. However, other presenters raised concerns that combining race and ethnicity might lead to erasure within racially and ethnically diverse communities. For example, Afro-Latino patients could feel compelled to choose between the ‘Hispanic or Latino’ and the ‘Black or African American’ categories, when they would otherwise have categorized themselves as both [20].

Conceptual challenges

Developing categories for r/e is an epistemically challenging undertaking with broad social consequences for medical research. To take just one example, scholars disagree on whether the ‘one drop rule’ (the idea that anyone with a Black ancestor is considered Black) is a reductionist approach and ignores the complexities of identity and culture, reinforcing negative stereotypes like that of racial purity [21]. Relatedly, some have advocated for using ‘multiracial’ as an r/e category [22], while others oppose this designation [23]. Moreover, there is discourse on whether r/e categories should be used at all, or whether they should be replaced by ancestry in biomedical research and clinical practice [24]. R/e categories are essential for understanding health disparities and developing targeted interventions, but there are those who criticize their oversimplification and failure to capture the complexity of identity and culture [25,26].

These historical debates were considered in the National Academies of Sciences, Engineering, and Medicine (NASEM) report on the Use of Race, Ethnicity, and Ancestry as Population Descriptors in Genomics Research [27]. The standardization of r/e categories for use in biomedical AI modeling should be guided by these particularly relevant recommendations:

• “[R]esearchers should avoid typological thinking, including the assumption and implication of hierarchy, homogeneity, distinct categories, or stability over time of the groups.”
• “Researchers, as well as those who draw on their findings, should be attentive to the connotations and impacts of the terminology they use to label groups.”
• “For each descriptor selected, labels should be applied consistently to all participants.”
• “Researchers should disclose the process by which they selected and assigned group labels…”

Quality control

Based on two decades of research on optimizing the collection of r/e data in hospital contexts, recommendations for best practices have emerged. In Box 1, we identify several of these best practices that are germane to the issues we raise here (Box 1). The first step is to prioritize this problem on an institutional level and develop a plan for improving r/e data classification accuracy. This plan should be patient-centered, facilitating patient autonomy over how their r/e data is reported and then shared. Patients should also be given a clear rationale for why their r/e information is being requested.

Given the diversity of self-identification preferences and significant cross-cultural and cross-national differences in how r/e is conceptualized, we recognize that it is impossible to create r/e categories that perfectly reflect the sociodemographic groups with which individuals identify. But if the representativeness of AI datasets is to be evaluated, it follows that a categorization scheme must be deployed. Our contention is that in refining those categories institutions should continue to develop more granular categories reflecting the diversity of their unique communities. Precisely because these categories are challenging to define, and vary across cultures, it is important for institutional policymakers to listen to concerns from members of their communities.

Our proposed solution seeks to reconcile the tension between allowing for flexibility in patient identification and facilitating cross-institutional comparisons via standardization. A lack of standardization, both in defining categories and developing procedures for operationalizing them, is at the core of the problems we have identified above (see Section III). In the absence of uniform guidelines for collecting and categorizing r/e, different healthcare institutions may use divergent approaches, leading to incomparable datasets that cannot be meaningfully merged or analyzed at a broader scale. Without a standardized approach to measuring and reporting r/e in AI healthcare training datasets, we would expect to see significant variations across institutions in the quality of r/e data. This variation would contribute to the perennial problem of aggregating across datasets from different hospitals [40]. Using fragmented datasets may also impede efforts to monitor and evaluate AI systems for bias. If standardization across major systems is not practical, then at a minimum, institutions should be transparent about their methods and ensure that however they have defined their categories, they are accurate and consistent in applying them. This transparency and granular data could facilitate standardization across institutions, even when those institutions adopt different categories.

Addressing the practical challenges involved in implementing these best practices will need to be a priority if institutions wish to bring about real change in the quality of the r/e data they generate. We identify three practical challenges to prioritize. The first is that we are not aware of a publicly available training program that can be readily imported to address r/e data collection issues. An initial step will be investment in a short training module. The second challenge, related to the first, is that r/e data is likely to be collected by several different types of staff members, including those who admit patients, clinical staff who treat patients, and IT staff who prepare data files and link r/e data from one system to another. It is therefore important for the training module to be completed by all who may be initially collecting, reviewing, and archiving r/e data. The third challenge is that developing a training module and implementing it will face competing priorities in terms of resource allocation. We believe that investing in the implementation of these best practices is likely to yield outsized benefits in the longer term by increasing patient trust and unlocking the full potential of AI systems to improve health outcomes. An incremental, iterative approach can also help alleviate some of these challenges, as pilot programs will enable hospitals to refine workflows, train staff, and measure outcomes before scaling up across hospital systems.

Warranting data quality

Leading AI ethics frameworks stress the importance of transparency and sharing with end users the contents of the black box supporting medical decision-making [47]. One promising strategy to promote transparency is the use of labeling for AI-based medical devices [48]. We advocate for a label on medical AI software which acknowledges data quality limitations and discloses the data collection processes used for the r/e datasets on which the models were created.

We suggest that this disclaimer could take one of the following forms:

This disclaimer could be added at minimal time cost. The collection methods that could be specified include patient self-reporting through pre-appointment online forms. Such disclaimers would serve the ethical priority of transparency in AI modeling and the scientific priority of ensuring that AI models are built on high quality data—and if not, making everyone aware of this deficiency.

We recognize that the entities and individuals developing medical AI models will often be distinct from those who are gathering demographic information. Inspired by the longstanding practice of “ethical sourcing” of materials in the field of corporate social responsibility [49], our approach places the onus on AI developers to use reasonable efforts to identify the source(s) and quality of the r/e data they deploy in their models. For example, missing r/e data is regularly imputed in healthcare datasets [50–52]. Developers should be aware of when the r/e data they are using has been imputed and should communicate this to those who use their AI systems. AI developers often work with multiple datasets to train their AI model. In such instances, we believe that developers should disclose the collection methodologies for each dataset that was utilized in the development of the medical AI model, especially when the r/e datasets differ in terms of quality.

Our approach is forward-looking and intended to spur systemic changes in r/e data collection practices. We anticipate that in the near term, many AI developers will need to select Option #2 because they will not know how the r/e data in their models was generated. This would be true, for instance, for those AI developers who are working with historical datasets. However, we also anticipate that precisely because they are initially scarcer, AI models that can provide Option #1 explanations will be more highly valued. For example, a firm could differentiate itself from competitors by pointing to its use of gold-standard r/e data. Relatedly, as medical AI faces increasing regulatory scrutiny, markets may favor those model developers who can offer an Option #1 explanation. Regulatory bodies such as the U.S. Food and Drug Administration and the E.U.’s AI Office will be actively monitoring the potential for bias in medical AI systems. In the case of the E.U. AI Act, high-risk AI systems must be trained, validated, and tested with data that meet quality criteria, which include “data collection processes and the origin of data” (Article 10(2)(b)). The path forward to more accurate r/e data in AI models may be slow, but we believe that without data quality disclaimers progress will be even slower.