Introduction

Ethnicity and race are cornerstones of individuals’ sense of self-identity, social belonging, and shared experiences that influence one’s health beliefs, behaviours, and outcomes [1]. Ethnicity and race are socially-defined constructs that are complex and multilayered. While they are sometimes used interchangeably, they are two different but related concepts. The term “race” suggests a biological basis for socially-constructed categories that in-group members are implied to share greater genetic homogeneity than out-group members [2]. However, in reality, the degree of additional genetic similarity shared among members of the same race is largely negligible and biologically inconsequential compared to the total genetic makeup shared between individuals from different races [3]. The term “ethnicity” generally refers to a wide range of socially-constructed categories that in-group members tend to share a common culture, language, heritage, or national origin. While race is often characterized by a person’s physical attributes such as body height, hair texture, facial feature, and skin color, ethnicity is a person’s subjective affinity towards an ethnic group that he or she feels most self-identifiable with [4]. Since ethnicity is more widely used than race in Canada and it is conceptualized more narrowly for research and surveillance purposes [2], it is the focus of this research.

With great ethnic diversity, Canada is challenged by pervasive and persistent public health and social issues that are disproportionately affecting specific ethnic groups, including health inequality and racial discrimination towards Aboriginals and visible minorities [5, 6]. Specifically, Aboriginals experience significantly higher disease prevalence and worse clinical outcomes for a large number of acute and chronic conditions [7–9], yet they are also burdened by lower access and awareness to available health resources compared to non-Aboriginals in Canada [10–12]. As a result, ethnicity (and race) is considered a key social determinant of health in Canada [13]. Despite its public health significance and recognition, ethnicity data is absent in the vast majority of disease registries, patient-care administrative data, vital statistics data, and major health surveys across Canada [5, 6, 14]. This ethnicity data gap impedes the pursuit and attainment of ethnically-specific health evidence that are essential for the understanding, development, and monitoring of effective health policies and programs [5, 6].

Implementing large-scale, systematic data collection on ethnicity in health data across Canada may encounter many practical, political, and legal challenges [6, 15]. While the general public and health researchers and practitioners largely agree that collecting ethnicity for research is ethical and desirable, the net benefit may not be realized by all individuals [6]. For example, directly asking questions about ethnicity and race has evoked anxieties about racism and racist classification in some patients, especially those who have personally experienced racism in the healthcare system. Some interviewers have raised concerns regarding if the process of asking about ethnic and racial identities may steer the respondents to believe that inequality is endemic in the healthcare system [6]. An alternative approach that is timelier, less costly, and indirect in acquiring ethnicity information needs to be explored in Canada.

Unlike ethnicity, personal names are often collected in most databases by default. An alternative is to automate the prediction of ethnicity using commonly-collected variables such as personal names. Personal names are typically recorded in the form of unstructured format of the entire name or more structured format with specific name entities (such as first, middle, and last name). Since many naming practices are influenced by cultural, religious, and familial traditions that intersect with ethnicity, individuals’ first, middle, last, and full names carry a degree of predictive quality for the associated ethnicity. Mateos [16] conducted a systematic review which identified 13 representative studies that developed name-based ethnicity classification methodologies. The sensitivity and positive predictive value (PPV) of these studies ranged between 67–95% and 70–96%, respectively. These studies included a number of common methodological process and research components: 1) a name reference list is independently developed or sourced from another study or domain experts; 2) a separate target population (or testing data) is manually or automatically classified into different ethnic categories; and 3) the performance metrics of the method is evaluated against the previously known ethnicity (“gold standards”) in the target population [16]. The process of predicting or classifying ethnicity for the target population were either done manually or automatically. The manual approach typically involved domain experts examining and making the judgment of what the most likely ethnicity was for a particular name, based on the expertise in linguistic and ethnocultural history. In contrast, the automated approach did not rely on human judgment. Instead, it was based on programming the computer to automatically detect and utilize signals from names in the form of rule-based patterns (such as regular expression), statistical patterns (such as ML), or a combination of both. Mateos [16] found nine studies that have employed automated, as opposed to manual, ethnicity prediction methods to predict the ethnicity of the target population. Machine learning (ML) frameworks have continuously been explored in this area [17–20]. Ambekar et al. [17] combined decision tree and Hidden Markov Model (HMM) to conduct classification on a taxonomy with 13 ethnic categories. Treeratpituk et al. [15] examined both alphabetical and phonetic sequences in names to improve predictions. Fiscella and Fremont [21] and Imai and Khanna [22] have found that combining name and residence location further improved the performance.

Most published studies to date are non-Canadian with ML models trained with databases outside of Canada, which were not optimized for the Canadian population. Aboriginal populations must be included in this line of work as they are vastly marginalized and disadvantaged. In many ML publications, a formal procedure of feature selection and hyperparameter optimization may not be conducted or reported, thus reducing research reproducibility. Past Canadian studies using automated ML approaches to predict ethnicity tend to be scarce, outdated, examining only a few ethnic groups, and not including Aboriginals and their subgroups as distinct categories [16]. To our knowledge, no studies have combined textual and phonetic name features and location features to predict ethnicity. The primary objective of this study is to determine if the ethnicity data gap in Canada may be potentially addressed using a novel automated ML approach. This is achieved by conducting and formally evaluating the predictive performance of a large-scale ML framework utilizing a novel name and location feature set for ethnicity prediction. The secondary objectives of this study are to provide detailed description and to share codebase of this ML framework to support future research.

Materials and methods

Data source and analytic framework

The Canadian census contains all the required person-level variable fields, including self-reported full name, census location, sex, and ethnicity. From the Statistics Act of 2005, the release of personal records in past censuses is restricted for 92 years after their respective years of collection [23]. However, individual records from censuses prior to 1906 were publicly accessible at the National Archives of Canada. Under these constraints, we chose census 1901 as data source to build our ML framework (Fig 1). Features were extracted from the text recorded in name and census location variable fields. The census was split randomly into 80% training set and 20% test set. A development (dev) set was created by randomly-sampling 12.5% of the training set (Fig 1). Using only the dev set, the feature selection and hyperparameter optimization steps were conducted to finalize name and census location feature sets and to automatically optimize the ML algorithms, respectively. Finally, the obtained final feature sets and optimized hyperparameter values were used to train the ML algorithms within the training set. The trained ML algorithms would then be applied to predict the ethnicity of individuals in the test set. The agreement and discrepancy between the recorded ethnicity in census (as “gold standards”) and our predicted ethnicity were quantified based on a predefined set of evaluation metrics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Overview of the machine learning framework.

https://doi.org/10.1371/journal.pone.0241239.g001

Feature engineering

The feature extraction process to create individual and grouped name features are described in Table 1, using “Wing Sun Lee” as an example. A technical challenge using the name variable in census 1901 was that it contained unstructured text, as opposed to the more recent censuses that divided the name into “given name” and “family name” at data collection. We labeled the first name entity (“Wing”) as “First name”, the last name entity (“Lee”) as “Last name”, and any remaining text string as “Middle name”. While we recognize that the most accurate split of the Chinese name “Wing Sun Lee” should divide “Wing Sun” as first name and “Lee” as last name, this name splitting would require the foreknowledge that it belongs to a Chinese person. Despite these exceptions, our selected name splitting method should be generalizable to most individuals in most databases where the information of ethnicity is unavailable. The key aspect was that all the textural information of the name was captured from the span of the extracted name features as a whole, even though there would be a small information lost due to the incorrect placement into the “First name”, “Middle name”, and “Last name” predictor slots for a small number of individuals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Extraction and groupings of name features.

https://doi.org/10.1371/journal.pone.0241239.t001

For the location features, in addition to the existing full location string values, the corresponding province/territory, district, and subdistrict were extracted into three separate features. For example, the original location string of “H, L'Assomption, Quebec, Canada” corresponds to “subdistrict number, district, province/territory, country”. The text was processed by removing the non-informative “Canada” and lowercasing into “h, l'assomption, quebec” (as “Full location string”). It was then further broken into individual features as “quebec” (as “Province/Territory”), “l’assomption” (as “District”), and “h” (as “Subdistrict”). “All location features” included all four location features “Full location string”, “Province/Territory”, “District”, and “Subdistrict”.

Results

Census 1901 initially contained 5,079,210 records. After removing missing and duplicated records, the total number of unique individuals was 4,812,958 (94.8%). From this, 20% randomly extracted as test set (N = 962,592). The remaining 80% became training set (N = 3,850,366). 12.5% of the training set was randomly-selected to form the dev set (N = 481,296). The breakdown by ethnicity is illustrated in Fig 2. Considerable class imbalance was observed in some of the minority ethnic categories (i.e., Inuit) with low occurrence frequencies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Frequency count breakdown by ethnic groups in census 1901 (N = 4,812,958).

Each Aboriginal person represents two counts: one in the Aboriginal label and another in one of the followings: First Nations, Métis, Inuit, or all-other-Aboriginal label.

https://doi.org/10.1371/journal.pone.0241239.g002

Feature selection

Table 2 shows that all the name and location features outperformed the benchmarks by at least 10% in F1, fulfilling the a-priori criteria for inclusion. The “All name features” outperformed all the individual name feature subsets by 7% to 275% in F1. The “All name and location features” outperformed the “All name features” and “All location features” by 20% and 76%, respectively. The content of the “All name features” and “All name and location features” were confirmed, as previously described, to be the final feature sets for both ML pipelines.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Feature set selection result for multiclass classification using 5-fold cross-validation in development set.

https://doi.org/10.1371/journal.pone.0241239.t002

Multiclass classification

Table 3 describes the predictive performance for multiclass classification. The DT and RF classifiers consistently and vastly underperformed compared to LR, SVC, and NB classifiers, thus their results were omitted. Overall, LR classifiers tended to perform marginally better than SVC, while both were more superior over NB. The “All name and location features” expectedly achieved the best overall performance. Improvement by adding the location features for specific ethnicity label varied widely between 2% (Ch and Fr) and 34% (Ab) in F1 for the LR classifiers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Multiclass classification predictive performance in test set with ML models trained in training set.

https://doi.org/10.1371/journal.pone.0241239.t003

The multiclass confusion matrix for “All name and location features” with LR is shown in Table 4. The multiclass confusion matrix for “All name features” with LR is shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Multiclass confusion matrix in test set for the “All name and location features” set trained in training set with regularised logistic regression classifier.

https://doi.org/10.1371/journal.pone.0241239.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Multiclass confusion matrix in test set for the “All name features” set trained in training set with regularised logistic regression classifier.

https://doi.org/10.1371/journal.pone.0241239.t005

Binary class classification

The binary classification predictive performance is described in Table 6 for LR. Similar tables for SVR and NB are presented in Tables 7 and 8, respectively. The addition of location features improved markedly for specific groups (i.e., more than 10% increase in F1 for Irish, Japanese, others, and all Aboriginal groups) (Table 6). For the “All name and location features”, 7 out of 13 (53.8%) and 10 out of 13 (76.9%) binary ethnicity classifications achieved 70% or greater in F1 and average PPV, respectively (Table 6). The accuracy and AUC-ROC ranged from 85–100% and 82–100%, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Binary classification predictive performance in test set based on regularised logistic regression classifiers trained in training set.

https://doi.org/10.1371/journal.pone.0241239.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Binary classification predictive performance in test set based on C-support vector classifiers trained in training set.

https://doi.org/10.1371/journal.pone.0241239.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Binary classification predictive performance in test set based on naïve Bayes classifiers trained in training set.

https://doi.org/10.1371/journal.pone.0241239.t008

Discussion

To our best knowledge, we conducted one of the most extensive ML research in Canada within the domain of ethnicity prediction using name and location information. Overall, we employed a two-pipeline approach to demonstrate our classifiers’ performance for a wider range of potential applications. The multiclass classifiers achieved 76% F1 and 91% accuracy. The confusion matrices showed that most frequently-misclassified labels for English, Irish, and Scottish individuals occurred among themselves. This is to be expected as these groups historically shared a large degree of cultural and linguistic heritage. To improve the performance, regrouping them all under one generic “British” category may be considered, as shown in Ambekar et al. [17].

Aboriginals are most frequently misclassified as En, Ir, Sc, and Fr. The Indian Act was first introduced in 1876 (prior to 1901 census) as a consolidation of previous colonial mandates and regulations that aimed to eradicate First Nations culture in favour of assimilation into Euro-Canadian societies [31]. Thus, the misclassifications among Aboriginals are likely a result of the Indian Act’s naming policies which unjustly forced Aboriginals to adopt new European names. Despite this inherent challenge, there was a large improvement in performance by adding location features onto the name feature sets. This aligned with the phenomenon that certain geographic regions in Canada were highly populated by Aboriginal populations [32]. For binary classifications, the sensitivity and F1 increased by 44% (36% to 80%) and 34% (50% to 84%) for “Aboriginal” predictions, and by 43% (35% to 78%) and 32% (49% to 81%) for “First Nations” predictions, respectively. Métis and Inuit binary classifications resulted in mediocre and poor predictions (65% and 31% F1, respectively), which is not unexpected due to the interracial marriage with Europeans among Métis [33] and the small available training data of Inuit in census data. Excluding the Ab, Ab-Fn, Ab-Mé, Ab-In, En, Ir, and Sc, the F1 ranged from 70% to 95% (median: 87%) in binary classification for the remaining six ethnic categories (Ch, Fr, It, Jp, Ru, Others). As mentioned in the Materials and Methods section, our name splitting method likely misplaced the first and last names from the correct name feature labels for a portion of Chinese individuals if their last names were recorded as the first (position) name entity. However, the consistently high predictive performance for Chinese (i.e., 0.88 sensitivity, 0.96 PPV, and 0.92 F1 in binary classification with “All name features” set) confirms our assumption that the information lost is negligible as long as all the name entities are considered and further extracted somewhere within the span of all the name features.

In Canada, the trends and degrees of urbanisation and migration differ by ethnic groups. From 1961 to 2006, the percent of Aboriginals living in urban areas (as opposed to Indian reserves and rural areas) increased from 13% to 53% [34]. More specifically from 1981 to 2006, it increased from 40% to 53% for Aboriginals and from 75% to 81% for non-Aboriginals. However, residential segregation by ethnicity or ethnicity-associated factors remains. For example, residential aggregation of visible minorities, recent immigrants, and Aboriginals exists and is found to be a key associative factor underpinning the increase of concentrated urban poverty in various regions [35]. In addition to being highly populated in Canadian territories (Northwest Territories, Yukon, and Nunavut), Aboriginals are found to become more concentrated in a number of Prairie cities, particularly Saskatoon, Regina, and Winnipeg in recent times. Families of visible minority account for up to 78% of the low-income families residing in high poverty neighborhoods in 2001, doubling the level in 1981 [35]. Despite the changes in geographic boundaries between censuses and the dynamic nature of residential mobility and migration by Canadians, these findings strongly suggest that discernable patterns in geographic segregation by specific ethnic groups may always exist. These serves as indirect evidence that our demonstrated ML method utilizing name and location features should remain relevant and applicable if it is to be adopted by modern datasets.

The feature selection step has shown alphabetic name features carry more predictive quality over phonetic name features, which aligned with findings by Treeratpituk and Giles [15]. Similar to Fiscella and Fremont [21] and Imai and Khanna [22], we found that combining name and location features markedly improved performance for some but not all ethnic categories. In terms of predictive performance, direct and complete comparison of our study to published studies is impossible since different studies have employed different methodologies (time periods, populations, data sources, ML classifiers, performance measures, and ethnicity labels and grouping). Nonetheless, a number of indirect comparisons are feasible. For the ethnic categories overlapped between our study and Ambekar et al. [17], we achieved better performance in French (our F1: 95%, their F1: 57%) and Japanese (our F1: 90%, their F1: 83%), and similar performance in Italian (our F1: 67%, their F1: 66%). For the ethnic categories overlapped between our and Treeratpituk and Giles [15], we achieved better performance in French (our F1: 95%, their F1: 80%), Russian (our F1: 84%, their F1: 81%), and Chinese (our F1: 94%, their F1: 91%), but worse performance in Italian (our F1: 67%, their F1: 85%) and Japanese (our F1: 90%, their F1: 96%). For previous Canadian studies, our Chinese classifier (92% sensitivity, 95% PPV, 94% F1) outperformed Choi et al. (1993) (73% sensitivity, 81–84% PPV) [36] and is comparable to Coldman et al. (1988) (89–97% sensitivity) [37].

In terms of potential real-life applications, the trained ML classifiers using “All name features” can be applied to the personal name field of other databases to generate predicted ethnicity at the individual-level. However, databases that also contain respondents’ residential addresses will not be able to use our trained ML classifiers with “All name and location features” directly since they are trained with the older (1901) census boundaries. An additional step is needed to standardize and remap the location information before using both name and location information from applied databases. For example, one option is to convert both the location information in both census 1901 and applied database to the Global Positioning System (GPS) coordinates. The individuals in the applied database will receive an approximated census location, via mapping the GPS of their addresses to the nearest corresponding census location, which can then be used directly by our trained ML classifiers. Available remapping tools exist that enable the geographic conversion of residential addresses to corresponding census location information [38].

Conclusions

This is the first comprehensive Canadian study to show that a wide range of ethnic categories can be accurately predicted using a ML framework that learns from relatively simple and widely-collected personal name and location information. There are many potential public health applications (i.e., disease and risk factor surveillance, effectiveness of intervention, and patterns in health service utilization and related costs) in which adding the ethnicity dimension will greatly multiply the value, utility, and relevance of the existing information. Wide-spread implementation of ethnicity classifiers will help generate ethnically-specific health evidence that together will fill many critical knowledge gaps that currently impede effective health program and policy development in Canada.