https://orcid.org/0000-0002-1927-1703
Figures
Abstract
Breast cancer remains a significant concern worldwide, with a rising incidence in Indonesia. This study aims to evaluate the applicability of risk-based screening approaches in the Indonesian demographic through a case-control study involving 305 women. We developed a personalized breast cancer risk assessment workflow that integrates multiple risk factors, including clinical (Gail) and polygenic (Mavaddat) risk predictions, into a consolidated risk category. By evaluating the area under the receiver operating characteristic curve (AUC) of each single-factor risk model, we demonstrated that they retained their predictive accuracy in the Indonesian context (AUC for clinical risk: 0.67 [0.61,0.74]; AUC for genetic risk: 0.67 [0.61,0.73]). Notably, our combined risk approach enhanced the AUC to 0.70 [0.64,0.76], highlighting the advantages of a multifaceted model. Our findings demonstrate for the first time the applicability of the Mavaddat and Gail models to Indonesian populations, and show that within this demographic, combined risk models provide a superior predictive framework compared to single-factor approaches.
Citation: Rabbani B, Tanu SG, Ramanto KN, Audrienna J, Fernandez EA, Aldila F, et al. (2025) A combined risk model shows viability for personalized breast cancer risk assessment in the Indonesian population: A case/control study. PLoS One 20(5): e0321545. https://doi.org/10.1371/journal.pone.0321545
Editor: Klaus Brusgaard, Odense University Hospital, DENMARK
Received: October 24, 2024; Accepted: March 7, 2025; Published: May 15, 2025
Copyright: © 2025 Rabbani et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: De-identified or anonymized datasets can not be shared publicly due to ethical and legal reasons. Researchers were prohibited from sharing research data which contains sensitive patient information such as health history data as specified by Komisi Etik Penelitian Kedokteran Dan Kesehatan MRCCC Hospital, the Ethics Committee that has approved the study protocol. Permission to access research data is restricted to the corresponding authors only and data requests could be sent to Komisi Etik Penelitian Kedokteran Dan Kesehatan MRCCC Hospital (Ph: +6221-2996-2888; Fax: +6221-2996-2778).
Funding: This study was primarily funded by NalaGenetics.
Competing interests: I have read the journal's policy and the authors of this manuscript have the following competing interests: The authors are affiliated with their respective organizations. BR, SGT, KNR, JA, FA, EAF, MGP, MDV, JH, LHU, YM, KIJ, MA, GG, FS, AF, SSM, LS and AI are affiliated with NalaGenetics. NalaGenetics is a biotech company focusing on personalized medications, diet, and screening. The risk prediction tool investigated in the study was developed by NalaGenetics. FAA, MW, STBS, AW, AS and SJH are affiliated with SJH Initiatives. SJH Initiatives is a research body under the MRCCC Siloam Hospitals network whose one of the main focus is on breast cancer research. SA is affiliated with Atma Jaya Catholic University of Indonesia. Atma Jaya is a private Catholic university based in Jakarta, Indonesia. The authors have no financial gain or loss in any form that could result from the publication of this manuscript, but the absence of authors without any organizational affiliation could be considered a non-financial competing interest.
Introduction
Breast cancer is a pressing global public health concern. It has emerged as the most commonly diagnosed type of cancer worldwide, with its prevalence steadily rising in recent years. For example, in 2020 alone, an estimated 2.48 million cases were reported worldwide, resulting in approximately 685,000 deaths [1]. In Indonesia, breast cancer mirrors the global trajectory, establishing itself as a leading cause of mortality among women [2]. Specifically, in 2020, GLOBOCAN documented over 68,000 new diagnoses and 22,000 resultant deaths, representing incidence and mortality rates of 15.3% and 11%, respectively [3]. Thus, addressing the burden of breast cancer is crucial for improving public health outcomes.
As with other cancers, prompt diagnosis and timely therapeutic interventions have been unequivocally associated with enhanced patient prognosis and diminished mortality rates. Specifically, survival rates for stage 1 breast cancer are estimated at 99% five years postdiagnosis, but this figure quickly diminishes to 30% for advanced stages [4]. As a result, numerous countries have established population screening programs to promote the early detection of breast cancer. Mammography remains the primary method for breast cancer screening and has been proven to decrease mortality rates [5,6]. Nonetheless, mammography is not devoid of challenges. Typically, it adopts a uniform approach, recommending that all women within a certain age bracket undergo the procedure [6]. Moreover, in Asian populations, women present higher breast tissue density, which complicates mammogram interpretation and can lead to false-positive diagnoses [7]. Compounding the challenges above, Indonesia has yet to incorporate breast cancer screening into its national health agenda, and the procedure remains unsupported by the national health insurance framework. This financial constraint dissuades women from pursuing mammographic screening and compromises diagnostic rates when the disease is the most actionable [8]. For instance, a longitudinal analysis conducted over three decades at multiple academic hospitals found that a significant proportion of patients, ranging from 68–73%, only sought medical consultations during the advanced stages of the disease [2].
Collectively, the observations highlighted above underscore the need for refining breast cancer screening strategies, emphasizing both accessibility and diagnostic precision. Recently, risk-based screening, which customizes recommendations according to individual risk profiles, has challenged traditional one-size-fits-all screening paradigms and is emerging as a promising approach for enhanced patient stratification [9–12]. A myriad of risk factors, encompassing clinical predispositions, familial history, and genetic markers, have been considered in these advanced screening methods. For example, the Gail model, a nongenetic risk assessment tool, offers insights into the probability of an individual developing breast cancer over a five-year span based on clinical risk factors, including age, reproductive history, and familial breast cancer incidence [13]. Similarly, pathogenic mutations in high-penetrance genes, such as BRCA1 (Breast Cancer Gene 1) and BRCA2 (Breast Cancer Gene 2), which are involved in DNA (deoxyribonucleic acid) repair, have long been recognized as pivotal risk determinants for breast cancer and have been incorporated into routine clinical practice [14]. More recently, Polygenic Risk Score (PRS) such as the Mavaddat model, which aggregates the effects of multiple low-penetrance genes, have been demonstrated to harbor predictive power comparable to their high-penetrance counterparts [15,16]. However, despite advances in risk-based screening, most research and model development has been focused on Western populations [17]. This leaves a gap in understanding the applicability of these models to Southeast Asian populations, such as Indonesia. Unique genetic, environmental, and clinical factors may affect breast cancer risk in these populations, so validating and adapting these models is essential.
An end-to-end workflow for personalized breast cancer risk assessment
In the present study, we assess the applicability of three notable breast cancer risk assessment models to the Indonesian demographic based on a case-control study involving 305 women. We focus on the predictive accuracy of the Gail model, which evaluates clinical risk, the Mavaddat model, which assesses polygenic risk, and pathogenic mutations in BRCA1/2 genes, which are representative of monogenic risk. Additionally, we utilized a combined risk model to categorize patients into either elevated or average risk groups. Our objective is to address the current knowledge gap and offer a risk assessment instrument specifically designed for the Indonesian context.
Methods
Ethics approval and consent to participate
This study received ethical approval from the MRCCC Siloam Hospitals IRB under the approval number 005/EA/KEPKKRSMRCCC/X/2020. Amendments to the study protocol, including the inclusion of a larger sample size, have also been approved by the IRB, under the approval number 005/EA/KEPKKRSMRCCC/X/2022.
Written informed consent was taken from all the participants. Participant identifying information was recorded. Only the researchers involved in the recruitment process had the ability to identify individual participants during or after data collection.
An end-to-end workflow for personalized breast cancer risk assessment
As part of this study, we developed an end-to-end workflow for personalized breast cancer risk assessment. A visual representation of the workflow is provided in Fig 1, with details on the methods described below.
The process begins with a pretest counselling session, during which eligible participants, upon providing informed consent, submit a buccal sample and complete a risk survey. This sample is then processed in a testing laboratory where it is genotyped using microarrays. Bioinformatic analyses are next employed to calculate ancestry-adjusted PRS and to translate these into 5-year absolute risk scores, leveraging localized breast cancer incidence and mortality data. Array results are also inspected to determine the presence of pathogenic/likely pathogenic mutations in BRCA1/2 genes. Last, responses from the clinical risk survey are analyzed to derive 5-year absolute risk scores in accordance with the Gail model. In a final step, both the genetic and clinical risk scores are combined into a risk category, with participants receiving individualized risk reports during a posttest consultation.
Pretest consultation.
Eligible female participants (women aged 25–75 who have never been diagnosed with breast cancer and have no known mutations in BRCA1 or BRCA2) initially undergo a pregenetic consultation, either on-site or online, where they receive information about the test. After the consultation, they are asked to consent to the collection of buccal samples for genetic testing and to complete a questionnaire that gathers nongenetic breast cancer risk factors, as outlined in the Gail model (see Additional File 1). Buccal samples were then collected using OraCollect (cat. no. DNA OCR-100, provided by DNA Genotek).
gDNA Extraction and genotyping.
Genomic DNA (gDNA) was extracted using the Monarch® Genomic DNA Purification Kit (cat. no. T3010 from NEB). The extraction procedure adheres to the manufacturer’s instructions, incorporating an additional dry-spin step at maximum speed for 1 minute following the second buffer washing step. The quality and concentration of the gDNA extracts were measured using BioDrop-µLITE. The acceptance criteria for DNA quality adhere to the manufacturer’s guidelines for the extraction kit, specifically requiring absorbance ratios of A260/230 and A260/280 to be greater than 1.7 and a DNA yield exceeding 500 ng.
Array genotyping.
Genotyping is conducted using standard processing on the Illumina GSA (Global Screening Array) chip (Infinium Global Screening Array-24 Kit) by Genomic Solidaritas Indonesia. Raw data files (IDAT files) are converted to VCF format (Variant Call Format) using iiap-cli & GTCtoVCF Illumina software (genome build GRCh37). Missing calls are inferred by performing imputation with Eagle2 [18] and minimac4 [19], using the 1000 Genomes project as the reference panel.
Calculation of clinical risk.
The calculation of clinical risk is based on the Gail model [13]. Five-year absolute risk scores are determined using responses from the pretest consultation questionnaire, which includes information on the patient’s age, age at menarche and first full-term pregnancy, number of first-degree relatives with breast cancer, history of breast biopsy, presence of breast biopsy with atypical hyperplasia, and ethnicity.
Calculation of polygenic risk.
Genetic risk is assessed using the Mavaddat PRS model [16]. Initially, microarray results are subsetted to focus on the 313 markers specified in the model. Direct genotyping and imputation results are integrated, prioritizing microarray data, and supplementing with imputation results only when the INFO SCORE exceeds 0.8.
Subsequently, genotype calls are utilized for PRS calculation through PLINK [20]. Briefly, this process involves computing a weighted average of alleles present in each individual’s genetic profile across the 313 variants specified in the model. Missing variants are inferred using the --read-freq option and allele frequency data from the GNOMAD database [21]. Next, the raw PRS scores from PLINK are standardized as Z scores using the mean and standard deviation from a Southeast Asian cohort (MEC study [11]). These standardized scores are then adjusted for population structure according to methods outlined by Hao et al., 2022, employing a linear regression model that incorporates the first four principal components derived from the 1000 Genomes dataset [22]. Ultimately, the ancestry-adjusted PRS are translated into 5-year absolute scores using established methods [9], which are factors in incidence and mortality data pertinent to the Indonesian population [23].
Calculation of monogenic risk.
The BRCA1 and BRCA2 genes were selected as the targets for monogenic risk calculations. All detected variants in these (based on direct genotyping from the Illumina GSA chip) genes were annotated using Nirvana 3.18.1 [24]. Following the annotation process, additional filtering was performed according to the following criteria: (i) the variant’s significance is either pathogenic or likely pathogenic; (ii) it possesses a ClinVar review rating of>=2 stars; (iii) there is no conflicting interpretation; (iv) it is a nonreference variant; and (v) it has a genotype quality score of>=3. The determination of the monogenic risk category is based on whether a pathogenic or likely pathogenic variant is detected in the sample.
Calculation of combined risk.
Initially, individual risk factors are transformed into categorical outcomes in the following manner: 5-year absolute risk scores for clinical and polygenic risk are classified as either elevated or average, utilizing a 1.7% threshold, while monogenic risk is considered elevated if a pathogenic variant is identified. Subsequently, a unified risk category was derived using two distinct approaches. In the first approach, the highest risk classification among the three inputs was selected as the final category. Alternatively, patients were stratified based on the total number of elevated risk factors identified (1, 2, or 3). These methodologies were then compared to assess the extent of co-occurrence among the risk factors.
Post-test consultation.
The risk assessment results are compiled into a personalized report that includes clinical and genetic risk scores and categories, combined risk categories, and customized health modification recommendations based on the test findings. Examples of these reports for both average and elevated-risk individuals can be found in Additional Files 2 and 3. These risk reports are then presented to patients during a posttest consultation with an oncologist.
Workflow validation
To validate the accuracy of our risk prediction workflow, we conducted both preclinical and clinical validation studies. On the one hand, the preclinical validation study employed well-characterized reference materials from the Genome In a Bottle (GIAB) project [25], along with a mock dataset simulating responses to the clinical risk survey, to assess our ability to accurately genotype variants of interest and verify the correct implementation of the Gail model. On the other hand, clinical validation, executed as a case-control study, aimed to evaluate the predictive accuracy of the selected models (including both the clinical risk and PRS, as well as their combined risk assessment) specifically within the Indonesian population.
Preclinical validation.
A preclinical validation study was conducted to assess the accuracy of our clinical risk prediction algorithm, which was developed based on the Gail model. This validation employed a mock dataset created without any ties to actual patients or individuals. The dataset consisted of manually generated clinical survey responses, encompassing a wide range of ethnicities, clinical risk factors, and risk outcomes. Using this mock dataset as input, we proceeded to calculate a clinical risk score using our algorithm. Simultaneously, the mock dataset was also analyzed using the NIH Breast Cancer Risk Assessment Tool (BRCAT) to determine the anticipated clinical risk score. Upon completing both analyses, a comparison was undertaken using Pearson correlation analysis.
A separate validation study was conducted to assess the performance of our genotyping algorithm. The validation process employed GIAB samples (HG001–005), with each being genotyped in duplicate. Each sample underwent processing through our data analysis pipeline to derive genotype calls at each of the 313 target sites. The resulting VCF files were compared against their respective truth sets. The accuracy of the genotyping was assessed on a per-sample basis by calculating various metrics, including:
- Callability: The percentage of loci successfully genotyped out of the 313 loci in the PRS model.
- Genotype concordance: The percentage of genotyped sites with a correct call.
- Analytical sensitivity: The percentage of variant sites correctly identified.
- Analytical specificity: The percentage of nonvariant sites correctly identified.
- Precision: The percentage of variants correctly genotyped relative to the number of reported variants.
Additionally, we genotyped 18 additional 1KGP cell lines to facilitate a per-site assessment of variant calling accuracy. The truthset for this dataset consisted of 30X WGS (Whole Genome Sequencing) results from the Registry of Open Data in AWS (Amazon Web Services) [22]. The combined set of results, including 5 GIAB and 19 1KGP cell lines, covered 306 out of the 313 PRS loci. Each site was evaluated for concordance against the truthset across all samples and deemed concordant if the results were correct in more than 95% of the samples.
We validated the implementation of the Mavaddat PRS model in our algorithm through a comparative analysis with a genetic risk prediction tool from a third-party software company. We utilized a subset of the samples from the current study, which included 12 healthy individuals and 20 breast cancer patients. The evaluation was based on two predefined criteria: first, we measured the correlation between the PRS; and second, we assessed the overlap in categorical outcomes with the phenotypes present in our cohort.
To assess genotyping accuracy in the BRCA1 and BRCA2 genes, we utilized two distinct reference materials from Horizon (HD793 and HD794), each engineered to contain mutations in these genes, and evaluated a total of 26 genomic loci. Each sample was genotyped three times and processed through our data analysis pipeline to derive genotype calls at each target site. The resulting VCF files were compared against Horizon’s verified mutations, and genotyping accuracy was assessed using the performance metrics introduced above (see “Assessing genotyping accuracy in Mavaddat PRS markers”).
Clinical validation study.
To assess the accuracy of our risk prediction workflow in a real-world setting, we initiated a case/control study (TRIP), during which healthy participants and breast cancer patients were recruited over a span of two years (from 23 May 2021–1 February 2023).
Power analysis was conducted to determine the sample size of the study using Python’s statsmodel module [26]. We estimated the sample size with a power analysis based on a t test for two independent groups. The effect size is calculated using Cohen’s D formula:
where is the effect size,
is the mean of the PRS,
is the standard deviation,
is the number of samples, and the subscripts
and
refer to the treatment and control groups respectively. The mean and standard deviation are based on the literature [27]. This calculation provided us with an effect size of 0.396. The estimated sample size for the two-tailed test is calculated using the following formula:
where is the sample size for each group,
is the critical value of
and
is the critical value of
. Based on this analysis, we estimated the minimum sample size would be 101 samples for both the case and control groups, or 202 samples, assuming a power of 0.8. We recruited additional samples to minimize error and anticipate dropouts.
Participants were recruited from MRCCC Siloam Hospitals Semanggi Jakarta, Indonesia, and other locations. Breast cancer patients were assigned to the case group, while healthy participants were categorized into the control group. All inclusion criteria specified in Table 1 must be met for the patients or individuals to be included in the case/control group. Individuals not affiliated with Indonesian ethnic groups or of Chinese-Indonesian descent were excluded from the study to align with the research’s specific population focus. Additionally, participants who did not provide informed consent for participation and subsequent follow-up were also excluded from the study cohort.
Statistical analysis.
An independent samples t test was conducted to compare the clinical score, adjusted PRS, and genetic score between cases and controls. This test was utilized to determine whether significant differences existed between the two groups. Additionally, logistic regression analysis was performed to examine the relationship between the categorical outcome variables and the variables of interest while controlling for the effects of age and ethnicity.
Receiver operating characteristics (ROC) analysis was conducted to assess the predictive performance of the clinical score, adjusted PRS, genetic score, and combined risk. The ROC curve, generated using the R package “pROC” [28], provided the AUC which was used to quantify the overall discriminative ability of our models. To calculate the AUC for combined risk, we first listed a set of possible thresholds using pROC. These thresholds were applied to both the clinical and genetic scores to infer a categorical combined risk, while also incorporating the presence of monogenic variant. We then calculated the sensitivity and specificity of each threshold. Finally, we plotted the ROC curve and calculated the AUC based on the sensitivity and specificity values.
To assess the strength of the association between the dependent variable (cases and controls) and combined risk categories (elevated and average), a contingency table was created using the “epitools” package [29]. The odds ratio (OR) was then calculated based on the contingency table using the “odds ratio” function, providing a measure of the association strength between the variables of interest.
Results
Participant demographics
Following the initiation of the recruitment process, we enlisted 191 female participants from MRCCC Siloam Hospitals Semanggi, both onsite and through online sessions via Zoom, to partake in the breast cancer risk prediction study. An additional 141 participants were incorporated from a baseline study, with consent for the utilization of their remaining DNA for further studies concerning breast cancer. Out of 332 participants, there were 158 (47.59%) cases and 174 (52.41%) controls. This dataset was later narrowed down to 314 participants after excluding 18 (5.42%) due to loss to follow-up (n=1), withdrawal (n=1), and failure to meet the inclusion criteria (n=16). This number was further revised to 305 participants after identifying samples that did not meet the quality control criteria for analysis (Fig 2).
The clinical validation study employed a retrospective case/control cohort consisting of female breast cancer patients (cases) and healthy females (controls), primarily recruited from MRCCC Siloam Hospitals Semanggi and other locations. Initially, 158 cases and 174 controls were enlisted. Following triage and biological sample quality control, 149 cases and 156 controls remained, totalling 305 eligible participants. The study maintained a balanced distribution with 48.85% of the participants in the case group and 51.15% in the control group. Demographic analysis revealed a diverse participant pool, encompassing 114 individuals of Indonesian Chinese descent and 191 from other Indonesian backgrounds. The mean ages were 47.86 years (± 8.20) for cases and 44.26 years (± 7.88) for controls.
The final cohort comprised a total of 305 female participants, including 149 individuals (48.85%) diagnosed with breast cancer (cases) and 156 individuals (51.15%) without the condition (controls). Notably, the demographic distribution ultimately included 114 participants of Chinese lineage, consisting of 73 cases and 41 controls, as well as 191 participants of Indonesian heritage, encompassing 76 cases and 115 controls (Table 2).
Accuracy of the Gail clinical risk model in the Indonesian population
First, we evaluated whether the Gail model can be applied successfully to the local population. We first sought to validate our tool by comparing answers from our data analysis pipeline to those from the NIH BRCAT (NIH Breast Cancer Risk Assessment Tool) tool [30]. We relied on a simulated dataset with mock answers, which were manually generated to cover a range of ethnicities and risk outcomes, and utilized Pearson correlation analysis to compare the clinical risk scores from both methods (see Methods). We observed a very strong correlation (Pearson correlation 0.94, p-value 3.38×10-15) between the results of the NIH BRCA tool and the bioinformatics pipeline used in our clinical workflow, supporting the validity of our software to calculate clinical risk.
Next, we evaluated the predictive accuracy of the Gail model by applying it to our patient cohort. We calculated 5-year absolute risk scores for all study participants and compared them between the case and control groups. As expected, the case group exhibited higher scores (Fig 3A), with a mean of 0.76 (±0.44), compared to 0.54 (±0.31) for controls (p-value of 5.16×10-04). Notably, the difference in scores among groups persisted when controlling for age and ethnicity as potential confounding factors in the study (p-value: 4.82×10-02). In addition, we observed an AUC of 0.67 with 95% CI [0.61,0.74] (Fig 3B), which aligns with previously reported outcomes for Western and Asian ancestry populations [15,31–35]. Altogether, these findings support the applicability of the Gail model to Indonesian populations.
(A) Distribution curve of the clinical risk score in cases vs. controls. Higher scores (0.76±0.44) were observed in cases compared than in controls (0.54±0.31), with the difference being statistically significant (p-value: 5.16e-04). (B) ROC curve for the clinical risk score. The observed AUC is 0.674.
Assessment of polygenic risk using the Mavaddat model
We next aimed to assess the genotyping accuracy of our array workflow by evaluating our ability to obtain correct genotype calls at PRS loci. For this assessment, we relied on a set of cell lines with well-established variant truthsets (GIAB) [25,36], which we genotyped in-house. Assessing the accuracy of genotype calls in Mavaddat loci indicated high analytical sensitivity and specificity across all samples tested (99.25±0.46 and 96.89±0.50, respectively; Table 3). To further assess calling accuracy in each of the individual 313 loci of the PRS model, we expanded our evaluation dataset with 19 cell lines from the 1000 Genomes Project [22], which combined with the previous GIAB dataset, covered 306 out of the 313 loci. This analysis showed that 98.69% of sites (302/306) had >95% concordance with the expected calls across all samples, thus demonstrating high per-site accuracy in our workflow (Fig 4).
Heatmap depicting the genotyping concordance for 313 variants of interest associated with breast cancer risk prediction out of 23 samples. Concordance was assessed by comparison to reference calls based on whole-genome sequencing of 1KGP or Genome-In-A-Bottle samples. TP (blue): True positive homozygous alternate call, TP (light blue): True positive heterozygous call, TN (white): true negative, FP (red): false positive, FN (pink): false negative, NC (dark gray): No call in the query sample, NA (light gray): No call in the truth set.
Subsequently, we calculated PRS using PLINK, accounting for ancestry using established methods (see Methods). We compared ancestry-adjusted PRS distributions across cases and controls and detected a significant difference across groups, with higher scores observed in the cases (0.41±0.96 vs. 0.02±0.89; p-value: 1.56×10-03) and an overall AUC of 0.63 (Fig 5). Since our final report includes absolute risk scores instead of the relative risk reported by PRS, we also analyzed the distribution of 5-year absolute risk scores between groups. Notably, the trend previously observed for PRS persisted, with cases exhibiting higher risk scores than controls (0.85±0.41 vs. 0.62±0.33; p-value: 1.86×10-07, Fig 6A). Similar to previous observations for clinical risk, the difference among groups remained significant even after adjusting for age and ethnicity as potential confounding factors (p-value: 6.56×10-04). In addition, we observed an AUC of 0.68 with 95% CI [0.61,0.73] (Fig 6B), which is aligned with ranges previously reported in Western and Asian populations [16,23], indicating that the Mavaddat model can be applied with equivalent predictive accuracy in the Indonesian population.
(A) Distribution curve of adjusted PRS in cases vs. controls. Higher PRS was observed in the case group compared to the controls (0.41±0.96 vs. 0.02±0.89; p-value: 1.56e-03). (B) ROC curve for adjusted PRS. The observed AUC is 0.6262.
(A) Distribution curve of the genetic risk score in cases vs. controls. We observed higher scores in cases than in controls (0.85±0.42 vs. 0.62±0.33; p-value: 6.56e-04). (B) ROC curve for the genetic risk score. The observed AUC is 0.6761.
Last, we sought to compare the genetic risk predictions from our software with those from an established third-party tool by analyzing array results from a subset of samples in our patient cohort (N=32) using both platforms (Table 4). As expected, we noted a significant correlation in PRS results across both tools (Pearson correlation coefficient: 0.95; p-value: 6.11×10–12). However, we identified differences in the approaches each tool uses to interpret PRS into categorical risk outcomes. Our workflow translates PRS into a 5-year absolute risk, incorporating localized disease incidence and mortality rates, and utilizes a 1.7% threshold to distinguish between elevated and average risk. In contrast, the third-party tool categorizes PRS above the 91st percentile as high risk, equating to a ≥20% lifetime disease risk, and relies on a broader population reference to determine a patient’s percentile score. Notably, when comparing the predictive accuracies of the risk categories defined by each platform, we observed higher concordance with phenotypic outcomes in our software compared to the third-party tool (59.38% vs. 37.50%). This finding underscores the importance of considering localized factors when determining categorical risk outcomes from PRS.
Identifying pathogenic mutations in BRCA1/2.
The Illumina GSA chip, which we utilized to genotype samples in our study, encompasses a total of 5168 markers in the BRCA1 and BRCA2 genes, with 2900 of them being annotated as pathogenic/likely pathogenic mutations according to ClinVar [37]. Our analytical pipeline has been developed to report pathogenic mutations in these loci, prioritizing results from direct genotyping and preceding any imputation steps. Specifically, we focus on known pathogenic variants in ClinVar, with a confidence score of 2 or higher (see Methods).
To assess the performance of our workflow, we genotyped two Horizon reference material cell lines, HD793 and HD794, each engineered to contain mutations in BRCA1/2 [38,39]. Each cell line was genotyped in triplicate, and we evaluated our results by comparing the obtained genotype calls to the verified mutations from Horizon, demonstrating 100% analytical sensitivity and specificity across 26 assessed markers (Table 5). After establishing the analytical validity of our workflow, we proceeded to interrogate the presence of pathogenic variants in our study cohort. We detected pathogenic mutations in 14 individuals, all of whom reassuringly belonged to the case group (Table 6).
Performance evaluation of the combined risk model.
In the final stage of our data analysis workflow, we integrate various risk predictions - clinical, polygenic, and monogenic - to determine an overall risk category. Individual risk factors are initially converted into a categorical outcome as follows: 5-year absolute risk scores for clinical and polygenic risk are categorized as either elevated or average using a 1.7% cutoff, while monogenic risk is deemed elevated if a pathogenic variant is detected. Subsequently, a consolidated risk category is established by either selecting the highest risk classification from the three inputs or stratifying patients based on the total number of elevated risk factors identified (e.g., 1, 2, or 3 elevated factors; see Methods).
To assess the accuracy of our combined risk predictions, we first evaluated the impact on accuracy when combining clinical and polygenic risk, as opposed to relying on single-risk factors alone. The analysis yielded an AUC of 0.71 [95% CI: 0.64,0.76] for the two-factor combined risk model, which was higher than that of the single-factor risk models (Fig 7A). Next, we examined the proportion of study participants classified as either average or elevated risk within the case and control groups, taking into account the three-factor consolidated risk categories. Our analysis revealed a higher proportion of elevated risk predictions in the case group compared to the controls (13% vs. 3%, respectively; Fig 7B), a trend that was statistically significant with an OR of 4.56 and a p-value of 1.11×10-03. We further examined the contribution of each risk factor by analyzing the distribution of patients classified as elevated risk due to a single risk factor, a combination of two, or all three. Notably, the majority of elevated-risk samples (22/25) were classified based on a single risk factor. Among the remaining three samples, elevated risk was attributed to two factors, with no samples exhibiting all three factors as elevated (Fig 7C). Finally, we estimated the positive predictive value (PPV) and negative predictive value (NPV) of our assay, with values of 80.00% and 53.92%, respectively.
(A) Comparison of ROC curves between clinical, polygenic and three-factor combined risk (clinical, polygenic, and monogenic). (B) Percentage of samples categorized as average or elevated based on three-factor combined risk (clinical, polygenic and monogenic). (C) UpSet plot showing the number of samples classified as elevated by each of the three factors and their intersections.
Discussion
Early detection of breast cancer significantly enhances patient prognosis and reduces mortality rates, yet current screening strategies predominantly adhere to a one-size-fits-all approach. Notably, not all countries have established population screening programs to promote mammography screening, and even in countries that have, the uptake rate remains low. For instance, in Indonesia, breast cancer screening has not yet been integrated into the national health agenda. Meanwhile, in Singapore, where it has been established, only a mere 28% of primary educated women undergo screening, and 30% of diagnoses occur in women below the suggested screening age [27,40]. Emerging risk-based screening approaches promise better patient stratification and increased screening rates. However, the prevalent ancestry bias in the development of underlying risk models poses a significant challenge for their widespread adoption, particularly for PRS models, which rely on common genetic variants that can be markedly influenced by population-specific allele frequencies. While creating new models based on diverse ancestries is technically feasible, the scarcity of data presents a substantial barrier, making the evaluation of the applicability of existing models to various ancestries a critical task. In this context, our study aims to assess the clinical validity of two established risk prediction models, the Mavaddat and Gail models, for the Indonesian population, which we combine with monogenic risk factors, particularly pathogenic mutations in BRCA1/2 genes. In addition, we propose a personalized breast cancer risk assessment workflow based on array genotyping as a first step toward the implementation of risk-based screening in the region.
To validate the predictive accuracy of the Gail and Mavaddat models in the Indonesian context, we conducted a case-control study involving 305 participants of local descent. Our analysis revealed that the observed AUC of 0.67 for the Gail model aligns with the range documented in published studies [15,31–35]. For genetic risk, we observed an AUC of 0.63 for ancestry-adjusted PRS, indicating a slight regression in performance compared to the original study (AUC=0.64), which utilized samples from the UK Biobank [16]. The AUC, a metric derived from the ROC curve, is commonly used when evaluating the predictive accuracy of a model, and a higher AUC implies that the model has a higher true positive rate and a lower false positive rate, which is crucial in minimizing misclassifications in risk prediction. Altogether, our observations on the AUC of ancestry-adjusted PRS suggest the potential for developing more accurate models in the Asian population, a prospect that could become a reality as larger Asian cohorts become available to identify population-specific variants. Current population-genomic initiatives, such as the Biomedical and Genome Science Initiative (BGSi) in Indonesia, are poised to bridge this gap. It is also important to emphasize that our method of interpreting risk scores into categories relies on absolute risk, not relative risk. By incorporating localized data on disease incidence and mortality from Indonesian statistics, we achieved improved performance in risk scores compared to solely interpreting PRS (AUC of 0.68 for absolute risk scores vs. 0.63 for ancestry-adjusted PRS). This localized approach demonstrates a more significant impact when comparing our risk prediction results with those from a third-party software that relies on a broader population reference to derive risk categories based on PRS percentiles, as demonstrated by the higher concordance with phenotypic outcomes in our software compared to the third-party tool (59.38% vs. 37.50%). Altogether, our observations show, for the first time, the applicability of the Gail and Mavaddat models in the Indonesian context and highlight the potential for further enhancing performance through the creation of new models based on localized datasets.
While the results presented above showcase the performance of both clinical and PRS models individually, they underscores that the best accuracy is achieved when these models are combined (AUC of 0.71 for the combined risk model vs. 0.67 for single-factor models). This finding aligns with previous research, suggesting that a multifaceted approach to risk prediction surpasses the efficacy of single-factor models. For example, research conducted by Hurson et al [41] utilizing multiple datasets of European ancestry illustrated that integrating classical factors with PRS can enhance the detection of individuals at increased risk. Yang et al reported a comparable trend in a dataset encompassing various Asian populations, including Chinese, Japanese, Korean, Thai, Singaporean, and Malaysian groups [42]. Additionally, our analysis reveals that individuals classified as elevated risk are more likely to exhibit a single elevated risk factor rather than multiple factors. This observation highlights the complementarity of the risk evaluations and underscores the potential for enhanced sensitivity when using a combined risk approach. While the integration of various risk factors can enhance predictive capabilities, it also introduces complexity into model development, validation, and interpretation. Our research addresses these challenges and extends previous observations to include data from the Indonesian population, thereby contributing additional demographic variability to the analysis of risk factors across heterogeneous populations. In addition, emerging studies suggest that developing risk models with additional input data can substantially enhance predictive accuracy. In this context, the emergence of advanced models such as BRISK [43], which aggregates mammographic density, polygenic risk, and clinical factors, and BOADICEA [44], which incorporates a comprehensive range of inputs including detailed family history, genetic data, PRS, and lifestyle factors. Thus, there is potential to further enhance the accuracy of models in the Indonesian population through the inclusion of additional datasets.
In addition to evaluating AUCs for risk models, our study also includes an assessment of the genotyping accuracy of our risk prediction workflow. To this end, we utilized GIAB reference materials and focused the performance evaluation on the markers needed for calculating PRS under the Mavaddat model. We determined the analytical sensitivity and specificity of our array genotyping workflow to be 99.25% (±0.46) and 96.89% (±0.5), respectively. Such evaluation is vital in establishing a clinical-grade testing workflow, as it ensures the accuracy of results inputted into the risk calculation software. Most importantly, it should be conducted with the target loci in mind. Given that PRS models encapsulate common variants from numerous genomic loci, which can vary significantly in number and nature between models and considering that different models might encompass a larger representation of SNPs (Single Nucleotide Polymorphisms) or INDELs (insertions/deletions), genome-wide evaluation alone may not necessarily be representative of the performance at selected loci. Additionally, we assessed genotyping accuracy in BRCA1/2 genes using Horizon reference materials, emphasizing that such assessments, while challenging to conduct, should be pursued when possible.
Conclusions
In conclusion, our study underscores the efficacy of the Gail and Mavaddat models in predicting breast cancer risk in the Indonesian population, demonstrating a performance comparable to studies conducted in other demographics. Furthermore, we illustrate that a combined risk model that integrates clinical, polygenic and monogenic risk scores, excels in accuracy compared to single-factor models. The study serves as a preliminary yet promising exploration, as larger sample sizes will be required to fully capture the rich genetic diversity in Indonesia. The emergence of novel datasets and biobanks from population genomics initiatives such as BGSi promises to facilitate further evaluation of risk models and potentially foster the development of new models finely tuned to local populations. Beyond accuracy evaluations, the integration of personalized risk assessment into routine clinical practice requires a broader focus that also encompasses considerations of utility and cost efficiency. While the practical application of enhanced breast cancer risk assessment models in hospital settings remains largely unexplored, pilot studies in other health domains show evidence that it could potentially foster behavioral changes, promoting risk-reducing behaviors among high-risk individuals [45]. This could in turn significantly reduce the incidence of advanced-stage breast cancer cases and the associated healthcare costs [12,44,46]. Moreover, this type of personalized risk assessment could be especially beneficial in regions such as Indonesia, where population-based screening is unavailable and where given the population size, resources are limited, aiding in the identification of individuals who would benefit most from more frequent monitoring. Current global initiatives, such as “Our Future Health” in the UK [47] and the Genomes2Veterans study in the US [48], are leading the way in integrating personalized risk assessment in healthcare settings, thereby setting a precedent for further localized studies to emulate. Although still in its infancy, our collective efforts, in conjunction with others, mark the onset of a transformative era in risk assessment, steering toward a more personalized and proactive approach to healthcare.
Supporting information
S1 File. Clinical risk questionnaire.
This questionnaire was administered to study participants after obtaining their informed consent.
https://doi.org/10.1371/journal.pone.0321545.s001
(PDF)
S2 File. Example of personalized risk report for an individual with average risk.
Patients will be provided with a comprehensive report delineating their average risk of developing breast cancer. This report includes an analysis of both genetic and clinical risk factors and offers guidance on steps that can be taken after the risk assessment.
https://doi.org/10.1371/journal.pone.0321545.s002
(PDF)
S3 File. Example of a personalized risk report for an individual with elevated risk.
Similar to Supplementary File 2, the following report is customized for individuals exhibiting an elevated risk. The interpretation of this report is to be conducted under the guidance of a medical professional.
https://doi.org/10.1371/journal.pone.0321545.s003
(PDF)
S4 File. Inclusivity-in-global-research-questionnaire.
This questionnaire is used to assess inclusivity in global research.
https://doi.org/10.1371/journal.pone.0321545.s004
(DOCX)
Acknowledgments
We would like to thank Dr. Dismas A. Chaspuri, SpB of MRCCC Siloam Hospitals Semanggi for his valuable input and assistance in recruitment by referring his patients to participate in this study. We would also like to thank Ns Ria and Ns Surtati for their assistance during the recruitment process.
Disclaimer*
Images of individuals included in the example report are models specifically incorporated as part of the report design for illustrative purposes. Furthermore, we emphasize that all patient information is entirely fabricated for similar reasons.
References
- 1. Arnold M, Morgan E, Rumgay H, Mafra A, Singh D, Laversanne M, et al. Current and future burden of breast cancer: global statistics for 2020 and 2040. The Breast. 2022;66:15–23. pmid:36084384
- 2. Indo J CancerGautama W. Breast cancer in Indonesia in 2022: 30 years of marching in place. Indo J Cancer. 2022;16(1):1.
- 3. Kemkes. Kanker Payudara Paling Banyak di Indonesia, Kemenkes Targetkan Pemerataan Layanan Kesehatan, https://www.kemkes.go.id/article/view/22020400002/health-care-equality-for-breast-cancer-patients.html (2022, accessed 12 July 2023. )
- 4. American Cancer Society. Understanding a breast cancer diagnosis. Survival Rates for Breast Cancer, https://www.cancer.org/cancer/types/breast-cancer/understanding-a-breast-cancer-diagnosis/breast-cancer-survival-rates.html (2023, accessed 12 July 2023).
- 5. Humphrey LL. Breast cancer screening: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2002;137:347–60. pmid:12204020
- 6. Oeffinger KC, Fontham ETH, Etzioni R, Herzig A, Michaelson JS, Shih Y-CT, et al. Breast cancer screening for women at average risk: 2015 guideline update from the american cancer society. JAMA. 2015;314:1599. pmid:26501536
- 7. Bae J-M, Kim EH. Breast density and risk of breast cancer in asian women: a meta-analysis of observational studies. J Prev Med Public Health 2016; 49:367–375.
- 8. Choridah L, Icanervilia AV, De Wit MJM, et al. Knowledge and acceptance towards mammography as breast cancer screening tool among yogyakarta women and health care providers (mammography screening in Indonesia) J Canc Educ 2021;36:532–537
- 9. Hao L, Kraft P, Berriz GF, Hynes ED, Koch C, Korategere V Kumar P, et al. Development of a clinical polygenic risk score assay and reporting workflow. Nat Med. 2022;28(5):1006–1013. pmid:35437332
- 10. Kim G, Bahl M. Assessing risk of breast cancer: a review of risk prediction models. J Breast Imaging. 2021;3:144–155. pmid:33778488
- 11. Liu J, Ho PJ, Tan THL, Yeoh YS, Chew YJ, Mohamed Riza NK, et al. BREAst screening Tailored for HEr (BREATHE)—A study protocol on personalised risk-based breast cancer screening programme. PLoS ONE. 2022;17:e0265965. pmid:35358246
- 12. Pashayan N, Morris S, Gilbert FJ, Pharoah PDP. Cost-effectiveness and benefit-to-harm ratio of risk-stratified screening for breast cancer: a life-table model. JAMA Oncol. 2018;4:1504. pmid:29978189
- 13. Gail MH, Brinton LA, Byar DP, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. JNCI 1989;81:1879–1886.
- 14. Mehrgou A, Akouchekian M. The importance of BRCA1 and BRCA2 genes mutations in breast cancer development. Med J Islam Repub Iran. 2016;30:369. pmid:27493913
- 15. Ho PJ, Lim EH, Hartman M, Wong FY, Li J. Breast cancer risk stratification using genetic and non-genetic risk assessment tools for 246,142 women in the UK Biobank. Genet Med. 2023;25:100917. pmid:37334786
- 16. Mavaddat N, Michailidou K, Dennis J, Lush M, Fachal L, Lee A, et al. Polygenic risk scores for prediction of breast cancer and breast cancer subtypes. Am J Hum Genet. 2019;104:21–34. pmid:30554720
- 17. Martin AR, Gignoux CR, Walters RK, Wojcik GL, Neale BM, Gravel S, et al. Human demographic history impacts genetic risk prediction across diverse populations. Am J Hum Genet. 2017;100:635–649. pmid:28366442
- 18. Loh P-R, Danecek P, Palamara PF, et al. Reference-based phasing using the Haplotype Reference Consortium panel. Nat Genet 2016;48:1443–1448.
- 19. Das S, Forer L, Schönherr S, et al. Next-generation genotype imputation service and methods. Nat Genet 2016;48:1284–1287.
- 20. Chang CC, Chow CC, Tellier LC, Vattikuti S, Purcell SM, Lee JJ. Second-generation PLINK: rising to the challenge of larger and richer datasets. Gigascience. 2015;4:7. pmid:25722852
- 21. Karczewski KJ, Francioli LC, Tiao G, et al. The mutational constraint spectrum quantified from variation in 141,456 humans. Nature 2020;581:434–443
- 22. The 1000 Genomes Project Consortium, Corresponding authors, Auton A, et al. A global reference for human genetic variation. Nature 2015;526:68–74.
- 23. Ho W-K, Tan M-M, Mavaddat N, Tai M-C, Mariapun S, Li J, et al. European polygenic risk score for prediction of breast cancer shows similar performance in Asian women. Nat Commun. 2020;11:3833. pmid:32737321
- 24. Nirvana: Clinical-grade varian annotation, https://illumina.github.io/NirvanaDocumentation/ (accessed 9 October 2023).
- 25. Zook JM, McDaniel J, Olson ND, et al. An open resource for accurately benchmarking small variant and reference calls. Nat Biotechnol 2019; 37: 561–566.
- 26.
Seabold S, Perktold J. statsmodels: Econometric and statistical modeling with python. 2010.
- 27. Teo CT, Yeo YWS, Lee S-C. Screening mammography behavior and barriers in Singaporean Asian women. Am J Hlth Behav. 2013;37(5):667–682. pmid:23985290
- 28. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77. pmid:21414208
- 29. Aragon TJ, Fay MP, Wollschlaeger D. epitools: Epidemiology Tools, https://cran.r-project.org/web/packages/epitools/epitools.pdf (2020, accessed 13 June 2023).
- 30. Gail M, Zhang F. National Cancer Institute. BCRA R Package, https://dceg.cancer.gov/tools/risk-assessment/bcra (2019, accessed 10 July 2023).
- 31. Chay WY, Ong WS, Tan PH, Jie Leo NQ, Ho GH, Wong CS, et al. Validation of the Gail model for predicting individual breast cancer risk in a prospective nationwide study of 28,104 Singapore women. Breast Cancer Res. 2012;14:R19. pmid:22289271
- 32. Costantino JP, Gail MH, Pee D, Anderson S, Redmond CK, Benichou J, et al. Validation studies for models projecting the risk of invasive and total breast cancer incidence. J Natl Cancer Inst. 1999;91:1541–1548. pmid:10491430
- 33. Gao F, Machin D, Chow K-Y, Sim Y-F, Duffy SW, Matchar DB, et al. Assessing risk of breast cancer in an ethnically South-East Asia population (results of a multiple ethnic groups study). BMC Cancer. 2012;12:529. pmid:23164155
- 34. Wang X, Huang Y, Li L, Dai H, Song F, Chen K. Assessment of performance of the Gail model for predicting breast cancer risk: a systematic review and meta-analysis with trial sequential analysis. Breast Cancer Res. 2018;20:18. pmid:29534738
- 35. Zhang L, Jie Z, Xu S, Guo X. Use of Receiver Operating Characteristic (ROC) curve analysis for tyrer-cuzick and gail in breast cancer screening in jiangxi province, China. Med Sci Monit. 2018;24:5528–5532. pmid:30089770
- 36. Index of Reference Samples GIAB Release, https://ftp-trace.ncbi.nlm.nih.gov/ReferenceSamples/giab/release/.
- 37. Landrum MJ, Lee JM, Benson M, Chao C, Chitipiralla S, et al. ClinVar: improving access to variant interpretations and supporting evidence. Nucleic Acids Res. 2018;46:D1062–D1067. pmid:29165669
- 38. BRCA Germline I (gDNA), https://horizondiscovery.com/en/reference-standards/products/brca-germline-i-gdna (accessed 9 October 2023).
- 39. BRCA Germline II (gDNA), https://horizondiscovery.com/en/reference-standards/products/brca-germline-ii-gdna (accessed 9 October 2023).
- 40. Meng TW. Screening Rate for Breast Cancer, https://www.moh.gov.sg/news-highlights/details/screening-rate-for-breast-cancer#:~:text=5th%20Oct%202020&text=However%2C%20while%20the%20breast%20cancer,with%20primary%20level%20of%20education. (2020).
- 41. Hurson AN, Pal Choudhury P, Gao C, Hüsing A, Eriksson M, Shi M, et al. Prospective evaluation of a breast-cancer risk model integrating classical risk factors and polygenic risk in 15 cohorts from six countries. Int J Epidemiol. 2022;50(6):1897–1911. pmid:34999890
- 42. Yang Y, Tao R, Shu X, Cai Q, Wen W, Gu K, et al. Incorporating polygenic risk scores and nongenetic risk factors for breast cancer risk prediction among Asian women. JAMA Netw Open. 2022;5:e2149030. pmid:35311964
- 43. Allman R, Mu Y, Dite GS, Spaeth E, Hopper JL, Rosner BA. Validation of a breast cancer risk prediction model based on the key risk factors: family history, mammographic density and polygenic risk. Breast Cancer Res Treat. 2023;198:335–347. pmid:36749458
- 44. Lee A, Mavaddat N, Wilcox AN, et al. BOADICEA: a comprehensive breast cancer risk prediction model incorporating genetic and nongenetic risk factors. Genet Med. 2019;21:1708–1718. pmid:30643217
- 45. Widén E, Junna N, Ruotsalainen S, et al. How communicating polygenic and clinical risk for atherosclerotic cardiovascular disease impacts health behavior: an observational follow-up study. Circ Genom Precis Med. 2022;15:e003459. pmid:35130028
- 46. Wong JZY, Chai JH, Yeoh YS, Mohamed Riza NK, Liu J, Teo Y-Y, et al. Cost effectiveness analysis of a polygenic risk tailored breast cancer screening programme in Singapore. BMC Health Serv Res. 2021;21:379. pmid:33892705
- 47. Our Future Health. Our Future Health. Our Future Health: Let’s prevent disease together, https://ourfuturehealth.org.uk/ (accessed 12 July 2023).
- 48. Genomes2Veterans. Genomes2Veterans (G2V) Research Program. Genomes2Veterans (G2V) Research Program, https://www.genomes2people.org/research/genomes2veterans/ (accessed 12 July 2023).