https://orcid.org/0000-0002-7368-498X
Figures
Abstract
Background
Breast cancer is the leading cause of cancer-related deaths among women in Mexico, and the outcomes for patients are often poor. Futility in healthcare (FHC) occurs when treatment fails to achieve its intended goals, resulting in negative consequences. This study aimed to estimate the proportion of female patients in Mexico with advanced-stage breast cancer (ABC) who perceive FHC. Additionally, we inform the development and validation of a questionnaire (FHC-Q) to assess this phenomenon.
Patients and methods
This cross-sectional study was conducted from May 17, 2024, to December 30, 2024, in three phases. It involved three convenience samples (S) of adult women with ABC: S-1 (n = 30), S-2 (n = 201), and S-3 (n = 257). Phase 1 focused on constructing the FHC-Q, evaluating its content validity (expert agreement), and conducting a pilot test to assess its feasibility, all within S-1. Phase 2 involved assessing the questionnaire’s reliability (internal consistency and temporal stability), construct validity (correlations between FHC-Q and DASS21 scores, along with exploratory factor analysis), and criterion validity (correlations between FHC-Q and SF-36 scores), all in S-2. Phase 3 estimated the FHC phenomenon in S-3. The definition of FHC was established using the Delphi method.
Results
Participants represented typical female Mexican patients with ABC. The FHC-Q comprised 16 items distributed into five dimensions. The FHC-Q demonstrated feasibility and validity, with expert agreement of ≥80% and a five-factor structure explaining 57.8% of the variance. Correlations between the FHC-Q, DASS21 and SF-36 scores were significant and low to moderate. The FHC-Q reliability was confirmed with a Cronbach’s alpha of 0.766 and an ICC of 0.845 (95% CI: 0.735–0.909).
FHC was perceived by 3.9% of the participants
Citation: Namba-Bando MdMY, Contreras-Yáñez I, Tenorio-Torres JA, Pascual-Ramos V (2025) Futility in healthcare among Mexican female patients with breast cancer in advanced stage: The patient perspective. PLoS One 20(6): e0326332. https://doi.org/10.1371/journal.pone.0326332
Editor: Xiong Xingyu, Sichuan University West China Medical Center, CHINA
Received: January 24, 2025; Accepted: May 29, 2025; Published: June 23, 2025
Copyright: © 2025 Namba-Bando et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because of concerns about patient identification and the sensitivity of the information. Data are available from the Chair of the Research Ethics Committee Dr. Germán Carreto Chávez, at cei.fucam@gmail.com for researchers who meet the criteria for access to confidential data.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Breast cancer is the leading cause of cancer-related deaths among women globally and in Latin America (LATAM) [1–3]. Factors such as advanced stage at diagnosis [3,4], high tumor grade [4], negative hormone receptors [4], HER-2 amplification [4], tumor subtype [3], and age [3] are linked to higher mortality rates. Additionally, socioeconomic and cultural factors significantly influence survival, affecting access to screening, timely diagnosis, and effective treatments [5–7]. This contributes to the higher mortality of breast cancer in LATAM compared to developed countries [8,9].
The cancer mortality rate in Mexico increased from 58 per 100,000 residents in 1998–67 in 2008 [2]. Since 2006, breast cancer has been the leading cause of cancer deaths among Mexican women, representing 9% of fatalities in 2023 [2,10,11]. By 2030, an estimated 38,420 women will be diagnosed, with 11,781 (30.7%) expected to die from the disease in LATAM, posing a significant challenge for healthcare systems [12]. Factors contributing to poor outcomes include advanced-stage diagnoses [3], a high prevalence of triple-negative breast cancer [3], unique gene expression patterns [3], tumor biology [3], delays in diagnostic procedures [5], and social determinants of health [13,14].
Futility in healthcare (FHC) is a phenomenon recognized across various languages, disciplines, and cultures [15], with roots tracing back to the time of Plato [16]. Although many authors have attempted to define it in clinical contexts, a widely accepted bioethical definition remains elusive [16–21]. The determination of what constitutes meaningful versus futile treatment is inherently subjective, influenced by individual values, beliefs, and experiences [15]. Generally, treatment is deemed futile when it fails to meet its intended goals. However, defining these goals is often challenging within the patient-centered care model, where patients are active participants in their treatment decisions [22,23]. Morata et al. [15] conducted an evolutionary concept analysis to explore the historical development of the FHC concept and its potential future evolution alongside technological and systemic changes in healthcare [24]. They propose a consensus definition for FHC as “complex patient conditions characterized by interventions or procedures that do not achieve meaningful recovery of the primary ailment, based on the patient’s and multidisciplinary team’s healthcare goals, while often maintaining a latent sense of hope.” The consequences of FHC are complex and typically negative, encompassing increased healthcare costs, potential resource rationing, legal challenges, and ethical dilemmas or moral distress [15].
To effectively address the public health challenge of breast cancer, comprehensive research is necessary across clinical, epidemiological, health systems, translational, and bioethical domains [2]. Such research is crucial for identifying and overcoming the specific challenges faced by different countries while also utilizing successful strategies from other regions. The World Health Organization has noted a significant research gap in LATAM countries, primarily due to insufficient funding and support for researchers, emphasizing the need for targeted interventions [25].
Currently, there is no valid, reliable, or objective method to accurately identify or measure FHC from the patient perspective, either quantitatively or qualitatively. This study aims to develop and validate a comprehensive questionnaire to assess FHC (FHC-Q) among female Mexican patients with advanced-stage breast cancer. Additionally, it will estimate the prevalence of this phenomenon in the target population, which represents our primary objective.
Patients and methods
Ethical considerations
The study was performed in compliance with the Helsinki Declaration [26]. The Research Ethics Committee of the Fundación para Cáncer de Mama (Breast Cancer Foundation, FUCAM A.C.) approved the study (Reference number: CEI/PI 24–01/2024). All participants provided written informed consent to join the study.
Study setting, population, and duration
FUCAM A.C. is a non-profit organization in Mexico and LATAM that specializes in breast cancer treatment and monitoring. Since 2005, it has treated over 11,000 women and provided a comprehensive approach that includes early detection, diagnosis, medical and surgical treatment, breast reconstruction, and psychological support [27]. According to its 2022 report, FUCAM conducted 57,089 mammograms, diagnosed 1,297 new breast cancer cases, and performed 72,381 medical consultations [28].
The study was conducted at FUCAM’s high specialty hospital unit in Mexico City from May 17, 2024, to December 30, 2024. Adult women with confirmed advanced-stage breast cancer [29] were invited to participate if they had received treatment at FUCAM or were attending the palliative care unit (inclusion criteria). Advanced-stage breast cancer was defined based on the following three criteria: 1. Clinical Stage IIB with T3 N0 and beyond; 2. Pathologically advanced stage: This refers to cases that may clinically appear to be stage I or II, but reveal more aggressive features upon pathological examination, such as microscopic lymph node involvement or vascular invasion; and 3. Evidence of metastatic disease: This involves cases where the cancer has spread to distant organs, irrespective of the initial clinical or pathological stages.
Patients were approached in waiting areas for consultations related to pain management, psycho-oncology, rehabilitation, nutrition, and various oncology specialties. Those with severe cognitive impairment were excluded.
Study design
The study utilized a cross-sectional design with three phases and is reported in accordance with the STROBE guidelines (see “Appendix 1. STROBE checklist”). Phase 1 involved the construction of the FHC-Q, which included item generation, pilot testing, and feasibility evaluation. Phase 2 assessed the reliability and validity of the FHC-Q. Phase 3 evaluated the prevalence of FHC in the target population according to the FHC-Q (see Appendix 2). We followed established guidelines for developing health measurement scales when no suitable instruments existed [30].
Description of samples and sample size calculation
This study analyzed three samples (S) of consecutive female outpatients with confirmed advanced-stage breast cancer. During Phases 1 and 2, two samples were used [31,32]: S-1, which included at least 30 participants for pilot testing, and S-2, comprising at least 200 participants for the validation of the FHC-Q. S-3 consisted of 257 participants and was utilized in Phase 3 to meet the primary objective.
We calculated the necessary sample size to detect FHC in the target population based on prevalence rates of 4%, 12%, and 25% for patients meeting three, two, and one criterion of therapeutic aggressiveness, respectively, as reported by Barón-Duarte et al. [33]. Additionally, the International Agency for Research on Cancer reported a 5-year prevalence of breast cancer in Mexico in 2020, with 99,288 cases, of which 66.4% were classified as advanced stages [34]. This results in an estimated 65,927 cases of advanced-stage breast cancer in Mexico. The final required sample size ranged from 59 to 287 patients, with a 95% confidence level and 5% precision, according to the three prevalence rates mentioned. This final sample size allowed us to achieve a confidence level of 95% and a precision of 5% for the primary objective.
Non-probabilistic, intentional sampling was employed, including consecutive women outpatients diagnosed with advanced-stage breast cancer [32].
Procedures
Phase 1- Construction of the conceptual model of the FHC-Q, instructions and items generation, pilot testing, scaling responses, and questionnaire feasibility.
Literature review: A literature review confirmed that there is no valid and reliable tool for assessing FHC [17]. Some prognostic tools have been proposed, such as the Injury Severity Score for trauma patients and the Acute Physiology and Chronic Health Evaluation score for critically ill adults, but these have had limited success [15,35,36]. These tools often lead to false positives and lack the statistical confidence needed for effective care, resulting in their limited adoption [37–39]. Currently, Earle’s aggressiveness criteria [40] are viewed as the gold standard for evaluating therapeutic aggressiveness in oncology for end-of-life care at the institutional level [33]; however, they do not consider the patient perspective. These criteria focus on three areas: the overuse of chemotherapy near death, inappropriate use of devices in the emergency department and ICU, and insufficient or delayed access to hospice and palliative care programs [33]. Accordingly, we decided to develop a new instrument.
Instructions and items generation: Two types of sources were used for instruction and item generation: theoretical literature [15–21] and suggestions from a coauthor with expertise in research, bioethics, and clinical involvement at FUCAM. The initial version of the FHC-Q (v1) was drafted and reviewed by four professionals: a bioethicist, a psycho-oncologist, a surgical oncologist, and a social worker. Their feedback informed the development of a second version (v2), which also included an item for participants to select their preferred term among three options related to the evaluated phenomenon: futility, obstinacy, and lack of therapeutic adequacy.
Version 2 (v2) was tested with eleven patients in four formats, allowing them to choose their preferred presentation style, either as questions or statements with corresponding response scales. The approved format of v2 was then reviewed by an expert committee of 14 members, including surgical oncologists (2), radiation oncologists (2), palliative care specialists (2), medical oncologists (4), a psychologist, a psychiatrist, and bioethicists (2). The committee evaluated the instructions and items based on clarity, appropriateness for the target population, and absence of emotional bias (content validity). Any poorly evaluated instructions or items were revised, leading to the creation of a third version (v3).
Pilot testing: Version 3 (v3) was pilot tested with 30 outpatients from the target population. Participants assessed the clarity of the instructions and items to evaluate face validity, as well as the feasibility of the questionnaire, which included its format, the time required for completion, and the patients’ willingness to fill out the survey. Based on the pilot results, version 4 (v4) was deemed suitable for the validation process (Phase 2).
Scale response: Two types of response scales were provided to match the item formats presented. The response scale could be displayed either as frequency or as a level of agreement, using a 5-point Likert scale in both cases.
Phase 2. FHC questionnaire validity and reliability
Construct validity was established through convergent validity by examining the correlations between the FHC-Q and the Depression, Anxiety, and Stress Scale-21 (DASS21) scores [41]. Additionally, factor analysis was conducted for further verification [42].
Content validity was evaluated by a committee of experts selected for their experience, reputation, availability, and impartiality [43].
Criterion validity was assessed by correlating FHC-Q scores with those from the 36-item Short Form Survey (SF-36) [44], a validated tool for measuring health-related quality of life (QoL) across diverse populations. The SF-36 encompasses eight scales and two distinct dimensions: the Physical Component Summary (PCS) and the Mental Component Summary (MCS). Scores range from 0 to 100, with higher scores indicating better QoL. Furthermore, MCS and PCS scores were compared between patients who had FHC and those who did not.
Reliability was assessed via internal consistency and temporal stability (test-retest) by administering the questionnaire to 58 patients at baseline and two weeks later [42].
Step 3. FHC prevalence.
A modified Delphi exercise [45] was proposed to establish criteria for the prevalence of FHC, based on the results of the FHC-Q. A panel of ten experts was assembled, including two specialists in palliative care, one surgical oncologist, two psycho-oncologists, two bioethicists, and one social worker. The experts participated in three rounds of questions, focusing on ranking the dimensions of the FHC-Q according to their importance for the construct (ordered from first to fifth) and determining their essentiality in addressing the construct (Yes/No).
The FHC-Q was administered alongside standardized assessments to collect sociodemographic (Table 1), disease-related (Table 2), and cancer-related (Table 3) variables. Data were obtained from patients and verified through chart reviews.
Sociodemographic variables included sex, age, education level, civil status, occupation, religious beliefs, and socioeconomic status. Disease-related characteristics encompassed disease duration, performance status measured by the Eastern Cooperative Oncology Group (ECOG) scale [46], comorbidities, and treatment-related factors. Mental health was assessed using the DASS21 [41], while functional status was evaluated with the Health Assessment Questionnaire Disability Index (HAQ-DI) [47]. Quality of life (QoL) was measured with the SF-36 questionnaire [44], and family functioning was assessed using the Adaptation, Partnership, Growth, Affection, and Resolve index (APGAR) [48].
Cancer treatment-related data included treatment options, the number of treatment modalities per patient, chemotherapy type, lines of therapy (LOT), and instances of reirradiation.
Finally, cancer-related variables included recurrence status, histological type [49], TNM staging [29], and molecular subtype [50].
Statistical analysis
The FHC score was computed by summing the individual item scores, and the final score ranged from a minimum of 16 to a maximum of 80. This calculation involved recoding items 6, 7, 10, 12, 13, and 15. A lower score reflects a greater patient perception of FHC. The decision to sum item scores aligns with standard practices in questionnaire scoring [51]. Additionally, for the assessment of FHC, we propose scoring the five dimensions of the FHC-Q. It is essential to ensure that at least one item from each dimension has a dichotomous response. This requirement guarantees a thorough evaluation of the FHC construct. By adopting this strategy, we can obtain a numerical representation of the patient’s perceived FHC, facilitating the routine care monitoring of any potential changes over time and the share-decision making process.
Descriptive statistics characterized the patient variables across three samples, with categorical variables reported as frequencies and percentages, and continuous variables summarized using medians and interquartile ranges (Q25-Q75). Statistical comparisons between groups employed the Chi-square (X²) test for categorical variables and the Mann-Whitney U test for continuous variables.
Convergent validity was analyzed using the Spearman rank correlation coefficient (rho) [30]. Construct validity was assessed via exploratory factor analysis (principal components) with Varimax rotation, extracting five factors. Sampling adequacy was confirmed with the Kaiser-Meyer-Olkin (KMO) measure (acceptable value ≥0.5), supported by Bartlett’s test of sphericity (p < 0.05) [52].
Content validity was examined with agreement percentages, and Lawshe/Tristan’s content validity ratio was calculated for individual items and the FHC-Q (mean of individuals’ content validity ratios) [53].
Criterion validity was evaluated using the Spearman rank correlation coefficient (rho) between the FHC-Q and the PCS and MCS scores from the SF-36 [30], as well as by comparing PCS and MCS scores between patients with and without FHC.
Internal consistency of the FHC-Q was assessed using Cronbach’s alpha (α). Temporal stability was determined via intra-class correlation coefficients (ICC) and 95% confidence intervals (CI), utilizing a single measurement, absolute-agreement, 2-way mixed-effects model. Interpretations of Cronbach’s α, ICC, and 95% CI followed established recommendations [54]. Floor and ceiling effects were calculated as the percentage of patients attaining the lowest and highest possible scores.
There were no missing data for the primary outcome; however, the missing data varied from 0% to 23.3% for patient-reported outcome measures (PROMs) in S-1. No imputation was performed.
All statistical analyses were conducted using the Statistical Package for the Social Sciences version 21.0 (SPSS, Chicago, IL), with statistical significance set at p < 0.05.
Results
Description of the participants included in the three samples and their breast cancer characteristics
Table 1 summarizes the sociodemographic characteristics of the patients, Table 2 presents the disease-related characteristics and patients’ treatment, and Table 3 details the breast cancer characteristics from the patients included in the three samples. Participants were primarily middle-aged, with a median of 12 years of formal education. Most were housewives, married or cohabiting, with self-reported religious beliefs. A substantial proportion reported a family income of less than $250 per month (equivalent) (Table 1).
Participants in the study had a short duration of illness, with the majority having an ECOG performance status of 1. A significant number of participants also had comorbid conditions, including current mental health disorders of at least moderate severity. While patients’ functional capabilities and overall quality of life were impacted, most reported that their family functioning remained intact. The treatment intention for most patients was curative, and nearly all participants underwent chemotherapy. Details regarding the treatment options for each patient, as well as the types of chemotherapy and radiotherapy used, are summarized in Table 2.
Data presented as median (Q25-Q75) as otherwise indicated. *Number (%) of patients. ¹Among those with the condition. ECOG = Eastern Cooperative Oncology Group. HAQ-DI = Health Assessment Questionnaire Disability Index. SF-36 = 36-items short form survey: PCS (Physical Component Summary) and MCS (Mental Component Summary). Family APGAR scale = Adaptation, Partnership, Growth, Affection, and Resolve. FHC = Futility in health care. MD = Missing data. ²There were 7 MD and 17 MD, respectively, but for FHC (no MD).
The predominant histological subtype of breast cancer was invasive ductal carcinoma. Tumors were predominantly classified at stage III according to the TNM staging system and exhibited positive hormone receptor (HR+) status, along with HER2 positivity (Table 3).
Phase 1- Construction of the conceptual model of the FHC-Q, instructions and items generation, pilot testing and scaling responses, and questionnaire feasibility
The first version of the FHC questionnaire included 47 items distributed among five dimensions. Fig 1 outlines these five dimensions (extraordinary measures, disproportionate measures, unnecessary prolongation of life, negative impact on the quality of life, and respect for patient autonomy) and their indicators.
The v2 of the FHC questionnaire included 16 items and resulted from a consensus achieved by 100% agreement among the four reviewers involved regarding which items should remain (S1 Table).
The majority of patients (n = 6 [58%]) opted for a version that included statements instead of questions and a frequency scale response based on a 5-point Likert scale. We chose a direct estimation method for the responses, where the scale was defined as follows: 1 = Always, 2 = Almost always, 3 = Sometimes, 4 = Almost never, and 5 = Never.
The results from the multidisciplinary expert committee indicated that there was at least 80% agreement regarding the clarity of the instructions and wording of the items, the appropriateness of the language for the target population, and the absence of emotional bias in the items. However, four members recommended minor changes to item 4 (for better differentiation from item 3) and item 16 (for better differentiation from item 14) to enhance their clarity, and all of these suggestions were adopted (S1 Table). Lawshe/Tristan’s content validity ratio for individual items was ≥ 0.6 and 0.93 for the FHC-Q. The results confirmed content validity.
Results from the pilot testing indicated that overall, patients generally agreed on the clarity of the instructions and items, with over 80% agreement. However, item 2 achieved only 73% agreement regarding its clarity. Some patients found the term “non-financial resources” unclear, although no specific proposals for improvement were suggested. Consequently, the item was updated to include examples.
Most patients (90%) agreed on the FHC-Q format, finding the time required to complete it convenient and expressing a willingness to do so. On average, patients took 18 minutes to complete the questionnaire.
Additionally, 82.4% of the patients and the physicians (14 specialists) directed to provide an answer opted for the term “therapeutic obstinacy” to describe the phenomenon evaluated.
Phase 2. FHC-Q validity and reliability
Construct validity.
Table 4 summarizes Spearman rank correlation coefficients (rho) between the FHC-Q score and specific dimensions scores of the DASS21: depression, anxiety, and stress. Overall, correlations were significant, but their strengths were low.
The 16 items were distributed into five domains that were renamed as follows: “Extraordinary Measures,” “Adequate/Proportionate Measures” (along with the conditions behind them), “Autonomy and Beneficence,” “Impact on Quality of Life,” and “Prolongation of Life.” The KMO measure was 0.735, and we observed a significant result (X² = 772.549, p ≤ 0.0001) for the Bartlett’s sphericity test. The 5-factor structure accounted for 57.8% of the variance (S2 Table). The structure of the FHC was slightly modified after factorial analysis (Fig 2).
Criterion validity.
Table 5 presents the Spearman rank correlation coefficients (rho) between the global FHC-Q score, its dimensions, and the MCS and PCS scores from the SF-36. Overall, the correlations were significant, except for the “Prolongation of Life” dimension of the FHC-Q. Additionally, higher correlations were observed between the FHC-Q and its dimension scores and the MCS of the SF-36 compared to the PCS. The correlation strength was moderate for the “Impact on Quality of Life” dimension of the FHC-Q, while the correlations with the other dimensions were weak. Finally, the MHC and PHC scores were significantly lower in patients with FHC compared to those without: 40.6 (33.3–44) vs. 62.5 (49.6–74.4), p = 0.001 and 45.8 (24.9–48.9) vs. 59.9 (48.3–77.2), p = 0.001, respectively.
Internal consistency and reliability.
Table 6 presents a summary of the internal consistency results (Cronbach’s α, ICC [95% CI]) for the FHC-Q and each of its domains, along with the floor and ceiling effects. The Cronbach’s α value for the FHC-Q was 0.766, while the ICC and its 95% Cl values were 0.845 (0.735–0.909). The mean (±SD) time between the two measurements in the test-retest analysis was 20 days (5.55).
Phase 3. FHC prevalence
Delphi exercise results.
Experts reached a consensus on the importance of various factors related to the FHC construct. “Adequate/proportionate measures” ranked first, with 90% agreement among experts, followed by the “(negative) impact on quality of life,” (100% agreement), “(unnecessary) prolongation of life” (70%), “extraordinary measures” (60%), and “autonomy and beneficence” (50% agreement). Additionally, experts identified several key dimensions essential for defining FHC. These include “adequate/proportionate measures” (80% agreement), “(negative) impact on quality of life” (90% agreement), and “(unnecessary) prolongation of life” (80% agreement).
FHC definition.
We determined that for a patient to be defined as with FHC, he or she must endorse at least four distinct dimensions from the FHC-Q. This includes all key dimensions previously identified. Furthermore, for each of these four dimensions, the patient must select at least one item response as “Always” or “Almost always.”
Discussion
Considerations on FHC definition
FHC is fundamentally a subjective judgment but remains an essential aspect of everyday clinical practice, with ethical and legal consequences [55]. Achieving a fully objective, concrete, and universal definition of FHC seems unattainable within this framework [56]. Various efforts have been made to define FHC using different approaches, including qualitative and quantitative methods [20,40,57], requiring physicians and patients (or their proxies) to refuse interventions intended to prolong life [56], and more controversial methods based on community [58,59] or institutional [56] standards. However, enforcing an objective definition of FHC could lead to situations where some patients receive interventions or face death due to judgments with which they disagree [56]. In this study, we adopted the approach outlined by Aghabarary et al. [60], which suggests that medical futility should be defined and assessed on an individual level, taking into account the unique circumstances of each case. We integrated this idea into a patient-centered care framework [23], as we endorse that a patient’s perspective and experience cannot be substituted for clinical observations and examinations conducted by specialists [61]. To this end, we developed a questionnaire to evaluate patients’ perceptions of FHC in end-of-life care, complementing existing scientific efforts that capture physicians’ perspectives [62].
Study strengths
We observed that 3.9% of Mexican women with advanced-stage breast cancer reported experiencing FHC. To address this, we developed and validated the FHC-Q specifically for this population. The creation of the questionnaire followed standardized test-construction methods. The final version demonstrated adequate psychometric properties, including construct, content, and criterion validity, as well as reliability, which was assessed through internal consistency and test-retest evaluations, as recommended [30,52,63]. Patient evaluations indicated that the questionnaire was feasible for use, particularly among individuals with low literacy levels. This suggests that it can be easily applied to patients across the LATAM region and beyond. Patients were also directly involved in establishing the face validity of the questionnaire. While clinicians may have better insights into certain aspects of cancer, such as disease classification, only patients can accurately report on the more subjective elements of their experience [64].
FHC prevalence
The prevalence rate of FHC in our study is comparable to that reported by Barón-Duarte et al. [33]. In their analysis of therapeutic aggressiveness based on Earle’s criteria, they studied 1,001 patients with advanced cancer who died between 2010 and 2013. The authors found that approximately 25% of the patients met at least one criterion for aggressiveness, while 4% of the patients met all three criteria. However, healthcare specialists have also noted higher rates of FHC in end-of-life care, extending beyond cancer patients [62]. Earle et al. [65] examined Medicare claims for 28,777 patients aged 65 and older who died within a year of a cancer diagnosis between 1993 and 1996. They found that 18.5% were still receiving treatment within two weeks of death, and 9.1% to 9.4% experienced multiple emergency visits or hospitalizations in their final month. In a 1993 study by Solomon et al. [66], 47% of 1,446 healthcare providers across five U.S. hospitals reported acting against their conscience in end-of-life care, with 55% feeling treatments were sometimes overly burdensome. A 2011 survey by Sirovich et al. [67] of 627 U.S. primary care physicians found that 42% believed their patients were overtreated. In a neurological intensive care unit study, Amoroso et al. [68] found that one-third of patients were considered for FHC, increasing to 50% when including loved ones. Chamberlain et al. [69] surveyed 349 physicians from two New York City academic centers, revealing that 91.3% had provided what they considered futile care in the past six months. Variations in prevalence rates among the studies, including ours, may be attributed to differences in how FHC is defined, the years of publication, and the characteristics of the participants. Notably, our study uniquely defines FHC based on patients’ perceptions. Additionally, the patients in our study had a short disease duration, despite being in an advanced stage of the disease. This may significantly shape local patients’ perceptions, particularly because individuals from lower socioeconomic backgrounds frequently encounter obstacles in accessing the Mexican healthcare system and may view any available treatment as a privilege [2,3].
FHC-Q psychometric properties
The FHC-Q was developed using a rigorous methodological approach. The construct was validated through a literature review [60], a report from the American Medical Association [56], and scientific publications on FHC concept analysis [15,70–73]. Input from healthcare professionals and patients with advanced-stage breast cancer was integral to reframing the construct, ensuring that the questionnaire reflected patient experiences. Content validity was enhanced by involving representative populations during questionnaire development. Item wording underwent optimization through face validity testing with multidisciplinary groups of healthcare experts and patients. The clarity of instructions and the final questionnaire were evaluated by patients in a pilot test before formal validation, leading to essential adjustments that improved patient acceptance and minimized data collection inaccuracies. Three patient samples were utilized, representing typical outpatients from Mexico with advanced-stage breast cancer. The FHC-Q demonstrated adequate internal consistency, with a Cronbach’s α coefficient for the total scale deemed acceptable. Additionally, test-retest reliability assessed in 58 patients by the same researcher showed an ICC with a 95% CI, indicating good reliability [54]. The construct validity was confirmed through KMO sampling and Bartlett’s test of sphericity, both of which validated the sample size’s adequacy for conducting factor analysis [52]. Moreover, the FHC-Q did not show any floor or ceiling effects, defined as occurring when more than 15% of patients score at the lowest or highest possible levels. The presence of floor and ceiling effects reduces the range of data, limiting its variability. This compression can hinder the instrument’s ability to detect differences or changes, as well as impact its reliability and validity. Consequently, it may lead to biased results and incorrect conclusions [30].
FHC-Q structure
A five-factor model was identified as the most appropriate for the questionnaire, and these five factors together accounted for 57.8% of the total variance. The structure of the FHC-Q underwent modifications from the initial conceptual model. The dimension of “Adequate/Proportionate Measures” was expanded to include three items that were previously categorized under the “Autonomy (and Beneficence)” dimension. Similarly, the “Impact on Quality of Life” dimension was enriched with one additional item. This change may reflect the increasing significance of autonomy as a central principle in the current patient-centered model of care, which permeates other dimensions [74].
The shift towards patient-centered care began with the recognition of patients’ fundamental rights to self-determination and informed consent prior to any intervention. This recognition is now acknowledged in the ethical field as the principle of autonomy. In their book “Principles of Biomedical Ethics,” Beauchamp and Childress [75] establish a framework for medical practice that articulates both utilitarian and deontological principles. They propose a non-hierarchical approach based on four principles: autonomy, beneficence, non-maleficence, and justice, all of which are viewed as absolute obligations. This framework has significantly influenced ethical standards in the US and LATAM [76]. However, Western societies have often overemphasized the principle of autonomy over the other principles [77,78], contributing to the ethical lattice upon which physicians base their decisions.
Rational for the questionnaire scoring system and the FHC prevalence
Finally, we propose a scoring system for the questionnaire that includes a total score and specific criteria to establish perceived FHC, ensuring its applicability in routine clinical practice. To develop this system, we utilized the Delphi technique, which has proven effective in addressing various critical issues across different health professions. The combination of anonymity, iterative feedback, controlled responses, and the statistical aggregation of group opinions makes this method particularly suitable for tackling emerging and less understood topics, as well as predicting important issues in medicine [79]. Moreover, as a consensus technique based on expert opinions, the Delphi method has the potential to lead to one of the first peer-reviewed publications in this new field, ultimately influencing a substantial body of literature [79].
Study limitations
Several limitations need to be addressed. First, the FHC-Q was defined solely from the patient’s perspective, without considering the opinions of physicians. It has been suggested that FHC should be approached as a shared decision-making process [33], and the provision of futile care should not only be viewed as an individual patient decision but also in a broader context [80]. Furthermore, while the proposed scoring system is easy to use in a busy clinical setting, it could potentially influence study results [81]. Second, the study focused on Spanish-speaking patients from a single center in Mexico City, which may limit the generalizability of the results to other Mexican or Latin American patients. Future research could benefit from including multicenter recruitment. Third, while the patients in this study were representative of typical Mexican women with breast cancer, the research lacked representation across the entire spectrum of patients with advanced stages of the disease. Fourth, the cross-sectional nature of our study restricts our ability to draw causal inferences. Finally, we assessed only a limited number of psychometric characteristics of the questionnaire, which were considered necessary for an initial evaluation; however, additional relevant factors, such as sensitivity to change, should also be defined in future studies.
Study implications
The current study investigates the prevalence of FHC among women with advanced-stage breast cancer and introduces a questionnaire to assess this construct.
Breast cancer is the leading cause of cancer deaths among Mexican women, who frequently present with advanced stages at their first consultation, highlighting the study’s public health relevance.
The FHC-Q demonstrated adequate psychometric properties and can be easily scored manually, allowing for immediate patient feedback. This enhances patient-centered care, which the Institute of Medicine defines as care that is respectful of and responsive to individual patient preferences, guiding clinical decisions [23]. This approach has been shown to improve treatment safety, effectiveness, adherence, and health-related outcomes [82], underlining the clinical implications of the study.
Moreover, the patient-centered care model emphasizes respecting patients as unique individuals, advocating for their autonomy and the consideration of their values [82], which further supports the study’s ethical implications.
Conclusions
The FHC was recognized by a small group of Mexican women who were suffering from advanced stages of breast cancer. The FHC-Q demonstrated strong psychometric properties for assessing this issue within the targeted population. In the context of patient-centered care, it is important that FHC encompasses more than just the physician’s perspective. The patient’s views must also be acknowledged and integrated to fully facilitate shared decision-making.
Supporting information
S2 Appendix. Futility in healthcare Questionnaire (FHC-Q).
https://doi.org/10.1371/journal.pone.0326332.s002
(PDF)
S1 Table. Successive FHC questionnaire versions.
https://doi.org/10.1371/journal.pone.0326332.s003
(DOCX)
S3 Table. Cronbach alfa values of the FHC-Q when items are removed.
https://doi.org/10.1371/journal.pone.0326332.s005
(DOCX)
Acknowledgments
We thank the patients and doctors of FUCAM for their enthusiastic participation in the study.
References
- 1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209–49. pmid:33538338
- 2. Chávarri-Guerra Y, Villarreal-Garza C, Liedke PER, Knaul F, Mohar A, Finkelstein DM, et al. Breast cancer in Mexico: a growing challenge to health and the health system. Lancet Oncol. 2012;13(8):e335-43. pmid:22846838
- 3. de Almeida LM, Cortés S, Vilensky M, Valenzuela O, Cortes-Sanabria L, de Souza M, et al. Socioeconomic, Clinical, and Molecular Features of Breast Cancer Influence Overall Survival of Latin American Women. Front Oncol. 2022;12:845527. pmid:35530311
- 4. Seneviratne SA, Campbell ID, Scott N, Lawrenson RA, Shirley R, Elwood JM. Risk factors associated with mortality from breast cancer in Waikato, New Zealand: a case-control study. Public Health. 2015;129(5):549–54. pmid:25753278
- 5. Ángeles-Llerenas A, Torres-Mejía G, Lazcano-Ponce E, Uscanga-Sánchez S, Mainero-Ratchelous F, Hernández-Ávila JE, et al. Effect of care-delivery delay on the survival of Mexican women with breast cancer. Salud Publica Mex. 2016;58(2):237–50. pmid:27557382
- 6. Bradley CJ, Given CW, Roberts C. Race, socioeconomic status, and breast cancer treatment and survival. J Natl Cancer Inst. 2002;94(7):490–6. pmid:11929949
- 7. Holm J, Eriksson L, Ploner A, Eriksson M, Rantalainen M, Li J, et al. Assessment of Breast Cancer Risk Factors Reveals Subtype Heterogeneity. Cancer Res. 2017;77(13):3708–17. pmid:28512241
- 8. Cazap E, Buzaid AC, Garbino C, de la Garza J, Orlandi FJ, Schwartsmann G, et al. Breast cancer in Latin America: results of the Latin American and Caribbean Society of Medical Oncology/Breast Cancer Research Foundation expert survey. Cancer. 2008;113(8 Suppl):2359–65. pmid:18837031
- 9. Justo N, Wilking N, Jönsson B, Luciani S, Cazap E. A review of breast cancer care and outcomes in Latin America. Oncologist. 2013;18(3):248–56. pmid:23442305
- 10. Instituto Nacional de Estadística y Geografía (INEGI) Mexico. Statistics about the world day against cancer national data. 2024. Available from: https://www.inegi.org.mx/contenidos/saladeprensa/aproposito/2024/EAP_CANCER24.pdf
- 11. Knaul FM, Nigenda G, Lozano R, Arreola-Ornelas H, Langer A, Frenk J. Breast cancer in Mexico: a pressing priority. Reprod Health Matters. 2008;16(32):113–23. pmid:19027629
- 12. International Agency for Research on Cancer, Global Cancer ObservatoryWHO. Cancer Tomorrow DATAVIZ. Estimated number of new cases and deaths from 2022 to 2030, Females, age [0-85+] for Breast Cancer. Available from: https://gco.iarc.who.int/tomorrow/en/dataviz/tables?populations=904&cancers=20&years=2030&sexes=2&types=1
- 13. Coughlin SS. Social determinants of breast cancer risk, stage, and survival. Breast Cancer Res Treat. 2019;177(3):537–48. pmid:31270761
- 14. Goel N, Hernandez A, Cole SW. Social Genomic Determinants of Health: Understanding the Molecular Pathways by Which Neighborhood Disadvantage Affects Cancer Outcomes. J Clin Oncol. 2024;42(30):3618–27. pmid:39178356
- 15. Morata L. An evolutionary concept analysis of futility in health care. J Adv Nurs. 2018;74(6):1289–300. pmid:29350780
- 16. Schneiderman LJ. Defining Medical Futility and Improving Medical Care. J Bioeth Inq. 2011;8(2):123–31. pmid:21765643
- 17. Šarić L, Prkić I, Jukić M. Futile Treatment-A Review. J Bioeth Inq. 2017;14(3):329–37. pmid:28634768
- 18. Ardagh M. Futility has no utility in resuscitation medicine. J Med Ethics. 2000;26(5):396–9. pmid:11055046
- 19. Rinehart A. Beyond the futility argument: the fair process approach and time-limited trials for managing dialysis conflict. Clin J Am Soc Nephrol. 2013;8(11):2000–6. pmid:23868900
- 20. Schneiderman LJ, Faber-Langendoen K, Jecker NS. Beyond futility to an ethic of care. Am J Med. 1994;96(2):110–4. pmid:8109595
- 21. Swetz KM, Burkle CM, Berge KH, Lanier WL. Ten common questions (and their answers) on medical futility. Mayo Clin Proc. 2014;89(7):943–59. pmid:24726213
- 22. Håkansson Eklund J, Holmström IK, Kumlin T, Kaminsky E, Skoglund K, Höglander J, et al. “Same same or different?” A review of reviews of person-centered and patient-centered care. Patient Educ Couns. 2019;102(1):3–11. pmid:30201221
- 23. Klimesch A, Martinez-Pereira A, Topf C, Härter M, Scholl I, Bravo P. Conceptualization of patient-centered care in Latin America: A scoping review. Health Expect. 2023;26(5):1820–31. pmid:37491799
- 24.
Rodgers BL, Knafl KA. Concept analysis: an evolutionary view. Concept Development in Nursing: Foundations, Techniques and Application. 2nd ed. Philadelphia: Saunders. 2000. p. 77–102.
- 25. Gilardino RE, Valanzasca P, Rifkin SB. Has Latin America achieved universal health coverage yet? Lessons from four countries. Arch Public Health. 2022;80(1):38. pmid:35063033
- 26. The World Medical Association Ethics Unit. WMA Declaration of Helsinki – Ethical Principles for medical research involving human subjects. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects.
- 27.
Breast Cancer Foundation FUCAM. Announcing Who we are? [cited 8 January 2025. ] In: Fucam AC. Web Site [Internet]. Mexico: FUCAM. 2025 Available from: https://www.fucam.org.mx/fucam/quienes-somos
- 28.
Breast Cancer Foundation FUCAM Annual report 2022. [cited 8 January 2025] In: Fucam AC. Web Site [Internet]. Mexico: FUCAM. 2025 Available from: https://cdn.prod.website-files.com/6656871611b7e0411305ed3b/66c768f75f40e21875c4435e_INFORME_ANUAL_2022.pdf
- 29.
Hortobagyi GN, Connolly JL, D’Orsi CJ, Edge SB, Mittendorf EA. Breast. In: Amin B, Edge S, Greene F, editors. AJCC cancer staging manual. 8th ed. New York, NY: Springer. 2017.
- 30.
Streiner DL, Norman GR. Health Measurement Scales. A Practical guide to their development and use. 4th ed. Oxford; Oxford Academic. 2008. https://doi.org/10.1093/acprof:oso/9780199231881.001.0001
- 31. Jung A, Challoumas D, Pagels L, Armijo-Olivo S, Braun T, Luedtke K. Guidelines for the development and validation of patient-reported outcome measures: a scoping review. BMJ Evid Based Med. 2024;29(6):363–73. pmid:38782559
- 32. Golzar J, Noor S, Tajik O. Convenience Sampling. IJELS. 2022;1:72–7.
- 33. Barón Duarte FJ, Rodríguez Calvo MS, Amor Pan JR. La Agresividad Terapéutica y la Oncología Líquida. Cuad Bioet. 2017;28:71–81.
- 34. Consenso mexicano sobre el diagnóstico y tratamiento del cáncer mamario. 10 Reunion. 2023 Jan 27-28 [Cited 2025 January 8. ]. Available from: http://consensocancermamario.com/documentos/FOLLETO_CONSENSO_DE_CANCER_DE_MAMA_10aRev2023a.PDF
- 35. Baker SP, O’Neill B, Haddon W Jr, Long WB. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14(3):187–96. pmid:4814394
- 36. Wagner DP, Draper EA. Acute physiology and chronic health evaluation (APACHE II) and Medicare reimbursement. Health Care Financ Rev. 1984;Suppl:91–105.
- 37. Duvall DB, Zhu X, Elliott AC, Wolf SE, Rhodes RL, Paulk ME, et al. Injury severity and comorbidities alone do not predict futility of care after geriatric trauma. J Palliat Med. 2015;18(3):246–50. pmid:25494453
- 38. Gabbay E, Calvo-Broce J, Meyer KB, Trikalinos TA, Cohen J, Kent DM. The empirical basis for determinations of medical futility. J Gen Intern Med. 2010;25(10):1083–9. pmid:20645019
- 39. Afessa B, Keegan MT, Mohammad Z, Finkielman JD, Peters SG. Identifying potentially ineffective care in the sickest critically ill patients on the third ICU day. Chest. 2004;126(6):1905–9. pmid:15596691
- 40. Earle CC, Park ER, Lai B, Weeks JC, Ayanian JZ, Block S. Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol. 2003;21(6):1133–8. pmid:12637481
- 41. Daza P, Novy DM, Stanley MA, Averill P. The Depression Anxiety Stress Scale-21: Spanish Translation and Validation with a Hispanic Sample. J Psychopathol Behav Assess. 2002;24:195–205.
- 42. Taherdoost H. Validity and reliability of the research instrument: How to test the validation of a questionnaire/survey in a research. Int J Acad Res in Management. 2016;5:28–36.
- 43. Escobar-Pérez J, Cuervo-Martínez A. Validez de contenido y juicio de expertos: una. Aproximación a su utilización. Avances en Medición. 2008;6:27–36.
- 44. Durán-Arenas L, Gallegos-Carrillo K, Salinas-Escudero G, Martínez-Salgado H. Hacia una base normativa mexicana en la medición de calidad de vida relacionada con la salud, mediante el Formato Corto 36. Salud pública Méx. 2004;46:306–15.
- 45. Green RA. The Delphi Technique in Educational Research. Sage Open. 2014;4(2).
- 46. Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5(6):649–55. pmid:7165009
- 47. Bruce B, Fries JF. The Stanford Health Assessment Questionnaire: dimensions and practical applications. Health Qual Life Outcomes. 2003;1:20. pmid:12831398
- 48. Gómez CFJ, Ponce RER. [A new proposal for the interpretation of Family APGAR]. Aten Fam. 2010;17: 102–6. [cited 2024 Mar 24. ]. Available from: https://www.revistas.unam.mx/index.php/atencion_familiar/article/view/21348
- 49. Cserni G. Histological type and typing of breast carcinomas and the WHO classification changes over time. Pathologica. 2020;112(1):25–41. pmid:32202537
- 50. Eliyatkın N, Yalçın E, Zengel B, Aktaş S, Vardar E. Molecular Classification of Breast Carcinoma: From Traditional, Old-Fashioned Way to A New Age, and A New Way. J Breast Health. 2015;11(2):59–66. pmid:28331693
- 51. Hofstee WKB, Ten Berge JMF, Hendriks AAJ. How to score questionnaires. Personality and Individual Differences. 1998;25(5):897–909.
- 52.
Pett M, Lackey N, Sullivan J. Making Sense of Factor Analysis. SAGE Publications, Inc. 2003. https://doi.org/10.4135/9781412984898
- 53. Tristan LA. Modification to the Lawshe model for the quantitative opinion of the content validity of an objective instrument. Avances en medición 2008;6: 37–48 [Cited 2024 October 24. ]. Available from: https://www.humanas.unal.edu.co/lab_psicometria/application/files/9716/0463/3548/VOL_6._Articulo4_Indice_de_validez_de_contenido_37-48.pdf
- 54.
Cohen J. Statistical Power Analysis for the Behavioural Sciences. New York: Academic Press; 1977. [cited 2024 March 24. ]. Available from: https://www.utstat.toronto.edu/~brunner/oldclass/378f16/readings/CohenPower.pdf
- 55.
Davis JK. Ethics at the End of Life: New Issues and Arguments. 1st ed. Routledge. 2017.
- 56. Medical futility in end-of-life care: report of the Council on Ethical and Judicial Affairs. JAMA. 1999;281(10):937–41. pmid:10078492
- 57. Schneiderman LJ, Jecker NS, Jonsen AR. Medical futility: its meaning and ethical implications. Ann Intern Med. 1990;112(12):949–54. pmid:2187394
- 58. Emanuel LL, Emanuel EJ. Decisions at the end of life. Guided by communities of patients. Hastings Cent Rep. 1993;23(5):6–14. pmid:8262772
- 59. Special edition on medical futility and a community policy. N C Med J. 1995;56:412–72.
- 60. Aghabarary M, Dehghan Nayeri N. Medical futility and its challenges: a review study. J Med Ethics Hist Med. 2016;9:11. pmid:28050241
- 61. Kaasa S, Loge JH, Aapro M, Albreht T, Anderson R, Bruera E, et al. Integration of oncology and palliative care: a Lancet Oncology Commission. Lancet Oncol. 2018;19(11):e588–653. pmid:30344075
- 62. Bacoanu G, Poroch V, Aniței M-G, Poroch M, Froicu EM, Pascu AM, et al. Therapeutic Obstinacy in End-of-Life Care-A Perspective of Healthcare Professionals from Romania. Healthcare (Basel). 2024;12(16):1593. pmid:39201152
- 63. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42. pmid:17161752
- 64. Lohr KN, Zebrack BJ. Using patient-reported outcomes in clinical practice: challenges and opportunities. Qual Life Res. 2009;18(1):99–107. pmid:19034690
- 65. Earle CC, Neville BA, Landrum MB, Ayanian JZ, Block SD, Weeks JC. Trends in the aggressiveness of cancer care near the end of life. J Clin Oncol. 2004;22(2):315–21. pmid:14722041
- 66. Solomon MZ, O’Donnell L, Jennings B, Guilfoy V, Wolf SM, Nolan K, et al. Decisions near the end of life: professional views on life-sustaining treatments. Am J Public Health. 1993;83(1):14–23. pmid:8417600
- 67. Sirovich BE, Woloshin S, Schwartz LM. Too Little? Too Much? Primary care physicians’ views on US health care: a brief report. Arch Intern Med. 2011;171(17):1582–5. pmid:21949169
- 68. Amoroso S, Chalela JA. Perception of Provision of Futile Care Among Clinicians in the Neuroscience Intensive Care Unit. J Neurosci Nurs. 2019;51(5):249–52. pmid:31469702
- 69. Chamberlin P, Lambden J, Kozlov E, Maciejewski R, Lief L, Berlin DA, et al. Clinicians’ Perceptions of Futile or Potentially Inappropriate Care and Associations with Avoidant Behaviors and Burnout. J Palliat Med. 2019;22(9):1039–45. pmid:30874470
- 70. Schneiderman LJ, Jecker N. Futility in practice. Arch Intern Med. 1993;153(4):437–41. pmid:8435023
- 71. Lo B, Jonsen AR. Clinical decisions to limit treatment. Ann Intern Med. 1980;93(5):764–8. pmid:6971066
- 72. Hanna Ruz V. Obstinación terapéutica y su límite con la ética: ¿cuándo detenerse?. Rev chil anest. 2021;50(1).
- 73. Casella C, Graziano V, Lorenzo PD, Capasso E, Niola M. Unreasonable obstinacy: Ethical, deontological and forensic medical problems. J Public Health Res. 2018;7(3):1460. pmid:30687677
- 74.
Committee on Quality of Health Care in America I of M. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington DC: National Academies. 2001.
- 75.
Beauchamp TL, Childress JF. Principle of biomedical ethics. 4th ed. New York: Oxford University Press. 1994.
- 76.
Stepke FL. Bioética en América Latina. Una década de evolución. Centro Interdisciplinario de Estudios de Bioética, Universidad de Chile OP/OMS. 2010.
- 77. Rodríguez Soriano RI. Reflections on the need for hierarchies in bioethical principialism. Revista Ciencias de la Salud. 2017;16:155–70.
- 78.
Macklin R. The doctor-patient relationship in different cultures. In: Kuhse H, Schüklenk U, Singer P, editors. Bioethics: An anthology. 3rd ed. Chichester, West Sussex; Malden, MA: John Wiley & Sons Inc. 2015. p. 642–53.
- 79. Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine (Baltimore). 2023;102(7):e32829. pmid:36800594
- 80. Niederman MS, Berger JT. The delivery of futile care is harmful to other patients. Crit Care Med. 2010;38(10 Suppl):S518-22. pmid:21164391
- 81. Soland J, Kuhfeld M, Edwards K. How survey scoring decisions can influence your study’s results: A trip through the IRT looking glass. Psychol Methods. 2024;29(5):1003–24. pmid:35834195
- 82. Voshaar MJH, Nota I, van de Laar MAFJ, van den Bemt BJF. Patient-centred care in established rheumatoid arthritis. Best Pract Res Clin Rheumatol. 2015;29(4–5):643–63. pmid:26697772