https://orcid.org/0000-0003-3454-6602
Figures
Abstract
Objective
To analyze the psychometric properties of the cross-culturally adapted version of the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS–PC) Compact Form Brazil.
Methods
A methodological study was conducted with 281 adult Primary Health Care users. Data collection took place online. Confirmatory Factor Analysis (CFA) was used to evaluate the psychometric properties of the PREOS–PC after the process of cross-cultural adaptation to the Brazilian context. Internal consistency was evaluated through Cronbach’s alpha coefficient (α) and McDonald’s omega coefficient (ω).
Results
The sample consisted of 73.3% women. The mean age was 36.1 years (SD = 12.2). Of the 23 items of the PREOS–PC that were eligible for CFA, a model with four correlated domains and 16 items presented satisfactory fit indexes. The domains were Practice Activation (PrA) (four items), Patient Activation (PaA) (two items), Experiences of patient safety events (EPaS) (five items) and Outcomes of patient safety (OPaS) (six items). One domain (GPeS) presented one question with a 0 to 10 response scale and two open questions, which cannot be inserted in the CPA due to the nature of the items, but can be included in the application of the scale, being evaluated individually. In this factorial model, five items (EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS8) presented factor loadings ≤ 0.30. The α and ω values demonstrated good internal consistency for all domains of the PREOS–PC.
Conclusions
The Brazilian version of the PREOS–PC Compact Form Brazil composed of four domains (PrA, PA, EPaS and OPaS) and 16 items presented evidence of validation of its psychometric properties and can be used to evaluate the experiences and results of patient safety in Primary Health Care in the Brazilian context.
Citation: Silva AEBdC, Sousa TP, Guimaraes RA, Pagotto V, Lima JCd, Sousa MRGd, et al. (2024) Validation of the patient reported experiences and outcomes of safety in primary care compact Form Brazil. PLoS ONE 19(7): e0305414. https://doi.org/10.1371/journal.pone.0305414
Editor: Immaculate Sabelile Tenza, North-West University, SOUTH AFRICA
Received: January 9, 2024; Accepted: May 29, 2024; Published: July 1, 2024
Copyright: © 2024 Silva et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: CFA, Confirmatory Factor Analysis; PHC, Primary Health Care; CFI, Comparative Fit Index; COVID-19, disease caused by the SARS-CoV-2 virus; EPaS, Experiences of patient safety events; df, Degrees of Freedom; GPeS, General perceptions of safety; 90%CI, 90% Confidence Interval; 95%CI, 95% Confidence Interval; CVI, Content Validity Index; OPaS, Outcomes of patient safety; PaA, Patient Activation; PrA, Practice Activation; PREOS–PC, Patient Reported Experiences and Outcomes of Safety in Primary Care; R$, Brazilian Reais; RMSEA, Root Mean Square Error of Approximation; PS, Patient Safety; SUS, Brazilian National Health System; TLI, Tucker-Lewis Index; α, Cronbach’s alpha; χ2, Chi-square goodness-of-fit test statistic; ω, McDonald’s omega
Background
Patient safety (PS) is conceptualized by the World Health Organization as “the reduction of risk of unnecessary harm associated with healthcare to an acceptable minimum” [1]. Patient safety consists of a set of organized actions that influence healthcare cultures, processes, behaviors, technologies and environments, with the aim of reducing adverse events, errors and their impacts on patients, families, professionals and health services [2].
Most of the research, discussions and efforts that have sought to understand the nature and magnitude of adverse events, evaluating the experiences and results of PS, have been focused on hospital contexts, care environments in which activities are more standardized, complex and carry a higher risk of adverse events [3]. Furthermore, previous research has focused on the experiences and outcomes in the perception of healthcare providers rather than patients [4–6].
Studies that have evaluated these experiences in Primary Health Care (PHC) are limited, especially in relation to the perception of patients [3, 7]. At this level of care, errors related to the use of medications, misdiagnoses and treatments, problems in communication and in the relationship of the healthcare provider with the patients are frequent, which can contribute to increase the magnitude of adverse events [8]. In low- and middle-income countries, the prevalence of PHC safety incidents can vary from 25 to 40% of the care provided, with 80% of these incidents being preventable [9]. Estimates indicate that about 20% and 25% of the general population suffers damage in this scenario in developed and developing countries, respectively [9]. Therefore, unsafe primary care can cause higher morbidity and mortality and increased costs for health services and countries [10].
The experience reported by patients is crucial to improve PHC services, with the aim of improving patient care and promoting the PS culture. This form of evaluation facilitates user involvement, healthcare planning for the improvement and delivery of safe care, increased communication, and trust in healthcare providers [11, 12]. Additionally, patients and their families can assist in the identification of failures in PS or in the quality of care [13].
The use of validated and reliable scales to measure and monitor PS in PHC is indispensable. Some methods are immediately available, easy to administer, do not require external involvement and have low cost [14]. Among them, the Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS–PC) is a multidimensional instrument developed by specialists and patients that has shown potential for implementation in PHC and use in research. The original instrument, in its final version, has 58 items, with several grouped into five PS domains: a) Practice Activation (PrA), b) Patient Activation (PaA), c) Experiences of patient safety events (EPaS), d) Outcomes of patient safety (OPaS) and e) General perceptions of safety (GPeS) [7].
The PREOSP-PC analyzes the experiences of patients and the results of PS problems in PHC, allowing the differentiation between the different levels of PS between practices and over time in this level of care [7]. It presents three versions, one screening form, one compact form and one comprehensive form, which can be used independently to evaluate five PS domains in PHC in a unique way [15]. The validation study of the comprehensive version was carried out in England with 6,736 patients. Psychometric methods were applied to evaluate its acceptability, reliability, and structural and construct validity, presenting satisfactory parameters [7]. Despite its relevance, this instrument has not yet been cross-culturally adapted and validated for the Brazilian population, which limits its use in the country.
The compact form of the PREOS–PC provides a balance between high psychometric standards and a reduced administrative burden, facilitating its implementation and use in the real practice. Therefore, by validating the scale in PHC care environments in Brazil, it will be possible to identify which experiences are most reported by patients and which PS-related problems need greater attention and, consequently, interventions. The analysis of patients’ experiences may allow an analysis of the current state of PS in PHC, providing relevant data for managers, aiming to develop improvements that meet the needs identified and improve the quality of the services. These actions would have a direct impact on patients’ lives due to the improvement of PS in PHC.
The aim of this study was to analyze the psychometric properties of the cross-culturally adapted version of the PREOS-PC Compact Form Brazil.
Methods
Outline
Methodological study to verify the psychometric properties of the PREOS–PC Compact Form Brazil, translated and cross-culturally adapted from the PREOS–PC [7, 16]. Methodological studies deal with the development, validation and evaluation of research tools and methods, such as PREOS–PC Compact Form Brazil.
Cross-cultural adaptation
The instrument was cross-culturally adapted according to the recommendations of Oxford University Innovation Limited [17] and the methodological guidelines of Beaton et al. [18, 19]. It is important to mention that we obtained the necessary permissions from the owner of the original questionnaire (https://innovation.ox.ac.uk/wp-content/uploads/2018/07/PREOS–PC-27-item-version-SAMPLE.pdf) to carry out the translation and cross-cultural adaptation of the instrument to Brazil.
The process of cross-cultural adaptation of the PREOS–PC Compact Form Brazil was carried out in several stages. First, the questionnaire was translated into Brazilian Portuguese (the language of the participants). The instrument was translated into the target language by two independent native Brazilian translators, with experience in this type of study, one of them with experience in scientific health terminology and the other in linguistics and culture of the Brazilian Portuguese language. The translation of the PREOS–PC instrument into Brazilian Portuguese was carried out in May 2019. Secondly, the synthesis of translations was performed by a third independent bilingual translator, with knowledge and expertise in research in the health area, generating an initial version. This stage was carried out in June 2019. Thirdly, in August 2019, this last version was back-translated by two independent professional translators (with British English as their native language, but fluent in Portuguese), who were not aware of the original version of the assessment instrument or the objectives of the study, which resulted in two more versions.
Fourth, all versions of the instrument were evaluated and approved by an expert committee composed of members of the research team who are experts in the field in question. The review by a committee of experts aimed to evaluate and consolidate all translation and back-translation versions of the instrument, with the aim of developing a single version of the instrument, in the target language, which would be used in the subsequent steps [18].
Finally, a pre-test (pilot) was performed using the final version of the instrument with 30 participants, aiming to evaluate the ability of individuals to interpret and understand the items of the instrument and to provide measures of internal consistency and content validity. This stage was carried out between August 1st and October 30th, 2020. Participants provided consent online, by reading the informed consent form and subsequently accepting the term. After these steps, the participant had access to the questionnaire. The participants in this stage were individuals from two PHC units with a mean age of 42.0 years. The majority (80.0%) were female.
The results of the pre-test showed good understanding of the items by the participants. The McDonald’s omega coefficient (ω) [20] was used to evaluate the internal consistency of the scale, with acceptable values above 0.6. The ω found were 0.689 for the practice activation domain, 0.749 for the patient activation domain and 0.850 for the PS results, demonstrating good internal consistency of the translated and cross-culturally adapted version.
Content validity was also performed by a committee of eight expert judges who evaluated the semantic (verifying whether the words have the same meaning in the original and translated versions), idiomatic (observing expressions difficult to translate that are not suitable for the place of study, to then elaborate an equivalent expression in the target language version), conceptual (verifying whether the words present a different conceptual meaning between cultures) and cultural (trying to find terms that correspond to those described in the instrument and that express situations or activities of daily life in the culture, so that the situations described in the instrument are in accordance with those experienced in the context of the local culture) equivalences of the final instrument [18, 19]. This stage was carried out between November 1st and December 29th, 2020. Each judge evaluated the items through a Likert-type response scale from 1 to 4 (1 = not relevant or not representative, 2 = need for major revision to be representative, 3 = needs minor revision to be representative, 4 = relevant or representative). The mean Content Validity Index (CVI) of 0.96 was calculated, being considered satisfactory (CVI≥0.80) [21].
The participants of the committee of judges were selected from searches on the Lattes Platform of the National Council for Scientific and Technological Development (CNPq) for health professionals with command of Brazilian Portuguese and English; with a master’s or doctorate degree; with experience or work in the area of quality and patient safety or in primary health care; with scientific production and experience in research involving the application of instruments or recognized knowledge in cross-cultural adaptation of measuring instruments, that is, with methodological knowledge in the preparation and/or adaptation of instruments. To avoid regional biases, the diversity of location and place of birth of the judges was considered, which is notably relevant in Brazil, given the large geographic dimension and its regional disparities. An invitation to participate in the committee was sent via email to each specialist. Thus, the committee was made up of eight experts. All judges with specialization in patient safety have already published works on adapting and validating instruments, and only two were not part of a working group related to patient safety.
Of the eight judges, half were nurses, and 7 had a doctorate as their highest degree, in terms of specialty, 4 were specialists in Patient Safety, 3 in measurement/psychometrics and 1 in primary health care. The average time working in the respective specialty areas was 10.25 years. As for the region of Brazil, 4 were from the Central-West, 2 were from the South, 1 was from the Southeast and 1 from the Northeast.
The flowchart with the stages of cross-cultural adaptation is shown in Fig 1.
The final version of the instrument is available in S1 File.
Context
The study to evaluate the psychometric properties was conducted in PHC units of Goiânia (estimated population of 1,555,626 inhabitants) [22], located in the state of Goiás, Midwest region of Brazil. All 75 PHC units of the study city were invited to participate in the study, however, only 40 (55.3%) units that agreed. The participating PHC units were Primary Health Units, that perform a set of individual, family and collective health actions that involve health promotion, disease prevention, protection, diagnosis, treatment, rehabilitation, harm reduction, palliative care, and health surveillance, carried out with a multidisciplinary team and directed toward the specific population. All Primary Health Units are considered to be potential spaces for education, training of human resources, research, in-service teaching, innovation, and technological assessment for the health care network [23]. The Primary Health Care coverage in the city at the time of the study was 59.3% [24]. The data was collected between March and July 2021.
Participants
The population was composed of users of the Brazilian public health system (Sistema Único de Saúde–SUS) attended in PHC. Adults aged 18 years or older, of both sexes, native speakers of the Brazilian Portuguese language, registered in one of the 40 units of the PHC network of Goiânia, who had used primary health services at least once in the previous 12 months, regardless of the reason, were included. Individuals without internet connection by mobile phone and/or who did not use the free messaging application WhatsApp were excluded, since the instrument was self-administered via google forms, through a link to access the questionnaire sent through the application.
Sample size
The minimum sample size was estimated according to the need for five to ten respondents for each item of the instrument [25]. Considering that the PREOS–PC Compact Form is composed of 27 items, a minimum sample size of 270 individuals was required.
Data collection
Data collection took place through an online access platform. The participants answered two self-administered instruments. The first included the collection of sociodemographic data on health and use of PHC health services in the previous 12 months. This instrument was adapted from questions used and validated in another study [26]. The second included the application of the cross-culturally adapted PREOS–PC Compact Form Brazil.
The selection of participants was performed using the simple random sampling technique, from the list of medical records of the PHC units, where the necessary contact information was obtained, such as the mobile phone number. Due to the pandemic of the disease caused by the SARS-CoV-2 virus (COVID-19), users were contacted by phone and invited to participate in the study. The data collection instruments were also transferred to the Google forms platform and the survey was made available to receive the online responses of the participants through WhatsApp. By accessing the link, the participant was able to view the information about the study and, to proceed, they had to agree to participate in the research. If calls are not answered by the participant, new calls were made to people treated at the same health unit
Measurements
Sociodemographic variables, health conditions and use of health services.
The sociodemographic data collected included age (in years), gender (male or female), self-declared race/skin color (white, mixed race, black or other–Asian or indigenous), schooling (up to complete elementary school, incomplete/complete high school or incomplete/complete higher education), family income (minimum wages in reais—R$) (< 1, 1–2 or > 2). Data were also collected on comorbidities (no or yes), use of medications (no or yes), considering the use of any medications as yes [27], and self-assessment health (very bad/bad or good/very good). Data were collected on the use of health services, the number of times they had used the unit’s services in the previous 12 months (1, 2, 3 or ≥4) and the number of consultations (with a doctor and/or nurse) in the unit in the previous 12 months (1, 2–4 or ≥4).
PREOS–PC Compact Br.
The PREOS–PC Compact Form Brazil was applied to obtain the necessary data for Confirmatory Factor Analysis (CFA).
The instrument presents eight questions and 27 items of different types, including 12 Likert-type items, nine multiple-choice items in which the individual needs to mark one or more items from a checklist of experiences with safety problems, three dichotomous items (yes and no) and two open questions. Question 1 presents four items (PrA1 to PrA4) with Likert-type responses (always, often, sometimes, rarely and never), with the option ‘does not apply’. These items are grouped in the PrA domain. Question 2 presents two items (PaA1 and PaA2) with Likert-type responses (always, often, sometimes, rarely, never), with the option ‘does not apply’. These items are grouped in the PaA domain. Question 3 presents nine multiple-choice items (EPaS 1–9). It should be noted that this question presented an item referring to “None of the above options”, which was excluded from the CFA. Question 4 presents three items (EPaS 10–12), with dichotomous answers (yes and no). The items of question 3 and 4 are grouped in the EPaS domain. Question 5 presents six items (OPaS 1–6) with Likert-type responses (no, yes, some; yes, many; yes, extreme levels), as well as ‘I don’t know’. These items are grouped in the OPaS domain. Finally, question 6, in the GPeS domain, is of the closed type (GPeS1) on a scale from 0 to 10 and questions 7 (GPeS2) and 8 (GPeS3) are open. Questions GPeS1, GPeS2 and GPeS3 were not included in the CFA (Table 1). In the original model, 24 items (excluding items EPaS 9, GPeS1, GPeS2 and GPeS3 due to their response natures) were part of the factorial structure of the scale and were included in the CFA.
Statistical analysis
The analyses were performed using the packages “lavaan”, “psych”, “semTools” and “semPlot” in the R language version 4.0.3 [28].
The Anderson-Darling test was initially performed to verify the normality of the quantitative variables. Next, a descriptive analysis of the variables was performed. Quantitative variables were presented as mean and standard deviation. For the PREOS–PC Compact Form Brazil, the median, 25th percentile (P25), 75th percentile (P75), minimum and maximum values were also presented. Qualitative variables were presented as absolute and relative frequency in percentage.
Construct validity was determined using CFA. This statistical technique aims to evaluate the relationship between latent variables (also called domains) and their relations with a pre-established theoretical covariance structure [29]. The Mardia normality test was performed to evaluate the multivariate normality of the PREOS–PC Compact Form Brazil [30]. Due to the absence of multivariate normality (p-value < 0.001), a CFA with likelihood estimation with robust variance was performed [31]. Two different factor structures of the PREOS–PC Compact Form Brazil were examined. The initial model of the scale was adjusted through first-order CFA, with four independent domains (PrA, PaA, EPaS, OPaS) and correlated with all 23 eligible items. Next, a final model was established by means of first-order CFA, with four independent and correlated domains, totaling 17 items. In the final model, only items with a factor loading > 0.30 in the initial model remained.
The following indices were used to verify the quality of fit of the models: (i) chi-square fit statistics (χ2)–non-significant p-values (≥0.05) indicate good quality of fit [32]; (ii) Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) with acceptable values equal to or greater than 0.95 [33] and (iii) Root Mean Square Error Of Approximation (RMSEA)–values < 0.06 indicate good fit, 0.06–0.07 acceptable fit, 0.08–0.10 limited fit, and > 0.10 inadequate fit [33].
After the estimates, the initial and final models were compared by the Satorra-Bentler Scaled Chi-Square test, which measures the difference between χ2 and the degrees of freedom (df) between the original and alternative models. The model chosen, if there was a statistical difference, presented the lowest chi-square rate, in addition to fit indices within the cutoff points [34].
The internal consistency of the instrument was evaluated through Cronbach’s alpha (α) and McDonald’s omega (ω), followed by the respective 95% confidence intervals (95%CI). These coefficients can range from zero to one, and the closer to one, the greater the internal consistency of the instrument. Values of α > 0.70 indicate good internal consistency [35], while values of ω > 0.6 indicate good internal consistency [20].
Results
A total of 1,650 telephone calls were made and, of these, 979 (59.3%) were answered. Of the total of 979 calls answered, 182 users (18.6%) refused to participate in the study. Thus, of the 797 (81.4%) individuals who agreed to participate in the study, 281 (35.2%) returned the completed questionnaire. Thus, the overall response rate was 28.7% (281/979 calls).
Characteristics of the participants
The mean age was 36.2 years (SD = 12.2) and the majority (73.3%) were female. The other characteristics of the participants are presented in Table 2.
Descriptive analysis of the PREOS–PC Compact Form Brazil
The Tables 3 and 4 present a descriptive synthesis of the quantitative and qualitative items of the PREOS–PC Compact Form Brazil, respectively.
Confirmatory Factor Analysis
The CFA was conducted to verify the factorial structure of the PREOS–PC Compact Form Brazil. The initial model was adjusted according to a first-order model, with four correlated latent factors (PrA, PaA, EPaS and OPaS) and all 23 eligible items of the PREOS–PC Compact Form Brazil. The “PrA” factor was composed of four items (PrA1 to PrA4), the “PaA” factor of two items (PaA1 and PaA2), the “EPaS” factor encompassed 11 items (EPaS 1 to EPaS12) and the “OPaS” factor included six items (OPaS 1 to OPaS6). This model presented the following quality of fit indices (χ2: 327,194; df: 224; p-value < 0.001): CFI: 0.908; TLI: 0.897; RMSEA: 0.068 [90%CI: 0.052; 0.085]). The CFI, TLI and RMSEA presented values lower than the cutoff points adopted. In this factorial model, five items (EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS8) presented factor loadings ≤ 0.30, which resulted in their exclusion from the final CFA model (Table 5).
The final model, after excluding the items, was like a first-order model, with four latent factors related to each other (PrA, PaA, EPaS and OPaS), totaling 17 items. The “PrA” factor was composed of four items (PrA1 to PrA4), the “PaA” factor two items (PaA1 and PaA2), the “EPaS” factor five items (EPaS1, EPaS7 and EPaS10 to EPaS12) and the “OPaS” factor six items (OPaS 1 to OPaS6). All items in final model had standardized factor loadings ≥0.3 and were significant, except for item 7 (EPaS7), which had a factorial loading of 0.236, although it was significant. This model had adequate fit indices (χ2: 170,850; df: 114; p-value < 0.001); CFI: 0.955; TLI: 0.951; RMSEA: 0.050 [90%CI: 0.038; 0.061]), indicating that the factorial structure of the PREOS–PC Compact Form Brazil corresponds to a model with four first-order correlated factors with 17 items (73.9% of the original 23 items). Thus, this model proved to be appropriated, even though item 7 (EPaS7) has a reduced factor loading (Fig 2 & Table 6).
The Satorra-Bentler Scaled Chi-Square test for differences between the initial and final models suggested a better fit of the final model to the factor structure. The p-value of the test was 0.0059, which demonstrates that the model with 17 items was significantly higher than the complete model with 23 items.
Internal consistency
The McDonald ω and Cronbach’s α coefficients ranged from 0.74 (95%CI:0.69–0.80) for the EPaS domain to 0.94 (95%CI:0.92–0.97) for the OPaS domain. The McDonald ω values for the PaA and PrA domains were 0.82 (95%CI: 0.76–0.88) and 0.90 (95%CI = 0.88–0.93), respectively. In each domain, Cronbach’s α values were: EPaS– 0.70 (95%CI: 0.65–0.76), PaA– 0.82 (95%CI: 0.77–0.86), PrA– 0.90 (95%CI: 0.88–0.92) and OPaS– 0.94 (95%CI: 0.93–0.95). The results of the measurements revealed good internal consistency of the PREOS–PC Compact Form Brazil.
Discussion
The present study provides evidence on the psychometric validity of the PREOS–PC Compact Form Brazil with a sample of PHC patients. This instrument has been used in other countries, but, to date, its validation had not been performed and its applicability remains limited in the Brazilian context. The results demonstrate the validity of the PREOS–PC Compact Form Brazil, with a factorial structure composed of 17 items and four correlated domains (PrA, PaA, EPaS and OPaS). The instrument presented good internal consistency. Therefore, it represents evidence of construct validity to measure PS perceptions, experiences and results from the perspective of PHC users.
The initial model (with 23 items and four correlated factors) presented low factorial validity indexes. However, after the exclusion of six items, the model proved to be adequate, with good fit indexes. The excluded items were EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS7, all dichotomous and belonging to the EPaS domain. They relate to a single question that covers the problems that patients may have faced during PHC care. The inclusion of dichotomous items in the CFA model may result in biased parameter estimates with inaccurate interpretation limitations [36]. Therefore, the absence of contribution to the domains of these items may have been caused by the very nature of the items and, not necessarily, because they did not contribute to the formation of the construct. Another point is that some excluded items relate to health practices that are uncommon in the context of PHC in Brazil, particularly the administration of medications in the health unit, treatments, blood tests, laboratory tests and diagnostic tests, which are predominantly performed in the secondary (emergency and urgency units) and tertiary (for example: hospitals) networks [37]. This can decrease the probability of the patient’s knowledge about aspects related to the problems faced during their PHC care. All the excluded items, in fact, presented low prevalence in the sample, which may have contributed to their non-fit in the domains. In addition, the collection was carried out in the context of the COVID-19 pandemic in a scenario of reduced supply of health services [38], which may have contributed to the low prevalence of these items.
The final CFA model showed four correlated domains: “PrA”, “PaA”, “EPaS” and “OPaS”. The first contains items that evaluate what the health unit does to create a safe environment and ensure PS. The second presents items that seek to understand how patients are proactive to ensure that the healthcare is safe. The third has items that assess the prevalence of safety errors related to healthcare. The fourth presents items that assess the presence of health damage experienced by users. A domain that is not adjusted to the CFA due to the nature of the items, is called “General perceptions of patient safety”, which aims to evaluate how safe patients perceive their health unit to be and should be analyzed in isolation [26].
Internal consistency was assessed using Cronbach’s α and McDonald’s ω. The values were adequate, higher than recommended (α > 0.70 and ω > 0.60) [20, 35]. McDonald’s ω is a measure of solid internal consistency, but is little used in scale validation studies [39]. The PREOS–PC is relatively recent. No studies were found that analyzed the internal consistency of the instrument, along with the psychometric properties of the compact version. To date, the reliability of the instrument has only been tested for the full version, using Cronbach’s α, but not McDonald’s ω. [7]. The results of the full version study showed high internal consistency for the domains (Cronbach’s α: PrA = 0.89; PaA = 0.80, EPaS = 0.75 and OPaS = 0.96) [7]. Although with differences in the items, the internal consistencies were similar in the present study (Cronbach’s α: PrA = 0.90; PaA = 0.82, EPaS = 0.74 and OPaS = 0.94). In the original study, the EPaS domain presented the lowest internal consistency value, which is like that found in this study, which may explain the removal of some of the items from the final model. It is important to note that the PREOS–PC Compact Form Brazil is a reduced version and, to date, there are no studies that can accurately compare this version, since the original extended instrument has 58 items [7], which prevents a more accurate comparison with the literature.
The validity of the structural construct of the General Perceptions of Patient Safety (GPeS) domain was not confirmed by factor analysis, since one item was composed of a scale from 0 to 10 and two were open questions. Therefore, the compact version does not allow a factorial evaluation of the instrument, since it does not include this domain. It is important to note that, in the full version, the “General perceptions of patient safety” domain presents a single scale question and two open questions [7].
The present study has some limitations. Data collection was performed in a single Brazilian city, which limits the ability to generalize the results. The experiences and reports of the patients in these PHC units may not be the same as the practice in the rest of the country. The study sample was composed of 73.3% women, which does not guarantee that the results found are not distorted by the lack of sampling balance between the sexes, and may not be generalized to men [40]. Additionally, the data was collected online, due to the COVID-19 pandemic. These individuals may have characteristics that differ from people who do not have access to the internet, which also limits generalization. In addition, the validation study of the full version was performed through the application of the physical instrument. Some calls went unanswered, which may have led to selection bias. All measures were self-reported, susceptible to memory and response biases. The inclusion of dichotomous items in the CFA is another limitation, which reduces the contribution of these items in the latent domain.
However, the study has strengths. First, the relatively large sample size allowed the inclusion of enough patients to conduct CFA. Second, this psychometric validation study is unprecedented in Brazil, being a starting point for future investigations that may be carried out in other Brazilian regions, as well as the implementation of the instrument in the PHC practice of the country. The relevance of this study is the possibility of improving the quality of care in PHC in Brazil, adapting methodologies to PS perceptions, experiences and results, through the evaluation of the patients themselves. This study can also help in the implementation of a process to improve the PHC care environment, as well as consolidate knowledge about the theme in the literature. However, further studies are still needed to evaluate the psychometric properties of the PREOS–PC Compact Form Brazil, especially in relation to some of the less prevalent items. Accordingly, future studies with more diverse samples should be carried out to have a clear understanding of users’ perceptions about PS in PHC, so that other comparisons and generalizations can be made. It is important to emphasize that, especially, conducting studies in other Brazilian regions, in a self-administered and face-to-face manner, could contribute to the improvement of the instrument and to the identification of new tests of validity and reliability for the PREOS–PC Compact Form Brazil in different contexts.
Conclusions
The Brazilian version of the PREOS–PC Compact Form Brazil presented satisfactory validity and psychometric properties for 16 items and four domains (PrA, PaA, EPaS and OPaS). The items EPaS2, EPaS3, EPaS4, EPaS5, EPaS6 and EPaS7 presented low factor loadings, which indicates that, in the Brazilian scenario, they do not contribute to the latent variables of the domains. However, these items should be incorporated into future psychometric studies for further assessment. Furthermore, the isolated analysis of the GPeS domain items is not suitable for CFA but should be considered when assessing PS in PHC. The instrument showed good internal consistency.
Supporting information
S1 File. Final version of the Patient Reported Experiences and Outcomes of Safety in Primary Care Compact Form Brazil (PREOS–PC Compact Form Brazil), in Portuguese.
https://doi.org/10.1371/journal.pone.0305414.s001
(PDF)
Acknowledgments
The authors would like to thank the following research assistants: Students Thiago Pereira Sampaio and Viviane Oliveira da Silva who were involved in this project at the data collection stage. We would also like to thank the participants for sharing their experiences. This work was supported by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), awarded to Tanielly Paula Sousa.
References
- 1.
World Health Organization. Conceptual Framework for the International Classification for Patient Safety Final Technical Report (Version 1.1). In: International Classification [Internet]. 2009 [cited 11 Feb 2023] p. 154. Available: https://apps.who.int/iris/bitstream/handle/10665/70882/WHO_IER_PSP_2010.2_eng.pdf
- 2.
World Health Organization. Global patient safety action plan 2021–2030: Towards eliminating avoidable harm e health care. In: World Health Organization [Internet]. 2021 [cited 12 Feb 2023] p. 96. Available: https://www.who.int/teams/integrated-health-services/patient-safety/policy/global-patient-safety-action-plan
- 3. Ornelas MD, Pais D, Souza P. Patient Safety Culture in Portuguese Primary Healthcare. Qual Prim Care. 2016;24: 214–218.
- 4. Fracolli LA, Gomes MFP, Nabão FRZ, Santos MS, Cappellini VK, de Almeida ACC. Primary health care assessment tools: A literature review and metasynthesis. Cien Saude Colet. 2014;19: 4851–4860. pmid:25388193
- 5. MacFarlane E, Carson-Stevens A, North R, Ryan B, Acton J. A mixed-methods characterisation of patient safety incidents by primary eye care practitioners. Ophthalmic Physiol Opt. 2022;42: 1304–1315. pmid:35908186
- 6. Silva L de LT, Dias FC de S, Maforte NTP, Menezes AC. Patient safety in Primary Health Care: Perception of the nursing team. Esc Anna Nery. 2022;26: e20210130.
- 7. Ricci-cabello I, Avery AJ, Reeves D, Kadam UT, Valderas JM. Measuring Patient Safety in Primary Care: The Development and Validation of the “Patient Reported Experiences and Outcomes of Safety in Primary Care” (PREOS–PC). Ann Fam Med. 2016;14: 253–261. pmid:27184996
- 8. Lawati MHA, Dennis S, Short SD, Abdulhadi NN. Patient safety and safety culture in primary health care: a systematic review. BMC Fam Pract. 2018;19: 104. pmid:29960590
- 9.
Auraaen A, Slawomirski L, Klazinga N. The economics of patient safety in primary and ambulatory care: Flying blind. OECD Heal Work Pap. 2018; OECD Publishing. https://doi.org/10.1787/baf425ad-en
- 10. Kuriakose R, Aggarwal A, Sohi RK, Goel R, NC R, Gambhir RS. Patient safety in primary and outpatient health care. J Fam Med Prim Care. 2020;9: 7–11. pmid:32110556
- 11. Lygre RB, Thuen VM, Gjestad R, Norekvål TM, Greve G, Mildestvedt T, et al. How can we improve specialist health services for children with multi-referrals? Parent reported experience. BMC Health Serv Res. 2020;20: 786. pmid:32831078
- 12. Rodríguez-Eguizabal E, Oliván-Blázquez B, Coronado-Vázquez V, Sánchez-Calavera Ma. A, Gil-de-Goméz Ma J, Lafita-Mainz S, et al. Perception of the primary health care response capacity by patients with and without mental health problems, and health professionals: qualitative study. BMC Health Serv Res. 2021;21: 285. pmid:33784998
- 13. Sundler AJ, Darcy L, Råberus A, Holmström IK. Unmet health-care needs and human rights-A qualitative analysis of patients’ complaints in light of the right to health and health care. Heal Expect. 2020;23: 614–621. pmid:32069375
- 14. Ricci-Cabello I, Gonçalves DC, Rojas-Garcóa A, Valderas JM. Measuring experiences and outcomes of patient safety in primary care: A systematic review of available instruments. Fam Pract. 2015;32: 106–119. pmid:25192905
- 15. Ricci-Cabello I, Valderas JM, Reeves D. Patient Reported Experiences and Outcomes of Safety in Primary Care (PREOS–PC). 2018 [cited 12 Feb 2023]. Available: https://innovation.ox.ac.uk/outcome-measures/patient-reported-experiences-outcomes-safety-primary-care-PREOS–PC/
- 16. Ricci-Cabello I, Pons-Vigués M, Berenguera A, Pujol-Ribera E, Slight SP, Valderas JM. Patients’ perceptions and experiences of patient safety in primary care in England. Fam Pract. 2016;33: 535–542. pmid:27312563
- 17. Oxford University Innovation Limited. Translation and cross-cultural adaptation of the “Patient Reported Experiences and Outcomes of Safety in Primary Care” (PREOS-PC) questionnaire(s) for use in other languages/ healthcare systems. 2018 [cited 12 Feb 2023]. Available: https://innovation.ox.ac.uk/wp-content/uploads/2018/09/PREOS-PC-Clinical-Outcomes-at-Oxford-University-Innovation-Translation-and-Linguistic-Validation-Process_June2018.pdf
- 18. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). 2000;25: 3186–91. pmid:11124735
- 19.
Beaton D, Bombardier C, Guillemin F, Ferraz MB. Recommendations for the Cross-Cultural Adaptation of the DASH & QuickDASH Outcome Measures. Toronto, Canadá: Institute for Work & Health; 2007 [cited 18 Feb 2023]. Available: https://dash.iwh.on.ca/sites/dash/files/downloads/cross_cultural_adaptation_2007.pdf
- 20.
McDonald RP. Test theory: A unified approach. 1st ed. Mahwah: Lawrence Erlbaum Associates Publishers; 1999.
- 21. Polit DF, Beck CT. The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Res Nurs Health. 2006;29: 489–497. pmid:16977646
- 22.
Instituto Brasileiro de Geograria e Estatística. IBGE Cidades. In: Brasília-DF: IBGE [Internet]. 2021 [cited 18 Feb 2023] p. 1. Available: https://cidades.ibge.gov.br/
- 23.
Ministério da Saúde. Portaria número 2.436, de 21 de setembro de 2017. Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes para a organização da Atenção Básica, no âmbito do Sistema Único de Saúde (SUS). In: Ministério da Saúde. Gabinete do Ministro [Internet]. Brasília-DF: Ministério da Saúde; 2017 [cited 18 Feb 2023]. Available: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2017/prt2436_22_09_2017.html
- 24.
Ministério da Saúde. e-Gestor Atenção Básica—Informação e Gestão da Atenção Básica. In: Ministério da Saúde [Internet]. 2020 [cited 18 Feb 2023] p. 1. Available: https://egestorab.saude.gov.br/paginas/acessoPublico/relatorios/relHistoricoCoberturaAB.xhtml
- 25. Alexandre NMC, Coluci MZO. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Cien Saude Colet. 2011;16: 3061–3068. pmid:21808894
- 26. Ricci-Cabello I, Marsden KS, Avery AJ, Bell BG, Kadam UT, Reeves D, et al. Patients’ evaluations of patient safety in English general practices: A cross-sectional study. Br J Gen Pract. 2017;67(660): e474–e482. pmid:28583945
- 27. Masnoon N, Shakib S, Kalisch-Ellett L, Caughey GE. What is polypharmacy? A systematic review of definitions. BMC Geriatr. 2017;17: 1–10. pmid:29017448
- 28.
R Core Team. R: A language and environment for statistical computing. In: Vienna, Austria: R Foundation for Statistical Computing [Internet]; 2022 [cited 18 Feb 2023]. Available: https://www.r-project.org/
- 29.
Byrne B. Structural equation modeling with AMOS: basic concepts, applications and programming. 2nd ed. New York: Routledge; 2010. https://doi.org/10.4324/9780203805534
- 30. Mardia K V. Measures of multivariate skewness and kurtosis with applications. Biometrika. 1970;57: 519–530.
- 31. Muthén B, Kaplan D. A comparison of some methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985;38: 171–189.
- 32. Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Electron J Bus Res Methods. 2008;6: 53–60.
- 33. Hu L, Bentler PM. Cutoff criterion for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J. 1999;6: 1–55.
- 34. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66: 507–514.
- 35. Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL. Análise multivariada de dados. 6th ed. Porto Alegre: Bookman; 2009.
- 36. DiStefano C. The impact of categorization with confirmatory factor analysis. Struct Equ Model A Multidiscip J. 2002;9: 327–346.
- 37. Erdmann AL, Regina De Andrade S, Lúcia A, Ferreira De Mello S, Drago LC. Secondary Health Care: best practices in the health services network. Rev Lat Am Enfermagem. 2013;21(spe): 131–9. Available from: pmid:23459900
- 38. Khalil-Khan A, Khan MA. The Impact of COVID-19 on Primary Care: A Scoping Review. Cureus. 2023;15(1): e33241. pmid:36618499
- 39.
Devellis RF. Scale Development: Theory and Applications. 4th ed. Chapel Hill: Sage Publications (CA); 2017.
- 40. Willmer M, Westerberg Jacobson J, Lindberg M. Exploratory and Confirmatory Factor Analysis of the 9-Item Utrecht Work Engagement Scale in a Multi-Occupational Female Sample: A Cross-Sectional Study. Front Psychol. 2019;10: 1–7. pmid:31866921