Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Family vulnerability scale: Evidence of content and internal structure validity

  • Evelyn Lima de Souza ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    evelyn.lima@einstein.br

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

  • Ilana Eshriqui,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

  • Flávio Rebustini,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Department of Gerontology, School of Arts, Sciences and Humanities (EACH), University of São Paulo, São Paulo, São Paulo, Brazil

  • Eliana Tiemi Masuda,

    Roles Formal analysis, Investigation, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

  • Francisco Timbó de Paiva Neto,

    Roles Formal analysis, Investigation, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

  • Ricardo Macedo Lima,

    Roles Data curation, Formal analysis, Investigation, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

  • Daiana Bonfim

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Hospital Israelita Albert Einstein, São Paulo, São Paulo, Brazil

Abstract

Introduction

Territory view based on families’ vulnerability strata allows identifying different health needs that can guide healthcare at primary care scope. Despite the availability of tools designed to measure family vulnerability, there is still a need for substantial validity evidence, which limits the use of these tools in a country showing multiple socioeconomic and cultural realities, such as Brazil. The primary objective of this study is to develop and gather evidence on the validity of the Family Vulnerability Scale for Brazil, commonly referred to as EVFAM-BR (in Portuguese).

Methods

Items were generated through exploratory qualitative study carried out by 123 health care professionals. The data collected supported the creation of 92 initial items, which were then evaluated by a panel of multi-regional and multi-disciplinary experts (n = 73) to calculate the Content Validity Ratio (CVR). This evaluation process resulted in a refined version of the scale, consisting of 38 items. Next, the scale was applied to 1,255 individuals to test the internal-structure validity by using the Exploratory Factor Analysis (EFA). Dimensionality was evaluated using Robust Parallel Analysis, and the model underwent cross-validation to determine the final version of EVFAM-BR.

Results

This final version consists of 14 items that are categorized into four dimensions, accounting for an explained variance of 79.02%. All indicators were within adequate and satisfactory limits, without any cross-loading or Heywood Case issues. Reliability indices also reached adequate levels (α = 0.71; ω = 0.70; glb = 0.83 and ORION ranging from 0.80 to 0.93, between domains). The instrument scores underwent a normalization process, revealing three distinct vulnerability strata: low (0 to 4), moderate (5 to 6), and high (7 to 14).

Conclusion

The scale exhibited satisfactory validity evidence, demonstrating consistency, reliability, and robustness. It resulted in a concise instrument that effectively measures and distinguishes levels of family vulnerability within the primary care setting in Brazil.

Introduction

The term vulnerability is at the core of the debate in different knowledge fields; etymologically it comes from Latin “vulnerare” (harm, impair) and “bile” (susceptible) [1]. However, the understanding about vulnerability shows variations when different dimensions linked to this topic are considered [2]. Vulnerability, in the Bioethics field, refers to being in danger or at risk due to individual weakness, which is a feature inherent to humans [3]. As for the healthcare field, this term has a broader meaning. It is linked to the recognition that humans are vulnerable to harm or risks within the health/illness continuum as a result of social disadvantages [4].

Although the literature presents different definitions for vulnerability and assesses individual predictors, the concept of family vulnerability is measured in different ways [4]. It is crucial to consider this phenomenon from multiple perspectives that are interconnected with the health needs of individuals within a given family. Factors such as the health status of family members, the community environment, and the social context should all be taken into consideration when examining vulnerability within families [5,6].

Thus, health services providers and managers must consider family vulnerability in order to structure healthcare practices, particularly from a population perspective [7]. In line with this, adopting a territorial perspective that considers vulnerability strata can support the identification of health needs in various population groups [2]. Besides, family vulnerability stratification is essential to consider when planning the services to offer in a given territory, since it may help achieving equity and qualified population-based care management. Therefore, in order to align the activities of healthcare teams and the order of care provision within the Health Care Network, it is essential to further explore the elements that contribute to family vulnerability. This deeper understanding will enable a more comprehensive response to the demands arising from family vulnerability.

There is currently a scarcity of tools and initiatives that specifically address the incorporation of aspects related to family vulnerability as integral components within labor process organizations. Some global experiences are linked to this phenomenon: vulnerability-measuring background is in the United Nations Program for Development (UNPD). In 2004, the Disaster Risk Index (DRI) was created to measure and compare countries within a process based on physical, social, economic, and environmental factors [8]. Subsequently, in 2006, the Autonomous University of Madrid (Spain) conducted an estimation of the vulnerability levels among citizens who were assumed to be at risk of lacking social protection. This estimation was based on countries that are part of the Organization for Economic Cooperation and Development (OECD) [9]. In Latin America, one research aimed at measuring family vulnerability rates in a Colombian municipality based on a sample of families from all socioeconomic strata living in urban and rural zones [10].

Other initiatives have aimed at developing tools to be used by primary healthcare (PHC) teams to measure family vulnerability and, consequently, to contribute to plan healthcare provision in the Brazilian territory [1115]. However, the tools so far developed and used for such a purpose still need robust evidence of their validity, as they present limitations to be used in a country with continental dimensions, and multiple socioeconomic and cultural realities like Brazil.

Therefore, it is crucial to contemplate the concept of family vulnerability, with a particular emphasis on diverse populations. From this perspective, there is a need to develop an instrument that can be utilized in a standardized manner at a national level. Therefore, the present study aims to seek validity evidence about the content and internal structure of the Family Vulnerability Scale for Brazil (EVFAM-BR, in Portuguese).

Materials and methods

The present study followed a psychometric design to gather evidence on both content validity (stage 1) and internal structure (stage 2), based on current recommendations by the Educational Research Association (AERA), the American Psychological Association (APA) and the National Council on Measurement in Education (NCME) [16]. The study was approved by the Ethics Research Committee of Hospital Israelita Albert Einstein on October 22, 2019 (n° 3.674.106, CAAE 12395919.0.0000.0071).

Stage 1: Content validity evidence

PHC professionals (such as physicians, nurses, nursing assistants, community health agents, oral health team and multi-professional team, as psychologists, social workers, speech therapists, physiotherapists, nutritionists, and pharmacists) from all geographic regions in Brazil were invited to join the first qualitative exploratory stage of the study to define the concept of “family vulnerability” and to identify factors likely associated with it. The insights gathered from this stage were used to inform the development of items for the instrument. This effort was done to identify different views of family vulnerability in different geographic regions countrywide.

The invitation was conducted using a snowball sampling method [17], through an online questionnaire on the RedCap® electronic tool [18,19] sent via WhatsApp and email. After reading the Informed Consent Form (ICF) and providing consent to participate in the study, participants were given access to a semi-structured questionnaire.

The questionnaire comprised (i) respondents’ socioeconomic, demographic and health profile identification, (ii) open questions about the concept of vulnerability and scale applicability, and (iii) multiple-choice questions about the relevance of measuring family vulnerability. The multiple-choice questions were distributed into 46 items elaborated from individual and domestic registration forms used at the Brazilian public healthcare system through e-SUS PHC software [20,21]. Frequencies of responses for items expressed in multi-choice questions and the group of aspects identified in open questions were taken into consideration according to the Content Analysis Technique, in order to elaborate the first version of identified items [22]–they were developed as questions to allow dichotomous answers (“0 –no” and “1 –yes”).

Items developed from the previous stage were subjected to an extensive panel of multi-regional and multi-disciplinary judges. The panel encompassed health professionals, scholars and psychometrists who were invited to join the study through the snowball method and like previously, an online RedCap® questionnaire with ICF was used.

Registered participants accepted to participate and signed an ICF, but did not access the questionnaires related to the family vulnerability concept in the first qualitative exploratory stage or the questionnaire used to evaluate the items in the initial version of the instrument during the panel of judges were excluded from the study.

The large number of judges was explained by the need to adjust and apply the instrument at national level. The judges evaluated the items in the first version of the instrument based on criteria of relevance and clarity, and were also asked whether there was a need to change the wording of any item.

Then, a choice was made to apply the Content Validity Ratio (CVR) [23] as the validity index to select the items. This choice was made because CVR is calculated based on the number of judges in the panel [24,25] and therefore more judges could be included. CVR was initially applied to assess the relevance of a given item in order to check whether it effectively measures the latent variable: family vulnerability. CVR was represented as CVR-1 (the Item’s CVR) and CVR-E (Scale CVR)–this last one corresponds to mean recorded for the CVR criteria. It is important to highlight that a modified CVR version with two points, namely “no” and “yes”, was adopted. The original version had three points and did not have an effective practical effect, since, for CVR calculation purposes, it turned dichotomous. This procedure has been adopted in other studies [26,27].

Oftentimes, the mean recorded for the assessed questionnaires was adopted; therefore, a hierarchic flow was adopted, since other indicators, such as item’s clarity, were only analyzed after judges showed its relevance to testify that the item actually assessed the instrument’s latent variable. The application of requirements in the same stage tends to inflate mean CVR and to launch items that do not measure the latent variable in the next stage. Accordingly, as pointed out by DeVellis [28], a given item can be relevant, but its words might be problematic. Thus, the second stage refers to items’ clarity (whether the item is well written in terms of its semantics). The third stage assessed the need of changing the items’ writing. The mean recorded for CVR was only applied to items that adhered to the phenomenon.

Stage 2: Evidence about internal structure validity

The version subjected to content evidence was applied to users of PHC services to find evidence of internal structure validity.

All data collectors were previously trained and received guidance about the ICF application, as well as the research aim, methodology and questions. Study presentation and data collection flow were also previously carried out with teams from the participating PHC services.

PHC services selection was based on municipalities with the largest population in which the methodology of Health Care Planning was being implemented [29]. This was done by including at least one PHC service from each geographic region in the country. Thus, data collection was carried out in 11 PHC services: one in Northern Brazil (Roraima State), one in the Northeastern region (Pernambuco State) and two in Midwestern Brazil (Mato Grosso State), five in the Southeastern region (São Paulo and Minas Gerais states) and two in Southern Brazil (Paraná State).

Due to the Covid-19 pandemic, during the first data collection period (June to November 2020), interviews were conducted via telephone contact with healthcare users in São Paulo. This was done with the consent of the service management, who provided the identifying information of individuals within the territory. As for the second data collection time (from May to August 2022), users who had attended the participating PHC services at data-collection day were asked to join the study.

PHC users over 18-years-old were invited to participate. The Informed Consent Form approved by Hospital Israelita Albert Einstein in Ethics Research Committee was applied by the data collector. Verbal acceptance or refusal was registered using the RedCap® software on a tablet device and the participant received a printed copy of the ICF. Next, the structured questionnaire about the family vulnerability scale was applied and registered offline using the RedCap® installed on a tablet device. At the end of each day, information registered on each tablet was uploaded to the RedCap® server.

PHC users who accepted to participate according to the ICF but had missing information regarding the family vulnerability scale were excluded.

Statistical analysis

Exploratory factor analysis.

The first stage of the analysis aimed at assessing whether the collected data were prone to factorial through Measure of Sampling Adequacy (MSA). Bartlett sphericity, which is determinant of the matrix and Kaiser-Meyer-Olkin (KMO), were assessed at this stage. Besides assessing the dataset items, an individual analysis was also assessed, as recommended by Lorenzo-Seva and Ferrando [30]. The inadequacy of items to be factored can affect model solution. Missing data were treated with the multiple imputation technique [31].

Dimensionality testing was carried out through Parallel Analysis, based on Optimal implementation of Parallel Analysis (PA) and Minimum rank factor analysis to minimize the common variance of residues [32]. PA was implemented through permutation with 500 random matrices. Dimensionality in exploratory factorial analysis (unrestricted model) was tested through Parallel Analysis, which has been considered more robust and accurate to test it [3337].

Tetrachoric matrix estimates were carried out through Bayes Modal Estimation [38], with Smoothing Ridge [39]. The use of tetrachoric/polychoric correlations tends to increase the model’s accuracy in comparison to Pearson’s correlation [40,41].

Factors’ extraction was performed through the RULS technique (Robust Unweighted Least Squares), which reduces the residues in matrices that are more robust in terms of abnormal data [42]. Promin oblique rotation would be used in case the instrument emerged as multi-dimensional [43].

UNICO (Unidimensional Congruence > 0.95), ECV (Explained Common Variance > 0.80 –Quinn, 2014) and MIREAL (Mean of Item Residual Absolute Loadings < 0.30) were adopted as unidimensionality assessment indicator [44].

Quality parameters of the instrument.

The tool variance must be close to 60% [45]. Initial factorial load of 0.30 is recommended when the sample comprises less than 300 individuals [45]; communities must present values higher than 0.40 [46].

The maintenance or removal of the item in the model will depend on the magnitude of the factorial loads, the communality, the absence of cross-loading, Heywood cases, and the interpretability of the factors.

The unique directional correlation (Eta) through Pratt’s Measure was adopted to increase the accuracy of decision-making about the retention or removal of a given item [47,48].

Reliability.

Reliability was measured through four indicators: Cronbach’s alpha [49], Greatest Lower Bound–glb [50], Omega [51]—all three by means of Bayesian estimates—and ORION (Overall Reliability of Fully-Informative prior Oblique N-EAP scores) [52].

Cross-validation was applied to increase the model’s reliability and replicability; the Houdolt technique was also applied [53]. This technique divides the dataset into a training sample—that can range from 10%, 30% to 50%—and into a dataset known as test dataset [53]. The dataset in the present study was split in half by randomly choosing the items. The Solomon technique [54] was adopted, so that dataset division could be random and respect factorability’s equivalence. The datasets were labeled as follows: Full Sample (FS; n = 1,255); Training Sample (TrS n = 627) and Test Sample (TsS; n = 628). According to Brown [55], cross-validation can be carried out either through EFA or Confirmatory Factor Analysis (CFA). FS analysis will only take place if the model found in TrS and TsS can be replicated. This procedure was already adopted in previous studies [56,57], and it follows current recommendations [58].

Descriptive study and standardization.

An exploratory descriptive study of general scores recorded for the Family Vulnerability Scale was carried out after a solution for the internal structure was found. Results recorded for the items and for total score were represented by the frequency of answer, median (Md), interquartile interval (IIQ), amplitude (amp), minimum (min) and maximum (max) value.

In the first stage, standardization was performed by identifying score cutoffs based on the distribution of participants. Despite this process, the fact that participants’ distribution is recurrent in standardization studies can lead to distortions, because the score is not directly analyzed, but considered according to participants’ positions according to cutoff points. To achieve greater precision regarding the proposed cutoffs in the ranges and to assess the predictive capacity of individual classification, a discriminant analysis was used for each of the ranges and the scores of the Family Vulnerability Scale.

Discriminant analysis of each one of the limits and Family Vulnerability Scale scores were used to improve the accuracy of proposed cutoffs (within the limit) and to assess the predictive ability to classify the individuals. The discriminant analysis aims to allow a better understanding of group differences and to predict the probability of an entity (individual or object) to perceive a specific class or group, based on several independent variables of the metrics [59]. Boedeker and Kearns [60] identified a better performance using the discriminating analysis, in comparison to many other techniques applied for the same purpose. Besides, it allows determining the independent variables that have the most significant impact on the differences observed in the mean score profiles in two or more groups [59]. Tabachnick and Fidell [61] added to this information by stating that the aim of the discriminating analysis is to predict the group’s participation based on a set of predictors. Accordingly, it is possible to confirm whether the groups formed from the distribution process have properly classified individuals within the established limits.

Data were analyzed in statistic software Factor 12.01.01, SPSS v.23 and JASP 16.04.

Results

Content validity evidence

A total of 123 professionals from five Brazilian regions joined the first stage of the research to define the concept of “family vulnerability”: 48.8% of them came from Northeastern Brazil; 21.1% from Southeastern Brazil; 17.9% from Southern Brazil, 8.9% from Northern Brazil and 3.3% from Midwestern Brazil. Most professionals were female (82.9%), approximately 40% of them had at least 10-year experience in PHC and 48.8% reported having a specialization degree as higher education level. Most participation in this stage were nurses (48.3%) and community health agents (10.7%). Based on the responses of the factors that could potentially be associated with the concept of family vulnerability, the first version of the instrument was developed, containing 92 items.

A panel of multi-regional and multi-disciplinary judges was set for the second stage; it aimed at identifying content validity evidence. This panel had 73 judges: 61.7% from Southeastern Brazil, 15.1% from Southern Brazil, 9.6% Northeastern Brazil, 6.8% from Northern Brazil and 6.8% from Midwestern Brazil. Most of them were female (79.5%), and had at least 10-year experience in PHC (57.5%) and a specialization degree as higher education level (51.4%). Most in the panel were nurses (50.7%) and physicians (16.55). CVR was applied to judges’ answers. CVR critical value was established at CVR > 0.12, which was defined based on the participation of 73 judges.

CVR calculation led to rule out 54 items resulting in a scale with 38 items that are related to socioeconomic and demographic aspects, access to healthcare services, health condition, and lifestyle. To make the scale even clearer, 12 of the 38 items were rewritten based on recommendations from the panel of judges. Only items 2 and 15 (Table 1) recorded a CVR lower than the critical value; therefore, their text was revised. The other 10 items did not suffer any changes; writing adjustments were made in the original text just to meet CVR’s critical value.

thumbnail
Table 1. Items that remained in the scale after applying the Content Validity Ratio (CVR).

https://doi.org/10.1371/journal.pone.0280857.t001

The version containing evidence of content validity (38 items) was considered in the stage to demonstrate internal structure validity.

Evidence about internal structure validity

In total, 1,584 users who attended the 11 PHC services during data collection were invited to join this research stage. However, only 1,505 (95%) of them accepted the invitation and signed the informed consent form. Only 1,255 completed the interview for the application of the scale version presenting content validity evidence. This sample represents the study final sample and this version presents content validity evidence. Table 2 introduces participants descriptions of this study stage.

Factorability.

The evaluation of sample adequacy measures is the first step in the factorability analysis; it aims at assessing dataset factorability and whether factorial analyses are applicable. General database data have shown good factorability: Fs recorded KMO (0.75), Bartlett Sphericity = 6,084.6 (df = 91; P < 0.0001) and determinant of the matrix = 0.00001. As for TrS: KMO (0.74), Bartlett Sphericity = 6,153.7 (df = 91; P < 0.0001) and determinant of the matrix = 0.00001; TsS: KMO (0.71), Bartlett Sphericity = 2,617.3 (df = 91; P < 0.0001) and determinant of the matrix = 0.0002. Although the general indices presented good indicators, 4 of the 38 initial items have shown factorability issues in three datasets (27—Does anyone in your household have HIV/AIDS?; 36—Are there children in your household out of school?; 37—Do you have any adolescent in your household out of school?; and 38—Are there under 14-year-old individuals in your household who have a job?). They were excluded from the analyses based on recommendations by Lorenzo-Seva and Ferrando [30].

Dimensionability.

The first analyses were carried out in TrS. Dimensionality analyzed through parallel analysis pointed towards a 4-dimension model. Closeness of dimensionality values kept the indication for multi-dimensional model: Single = 0.82; ECV = 0.65 and MIREAL = 0.37. Thirteen (13) of the 34 items forming the initial analysis did not present substantial factorial load in the model. Accordingly, the process to remove items to adjust the model followed two principles: quantitative (statistical adjustment) and qualitative (interpretability) adjustment. The choice for removing an item was carried out by taking into consideration the set of primary indicators: factorial load, communality, Eta of Pratt’s Importance Measure, existence of cross-loading, Heywood case and model adjustment indices. The items were removed from the scale until the two principles were congruent, resulting in a model for 4-dimension TrS with 14 items showing appropriate statistical fit and interpretability.

The parallel analysis kept on pointing out a 4-dimension solution and explained variance of 78.66%. This model was replicated in TsS and FS. Both datasets confirmed the 4-dimension model; furthermore, the closeness of dimensionality values reinforced the multi-dimensional model (Table 3) for the three datasets.

TrS primary data (Table 4) presents factorial data ranging from 0.597 to 0.975, communality ranging from 0.449 to 0.967, and Eta ranging from 0.640 to 0.941. The model recorded explained variable of 76.18%. Items 1 and 3 comprised the dimension Income, items from 4 to 8 comprised the dimension Healthcare, dimension Family was in items 9 to 11, and the single dimension Violence was observed in items 12 to 14. Accordingly, the model points towards good factorial and interpretable (quantitatively) solution, with content alignment in coherent and interpretable (quantitatively) items. The final version of the scale represented reduction by approximately 63% in the 38 items assessed through judges’ panel in the first stage. This value is close to that presented by DeVellis [28], according to whom the researcher must project items lost by 50% throughout the process.

thumbnail
Table 4. Training dataset: Factorial loads, communality, and Eta.

https://doi.org/10.1371/journal.pone.0280857.t004

Based on the four dimensions forming the scale, it is possible to determine the key role played by social determinants within the health/illness process. Dimensions embody items related to income, social and family cohesion, and to life and housing conditions associated with psychosocial and behavioral aspects [62]. Thus, they extrapolate the biological view of health, which is overall acknowledged in a reductionist way, focused at the medical practices [6265]. Nevertheless, the present instrument emerges as a multi-disciplinary work-tool available for PHC teams that have the potential to promote social justice by taking into consideration social inequities when planning healthcare services.

Factorial loads recorded for the TsS dataset (Table 5) ranged from 0.643 to 0.976, communality ranged from 0.482 to 0.910 and Eta ranged from 0.658 to 0.951. This model presented explained variance of 76.18%. Once again, the observed model was equal to the training dataset model; consequently, it could be interpreted quantitatively and qualitatively.

thumbnail
Table 5. Test dataset: Factor loading, communality and Eta.

https://doi.org/10.1371/journal.pone.0280857.t005

In the FS dataset (Table 6), these results ranged from 0.308 to 0.785 for factorial loads, from 0.212 to 0.967 for communality, and Eta values from 0.640 to 0.941. The model using the entire sample achieved an explained variance of 79.02%. An essential aspect is the stability in the number of dimensions and the interpretability of the model, which reinforces the importance of conducting cross-validation.

thumbnail
Table 6. Full dataset: Factor loading, communality, and Eta.

https://doi.org/10.1371/journal.pone.0280857.t006

The reliability indices across analysis datasets ranged from 0.69 to 0.71 for Cronbach’s alpha, reaching 0.70 in all three datasets for Omega. Additionally, it ranged from 0.83 to 0.84 for glb and from 0.80 to 0.96 for ORION, indicating strong consistency between dimensions and datasets. The quality of the factorial solution indices also demonstrated adequate levels, further reinforcing the stability of the model (Table 7). Consequently, the combination of applied techniques and indices provided strong evidence of internal structure validity that is appropriate, consistent, robust, and interpretable.

Normalization.

Once the internal structure of the instrument was established and validated, we proceeded with the descriptive study and normalization of scores to enable the interpretability of the instrument and proper classification of participants’ scores. In this regard, Table 8 displays the response frequency for each questionnaire item. There is a clear prevalence of respondents selecting "No" for all items of the instrument. Some items had a higher frequency of "Yes" responses, such as "Is money short to meet household needs?" with 40.34%, "Does anyone in the household use medication?" with 36.41%, "Does anyone in the household have a health condition that requires continuous care?" with 36.77%, and "Did anyone in the household have an absent father during childhood?" with 34.07%.

Table 9 presents the scores recorded for the dimensions and the general score of the Family Vulnerability Scale. All dimensions had all their amplitudes answered. Dimensions Income, Family and Violence recorded median = 0, Healthcare showed median = 1 and total score recorded median = 2. An interesting aspect of the total score lies on the fact that the amplitude ranged from 0 to 14 and the maximum score recorded in the current sample reached 12. Medians in the minimum limit, and close to it, had previously indicated that the instrument could accurately differentiate individuals eventually facing family vulnerability situations.

thumbnail
Table 9. Description of dimensions and scores recorded for the Family Vulnerability Scale.

https://doi.org/10.1371/journal.pone.0280857.t009

Because these scores are closer to the bottom limit, they started to be impactful only when they move away from the median that, in this case, was close to the bottom limit and thus in the upper quartile. Thus, three initial classifications were suggested: model 1 had a cutoff in the median (low and high vulnerability), model 2 scores were separated up to percentile 75, from 76 to 89, and 90 and above; model 3 scores were separated up to percentile 70, from 70 to 89, or higher.

The discriminant analysis of each classification developed to assess whether it was possible to accurately identify participants within the limit was applied after the first cutoffs were made.

The first analysis adopted a binary classification (low and high). The discriminant analysis showed MBox = 446.58 p < 0.001. λwilks = 0.36; F(1, 1257) = 1,268.77; p < 0.001; canonical correlation = 0.797; the model with two limits properly classified 85.7% of the cases. The discriminant analysis applied to model 2 was MBox = 49.64 p < 0.001. λwilks = 0.18; F(2, 1256) = 2,094.25; p < 0.001; canonical correlation = 0.907. Model 2 properly classified 100% of cases. The analysis applied to model 3 presented MBox = 49.64 p < 0.001. λwilks = 0.25; F(2, 1256) = 1,838.71; p < 0.001; canonical correlation = 0.863; it was possible to properly classify 89% of the cases. The recommended classification and scores interpretations are depicted in Table 10.

thumbnail
Table 10. Limits, classification, and interpretation of Family Vulnerability Scale scores.

https://doi.org/10.1371/journal.pone.0280857.t010

Family Vulnerability Scale for Brazil–final version

The final version of the Family Vulnerability Scale for Brazil (EVFAM-BR) comprised 14 items (or questions) applied to each family in the PHC territory in Brazil, due to the action by community health agents. EVFAM-BR has four dimensions; each one has a score corresponding to the number of items in the dimension; at the end of its application, the score must range from zero (0) to fourteen (14). Healthcare is the dimension accounting for the largest number of items; consequently, it has the greatest scoring potential on the scale (n = 5).

Through the sum of scores for each dimension, the final score of EVFAM-BR presents three classification ranges of family vulnerability: Low (0 to 4), Moderate (5 to 6), and High (7 to 14). Table 11 (in English) and Table 12 (in Portuguese) summarize the EVFAM-BR, with their respective dimensions, items, and scores.

thumbnail
Table 11. Family Vulnerability Scale for Brazil (EVFAM-BR).

https://doi.org/10.1371/journal.pone.0280857.t011

thumbnail
Table 12. Family Vulnerability Scale for Brazil (Escala de Vulnerabilidade Familiar, EVFAM-BR, in Portuguese).

https://doi.org/10.1371/journal.pone.0280857.t012

The final version for application of the EVFAM-BR is presented as S1 File (in English) and S2 File (in Portuguese).

Discussion

The Family Vulnerability Scale for Brazil (EVFAM-BR) demonstrated evidence of content validity and internal structure in a multi-regional and multi-professional context, allowing for the measurement of family vulnerability in Brazil.

The EVFAM-BR consists of 14 items distributed across the dimensions: Income, Healthcare, Family, and Violence. It is worth noting that the EVFAM-BR aims to measure social vulnerability within the family context, and as such, all items refer to the family nucleus, rather than being directed to a specific household member or respondent.

Next is an example of the EVFAM-BR applied to a family living in a given local and time: one of the household residents in a given house is facing financial difficulties. However, lack of money to meet household needs is not identified as an issue and there is no difficulties in having access to different food types; one of the household residents has a chronic disease that requires continuous care and needs controlled medication (less than five medication types daily), but none of the residents has any difficulties in performing daily activities and does not need help to perform daily healthcare procedures; none of the residents had either an absent mother or father during childhood and no family member was abandoned by the family; there was no violence in the house and no one in the house was coexisting with violent people, but one of the household residents was the victim of violence. Given the positive answers to items related to financial difficulties by one of the household residents (1), a health condition requiring continuous care (1), use of medication (1) and a person who was the victim of violence (1), this family would reach score 4, which according to the EVFAM-BR it represents a family classified as “low family vulnerability”.

The literature consistently states that income is a relevant social health determinant that must be taken into account to allow planning an equitable healthcare provision [66]. More recent discussions about this topic led to different views on how income inequality affects health. Rich or poor individuals, in societies with low social cohesion, may become susceptible to a range of issues, including crime, insufficient public investments, and the adoption of unhealthy behaviors like smoking, excessive alcohol consumption, and sedentary lifestyles [67]. These outcomes can help us better understand the relationship between income-linked variables, facilitating interventions at macroeconomic level, and evaluating shifts in population health.

The healthcare dimension is timely, given the rapid aging of the population in Brazil and worldwide, a fact that demands reorganization of healthcare services in order to continuously meet population needs, in an organized way and based on quality and safety. With respect to the family dimension, several studies have shown the association between the absence of parents and a number of health outcomes, including cognitive development loss [68,69], impact on mental health [70,71], and early development of a health-related risk behaviors such as smoking and alcohol abuse [72].

Finally, the dimension “violence” reaffirms the observation outlined in the WHO’s 2030 agenda for the sustainable development of millennium goals. This agenda highlights violence prevention as a pivotal component for fostering global development and enhancing overall quality of life. It is known that violence has a significant impact on health, extending beyond the initial trauma, exacerbating the likelihood of other significant causes of illness and death [73]. An association among exposure to violence, undesired health outcomes and unhealthy behaviors, such as drug and alcohol abuse, mainly among low-income mothers in urban locations, can be identified [74].

While there is a recognized urgency for research in low and middle-income countries, where 90% of global violence occurs, only 10% of studies on violence are conducted in these contexts [73]. Thus, EVFAM-BR emerges as a tool that enables the structured and routine introduction of social factors known to be associated with the health-disease process in the healthcare services in developing countries.

With the evidence of EVFAM-BR validity and the three proposed strata of family vulnerability resulting from its application, there is potential to contribute to the role of Primary Health Care (PHC) in executing its attributes, particularly in coordinating care from population-based management in the community and family context [75].

It is essential to highlight that the EVFAM-BR is a robust and concise instrument that initially demands a low investment of professional workload and financial resources. Because it is an objective instrument (only “yes” and “no” answers are expected), all PHC professionals must be trained to use it. This instrument emerges as a powerful tool to support the work by community health agents, since these actors are community members and are closely bond to families in the territory; this process makes the “interview environment” more comfortable and trustful for users who answer the questions on behalf of the household. Therefore, EVFAM-BR can be implemented using both printed materials and online platforms, as it serves as a valuable tool integrated into the routine of healthcare teams. Also, there is potential to incorporate EVFAM-BR into the digital registration system of the Brazilian Unified Health System (SUS).

It is possible to prepare the teams at the time to plan their actions and health interventions (based on exposing families to conditions that increase vulnerability and risk to develop illnesses) by applying EVFAM-BR and by interpreting its household-classification results based on the three predicted strata. It is crucial to consider that vulnerability contexts can change overtime, so the instrument periodic application is necessary to keep teams planning updated according to the population’s needs.

Among the limitations of this study is the convenience sampling, which does not allow for statistical representativeness of the results. Nevertheless, the study was conducted in various socioeconomic, demographic, and cultural contexts, including participants from all five Brazilian geographical regions. Additionally, the possibility of memory bias should be mentioned considering that participants answered questions regarding retrospective information on all residents of the household.

As strengths, data collection by external professionals trained to conduct the interviews and the use of robust techniques for identifying evidence of EVFAM-BR validity are noteworthy. One such technique is the Content Validity Ratio (CVR), which is a sophisticated method [76] if compared to other proposed alternatives [77]. CVR calculation takes into consideration the number of judges [24,25] and it minimizes the inflation of chance agreement [24], a fact that allows having more judges to assess the instrument. The adoption of a multi-disciplinary panel of judges comprising of researchers, translators, health professionals, methodology experts and lay people ensures better consistency of the results obtained [7779].

It is important to highlight the need to implement research to identify the potentialities and challenges for the practical use of EVFAM-BR in the routine of PHC services in different Brazilian contexts, as well in countries with similar health systems, considering the socioeconomic, demographic, sanitary, and epidemiological profiles of populations.

Conclusion

Given the techniques here applied and the results obtained, it is possible stating that the set of content validity and EVFAM-BR internal structure are adequate, consistent, reliable and robust. Also, the cross-validation method ensured model reliability and replicability. Here we present a concise scale capable of accurately measuring and distinguishing family vulnerability.

Supporting information

S1 File. Family Vulnerability Scale for Brazil (EVFAM-BR).

https://doi.org/10.1371/journal.pone.0280857.s001

(DOCX)

S2 File. Family Vulnerability Scale (Escala de Vulnerabilidade Familiar, EVFAM-BR, in Portuguese).

https://doi.org/10.1371/journal.pone.0280857.s002

(DOCX)

References

  1. 1. Carmo ME, Guizardi FL. O conceito de vulnerabilidade e seus sentidos para as políticas públicas de saúde e assistência social. Cad. Saude Publica [Internet]. 2018 [cited 2022 Nov 22];34(3):e00101417. https://doi.org/10.1590/0102-311X00101417. Available at: https://www.scielo.br/j/csp/a/ywYD8gCqRGg6RrNmsYn8WHv/abstract/?lang=pt#.
  2. 2. Ferreira JBB, Santos LL, Ribeiro LC, Fracon BRR, Wong S. Vulnerability and Primary Health Care: An Integrative Literature Review. J Prim Care Community Health [Internet]. 2021 [cited 2022 Dec 21];12 Available from: https://pubmed.ncbi.nlm.nih.gov/34654333/. pmid:34654333
  3. 3. Oviedo RAM, Czeresnia D. conceito de vulnerabilidade e seu caráter biossocial. Interface [Internet]. 2015 [cited 2022 Nov 22];19(53):237–250. https://doi.org/10.1590/1807-57622014.0436. Available at: https://www.scielo.br/j/icse/a/5BDdb5z4hWMNn58drsSzktF/abstract/?lang=pt#.
  4. 4. Ayres JRCM , Calazans GJ, Saletti Filho HC, França Junior I. Risco, vulnerabilidade e práticas de prevenção e promoção da saúde. In: Campos GWS, Bonfim JRA, Minayo MCS, Akerman M, Drumond Junior M, Carvalho YM (eds) orgs. Tratado de Saúde Coletiva. Hucitec Editora; 2016;399–442.
  5. 5. Ayres JRCM, França Junior I, Calazans GJ, Saletti Filho HC. O conceito de vulnerabilidade e as práticas de saúde: novas perspectivas e desafios. In: Czeresnia D, Freitas CM, editors. Promoção da saúde: conceitos, reflexões, tendências. Rio de Janeiro: Fiocruz; 2003.; p. 117–139.
  6. 6. Ayres JRCM, Paiva V, França Junior I, Gravato N, Lacerda R, Negra MD, et al. Vulnerability, human rights, and comprehensive health care needs of young people living with HIV/AIDS. Am J Public Health [Internet]. 2006 [cited 2022 Nov 14];96(6):1001–1006. Available at: https://pubmed.ncbi.nlm.nih.gov/16449593/. pmid:16449593
  7. 7. Silva TMR, Alvarenga MRM, Oliveira MAC. Evaluation of the vulnerability of families assisted in primary care in Brazil. Rev Lat Am Enfermagem [Internet]. 2012 [cited 2022 Dec 21];20(5):935–43. Available from: https://pubmed.ncbi.nlm.nih.gov/23174839/ pmid:23174839
  8. 8. Programa de las Naciones Unidas para el Desarrollo (PNUD), editor. La reducción de riesgos de desastres: un desafío para el desarrollo [bibliography on the Internet]. New York: Programa de las Naciones Unidas para el Desarrollo; 2004 [cited 2023 Jun 30]. Available at: https://www.undp.org/sites/g/files/zskgke326/files/publications/es/rdr_esp_reduccion%20de%20riesgos%20de%20desastres.pdf.
  9. 9. Ayuso IA, Vargas EC. Índice de Vulnerabilidad Social en los países de la OCDE. Quivera [Internet]. 2006 [cited 2022 Nov 22];8(2):248–274. Available at: https://www.redalyc.org/pdf/401/40180212.pdf.
  10. 10. Hidalgo-Troya A, Guerrero-Díaz GF, Estupiñan-Ferrín VL, Rocha-Buelvas A. Índice de vulnerabilidad de los hogares en el municipio de Pasto, Colombia, 2012. Cad Saude Publica [Internet]. 2017 [cited 2022 Nov 14];33(3):e00122315. Available at: https://www.scielo.br/j/csp/a/FP5Mfs5srgx6CMKX7hf7GGd/?lang=es. pmid:28380142
  11. 11. Savassi LCM, Lage JL, Coelho FLG. Sistematização de um instrumento de estratificação de risco familiar: Escala de risco familiar de Coelho-Savassi. J Manag Prim Health Care [Internet]. 2012 [cited 2022 Nov 14];3(2):179–185. https://doi.org/10.14295/jmphc.v3i2.155. Available at: https://www.jmphc.com.br/jmphc/article/view/155.
  12. 12. Amendola F, Alvarenga MRM, Gaspar JC, Yamashita CH, Oliveira MAC. Validade aparente de um índice de vulnerabilidade das famílias a incapacidade e dependência. Rev Esc Enferm USP [Internet]. 2011 [cited 2022 Nov 14];45(spe2):1736–1742. https://doi.org/10.1590/S0080-62342011000800017. Available at: https://www.scielo.br/j/reeusp/a/WN3Xh9ySynhWsQmKr6zbQVH/abstract/?lang=pt.
  13. 13. Amendola F, Alvarenga MRM, Latorre MRDO, Oliveira MAC. Development and validation of the Family Vulnerability Index to Disability and Dependence (FVI-DD). Rev Esc Enferm USP [Internet]. 2014 [cited 2022 Nov 14];48(1):82–90. Available at: https://www.scielo.br/j/reeusp/a/GHNyt8QQbG7FxhFNBWzthpB/?format=pdf&lang=en. pmid:24676112
  14. 14. Amendola F, Alvarenga MRM, Latorre MRDO, Oliveira MAC. Índice de vulnerabilidade a incapacidades e dependência (IVF-ID), segundo condições sociais e de saúde. Cien Saude Colet [Internet]. 2017 [cited 2022 Nov 14];22(6):2063–2071. Available at: https://www.scielo.br/j/csc/a/jtCXRphGrsGjLSTpMsqzZjC/?lang=pt pmid:28614524
  15. 15. Associação Brasileira de Empresas de Pesquisa—ABEP [Internet]. [place unknown]; 2018. Critério Brasil 2018; [cited 2022 Nov 14]; Available at: https://www.abep.org/criterio-brasil.
  16. 16. American Psychological Association—APA [Internet]. Washington: American Educational Research Association (AERA), American Psychological Association (APA), National Council on Measurement in Education (NCME); 2014. The Standards for Educational and Psychological Testing; [cited 2022 Nov 14]; Available at: https://www.apa.org/science/programs/testing/standards.
  17. 17. Biernacki P, Waldorf D. Snowball Sampling—Problems and Techniques of Chain Referral Sampling. Sociol Methods Res [Internet]. 1981 [cited 2022 Nov 14];10(2):141–163. Available at: https://journals.sagepub.com/doi/10.1177/004912418101000205.
  18. 18. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform [Internet]. 2009 [cited 2022 Nov 14];42(2):377–381. Available at: https://pubmed.ncbi.nlm.nih.gov/18929686/. pmid:18929686
  19. 19. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O‘Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform [Internet]. 2019 [cited 2022 Nov 14];95:103208. Available at: https://pubmed.ncbi.nlm.nih.gov/31078660/ pmid:31078660
  20. 20. Schönholzer TE, Pinto IC, Zacharias FCM, Gaete RAC, Serrano-Gallardo MDP. Implementation of the e-SUS Primary Care system: Impact on the routine of Primary Health Care professionals. Rev Lat Am Enfermagem [Internet]. 2021 [cited 2022 Nov 15];19(29):e3447. Available at: https://pubmed.ncbi.nlm.nih.gov/34287545/. pmid:34287545
  21. 21. Brazil. Ministry of Health. e-SUS APS [Internet]. Brasília: Ministry of Health; 2022 [cited 2022 Nov 15]. Available at: https://sisaps.saude.gov.br/esus/.
  22. 22. Bardin L. Análise de Conteúdo. 2010th ed. Lisboa: Edições 70; 1977. 225 p. ISBN: 978-972-44-1154-5.
  23. 23. Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–575.
  24. 24. Baghestani AR, Ahmadi F, Tanha A, Meshkat M. Bayesian critical values for Lawshe‘s content validity ratio. Meas Eval Couns Dev [Internet]. 2019 [cited 2022 Nov 15];52(1):69–73. Available at: https://www.tandfonline.com/doi/abs/10.1080/07481756.2017.1308227
  25. 25. Ayre C, Scally AJ. Critical values for lawshe’s content validity ratio: revisiting the original methods of calculation. Meas Eval Couns Dev [Internet]. 2014 [cited 2022 Nov 15];47(1):79–86. Available at: https://www.tandfonline.com/doi/abs/10.1177/0748175613513808.
  26. 26. Ortiz La Banca R, Rebustini F, Alvarenga WA, Carvalho EC, Lopes M, Milaszewski K, et al. Checklists for Assessing Skills of Children With Type 1 Diabetes on Insulin Injection Technique. J Diabetes Sci Technol [Internet]. 2022 [cited 2022 Dec 21];16(3):742–750. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9294572/ pmid:33423543
  27. 27. Palacio DC, Rebustini F, Oliveira DB, Peres Neto J, Barbieri W, Sanchez TP. Dental vulnerability scale in primary health care: evidence of content and structure internal validity. BMC Oral Health [Internet]. 2021 [cited 2022 Dec 21];21(1) Available from: https://pubmed.ncbi.nlm.nih.gov/34454449/. pmid:34454449
  28. 28. DeVellis RF. Scale development: theory and applications. 4th ed. Thousand Oaks: Sage; 2017. 261 p.
  29. 29. PlanificaSUS. e-Planifica [Internet]. São Paulo: PlanificaSUS; 2022 [cited 2022 Nov 15]. Available at: https://planificasus.com.br/.
  30. 30. Lorenzo-Seva U, Ferrando PJ. MSA: The Forgotten Index for Identifying Inappropriate Items Before Computing Exploratory Item Factor Analysis. Methodology [Internet]. 2021 [cited 2022 Nov 15];17(4):296–306. https://doi.org/10.5964/meth.7185. Available at: https://meth.psychopen.eu/index.php/meth/article/view/7185.
  31. 31. Lorenzo-Seva U, Van Ginkel JR. Multiple Imputation of missing values in exploratory factor analysis of multidimensional scales: estimating latent trait scores. Anal. Psicol [Internet]. 2016 [cited 2022 Nov 15];32(2):596–608. Available at: https://scielo.isciii.es/scielo.php?pid=S0212-97282016000200034&script=sci_abstract&tlng=en.
  32. 32. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods [Internet]. 2011 [cited 2022 Nov 15];16(2):209–220. Available at: https://pubmed.ncbi.nlm.nih.gov/21500916/. pmid:21500916
  33. 33. Lim S, Jahng S. Determining the number of factors using parallel analysis and its recent variants. Psychol Methods [Internet]. 2019 [cited 2022 Nov 15];24(4):452. Available at: https://pubmed.ncbi.nlm.nih.gov/31180694/. pmid:31180694
  34. 34. Auerswald M, Moshagen M. How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychol Methods [Internet]. 2019 [cited 2022 Nov 15];24(4):468–491. Available at: https://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000200. pmid:30667242
  35. 35. Dobriban E, Owen AB. Deterministic parallel analysis: an improved method for selecting factors and principal components. J R Stat Soc Series B Stat Methodol [Internet]. 2019 [cited 2022 Nov 15];81(1):163–183. Available at: https://rss.onlinelibrary.wiley.com/doi/abs/
  36. 36. Cho SJ, Li F, Bandalos D. Accuracy of the parallel analysis procedure with polychoric correlations. Educ Psychol Meas [Internet]. 2009 [cited 2022 Nov 15];69(5):748–759. Available at: https://journals.sagepub.com/doi/10.1177/0013164409332229.
  37. 37. Briggs NE, MacCallum RC. Recovery of Weak Common Factors by Maximum Likelihood and Ordinary Least Squares Estimation. Multivariate Behav Res [Internet]. 2003 [cited 2022 Nov 15];38(1):25–56. Available at: https://pubmed.ncbi.nlm.nih.gov/26771123/. pmid:26771123
  38. 38. Choi J, Kim S, Chen J, Dannels S. A Comparison of Maximum Likelihood and Bayesian Estimation for Polychoric Correlation Using Monte Carlo Simulation. J Educ Behav Stat [Internet]. 2011 [cited 2022 Nov 15];36(4):523–549. Available at: https://journals.sagepub.com/doi/10.3102/1076998610381398.
  39. 39. Jöreskog KG, Sörbom D. LISREL V: analysis of linear structural relationships by maximum likelihood and least squares methods. 5th ed. Uppsala, Sweden: University of Uppsala; 1981.
  40. 40. Holgado-Tello FP, Chacón-Moscoso S, Barbero-García I, Vila-Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant [Internet]. 2010 [cited 2022 Nov 15];44(1):153–166. Available at: https://link.springer.com/article/10.1007/s11135-008-9190-y.
  41. 41. Baglin J. Improving your exploratory factor analysis for ordinal data: A demonstration using FACTOR. Practical Assessment, Research & Evaluation [Internet]. 2014 [cited 2022 Nov 15];19(5) https://doi.org/10.7275/dsep-4220. Available at: https://scholarworks.umass.edu/pare/vol19/iss1/5/.
  42. 42. Osborne JW, Banjanovic ES. Exploratory factor analysis with SAS. North Carolina: SAS Institute Inc.; 2016.
  43. 43. Lorenzo-Seva U. Promin: a method for oblique factor rotation. Multivariate Behav Res [Internet]. 1999 [cited 2022 Nov 15];34(3):347–365. Available at: https://www.tandfonline.com/doi/abs/10.1207/S15327906MBR3403_3.
  44. 44. Ferrando PJ, Lorenzo-Seva U. Assessing the quality and appropriateness of factor solutions and factor score estimates in exploratory item factor analysis. Educ Psychol Meas [Internet]. 2018 [cited 2022 Nov 15];78(5):762–780. Available at: https://pubmed.ncbi.nlm.nih.gov/32655169/. pmid:32655169
  45. 45. Hair JF, Babin BJ, Anderson RE, Black WC. Multivariate Data Analysis. 8th ed. [place unknown]: CENGAGE INDIA; 2018. 832 p.
  46. 46. Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation [Internet]. 2005 [cited 2022 Nov 15];10(1):7. https://doi.org/10.7275/jyj1-4868. Available at: https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1156&context=pare.
  47. 47. Wu AD, Zumbo BD, Marshall SK. A method to aid in the interpretation of EFA results: An application of Pratt’s measures. Int J Behav Dev [Internet]. 2014 [cited 2022 Nov 15];38(1):98–110. Available at: https://journals.sagepub.com/doi/abs/10.1177/0165025413506143.
  48. 48. Wu AD, Zumbo BD. Using Pratt‘s Importance Measures in Confirmatory Factor Analyses. J Mod Appl Stat Methods [Internet]. 2017 [cited 2022 Nov 15];16(2):81–98. Available at: https://digitalcommons.wayne.edu/jmasm/vol16/iss2/5/.
  49. 49. Crobach LJ. Coefficient alpha and the internal structure of tests. Psychometrika [Internet]. 1951 [cited 2022 Nov 15];16(3):297–334. Available at: https://link.springer.com/article/10.1007/BF02310555
  50. 50. Woodhouse B, Jackson PH. Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: A search procedure to locate the greatest lower bound. Psychometrika [Internet]. 1977 [cited 2022 Nov 15];42(4):579–591. Available at: https://link.springer.com/article/10.1007/BF02295980.
  51. 51. McDonald RP. Test theory: A unified treatment. 1st ed. [place unknown]: Psychology Press; 2013.
  52. 52. Ferrando PJ, Lorenzo-Seva U. A note on improving EAP trait estimation in oblique factor-analytic and item response theory models. Psicologica [Internet]. 2016 [cited 2022 Nov 15];37:235–247. Available at: https://www.uv.es/revispsi/articulos2.16/7Ferrando.pdf
  53. 53. Gütlein M, Helma C, Karwath A, Kramer S. A Large-Scale Empirical Evaluation of Cross-Validation and External Test Set Validation in (Q)SAR. Mol Inform [Internet]. 2013 [cited 2022 Nov 15];32(5–6):516–528. Available at: https://pubmed.ncbi.nlm.nih.gov/27481669/. pmid:27481669
  54. 54. Lorenzo-Seva U. SOLOMON: a method for splitting a sample into equivalent subsamples in factor analysis. Behav Res Methods [Internet]. 2021 [cited 2022 Nov 15];:1–13. Available at: https://link.springer.com/article/10.3758/s13428-021-01750-y
  55. 55. Brown TA. Confirmatory factor analysis for applied research. 2nd ed. [place unknown]: Guilford publications; 2015. 462 p. ISBN: 9781462515363.
  56. 56. Mialhe FL, Sampaio HAC, Moraes KL, Brasil VV, Rebustini F. Psychometric properties of the Brazilian version of the European Health Literacy Survey Questionnaire short form. Health Promot Int [Internet]. 2022 [cited 2022 Nov 15];37(4):daac130. Available at: https://pubmed.ncbi.nlm.nih.gov/36102478/. pmid:36102478
  57. 57. Mialhe FL, Moraes KL, Bado FMR, Brasil VV, Sampaio HAC, Rebustini F. Psychometric properties of the adapted instrument European Health Literacy Survey Questionnaire short-short form. Rev Lat Am Enfermagem [Internet]. 2021 [cited 2022 Nov 15];29:e3436. Available at: https://pubmed.ncbi.nlm.nih.gov/34231791/. pmid:34231791
  58. 58. Koul A, Becchio C, Cavallo A. Cross-validation approaches for replicability in psychology. Front Psychol [Internet]. 2018 [cited 2022 Nov 15];9:1117. Available at: https://pubmed.ncbi.nlm.nih.gov/30034353/. pmid:30034353
  59. 59. Hair JR, Black WC, Babin BJ, Anderson R, Tathm RL. Multivariate Data Analysis. 7th ed. Upper Saddle River: Pearson Prentice Hall; 2019.
  60. 60. Boedeker P, Kearns NT. Linear Discriminant Analysis for Prediction of Group Membership: A User-Friendly Primer. Adv Methods Pract Psychol Sci [Internet]. 2019 [cited 2022 Dec 21];2(3):250–263. Available. from: https://journals.sagepub.com/doi/full/10.1177/2515245919849378
  61. 61. Tabachnick BG, Fidell LS. Using multivariate statistics. Boston: Pearson; 2019.
  62. 62. Marmot M. Social justice, epidemiology and health inequalities. Eur J Epidemiol [Internet]. 2017 [cited 2022 Nov 22];32(7):537–546. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5570780/#CR28 pmid:28776115
  63. 63. Marmot M. Social determinants of health inequalities. The Lancet [Internet]. 2005 [cited 2022 Nov 22];365(9464):1099–1104. Available at: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(05)71146-6/fulltext. pmid:15781105
  64. 64. Dahlgren G, Whitehead M, compilers. Policies and strategies to promote social equity in health [bibliography on the Internet]. [place unknown: publisher unknown]; 1991 [cited 2022 Nov 22]. Available at: https://core.ac.uk/download/pdf/6472456.pdf.
  65. 65. Preda A, Voigt K. The Social Determinants of Health: Why Should We Care?. Am J Bioeth [Internet]. 2015 [cited 2022 Nov 22];15(3):25–36. Available at: https://pubmed.ncbi.nlm.nih.gov/25786009/. pmid:25786009
  66. 66. Barata RB, Almeida MF, Montero CV, Silva ZP. Health inequalities based on ethnicity in individuals aged 15 to 64, Brazil, 1998. Cad Saude Publica [Internet]. 2007 [cited 2022 Dec 21];23(2):305–313. Available from: https://www.scielosp.org/article/csp/2007.v23n2/305-313/#ModalArticles. pmid:17221079
  67. 67. Mechanic D, Tanner J. Vulnerable people, groups, and populations: societal view. Health Aff (Millwood) [Internet]. 2007 [cited 2022 Nov 22];26(5):1220–1230. Available at: https://pubmed.ncbi.nlm.nih.gov/17848429/. pmid:17848429
  68. 68. Mao M, Zang L, Zhang H. The Effects of Parental Absence on Children Development: Evidence from Left-Behind Children in China. Int J Environ Res Public Health [Internet]. 2020 [cited 2022 Nov 24];17(18):6770. Available at: https://pubmed.ncbi.nlm.nih.gov/32957472/. pmid:32957472
  69. 69. Chen L, Wulczyn F, Huhr S. Parental absence, early reading, and human capital formation for rural children in China. J Community Psychol [Internet]. 2022 [cited 2022 Nov 24];:1–14. Available at: https://onlinelibrary.wiley.com/doi/ pmid:34985781
  70. 70. Fu M, Xue Y, Zhou W, Yuan T. Parental absence predicts suicide ideation through emotional disorders. PLOS ONE [Internet]. 2017 [cited 2022 Nov 24];12(12):e0188823. Available. at: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0188823 pmid:29216233
  71. 71. Inoue Y, Fukunaga A, Stickley A, Yzawa A, Pham TTP, Nguyen CQ, et al. Association between parental absence during childhood and depressive symptoms in adulthood in rural Vietnam. J Affect Disord [Internet]. 2022 [cited 2022 Nov 24];311:479–485. Available at: https://www.sciencedirect.com/science/article/pii/S016503272200622X. pmid:35598745
  72. 72. Lacey RE, Zilanawala A, Webb E, Abell J, Bell S. Parental absence in early childhood and onset of smoking and alcohol consumption before adolescence. Arch Dis Child [Internet]. 2018 [cited 2022 Nov 24];103(7):691–694. Available at: https://pubmed.ncbi.nlm.nih.gov/27831906/. pmid:27831906
  73. 73. Lee BX, Donnelly PD, Cohen L, Garg S. Violence, health, and the 2030 agenda: Merging evidence and implementation. J Public Health Policy [Internet]. 2016 [cited 2022 Nov 24];37(1):1–12. Available at: https://pubmed.ncbi.nlm.nih.gov/27638239/. pmid:27638239
  74. 74. Huang X, King C, McAtee J. Exposure to violence, neighborhood context, and health-related outcomes in low-income urban mothers. Health & Place [Internet]. 2018 [cited 2022 Nov 24];54:138–148. Available at: https://www.sciencedirect.com/science/article/pii/S1353829217307165?via%3Dihub. pmid:30265943
  75. 75. Oliveira MAC, Pereira IC. Atributos essenciais da Atenção Primária e a Estratégia Saúde da Família. Rev Bras Enferm [Internet]. 2013 [cited 2022 Nov 24];66 https://doi.org/10.1590/S0034-71672013000700020. Available at: https://www.scielo.br/j/reben/a/5XkBZTcLysW8fTmnXFMjC6z/?lang=pt.
  76. 76. Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A practical guide to their development and use [Internet]. 5th ed. Oxford: Oxford University Press; 2014 [cited 2022 Dec 21]. ISBN: 9780191765452. https://doi.org/10.1093/med/9780199685219.001.0001. Available from: https://academic.oup.com/book/24920.
  77. 77. Pedrosa I, Suárez-Álvarez J, García-Cueto E. Content Validity Evidences: Theoretical Advances and Estimation Methods. Acción Psicológica [Internet]. 2014 [cited 2022 Dec 21];10(2):3–18. https://doi.org/10.5944/ap.10.2.11820. Available from: https://revistas.uned.es/index.php/accionpsicologica/article/view/11820.
  78. 78. Gjersing L, Caplehorn JRM, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol [Internet]. 2010 [cited 2022 Dec 21];10:10–13. Available from: https://pubmed.ncbi.nlm.nih.gov/20144247/. pmid:20144247
  79. 79. Wilson FR, Pan W, Schumsky DA. Recalculation of the Critical Values for Lawshe’s Content Validity Ratio. Meas Eval Couns Dev [Internet]. 2012 [cited 2022 Dec 21];45(3):197–210. Available from: https://www.tandfonline.com/doi/full/10.1177/0748175612440286.