https://orcid.org/0000-0003-1106-2883
Figures
Abstract
Social axioms or general social beliefs represent people’s cognitive map of their social world acquired through social experiences. Empirical research has related the central constructs in the study of psychology and social axioms, establishing a broad nomological network in various cultural settings. This paper studies the validity of the Social Axioms Survey II (SAS-II) short form, Spanish version, on the individual level in Melilla as North Africa´s borderland. Participants were 410 high school students from 14 to 18 years of age. The reliability analysis, the discriminant validity analysis, and the confirmatory factor analysis through the structural model equation, showed similar results to previous studies in other contexts and allowing the use of the survey in Melilla. In addition it is presented a fitted model that improves the psychometric results showing significant differences with the initial model. The confirmatory multi-group analysis of the fitted model shows measurement invariance across educational centers, allowing new research possibilities in the cultural context of Melilla.
Citation: García-Alonso M, Gallardo-Vigil MÁ, Melgar Alcantud P, Segura-Robles A (2020) Social axioms on high school students in the North African context: Validation and fit of the SAS-II. PLoS ONE 15(11): e0241510. https://doi.org/10.1371/journal.pone.0241510
Editor: Frantisek Sudzina, Aalborg University, DENMARK
Received: September 19, 2020; Accepted: October 15, 2020; Published: November 2, 2020
Copyright: © 2020 García-Alonso et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data underlying the results presented in the study are available from https://doi.org/10.7910/DVN/WAL0HO.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Leung and Bond [1,2] argue from a functionalist perspective that social axioms or general social beliefs represent people’s cognitive map of their social world. They propose the construct of general social beliefs as an alternative and possible complement to values in interpreting culture and explaining the responses of its members, and suggested four specific functions of these general beliefs: “facilitate the attainment of important goals (instrumental), help people protect their self-worth (ego-defensive), serve as a manifestation of people’s values (value-expressive), and help people understand the world (knowledge)” (p. 288).
Social axioms are defined as generalized beliefs about people, social groups, social institutions, the physical environment, or the spiritual world, as well as categories of events and phenomena in the social world. These generalized beliefs are encoded in the form of an assertion about the relationship between two entities or concepts [1–4]. The term social refers to the assumption that axioms are acquired through social experiences and are concerned with living as inherently social beings. The term axiom refers to the assumption that these general beliefs represent basic premises that people endorse without too much scrutiny of their validity.
On the basis of a literature review, interviews and content analysis of various sources, Leung and Bond [2] developed the Social Axioms Survey, a scale of 60 items with five factors at the individual level, involving over 50 collaborators from 40 national/cultural groups. The five-factor structure, was subsequently confirmed by multilevel factor analysis technique [5].
Social axioms always involve the relationship between two conceptual entities, and the relationship may be causal or correlational. Following this approach, empirical research has related the central constructs in the study of psychology and social axioms, establishing a broad nomological network in various cultural settings [2,6–16].
The development of the Social Axioms Survey II (SAS-II) was developed using a culturally decentered approach through collaborations with psychologists from 10 countries: Brazil, China, Germany, Ghana, Israel, Japan, Malaysia, Mexico, Russia and United States [17]. The items in the later version were then consolidated, reviewed, selected and refined by a principal group of researchers, obtaining a long form of 109 items and a short form of 40 items [18].
The validation of the Social Axioms Survey II allows the research network to be extended to a multitude of cultural contexts and, therefore, to the cultural context of the city of Melilla. Melilla is a Spanish territory located in North Africa with an area of 12'32 km2. The National Statistics Institute in Spain registered a population of 86.487 in 2019. Due to its location, it is part of the Western Mediterranean Route used by both the Maghreb and West Africans with the intention of relocating or moving to other European countries. Irregular arrivals to Spain registered 4,984 migrants in 2019, using Melilla as the point of entry [19]. As Frank Meyer [20] describes, the religious denomination in Melilla is not only associated with questions of belief or religious practices, are mostly perceived as cultures with corresponding and clear-cut values, traditions and customs as well as a territorial rootedness. The urban societies in Melilla can be seen as good examples of the significance and interconnection of identity, culture, space and time for human co-existence and the difficult relationship of the familiar and the strange.
The particular geopolitical context and multicultural coexistence make Melilla an especially attractive enclave for social research. Therefore, the main aim of this study is guided by previous studies in other social and cultural contexts [21,22]: Validate the replication of social axioms and its five-factor structure as a representative people’s cognitive map in the cultural context of Melilla. Consequently, two specific aims were set. The first specific aim was to validate the Social Axioms Survey II (SAS-II) short form, Spanish version, on the individual level, in Melilla. The second specific aim was to develop a fitted model to improve the statistical results [18].
Materials and methods
Participants
The research project in which this study is framed has been developed in accordance with the ethical principles expressed in the Declaration of Helsinki. Before accessing the school-age population, the research project was firstly approved by the academic commission of the Educational Sciences Program of the University of Granada. Subsequently, it was sent to the call for institutional projects that require the participation of centers dependent on the Ministry of Education in Spain, being approved by the commission of the Vice-Dean Office for Research, International Projects and Transfer of the University of Granada. Finally, it was approved by the commission of the Ministry of Education through the Educational Programs Unit of the Provincial Department of Education in Melilla. The collaboration of the high school principals was requested through the Head of the Educational Programs Unit providing the data of the principal researcher; and in person, giving a cover letter including the research process, the ethical commitment, and a copy of the instrument to be applied. The educational centers informed the parents and legal guardians of the participants, requesting consent to participate in this research activity and keeping the informed consent in their custody. Once the consents were obtained, the centers gave the researchers a specific date and time to access the participants. In addition, the participants were previously informed by the researchers about the objectives, purpose, method, benefits of the research, their free decision to participate or not and the commitment to keep their anonymity. Finally, once the data had been analyzed, the Head of the Educational Programs Unit offered the secondary directors of the centers the possibility of obtaining the results of the investigation.
Survey procedure and sample
The method used to select a representative sample from this population is cluster sampling. Cluster sampling does not select the subjects individually, but rather the homogeneous intergroups and heterogeneous intragroups in which they are previously inserted [23]. Clusters are represented by high schools, while the different groups available at each educational level were selected by simple random sampling. The number of students enrolled in the educational level as population (N = 845) was provided by the Provincial Department of Education in Melilla. Assuming a margin of error of 5%, a confidence level of 99% and a variability of 50%, a sample size of 372 subjects would have been representative enough to generalize results to the entire population. Finally, as shown in Table 1, the sample consisted of 410 students (200 males and 210 females) from 14 to 18 years of age (M = 15.7; SD = 0.915), from 6 high schools in Melilla.
Instrument
The Social Axioms Survey II (SAS-II) short form [17], Spanish version translated by Judith Gibbons, comprises 40 items, rated on a five-point Likert scale from totally disagree to totally agree. Leung et al. [17] defined the five axiom dimensions as follows: Social cynicism asserts that human nature and the social world yield negative outcomes; reward for application refers to the belief complex that people’s use of effort, knowledge, careful planning and other resources will lead to positive outcomes; social complexity asserts that people’s behavior may vary across situations and that problems have multiple solutions; fate control refers to the belief complex that life events are pre-determined by fatalistic forces, but that people may be able to predict and alter the decree of fate by various means; finally, religiosity asserts the existence of a supernatural being and the beneficial functions of religious practice.
Distribution of responses to SAS-II is shown in Fig 1.
Items are ordered by factors: RL, Religiosity; RC, Reward for Application; DT, Fate Control; CN, Social Cynicism; CP, Social Complexity.
Data analysis
A descriptive analysis was first performed to choose the most appropriate statistical processes. The assumption of a normal distribution of the population without prior verification makes it difficult to draw precise conclusions about reality, being a common error in a high percentage of scientific publications [24].
Some questionnaires were found to be incomplete. This is a problem since some statistical procedures do not consider the rest of the data provided by a case if it is incomplete. The elimination of cases implies the loss of present partial data, a decrease in the sample size and, therefore, a decrease in statistical power [25]. The data was missing completely at random, and the chosen method for the imputation of missing values was Multivariate Imputation by Chained Equations, using the R MICE package, a recommended choice when the data contains different types of numerical and categorical variables [26]. Missing values are imputed based on the observed values for a given individual and the observed relationships in the data for other participants, assuming that the observed variables are included in the imputation model [27].
Psychometric analysis
The following analyses used the Lavaan package and the Semplot package for modeling structural equations in R v.3.3.2 [28]. These analyses were applied to the initial SAS-II model and a fitted model, which reported better results by deleting items DT1, DT4, DT7, CP1, CP5, CP8, CN5 and CN8.
The reliability analysis, as internal consistency, is traditionally reported using Cronbach's α [29]. Revelle and Zinbarg [30] reported the biases of this measure, encouraging researchers to use better estimators. Therefore, the reliability was analyzed by reporting Cronbach's α [29], ω1 [31,32], ω2 [33–35], and ω3 [36], for each factor. Following the reliability criteria [37], values above 0.90 were considered excellent, between 0.90 and 0.70 were high, between 0.70 and 0.50 were moderate, and lower than 0.50 were low.
The discriminant validity analysis indicates whether the variables are strongly uncorrelated to the factor they are measuring, allowing one factor to be distinguished from the others. The AVE, or Average Variance Extracted values of each latent construct must be greater than the maximum square correlation, AVEsqrt, with other latent construct [38].
Confirmatory factor analysis was performed using structural equation models based on covariance, using the weighted least squares mean and variance or WLSMV estimator because it is “a robust estimator that does not assume distributed variables normally and provides the best option for modeling categorical or ordered data” [39–41]. The criteria to determine an adequate fit of the models [42] are: division χ2 and degrees of freedom (χ2/df < 3), significance (p < 0.05), Comparative Fit Index (CFI > 0.95), Goodness of Fit Index (GFI > 0.9), Adjusted Goodness of Fit Index (AGFI > 0.9), Tucker Lewis Index (TLI > 0.9), Root Mean Square Residual (RMR < 0.05), Standardized Root Mean Square Residual (SRMR < 0.08), Root Mean Square Error of Approximation (RMSEA = 0.06–0.08).
The confirmatory analysis of multiple groups [43], using the variable schools, allowed to test the measurement invariance. The analysis of the measurement invariance proves that different groups understand the items in the same way, and are, therefore, comparable. This analysis, involves the generation of models that are evaluated in a staggered manner with greater restriction in each test: configural, metric, scalar and strict. The values Δ must be significant (p < 0.05) in each test.
Results
Distribution analysis and imputation of missing values
The following tests were performed to choose the most appropriate statistical processes: goodness-of-fit Kolmogorov-Smirnov Lilliefors (KS = 0.157 to 0.438; p < 0.001); Shapiro-Wilks (W = 0.94623; p < 0.001); multivariate asymmetry (γˆ1, p = 1368.164, p = 0); multivariate kurtosis (γˆ2, p = 5943.631, p = 0); and Levene's test (W = 0.011 to 3486; p = 0.918 to 0.063), using the sex variable to segment the sample into two independent groups. The results show that the normality assumptions are not fulfilled.
Cases with incomplete data (n = 18) show a proportion of missing values less than or equal to 4.8%. These results confirm that the decision to eliminate cases is not recommended. Little's χ2 test statistic for MCAR [27] shows that the data was missing completely at random (χ2 = 1315,572; df = 1275; p = 0.209). The Polyreg function in the R MICE package was the most suitable to impute responses in categorical variables with more than two levels, using the Bayesian polytomous regression model. The imputation of missing values increases the number of valid cases from 392 to 410.
Reliability analysis
The reliability analysis for the initial model, shown in Table 2, report excellent values for the religiosity factor from 0.919 to 0.918; moderate values for the reward for application factor from 0.694 to 0.646, fate control factor from 0.685 to 0.652, social cynicism factor from 0.662 to 0.657; and moderate values for social complexity from 0.545 to 0.512. The fitted model reports moderately better values for each factor. The total reliability of the test allows for observing better values in the fitted model for each estimator.
Discriminant validity analysis
The discriminant validity analysis for the initial model, shown in Table 3, does not report discriminant validity for the social complexity factor being correlated with the reward for application factor (r = 0.384 > AVEsqr = 0.362). The discriminant validity analysis for the fitted model, shown in Table 4, overcomes this problem achieving discriminant validity in all factors.
Confirmatory factor analyses
The confirmatory factor analyses, shown in Table 5, for the initial model reports a good model fit: χ2 (730) = 1254.62; CFI = 0.93, TLI = 0.92, RMSEA = 0.042. The fitted model also reports a good model fit with better values than the initial model: χ2 (454) = 705.69; CFI = 0.96, TLI = 0.96, RMSEA = 0.037.
The loadings of each item to its corresponding factor for the initial model is shown in Fig 2, and the loadings of each item to its corresponding factor for the fitted model is shown in Fig 3.
The difference test for the initial model and fitted model, shown in Table 6, reports significant differences (p < 0.001).
Measurement of invariance
The measurement invariance test, shown in Table 7, supports that the instrument measures the same psychological construct in all educational centers. Holding this assumption, the comparisons between groups are valid and can be meaningfully interpreted.
Discussion
The main aim of this study was to validate the replication of social axioms and its five-factor structure as a representative people’s cognitive map in the cultural context of Melilla for the first time, as one of the two European Union’s land borders with North Africa. This geographical, political and multicultural situation provides its own social characteristics that cannot be directly attributed to the nations of Spain or Morocco. The second aim was to develop a fitted model to improve the psychometrical measures of the Social Axioms Survey II (SAS-II).
The reliability analysis for the initial model, shown in Table 2, reported moderate values for social complexity from 0.545 to 0.512. The fitted model reports moderately better values for each factor. Social Complexity and Fate Control had shown marginal reliability as some previous researches [17]. The Social Complexity factor thus remains problematic; this is the same factor from the original social axioms survey that Barnard et al. [4] were unable to replicate in their South African study.
The discriminant validity problem for the social complexity factor was overcome for the fitted model, as shown in Tables 3 and 4, achieving discriminant validity in all factors.
Confirmatory factor analyses through the structural model equation, shown in Fig 2, report low item loads at their corresponding factor for both models. However, goodness-of-fit estimators report good values for both models, as shown in Table 6, reporting better values for the fitted model.
The measurement invariance test, shown in Table 7, supports that the comparisons between groups are valid and can be meaningfully interpreted. This suggests that the fitted SAS-II model could be a valid instrument to study the differences or similarities between educational centers as microcultural contexts.
Conclusions
Considering the results of the analysis, we can affirm that the first objective has been achieved moderately, finding problems of discriminant validity for the factor of social complexity. Likewise, we can affirm that the second objective has been achieved, obtaining a valid model to investigate social axioms in the cultural context of Melilla. The findings of this study provide possibilities for expanding the nomological network of correlations in the field of social psychology and invite the study of possible causal relationships from these investigations.
Limitations of the study
This study is limited by access to the population. Two institutes did not respond to our request for participation in this research, one of them characterized by students of Sephardic culture as a representative cultural identity of the city, which could have influenced the results of the survey. In addition, as has been shown in previous research, the social complexity factor showed lower values than the other factors, therefore, this factor should be approached with caution in future research.
Acknowledgments
Our grateful acknowledgment to Michael Harris Bond of the Hong Kong Polytechnic University for providing us with the Spanish version of Social Axioms Survey II done by Judith Gibbons.
References
- 1. Leung K, Bond MH, de Carrasquel SR, Muñoz C, Hernández M, Murakami F, et al. Social axioms: The search for universal dimensions of general beliefs about how the world functions. Journal of Cross-Cultural Psychology. 2002;33(3):286–302.
- 2.
Leung K, Bond MH. Social Axioms: A Model for Social Beliefs in Multicultural Perspective. In: Advances in Experimental Social Psychology [Internet]. Elsevier; 2004 [cited 2020 Aug 12]. p. 119–97. Available from: https://linkinghub.elsevier.com/retrieve/pii/S006526010436003X.
- 3.
Chiu C, Chia SI, Wan WWN. Chapter 22—Measures of Cross-Cultural Values, Personality and Beliefs. In: Boyle GJ, Saklofske DH, Matthews G, editors. Measures of Personality and Social Psychological Constructs [Internet]. San Diego: Academic Press; 2015. p. 621–51. Available from: http://www.sciencedirect.com/science/article/pii/B978012386915900022X.
- 4. Barnard A, Rothmann S, Meiring D. The cross-cultural application of the social axioms survey in The South African police service. SA j ind psychol [Internet]. 2008 Nov 24 [cited 2020 Aug 12];34(2):1–9. Available from: http://sajip.co.za/index.php/sajip/article/view/474.
- 5. Cheung MW-L, Leung K, Au K. Evaluating Multilevel Models in Cross-Cultural Research: An Illustration With Social Axioms. Journal of Cross-Cultural Psychology [Internet]. 2006 Sep [cited 2020 Aug 12];37(5):522–41.
- 6. Chen L, Zhang J. Relation between general social beliefs and interpersonal conflict resolution styles. Chinese Journal of Clinical Psychology [Internet]. 2004 [cited 2020 Aug 12];12(2):151–3. Available from: http://papi.psych.ac.cn/members/references/04GSB.PDF.
- 7. Chen SX, Cheung FM, Bond MH, Leung J-P. Decomposing the construct of ambivalence over emotional expression in a Chinese cultural context. Eur J Pers [Internet]. 2005 Apr [cited 2020 Aug 12];19(3):185–204.
- 8. Dangel TJ, Webb JR, Hirsch JK. Forgiveness and Suicidal Behavior: Cynicism and Psychache as Serial Mediators. The Journal of Psychology [Internet]. 2018 Feb 17 [cited 2020 Aug 12];152(2):77–95. pmid:29261433
- 9. Fu PP, Kennedy J, Tata J, Yukl G, Bond MH, Peng T-K, et al. The impact of societal cultural values and individual social beliefs on the perceived effectiveness of managerial influence strategies: a meso approach. J Int Bus Stud [Internet]. 2004 Jul [cited 2020 Aug 12];35(4):284–305.
- 10. Hackney CH, Sanders GS. Religiosity and Mental Health: A Meta-Analysis of Recent Studies. J Scientific Study of Religion [Internet]. 2003 Mar [cited 2020 Aug 12];42(1):43–55.
- 11. Hui VK-Y, Bond MH, Ng TSW. General Beliefs about the World as Defensive Mechanisms against Death Anxiety. Omega (Westport) [Internet]. 2007 May [cited 2020 Aug 12];54(3):199–214. pmid:17847954
- 12. Iliescu D, Dincă M, Bond MH. The Increment of Social Axioms over Broad Personality Traits in the Prediction of Dyadic Adjustment: An Investigation Across Four Ethnic Groups: Predictors of dyadic adjustment. Wrzus C, editor. Eur J Pers [Internet]. 2017 Nov [cited 2020 Aug 12];31(6):630–41.
- 13. Kuo BCH, Kwantes CT, Towson S, Nanson KM. Social Beliefs as Determinants of Attitudes Toward Seeking Professional Psychological Help among Ethnically Diverse University Students. Canadian Journal of Counselling and Psychotherapy [Internet]. 2006 [cited 2020 Aug 12];40(4):224–41. Available from: https://cjc-rcc.ucalgary.ca/article/view/58785.
- 14. Lai JH-W, Bond MH, Hui NH-H. The role of social axioms in predicting life satisfaction: a longitudinal study in Hong Kong. J Happiness Stud [Internet]. 2007 Dec [cited 2020 Aug 12];8(4):517–35.
- 15. Nartova-Bochaver SK. Connection between the psychological sovereignty and social beliefs: personal boundaries in the social world. Social Psychology and Society [Internet]. 2017 [cited 2020 Aug 12];8(3):100–14. Available from: https://psyjournals.ru/en/social_psy/2017/n3/Nartova-Bochaver.shtml.
- 16. Neto F. Dimensions and Correlates of Social Axioms Among a Portuguese Sample. Individual Differences Research. 2006;4(5):340–51.
- 17. Leung K, Lam BCP, Bond MH, Conway LG III, Gornick LJ, Amponsah B, et al. Developing and evaluating the social axioms survey in eleven countries: Its relationship with the five-factor model of personality. Journal of Cross-Cultural Psychology. 2012;43(5):833–57.
- 18.
Boyle GJ, Saklofske DH, Matthews G. Measures of Personality and Social Psychological Constructs [Internet]. Elsevier; 2015 [cited 2020 Aug 12]. https://doi.org/10.1016/C2010-0-68427-6
- 19.
Idemudia E, Boehnke K. Travelling Routes to Europe. In: Idemudia E, Boehnke K, editors. Psychosocial Experiences of African Migrants in Six European Countries: A Mixed Method Study [Internet]. Cham: Springer International Publishing; 2020. p. 33–49. https://doi.org/10.1007/978-3-030-48347-0_3
- 20. Meyer F. “Wer ist fremd an diesen Orten?” Zur Bedeutung von Identität, Kultur, Raum und Zeit in den spanisch-nordafrikanischen Städten Ceuta und Melilla (“Who Is Foreign in These Places?” On the Significance of Identity, Culture, Space and Time in the Spanish North African Towns of Ceuta and Melilla). Erdkunde [Internet]. 2004 [cited 2020 Oct 4];58(3):235–51. Available from: http://www.jstor.org/stable/25647678.
- 21. Safdar S, Lewis JR, Daneshpour M. Social axioms in Iran and Canada: Intercultural contact, coping and adjustment. Asian J Soc Psychol [Internet]. 2006 Aug [cited 2020 Aug 11];9(2):123–31.
- 22. Barnard A, Meiring D, Rothmann I. The factor structure of the Social Axioms Survey II (SASII) in the South African context. Journal of Psychology in Africa [Internet]. 2017 Dec 11 [cited 2020 Aug 12];27(6):522–9.
- 23.
Buendía L, Colás P, Hernández F. Métodos de Investigación en Psicopedagogía. Madrid, España: McGraw-Hill; 1998.
- 24. Ghasemi A, Zahediasl S. Normality Tests for Statistical Analysis: A Guide for Non-Statisticians. Int J Endocrinol Metab [Internet]. 2012 Dec 1 [cited 2020 Aug 12];10(2):486–9. Available from: https://sites.kowsarpub.com/ijem/articles/71904.html. pmid:23843808
- 25. Graham JW. Missing Data Analysis: Making It Work in the Real World. Annu Rev Psychol [Internet]. 2009 Jan [cited 2020 Aug 12];60(1):549–76. pmid:18652544
- 26.
Enders CK. Applied missing data analysis. New York: Guilford Press; 2010. 377 p. (Methodology in the social sciences).
- 27. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods [Internet]. 2002 [cited 2020 Aug 12];7(2):147–77. pmid:12090408
- 28.
R Core Team. R: A language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2016. Available from: https://www.R-project.org/.
- 29. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika [Internet]. 1951 Sep [cited 2020 Aug 12];16(3):297–334.
- 30. Revelle W, Zinbarg RE. Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika [Internet]. 2009 Mar [cited 2020 Aug 11];74(1):145–54. pmid:20037638
- 31. Bollen KA. Issues in the Comparative Measurement of Political Democracy. American Sociological Review [Internet]. 1980 Jun [cited 2020 Aug 12];45(3):370. Available from: http://www.jstor.org/stable/2095172?origin=crossref.
- 32. Bollen KA. Political Democracy: Conceptual and Measurement Traps. St Comp Int Dev [Internet]. 1990 Mar [cited 2020 Aug 12];25(1):7–24.
- 33. Bentler PM. A lower-bound method for the dimension-free measurement of internal consistency. Social Science Research [Internet]. 1972 Dec [cited 2020 Aug 12];1(4):343–57. Available from: https://linkinghub.elsevier.com/retrieve/pii/0049089X72900828.
- 34. Bentler PM. Alpha, Dimension-Free, and Model-Based Internal Consistency Reliability. Psychometrika [Internet]. 2009 Mar [cited 2020 Aug 12];74(1):137–43. pmid:20161430
- 35. Bentler PM. Specificity-enhanced reliability coefficients. Psychological Methods [Internet]. 2017 Sep [cited 2020 Aug 12];22(3):527–40. pmid:27732051
- 36.
McDonald RP. Test Theory: A Unified Treatment [Internet]. 1st ed. Psychology Press; 2013 [cited 2020 Aug 12]. Available from: https://www.taylorfrancis.com/books/9781410601087.
- 37.
Hinton P. SPSS Explained [Internet]. 2nd ed. Routledge; 2014 [cited 2020 Aug 12]. Available from: https://www.taylorfrancis.com/books/9781315797298.
- 38. Fornell C, Larcker DF. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research [Internet]. 1981 Feb [cited 2020 Aug 12];18(1):39–50.
- 39. Motulsky HJ, Brown RE. Detecting outliers when fitting data with nonlinear regression–a new method based on robust nonlinear regression and the false discovery rate. BMC Bioinformatics [Internet]. 2006 [cited 2020 Aug 12];7(1):123. pmid:16526949
- 40.
Powell G. Disproportionate Collapse: The Futility of Using Nonlinear Analysis. In: Structures Congress 2009 [Internet]. Austin, Texas, United States: American Society of Civil Engineers; 2009 [cited 2020 Aug 12]. p. 1–10. https://doi.org/10.1061/41031(341)211
- 41.
Wang J, Wang X. Structural Equation Modeling: Applications Using Mplus [Internet]. Chichester, UK: John Wiley & Sons, Ltd; 2012 [cited 2020 Aug 11]. https://doi.org/10.1002/9781118356258
- 42. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal [Internet]. 1999 Jan [cited 2020 Aug 12];6(1):1–55.
- 43.
Byrne BM. Multivariate applications series. Structural equation modeling with AMOS: Basic concepts, applications, and programming. 2nd ed. New York, NY, US: Routledge/Taylor & Francis Group; 2010.