The aims of our study were to explore the factor structure and psychometric properties of the French version of the Beliefs about Medicines Questionnaire (BMQ) in patients with severe mental illness and in specific populations of patients with schizophrenia, bipolar disorder and major depressive disorder.
A cross-sectional study including patients with schizophrenia, bipolar disorder and major depressive disorder was conducted (n = 150). Principal component analysis (PCA), reliability and validity of the French version of the BMQ were performed.
PCA revealed a two-factor structure similar to the original structure for the BMQ-Specific scale but only a one-component solution for the BMQ-General scale in both the total sample and the three subgroups. These subscales have satisfactory internal consistency. Validity was supported by the significant correlations of all BMQ subscales with the Drug Attitude Inventory.
Citation: Samalin L, de Chazeron I, Belzeaux R, Llorca P-M (2017) Exploratory analysis of the French version of the beliefs about medicines questionnaire in patients with severe mental disorders: Factorial structure and reliability in specific populations of schizophrenic, bipolar and depressive patients. PLoS ONE 12(3): e0173267. https://doi.org/10.1371/journal.pone.0173267
Editor: Michel Botbol, Universite de Bretagne Occidentale, FRANCE
Received: June 9, 2016; Accepted: February 8, 2017; Published: March 3, 2017
Copyright: © 2017 Samalin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Psychotropic medications are the cornerstone and the standard of care for the treatment of severe mental illness (SMI). However, medication non-adherence is frequent in these chronic illnesses with an estimated prevalence rate in schizophrenia of about 50%  and in mood disorders of about 40% . Non-adherence is associated with poor clinical outcomes in these patients such as higher risk of recurrence/relapse and hospitalization [3, 4], poorer psychosocial functioning  and higher rates of suicide [4, 6]. The negative attitudes and beliefs toward medication as well as poor insight, poor therapeutic alliance and low social support represent the most important factors associated with non-adherence [2, 7, 8].
To improve the understanding of non-adherence to medication, Horne et al. developed the Beliefs about Medicines Questionnaire (BMQ) on the basis of social cognitive models, such as the extension of Leventhal’s self-regulation model . This is the most specific model about medication, suggesting that adherence is heavily influenced by the patients’ own beliefs and representation of the illness. This scale measures the specific patient’s perception of the need to take medication and concerns about it (BMQ-Specific, with two subscales named “Necessity” and “Concern”) as well as the general beliefs about pharmacotherapy (BMQ-General, with two subscales named “Harm” and “Overuse”).
This scale has been used in relation to a broad range of chronic diseases in different language versions . Studies in SMI patients using the BMQ have shown its potential to explore and understand the determinants of non-adherence and to help the clinician to determine individual targets for interventions to improve patients’ negative beliefs. In these studies with schizophrenic patients , bipolar patients  or patients with major depressive disorder , higher adherence was associated with stronger perceived necessity of psychotropic treatment and fewer concerns regarding treatment. These specific beliefs (BMQ-Specific) about medication impacted medication adherence independently of the general beliefs (BMQ-General) toward pharmacotherapy .
In France, one study has validated the BMQ in patients with diabetes and HIV and found the same factorial structure as the English original version  even if the process does not follow steps recommended for validation scale in the COSMIN guidelines . Theoretical concepts need to be explored in order to confirm that the French version of this scale is a suitable instrument in SMI patients. Most validation studies of BMQ in SMI patients were conducted in a mixed sample of psychiatric patients despite their clinical heterogeneity [9, 16–18]. Specific studies in defined samples of schizophrenic, bipolar and depressive patients are needed in order to support the stability of the factor structure and the psychometric properties of the BMQ among them.
The aims of our study were to explore the factor structure and some psychometric properties of the French version of the BMQ in SMI patients and to describe the factor structure and psychometric properties in specific populations of patients with schizophrenia, bipolar disorder and major depressive disorder.
Materials and methods
All adult patients (outpatients or inpatients) treated by at least one psychotropic medication (i.e. antipsychotic, mood stabilizer or antidepressant), who met the diagnosis of schizophrenia, bipolar disorder or major depressive disorder according to DSM-IV-TR criteria were recruited consecutively at two different French mental health hospital sites during a 1-year period. Exclusion criteria were inability to understand the self-administered questionnaires or the interviewer because of language problems or a learning disability. After the procedure was fully explained, written informed consent was obtained from all participants. Formal assessment of capacity was performed where applicable; patients lacking the capacity to provide written, informed consent were not approached to participate. The study was approved by the Institutional Review Board (CECIC Rhône-Alpes-Auvergne, Grenoble, IRB 5921) and was conducted in accordance with the Helsinki Declaration of 1975, as revised in 1983.
Socio-demographic and clinical data were collected through a patient interview. The attitudes toward medication were assessed using the French versions of the BMQ and the 10-item Drug Attitudes Inventory (DAI). All patients filled in the BMQ and the 10-item DAI simultaneously.
The BMQ is an 18-item self-questionnaire assessing patients’ beliefs about their medication . The original version comprises a Specific and a General scale, and each has two subscales. The BMQ-Specific has a 5-item Necessity scale that assesses beliefs about the necessity of the prescribed medication and a 5-item Concerns scale that assesses concerns about potential adverse effects of treatment. The BMQ-General assesses more general beliefs or social representations about drugs, including 8 items in two subscales of 4-item (Harm and Overuse subscales). Each item of the questionnaire is rated on a 5-point Likert scale (1 = “strongly disagree” to 5 = “strongly agree”). Each Necessity and Concerns scale ranges from 5 to 25 and each Harm and Overuse scale ranges from 4 to 20. Higher Necessity scores represent stronger perceptions (or positive beliefs) of personal need for the medication, and higher Concerns scores represent stronger concerns (or negative beliefs) about the potential negative effects of the medication. Higher scores on the General subscales represent more negative views about medicines.
In this study, the French version of the BMQ was used . It showed satisfactory properties with the same factorial structure as the original version (two factors for the BMQ-Specific scale and two factors for the BMQ-General scale) in both diabetes and HIV patients.
The 10-item DAI is a self-report instrument that includes 10 questions on the subjective response to medication regarding both positive and negative effects [19, 20]. The respondents rate statements as true or false. The total score ranges from -10 to +10, a positive total score indicating a positive attitude towards psychiatric medication and a negative total score indicating a negative attitude towards psychiatric medication. The French version of the DAI showed satisfactory psychometric properties similar to the original version in SMI patients . Despite some limitations (e.g. dichotomous ratings, items about both subjective states and attitudes towards medication), the 10-item DAI was used for the criterion validity of the BMQ because it is one of the most frequently used patient self-report measures to assess attitudes towards medication and can also predict medication adherence in SMI patients .
Exploratory factor analysis was performed to assess measurement items within scale to check dimensionality, factor structure, and strength of factor loadings. Construct validity was first explored using principal component analysis (PCA) with varimax rotation for all included patients (considered as SMI patients). Then, PCAs were conducted for the three subgroups of patients (schizophrenia, bipolar disorder and major depressive disorder). A PCA was performed for each scale (Specific and General). In line with recommendations [23, 24], factors generated by the PCA were extracted as valid if at least two of the following criteria were met: i) eigenvalues were equal to or greater than unity ; ii) scree (Cattel’s) test (by the identification of the point of inflection in the scree plot)  and; iii) eigenvalues exceed the 95th percentile of the corresponding simulated eigenvalue from Horn’s parallel analysis .
Mean, standard deviation and median were calculated for the BMQ subscales. Normality was explored based on skewness and kurtosis values. Floor and ceiling effects were measured by the percentage of patients with minimum and maximum scores respectively and should be below 15% to meet the acceptable measurement .
Cronbach’s alpha was used to estimate internal consistency. Pearson’s correlation coefficient was used for inter-factor correlation and factor correlation with DAI in order to establish criterion validity.
All statistical tests were two-tailed and the significance level was set at 5%. Statistical analyses were performed using SAS 9.3 ® software and Monte-Carlo parallel analysis software.
A total of 150 treated patients with schizophrenia (n = 50), bipolar disorder (n = 50) and major depressive disorder (n = 50) were recruited (Table 1). The mean ± SD age was 51.3 ± 16.6 and more than half of the patients were males (67%) without employment (74.7%). The mean ± SD illness duration was 14.3 years ± 10.4. Second-generation antipsychotics were the most frequent medication used in schizophrenic patients (98.0%), antidepressants the most frequent medication used in major depressive disorder (92.0%) and mood stabilizers the most frequent treatment prescribed for bipolar patients (86.0%).
Principal component analysis
Scree plots for the PCAs of the BMQ scales in total sample (A); patients with schizophrenia (B); patients with bipolar disorder (C); and patients with major depressive disorder (D). BMQ, Beliefs about Medicines Questionnaire.
Kaiser recommends extracting factors with eigenvalue ≥ 1; Scree (Cattel’s) test recommends extracting factors above the elbow of the scree plot; Horn’s parallel analysis recommends extracting factors above the randomly generated eigenvalue (95th percentile).
In the total sample, the first factor, “Necessity” in the original scale, explained 32% of the variance and the second, “Concerns” in the original scale, explained 23% (Table 3). All items loaded higher than 0.40 on the respective factors. No item loaded high on two factors. In the subgroups of patients with schizophrenia, bipolar disorder and major depressive disorder, the first factor (Necessity) explained 33%, 33% and 36% of the variance respectively, and the second factor (Concerns) explained 21%, 26% and 20% respectively (Table 4). All items loaded higher than 0.40 on the respective factors, except for item 5 (“I sometimes worry about the long-term effects of my medicines”) in the schizophrenia group, which obtained small loading on both factors, and item 9 (“I sometimes worry about becoming too dependent on my medicines”) in the major depressive disorder group, which loaded higher on the Necessity factor.
PCA of the BMQ-General revealed a one-component solution for the total sample and the three subgroups of patients (Fig 1, Table 2). This factor explained 42% of the variance in the total sample and 44%, 46% and 37% of the variance in the schizophrenia group, bipolar disorder group and major depressive disorder group respectively. In the total sample and the three subgroups, this factor comprised all items of the BMQ-General scale with all items loaded higher than 0.40. It indicated that SMI patients, as well as patients with schizophrenia, bipolar disorder and major depressive disorder, do not distinguish the perceived Harm and Overuse of medication as the original scale.
The mean scores of the BMQ-Specific subscales and the BMQ-General scale (Necessity = 19.04 ± 4.27, Concerns = 15.08 ± 4.37, General = 21.50 ± 5.89) in the total sample (Table 5) were similar to the previous studies in SMI patients . The mean scores of the three factors were also similar in the three subgroups of patients without significant difference between groups. There were no significant floor or ceiling effects for subscales in different psychiatric groups (<15%) and results also revealed no severe normality violations (skewness and kurtosis) (Table 5).
There was good or acceptable consistency between items in the total sample, Cronbach’s alpha for the BMQ-Specific Necessity and Concerns and the BMQ-General were 0.83, 0.68 and 0.80 respectively (Table 6). There was good consistency between items in the three subgroups except for the BMQ-Specific Concerns in patients with schizophrenia and major depressive disorder, which showed questionable values (Table 6). Inter-factor correlations were acceptable with weak to moderate relationships in both the total sample and the three subgroups of patients (Table 6). No significant relationships were found between the BMQ-Specific subscales for all groups.
The BMQ-Specific subscales and the BMQ-General scores significantly correlated with the DAI scores in the total sample and the three subgroups of patients, indicating that the BMQ assesses a construct that is similar to the DAI in patients with SMI as well as patients with schizophrenia, bipolar disorder and major depressive disorder (Table 6).
The French version of the BMQ partially supports the original factor structure (BMQ-Specific, two factors; BMQ-General, two factors) in SMI patients. Our findings replicate the original factor structure of the BMQ-Specific scale but not the BMQ-General, for which a monofactorial solution was found. Similarly, factor analysis in the specific populations of patients revealed the same factor structure as the total sample. The French version of the BMQ presented satisfactory psychometric properties for use in SMI patients. It is also psychometrically sound for measuring the beliefs about medications in patients with schizophrenia, bipolar disorder and major depressive disorder, despite their clinical heterogeneity.
In contrast with the French version validated in patients with diabetes and HIV  and similar to previous validation in schizophrenic patients  or psychiatric outpatients , the French version of the BMQ appeared as a three-dimensional scale in SMI patients. It is interesting to note that this factor structure appeared stable whatever the psychiatric diagnosis of the included patients.
From a methodological perspective, most previous validation studies of the BMQ revealing the same factor structure as the original scale used the Kaiser criterion (eigenvalues > 1 rule) to identify the number of factors as the primary criterion [30–32]. Our findings also showed a two-factor solution for the BMQ-General based on the Kaiser criterion, but only one factor solution according to the scree plot and parallel analysis criteria. In subgroups of patients, on the basis of the Kaiser criterion, the PCA revealed between three and four factors for the BMQ-specific and between one and three factors for the BMQ-General (Table 2). A recent systematic review assessed which factor analysis determined the correct number of latent dimensions when applied to ordered-categorical survey items (specifically for five-category Likert items) . Strong consensus recommended avoiding use of the Kaiser criterion because it can overestimate the number of factors, not considering the number of items tested and the sample size. Parallel analysis appeared to be the best choice of eigenvalue-based criteria for assessing dimensionality. In line with these recommendations, and according to our findings, we can formulate the hypothesis that previous validation studies could have overestimated the number of factors for the BMQ-General due to the limitations of the Kaiser criterion.
The second explanation could be related to the specificity of the SMI population concerning their general beliefs about medications in comparison with other chronic illness populations. As Beck et al. pointed out, the monofactorial solution for the BMQ-General suggests that SMI patients do not differ between harm and overuse of medication in general . In line with these results, a previous study of SMI patients showed that neither BMQ-General subscale appears to measure independent dimensions, because Harm and Overuse subscales were strongly inter-correlated .
The PCA revealed the same items for each factor as the original scale in the total sample and in patients with bipolar disorder. In the schizophrenia subgroup, the Specific item 5 (“I sometimes worry about the long-term effects of my medicines”) did not reach a loading greater than 0.40 for the either factor, even if the higher loading was obtained for the Concerns factor than in the original scale. Unexpected and well-nigh incomprehensible was the higher loading of Specific item 9 (“I sometimes worry about becoming too dependent on my medicines”) on the Necessity factor in patients with major depressive disorder. We can hypothesise that there is a potential cultural effect on this loading. In a recent large epidemiological study in France, major depressive episodes remained associated with a high co-prescription rate of long-term treatment with benzodiazepines in comparison to other Western countries , treatment that is well-known in the general population to be associated with dependence. In this context, we can make the hypothesis that French patients may consider that the necessary treatment for major depressive disorder is associated with a risk of dependence. Another explanation could be due to the lack of representativeness of the sample because patients were only recruited in two French mental health hospital sites.
The means Necessity, Concerns and General scores of our whole sample and our three subgroups were similar to previous studies using BMQ with schizophrenic patients [11, 28], with mixed sample of patients with bipolar disorder and schizophrenia  or with psychiatric outpatients .
Both BMQ-Specific subscales and the BMQ-General scale showed satisfactory internal consistency, except for the BMQ-Specific Concerns in the subgroups of patients with schizophrenia and major depressive disorder. The significant correlations of the BMQ-Specific subscales and the BMQ-General scale with the DAI suggested the criterion validity of the BMQ to assess patients’ attitudes towards medications for all groups.
However, this study presents several limitations. The main limitation is the cross-sectional design, which cannot support causal relationships, along with the absence of data testing the test-retest reliability. The inclusion of SMI patients who consented to participate in the study represents a potential selection bias that can overestimate the percentage of patients with positive attitudes towards medication. The findings also need to be interpreted with caution due to the minimal sample size reached in each subgroup of patients  and the limited number of French hospital sites for inclusions. Finally, it will be interesting to use cross-validation procedures in future work to test models.
Our findings highlight the interest to explore the factor structure and the psychometric properties of a scale in specific populations, because the theoretical equivalence between our population sample and patients with diabetes or HIV populations from the first French validation study of the BMQ was partially supported.
This questionnaire represents a useful tool to facilitate the identification of non-adherence and to implement strategies to improve negative beliefs in clinical practice. Future studies using the French version of the BMQ in patients with schizophrenia, bipolar disorder or major depressive disorder will be needed to confirm our findings.
- Conceptualization: LS PML.
- Data curation: IdC.
- Formal analysis: IdC LS.
- Investigation: LS RB PML.
- Methodology: LS IdC PML.
- Project administration: LS.
- Resources: LS RB PML.
- Supervision: PML.
- Visualization: LS IdC.
- Writing – original draft: LS.
- Writing – review & editing: LS IdC RB PML.
- 1. Acosta F, Hernandez J, Pereira J, Herrera J, Rodriguez C. Medication adherence in schizophrenia. World J Psychiatry. 2012;2(5):74–82. pmid:24175171
- 2. Pompili M, Venturini P, Palermo M, Stefani H, Seretti ME, Lamis DA, et al. Mood disorders medications: predictors of nonadherence—review of the current literature. Expert Rev Neurother. 2013;13(7):809–25. pmid:23898852
- 3. Novick D, Haro JM, Suarez D, Perez V, Dittmann RW, Haddad PM. Predictors and clinical consequences of non-adherence with antipsychotic medication in the outpatient treatment of schizophrenia. Psychiatry Res. 2010;176(2–3):109–13. pmid:20185182
- 4. Pompili M, Serafini G, Del Casale A, Rigucci S, Innamorati M, Girardi P, et al. Improving adherence in mood disorders: the struggle against relapse, recurrence and suicide risk. Expert Rev Neurother. 2009;9(7):985–1004. pmid:19589049
- 5. Velligan DI, Weiden PJ, Sajatovic M, Scott J, Carpenter D, Ross R, et al. The expert consensus guideline series: adherence problems in patients with serious and persistent mental illness. J Clin Psychiatry. 2009;70 Suppl 4:1–46; quiz 7–8.
- 6. Higashi K, Medic G, Littlewood KJ, Diez T, Granstrom O, De Hert M. Medication adherence in schizophrenia: factors influencing adherence and consequences of nonadherence, a systematic literature review. Ther Adv Psychopharmacol. 2013;3(4):200–18. pmid:24167693
- 7. Lacro JP, Dunn LB, Dolder CR, Leckband SG, Jeste DV. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia: a comprehensive review of recent literature. J Clin Psychiatry. 2002;63(10):892–909. pmid:12416599
- 8. Llorca PM. Partial compliance in schizophrenia and the impact on patient outcomes. Psychiatry Res. 2008;161(2):235–47. pmid:18849080
- 9. Horne R, Weinman J, Hankins M. The Beliefs about Medicines Questionnaire: the development and evaluation of a new method for assessing the cognitive representation of medication. Psychol Health. 1999;14:1–25.
- 10. Horne R, Chapman SC, Parham R, Freemantle N, Forbes A, Cooper V. Understanding patients' adherence-related beliefs about medicines prescribed for long-term conditions: a meta-analytic review of the Necessity-Concerns Framework. PLoS One. 2013;8(12):e80633. pmid:24312488
- 11. Beck E, Cavelti M, Kvrgic S, Kleim B, Vauth R. Are we addressing the 'right stuff' to enhance adherence in schizophrenia? Understanding the role of insight and attitudes towards medication. Schizophr Res. 2011;132(1):42–9. pmid:21820875
- 12. Clatworthy J, Bowskill R, Parham R, Rank T, Scott J, Horne R. Understanding medication non-adherence in bipolar disorders using a Necessity-Concerns Framework. J Affect Disord. 2009;116(1–2):51–5. pmid:19101038
- 13. Aikens J, Nease D, Nau D, Klinkman M, Schwenk T. Adherence to maintenance-phase antidepressant medication as a function of patient beliefs about medication. Ann Fam Med. 2005;3(1):23–30. pmid:15671187
- 14. Fall E, Gauchet A, Izaute M, Horne M, Chakroun N. Validation of the French version of the Beliefs about Medicines Questionnaire (BMQ) among diabetes and HIV patients. Eur Rev Appl Psychol. 2014;64:335–43.
- 15. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HC. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737–45. pmid:20494804
- 16. Argentero P, Torchio E, Tibaldi G, Horne R, Clatworthy J, Munizza C. [The beliefs about drug treatment. The Italian version of the BMQ (the Beliefs about Medicines Questionnaire): its validity and applicability]. Epidemiol Psichiatr Soc. 2010;19(1):86–92. pmid:20486428
- 17. De las Cuevas C, Rivero-Santana A, Perestelo-Perez L, Gonzalez-Lorenzo M, Perez-Ramos J, Sanz EJ. Adaptation and validation study of the Beliefs about Medicines Questionnaire in psychiatric outpatients in a community mental health setting. Hum Psychopharmacol. 2011;26(2):140–6. pmid:21455972
- 18. Jonsdottir H, Friis S, Horne R, Pettersen KI, Reikvam A, Andreassen OA. Beliefs about medications: measurement and relationship to adherence in patients with severe mental disorders. Acta Psychiatr Scand. 2009;119(1):78–84. pmid:18983630
- 19. Awad AG. Subjective response to neuroleptics in schizophrenia. Schizophr Bull. 1993;19(3):609–18. pmid:7901897
- 20. Hogan TP, Awad AG, Eastwood R. A self-report scale predictive of drug compliance in schizophrenics: reliability and discriminative validity. Psychol Med. 1983;13(1):177–83. pmid:6133297
- 21. Bonsack C, Conus P, Philippoz R, Bovet J, Spagnoli J, Dufour H. [Subjective perception of the effect of neuroleptics by outpatient schizophrenic patients: a cross-sectional study]. Encephale. 1998;24(4):315–23. pmid:9809236
- 22. Brain C, Allerby K, Sameby B, Quinlan P, Joas E, Karilampi U, et al. Drug attitude and other predictors of medication adherence in schizophrenia: 12 months of electronic monitoring (MEMS) in the Swedish COAST-study. Eur Neuropsychopharmacol. 2013;23(12):1754–62. pmid:24091164
- 23. Dinno A. Exploring the sensitivity of horn's parallel analysis to the distributional form of random data. Multivariate Behav Res. 2009;44(3):362–88. pmid:20234802
- 24. van der Eijk C, Rose J. Risky business: factor analysis of survey data—assessing the probability of incorrect dimensionalisation. PLoS One. 2015;10(3):e0118900. pmid:25789992
- 25. Kaiser HF. The varimax criterion for analytic rotation in factor analysis. Psychometrika. 1958;23(3):187–200.
- 26. Cattell RB. The scree test for the number of factors. Multivariate Behav Res. 1966;1(2):245–76. pmid:26828106
- 27. Horn JL. A rationale and test for the number of factors in factor analysis Psychometrika. 1965;30:179–85. pmid:14306381
- 28. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293–307. pmid:7550178
- 29. Beck E, Vogelin R, Wirtz M, Cavelti M, Kvrgic S, Vauth R. Do patients with schizophrenia distinguish between attitudes toward antipsychotic medication and pharmacotherapy in general? Validation of the Beliefs About Medication Questionnaire. J Nerv Ment Dis. 2012;200(1):33–43. pmid:22210360
- 30. Komninos I, Micheli K, Roumeliotaki T, Horne R. Adaptation and validation of the Beliefs about Medicines Questionnaire (BMQ) in primary care patients in Greece. Eur J Pers Cent Healthc. 2012;1(1):224–31.
- 31. Mahler C, Hermann K, Horne R, Jank S, Haefeli W, Szecsenyi J. Patients' beliefs about medicines in a primary care setting in Germany. J Eval Clin Pract. 2012;18(2):409–13. pmid:21087373
- 32. Salgado T, Marques A, Geraldes L, Benrimoj S, Horne R, Fernandez-Llimos F. Cross-cultural adaptation of the Beliefs about Medicines Questionnaire into Portuguese. Sao Paulo Med J. 2013;131(2):88–94. pmid:23657510
- 33. Pélissolo A, Maniere F, Boutges B, Allouche M, Richard-Berthe C, Corruble E. [Anxiety and depressive disorders in 4,425 long term benzodiazepine users in general practice]. Encephale. 2007;33(1):32–8. pmid:17457292
- 34. de Winter J, Dodou D, Wieringa P. Exploratory factor analysis with small sample sizes. Multivariate Behav Res. 2009;44(2):147–81. pmid:26754265