http://orcid.org/0000-0003-1252-0479
Figures
Abstract
Background
The Alcohol Use Disorders Identification Test (AUDIT) is the gold standard in assessing harmful alcohol intake, which is responsible for substantial morbidity and mortality.
Objective
The goal of this study is to evaluate the psychometric properties and factor structure of an Ecuadorian adaptation of a Spanish translation of the AUDIT in a large sample of college students in Ecuador.
Methods
A total of 7905 students, including 46.26% males, and 53.75% females, from 11 universities in Ecuador, were surveyed. The questionnaire was tested for two- and three-factor structures, reliability, and correlations with other health related measures.
Results
The Kaiser-Meyer-Olkin test for sampling adequacy was satisfactory (.0885), and Bartlett´s test for sphericity was significant (p < .001). Although both models showed a good fit to the data, the two-factor model was preferred based on the high correlations between the factors 2 and 3 within the three-factor model (.86 for the total sample, .77 for females, and .91 for males). The reliability for the two-factor model was good, as indicated by Cronbach´s α = .806 (factor I) and .716 (factor II) for the total sample, .808 (factor I) and .667 (factor II) for females, and .787 (factor I) and .728 (factor II) for males. Additionally, the AUDIT scores positively correlated with several health-related measures: stress, psychological inflexibility, loneliness and depression/anxiety symptomatology.
Conclusion
The Ecuadorian adaptation of the Spanish version of the AUDIT has good reliability, and internal consistency and correlates with other health related measures, proving to be a reliable tool that can be used by researchers and clinicians to screen hazardous alcohol intake in college students.
Citation: López V, Paladines B, Vaca S, Cacho R, Fernández-Montalvo J, Ruisoto P (2019) Psychometric properties and factor structure of an Ecuadorian version of the Alcohol Use Disorders Identification Test (AUDIT) in college students. PLoS ONE 14(7): e0219618. https://doi.org/10.1371/journal.pone.0219618
Editor: Gillian Shorter, Ulster University, UNITED KINGDOM
Received: January 5, 2019; Accepted: June 27, 2019; Published: July 10, 2019
Copyright: © 2019 López et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: This study was funded by the Particular Technical University of Loja (Ecuador) (Research Project PROY.PSC.1055 and the National Council for the Control of Narcotic Drugs and Psychotropic Substances (CONSEP). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist within 5 years of conducting the research or preparing the article for publication.
Introduction
Harmful use of alcohol causes substantial morbidity and mortality, including depression and anxiety symptoms, cognitive deficits, the abuse of other drugs, heart disease, cancer, road traffic accidents and violence, including suicide [1–2] with significant differences according to sex [3]. As a consequence, the reduction in the harmful consumption of alcohol is currently one of the major public health challenges in the western world [1,4,5,6].
The World Health Organization encourages the development and use of standardized instruments to enable cross-national comparisons. As such, the Alcohol Use Disorders Identification Test (AUDIT) was developed from a six-country WHO collaborative project as an instrument for the screening or early detection of hazardous and harmful alcohol use [7–8]. The original English-language version as well as many translated and language-adapted versions of the AUDIT are currently the gold standard a wide range of cross-national studies of both clinical samples [9–20] and the general population. College students represent an especially vulnerable population due to the underdevelopment of their prefrontal cortex and executive functions in comparison with those of adults [21–26].
The original version of the AUDIT developed by the World Health Organization originally consisted of three factors: alcohol consumption (items 1–3), drinking behavior (items 4–6), and alcohol-related problems (items 7–10). However, other studies reported a different factor structure [27–28]. For example, Bergman and Källmén [29] used a two-factor structure in a Swedish sample: “hazardous consumption” factor (items 1–3) and an “alcohol-related problems” factor (items 4–10). Shevlin & Smith [30] also employed a two-factor model in a large sample in Great Britain. Moreover, the use of two-factor models has been reported for both males and females in previous studies [27, 31, 32].
Ecuador is one of the countries with the highest rate of alcohol consumption in Latin America, according to the latest data from the World Health Organization [2]. However, an adaptation of the Spanish version of the AUDIT has yet to be developed. This work will address this gap in the currently available instruments to measure harmful alcohol consumption in the Ecuadorian population.
The aim of this study is to evaluate the psychometric properties and factor structure of an Ecuadorian adaptation of the Spanish version of the AUDIT in a large sample of college students, exploring potential differences by sex.
Methods
Participants
College students enrolled in 11 universities in Ecuador were invited via email to participate in the study, and they then completed a computerized survey within the 3-week assessment period. A total of 7905 participants met the inclusion criteria of being enrolled in, at least, one whole academic year; and completing the entire survey (the average response rate across universities was 47,8%, ranging from 39.10% to 56.30%). The average age was 21.49 years (SD = 3.68). Among the participants, 3656 (46.26%) were males, with an average age of 21.8 (SD = 3.7), and 4249 (53.75%) were females, with an average age of 21.2 (SD = 3.7).
Design
A descriptive cross-sectional study was conducted. Data were collected during the first three weeks of November 2015. Participation was confidential, and fully anonymous, and a brief summary of individual scores was freely provided after the completion of the survey to encourage honest answers and a higher response rate.
The study was approved by the Ethics Committee for Research in Human Beings (Comité de Ética de Investigación en Seres Humanos, CEISH) of the Ministry of Public Health of the Republic of Ecuador in June 2015 (MSP-DIS-2015-0088-O) and was conducted according to the principles expressed in the Declaration of Helsinki. Written informed consent was obtained from all participants. A language adaptation of the Spanish version of the AUDIT into Ecuadorian was carried out by a panel of three experts (see S1 Appendix). The final Ecuadorian Spanish adaptation was tested on 30 candidates as a pilot trial to test for the clarity and comprehensiveness of the questionnaire.
Measures
Alcohol Use Disorders Identification Test (AUDIT, Self-report version) [33]. The AUDIT consists of a 10-item questionnaire designed to screen for hazardous alcohol intake. Subjects respond to each question by indicating the frequency of alcohol consumption and/or the experience of symptoms related to problematic drinking on a scale of 0 ("never"), 4 ("4 or more times a week"), yielding a maximum possible score of 40. Higher scores indicate a higher risk of problematic alcohol consumption and a score of 8 or above is considered as a cut-off point for a hazardous score [34–35].
Perceived Stress Scale (PSS-14) [36]. This scale consists of 14 items to assess the degree to which people perceive a lack of control in their daily lives. Participants respond to a 5-point Likert-type scale ranging from 0 (“never”) to 4 (“very often”). Scores range from 0 to 56 points. Higher scores indicate higher levels of psychological stress. The Cronbach’s α coefficient for internal consistency reliability was .82 for males and .83 for females.
Loneliness Scale Revised-Short (UCLA) [37]. This instrument consists of a brief 3-item scale evaluating the subjective feeling of loneliness, understood as the perception of less social support being available than desired. Participants respond based on their agreement with a series of statements, where 1 = “hardly ever”, 2 = “sometimes”, and 3 = “often”. Scores range from 0 to 9. Higher scores indicate a greater feeling of loneliness or lack of social support. The Cronbach’s α coefficient for internal consistency reliability was .76 for males and .84 for females.
Acceptance and Action Questionnaire (AAQ-7) [38]. This questionnaire is the most widely used general measure of psychological inflexibility, defined as rigidity in the handling of emotions or unpleasant internal events. It consists of 7 items and participants respond to a 7-point Likert-type scale, from 1 = “never” to 7 = “always”. Scores range from 7 to 49. Higher scores indicate a tendency to act under the need to control or avoid aversive thoughts, memories or feelings. The Cronbach’s α coefficient for internal consistency reliability was .93 for males and .95 females.
Patient Health Questionnaire of Depression and Anxiety (PHQ-4) [39]. This questionnaire assesses depression and anxiety associated with functional impairment and disability days. Scores range from 0 to 12. A higher score indicates a greater anxiety and depression level.
Data analysis
The statistical analyses were carried out using the IBM Statistical Package for the Social Sciences (SPSS) software and AMOS version 21.0 for Windows. All analyses were carried out separately for the total, male and female samples (see S1 Database). Considering the skewed distribution of AUDIT scores, descriptive analyses were carried out using the median and interquartile range (IQR). Mann-Whitney U test was conducted as a non-parametric equivalent to the independent group test to compare differences between males and females. Then, a confirmatory factor analysis was conducted to reveal the factor structure of the AUDIT. The Kaiser-Meyer-Olkin test was used to assess sampling adequacy, and Bartlett´s test was used to assess sphericity assumptions. To assess the overall fit of the factor model, the following indexes were calculated: chi squared (χ2), comparative fit index (CFI), Tucker Lewis Index (TLI), root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR) and Akaike information criterion (AIC). Cronbach´s α was used to assess the reliability for the global score of two- and three-factor solutions in men, women and the total sample. In addition, Cronbach´s α was calculated independently for each item is displayed independently. Finally, two-tailed Pearson's correlations of the AUDIT scores with other health-related measures were conducted.
Results
AUDIT scores
Total scores, and scores by item are displayed in Table 1 for the total, male and female samples independently. Males scored significantly higher than females on the total scale and each one of the items. In fact, 46.74% of the male college students (n = 1709), and 24.14% of the females students in the sample (n = 1026), reported a score above the cut-off of 8 points (AUDIT score ≥ 8) for harmful alcohol consumption, and the 34.59% (n = 2735) of the total sample reported a score above the cutoff.
Factor structure
To determine whether the sample was large enough to perform a factor analysis, the Kaiser-Meyer-Olkin (KMO) measure was analyzed, yielding a KMO value = .885 for the total sample (.865 for females and .883 for males) indicating that the sampling was close to optimal. In addition, the assumption of sphericity was met as indicated by Bartlett's sphericity test: χ2(45) = 25138.625, p < .001 for the total sample; χ2(45) = 12179.569, p < .001 for females and χ2(45) = 11388.985, p < .001 for males. The confirmatory factor analysis performed by sex suggested that the questionnaire performed equally well for males and females with both the 2 and three-factor solutions (Table 2).
The confirmatory factor analysis reported goodness of fit indexes for both models: the 2-factor and 3-factor solutions. The AIC index was also low for both solutions (2-factors AIC = 532.6; 3-factors AIC = 303.1). Therefore, based on the logic of parsimony, the 2-factor solution is preferred, with the first factor corresponding to items 1–3, the second factor corresponding to items 3–10 (Table 3).
Reliability
The Ecuadorian adaptation of the Spanish AUDIT scale was found to be highly reliable for the total sample (α = .818), for females α = .795, and males (α = .816). Table 4 displays the matrix of Cronbach α coefficients for each item with the total score and α coefficients when each item is removed. Items 6, 9 and 10 failed to reach α coefficients above .50 for the total, female and male samples. Item 4 was also slightly below the expected .50 value. The reliability for the two-factor model was superior to that of the three-factor model for the total, female and male samples (Table 4).
AUDIT correlations with other health-related measures
Correlations between the AUDIT scores and 4 well-established health related measures in the scientific community were statistically significant: stress (r = .156, p < .001 in males and r = .157, p < .001 in females), loneliness (r = .114, p < .001 in males and r = .150, p < .001 in females), psychological inflexibility (r = .176, p < .001 in males and r = .213, p < .001 in females), and depressive and anxiety symptoms (r = .169, p < .001 in males and r = .171, p < .001 in females).
Discussion
A linguistically adapted and culturally acceptable equivalent of the Spanish version of the Alcohol Use Disorders Identification Test (AUDIT) was tested among a large sample of college students in Ecuador, reporting good reliability and a factor structure consistent with the original version.
As far as we know, this is the first and largest study to assess the appropriateness, meaningfulness, and usefulness of an adaptation of the AUDIT for measuring the risk of harmful alcohol consumption in college students in Ecuador; thus, this study responds to the need to explore psychometric properties in non-English versions and translated adaptations demanded by Reinert and Allen [40–41].
Sex differences deserve special attention. Males reported significantly higher levels of alcohol consumption than females. The score for the total sample was almost 2 points below the cut-off point of 8 assumed for harmful alcohol consumption. However, it is worth noting that alcohol abstainers were included in the calculation of that value, which explains the 5-point standard deviation. Furthermore, the prevalence of harmful alcohol consumption, above the cut-off score of 8, was strikingly high. Almost 50% of male college students and 25% female college students reported harmful alcohol consumption. Previous studies have reported a significantly higher risk for harmful drinking (AUDIT score > 8) in college students than in noncollege youth [42] and differences across countries [43]. The rates of harmful alcohol consumption (AUDIT score > 8) reported by college students in this study are similar to the rates reported in North America or New Zealand (countries with the highest rates, above 50% for males and females) [43], higher than Europe and South America (where rates range between 23–33% for males and 10–22% for females)[43], Africa, Asia, and counties from the Arab region (with the lowest reported rates, below 10%) [43]. In comparison with Ecuador, the rates of harmful alcohol consumption (AUDIT score > 8) found in this study are similar to the 39% prevalence of abusive drinking reported almost 15 years ago in the first research conducted in a college sample in Ecuador [44].
Sex differences are consistent with previous studies, which have already found both a high prevalence of harmful consumption among the general population and higher rates in males than in females [2,5]. The aforementioned pattern of sex differences was found not only in the AUDIT total scores but also in the scores for each item. Different scores between males and females may either support the use of different cut-off scores or evidence lower levels of harmful drinking in women [45]. Previous studies consider that women are more sensitive to alcohol than men, recommending lower the cut-off score for women from the recommended value of 8 points to a value of 6 points [27, 29, 40].
Regarding the reliability of the AUDIT, our results yielded a Cronbach's α = .818 for the total scale, .816 for males, and .795 for females, indicating the good reliability of the Ecuadorian adaptation of the Spanish version of the AUDIT. These results are in line with previous studies, which generally have reported Cronbach's α and item-total correlations values of approximately 0.80 [46].
Regarding the factor structure, confirmatory factor analysis supported the two-factor model and the three-factor models consistent with the original conceptual domains [28]. Indeed, the goodness-of-fit indexes provided support for both, the two and three-factor models, with both fitting the data equally well. However, the two-factor model is preferred in Ecuador based on the principle of parsimony, given the very high correlations between the factors 2 and 3 in the three-factor model, in accordance with previous studies aimed at detecting patients at risk for problematic drinking behavior in primary care [27,29,30,31]. It is worth noting that the factor loadings for item 9 were very low for both models, especially in the two factor models for females (0.29). Moreover, the correlation of item 9 with the total score was the lowest for the total sample (.33), females (.27) and males (.36). The poor functioning of this item has already been reported previously by the literature [32] suggesting the exclusion of this item because it does not generate score variability.
In addition, the AUDIT scores positively correlated with multiple well-established health-related measures, including psychological stress [36,43], loneliness [37], psychological inflexibility [38] and depressive/anxiety symptoms [39]. See Cohen, Janicki-Deverts, & Miller [47] or Remor [36] for a detailed description of the impact of stress on health; Cacioppo [48] or Hughes, et al. [37] for a review of the impact of loneliness on health; Kashdan & Rottenberg [49] or Bond et al. [38] for the role of psychological inflexibility on health; and Wingenfeld, et al. [39] for the link between depressive and anxiety symptoms and health. Moreover, although we acknowledge that correlations between AUDIT scores and other health outcomes measures of alcohol do not provide evidence about the validity of the AUDIT, such correlations have been used by previous studies to highlight the health implications of alcohol consumption for other health dimensions [32]. Future studies should explore further the validity of the AUDIT against other measures of alcohol use in college students in Ecuador.
The remarkably high AUDIT scores in our sample, especially among males, might be useful for anticipating and raising concerns about expected adverse health outcomes in this population in the near future. Other studies have already used AUDIT scores to predict alcohol-related illness, social problems, hospital admission [35], the development of alcohol withdrawal symptoms during inpatient detoxification [50], and even mortality over a 2-3-year period [35].
Finally, our results are consistent with previous studies that have reported the good psychometric properties of the English version of the AUDIT and other versions. Future studies should further explore the following areas: 1) the psychometric properties and factor structure of short versions of the AUDIT, for example, AUDIT-C, the most popular short version, which consists of the 3 consumption items of the AUDIT, and which seems to work as well as the full AUDIT [40,41,51,52]; 2) the correlation between AUDIT scores and biomarkers of alcohol drug abuse, including measures of liver enzymes, blood volume, alcohol byproducts, such as acetaldehyde, or differences in beta-endorphin or gamma-aminobutryic acid [53], in order to identify harmful cut-off scores for males and females in nonclinical samples [9]; and 3) the potential role of the item sequence should also be explored more systematically [54].
Supporting information
S1 Appendix. Ecuadorian-Spanish version of AUDIT.
https://doi.org/10.1371/journal.pone.0219618.s001
(DOCX)
Acknowledgments
The authors would like to thank to National Secretary for Higher Education, Science, Technology and Innovation (SENESCYT) of the Republic of Ecuador for its non-financial support via Project Prometeo.
References
- 1. Rehm J, Mathers C, Popova S, Thavorncharoensap M, Teerawattananon Y, Patra J. Global burden of disease and injury and economic cost attributable to alcohol use and alcohol-use disorders. Lancet. 2009;373:223–233.
- 2.
World Health Organization. Global status report on alcohol and health 2018. World Health Organization: Switzerland. 2018.
- 3. Nolen-Hoeksema S. Gender differences in risk factors and consequences for alcohol use and problems. Clinical Psychology Review. 2004;24: 981–1010. pmid:15533281
- 4. Nutt D, King LA, Saulsbury W, Blakemore C. Development of a rational scale to assess the harm of drugs of potential misuse. Lancet. 2007;369:1049–1053.
- 5.
World Health Organization. Global strategy to reduce the harmful use of alcohol. World Health Organization: Switzerland. 2010.
- 6. Wahlbeck K, McDaid D. Actions to alleviate the mental health impact of the economic crisis. World Psychiatry. 2012;11: 139–145. pmid:23024664
- 7. Bohn MJ, Babor TF, Kranzler HR. The Alcohol Use Disorders Identification Test (AUDIT): validation of a screening instrument for use in medical settings. Journal of studies on alcohol. 1995;56:423–32. pmid:7674678
- 8. Saunders JM, Aasland OG, Babor TF, Fuente JR, Grant ME. Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO Collaborative Project on Early Detection of Persons with Harmful Alcohol Consumption—II. Addiction. 1993; 88: 791–804. pmid:8329970
- 9. Allen JP, Reinert DF, Volk RJ. The alcohol use disorders identification test: An aid to recognition of alcohol problems in primary care patients. Preventive Medicine. 2001;33: 428–433. pmid:11676584
- 10. Almarri TS, Oei TP, Amir T. Validation of the alcohol use identification test in a prison sample living in the Arabian Gulf region. Substance Use & Misuse. 2009;44:2001–2013.
- 11. Carey KB, Carey MP, Chandra PS. Psychometric evaluation of the alcohol use disorders identification test and short drug abuse screening test with psychiatric patients in India. Journal of Clinical Psychiatry. 2003;64:767–773. pmid:12934976
- 12. Davey JD, Obst P L, Sheehan MC The use of AUDIT as a screening tool for alcohol use in the police work-place. Drug and Alcohol Review. 2000;19:49–54.
- 13. Dawe S, Seinen A, Kavanagh D. An examination of the utility of the AUDIT in people with schizophrenia. Journal of Studies on Alcohol. 2000;61:744–750. pmid:11022815
- 14. Gache P, Michaud P, Landry U, Accietto C, Arfaoui S, Wenger O, et al. The alcohol use disorders identification test (AUDIT) as a screening tool for excessive drinking in primary care: Reliability and validity of a French version. Alcoholism: Clinical and Experimental Research. 2005;29:2001–2007.
- 15. Kawada T, Inagaki H, Kuratomi Y. The alcohol use disorders identification test: reliability study of the Japanese version. Alcohol. 2011;45:205–207. pmid:20843637
- 16. Kim SS, Gulick EE, Nam KA, Kim SH. Psychometric properties of the alcohol use disorders identification test: a Korean version. Archives of Psychiatric Nursing. 2008; 22(4):190–199. pmid:18640538
- 17. Moussas G, Dadouti G, Douzenis A, Poulis E, Tzelembis A, Bratis D, et al. The Alcohol Use Disorders Identification Test (AUDIT): reliability and validity of the Greek version. Annals of General Psychiatry. 2009;8:11. pmid:19442281
- 18. O’Hare T, Sherrer MV, LaButti A, Emrick K. Validating the alcohol use disorders identification test with persons who have a serious mental illness. Research on Social Work Practice. 2004;14:36–42.
- 19. Pal HR, Jena R, Yadav D. Validation of the alcohol use disorders identification test (AUDIT) in urban community outreach and de-addiction center samples in North India. Journal of Studies in Alcohol. 2004;65:794–800.
- 20. Tsai MC, Tsai YF, Chen CY, Liu CY. Alcohol use disorders identification test (AUDIT): Establishment of cut-off scores in a hospitalized Chinese population. Alcoholism: Clinical and Experimental Research. 2005;29:53–57.
- 21. Aertgeerts B, Buntinx F, Bande-Knops J, Vandermeulen C, Roelants M, Ansoms S, et al. The value of CAGE, CUGE, and AUDIT in screening for alcohol abuse and dependence among college freshmen. Alcoholism: Clinical and Experimental Research. 2000;24:53–57.
- 22. Fleming MF, Barry KL, Macdonald R. The alcohol use disorders identification test (AUDIT) in a college sample. The International Journal of the Addictions. 1991;26: 1173–85. pmid:1743817
- 23. Hingson R, Heeren T, Zakocs R, Winter M, Wechsler H. Age of first intoxication, heavy drinking, driving after drinking and risk of unintentional injury among US college students. Journal of Studies in Alcohol. 2003;64:23–31.
- 24. Knight JR, Sherritt LR, Harris SK, Gates EC, Chang GC. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcoholism, clinical and experimental research. 2003;27:67–73. pmid:12544008
- 25. Kokotailo PK, Egan J, Gangnon R, Brown D, Mundt M, Fleming M Validity of the alcohol use disorders identification test in college students. Alcoholism: Clinical and Experimental Research. 2004;28:914–920.
- 26. Selin KH. Test-retest reliability of the alcohol use disorder identification test in a general population sample. Alcoholism: Clinical and Experimental Research. 2003;27:1428–1435.
- 27. Karno M, Granholm E, Lin A. Factor structure of the Alcohol Use Disorders Identification Test (AUDIT) in a mental health clinic sample. Journal of Studies on Alcohol. 2000;61:751–758. pmid:11022816
- 28. Maisto SA, Conigliaro J, McNeil M, Kraemer K, Kelley ME. An empirical investigation of the factor structure of the AUDIT. Psychological Assessment. 2000;12:346–353. pmid:11021159
- 29. Berman H, Källmén H. Alcohol use among Swedes and a psychometric evaluation of the alcohol use disorders identification test. Alcohol and alcoholism. 2002;37:245–51. pmid:12003912
- 30. Shevlin M, Smith GW. The factor structure and concurrent validity of the Alcohol Use Disorder Identification Test based on a nationally representative UK sample. Alcohol and Alcoholism. 2007;42: 5827.
- 31. Chung T, Colby SM, Barnett NP, Monti PM. Alcohol use disorders identification test: Factor structure in an adolescent emergency department sample. Alcoholism: Clinical and Experimental Research. 2002;26:223–231.
- 32. Kelly TM, Donovan JE. Confirmatory factor analyses of the alcohol use disorders identification test (AUDIT) among adolescents treated in emergency departments. Journal of Studies in Alcohol. 2001;62:838–842.
- 33.
Babor TF, Higgins-Biddle JC, Monteiro MG. AUDIT. The Alcohol Use Disorders Identification Test. Geneva (Switzerland): World Health Organization. 2001.
- 34. Berner MM, Kriston L, Bentele M, Härter M. The alcohol use disorders identification test for detecting at-risk drinking: A systematic review and meta-analysis. Journal of Studies on Alcohol and Drugs, 2007;68:461–473. pmid:17446987
- 35. Conigrave KM, Hall WD, Saunders JM. The AUDIT questionnaire: choosing a cut-off score. Alcohol Use Disorder Identification Test. Addiction. 1995; 90:1349–56. pmid:8616463
- 36. Remor E. Psychometric Properties of a European Spanish Version of the Perceived Stress Scale (PSS). Spanish Journal of Psychology. 2006;9:86–93. pmid:16673626
- 37. Hughes ME, Waite LJ, Hawkley LC, Cacioppo JT. A short scale for measuring loneliness in large surveys: Results from two population-based studies. Research in Aging. 2004;26:655–672.
- 38. Bond FW, Hayes SC, Baer RA, Carpenter KM, Guenole N, Orcutt HK, et al. Preliminary Psychometric Properties of the Acceptance and Action Questionnaire–II: A Revised Measure of Psychological Inflexibility and Experiential Avoidance. Behavior Therapy. 2011;42:676–688. pmid:22035996
- 39. Wingenfeld K, Schneider A, Brähler E. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. Journal of Affective Disorders. 2010;122:86–95. pmid:19616305
- 40. Reinert DF, Allen JP. The Alcohol Use Disorders Identification Test (AUDIT): a review of recent research. Alcoholism, clinical and experimental research. 2002;26:272–279. pmid:11964568
- 41. Reinert DF, Allen JP. The alcohol use disorders identification test: an update of research findings. Alcoholism: Clinical and Experimental Research. 2007;31(2):185–199.
- 42. Kypri K, Cronin M, Wright CS. Do university students drink more hazardously than their nonstudent peers? Addiction 2005;100:713–714. pmid:15847629
- 43. Karam E, Kypri K, Salamoun M. Alcohol use among college students: an international perspective. Current opinion in psychiatry. 2007;20(3):213–21. pmid:17415072
- 44. Chavez K, O’Brien B, Pillon S. Drugs use and risk behavior in a university community. Rev Lat Am Enfermagem 2005;13:1194–1200. pmid:16501792
- 45. DeMartini KS, Carey KB. Optimizing the use of the AUDIT for alcohol screening in college students. Psychological assessment. 2012;24(4):954. pmid:22612646
- 46. Meneses-Gaya CD, Zuardi AW, Loureiro SR, Crippa JA. Alcohol Use Disorders Identification Test (AUDIT): an updated systematic review of psychometric properties. Psychology & Neuroscience. 2009;2:83–97.
- 47. Cohen S, Janicki-Deverts D, Miller GE. Psychological stress and disease. Jama. 2007;298(14):1685–1687. pmid:17925521
- 48. Cacioppo JT. The growing problem of loneliness. Lancet. 2018; 391:426.
- 49. Kashdan T, Rottenberg J. Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review. 2010;30:865–878 pmid:21151705
- 50. Reoux JP, Malte CA, Kivlahan DR, Saxon AJ. The Alcohol Use Disorders Identification Test (AUDIT) predicts alcohol withdrawal symptoms during inpatient detoxification. Journal of addictive diseases. 2002;21:81–91. pmid:12296504
- 51. Gomez A, Conde A, Santana JM, Jorrin A. Diagnostic usefulness of brief versions of alcohol use disorders identification test (AUDIT) for detecting hazardous drinkers in primary care settings. Journal of Studies on Alcohol. 2005;66:305–308. pmid:15957683
- 52. Ivis FJ, Adlaf EM, Rehm J. Incorporating the AUDIT into a general population telephone survey: A methodological experiment. Drug and Alcohol Dependence. 2000;60:97–104. pmid:10821994
- 53. Peterson K. Biomarkers for alcohol use and abuse: a summary. Alcohol Research & Health. 2004;28:30–38.
- 54. Bischof G, Reinhardt S, Grothues J, Dybek I, Meyer C, Hapke U, et al. Effects of item sequence on the performance of the AUDIT in general practices. Drug and Alcohol dependence. 2005;79:373–377. pmid:16102379