Intolerance of aloneness is considered a core feature of borderline personality disorder (BPD) that is clinically significant, yet under-researched. This study developed a measure of aloneness for individuals with BPD.
Interviews investigating the experience of aloneness for BPD participants (n = 12) formed the basis for the development of the measure. Pilot testing then occurred with BPD participants, control participants and qualified respondents. Validity, reliability and factor analysis of an Experience of Time Alone Scale (ETAS) was conducted with BPD participants (n = 112) and a comparison control group (n = 105).
A three factor structure was revealed: (a) Cannot Cope Alone (α = .92), (b) Need to Escape from Others (α = .90), and (c) Consumed in Intolerable Distress (α = .88). The measure correlated significantly (p < .01) with the Mental Health Inventory, the Aloneness and Evocative Memory Scale, and the Hurvich Experience Inventory- Revised. Comparisons revealed highly significant differences between the BPD sample and control group on all subscales and the total score (U = 75.5, p = < .001, r = -.85).
Citation: Vardy Y, Day NJS, Grenyer BFS (2019) Development and validation of an experience of time alone scale for borderline personality disorder. PLoS ONE 14(5): e0217350. https://doi.org/10.1371/journal.pone.0217350
Editor: Stephan Doering, Medical University of Vienna, AUSTRIA
Received: January 8, 2019; Accepted: May 9, 2019; Published: May 23, 2019
Copyright: © 2019 Vardy et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data restrictions imposed by the University of Wollongong’s Human Research Ethics Committee prevent the uploading of participant data as it would constitute a violation of the approved procedure. Participants provided informed consent to participate in the current research and gave permission for the research team to use the data and for it to be confidentially secured. Participants did not provide informed consent for their data to be provided to online databases. Applications to access participant data will need to be addressed to Ethics Officer, Human Research Ethics Committee, University of Wollongong (email@example.com).
Funding: This study is part of a larger project funded by the New South Wales Ministry of Health (Grant DOH09-33) to BFSG. The funding body had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Borderline personality disorder (BPD) involves instability of interpersonal relationships, self-image, behaviours and affects . Globally, personality disorder prevalence is estimated at 6% , but individuals with BPD can account for up to 26% of emergency presentations and 25% of inpatient admissions within Australia , reflecting a mental health priority area . Intolerance of being alone was once a criteria for BPD in the Diagnostic and Statistical Manual of Mental Disorders , however, in spite of it being one of the more discriminating features, its inclusion in DSM-IV was voted down by committee members who favoured an atheoretical perspective . Despite its deletion, intolerance of being alone remains a key descriptive feature for BPD [7, 8]. As revision of personality disorder criteria is still ongoing , intolerance of being alone remains a clinically relevant yet under-examined feature of BPD that is worthy of further study.
Psychoanalytic theorists have explored both the concept of aloneness and its relationship to borderline pathology. One explanation for the intolerance of being alone in BPD may be that individuals experience annihilation anxiety . This is a traumatic anxiety based on an actual experience of danger and psychic helplessness , reflecting a fear of impending psychic or physical destruction . Helplessness may arise post-birth, when a child experiences independent needs and yet total dependency. The dependency is thought to evoke anxiety in the infant that if not met with consistent care and the subsequent internalization of a soothing other (an introject which continues the carer’s function in their absence), is experienced as annihilation of the self [13–18]. Empirical evidence from the field of developmental psychology demonstrates this fundamental need for connection and for consistent care of caregivers, as well as the distress experienced when this care is absent or removed. Meta-analyses of well-known paradigms such as the ‘strange situation’ and the ‘still face task’ indicate that the quality of parental connection is related to the subsequent attachment style of the infant [19, 20] with implications for psychopathology [21–23] and peer relations [23, 24] in later life. Thus aloneness has been contrasted with loneliness , as loneliness depends on an internalized sense of another who is absent, aloneness is instead a disorganizing feeling of utter helplessness because of a failure to evoke any sense of a soothing other . This annihilation anxiety may also be seen as a key component of the after-effects of trauma, neglect or abuse , commonly thought to be contributing factors in the etiology of BPD .
Fonagy draws on deficits in mentalization to understand intolerance of aloneness, and again suggests that these problems may arise from deficiencies in emotionally attuned care for the infant, including a lack of contingent and marked mirroring of the child’s internal states . Problems in mentalizing, or in the ability to think about one’s own or others’ mental states, is hypothesized to lead to a sense of emptiness and isolation, a depreciated sense of connection, discontinuity between past and present  and ultimately a diminished sense of identity. Furthermore, with chronic inaccurate reflection or maltreatment from a care-giver during development, Fonagy, et al  postulate that a malevolent and critical ‘alien self’ can develop and become internalized within the child, persisting into adulthood. When alone this alien self can attack the individual from within and thus creates intolerable distress from internal persecution. Suicide, self-harm and externalization of this difficult part of the self become a means of escape. A failure in secondary representation, or mentalization, akin to Meares, et al  loss of self-awareness or of the stream of consciousness in BPD, would result in these punitive thoughts being experienced as real, intense and immediate, rather than representations of reality that are one step removed. Thus, both deficits in mentalization and the resulting immediate experience of punitive internal states may result in distressing aloneness.
Adler and Buie  argued this painful aloneness is at the core of BPD and explains clingy desperate behaviours, rage when needs are unmet, self-hatred when fears of rejection arise, and impulsive attempts to avoid the feeling via drinking, sexual activity, suicidal and self-harming behaviours [34, 35]. Similarly, intolerance of aloneness has been argued to be a key feature underpinning most if not all DSM criteria for BPD . As such, the frequency and intensity of aloneness may reflect the severity of BPD, and therefore it is of major importance in treatment [33, 37].
Given the diagnostic relevance of this experience in BPD, it is surprising that only one empirical examination of aloneness has been conducted. By adding items drawn from psychoanalytic traditions to the University of California, Los Angeles (UCLA) Loneliness Scale , Richman and Sokolove  created the Aloneness and Evocative Memory Scale (AEMS). This scale focuses on emptiness, inner deadness, hollowness, the inability to self-soothe and the inability to recall comforting images. Using a BPD sample and a neurotic sample, the authors found that the experience of aloneness was the strongest predictor of BPD diagnosis. When aloneness was combined with memory quotient and level of depression, use of the scale could discriminate between 87.5% (35 of 40) of borderline and neurotic patients and the authors concluded that aloneness represents a defining feature of BPD. While not explicitly examining ‘aloneness’, recent empirical research using network analyses on the related concepts of ‘loneliness’ and ‘abandonment avoidance’ both highlight that the experience of being alone is intolerable for many individuals with BPD and that this is a defining feature of the disorder [40, 41].
More empirical research is needed regarding intolerance of aloneness in BPD. Consequently, this paper first aimed to investigate the actual experience of time alone for individuals with BPD. From this, the study aimed to develop a measure that reflects this described experience, in contrast to the AEMS  that evolved from an existing loneliness scale and was based on theoretical notions only. Finally, the study then aimed to evaluate the developed measure in terms of consistency and reliability. If considered a core feature of BPD, understanding and being able to measure the nature of the experience of being alone may give clinicians a direct measure of the severity of the illness. It may also provide a measure of the integrity of the self, an important insight into the capacity to soothe and regulate the self and a vital way to empathize with this painful experience. Given therapeutic relationships are self-evidently not in the context of being alone, a scale assessing this state may also yield important clinical information about specific problems and deficits that are difficult for the client to recall in an interpersonal environment.
Study 1: Investigation into the experience of aloneness in BPD
Aims and hypotheses
The aim of this study is to investigate the experience of time alone for individuals with BPD. It is hypothesized that participants will report an overall negative experience of time alone characterised by distressing internal experiences. Whilst the exact characterisation is unknown and provides the rationale for this qualitative study, it is expected time alone may be described as frightening in some way and aversive, and perhaps even intolerable, overwhelming or annihilating. It is expected that participants will report utilizing a range of strategies to avoid this experience.
Twelve women with a primary diagnosis of BPD attending the Affect Regulation Clinic were invited and agreed to participate in consultation with their treating clinician. The clinic, specializing in providing evidence-based psychological therapy for those with borderline personality disorder , operates in the local health service under a university-health service partnership . Six of the participants with BPD had been assessed as having comorbid diagnoses of mood and anxiety disorders, and one participant also identified as engaging in alcohol abuse. Diagnosis was initially established by their treating psychiatrist or clinical psychologist, and confirmed with the SCID-II . This number of participants is satisfactory for qualitative analysis [44, 45]. Participants’ age ranged from 19 to 56 (M = 36.3 years) and half were in a romantic relationship. All individuals agreed to participate following informed written consent procedures approved by the University of Wollongong’s Human Research Ethics Committee (HE2006/008).
Interviews were conducted face-to-face and audiotaped with consent from participants. Interviews involved five open ended questions, carefully constructed so as not to influence the nature of the responses. These were as follows:
- Generally speaking, how do you feel about time alone?
- What influences your experience of time alone?
- What do you do when you’re alone? How does this differ in quality (feel) or content (the things you do) when around others?
- Describe a typical experience of a time when you were alone. Try to get as detailed as possible so I really understand exactly how it felt for you and what you were thinking.
- Tell me about any times and ways you’ve tried to avoid the experience of time alone. Describe any times and ways you’ve deliberately sought the experience of time alone.
For complete transcripts and a more detailed account of the procedures followed, see Vardy .
Phenomenological analysis followed the procedure outlined by Giorgi  which assumes that if there is a common phenomenon of aloneness that people with BPD experience, then each individual with BPD would convey elements of that experience in their interview. To ensure reliability and clinical utility two researchers independently identified ‘meaning units’ of aloneness from the scripts . The researchers were both doctoral level clinical psychologists working within a personality disorders program. Through collaboration these units were synthesized into ‘consensus categories’ and further again into overall ‘themes’ that expressed the underlying psychological phenomena being described. For instance, the meaning units of: ‘more thoughts, worries, negative thoughts, constant thoughts’, ‘hears voices when alone’, ‘rumination increase’ and ‘busy mind when alone’ where all grouped together into one consensus category, as they were all viewed as expressing the same psychological phenomenon. This consensus category was labelled ‘intrusive negative thoughts’. The process of grouping meaning units into consensus categories helped to clarify when the different language or labels used by the two independent raters shared the same underlying psychological characteristics.
The categories were condensed further into themes based on function. Both researchers collaborated and used imaginal variation to reflect on when and how each category operated, and whether any tended to co-occur. For example, “alcohol/substances to escape or avoid”, and “sleep/dissociate as avoidant behaviour” were both considered part of the same theme “maladaptive attempts to escape distress” as they both occurred at the same times, and were used by participants in the same ways to manage time alone. The researchers then independently calculated the number of participants they endorsed as experiencing each theme and the Spearman’s Rank Reliability Coefficient highlighted strong, positive agreement between the two sets of ratings (r = 0.78).
Tables 1 and 2 display the themes and their latent meanings identified in the interviews. These themes encapsulate the essence of aversive (negative) and craved (positive) experiences of time alone for the twelve participants. As shown in Table 1, nine overarching themes were identified for a negative experience alone, consisting of (a) Intrusive negative thoughts; (b) Insecurity, conflict and indecision; (c) Self-recrimination; (d) Inability to control escalating panic and rage; (e) Maladaptive attempts to escape distress; (f) Avoidance of aloneness by structuring and filling time; (g) Reassurance and direction from others; (h) Lethargy, inactivity and amotivation; and, (i) Feelings of depression and disconnection.
Table 2 shows the seven themes identified for a positive experience alone arising from difficulties relating, namely (a) Freedom without compromise; (b) Settled and in touch with self; (c) Feelings of inadequacy, insecurity and rejection fears around others; (d) Effortful and self-sacrificing relating; (e) Invasion of senses and boundaries by others; (f) Overwhelming stress and anger around others; and, (g) Desperate need to escape from others.
Some participants described feelings of being continually uneasy and restless both alone and with others and a sense that inner peace was impossible.
Study 2: Development, validity and reliability of a self-report measure of the experience of time alone
Aims and hypotheses
First, this study aims to develop a measure that reflects the experience of being alone for individuals with BPD following from the results of the previous study. Second, this study aims to evaluate the developed measure in terms of validity and reliability. It is hypothesized that scores on the developed measure will be significantly higher for participants with BPD compared to healthy controls. Furthermore, that higher ratings of aloneness will significantly correlate with psychopathology, higher ratings of annihilation anxiety, and higher scores on a previous measure of aloneness which includes more failures in evocative memory.
Item development and pilot testing
Preliminary work was undertaken to design the items for a brief self-report scale to assess the nature and severity of the experience of time alone for people with BPD. The results from Study 1 were used to inform the development of the instrument.
Items were generated for each theme derived in Study 1, including some negatively weighted items. Language in the original interviews was used whenever possible to ensure the closest fit with the essential experience. In collaboration with the research team, items were selected that most clearly expressed a single idea without ambiguity. A four part response format was chosen for simplicity and ease of use (‘Not at all’, ‘A little bit’, ‘A moderate amount’, ‘A great deal’), mirroring the response format of Richman  in the Aloneness and Evocative Memory Scale. Items were placed in a random order so that similar items were separated and the reverse-scored items were spread throughout the scale to minimize response bias. Sixty two items underwent pilot testing and these reflected both the themes describing the experience of time alone and those that depicted the challenging nature of time with others.
Stage 1: Pilot testing with BPD participants.
Two female participants, aged 33 and 23 years old, both diagnosed with BPD and meeting the DSM-IV criteria according to the SCID-II  agreed to participate in the pilot testing. Neither participant took part in the original study and both were receiving treatment from a local health service. Based on feedback from these participants, revisions were made to various items regarding ease of understanding and clarity of items and one item, “I need time alone to be my true self”, was deleted due to reported trouble with identity and in understanding what a true self would be. Additionally, both participants raised issues with the “Time with Others” items, because their experience varied greatly depending on who the “others” were. The Time with Others section was therefore reviewed and modified in an attempt to make items clearer. Overall, participants reported that they found the questions clear, easy to understand and that the scale covered all the important areas of their experience of time alone.
Stage 2: Pilot testing with control participants.
Five healthy control participants, two males aged 35 and 63, and three females aged 32, 34 and 34 agreed to participate in pilot testing. Feedback in general was of a positive nature, and no modifications were made to items depicting time alone. However items describing time with other people continued to be reported as ambiguous or difficult to answer. Consequently, items were converted into statements depicting the positive experience of time alone once away from others. For example “I find myself getting edgy and irritable around others” was converted into “I need to have time alone because I get irritable and edgy around others”. Those with the clearest wording were selected for inclusion in the scale, also ensuring each theme from the original study was represented by at least one item.
Stage 3: Pilot testing with experienced and qualified clinicians.
Six health professionals (50% female) with a minimum experience of 10 years (mean = 16, SD = 4.18, range 10 to 20 years) in the treatment of BPD agreed to review the aloneness scale. These participants were clinical psychologists (n = 3), a research psychologist (n = 1), an occupational therapist (n = 1) and one nurse (n = 1). Respondents were asked to rate each item on the basis of clarity (ranging from ‘not clear’, ‘somewhat clear’, ‘quite clear’ and ‘highly clear’) and clinical relevance (ranging from ‘not relevant’, ‘somewhat relevant’, ‘quite relevant’ and ‘highly relevant’) as recommended by Davis . The content validity index (I-CVI) was calculated for each item on the basis of these expert ratings. This was done by computing the number of experts scoring each item as either ‘quite’ or ‘highly’ (dichotomizing the ratings into clear and not clear, or relevant and not relevant), divided by the total number of experts. I-CVI’s ranged from .50–1.0 for clarity and .67–1.0 for relevance. As specified by Lynn , acceptable level of agreement for six or more judges’ I-CVI ratings is to be no lower than 0.78. All those with an index score below 0.78 were modified until they met criteria or deleted, any that had an I-CVI below 1.0 were improved wherever possible.
Items retained at the end of pilot testing are presented in Table 3 as grouped into their themes. These modifications resulted in 58 items remaining in the scale.
After the pilot testing and subsequent modification, this study aimed to demonstrate the reliability and validity of the developed measure.
Participants were recruited online via psychology related email lists, links on mental health websites and from BPD Facebook pages. All individuals agreed to participate following informed written consent procedures approved by the University of Wollongong’s Human Research Ethics Committee (HE2006/192). This data collection strategy via online platforms has been found to be both effective and reliable . Inclusion criteria for the study were, (a) minimum age of 18; (b) completion of the survey–incomplete responses were assumed to indicate withdrawal; and (c) instances of response bias (e.g. responding the same to all items regardless of weighting). The participants were sorted into two groups on the basis of their response to the question “Have you been diagnosed with borderline personality disorder?” and their score on the McLean Screening Instrument for BPD (MSI-BPD) . All respondents self-declaring a diagnosis of BPD and with a score of seven or more on the MSI-BPD were allocated to the BPD group (n = 112). Respondents with no reported diagnosis of BPD and with a score of four or less on the MSI-BPD were allocated to the control group (n = 105). Participants who did not meet either criteria were excluded from analysis (n = 51). See Fig 1 for this screening and group allocation process.
McLean Screening Instrument–BPD (MSI-BPD).
The MSI-BPD  is a 10-item scale, with a Yes/No response format designed to reflect the nine DSM-IV criteria for BPD. This scale demonstrated very good internal consistency (α = .95, n = 217). A cut-off score of seven out of ten endorsed items has been shown to yield both good sensitivity (.81 correctly identified cases) and good specificity (.85 correctly identified non-cases) .
Hurvich Experience Inventory–Revised (HEI-R).
The HEI-R  is a 30-item self-report measure designed to assess annihilation anxiety, with a four-part response format of ‘Never’, ‘Not very often’, ‘Often’ and ‘Very often’. Internal consistency was demonstrated to be excellent (α = .98, n = 217). Content validity has been established via expert ratings and factor analysis revealed a six factor solution accounting for 54.3% of the variance, with factor loadings ranging from .40 to .81. Scores were found to significantly differ between non-clinical and clinical groups (at p < .001) 
Aloneness and Evocative Memory Scale (AEMS).
The AEMS  consists of two subscales that form a 33-item self-report scale, with answers ranging from ‘Not at all’, ‘A little bit’, ‘A moderate amount’ and ‘A great deal’. Factor 1 aloneness subscale items demonstrated an excellent Cronbach's alpha of α = .97 (n = 217), and the factor 2 evocative memory subscale had a very high Cronbach's alpha of α = .88 (n = 217).
Mental Health Inventory-5 (MHI-5).
The MHI-5  is a five item scale that measures five dimensions (anxiety, depression, positive affect, loss of behavioural or emotional control, and psychological well-being). The MHI-5 forms the Mental Health Scale from the Medical Outcomes Study Short Form Health Survey . Consistent with previous research, scores on the MHI-5 can be linear transformed to a scale of 0 to 100 . Higher scores are indicative of better mental health. The MHI-5 demonstrated strong internal consistency (α = .84, n = 217).
Participants were asked for general descriptive information about age, gender, education, employment status, living arrangements, children and relationship status. Individuals were asked whether they had ever been diagnosed with BPD and whether they had any other current mental illnesses.
Data screening of the BPD sample was conducted prior to running the main analyses. Z scores were calculated to assess for univariate outliers, which were defined at +/- 3.29 standard deviations from the mean . Scores ranged from z = .76 to z = 3.16 in the sample, and as such no data were excluded. Mahalanobis distance was also calculated to assess for multivariate outliers in the BPD sample, using the chi-square cut off of p < .001 for 57 items (degrees of freedom). The maximum distance 80.69 (M = 56.49, SD = 10.26) was below the critical value of 95.75, thus no data were excluded from the BPD sample. Data screening of control participant responses were also conducted. Analyses revealed a response bias to the ‘not at all’ category (reflecting comfort with time alone), and as such those who scored themselves as ‘a moderate amount’ or ‘a great deal’ were classed as outliers with z-scores above 3.29. Additionally, examination using the Kolomogorov-Smirnov text revealed that item responses were not normally distributed in either group (p <. 001). The response distribution in the BPD sample showed individuals responded most to the ‘a great deal’ category and least to the ‘not at all’ category, and the control sample largely demonstrated the opposite pattern. These differing response tendencies actually indicated that the items performed as expected, successfully reflecting the prototypical experiences of these samples and, consequently, no individual’s data were removed despite the outliers in the control sample. Visual inspection of the means obtained by both the BPD and control groups indicated that all but one item ‘I need time alone to unwind and de-stress’ performed as expected and hence this item was removed, leaving 57 items.
Where data was not normally distributed, non-parametric tests were utilized for between-groups comparisons. As factor analysis is typically considered robust enough to handle deviations in normality  no data transformations were required for this test. Chi-square and t-tests were used to examine demographic differences, and differences between other measures (i.e. MHI-5, AEMS). Principle Axis Factoring with promax rotation was used to investigate the latent structure of the experience of time alone along with the association the current items have with this latent structure . The items retained after factor analysis formed the scale. The scale was then subjected to correlation analysis with theoretically relevant measures (i.e. MHI-5, AEMS) to demonstrate validity. Finally, the Mann Whitney U test was utilized to test for differences between samples (BPD vs. control) for the developed measure.
Demographic details for both BPD and control groups are presented in Table 4. The majority (n = 94, 83.9%) of BPD participants identified that they were suffering from at least one co-morbid disorder, and given the common nature of such co-morbidities, no individual was excluded on this basis. Seven control participants indicated that they were currently suffering from a mental illness, however no significant difference was evident in their MHI-5 scores (M = 23.00, SD = 3.06) compared to the rest of the control sample (M = 24.78 SD = 3.07), t(103) = -1.48, p = .142 (two-tailed), therefore they were not excluded from the analyses.
Means and standard deviations for the questionnaires administered in the battery are presented in Table 5. Differences between groups on all measures were significant.
The 57 prospective items retained after pilot testing and data screening were subjected to principal axis factoring with promax rotation utilizing the BPD sample data. The Kaiser-Meyer-Olkin measure of sampling adequacy (.80) indicated the data were suitable for factor analysis, as did Bartlett’s Test of Sphericity (p < .001). Horn’s parallel analysis  was conducted using SPSS syntax proposed by O'Connor  to identify the number of factors within the data, as this has been identified as the most reliable method . Three factors were identified with eigenvalues of 13.41, 7.85 and 2.98 respectively. Items with factor loadings of less than .50 were suppressed (n = 20). Factor 1 loaded highly on 15 items, Factor 2 on 10 items and Factor 3 on 12 items, as displayed in Table 6. These factors accounted for around 41.11% of the variance in the questionnaire data. Factor 1 was labeled Cannot Cope Alone, depicting a need for another person in order to function and feel comfortable, the rising anger and anxiety about being left alone, and the use of alcohol and sleep to avoid the feeling of being alone. Factor 2 was labeled Need to Escape from Others, representing the need for time alone to escape the intrusion and demands of relating, and to escape from the negative emotions (e.g. self-consciousness) associated with being around others. Factor 3 was labeled Consumed in Intolerable Distress, depicting the increasing and all-consuming nature of distress when alone, in which hopelessness prevailed, and thoughts of self-harm and suicide dominated. Factor correlations revealed factor 1 and 3 to be significantly correlated r = .58, p < .001. Factor 2 did not correlate significantly with either of the other two factors and may therefore tap into independent underlying psychological characteristics. A total of 37 items across the three factors were retained after these analyses.
Factor 1 had an alpha coefficient of α = .92. Corrected item-total correlations ranged from .50 to .75 which classifies it as ‘very good’ . Inter-item correlations ranged from .18 to .69. The lowest correlation was below the minimum r of .20 as specified by Ferketich  and was between the items ‘Alone I’m overwhelmed by simple tasks and have to push myself to do them’ and ‘When I’m alone I crave a deep personal connection with someone’. The latter item was removed from the scale because it affected alpha less and was conceptually similar to other items. The final coefficient alpha for Factor 1: Cannot Cope Alone (14 items) was α = .92 which is classified as ‘excellent’ .
Factor 2 alpha coefficient increased from α = .92 to α = .93 when the item ‘I need time alone to work out how I feel about things’ was removed from the scale, due to weak loading. Four items correlated above the recommended cut-off point of .70  to the item ‘I need to escape and be by myself to avoid being totally overwhelmed by others’, so two items were removed (‘It is a relief to be on my own because I find it too intense around others’ and ‘Time alone is a relief from feeling self-conscious around others’). The other two items (‘Time alone is freedom from the worry that I will say or do the wrong thing around others’ and ‘It is a relief to be alone because I do not have to maintain a false self or mask to conceal the real me’) were retained, as they were thought to reflect unique content (i.e. being overwhelmed by others is different to the effort of putting on a false persona or from fears of saying or doing the wrong thing). The inter-item correlations for the remaining seven items ranged from .41 to .74. Alpha was .90 or ‘excellent’  and corrected item-total correlations were between .63 and .83 or ‘very good’ .
Factor 3 alpha coefficient was α = .88 or ‘good’ . Alpha was not increased by removing any item, inter-item correlations ranged from .21 to .65 and corrected item-total correlations ranged from .45 to .70 or ‘very good’ . As such, no items were removed from this factor.
The remaining 33-items after these deletions formed the Experience of Time Alone Scale (ETAS), each factor constituting a subscale of the measure, as displayed in Table 6.
Content validity was built into the test from the outset, with items being derived from Study 1. Qualitative analysis of the interviews with people with BPD ensured the items reflected the experiences as described by participants. Validity was then further strengthened by the pilot studies involving individuals with BPD, healthy participants and experienced and qualified respondents. Validity of the scale was also demonstrated through significant correlations with theoretically relevant clinical measures (AMES, HEI-R, MHI-5) as displayed in Table 7.
As items for the scale were developed using a female-only sample, investigation into any potential gender effects was undertaken. Results revealed no statistically significant differences between the genders for any of the subscales: Cannot Cope Alone, U = 382, z = -.38, p = .71; Need to Escape from Others, U = 325.50, z = -1.02, p = .31; Consumed in Intolerable Distress, U = 387, z = -.33, p = .75. Total ETAS score was also non-significant across genders, U = 393.50, z = -.25, p = .81. However, the small sample size of men means these results should be interpreted cautiously.
The BPD scores on the ETAS were compared with the control sample scores and revealed significant differences between groups on all three subscales and the total ETAS score, as displayed in Table 8. Effect sizes of these between-group differences were estimated by Pearson’s correlation coefficient r  and were all found to be “large” .
The final version of the scale with randomized items is presented in S1 Table.
The aim of these studies was to develop a measure of the experience of time alone for individuals with BPD. First, Study 1 examined the essential experience of time alone for BPD participants. Based on these findings a self-report scale was created and pilot-tested to assess the soundness and clarity of the measure. Study 2 then aimed to demonstrate the reliability and validity of the developed scale through correlations with theoretically relevant measures and comparisons between groups in their responses to the scale. As hypothesized, mean scores on the ETAS and all other measures demonstrated that there were significant differences between BPD and control participants in the aversive experiences of aloneness, in general psychopathology, in the intensity of annihilation fears and in both aloneness and evocative memory. BPD participants strongly endorsed ETAS items yet control participants did not indicate that they related to or shared in these experiences to the same extent or even, sometimes, at all. Furthermore, as predicted, higher ratings of aloneness did significantly correlate to poorer general psychopathology, higher ratings of annihilation anxiety, and higher scores on existing measures of aloneness , including poorer evocative memory. These results suggest the ETAS to be a valid measure of the experience of time alone for people with BPD.
The accounts of a negative experience of time alone captured in Study 1 reflect previous depictions of the intolerance of aloneness in BPD. As described by previous theorists participants reported desperately trying to fill time alone with activities and by contacting others, as well as resorting to maladaptive means such as self-harm to rid themselves of their overwhelming distress and rage [33, 34]. These findings are consistent with recent research into the separate but related construct of chronic emptiness, in which chronic feelings of emptiness were significantly related to self-harm behaviours for individuals with BPD . It would be expected that these chronic feelings would be experienced as more intense in the context of being alone. Indeed, Köhling et al.  report that for individuals with major depressive disorder and BPD, significant reductions in mood were reported after a period of being alone. This reduction in mood was significantly more intense than a comparison group of individuals with major depression only, indicating that a lower tolerance for being alone may be specific to BPD.
Pazzagli and Monti  argue that in line with Kernberg’s [71, 72] pain-anger-hate-vengefulness cycle this rage may function to punish yet preserve a sense of an absent other. Rumination, worry and negative thinking may be other ways individuals preserve a sense of the other in their absence, and participants reported that it was the intensity and intrusiveness of their negative thoughts that made time alone intolerable. However, persistent rumination (as opposed to escalating distress and emotion) and the experience of being bombarded by intrusive negative thoughts has not been the focus of previous descriptions of aloneness. Similarly, theories have not tended to focus on individuals difficulties initiating activity via indecision and a pattern of becoming lethargic, depressed, depleted and isolating themselves away from others. This tendency to sleep or become completely inactive was reported as lasting for days or weeks on end, and may represent Masterson’s [18, 73] abandonment depression, an underlying anaclitic depression  or the core of aloneness that Gunderson and Links  suggest can result in psychotic symptoms and dissociation.
A further significance of Study 1 is that participants described actually craving time alone, an aspect that is missing from many accounts of aloneness in BPD. This positive experience was not due to feelings of contentment and peace when alone, but rather, because of the relief experienced when escaping the demands of relating, supporting the idea that alone time can be conceptualized as a need [75, 76]. Together, the positive and negative experiences of time alone can be understood as consistent with Dazzi’s  view that both intolerance of aloneness and intolerance of relating are pivotal in BPD. Individuals reported struggling with feelings of alienation, and would hide themselves with a false persona out of fears of rejection [16, 77]. Additionally, fears of intrusion and engulfment preoccupied individuals when around others.
In Study 2, group comparisons revealed significant differences between control and BPD participants in all demographic comparisons, many of which were unsurprising given the nature of the diagnosis and the psychosocial impact of having a serious mental illness. As such these differences were not anticipated to impact meaningfully on the validity of results. Differences between groups on all measures were significant indicating those with BPD were more distressed (MHI-5), had more severe problems with aloneness and evocative memory (AEMS), and experienced more annihilation anxiety (HEI-R) compared to non-BPD controls.
Results from the between-groups comparison also showed that the ETAS clearly taps into the borderline experience of aloneness, with very large differences between the individuals with BPD and controls on all three subscales and the total score. The three factors that make up the ETAS reflect theoretically relevant areas of concern for individuals with BPD that healthy controls do not tend to struggle with. The ‘Cannot Cope Alone’ factor encapsulates notions of dependency, fears of abandonment as consistent with DSM-5 criteria  and is similar to a presentation of preoccupied attachment in BPD . The ‘Need to Escape from Others’ factor depicts a fear of being judged, being intruded upon and rising irritability around others. This factor may be understood by the notion of an ‘alien self’ , or of hyper-mentalization, reflecting a consuming preoccupation with the assumed thoughts and feelings of others as well as an automatically negative self-appraisal . It may also reflect annihilation from a weak self being encroached upon by external demands and stimuli . As such, time alone is felt as a relief from the effortful nature of interacting. The ‘Consumed in Intolerable Distress’ factor most clearly resembles Fromm-Reichmann’s  description of aloneness, as being unable to recall a soothing presence, and thus left helpless with annihilating feelings and consuming negative mental states.
Finally, validity of the ETAS was also demonstrated through significant correlations with theoretically relevant measures reflecting that, as expected, individuals with more severe experiences of time alone have greater distress, increased difficulties with both aloneness and evocative memory, and stronger reported feelings of annihilation.
There are two caveats to consider with respect to these studies. First, participants in Study 1, which formed the basis of the ETAS, were all women. While further examination of differences between genders on scores of the ETAS revealed no significant differences, the small sample size of men means these results should be interpreted with caution. It may also indicate that there are core themes of the experience of time alone that men encounter that are not reflected in the scale items. Qualitative interviews with males diagnosed with BPD investigating their experience would be a possible avenue for future research.
The second caveat is that an online format was used for administering the scale in Study 2, which prevents assessing BPD presentation via a formal structured clinical interview and potentially impacts validity and generalizability of results. However, data collection via online platforms has previously been found to be effective and reliable, particularly for personality disorder research . Further, as results demonstrate clear discrimination between BPD and control participants, this indicates that this method is adequate and that the measure was sensitive. However, as the reliability and preliminary validity was established in an online format, it needs to be recognised that findings may not be generalisable to paper and pencil versions of the scale, or versions of the scale conducted in group or interpersonal settings. Whilst there is no a priori reason that results in different contexts or formats would be different, this is something that has not been established with the ETAS and could be a direction for future research.
Given BPD participants’ scores were so clearly different to control scores, and given fear and intolerance of being alone is linked to most, if not all, current DSM-5 criteria , the ETAS has clinical potential as a diagnostic aide. The scale also demonstrates clinical utility through various individual items that would prove useful in a therapy setting. For instance, endorsement of items such as ‘I get so upset when I am alone that I hurt myself’, and ‘I sleep to avoid being alone’ would possibly warrant further exploration in psychotherapy, and as such, use of the ETAS may enhance and guide treatment. Future studies utilizing a large cohort of representative participants could assess the robustness of these findings, could further investigate the relationship between aloneness and BPD diagnosis, and could explore interactions with self-harming or suicidal behaviors. Further, while these studies focused on aloneness in the context of a specific BPD population, the current conversation regarding dimensional versus categorical approaches [9, 81] may indicate the experience of being alone as a relevant feature for a wide variety of severe personality (or other) pathology. As such, investigating the experience of being alone within a transdiagnostic or dimensional trait based framework such as the borderline personality ‘organisation’  or borderline ‘pattern’  would be of interest. Alternatively, if a categorical approach is favored, exploring the experience of time alone for other personality disorders (e.g. narcissistic, dependent) may also be an avenue for future research.
The ETAS is the only measure that assesses the experience of time alone in BPD as described by individuals with the disorder. Time alone for people with BPD is a psychological challenge and, along with the experience of relating, warrants renewed attention. As such, this measure provides clinicians with an evidence-based instrument to assist the treatment process, responding to the need for continued focus on effective treatment and research into BPD [4, 84].
The authors would like to acknowledge Dr Hamish McLeod for reviewing an earlier version of this study.
- 1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Arlington, VA: American Psychiatric Association; 2013.
- 2. Tyrer P, Mulder R, Crawford M, Newton-Howes G, Simonsen E, Ndetei D, et al. Personality disorder: a new global perspective. World Psychiatry. 2010;9: 56–60. pmid:20148162
- 3. Grenyer BFS. An integrative relational step-down model of care: The Project Air Strategy for personality disorders. The ACPARIAN. 2014;9: 8–13. Available from: https://projectairstrategy.org/content/groups/public/@web/@ihmri/documents/doc/uow177533.pdf.
- 4. Grenyer BFS, Ng FY, Townsend ML, Rao S. Personality disorder: A mental health priority area. Australian and New Zealand Journal of Psychiatry. 2017;51: 872–5. pmid:28681611
- 5. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 3rd ed. Washington, DC: American Psychiatric Association; 1980.
- 6. Gunderson JG. The borderline patient's intolerance of aloneness: insecure attachments and therapist availability. The American journal of psychiatry. 1996;153: 752–8. pmid:8633685
- 7. Gunderson JG, Kolb JE. Discriminating features of borderline patients. The American journal of psychiatry. 1978;135: 792–6. pmid:665789
- 8. Dazzi S. Some thoughts concerning borderline pathology and fear of aloneness. Journal of the American Academy of Psychoanalysis. 1998;26: 69–84. pmid:9724914
- 9. Grenyer BFS. Revising the diagnosis of personality disorder: Can it be single, clinical and factorial? The Australian and New Zealand journal of psychiatry. 2017;52: 202–3. pmid:29143533
- 10. Freud S. Inhibitions, symptoms and anxiety. Standard edition. 1926;20: 87–174.
- 11. Hurvich M. Fear of being overwhelmed and psychoanalytic theories of anxiety. The Psychoanalytic review. 2000;87: 615–49. pmid:11242776
- 12. Benveniste PS, Papouchis N, Allen R, Hurvich M. Rorschach assessment of annihilation anxiety and ego functioning. Psychoanalytic psychology. 1998;15: 536–66.
- 13. Winnicott DW. The capacity to be alone. The International Journal of Psycho-Analysis. 1958;39: 416–20. Available from: http://icpla.edu/wp-content/uploads/2012/10/Winnicott-D.-The-Capacity-to-be-Alone.pdf. pmid:13610513
- 14. Modell A. Primitive object relationships and the predisposition to schizophrenia. The International Journal of Psychoanalysis. 1963;44: 282–92. Available from: http://psycnet.apa.org/record/1965-05761-001.
- 15. Buie DH, Adler G. Definitive treatment of the borderline personality. International Journal of Psychoanalytic Psychotherapy. 1982;9: 51–87. pmid:6759434
- 16. Singer M. The experience of emptiness in narcissistic and borderline states: II. The struggle for a sense of self and the potential for suicide. International Review of Psycho-Analysis. 1977;4: 471–9. Available from: http://psycnet.apa.org/record/1979-13573-001.
- 17. Singer M. The experience of emptiness in narcissistic and borderline states: I. Deficiency and ego defect versus dynamic-defensive models. International review of Psycho-Analysis. 1977;4: 459–69. Available from: http://psycnet.apa.org/record/1979-13572-001.
- 18. Masterson JF. Treatment of the adolescent with borderline syndrome: A problem in separation-individuation. Bulletin of the Menninger Clinic. 1971;35: 5–18. Available from: http://psycnet.apa.org/record/1971-25212-001. pmid:5100653
- 19. Lucassen N, Tharner A, Van Ijzendoorn MH, Bakermans-Kranenburg MJ, Volling BL, Verhulst FC, et al. The association between paternal sensitivity and infant-father attachment security: a meta-analysis of three decades of research. J Fam Psychol. 2011;25: 986–92. pmid:22004434
- 20. Mesman J, van Ijzendoorn MH, Bakermans-Kranenburg MJ. The many faces of the Still-Face Paradigm: A review and meta-analysis. Developmental Review. 2009;29: 120–62.
- 21. Madigan S, Atkinson L, Laurin K, Benoit D. Attachment and internalizing behavior in early childhood: A meta-analysis. Developmental Psychology. 2013;49: 627–89. pmid:22686171
- 22. Colonnesi C, Draijer EM, Jan JMSG, Van der Bruggen CO, Bogels SM, Noom MJ. The relation between insecure attachment and child anxiety: a meta-analytic review. J Clin Child Adolesc Psychol. 2011;40: 630–45. pmid:21722034
- 23. Groh AM, Roisman GI, van Ijzendoorn MH, Bakermans-Kranenburg MJ, Fearon RP. The significance of insecure and disorganized attachment for children's internalizing symptoms: a meta-analytic study. Child Dev. 2012;83: 591–610. pmid:22235928
- 24. Pallini S, Baiocco R, Schneider BH, Madigan S, Atkinson L. Early child-parent attachment and peer relations: a meta-analysis of recent research. J Fam Psychol. 2014;28: 118–23. pmid:24512287
- 25. Fromm-Reichmann F. Loneliness. Psychiatry. 1959;22: 1–15. pmid:13634274
- 26. Frailberg S, Adelson E, Shapiro V. Ghosts in the nursery: A psychoanalytic approach to the problems of impaired infant-mother relationships. Journal of the American Academy of Child Psychiatry. 1975;14: 387–421. pmid:1141566
- 27. Davies JM, Frawley MG. Treating the adult survivor of childhood sexual abuse: A psychoanalytic perspective. New York: Basic Books; 1994.
- 28. Winsper C. The aetiology of borderline personality disorder (BPD): contemporary theories and putative mechanisms. Curr Opin Psychol. 2018;21: 105–10. pmid:29182951
- 29. Fonagy P. Thinking about thinking: some clinical and theoretical considerations in the treatment of a borderline patient. The International Journal of Psycho-Analysis. 1991;72: 639–56. Available from: http://psycnet.apa.org/record/1992-20749-001. pmid:1797718
- 30. Fonagy P, Target M, Gergely G, Jurist EL. Affect regulation, mentalization, and the development of the self. New York: Other Press; 2002.
- 31. Fonagy P, Target M, Gergely G, Allen JG, Bateman A. The Developmental Roots of Borderline Personality Disorder in Early Attachment Relationships: A Theory and Some Evidence. Psychoanalytic Inquiry. 2003;23: 412–59.
- 32. Meares R, Stevenson J, Gordon E. A Jacksonian and biopsychosocial hypothesis concerning borderline and related phenomena. Australia and New Zealand Journal of Psychiatry. 1999;33: 831. pmid:10619209
- 33. Adler G, Buie DH. Aloneness and borderline psychopathology: the possible relevance of child development issues. International journal of psychoanalysis. 1979;60: 83–96. Available from: https://www.ncbi.nlm.nih.gov/pubmed/457345. pmid:457345
- 34. Chessick RD. Defective ego feeling and the quest for being in the borderline patient. International Journal of Psychoanalytic Psychotherapy. 1974;3: 73–89. Available from: http://europepmc.org/abstract/med/4430567. pmid:4430567
- 35. Pazzagli A, Monti MR. Dysphoria and aloneness in borderline personality disorder. Psychopathology. 2000;33: 220–6. pmid:10867581
- 36. Gunderson JG, Links PS. Borderline personality disorder: A clinical guide. Arlington, VA: American Psychiatric Publishing; 2008.
- 37. Adler G. The psychotherapy of core borderline psychopathology. American Journal of Psychotherapy. 1993;47: 194–205. pmid:8517469
- 38. Russell D, Peplau LA, Cutrona CE. The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. Journal of personality and social psychology. 1980;39: 472–80. pmid:7431205
- 39. Richman NE, Sokolove RL. The experience of aloneness, object representation, and evocative memory in borderline and neurotic patients. Psychoanalytic psychology. 1992;9: 77–91.
- 40. Southward MW, Cheavens JS. Identifying Core Deficits in a Dimensional Model of Borderline Personality Disorder Features: A Network Analysis. Clinical Psychological Science. 2018;6: 685–703. pmid:30854263
- 41. Richetin J, Preti E, Costantini G, De Panfilis C. The centrality of affective instability and identity in Borderline Personality Disorder: Evidence from network analysis. PloS one. 2017;12: e0186695. pmid:29040324
- 42. Carter PE, Grenyer BFS. Expressive language disturbance in borderline personality disorder in response to emotional autobiographical stimuli. Journal of Personality Disorders. 2012;26: 305–21. pmid:22686220
- 43. First MB, Spitzer RL, Gibbon M, Williams JBW, Benjamin L. Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II), version 2.0. New York: New York State Psychiatric Institute, Biometrics Research Department; 1994.
- 44. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18: 59–82.
- 45. Crouch M, McKenzie H. The logic of small samples in interview-based qualitative research. Social Science Information. 2006;45: 483–99.
- 46. Vardy Y. The experience of aloneness in borderline personality disorder: University of Wollongong; 2011.
- 47. Giorgi A. Sketch of a psychological phenomenological method. Phenomenology and psychological research. 1985;1: 23–85. Available from: https://ci.nii.ac.jp/naid/10030239159/.
- 48. Giorgi A. Validity and Reliability from a Phenomenological Perspective. In: Baker J, Mos L, Rappard H, Stam H, editors. Recent Trends in Theoretical Psychology. Canada, Alberta: Banff: Springer-Verlag; 1987.
- 49. Richman NE. Borderline personality: An investigation of object representation, evocative memory, and the experience of aloneness. [Unpublished doctoral dissertation]. In press 1986.
- 50. Davis LL. Instrument review: Getting the most from a panel of experts. Applied nursing research. 1992;5: 194–7.
- 51. Lynn M. Determination and quantification of content validity. Nursing research. 1986;35: 382–5. pmid:3640358
- 52. Miller JD, Crowe M, Weiss B, Maples-Keller JL, Lynam DR. Using online, crowdsourcing platforms for data collection in personality disorder research: The example of Amazon's Mechanical Turk. Personal Disord. 2017;8: 26–34. pmid:28045305
- 53. Zanarini MC, Vujanovic AA, Parachini EA, Boulanger JL, Frankenburg FR, Hennen J. A screening measure for BPD: The Mclean screening instrument for borderline personality disorder (MSI-BPD). Journal of Personality Disorders. 2003;17: 568–73. pmid:14744082
- 54. Hurvich M. Hurvich Experience Inventory—Revised. [Unpublished scale.]. In press 1990.
- 55. Hurvich M, Allen R, McGuire H. Hurvich experience inventory/50 manual, revised. Brooklyn, NY: Long Island University; 2007.
- 56. Berwick DM, Murphy JM, Goldman PA, Ware JE Jr, Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Medical care. 1991;29: 169–76. pmid:1994148
- 57. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical Care. 1992;30: 473–83. Available from: https://www.jstor.org/stable/3765916. pmid:1593914
- 58. Cuijpers P, Smits N, Donker T, ten Have M, de Graaf R. Screening for mood and anxiety disorders with the five-item, the three-item, and the two-item Mental Health Inventory. Psychiatry Res. 2009;168: 250–5. pmid:19185354
- 59. Tabachnick BG, Fidell LS. Using multivariate statistics. Boston: Allyn and Bacon; 2001.
- 60. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychological methods. 1999;4: 272.
- 61. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30: 179–85. A rationale and test for the number of factors in factor analysis. pmid:14306381
- 62. O'Connor BP. SPSS and SAS Programs for Determining the Number of Components Using Parallel Analysis and Velicer's MAP Test. Behavior research methods. 2000;32: 396–402.
- 63. Zwick W, Velicer W. Comparison of five rules for determining the number of components to retain. Psychological bulletin. 1986;99: 432–42.
- 64. Ebel RL. Essentials of educational measurement. Third ed. Englewood Cliffs, NJ: Prentice-Hall; 1979.
- 65. Ferketich S. Internal consistency estimates of reliability. Research in nursing & health. 1990;13: 437–40.
- 66. George G, Mallery P. SPSS for Windows Step by Step: A simple guide and reference, 11.0 update. Boston, MA: Allyn & Bacon; 2003.
- 67. Fritz CO, Morris PE, Richler JJ. Effect size estimates: current use, calculations, and interpretation. J Exp Psychol Gen. 2012;141: 2–18. pmid:21823805
- 68. Cohen J. Statistical power analysis for the behavioral sciences. Second Edition ed. Hillsdale, NJ: Lawrence Earlbaum Associates; 1988.
- 69. Miller CE, Lewis KL, Huxley E, Townsend ML, Grenyer BFS. A 1-year follow-up study of capacity to love and work: What components of borderline personality disorder most impair interpersonal and vocational functioning? Personal Ment Health. 2018;12: 334–44. pmid:30136443
- 70. Köhling J., Moessner M., Ehrenthal JC., Bauer S., Cierpka M., Kämmerer A., Schauenburg H, Dinger U. (2016). Affective instability and reactivity in depressed patients with and without borderline pathology. Journal of Personality Disorders, 30(6), 776–795. pmid:26623534
- 71. Kernberg O. Aggression in personality disorders and perversions. New Haven: Yale University Press; 1992.
- 72. Kernberg O. Aggression, trauma, and hatred in the treatment of borderline patients. The Psychiatric Clinics of North America. 1994;17: 701–14. pmid:7877899
- 73. Masterson JF. The personality disorders: A new look at the developmental self and object relations approach. Pheonix: Tucker; 2000.
- 74. Blatt SJ, Levy KN. Attachment theory, psychoanalysis, personality development, and psychopathology. psychoanalytic Inquiry. 2003;23: 102–50.
- 75. Buchholz ES, Helbraun E. A psychobiological developmental model for an" alonetime" need in infancy. Bulletin Of The Menninger Clinic. 1999;63: 143–58. Available from: http://psycnet.apa.org/record/1999-13375-001.
- 76. Buchholz ES, Chinlund C. En route to a harmony of being: Viewing aloneness as a need in development and child analytic work. Psychoanalytic Psychology. 1994;11: 357–74.
- 77. Winnicott DW. The maturational processes and the facilitating environment: Studies in the theory of emotional development. New York: International University Press Inc; 1960. 1–182 p.
- 78. Gunderson JG, Lyons-Roth K. BPD's interpersonal hypersensitivity phenotype: A gene-environment-developmental model. Journal of Personality Disorders. 2008;22: 22–41. pmid:18312121
- 79. Dinsdale N, Crespi B. The Borderline Empathy Paradox: Evidence and Conceptual Models for Empathic Enhancements in Borderline Personality Disorder. Journal of Personality Disorders. 2013;27: 172–95. pmid:23514182
- 80. Hurvich M. Annihilation anxiety: an introduction. In: Siegel HB, Barbanel L, Hirsch I, Lasky J, Silverman H, Warshaw S, editors. Psychoanalytic Reflections on Current Issues. New York: New York University Press; 1991. p. 135–54.
- 81. McWilliams N, Grenyer BFS, Shedler J. Personality in PDM-2: Controversial issues. Psychoanalytic Psychology. 2018;35: 299–305.
- 82. Lingiardi V, McWilliams N. Psychodynamic Diagnostic Manual (PDM-2). 2nd ed. New York: The Guilford Press; 2017.
- 83. World Health Organization. International statistical classification of diseases and related health problems. 11 ed2018.
- 84. McCarthy KL, Carter PE, Grenyer BF. Challenges to getting evidence into practice: expert clinician perspectives on psychotherapy for personality disorders. J Ment Health. 2013;22: 482–91. pmid:24206435