Introduction

Huntington’s disease (HD) is an autosomal dominant neurodegenerative disorder, related to the mutation of the Huntingtin (HTT) gene on chromosome 4. It progresses towards disability through a triad of motor, psychiatric, and cognitive symptoms. Among those, psychiatric manifestations encompassing anxiety, despair, obsessive-compulsive behaviors, apathy and hallucinations, affect up to 98% of patients [1]. They can be the first manifestation of the disease [2], with major consequences like social withdrawal or suicide [1]. They constitute the major factor of burden in HD at moderate stages [3]. However, despite their high weight on patients and caregivers’ relationships and quality of life, psychiatric disorders are difficult to assess; physicians might rather focus on motor symptoms, easier to detect and assess [4]. In addition, psychiatric disorders not only rarely constitute the endpoint of therapeutic trials and also may constitute an exclusion criterion for being part of studies [4]. Currently, psychiatric symptoms in HD are assessed in a yearly face-to-face interview conducted by trained specialists using the Problem Behaviors Assessment Short version (PBA-s) [5]. This 11-item semi-structured interview captures depression, suicidal ideation, anxiety, irritability, lack of initiative, obsessive/compulsive behavior, and hallucinations [1,5]. Although valuable, this approach has its limitations: neurologists and psychiatrists trained in this rare disease are not so common, patients may come unaccompanied and not express their symptoms in the absence of a caregiver or even avoid consultations because of psychiatric disorders [6]. Thus, more accessible and objective tools for monitoring psychiatric status are mandatory in HD.

Speech analysis appears as a readily accessible and affordable marker due to its ease of recording. The recent rise of machine learning methods, capable of analyzing complex and high-dimensional data, has paved the way for the development of speech-based biomarkers. Speech encompasses two main components: (1) acoustic features representing articulation including prosody derived through signal processing and (2) linguistic features representing all aspects of spoken language production.

To date, there has been an overwhelming prevalence of acoustic analyses in literature in neurodegenerative disorders. This is explained by relative ease of automatic extraction, compared to linguistic features which remains resource-intensive to annotate, especially when articulation is impaired [7]. Indeed, acoustic analyses are effective for psychiatric disorders (mostly depression) and neurologic disorders (mostly Parkinson’s (PD) or Alzheimer’s diseases (AD)) [8–15], including HD. They allow distinguishing manifest HD from controls [16], from prodromal stages [17], or predicting disease severity for motor, functional, and cognitive scores [18–20].

Nevertheless, prior work suggests that linguistic features (though grammatical, lexical or syntactical patterns) might be essential in some cases, such as diagnosing psychotic disorders [10], AD [13], or mild cognitive impairment [13], or depression in AD [21] and PD [22]. In HD, Gallezot et al [23] compared acoustic and linguistic performance (using language-agnostic embeddings, Language-Agnostic SEntence Representations, LASER) for emotional expression (not psychiatric symptoms), showing that patients struggle to express emotions and that linguistic analyses may be more effective in discriminating emotions than acoustic one. To our knowledge, speech-based approaches to PBA-S–assessed symptoms in HD has not yet been investigated. This calls for a comprehensive approach using both acoustic and linguistic feature (including classical linguistic analysis and LASER). The acoustic features consisted of the Geneva Minimalistic Acoustic Parameter Set (GeMAPS) created from an international initiative for emotion and psychiatric analyses [9,23,24]. Since no equivalent consensus exists in linguistic features, an original combination described in literature combining lexical, syntactical, emotional or grammatical characteristics [9–14,21] was constructed, along LASER [23].

Here, we developed and validated a machine learning procedure to analyze speech extracted from narrative tasks to detect the psychiatric symptoms assessed by the PBA-s, in participants carrying the HTT mutation.

Materials and methods

Machine learning procedure

The whole pipeline was implemented using sklearn [41] and designed to balance robustness (given the small sample size relatively to the high dimensionality of the feature sets), simplicity and interpretability (given the originality of combining both component of speech).

A general illustration of the pipeline with application in the depression symptoms detection using the linguistic features from the cookie speech task as an example in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Example of machine learning procedure for the detection of depression symptoms with linguistic features extracted from the cookie task.

This procedure was applied for each of the four psychiatric symptom classes (depression, irritability, apathy, and obsessive/compulsive behavior), with each of the three sets of features, extracted from each of the six speech tasks.

https://doi.org/10.1371/journal.pone.0350118.g001

This study used sklearn [41] in python to analyze the features obtained from each of the six speech tasks for each participant. To calculate robust average metrics, 50-fold cross-validation (from steps 1–5) [42] was used. (1) To limit model’s true performance biases, random splits were used: the model was trained on dataset issued from 80% participants (training set) and evaluated it on the data from the 20% remaining participants (test set) [42] (type 2a from the Tripod Statement [43]) (2) Standardization of features was performed using Robust Scaler, thus limiting their intrinsic variability within participants. (3) A logistic regression classifier was trained for detecting psychiatric symptoms. To avoid overfitting and improve model generalization, a L2 penalty was used with low regularization coefficient C = 0.01. (4) The psychiatric symptoms were detected for each participant on the test set. The predicted classifications were then compared to the observed psychiatric classification defined by the PBA using several metrics.

The chance level was calculated with a “dummy model” [41] whose prediction reflected distribution of psychiatric symptoms in the training set.

Metrics and statistical analysis

Psychiatric classes.

Participants were classified as symptomatic for a given psychiatric symptom (depression, irritability, apathy, and OCB) if their score on the corresponding assessment reached the cohort median score for that symptom. Psychosis was excluded from the machine learning procedure because it was only observed in a single patient.

Main metric and secondary metrics.

The primary endpoint was the F1-score that compares directly predicted and observed psychiatric class by combining precision (True positive/ (True positive + False positive)) and sensitivity (Se, True positive/ (true positive + false negative)) per class. When dealing with unbalanced classes, the F1-score mitigates the potential bias towards the majority class [9].

Accuracy, sensitivity, specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) were also reported, as these are indicative of clinical utility. For simplicity, the mean values of these metrics over 50-folds for cross-validation were systematically referred to in the following text.

Statistical analysis.

Whether the F1-score differed from the chance level was evaluated; the closer the F1-score is to 1, the better the prediction performance. An independent t-test was used when the normality requirement was met with the Shapiro-Wilk test; otherwise, a non-parametric equivalent (Mann-Whitney U-test). P-values were corrected for multiple comparisons per feature set (linguistic, LASER or acoustic features) and per task (cookie, 24h, red-riding hood, sadness, joy, and anger), thus multiplied by 18.

Exploratory post-hoc analysis.

A line plot (S2 Fig) displayed the resulting F1-scores and their confidence interval. The validity of the sample size was assessed by running the machine learning procedure while increasing the size of the training set, resulting in line plots (S3 Fig). A hierarchically-clustered Heatmap (with linguistic features in S4 Fig and acoustic features in S5 Fig) displayed logistic regression coefficients of the features.

Results

Population description

Among the selected 99 participants, 10 were excluded for not having completed all the speech tasks, leaving 89 participants included in the machine learning procedure. PBA’ scores were as follows: Irritability mean: 0.71 (standard deviation: 1.5, range: [0.0–8.0]), OCB 1.55 (2.96, [0.0–16.0]), Apathy 0.8, (2.34, [0.0–16.0]), and Depression 5.08 (6.11, [0.0–24.0]). The figure S2 Fig illustrated the score distributions. The median score was four for depression and zero for all the other psychiatric scores: thus, the classification thresholds were set to four for depression and one for the other symptoms.

The excluded participants did not clinically differ from the 89 analyzed (Table 1) with a mean TFC of 11.0 (2.79, [4–13]), cUHDRS 10.65 (4.97, [2.20–17.40]), TMS 23.8 (20.47, [0–55]); four were classified as depressed, three irritable, three obsessed, and one apathetic.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Demographics and characteristics of the participants. The psychiatric (aggregated) scores and (isolated symptoms) subscores are calculated by multiplying severity by frequency [1,5]. Values are mean (standard deviation, range) or n (number of participants (%)). TFC: Total Functional Capacity. cUHDRS: composite Unified Huntington’s Disease Rating Scale. OCB: Obsessive/compulsive behavior. Study level: 12 = high school degree. N CAG: Number of CAG repeats. ISS: Integrated Staging System. Not applicable if CAG < 40. ^aSubscore on non-aggregated symptom.

https://doi.org/10.1371/journal.pone.0350118.t001

Psychiatric symptoms detection

Linguistic features allowed to detect better than chance the obsessive/compulsive behavior (OCB) with all tasks (cookie, anger and red-riding hood tasks p ≦ 0.001, 24h and sadness p ≦ 0.01) but not with joy task, depression with both the red-riding hood (p ≦ 0.001) and the anger p ≦ 0.01 tasks, apathy with the joy (p ≦ 0.001), sadness (p ≦ 0.01), cookie (p = 0.018) and the red-riding hood (p = 0.012) tasks. LASER features extracted enabled obsessive/compulsive behavior detection from the cookie, red-riding hood and anger tasks (p ≦ 0.001), depression with red-riding hood task (p ≦ 0.01), apathy with red-riding hood, sadness (p ≦ 0.001), 24h (p ≦ 0.01) and cookie (p = 0.020) tasks. Acoustic features enabled obsessive/compulsive behavior detection from the sadness task (p ≦ 0.001), depression with joy, sadness (p ≦ 0.001) and red-riding hood tasks (p = 0.031). Apathy was not detected using acoustic features Irritability was not detected with any of the feature sets. Results are summarized in Table 2, Fig 2 and detailed in S1 Table.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Results of the statistical analyses between the F1-score from logistic regression of linguistic or LASER features, compared to chance level. The mean value of F1-score was calculated over the 50 folds of cross validation. The table summarizes only the main results with *: 0.01 < p ≤ 0.05, **: 0.001 < p ≤ 0.01, ***: p ≤ 0.001. These statistics are Bonferroni corrected for multiple comparisons. CI95%: 95% Confidence interval. N.S.: non-significant.

https://doi.org/10.1371/journal.pone.0350118.t002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Box plot illustrating the results using both linguistic and LASER sets of features compared to chance level.

The boxes extend to the first and third quartiles of the F1-scores across the 50 folds of cross validation. The horizontal line showed the median, and the whiskers show the remainder of the distribution, except for outliers’ points. The square figured the mean within its 95% confidence interval. *: 0.01 < p ≤ 0.05, **: 0.001 < p ≤ 0.01, ***: p ≤ 0.001, these statistics are Bonferroni corrected for multiple comparison.

https://doi.org/10.1371/journal.pone.0350118.g002

Discussion

This prospective cross-sectional study examined whether speech-based machine learning analyses could detect psychiatric symptoms, as measured by the PBA-s, in 89 French HTT gene carriers from both Bio-HD and Repair HD cohorts. Samples were derived from both emotional (sad, angry, and happy stories) and non-emotional speech (the cookie theft description, the red-riding hood, and the last 24 hours). The machine learning model using linguistic, LASER and acoustic feature sets demonstrated the ability to detect several psychiatric symptoms. None of the models successfully detected irritability, likely reflecting the low irritability levels observed in this cohort (median score: 0,71). Linguistic feature analyses showed the strongest predictive performance, enabling the detection of obsessive-compulsive behavior (OCB), apathy, and depression. LASER features detected OCB and apathy across most speech tasks and detected depression with the red-riding hood task. Acoustic features detected depression in the sadness, joy and red-riding hood tasks, and OCB with sadness task, but did not detect apathy. Overall, these findings extended previous HD speech research, which has primarily focused mainly on motor, cognitive, and functional domains with acoustic analysis, and highlighted the added value of linguistic approaches for the characterization of psychiatric symptoms.

The linguistic features improved the model’s capacity to detect psychiatry in HD, as in previous model targeting emotional expression [23]. This added value of linguistic features was unexpected due to the overwhelming predominance of acoustic-based studies in both psychiatry and neurology [8–10,12,14,15,21,22,44]. The linguistic set comprised grammatical, lexical, syntactic, and affective markers, all of which have been shown to be powerful predictors of psychiatric symptoms, for example reduced lexical diversity, altered syntactic complexity, or the use of affective vocabulary [10,12,21]. Post-hoc analyses further illustrated the complex interactions between these features. Although no single linguistic pattern emerged to detect psychiatric class in this analysis, several linguistic features previously reported may reflect HD symptoms. For example, the use of first‑ and third‑person pronouns have been linked to depression, extended silence duration to apathy, and reduced lexical and syntactic complexity to different psychiatric conditions [9,10,12]. In contrast, LASER embeddings (which primarily encode sentenced-level semantics) performed relatively less consistently than the linguistic feature set. While useful for cross‑linguistic applications, LASER may not capture subtle linguistic mechanisms relevant to psychiatric states. In HD, difficulties in emotional expression or recognition [23,42] may further limit the discriminative power of purely semantic representations. Moreover, LASER features’ high dimensionality (>3,000) could have exceeded the simple logistic regression capabilities.

In contrast, compared to linguistic and LASER feature sets, GeMAPS acoustic features broadly less informative, except for depression, and did not detect apathy. This partial, and relative weaker success of acoustic features may stem from the early presence of dysarthria in HD or medication which may distort acoustic markers [46]. In HD, dysarthria induces imprecise vowel articulation, altered pitch, slower articulation rate, fewer pauses, and prolonged pause ratios, while the antipsychotics medication can cause excessive pitch and loudness variations [46]. These effects may confound the acoustic patterns usually associated with psychiatric disorders (such as reduced pitch variability, energy, fundamental frequency, or speech rate in depression [9,10,44]; this pattern seemed partly found in our post-hoc analysis) while probably leaving linguistic choices relatively intact. Thus, while acoustic features enhanced the performance (particularly for depression detection) linguistic features appeared more robust overall in HD, especially in the context of early confounding effects of dysarthria and treatment on speech production. More broadly, our findings suggest that different feature sets offer complementary strengths, which may enhance the robustness and clinical relevance of speech‑based tools for psychiatric symptom assessment.

Speech‑based studies addressing OCB are scarce in psychiatric literature review [9,10]. Two previous studies [47,48] have examined psychiatric obsessive-compulsive disorders detection: one using speech graph analysis and one using acoustic features in non‑adults populations. In the present study, OCB could be detected with linguistic features, including LASER embeddings, and to a lesser extent acoustic features. A recent review [10] also highlighted the lack of speech‑based studies focusing specifically on apathy and irritability in neurological and psychiatric disorders. Since LASER embeddings have previously been shown to classify emotions in HD [23], the findings suggest a counterintuitively non-straightforward relationship between emotional expression and PBA-s-assessed symptoms. For example, one might expect the “anger” story to be associated with the irritability symptoms. However, irritability levels were low in our cohort: it is possible that participants experienced mild irritability rather than overt anger. Supporting this interpretation, a systematic review [45] showed individuals with HD exhibit reduced recognition of negative vocal emotions (particularly anger), and two recent studies [49,50] showed that HD patient struggle to recall and narrate anger-shared stories and that they express that emotion less.

In the present work, speech analyses did not enable the detection of all psychiatric symptoms across all speech tasks. This could be explained by two interrelated factors: the relatively small sample size, and the mild severity and imbalance of psychiatric symptoms. Here, the sample size was comparable to those reported in the psychiatric speech-based studies (median size fewer than 100 participants [8,9,22,51]). Pérez-Toro et al [22] successfully classified depression in PD with 60 participants. Given the practical constraints of conducting speech-based (even more with linguistics) studies, recommended sample sizes range from 74 to 353 participants per group [9,51], depending on the studied disorders. Notably, our post-hoc analysis confirmed psychiatric symptom detection using even smaller training sets, consistent with previous studies [47,48] that reported OCB classification with fewer than 50 participants.

In addition, the low and unbalanced psychiatric symptoms may have further limited detection performance. Classification thresholds were set at 4 for depression and 1 for other symptoms. Irritability, which could not be detected by our model, scored the lowest scores (median 0 [interquartile range: 0–1]). These values were below those reported in the international HD cohort used to validate PBA-s (median 1 (IQR 0–5) [1] and were consistent with other HD cohorts [1,5]. Similar low values patterns have been reported in PD or AD studies [21,22,52], in which depression scores were also slight or mild at most: unlike dedicated psychiatric studies, neurological cohorts often avoid severe psychiatric symptoms. Future studies including participants with more severe psychiatric symptoms may improve generalizability.

Literature reviews report a wide range of speech tasks used across disorders and feature types [8,9,12]. Spontaneous speech tasks, which aims at capturing ecological language use (word choice, and pauses could be important), are particularly suitable for linguistic analysis than constrained tasks such as reading (more control on sound production or evoked emotions) or sustained vowels (more suitable for some acoustic features) [9]. Spontaneous speech has also been used in acoustic analyses in AD, depression or bipolar disorders [8] and this choice of task aligns with its predominance in literature [8,13,21,22,47,48]. Emotional speech tasks were included with the expectation that they might amplify symptom‑related differences [31–33]. However, aside from detecting depression through acoustic features in the sadness and joy tasks, there was no clear evidence of such an effect in either the linguistic or LASER feature sets, nor did these tasks support the detection of other psychiatric symptoms (particularly irritability). These findings suggest that neutral tasks may suffice for extracting linguistic markers, though further work is needed to clarify the role of emotional tasks and HD‑specific peculiarities.

The finding that linguistic features play an important role in psychiatric symptom detection in HD raises two main considerations related to their inherently language-dependent nature. First, our dataset was collected at Henri Mondor hospital in French and the pipeline, although based on a simple logistic regression classifier, will require language-specific training to ensure its generalizability to other languages. However, the nature of dysarthria and the role of medication, which limit the direct use of Automatic Speech Recognition (ASR), are likely to affect speech similarly across languages, making it plausible that their impact on acoustic features would not differ substantially between languages. Although LASER was trained on 200 languages, its modest performance in this study suggests that future work should explore alternative language‑agnostic embeddings to facilitate cross‑linguistic transfer. Second, the present study relied on time-consuming manual annotations. The use of ASR could reduce this burden in future work; however, dedicated evaluations remain necessary, as ASR accuracy declines in individuals with neurological disorders and varies across algorithms [7]. Future studies should therefore compare gold‑standard human annotations with ASR‑derived features to assess the predictive value of linguistic features in HD and potentially other neurological diseases, across multiple languages.

Conclusions

This study provides the first classification of PBA-s assessed psychiatric symptoms in carriers of the mutant HTT gene using speech-based analysis. The findings demonstrate the potential of speech to detect psychiatric symptoms, including obsessive-compulsive behavior, in individuals with premanifest and early-stage Huntington’s Disease. Linguistic features appear particularly relevant for psychiatric symptom detection in HD, potentially due to the influence of dysarthria on acoustic parameters. Although complete automation of the linguistic features extraction remains challenging, this study highlighted the importance of combining linguistic and acoustic speech features in the development of future psychiatric evaluation tools in HD.

Supporting information

Acknowledgments

We are very thankful to the patients that participated in our study, neuropsychologists who collected data and speech pathologists who transcribed and annotated the recordings.