Study goal and objectives

During an acute stage of an early wave of the COVID-19 pandemic, we set out to conduct PanDIRECT, a pilot RCT, to compare two brief self-care intervention strategies for management of symptoms of depression and anxiety. The first involved self-directed use of CBT self-care tools supplied by the study, while the second employed the same tools, but supported by a telephone-based, trained lay coach. Objectives were to assess 1) feasibility of delivering the interventions during a pandemic based on: a) successful recruitment of eligible participants into the study, b) participant completion of the interventions, and c) fidelity of delivery of the coaching; 2) comparative acceptability of the interventions in the two groups based on degree of use of the self-care tools and satisfaction with the assigned intervention; 3) preliminary comparative effectiveness of coached vs non-coached interventions on severity of depression and anxiety symptoms at eight weeks; and 4) participating patients’ family physicians’ views on the value and acceptability of study-generated information sent to them about their patients.

Study design and timeline

A single blind, individually randomized, pragmatic pilot RCT of the two self-care strategies was conducted during the SARS-COVID-19 pandemic (local onset March 2020) between October 2020 and April 2021. The strategies tested were 1) selected components of previously validated self-care tools (internet and paper-based), with guidance from up to three coach telephone calls over eight weeks; and 2) use of these self-care tools alone. The trial design adhered to the CONSORT criteria [29], the CONSORT extension for pragmatic trials [30], and guidelines for behavioral trials [31]. The protocol was registered on clinicaltrials.gov (Identifier: NCT04609371) on 30/10/2020, and was approved by the hospital research ethics committee (protocol # SMHC 20–10).

Sample and eligibility criteria

Our study population was obtained from a cohort of 235 patients 65+ years old previously enrolled in a study our team conducted on individuals admitted for physical health problems to either of two acute care hospitals prior to the COVID-19 pandemic [32]. On discharge they were invited to consent to be contacted again for possible participation in other research projects.

Eligibility for the PanDIRECT study included being age 65 or older, English or French speaking, and autonomous home dwelling. People living in long-term care or other medicalized facilities were excluded, but those receiving medical or social services in their own homes were not. Other exclusion criteria were: moderate to severe cognitive impairment (measured by a validated telephone screening interview, see below), inability to read in English or French (self-reported), hearing impairment (judged by research staff over the telephone), self-reported ongoing counseling or psychological therapy begun prior to recruitment (since such treatments might conflict with the self-care interventions [28]). Participants who started therapy after enrolment were not removed from the study. Those expressing suicidal intent (assessed during the screening interview) received further assessment / possible emergency referral and exclusion from the study. Contrary to our previous studies of mental health self-care interventions, we did not limit enrolment to individuals with at least mild depression or anxiety symptoms. Asymptomatic individuals who were interested in the intervention and were otherwise eligible were included in the study sample since tool use might improve overall coping skills during the pandemic, and possibly limit development or worsening of symptoms over a longer time frame [33].

Contact and consent

Research assistants (RAs) attempted telephone contact with the 235 individuals to introduce the study, establish interest, conduct a brief screening interview to assess eligibility and obtain verbal consent following description of study goals and involvement. Consent was documented by the research assistant and calls were audio-recorded to preserve the consent process. A copy of the consent form was postal mailed or sent electronically to respondents.

Randomization

On study entry, participants were randomized to the self-directed arm (tools only) or the guided arm (tools with coaching) by a computer-generated randomization schedule using random block sizes, with an allocation ratio of 1:1 (using SAS version 9.4). The sequence was concealed and a participant’s assignment appeared only once the coordinator entered the unique participant ID, date of enrolment and information on symptom severity. Participants were stratified based on whether their depression and/or anxiety symptoms were none to mild (as measured by the 9-item Patient Health Questionnaire (PHQ-9) and/or 7-item Generalized Anxiety Disorder questionnaire (GAD-7), scores <10) or moderate to severe (PHQ-9 and/or GAD-7 scores ≥10). A discussion of the psychometrics of these tools follows.

Measures used, data collected

In the eligibility screening interviews RAs administered the Blessed Orientation-Memory-Concentration (BOMC) test, a reliable (retest reliability coefficient of 0.88) and valid (internal consistency: Cronbach’s alpha coefficient of 0.89; construct validity as compared to the Mini-Mental State Examination: correlation coefficient of 0.84) six item cognitive screen to assess possible cognitive impairment (with scores of eligible, consenting patients retained as study data for analyses) [34]. There were also questions on living arrangements, therapy, and ability to read English or French. The suicidal ideation item from the PHQ-9 was used to identify possible suicidal intent.

Among eligible and consenting participants, PHQ-9 results were used as continuous and categorical measures of severity of depressive symptoms [35]. This tool was selected because of its wide use in medical populations, and sensitivity to change [36]. The PHQ-9 has demonstrated good internal consistency across various populations (Cronbach’s alpha ranging from 0.79 to 0.89); high test-retest correlations, with coefficients ranging from 0.84 to 0.96 over periods ranging from 1 to 8 weeks; and good construct validity, showing strong correlations with other depression measures and diagnostic criteria for depression [35, 36]. Scores range from 0–27 with established severity ranges (mild: 5–9; moderate: 10–14; moderately severe: 15–19; severe: 20+), and criterion for diagnosis of clinically significant depression being 10 [37].

The 7-item GAD-7 questionnaire was similarly used for continuous and categorical measures of severity of anxiety symptoms [38], with scores ranging from 0–21, with established severity ranges (mild: 5–9; moderate: 10–14; severe: 15+), and criterion for diagnosis of clinically-significant anxiety being 10. The GAD-7 has demonstrated good sensitivity and specificity, a Cronbach’s alpha ranging from 0.83 to 0.93 and high positive correlation with other measures of anxiety [39].

A 4-item CAGE alcohol abuse screening questionnaire (score ranging from 0–4) was used for descriptive purposes of possible co-morbid alcohol abuse or dependency [40]. Although a score of 2 or more is considered clinically significant, a score of 1 or more results in greater sensitivity and has been used in some studies [41]. The screening tool has been compared with the Diagnostic Interview Schedule (DIS) and found to have a sensitivity of 92% and a specificity of 82% for identifying alcohol dependence [42].

The baseline interview also included questions on hospitalizations, emergency department visits, counselling in the previous six months, COVID diagnoses in participants or family members, presence of a family caregiver, and reception of homecare services. To reduce interview burden, data collected in the previous study (age, sex, country of origin, and education level) were extracted.

After the eight-week intervention, an RA telephoned participants for a structured follow-up interview. All baseline measures (except the CAGE questionnaire) were re-administered, along with questions about experience in the study: use of the self-care materials, perceived helpfulness in managing mental health problems (on a four point scale, from “not at all helpful” to “very helpful”), use of any health services including counseling or therapy started during the study period, and satisfaction with the intervention (with a three item version of the Client Satisfaction Questionnaire (CSQ-3) [43, 44]). The CSQ has demonstrated internal consistency, with reported alpha coefficients ranging from 0.80 to 0.93 [45, 46]. It correlates with other measures of client satisfaction, treatment outcome, and therapist ratings [45]. Stage of tool use was assessed through three categories: ‘didn’t use’, ‘just started’, and ‘well underway’, inspired by the stage of change measure validated by Sarkin et al. (2001) [47]. Participants reporting at least some material use were queried to assess level of use (level 1 was limited to reading contents of the materials; level 2 to having some engagement with materials to identify relevant self-care approaches; and level 3 indicating that some approaches were at least somewhat applied) [48]. We cross-validated these self-reported tool use reports with information recorded in the coach logs (described below).

Participants were invited to provide their family physicians’ contact information so that the latter could be mailed a report of their patients’ study involvement once the patients had completed the study. The reports included brief information on the study, participants’ PHQ-9 and GAD-7 scores at baseline and follow-up, instructions on how to interpret the scores, and a brief postal survey (for return by pre-paid stamped/addressed envelope, fax, or email). This survey enquired about (1) usefulness of the report; (2) their general familiarity with PHQ-9 and GAD-7 screening tools; and (3) their interactions with participants during the study period. Reminders were sent to non-responders four weeks after the first mailing.

Interventions

A mailing to participants’ homes using a premium guaranteed delivery service contained a study welcome letter, self-use materials, description of what to expect next (coach calls, if relevant), and expected timeline for a follow-up questionnaire. The self-care CBT-based toolkit was adapted from tools used in our previous studies [25, 26, 49] to accommodate the shorter intervention period. The main changes were: no initial screen; making toolkit available to those without clinically significant depression or anxiety symptoms; limiting the number of tools sent to two (versus the entire toolkit of over eight tools); reduction in the length of certain tools (i.e. providing one chapter of a workbook as opposed to the whole workbook); and an abbreviated guidance schedule of up to three calls over eight weeks (compared to up to 15 calls over six months).

In this study, specific self-care tools were assigned to each participant according to a study-developed algorithm for particular positive findings identified on the PHQ-9 and GAD-7 at baseline. Each participant was therefore assigned one CBT primary tool, either the Reactivating Your Life chapter from the Antidepressant Skills Workbook [50, 51] or the Managing Worry chapter from the Positive Coping With Health Conditions Workbook [52]; and one secondary tool, either a mood monitoring tool, a relaxation audio tool, information on exercise and healthy eating, information on sleep [50] or summaries on emotional eating (Toolkit described in S1 Appendix).

Guidance on the use of the materials in the intervention arm was delivered by individuals who had recently obtained Bachelor’s degrees in psychology and whose training and activities were supervised by a coach supervisor. The latter, a registered psychotherapist and clinical psychology PhD candidate, was familiar with the interventions used in the team’s previous RCTs, had worked previously with the co-investigators, and had led the creation of the algorithm in the current study that personalized the selection of tools for each participant. Each of the intervention arm participants was followed by one of three study coaches who were bilingual (English, French), had strong organizational and interpersonal skills, but no formal training in CBT. They received a half day training via videoconference, followed by mock practice sessions over the telephone. Initial calls with participants took place one week following toolkit deliveries, with a mandate of a maximum three calls over eight weeks and a recommended 15 to 20 minutes per call. Coach activities were guided by a Coach Manual adapted from that used in our previous two RCTs [25, 26]. They had no role in psychotherapy or counselling, and their function was limited to assisting participants by encouraging use of assigned tools and helping to formulate goals to meet objectives related to the strategies described in the tools.

The coaches maintained logs of all telephone contacts attempted or completed, including their duration. They noted which tools were recommended and used, as well as participants’ levels of tool use using a 3-point scale: 1) coach introduced tool, 2) participant read tool, and 3) participant implemented strategies in the tool or tried the exercises at each of the calls. The coach supervisor reviewed the logs, along with random audio taped telephone contacts, employing a fidelity checklist (adapted from our previous research [25, 26]), for assessing coaching done in accordance with the protocol, and for verifying coaching data in the logs addressing the feasibility objectives.

Data analysis

Data analysis was conducted according to the CONSORT guidelines [29, 30]. All the quantitative analyses were carried out with SAS version 9.4 and STATA 15.0 software.

Baseline imbalance was assessed by computing the standardized difference [53] between study arms (coached and self-directed) [54] with respect to six pre-specified baseline variables (age, sex, hospital recruited from in the original study from which the cohort was created, COVID-19 diagnosis, PHQ-9, GAD-7), and nine other variables selected by the research team (see Table 1). Any variable with a standardized difference ≥ 0.15 was considered evidence of imbalance [55].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Assessing baseline imbalance variables across study group (n = 90).

https://doi.org/10.1371/journal.pone.0297937.t001

The feasibility of the intervention was assessed using descriptive statistics to describe refusal and uptake rates, completion of surveys, and the rates of missing data. Analyses including only participants who completed the eight-week follow-up were performed. The standardized difference and 95% confidence interval was computed to compare the study arms with respect to stage and level of self-care tool use and the satisfaction [53, 56]. Unadjusted effect sizes (Cohen d) defined as the study group mean difference in the outcome score, divided by the pooled standard deviation [57] and 95% confidence intervals [56, 58] were computed for both primary outcomes. In addition, a linear regression model was fitted to adjust for baseline imbalance [59], while adjusted effect sizes and 95% confidence intervals (95% CI) were computed as the Beta estimate of the intervention group divided by the pooled standard deviation obtained from the unadjusted analysis. For the regression analyses, standard model checking techniques were applied (e.g.: residual plots) [59]. Finally, the standardized difference was also used to compare characteristics of participants who completed the eight week follow-up survey with participants who did not (see Table A in S2 Appendix). The analysis by intention-to-treat was performed to account for potential selection bias due to differential losses to follow up; missing data were imputed using Multiple Imputation by Chained Equations [60, 61], see S2 Appendix for more details.

We cross-validated the self-reported stage of use (for primary and secondary tools) and level of use (for primary tools only, as participants only self-reported level of use for the workbooks) variables against tool use information from coach logs, using the Cramer-V statistic [62]; this statistic ranges from 0 to 1 and can be interpreted as a) weak: 0.0 to 0.10, b) moderate: >0.10 to 0.15, c) strong: >0.15 to 0.25, and d) very strong >0.25 [63].

Sample size

We followed guidelines for minimum sample sizes for pilot studies [64]. With a final sample size of over 40 participants in each group, we were well above the recommendations.

Feasibility: Recruitment and study sample

Of 235 older adults in our previous study consenting to follow-up [32], 75 were not reachable by telephone due to numbers no longer in service, patients having died, calls not answered or messages not returned. Fig 1 summarizes the outcome of successful contact of 160 individuals. Forty-four were not interested, with over half indicating they did not have time or not providing a reason. Twenty-three did not meet eligibility criteria (mostly due to either visual, hearing or cognitive impairment). Of the remaining 93 who engaged in the current study’s consent process, 90 consented to participation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flowchart.

https://doi.org/10.1371/journal.pone.0297937.g001

Characteristics of these 90 older adults (see Table 1) include a mean age (standard deviation) of 79.4 (6.8), slightly more females, 2/3 with at least post-secondary education, high cognitive functioning (as determined by the BOMC at screening), 1 having had a positive COVID-19 diagnosis, 1/3 having been to an ER during the COVID pandemic, and 2/3 having a family caregiver. Since the pandemic’s onset, 31.4% perceived experiencing worse physical health, and 47.2% worse mental health (not indicated in table), while at the time of the study PHQ-9 and GAD-7 scores identified no or only mild symptoms of depression or anxiety.

The 90 were randomized to the coached group (n = 44), or the self-directed group (n = 46). Table 1 identifies nine variables that were imbalanced between the two groups. Compared to the self-directed group, the following were more frequent in the coached group: scores in the less depressed or anxious categories (although mean scores were similar); less than 85 years of age; female; completed less than a university education; no caregiver; a CAGE score of 0; previous counseling; and hospitalization or emergency room visit during the previous six months. One participant (in the control group) started therapy after enrolment. 68.9% (62 /90) completed the follow-up questionnaire, comprising 63.6% (28/44) of the coached cohort and 73.9% (34/46) of the self-directed cohort.

Feasibility: Intervention delivery

All randomized participants received their tools, except one in the self-directed group (multiple attempts at delivery were unsuccessful). In the coached group, 37/44 received at least one coach call (84%), 52% completed all three calls (Table 2). Among the seven who did not receive at least one call, five could not be reached despite multiple attempts and two declined the coach calls indicating they were not interested. The average call length was 15.0 minutes, generally occurring at two week intervals. Participants who did not complete follow-up (n = 16) received fewer calls than completers of follow-up (mean 1.2 vs 2.5 calls, respectively).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Feasibility of intervention delivery.

https://doi.org/10.1371/journal.pone.0297937.t002

At follow-up, over 80% of participants who had received at least one coach call reported that the number of calls felt appropriate to them and had helped them understand and use the tools. Only 32% indicated that they would have preferred to use the tools without coaching support.

Twenty-eight coach calls were randomly selected for fidelity review. On average, 96% of the coaching components were delivered as per the intervention protocol (see Table 2).

Acceptability of intervention

Table 3 describes participants’ tool use (the participant in the self-directed groups who did not receive the tools is excluded). We noted a significant effect of coaching on stage of use of both the workbook and secondary tool assigned, as well as on perceived helpfulness of the secondary tool. Coaching was not associated with level of use among users or with satisfaction about the intervention, which was generally high in both groups.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Comparative acceptability of the two interventions (n = 61).

https://doi.org/10.1371/journal.pone.0297937.t003

An exploration of the association between coach-recorded tool use and participants’ self-reported tool use was examined. The Cramer-V statistic computed by comparing self-reported vs coach reported stage of use was 0.47 (n = 27), while for level of use it was 0.39 (n = 27). Both of these are considered to be very strong associations [63].

Potential effectiveness of the coaching intervention

Table 4 summarizes the effects of the eight week coaching intervention on PHQ-9 and GAD-7. Effect sizes were small (less than 0.2) in unadjusted and adjusted analyses. Descriptive results using depression and anxiety categories do not suggest important differences between the study groups or between baseline and follow-up. Sensitivity analyses taking into account missing data found similar results (S2 Appendix).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparative outcomes of the two interventions (n = 62).

https://doi.org/10.1371/journal.pone.0297937.t004

Feedback from family physicians

69% (43/62) of study completers agreed to have reports sent to their family physicians at the end of the study. 46.5% (20/43) returned completed surveys included with the report. The respondents appeared to be involved in mental health care given that 80% and 75% respectively used the PHQ-9 and the GAD-7. 55% (11/20) found the patient information sent to them useful, while the remainder held a neutral opinion. Nonetheless 70% (14/20) of physicians kept the information for filing in the patients’ medical records, 20% (4/20) used it to try to initiate or confirm patient follow-up, and 90% (18/20) indicated interest in future reception of similar patient information, if it was available.