Internet-based cognitive behavioural therapy (iCBT) for generalized anxiety disorder (GAD) has been shown to be effective when guided by a clinician. The present study sought to replicate this finding, and determine whether support from a technician is as effective as guidance from a clinician.
Randomized controlled non-inferiority trial comparing three groups: Clinician-assisted vs. technician-assisted vs. delayed treatment. Community-based volunteers applied to the VirtualClinic (www.virtualclinic.org.au) research program and 150 participants with GAD were randomized. Participants in the clinician- and technician-assisted groups received access to an iCBT program for GAD comprising six online lessons, weekly homework assignments, and weekly supportive contact over a treatment period of 10 weeks. Participants in the clinician-assisted group also received access to a moderated online discussion forum. The main outcome measures were the Penn State Worry Questionnaire (PSWQ) and the Generalized Anxiety Disorder-7 Item (GAD-7). Completion rates were high, and both treatment groups reduced scores on the PSWQ (p<0.001) and GAD-7 (p<0.001) compared to the delayed treatment group, but did not differ from each other. Within group effect sizes on the PSWQ were 1.16 and 1.07 for the clinician- and technician-assisted groups, respectively, and on the GAD-7 were 1.55 and 1.73, respectively. At 3 month follow-up participants in both treatment groups had sustained the gains made at post-treatment. Participants in the clinician-assisted group had made further gains on the PSWQ. Approximately 81 minutes of clinician time and 75 minutes of technician time were required per participant during the 10 week treatment program.
Both clinician- and technician-assisted treatment resulted in large effect sizes and clinically significant improvements comparable to those associated with face-to-face treatment, while a delayed treatment/control group did not improve. These results provide support for large scale trials to determine the clinical effectiveness and acceptability of technician-assisted iCBT programs for GAD. This form of treatment has potential to increase the capacity of existing mental health services.
Australian New Zealand Clinical Trials Registry ACTRN12609000563268
Citation: Robinson E, Titov N, Andrews G, McIntyre K, Schwencke G, Solley K (2010) Internet Treatment for Generalized Anxiety Disorder: A Randomized Controlled Trial Comparing Clinician vs. Technician Assistance. PLoS ONE 5(6): e10942. https://doi.org/10.1371/journal.pone.0010942
Editor: Antonio Verdejo García, University of Granada, Spain
Received: January 6, 2010; Accepted: May 1, 2010; Published: June 3, 2010
Copyright: © 2010 Robinson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was investigator initiated. It was funded by departmental funds and by a grant from the Australian Rotary Health Research Fund (http://www.australianrotaryhealth.org.au/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Generalized anxiety disorder (GAD) is a common anxiety disorder characterized by chronic, excessive, and uncontrollable worry. The 12-month prevalence of GAD in Australia and the US is 2.7% and 3.1%, respectively , . GAD commonly co-occurs with other anxiety disorders and/or depression and is highly disabling, resulting in levels of disability comparable to depression , . Although people with GAD frequently utilise health care facilities they often report somatic rather than psychological symptoms, making diagnosis difficult .
GAD can be treated effectively with cognitive behavioural therapies (CBT) –, but numerous barriers to treatment exist, including the direct and indirect costs of treatment, the limited availability of mental health professionals, stigma, and the difficulty of patients attending treatment during office hours , . One promising strategy for reducing these barriers involves Internet-based CBT (iCBT) programs. iCBT programs result in clinically significant improvements in patients with depression –, panic disorder –, social phobia –, with encouraging results recently reported from the first iCBT program for treating GAD . In that study, treatment group participants obtained clinically significant reductions in severity of GAD symptoms relative to a control group.
The successful use of a non-clinician (technician) to administer iCBT, without compromising clinical outcomes, has been reported in iCBT programs for depression and social phobia , , . The use of technicians to oversee administration of iCBT programs has considerable implications for the cost-effectiveness of iCBT. An important question is whether similar effects could be obtained from iCBT for GAD.
The present CONSORT-Revised compliant randomized controlled trial (RCT)  had two aims: To replicate the recent finding that people with a DSM-IV  diagnosis of GAD could be treated using the Worry program , a diagnosis-specific iCBT program developed to treat GAD, and; to examine the relative clinical efficacy and acceptability of clinician- and technician-assisted iCBT using the Worry program . We tested four hypothesises: Firstly, that clinician-assisted (CA) treatment would be efficacious; secondly, that participants in a technician-assisted (TA) group would show similar clinical improvements on measures of GAD, depression and disability as the clinician-assisted (CA) group; thirdly, that both treatment groups would have better outcomes than a delayed treatment (control) group; and finally; improvements would be sustained at follow-up.
This study was approved by the St Vincent's Hospital Human Research Ethics Committee and by the University of New South Wales Human Research Ethics Committee. Written informed consent was obtained from all participants.
Participants were recruited from July to September 2009 via a website (www.virtualclinic.org.au) providing information about common mental disorders including GAD, and a link to apply online to join a research treatment program. Participants first applied online, completing several screening questionnaires about the presence and severity of symptoms of anxiety and depression, including the Patient Health Questionnaire - 9 Item (PHQ-9) , the Generalized Anxiety Disorder 7-Item Scale (GAD-7)  and the Social Phobia Screening Questionnaire (SPSQ) . Questions were also asked to determine demographic details of participants (see Table 1).
Exclusions were (i) not a resident of Australia; (ii) less than 18 years of age; (iii) no regular access to a computer, the Internet, and use of a printer; (iv) currently participating in CBT; (v) using illicit drugs or consuming more than three standard drinks/day; (vi) experience of a psychotic mental illness (schizophrenia or bipolar disorder) or current severe symptoms of depression (defined as a total score >23 or responding >2 to Question 9 (suicidal ideation) on the PHQ-9; and (vii) if taking medication, had been taking the same dose for less than 1 month or intending to change that dose during the course of the program. Excluded applicants immediately received an on-screen message and email thanking them for their application, and encouraging them to discuss their symptoms with their physician.
Participants who passed the screening phase were telephoned for a diagnostic interview using the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI)  to determine whether they met DSM-IV criteria for GAD. Participants who satisfied all criteria were informed of the study design and invited to return a completed consent form by email. The study was approved by the Human Research Ethics Committees of St Vincent's Hospital, Sydney, and the University of New South Wales.
Treatment groups received access to the Worry program, an iCBT program with demonstrated efficacy at reducing symptoms of GAD . The Worry program consists of six online lessons, printable summary and homework assignments, automatic emails, and additional resource documents. The six online lessons represent best practice principles used in CBT programs for GAD including cognitive therapy, challenging meta-beliefs about worry, graded exposure, challenging core beliefs, and relapse prevention. Part of the content of each lesson is presented in the form of an illustrated story about a woman with GAD who, with the help of a clinical psychologist, learns to gain mastery over her symptoms. Automatic emails are sent to congratulate participants for completing each lesson, to remind them to complete materials, and to notify them of new resources. As people progress through each lesson they have access to additional written documents providing supplementary information about techniques such as managing sleep problems, assertiveness and problem solving skills, managing low mood, panic, and other common comorbid symptoms. They are also provided with access to vignettes written by previous participants about their own experiences in the Worry program of managing GAD. Participants are expected to complete the homework tasks prior to completing the next lesson, and to complete all lessons within 10 weeks.
All participants in the treatment groups began the 10-week treatment program at the same time. Participants were advised to complete one lesson every 7–10 days and to complete the six lessons within 10 weeks of starting. All participants received automatic emails informing them when a lesson was to be completed, and reminder emails if they had not completed a lesson within 7 days of notification.
Three staff conducted the study, with supervision from NT. The technician (KM) was employed in an administrative role as a Clinic Manager, Anxiety Disorders Clinic, Mental Health Service, St Vincent's Hospital, Sydney. She reported no prior experience with research programs, no qualifications in health care or counseling, and had no clinical duties in her usual role. The clinician (ER) was a qualified and registered clinical psychologist, employed at the same unit as KM. The third staff member (KS) was a research assistant, who provided administrative support to the technician and clinician.
During treatment the technician provided TA group participants with weekly email or telephone contact. The technician was given a guideline script which identified the topics covered in each lesson of the program and activities participants should be encouraged to practice for each lesson. The technician was instructed to contact each TA group participant weekly to provide encouragement and support, and where possible to respond to participants' general questions by referring them to the materials in the Worry program. The technician was not permitted to provide clinical advice. The technician received supervision from the clinician and was instructed to inform the clinician of any perceived deterioration in the participants' mental health status, or of any concerns about participants' wellbeing. While conducting this research the technician maintained her full-time role as a Clinic Manager.
CA group participants had weekly email or telephone contact with the clinician and access to an online discussion forum where they could post questions to the clinician about the program content. Information posted on the discussion forum could be read by other participants in the CA group. The clinician was provided with the same guideline script as the technician but was also instructed to answer participants' questions via forum, email, or telephone. The clinician was instructed to actively engage with each participant in treatment including goal setting, problem solving, and discussion of strategies for overcoming hurdles to progress. Because of the clinical nature of messages on the forum, the TA group did not have access to a forum. The clinician and technician were instructed to try to spend no more than 10 minutes in contact with each participant per week. The total time required and nature of all contacts with participants during treatment was recorded.
This study was a 3 group randomized controlled non-inferiority trial to determine whether technician-assisted iCBT was equivalent to clinician-assisted iCBT but superior to delayed treatment (control).
Primary and Secondary Outcomes
One week prior to beginning the trial participants completed the following questionnaires online: The Penn State Worry Questionnaire (PSWQ) ; the GAD-7; the Patient Health Questionnaire 9-Item (PHQ-9); the Kessler 10 (K-10) ; the Sheehan Disability Scales (SDS) ; and the Credibility/Expectancy Questionnaire (CEQ) , . The PSWQ is a 16-item measure with scores ranging from 16–80. The GAD-7 is a 7-item measure with scores ranging from 0–27. The PSWQ and GAD-7 are frequently used clinical and research measures of GAD. A score of 10 on the GAD-7 has been identified as providing an important threshold for identifying DSM-IV congruent GAD . The PHQ-9 is a 9-item measure of depressive symptoms with scores ranging from 0–27. The K-10 is a 10-item measure of psychological distress with scores ranging from 10–50. The SDS is a 3-item measure of disability with scores ranging from 0 to 30 and the CEQ is a widely used measure of the expectancies or perception of treatment credibility.
The PSWQ, GAD-7, PHQ-9, K-10, SDS and a treatment satisfaction questionnaire (based on the CEQ) were re-administered one week post-treatment and at three-months post-treatment (follow-up), while the GAD-7 and PHQ-9 was also administered mid-treatment (at week 5). All of these measures are considered reliable, valid, and appropriate for clinical and research purposes, with recent research indicating that online administration of questionnaires results in acceptable reliability of responses , . Changes in the PSWQ and GAD-7 were considered the primary outcome measures, while changes in the PHQ-9, K-10, SDS, and treatment satisfaction questionnaire were the secondary outcome measures. Results are reported at the end of treatment. Follow-up results were not available for the control group, who had started treatment by that time.
Sample Size and Randomization
Power calculations were based on a non-inferiority trial design comparing parallel-groups. Alpha was set at 0.025, power at 90%, and the mean minimal reliable change index on the GAD-7 (based on earlier findings) and standard deviations for each group were expected to be equivalent (5 and 4, respectively). Using Table V from Julious , the minimum sample size for each group was identified as 39, but more were recruited to hedge against attrition.
The 150 people accepted into the program were randomised by NT via a true randomisation process (www.random.org) to either the CA (n = 51), TA (n = 50), or control groups (n = 49) (see Figure 1). Allocation preceded the diagnostic telephone call. Dependence on self-report measures precluded blinding.
Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analyzed with one-way analysis of variance (ANOVAs) and chi-square tests, followed by t-tests with Bonferonni corrected p values. Changes in participants' questionnaire scores from pre to post-treatment and from pre-treatment to follow-up were analyzed using repeated measures analyses of covariance (ANCOVAs). This approach is recommended as a robust and reliable statistical strategy for analyzing the results of RCTs , . Changes in questionnaire scores between post-treatment and follow-up were analyzed using paired samples t-tests. Effect sizes (Cohen's d) were calculated both within- and between-groups, based on the pooled standard deviation.
All post-treatment and follow-up analyses adopt an intention-to-treat (ITT) design where missing data is replaced by the last observation carried forward (LOCF).
Two scores (credibility and expectancy) were derived from the CEQ as described in .
Three measures of clinical significance were employed. Pre-treatment and post-treatment GAD-7 scores were compared with optimum cut-offs for a probable diagnosis of GAD , to provide an index of remission. This was defined as the proportion of participants who initially scored above the optimum cut-off (GAD-7 total score of 10 or more) and subsequently scored below this cut-off. Secondly, an estimate of recovery was made by identifying the proportion of participants in each group who demonstrated a significant reduction in their symptoms (defined here, as a reduction of 50% of pre-treatment GAD-7 scores), as described in recent dissemination studies . Thirdly, the percentage of participants in each group who met criteria for reliable change on the PSWQ was calculated. This was defined as the proportion of participants who met the criteria of statistically reliable change as described in Jacobson and Truax . A reliable change index for the PSWQ was calculated separately for each of the three groups using their pre-treatment standard deviation, and a test-retest reliability coefficient of 0.93, as reported in .
Two hundred and six individuals expressed interest in the study (Figure 1), 150 met the eligibility criteria and were randomized to one of the three groups. Fifty TA and 47 CA group participants completed the pre-treatment questionnaires and began Lesson 1 and are eligible for analysis along with 48 control group participants who completed the pre-treatment questionnaires. Five of the original 150 participants did not complete the pre-treatment questionnaires, and so are ineligible for analysis.
Characteristics of the groups are presented in Table 1. There were no significant between-group differences in gender, education, employment, previous discussions of symptoms with a health professional, use of medication, weekly use of the internet, or confidence in using computers. Treatment groups were also equally chronic, with 71% of treatment group participants reporting onset of GAD before age 30 years. At pre-treatment, both treatment groups also rated the likely benefits of the Worry program as similar, although participants in the CA group rated the expectation of benefit as marginally greater than the TA group (F1, 91 = 3.77, p<0.06).
There was a significant difference between groups in marital status (χ2 = 19.48, df = 4 p<0.001), but post-hoc tests removing the control group revealed no differences between the treatment groups. A one-way ANOVA revealed significant between-groups differences in age (F2, 142 = 4.41, p<0.01), with Bonferronni corrected post-hoc t-tests revealing that participants in the control group (M = 51.23, SD = 11.61) were significantly older than TA group (M = 44.16, SD = 12.44) and CA group (M = 45.57, SD = 13.14) participants, with no differences between treatment groups. ANOVAs were also conducted to explore pre-treatment differences in symptom severity. No between group differences were found on the PSWQ, GAD-7, PHQ-9, K-10, or SDS (p>0.05).
Forty (40/50, 80%) TA and 35/47 CA (74%) group participants completed all 6 lessons within the required time. Reasons for not completing all lessons were not collected. Post-treatment data was collected from 45 (90%) TA, 46 (98%) CA group members, and from 47/48 (98%) of control group participants. Follow-up data (3 months post-treatment) were collected from 38/50 (76%) TA and 33/47 (70%) CA group participants. In accordance with the ITT and LOCF paradigm, the pre-treatment scores of the participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.
Post-Treatment (11-week) Outcomes
Univariate ANCOVAs on post-treatment PSWQ and GAD-7 scores, controlling for pre-treatment scores (see Table 2), revealed significant effects for PSWQ (F2, 141 = 23.02, p<0.001) and GAD-7 (F2, 141 = 27.04, p<0.001) scores. Post-hoc pairwise comparison of groups revealed no difference on either measure between treatment groups, but significant differences between the treatment groups and the control group (p<0.001). The effect of the differences in age between groups were explored by repeating these calculations and adding age as a covariate, but age was not significantly related after controlling for pre-treatment scores.
Univariate ANCOVAs conducted on the PHQ-9, K-10, and SDS post-treatment scores, while controlling for pre-treatment scores revealed significant effects over time for the PHQ-9 (F2, 141 = 16.63, p<0.001), K-10 (F2, 141 = 17.52, p<0.001), and SDS (F2, 141 = 17.57, p<0.001) scores. Post-hoc pairwise comparison of groups revealed no difference on either measure between treatment groups, but significant differences between the treatment groups and the control group (p<0.001).
Within-group effect sizes on the PSWQ were 1.16 and 1.07 for the clinician- and technician-assisted groups, respectively, and on the GAD-7 were 1.55 and 1.73, respectively. Large (>0.80) within-group effect sizes (ESs) (Table 2) were found for both treatment groups on the PHQ-9 and K-10, and on the SDS for the TA group. Large ESs between each treatment group and the control group were found for most measures.
Clinical Significance: Remission, Recovery, and Reliable Clinical Change.
At pre-treatment 37/50 (74%) of TA group, 32/47 (68%) of CA group, and 36/48 (75%) of control group participants had a GAD-7 score of 10 or more, indicating a diagnosis of GAD. At post-treatment (using the intention-to-treat and LOCF design), 8/50 (16%) of TA group, 8/47 (17%) of CA group, and 29/48 (60%) of the control group participants continued to have a GAD-7 score above 9. Based on the criteria for recovery (a reduction of pre-treatment GAD-7 scores of at least 50%) at post-treatment, 28/50 (56%) of TA group, 33/47 (70%) of CA group and 4/48 (10%) of control group participants were classified as recovered. Based on the criteria for reliable clinical change (statistically reliable change), 48% of TA group, 47% of CA group and 6% of control group participants were classified as having achieved reliable change at post-treatment on the PSWQ.
Chi-squared tests and one-way ANOVAs failed to reveal any differences between treatment groups' ratings of satisfaction with the program with respect to: Overall satisfaction (p = .17); quality of the treatment lessons (p = .84); and quality of the support they received from the technician or clinician (p = .25). Overall, treatment group participants reported an acceptable level of satisfaction with the overall program, with 74/85 (87%) reporting being either very satisfied or mostly satisfied, and 11/85 (13%) neutral/somewhat dissatisfied, with 0% reporting very dissatisfied. Most responding participants (90%) rated the quality of the treatment modules as excellent or good, and 10% rated them as satisfactory; 83% rated the quality of contact with the clinician or technician as excellent or good, 15% rated it as satisfactory, and 2% as unsatisfactory.
When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, the average participant rated the treatment as logical (9/10); they reported feeling confident that the treatment would be successful at teaching them techniques for managing their symptoms (8/10); and they reported a high level of confidence in recommending this treatment to a friend with GAD (9/10). No between treatment group differences were found on these items.
Time/Contact Events Per Participant.
One-way ANOVAs revealed that each participant in the CA group received a greater mean (and SD) (33.2, 4.0) total number of contacts (telephone calls and emails) during the 8 week program than participants in the TA group (31.1, 3.1) (F1, 86 = 7.94, p<0.01). However, no difference was found in the total mean (and SD) time spent by the technician (74.5 mins, 7.8) and clinician (80.8, 22.6) with each participant during the program (p = .08). These time estimates included monitoring individual progress, reading and responding to emails, discussing cases with the clinician, and attending weekly supervision sessions. Conducting this research added approximately 7 hours per week to the technician and the clinician's existing workload. The technician reported that four (10%) participants were discussed with the clinician who emailed these participants once only. No differences in pre-treatment symptom scores, demographic characteristics, or post-treatment symptom scores were observed between the TA group participants who were discussed with the clinician and the other participants.
Follow-Up (3 Month) Outcomes
A univariate ANCOVA controlling for pre-treatment scores (see Table 2), revealed that the CA group had significantly lower PSWQ scores at follow-up than the TA group (F1, 94 = 5.01, p<0.03). A univariate ANCOVA, controlling for pre-treatment scores, failed to reveal differences between the CA and TA groups on the GAD-7 (p = 0.31). Paired samples t-tests for each intervention group revealed that the CA group made significant improvements between post-treatment and follow-up assessments on the PSWQ (t(40) = 2.72, p<0.01) but no change on the GAD-7 (p = 1.00), while the TA had no change on either PSWQ (p = 0.82) or GAD-7 (p = 0.58).
Univariate ANCOVAs conducted on the PHQ-9, K-10, and SDS follow-up scores, while controlling for pre-treatment scores, failed to reveal differences between the CA and TA groups on either the PHQ-9 (p = 0.92), K-10 (p = 0.49) or the SDS (p = 0.37). Paired samples t-tests for each intervention group failed to reveal any differences between post-treatment and follow-up for either the TA or CA groups on the secondary measures.
Pre to follow-up within-group effect sizes (Table 2) on the PSWQ were 1.42 and 0.97 for the clinician and technician-assisted groups, respectively, and 1.48 and 1.61 on the GAD-7, respectively. Large (>0.80) within-group effect sizes (ESs) were found for both treatment groups on the PHQ-9 and K-10, and for the TA group on the SDS.
Clinical Significance: Remission, Recovery, and Reliable Clinical Change.
At follow-up (using the intention-to-treat and LOCF design), 10/50 (20%) of TA group and 9/47 (19%) of CA group participants continued to have a GAD-7 score above 9. Based on the criteria for recovery (a reduction of pre-treatment GAD-7 scores of at least 50%) at follow-up, 30/50 (60%) of TA group and 33/47 (70%) of CA group participants were classified as recovered. Based on the criteria for reliable clinical change (statistically reliable change), 42% of TA group and 59% of CA group were classified as having achieved reliable change at post-treatment on the PSWQ.
Differences Between 3-Month Completers and Non-Completers.
Analyses were conducted to explore differences in pre-treatment, post-treatment, and changes scores of participants in the treatment groups who completed the follow-up questionnaires vs. participants who did not complete these measures. One-way ANOVAs revealed that the group who did not complete the 3-month follow-up questionnaires had significantly higher post-treatment PHQ-9 (F1, 45 = 4.33, p<0.04), K-10 (F1, 45 = 5.06, p<0.03), and SDS scores (F1, 45 = 6.62, p<0.01), but no other differences were found (p range = 0.06–0.60).
This trial compared the efficacy and acceptability of technician- vs. clinician-assisted iCBT for GAD. At intake all participants met DSM-IV diagnosis of GAD, and the majority reported onset before the age of 30 years. In addition to access to the components of the Worry program, CA group participants had weekly email or telephone contact with the clinician and access to an online discussion forum. The clinician actively engaged in treatment with CA group participants. In addition to having access to the Worry program, TA group participants had weekly email or telephone contact with the technician, but did not have access to an online forum. The technician provided support and encouragement, did not provide clinical advice, but was instructed to refer clinical questions or concerns to the clinician.
The first hypothesis, that clinician-assisted treatment using the Worry program would be efficacious was supported. Large within-group effect sizes were obtained for the CA group on measures of GAD and importantly, satisfaction with treatment was high. This replicates the outcomes of an earlier preliminary study using the Worry program  and extends those results by confirming the stability or improvement of clinical gains 3 months post-program.
The second hypothesis, that participants in the TA group would show similar clinical improvements on measures of GAD, depression, and disability, to those in the CA group, was also supported. At post-treatment outcomes for both treatment groups were superior to the control group, satisfaction with treatment was high in both treatment groups, and there were no differences between the two treatment groups in clinical outcomes. At post-treatment more than 50% of participants in the treatment groups were classified as recovered compared to 10% of controls, while reliable clinical change was observed in almost 50% of participants in the treatment groups compared to 6% of controls. Consistent with the final hypothesis, at follow-up this pattern of results was maintained for participants in the TA group, while at follow-up the CA group obtained significantly lower scores on the PSWQ than at post-treatment, indicating that they made additional gains.
At post-treatment and follow-up ESs in the treatment groups on both GAD measures were greater than 1.0 indicating that the treatment effect was considerable. The magnitude of these ESs is comparable to improvements typically reported in meta-analyses of face-to-face CBT-based treatment of GAD , . These results were obtained with a relatively low level of total contact time per participant but a large number of total contacts. It is estimated that 7 hours per week of clinician or technician time was required to conduct each group of more than 45 participants. However, during treatment CA group participants received approximately 33 contacts, prompts, and reminders compared to 31 in the TA group, a difference that was statistically, but unlikely to be clinically significant. More than half of these contacts were managed by the automated email system. These results indicate the importance of regular contact, even if the contact is automatic or of relatively short duration.
These results replicate the findings of a recent RCT  reporting the preliminary results of the Worry program. The present results indicate the Worry program can reliably produce good clinical outcomes that are sustained for at least 3 months post-treatment, and that the procedure is acceptable to consumers with GAD. A total of 80 minutes of staff time was required per participant using the Worry program, which compares favorably to the 8 to 16 hours of clinician contact usually required in face-to-face treatment, indicating this approach is cost-effective. These results also support recent evidence indicating that Internet-based treatment programs may be effectively administered by a non-clinician , , , when supervised by a clinician. Importantly, if the technician in the present study was concerned about a participant in their group they were able to “step-up” participants to the clinician, but did so with only 10% of participants. This indicates that the majority found the intervention by the technician sufficient for their needs, and demonstrates a potential model for integrating clinician and technician support during iCBT programs.
The relatively small sample size is one limitation of this study. The low completion rates of questionnaires at follow-up is another limitation, and analyses revealed that non-completers had elevated post-treatment scores on the measures of depression, psychological distress, and disability relative to completers, indicating that the follow-up results should be interpreted with caution.
An important potential limitation is the use of a delayed treatment control group rather than an attention-control placebo. This choice was grounded in concerns about the impact of raised expectations of symptom resolution in anxious participants placed in a placebo, attention-control condition. These concerns were heightened by the geographical spread of participants, who were from all around Australia, and hence unable to be reached by the investigators should additional help have been required.
This randomized controlled trial found no difference between a clinician- and technician-assisted Internet-based treatment program for GAD. Both conditions resulted in large effect sizes, clinically significant improvements, and high levels of acceptability, while a delayed treatment control group did not improve. These results were sustained at 3-month follow-up in the technician-assisted group, while the clinician-assisted group showed evidence of continued improvement. These findings are consistent with emerging evidence indicating that Internet-based treatment programs may be effectively administered by a non-clinician , , , when supervised by a clinician. Furthermore, this model of implementation requires considerably less time than face-to-face treatment, and appears highly acceptable to people with GAD. The question is not whether to accept such an innovative model of service delivery, but how to do so in an ethical, competent, safe, and cost-effective way, while maintaining excellent clinical standards.
The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the VirtualClinic program by Mr Jason Hando, and Mr Daniel Winter.
Conceived and designed the experiments: ER NT GA. Performed the experiments: ER KM. Analyzed the data: NT GS KS. Wrote the paper: ER NT GA KM GS KS.
- 1. Australian Bureau of Statistics (2008) Australian Bureau of Statistics2008National Survey of Mental Health and Wellbeing 2007: Summary of Results. Released 10/23/2008. Available: http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/6AE6DA447F985FC2CA2574EA00122BD6/File/43260_2007.pdf. Accessed 27 Nov 2009. National Survey of Mental Health and Wellbeing 2007: Summary of Results. Released 10/23/2008. Available: http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/6AE6DA447F985FC2CA2574EA00122BD6/File/43260_2007.pdf. Accessed 27 Nov 2009.
- 2. Kessler RC, Berglund PA, Chiu WT, Demler O, Glantz W, et al. (2008) The National Comorbidity Survey Replication (NCS-R). In: Kessler RC, Ustun TB, editors. The WHO World Mental Health Surveys. Cambridge: Cambridge University Press. pp. 165–210.RC KesslerPA BerglundWT ChiuO. DemlerW. Glantz2008The National Comorbidity Survey Replication (NCS-R)RC KesslerTB UstunThe WHO World Mental Health SurveysCambridgeCambridge University Press165210
- 3. Hoffman DL, Dukes EM, Wittchen HU (2008) Human and economic burden of generalized anxiety disorder. Depress Anxiety 25: 72–90.DL HoffmanEM DukesHU Wittchen2008Human and economic burden of generalized anxiety disorder.Depress Anxiety257290
- 4. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H (2004) Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry 184: 526–533.G. AndrewsC. IssakidisK. SandersonJ. CorryH. Lapsley2004Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders.Br J Psychiatry184526533
- 5. Wittchen HU (2002) Generalized anxiety disorder: prevalence, burden, and cost to society. Depress Anxiety 16: 162–171.HU Wittchen2002Generalized anxiety disorder: prevalence, burden, and cost to society.Depress Anxiety16162171
- 6. Lang AJ (2004) Treating generalized anxiety disorder with cognitive-behavioral therapy. J Clin Psychiatry 65: Suppl 1314–19.AJ Lang2004Treating generalized anxiety disorder with cognitive-behavioral therapy.J Clin Psychiatry65Suppl 131419
- 7. Ladouceur R, Dugas MJ, Freeston MH, Léger E, Gagnon F, et al. (2000) Efficacy of a cognitive-behavioral treatment for generalized anxiety disorder: evaluation in a controlled clinical trial. J Consult Clin Psychol 68: 957–964.R. LadouceurMJ DugasMH FreestonE. LégerF. Gagnon2000Efficacy of a cognitive-behavioral treatment for generalized anxiety disorder: evaluation in a controlled clinical trial.J Consult Clin Psychol68957964
- 8. Dugas MJ, Ladouceur R, Léger E, Freeston MH, Langlois F, et al. (2003) Group cognitive-behavioral therapy for generalized anxiety disorder: treatment outcome and long-term follow-up. J Consult Clin Psychol 71: 821–825.MJ DugasR. LadouceurE. LégerMH FreestonF. Langlois2003Group cognitive-behavioral therapy for generalized anxiety disorder: treatment outcome and long-term follow-up.J Consult Clin Psychol71821825
- 9. Marks I, Cavanagh K, Gega L (2007) Hands-on help: Computer-aided psychotherapy. New York: Psychology Press Taylor and Francis.. 223 p.I. MarksK. CavanaghL. Gega2007Hands-on help: Computer-aided psychotherapy.New YorkPsychology Press Taylor and Francis.223
- 10. Titov N (2007) Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 41: 95–114.N. Titov2007Status of computerized cognitive behavioural therapy for adults.Aust N Z J Psychiatry4195114
- 11. Andersson G, Cuijpers P (2009) Internet based and other computerized psychological treatments for adult depression: a meta-analysis. Cog Behav Ther 8: 196–205.G. AnderssonP. Cuijpers2009Internet based and other computerized psychological treatments for adult depression: a meta-analysis.Cog Behav Ther8196205
- 12. Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, et al. (2005) Internet-based self-help for depression: randomized controlled trial. Br J Psychiatry 187: 456–461.G. AnderssonJ. BergstromF. HollandareP. CarlbringV. Kaldo2005Internet-based self-help for depression: randomized controlled trial.Br J Psychiatry187456461
- 13. Perini S, Titov N, Andrews G (2009) Clinician-assisted Internet-based treatment is effective for depression: a randomized controlled trial. Aust N Z J Psychiatry 43: 571–578.S. PeriniN. TitovG. Andrews2009Clinician-assisted Internet-based treatment is effective for depression: a randomized controlled trial.Aust N Z J Psychiatry43571578
- 14. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, et al. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLos ONE. N. TitovG. AndrewsM. DaviesK. McIntyreE. RobinsonInternet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance.PLos ONEin press. in press.
- 15. Klein B, Richards JC, Austin DW (2006) Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 37: 213–238.B. KleinJC RichardsDW Austin2006Efficacy of Internet therapy for panic disorder.J Behav Ther Exp Psychiatry37213238
- 16. Kiropoulos LA, Klein B, Austin DW, Gilson K, Pier C, et al. (2008) Is Internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J Anxiety Disord 22: 1273–1284.LA KiropoulosB. KleinDW AustinK. GilsonC. Pier2008Is Internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?J Anxiety Disord2212731284
- 17. Shandley K, Austin DW, Klein B, Pier C, Schattner P, et al. (2008) K. ShandleyDW AustinB. KleinC. PierP. Schattner2008Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res 10(2):e14. Available: http://www.jmir.org/2008/2/e14/via the Internet. Accessed 26 Nov 2009. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res 10(2):e14. Available: http://www.jmir.org/2008/2/e14/via the Internet. Accessed 26 Nov 2009.
- 18. Carlbring P, Nilsson-Ihrfelt E, Waara J, Kollenstam C, Buhrman M, et al. (2005) Treatment of panic disorder: live therapy vs. self-help via the Internet. Behav Res Ther 43: 1321–1333.P. CarlbringE. Nilsson-IhrfeltJ. WaaraC. KollenstamM. Buhrman2005Treatment of panic disorder: live therapy vs. self-help via the Internet.Behav Res Ther4313211333
- 19. Berger T, Hohl E, Caspar F (2009) Internet-based treatment for social phobia: a randomized controlled trial. J Clin Psychology 65: 1–15.T. BergerE. HohlF. Caspar2009Internet-based treatment for social phobia: a randomized controlled trial.J Clin Psychology65115
- 20. Titov N, Andrews G, Schwencke G, Drobny J, Einstein D (2008) Shyness 1: a randomized controlled trial of an Internet-based treatment for social phobia. Aust N Z J Psychiatry 42: 585–594.N. TitovG. AndrewsG. SchwenckeJ. DrobnyD. Einstein2008Shyness 1: a randomized controlled trial of an Internet-based treatment for social phobia.Aust N Z J Psychiatry42585594
- 21. Titov N, Andrews G, Schwencke G (2008) Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 42: 595–605.N. TitovG. AndrewsG. Schwencke2008Shyness 2: treating social phobia online: replication and extension.Aust N Z J Psychiatry42595605
- 22. Titov N, Andrews G, Choi I, Schwencke G, Mahoney A (2009) Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 42: 1030–1040.N. TitovG. AndrewsI. ChoiG. SchwenckeA. Mahoney2009Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia.Aust N Z J Psychiatry4210301040
- 23. Andersson G, Carlbring P, Holmstrom A, Sparthan E, Furmark T, et al. (2006) Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol 74: 677–686.G. AnderssonP. CarlbringA. HolmstromE. SparthanT. Furmark2006Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial.J Consult Clin Psychol74677686
- 24. Carlbring P, Gunnarsdóttir M, Hedensjö L, Andersson G, Ekselius L, et al. (2007) Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 190: 123–128.P. CarlbringM. GunnarsdóttirL. HedensjöG. AnderssonL. Ekselius2007Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support.Br J Psychiatry190123128
- 25. Titov N, Andrews G, Choi I, Schwencke G, Johnston L (2009) Internet-based cognitive behavior therapy for social phobia without clinical input is effective: a pragmatic RCT of two types of reminders. Aust N Z J Psychiatry 43: 913–919.N. TitovG. AndrewsI. ChoiG. SchwenckeL. Johnston2009Internet-based cognitive behavior therapy for social phobia without clinical input is effective: a pragmatic RCT of two types of reminders.Aust N Z J Psychiatry43913919
- 26. Titov N, Andrews G, Schwencke G, Solley K, Johnston L, et al. (2009) An RCT comparing two types of support on severity of symptoms for people completing Internet-based cognitive behavior therapy for social phobia. Aust N Z J Psychiatry 43: 920–926.N. TitovG. AndrewsG. SchwenckeK. SolleyL. Johnston2009An RCT comparing two types of support on severity of symptoms for people completing Internet-based cognitive behavior therapy for social phobia.Aust N Z J Psychiatry43920926
- 27. Titov N, Andrews G, Robinson E, Schwencke G, Johnston L, et al. (2009) Clinician-assisted Internet-based treatment is effective for generalized anxiety disorder: a randomized controlled trial. Aust N Z J Psychiatry 43: 905–912.N. TitovG. AndrewsE. RobinsonG. SchwenckeL. Johnston2009Clinician-assisted Internet-based treatment is effective for generalized anxiety disorder: a randomized controlled trial.Aust N Z J Psychiatry43905912
- 28. Moher D, Schulz KF, Altman DG, for the CONSORT group (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. JAMA 285: 1987–91.D. MoherKF SchulzDG Altmanfor the CONSORT group2001The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials.JAMA285198791
- 29. American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association.. 943 p.American Psychiatric Association2000Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision.Washington, DCAmerican Psychiatric Association.943
- 30. Kroenke K, Spitzer RL, Williams JBW (2001) The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16: 606–613.K. KroenkeRL SpitzerJBW Williams2001The PHQ-9: validity of a brief depression severity measure.J Gen Intern Med16606613
- 31. Spitzer RL, Kroenke K, Williams JBW, Löwe B (2006) A brief measure for assessing generalized anxiety disorder the GAD-7. Arch Intern Med 166: 1092–1097.RL SpitzerK. KroenkeJBW WilliamsB. Löwe2006A brief measure for assessing generalized anxiety disorder the GAD-7.Arch Intern Med16610921097
- 32. Furmark T, Tillfors M, Everz P-O, Marteinsdottir I, Gefvert O, et al. (1999) Social phobia in the general population: prevalence and sociodemographic profile. Soc Psychiatry Psychiatr Epidemiol 34: 416–424.T. FurmarkM. TillforsP-O EverzI. MarteinsdottirO. Gefvert1999Social phobia in the general population: prevalence and sociodemographic profile.Soc Psychiatry Psychiatr Epidemiol34416424
- 33. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, et al. (1998) The Mini-International Neuropsychiatric Interview (M.I.N.I.): the Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10. J Clin Psychiatry 59(S20): 22–33.DV SheehanY. LecrubierKH SheehanP. AmorimJ. Janavs1998The Mini-International Neuropsychiatric Interview (M.I.N.I.): the Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10.J Clin Psychiatry59(S20)2233
- 34. Meyer TJ, Miller ML, Metzger RL, Borkovec TD (1990) Development and validation of the Penn State Worry Questionnaire. Behav Res Ther 28: 487–495.TJ MeyerML MillerRL MetzgerTD Borkovec1990Development and validation of the Penn State Worry Questionnaire.Behav Res Ther28487495
- 35. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, et al. (2002) Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 32: 959–976.RC KesslerG. AndrewsLJ ColpeE. HiripiDK Mroczek2002Short screening scales to monitor population prevalences and trends in non-specific psychological distress.Psychol Med32959976
- 36. Sheehan DV (1983) The Anxiety Disease. New York: Scribner.. 151 p.DV Sheehan1983The Anxiety Disease.New YorkScribner.151
- 37. Devilly GJ, Borkovec TD (2000) Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 31: 73–86.GJ DevillyTD Borkovec2000Psychometric properties of the credibility/expectancy questionnaire.J Behav Ther Exp Psychiatry317386
- 38. Borkovec TD, Nau SD (1972) Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry 3: 257–260.TD BorkovecSD Nau1972Credibility of analogue therapy rationales.J Behav Ther Exp Psychiatry3257260
- 39. Garb HN (2007) Computer-administered interviews and rating scales. Psychol Assess 19: 4–13.HN Garb2007Computer-administered interviews and rating scales.Psychol Assess19413
- 40. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, et al. (2007) Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput Human Behav 23: 1421–1434.P. CarlbringS. BruntS. BohmanD. AustinJ. Richards2007Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research.Comput Human Behav2314211434
- 41. Julious SA (2004) Tutorial in biostatistics: samples sizes for clinical trials with normal data. Stat Med 23: 1921–1986.SA Julious2004Tutorial in biostatistics: samples sizes for clinical trials with normal data.Stat Med2319211986
- 42. Vickers AJ (2005) Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med Res Methodol 5: 35.AJ Vickers2005Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data.BMC Med Res Methodol535
- 43. Vickers AJ (2005) Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom Med 67: 652–655.AJ Vickers2005Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches.Psychosom Med67652655
- 44. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, et al. (2008) Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care 46: 266–274.B. LöweO. DeckerS. MüllerE. BrählerD. Schellberg2008Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population.Med Care46266274
- 45. Richards DA, Suckling R (2009) Improving access to psychological therapies: Phase IV prospective cohort study. Br J Clin Psychol 48: 377–396.DA RichardsR. Suckling2009Improving access to psychological therapies: Phase IV prospective cohort study.Br J Clin Psychol48377396
- 46. Jacobson NS, Truax P (1999) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59: 12–19.NS JacobsonP. Truax1999Clinical significance: a statistical approach to defining meaningful change in psychotherapy research.J Consult Clin Psychol591219
- 47. Hofmann SG, Smits JA (2008) Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. J Clin Psychiatry 69: 621–632.SG HofmannJA Smits2008Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials.J Clin Psychiatry69621632
- 48. Mitte K (2005) Meta-analysis of cognitive-behavioural treatments for generalized anxiety disorder: a comparison with pharmacotherapy. Psychol Bull 131: 785–795.K. Mitte2005Meta-analysis of cognitive-behavioural treatments for generalized anxiety disorder: a comparison with pharmacotherapy.Psychol Bull131785795