Internet-based cognitive behavioural therapy (iCBT) for depression is effective when guided by a clinician, less so if unguided. Question: Would guidance from a technician be as effective as guidance from a clinician?
Randomized controlled non-inferiority trial comparing three groups: Clinician-assisted vs. technician-assisted vs. delayed treatment. Community-based volunteers applied to the VirtualClinic (www.virtualclinic.org.au) research program, and 141 participants with major depressive disorder were randomized. Participants in the clinician- and technician-assisted groups received access to an iCBT program for depression comprising 6 online lessons, weekly homework assignments, and weekly supportive contact over a treatment period of 8 weeks. Participants in the clinician-assisted group also received access to a moderated online discussion forum. The main outcome measures were the Beck Depression Inventory (BDI-II) and the Patient Health Questionnaire-9 Item (PHQ-9). Completion rates were high, and at post-treatment, both treatment groups reduced scores on the BDI-II (p<0.001) and PHQ-9 (p<0.001) compared to the delayed treatment group but did not differ from each other. Within group effect sizes on the BDI-II were 1.27 and 1.20 for the clinician- and technician-assisted groups respectively, and on the PHQ-9, were 1.54 and 1.60 respectively. At 4-month follow-up participants in the technician group had made further improvements and had significantly lower scores on the PHQ-9 than those in the clinician group. A total of approximately 60 minutes of clinician or technician time was required per participant during the 8-week treatment program.
Both clinician- and technician-assisted treatment resulted in large effect sizes and clinically significant improvements comparable to those associated with face-to-face treatment, while a delayed treatment control group did not improve. These results provide support for large scale trials to determine the clinical effectiveness and acceptability of technician-assisted iCBT programs for depression. This form of treatment has potential to increase the capacity of existing mental health services.
Australian New Zealand Clinical Trials Registry ACTRN12609000559213
Citation: Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K (2010) Internet Treatment for Depression: A Randomized Controlled Trial Comparing Clinician vs. Technician Assistance. PLoS ONE5(6): e10939. https://doi.org/10.1371/journal.pone.0010939
Editor: Antonio Verdejo García, University of Granada, Spain
Received: January 6, 2010; Accepted: May 11, 2010; Published: June 8, 2010
Copyright: © 2010 Titov et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was investigator initiated. It was funded by departmental funds. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Depression and anxiety are both common and disabling –. Although effective treatments such as medication and cognitive behavioural therapy (CBT) are available, less than half report seeking any treatment and few receive specialized treatment from trained mental health professionals . Barriers to effective treatment include limited availability of skilled clinicians, direct and indirect costs of treatment, and difficulty attending treatment during office hours.
One strategy for reducing barriers to effective treatment is the development of computerized cognitive behavioural therapy (CCBT) programs, which present structured sessions of CBT via a computer interface , . Such programs may be accessed by patients at primary or specialist care facilities, via the Internet or DVD, and may be completed with the assistance of a clinician or entirely self-guided. CCBT reduces the demands on clinician time but the results of entirely self-guided treatment has been significantly inferior –.
A growing body of evidence indicates that clinician-assisted Internet-based CBT (iCBT) can result in significant improvements in patients with depressive – and anxiety disorders –, with results comparable to those obtained from good face-to-face treatment. Recent studies of iCBT programs for generalized anxiety disorder and social phobia indicate that clinician time can be reduced considerably by delegating the majority of patient contact to a non-clinician (technician), without compromising clinical outcomes or acceptability , , . iCBT programs that can be safely and effectively administered by technicians who are supervised by clinicians, have considerable potential for enhancing existing mental health services. An important question is whether similar effects could be obtained from iCBT for depression when administered by a technician.
The present CONSORT-Revised compliant randomized controlled trial (RCT)  examined the clinical efficacy and acceptability of clinician- and technician-assisted iCBT using the Sadness program ,  for people with a DSM-IV  diagnosis of depression. We hypothesised that participants in a technician-assisted (TA) group would show similar clinical improvements on measures of depression and disability as a clinician-assisted (CA) group; that improvements would be sustained at follow-up; and that both treatment groups would have better outcomes than a delayed treatment (control) group.
This study was approved by the St Vincent's Hospital Human Research Ethics Committee (HREC) and by the University of New South Wales HREC. Written informed consent was obtained from all participants.
Participants were recruited from June to July 2009 via a website (www.virtualclinic.org.au) providing information about common mental disorders including depression, and a link to apply online to join a research treatment program. Participants first applied online, completing questionnaires about severity of symptoms, demographic details, and chronicity of symptoms (see Table 1).
Exclusions were (i) not a resident of Australia; (ii) less than 18 years of age; (iii) no regular access to a computer, the Internet, and use of a printer; (iv) currently participating in CBT; (v) using illicit drugs or consuming more than three standard drinks/day; (vi) experience of a psychotic mental illness (schizophrenia or bipolar disorder) or current severe symptoms of depression (defined as a total score >23 or responding >2 to Question 9 (suicidal ideation) on the Patient Health Questionnaire - 9 Item (PHQ-9) ; (vii) a total PHQ-9 score below 10; and (viii) if taking medication, had been taking the same dose for less than 1 month or intending to change that dose during the course of the program. Excluded applicants immediately received an on-screen message and email thanking them for their application, and encouraging them to discuss their symptoms with their physician.
Participants who passed the screening phase were telephoned for a diagnostic interview using the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI)  to determine whether they met DSM-IV criteria for major depressive episode. Participants who satisfied all criteria were informed of the study design and invited to return a completed consent form by email. The study was approved by the Human Research Ethics Committees of St Vincent's Hospital, Sydney, and the University of New South Wales.
Treatment group participants received access to the Sadness program, an iCBT program with demonstrated efficacy at reducing symptoms of depression –. The Sadness program consists of six online lessons, printable summary and homework assignments, automatic emails, and additional resource documents. The six online lessons represent best practice principles used in CBT programs for depression including behavioural activation, cognitive restructuring, problem solving, and assertiveness skills. Part of the content of each lesson is presented in the form of an illustrated story about a woman with depression who, with the help of a clinical psychologist, learns to gain mastery over her symptoms. Automatic emails are sent to congratulate participants for completing each lesson, to remind them to complete materials, and to notify them of new resources. As people progress through each lesson they have access to additional written documents providing supplementary information about techniques such as managing sleep problems, panic, and other common comorbid symptoms. They are also provided with access to vignettes written by previous participants about their own experiences of managing depression during the Sadness program. Participants are expected to complete the homework tasks prior to completing the next lesson, and to complete all lessons within 8 weeks.
All participants began the 8-week treatment program at the same time. Participants were advised to complete one lesson every 7–10 days and to complete the six lessons within 8 weeks of starting. All participants received automatic emails informing them when a lesson was to be completed, and reminder emails if they had not completed a lesson within 7 days of notification.
Three staff conducted the study, with supervision from NT. The technician (KM) was employed in an administrative role as a Clinic Manager, Anxiety Disorders Clinic, Mental Health Service, St Vincent's Hospital, Sydney. She reported no prior experience with research programs, no qualifications in health care or counseling, and had no clinical duties in her usual role. The clinician (MD) was a qualified and registered psychiatrist, employed at the same unit as KM. The third staff member (KS) was a research assistant, who provided administrative support to the technician and clinician.
During treatment the technician provided TA group participants with weekly email or telephone contact. The technician was given a guideline script which identified the topics covered in each lesson of the program and activities participants should be encouraged to practice for each lesson. The technician was instructed to contact each TA group participant weekly to provide encouragement and support, and where possible to respond to participants' general questions by referring them to the materials in the Sadness program. The technician was not permitted to provide clinical advice. The technician received supervision from the clinician and was instructed to inform the clinician of any perceived deterioration in their participants' mental health status, or of any concerns about participants' wellbeing. While conducting this research the technician maintained her full-time role as a Clinic Manager.
CA group participants had weekly email or telephone contact with the clinician and access to an online discussion forum where they could post questions to the clinician about the program content. Information posted on the discussion forum could be read by other participants in the CA group. The clinician was provided with the same guideline script as the technician but was also instructed to answer participants' questions via forum, email, or telephone. The clinician was instructed to actively engage with each participant in treatment including goal setting, problem solving, and discussion of strategies for overcoming hurdles to progress. Because of the clinical nature of messages on the forum, the TA group did not have access to a forum. The clinician and technician were instructed to spend no more than 10 minutes in contact with each participant per week. The total time required and nature of all contacts with participants during treatment was recorded.
This study was a 3 group randomized controlled non-inferiority trial to determine whether technician-assisted Internet treatment was equivalent to clinician-assisted Internet treatment but superior to delayed treatment (waitlist control).
Primary and Secondary Outcomes
One week prior to beginning the trial participants completed the following questionnaires online: The PHQ-9, the Beck Depression Inventory (second edition) (BDI-II) , the Kessler 10 (K-10) , the Sheehan Disability Scales (SDS) , and the Credibility/Expectancy Questionnaire (CEQ) , . The PHQ-9 is a 9-item measure with scores ranging from 0–27. The BDI-II is a 21-item measure with scores ranging from 0–64. A score of 10 on the PHQ-9 has been identified as providing an important threshold for identifying DSM-IV congruent depression , . The K-10 is a 10-item measure of psychological distress with scores ranging from 10–50. The SDS is a 3 item measure of disability with scores ranging from 0–30 and the CEQ is a widely used measure of the expectancies or perception of treatment credibility.
The PHQ-9, BDI-II, K-10 and SDS were re-administered one week post-treatment and at four month post-treatment (follow-up), while the PHQ-9 was also administered mid-treatment (at week 5). All of these measures are considered reliable, valid, and appropriate for clinical and research purposes, with recent research indicating that online administration of questionnaires results in acceptable reliability of responses , . Changes in the PHQ-9 and BDI-II were considered the primary outcome measures, while changes in the K-10, SDS and a treatment satisfaction questionnaire (based on the CEQ) were the secondary outcome measures. Follow-up results were not available for the control group, who had started treatment by that time.
Sample Size and Randomization
Power calculations were based on a non-inferiority trial design comparing parallel-groups. Alpha was set at 0.025, power at 90%, and the mean minimal reliable change index for the PHQ-9 (based on earlier findings) and standard deviations for each group were expected to be equivalent (5 and 4, respectively). Using Table V from Julious , the minimum sample size for each group was identified as 39, but more were recruited to hedge against attrition.
The 141 people accepted into the program were randomised by NT via a true randomisation process (www.random.org) to either the CA (n = 49), TA (n = 47), or control groups (n = 45) (see Figure 1). Allocation preceded the diagnostic telephone call, and self report measures precluded blinding.
Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analyzed with one-way analysis of variance (ANOVAs) and chi-square tests, followed by t-tests with Bonferonni corrected p values. Changes in participants' questionnaire scores from pre to post-treatment and from pre-treatment to follow-up were analyzed using repeated measures analyses of covariance (ANCOVAs). This approach is recommended as a robust and reliable statistical strategy for analyzing the results of RCTs , . Changes in questionnaire scores between post-treatment and follow-up were analyzed using paired samples t-tests. Effect sizes (Cohen's d) were calculated both within- and between-groups, based on the pooled standard deviation. We also calculated 95% confidence intervals for the between-group effect sizes between the two treatment groups for each primary outcome measure. All post-treatment and follow-up analyses adopt an intention-to-treat (ITT) design where missing data is replaced by the last observation carried forward (LOCF).
Two scores (credibility and expectancy) were derived from the CEQ by subjecting the scores to a linear transformation as described in . Change scores for the two primary outcome measures, from pre to post and from pre to follow-up, were calculated for each intervention group participant. Pearson product moment correlations were calculated between these change scores and the total number of contacts with participants during the program and with the total time spent per participant, to explore variables that may affect outcomes.
Three measures of clinical significance were employed. First, pre-treatment and post-treatment PHQ-9 scores were compared with optimum cut-offs for a probable diagnosis of depression using evidence from the literature , , to provide an index of remission. This was defined as the proportion of participants who initially scored above the optimum cut-off (PHQ-9 total score of 10 or more) and subsequently scored below this cut-off. Secondly, an estimate of recovery was made by identifying the proportion of participants in each group who demonstrated a significant reduction in their symptoms (defined here, as a reduction of 50% of pre-treatment PHQ-9 scores) as described in recent dissemination studies . Thirdly, the percentage of participants in each group who met criteria for both reliable and clinically significant change on the BDI-II was calculated. This was defined as the proportion of participants who met the criteria of statistically reliable change as described in Jacobson and Truax  and who also demonstrated a reduction in BDI-II scores of at least 10 points, a magnitude of change that has been used in similar research . A reliable change index for the BDI-II was calculated separately for each of the three groups using their pre-treatment standard deviation, and a test-retest reliability coefficient of 0.93, as reported in the BDI-II manual .
Two hundred and fifty eight individuals expressed interest in the study (Figure 1), 141 met all inclusion criteria, and were randomized to one of the three groups. Forty one TA and 46 CA group participants completed the pre-treatment questionnaires and began lesson 1 and are eligible for analysis, as are 40 control group participants, who completed the pre-treatment questionnaires.
Characteristics of the groups are presented in Table 1. There were no significant between-group differences in age, marital status, education, employment, previous discussions of symptoms with a health professional, use of medication, weekly use of the internet, confidence in using computers, treatment expectancies, or pre-treatment scores on the outcome measures. Treatment groups were also equally chronic, with most treatment group participants reporting at least 5 previous episodes of depression and fewer than 2 symptom free months in the past 2 years. However, there were significantly fewer males in the CA group (χ2 = 6.98, p<0.05).
Thirty three (33/41, 80%) TA and 32/46 CA (70%) group participants completed all 6 lessons within the required time. Post-treatment data was collected from 37/41 (90%) TA, 41/46 (89%) CA, and 39/40 (98%) control group participants. Follow-up data (4 months post-treatment) were collected from 30/41 (73%) TA and 38/46 (83%) CA group participants. In accordance with the intention-to-treat and LOCF models, missing post-treatment or follow-up data was addressed by replicating that participant's last observed score on that measure. Follow-up data were not available for control group participants who had begun treatment by that time.
Post-Treatment (11-Week) Outcomes
Univariate ANCOVAs on post-treatment PHQ-9 and BDI-II scores, controlling for pre-treatment scores (see Table 2), revealed significant effects for PHQ-9 (F2, 123 = 29.72, p<0.001) and BDI-II (F2, 123 = 21.72, p<0.001) scores. Post-hoc pairwise comparisons revealed no difference on either measure between treatment groups, but significant differences between the treatment groups and the control group (p<0.001).
Univariate ANCOVAs conducted on the K-10 and SDS post-treatment scores, while controlling for pre-treatment scores, revealed significant effects for K-10 (F2, 124 = 13.06, p<0.001) and SDS (F2, 124 = 63.87, p<0.001) scores. Post-hoc pairwise comparisons revealed no difference on either measure between treatment groups, but significant differences between the treatment groups and the control group (p<0.001).
Pre to post-treatment within-group effect sizes on the BDI-II were 1.27 and 1.20 for the clinician and technician-assisted groups, respectively, and on the PHQ-9 were 1.54 and 1.60, respectively (Table 2). The between-group effect size for the treatment groups on the BDI-II was 0.07 (CI95% = −0.35 to 0.49) and on the PHQ-9 was 0.07 (CI95% = −0.36 to 0.49). Large (>0.80) within-group effect sizes (ESs) were found for both treatment groups on the K-10 and SDS. Large ESs between each treatment group and the control group were found for all measures.
Clinical Significance: Remission, Recovery, and Reliable Clinical Change.
At pre-treatment 37/41 (90%) of TA group, 41/46 (89%) of CA group, and 31/40 (78%) of control group participants had a PHQ-9 score of 10 or more, indicating a persisting diagnosis of depression. At post-treatment (using the ITT and LOCF design) 11/41 (27%) of TA group, 18/46 (39%) of CA group, and 31/40 (78%) of control group participants continued to have a PHQ-9 score above 9. Based on the criteria for recovery (a reduction of pre-treatment PHQ-9 scores of at least 50%) at post-treatment, 61% of TA group, 43% of CA group and 3% of control group participants were classified as recovered. Based on the criteria for reliable clinical change (statistically reliable change and a reduction of pre-treatment BDI-II scores of at least 10 points), 56% of TA group, 61% of CA group and 8% of control group participants were classified as having achieved reliable clinical change at post-treatment.
Chi-squared tests and one-way ANOVAs failed to reveal any differences between treatment groups' ratings of satisfaction with the program with respect to: Overall satisfaction (p = .13); quality of the treatment lessons (p = .82); and quality of the support they received from the technician or clinician (p = .40). Overall, treatment group participants reported an acceptable level of satisfaction with the overall program, with 67/77 (87%) reporting being either very satisfied or mostly satisfied, 10/77 (13%) neutral/somewhat dissatisfied, and 0% reporting very dissatisfied. Most responding participants (90%) rated the quality of the treatment modules as excellent or good; 81% rated the quality of Internet correspondence with the clinician or technician as excellent or good, 14% rated it as satisfactory, and 4% as unsatisfactory.
When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, the average participant rated the treatment as logical (8/10); they reported feeling confident that the treatment would be successful at teaching them techniques for managing their symptoms (8/10); and they reported a high level of confidence in recommending this treatment to a friend with depression (8/10). No between treatment group differences were found on these items.
Time/Contact Events Per Participant.
One-way ANOVAs did not reveal any difference in the total number of contacts (telephone calls and emails) provided during the 8 week program to each participant in the TA (mean, SD; 36.9, 5.3) and CA groups (34.7, 6.3) (p = .10). One-way ANOVAs also failed to reveal any difference in the total time spent by the technician (mean, SD, 61.0 mins, 9.8) or clinician (60.5, 19.0) with each participant during the program (p = .88). These time estimates included time required monitoring individual progress, reading and responding to emails, discussing cases with the clinician, and attending weekly supervision sessions. Conducting this research added approximately 5 hours per week to the technician and the clinician's existing workload. The technician reported that most TA group participants were satisfied during treatment but four (10%) participants were discussed with the clinician because of concerns about their progress. The clinician intervened in two of these cases, withdrawing them from the program with their consent, and referring them to other services.
Follow-Up (4-Month) Outcomes
Univariate ANCOVAs on follow-up PHQ-9 and BDI-II scores, controlling for pre-treatment scores (see Table 2), revealed a significant difference between the CA and TA groups on the PHQ-9 (F1, 84 = 4.94, p<0.05) in favour of the TA group, but no difference between groups on the BDI-II (p = .10). Paired samples t-tests for each intervention group revealed that the TA group made significant improvements between post-treatment and follow-up assessments on the PHQ-9 (t(40) = 2.48, p<0.05) and BDI-II (t(40) = 3.29, p<0.05). They also revealed that the CA group deteriorated between post-treatment and follow-up assessments on the PHQ-9 (t(45) = 2.19, p<0.05) but had not changed on the BDI-II (p = 0.18).
Univariate ANCOVAs conducted on the K-10, and SDS follow-up scores, while controlling for pre-treatment scores, revealed a significant difference between the CA and TA groups on the K-10 (F1, 84 = 4.18, p<0.05), but not on the SDS.
Paired samples t-tests for each intervention group revealed that the TA group made significant improvements between post-treatment and follow-up assessments on the K-10 (t(40) = 2.59, p<0.05) and SDS (t(40) = 2.04, p<0.05), with no significant changes in CA group scores.
Effect Sizes. Pre to follow-up within-group effect sizes (Table 2) on the BDI-II were 1.03 and 1.59 for the clinician and technician-assisted groups, respectively, and 1.11 and 1.89 on the PHQ-9, respectively. The between-group effect size for the treatment groups on the BDI-II was 0.39 (CI95% = −0.03 to 0.82) and on the PHQ-9 was 0.46 (CI95% = 0.03 to 0.88). Large (>0.80) within-group effect sizes (ESs) were found for both treatment groups on the K-10 and SDS.
Clinical Significance: Remission, Recovery, and Reliable Clinical Change.
At follow-up (using the intention-to-treat and LOCF design) 9/41 (22%) of TA group and 18/46 (39%) of CA group participants continued to have a PHQ-9 score above 9. Based on the criteria for recovery (a reduction of pre-treatment PHQ-9 scores of at least 50%) at follow-up 66% of TA group and 43% of CA group participants were classified as recovered. Based on the criteria for reliable clinical change (statistically reliable change and a reduction of pre-treatment BDI-II scores of at least 10 points) 59% of TA group and 59% of CA group participants were classified as having achieved reliable clinical change at follow-up.
This trial examined the relative efficacy and acceptability of technician- vs. clinician-assisted Internet-based CBT for depression. At intake all participants met DSM-IV diagnosis of major depressive episode, and the majority reported a history of multiple depressive episodes. In addition to having access to the components of the Sadness program, CA group participants had weekly email or telephone contact with a clinician and access to an online discussion forum. The clinician actively engaged in treatment with CA group participants. TA group participants also had access to the Sadness program and weekly email or telephone contact with the technician, but did not have access to an online forum. The technician provided support and encouragement, but did not provide clinical advice, and was instructed to refer clinical questions or concerns to the clinician.
At post-treatment outcomes for both treatment groups were superior to the control group and there were no differences between the two treatment groups in either clinical outcomes or acceptability. ESs within each treatment group on both depression measures was equal to or greater than to 1.2 indicating that the treatment effect was considerable. At post-treatment approximately 50% of participants in the treatment groups were classified as recovered and/or had achieved reliable clinical change, providing further evidence of clinical efficacy. Satisfaction with treatment was high, even though the amount of contact time per participant was approximately one hour. The total contact time required for each group of more than 40 participants was estimated at approximately 5 hours per week of clinician or technician time. The automated email system facilitated the efficiency of contact with participants and during treatment participants received more than 30 contacts, prompts, and reminders.
At follow-up this pattern of results was generally stable, although the technician-assisted group continued to make improvements. At follow-up, the technician-assisted had significantly lower PHQ-9 and K-10 scores than the clinician-assisted group. At follow-up the between-group effect size for the technician-assisted group (1.89) on the PHQ-9 was considerably larger than for the clinician-assisted group (1.11).
These results support the study hypotheses and are consistent with recent findings indicating that less than one hour of therapist support during I-CBT for depression can produce excellent clinical outcomes . The present results also provide evidence that a technician, when supervised by a clinician, can produce results similar to those obtained by mental health professionals in the treatment of depression. Similar outcomes have been reported in I-CBT programs for treating social phobia ,  and generalized anxiety disorder , providing evidence that, when supervised by a clinician, I-CBT programs may be effectively and safely administered by a non-clinician. The technician in the present study was able to “step-up” participants to the clinician, but did so with only 10% of participants. This indicates that the majority found the intervention by the technician sufficient for their needs, and demonstrates a potential model for integrating clinician and technician support during iCBT programs. The magnitude of clinical effects observed in this study are consistent with those observed in previous evaluations of clinician-assisted Internet-based treatment of depression –. These results are also comparable to those observed in face-to-face treatment , but considerably larger than those observed in self-guided Internet-based treatment programs for depression , .
An unexpected finding was that the technician-assisted group continued to make improvements between post-treatment and follow-up, while the clinician-guided group did not improve and deteriorated on one of the primary outcome measures. It is difficult to account for these findings as pre-treatment expectancies and post-treatment satisfaction did not differ between intervention groups, and no between-group differences were found in the total number of contacts with participants or in the time spent on each participant. Replication of this research design and detailed analysis of the messages sent by the clinician and technician will be conducted to explore the reliability of this finding and possible reasons for differences.
The relatively small sample size is one limitation of this study. Other possible limitations include that the TA group had fewer males than the other groups, and that the CA group had access to more resources than the TA group. However, there is no indication that these issues affected results. A fourth limitation is the self-selecting nature of the sample. Those volunteering may well be inherently motivated, but this does not challenge the internal validity of the results. Another important potential limitation is the use of a delayed treatment control group rather than an attention-control placebo. This choice was grounded in concerns about the impact of raised expectations of symptom resolution in depressed participants placed in a placebo, attention-control condition, and the difficulty contacting Internet participants should additional help have been required.
This randomized controlled trial found no difference between a clinician- and technician-assisted Internet-based treatment program for depression at post-treatment. Both conditions resulted in large effect sizes, clinically significant improvements, and high levels of acceptability, while a delayed treatment control group did not improve. The technician-assisted group continued to improve at the 4 month follow-up, while the clinician-assisted group did not. These results demonstrate the potential of a new model of providing evidence-based treatment for depression. The question is not whether to accept such an innovative model of service delivery, but how to do so in an ethical, competent, safe, and cost-effective way, while also maintaining excellent clinical standards.
The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the VirtualClinic program by Mr Jason Hando, and Mr Daniel Winter.
Conceived and designed the experiments: NT GA ER. Performed the experiments: MD KM. Analyzed the data: NT. Wrote the paper: NT GA MD KM ER KS.
- 1. Andrews G, Titov N (2007) Depression is very disabling. Lancet 370: 808–809.G. AndrewsN. Titov2007Depression is very disabling.Lancet370808809
- 2. Andrews G, Henderson S, Hall W (2001) Prevalence, comorbidity, disability and service utilisation. Br J Psychiatry 178: 145–153.G. AndrewsS. HendersonW. Hall2001Prevalence, comorbidity, disability and service utilisation.Br J Psychiatry178145153
- 3. Australian Bureau of Statistics (2008) National Survey of Mental Health and Wellbeing 2007: summary of Results. Released 10/23/2008.Australian Bureau of Statistics2008National Survey of Mental Health and Wellbeing 2007: summary of Results. Released 10/23/2008.Available: http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/6AE6DA447F985FC2CA2574EA00122BD6/File/43260_2007.pdf. Accessed 2009 Nov 27. Available: http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/6AE6DA447F985FC2CA2574EA00122BD6/File/43260_2007.pdf. Accessed 2009 Nov 27.
- 4. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H (2004) Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders. Br J Psychiatry 184: 526–533.G. AndrewsC. IssakidisK. SandersonJ. CorryH. Lapsley2004Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders.Br J Psychiatry184526533
- 5. Marks I, Cavanagh K, Gega L (2007) Hands-on help: Computer-aided psychotherapy. New York: Psychology Press Taylor and Francis. 223 p.I. MarksK. CavanaghL. Gega2007Hands-on help: Computer-aided psychotherapy.New YorkPsychology Press Taylor and Francis223
- 6. Titov N (2007) Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 41: 95–114.N. Titov2007Status of computerized cognitive behavioural therapy for adults.Aust N Z J Psychiatry4195114
- 7. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D (2004) Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 34: 9–18.IM MarksM. KenwrightM. McDonoughM. WhittakerD. Mataix-Cols2004Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder.Psychol Med34918
- 8. Marks I, Cavanagh K (2009) Computer-aided psychological treatments: evolving issues. Annu Rev Clin Psychol 5: 121–141.I. MarksK. Cavanagh2009Computer-aided psychological treatments: evolving issues.Annu Rev Clin Psychol5121141
- 9. Andersson G, Cuijpers P (2009) Internet based and other computerized psychological treatments for adult depression: a meta-analysis. Cog Behav Ther 8: 196–205.G. AnderssonP. Cuijpers2009Internet based and other computerized psychological treatments for adult depression: a meta-analysis.Cog Behav Ther8196205
- 10. Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, et al. (2005) Internet-based self-help for depression: randomized controlled trial. Br J Psychiatry 187: 456–461.G. AnderssonJ. BergstromF. HollandareP. CarlbringV. Kaldo2005Internet-based self-help for depression: randomized controlled trial.Br J Psychiatry187456461
- 11. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, et al. (2010) Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. K. VernmarkJ. LenndinJ. BjärehedM. CarlssonJ. Karlsson2010Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression.Behav Res TherAvailable: doi: 10.1016/j.brat.2010.01.005. Accessed 2010 Mar 18. Available: doi: 10.1016/j.brat.2010.01.005. Accessed 2010 Mar 18.
- 12. Perini S, Titov N, Andrews G (2008) The Sadness program: an open trial of Internet-based treatment for depression. E- J Appl Psychol 4: 2.S. PeriniN. TitovG. Andrews2008The Sadness program: an open trial of Internet-based treatment for depression.E- J Appl Psychol42Available: http://ojs.lib.swin.edu.au/index.php/ejap/article/view/135/140. Accessed 2009 Nov 26. Available: http://ojs.lib.swin.edu.au/index.php/ejap/article/view/135/140. Accessed 2009 Nov 26.
- 13. Perini S, Titov N, Andrews G (2009) Clinician-assisted Internet-based treatment is effective for depression: a randomized controlled trial. Aust N Z J Psychiatry 43: 571–578.S. PeriniN. TitovG. Andrews2009Clinician-assisted Internet-based treatment is effective for depression: a randomized controlled trial.Aust N Z J Psychiatry43571578
- 14. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, et al. (2009) Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther 38: 66–82.P. CuijpersIM MarksA. van StratenK. CavanaghL. Gega2009Computer-aided psychotherapy for anxiety disorders: a meta-analytic review.Cogn Behav Ther386682
- 15. Titov N, Andrews G, Robinson E, Schwencke G, Johnston J, et al. (2009) Clinician-assisted Internet-based treatment is effective for generalized anxiety disorder: a randomized controlled trial. Aust N Z J Psychiatry 43: 905–912.N. TitovG. AndrewsE. RobinsonG. SchwenckeJ. Johnston2009Clinician-assisted Internet-based treatment is effective for generalized anxiety disorder: a randomized controlled trial.Aust N Z J Psychiatry43905912
- 16. Robinson E, Titov N, Andrews G, McIntyre K, Schwencke G, et al. Internet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance.E. RobinsonN. TitovG. AndrewsK. McIntyreG. SchwenckeInternet treatment for generalized anxiety disorder: a randomized controlled trial comparing clinician vs. technician assistance.PLos ONE, In press. PLos ONE, In press.
- 17. Klein B, Richards JC, Austin DW (2006) Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 37: 213–238.B. KleinJC RichardsDW Austin2006Efficacy of Internet therapy for panic disorder.J Behav Ther Exp Psychiatry37213238
- 18. Kiropoulos LA, Klein B, Austin DW, Gilson K, Pier C, et al. (2008) Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J Anxiety Disord 22: 1273–1284.LA KiropoulosB. KleinDW AustinK. GilsonC. Pier2008Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?J Anxiety Disord2212731284
- 19. Shandley K, Austin DW, Klein B, Pier C, Schattner P, et al. (2008) Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res 10(2): e14.K. ShandleyDW AustinB. KleinC. PierP. Schattner2008Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists?J Med Internet Res10(2)e14Available: http://www.jmir.org/2008/2/e14/. Accessed 2009 Nov 26. Available: http://www.jmir.org/2008/2/e14/. Accessed 2009 Nov 26.
- 20. Carlbring P, Nilsson-Ihrfelt E, Waara J, Kollenstam C, Buhrman M, et al. (2005) Treatment of panic disorder: live therapy vs. self-help via the Internet. Behav Res Ther 43: 1321–1333.P. CarlbringE. Nilsson-IhrfeltJ. WaaraC. KollenstamM. Buhrman2005Treatment of panic disorder: live therapy vs. self-help via the Internet.Behav Res Ther4313211333
- 21. Berger T, Hohl E, Caspar F (2009) Internet-based treatment for social phobia: a randomized controlled trial. J Clin Psychology 65: 1–15.T. BergerE. HohlF. Caspar2009Internet-based treatment for social phobia: a randomized controlled trial.J Clin Psychology65115
- 22. Titov N, Andrews G, Schwencke G, Drobny J, Einstein D (2008) Shyness 1: a randomized controlled trial of an internet-based treatment for social phobia. Aust N Z J Psychiatry 42: 585–594.N. TitovG. AndrewsG. SchwenckeJ. DrobnyD. Einstein2008Shyness 1: a randomized controlled trial of an internet-based treatment for social phobia.Aust N Z J Psychiatry42585594
- 23. Titov N, Andrews G, Schwencke G (2008) Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 42: 595–605.N. TitovG. AndrewsG. Schwencke2008Shyness 2: treating social phobia online: replication and extension.Aust N Z J Psychiatry42595605
- 24. Titov N, Andrews G, Choi I, Schwencke G, Mahoney A (2009) Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 42: 1030–1040.N. TitovG. AndrewsI. ChoiG. SchwenckeA. Mahoney2009Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia.Aust N Z J Psychiatry4210301040
- 25. Andersson G, Carlbring P, Holmstrom A, Sparthan E, Furmark T, et al. (2006) Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial. J Consult Clin Psychol 74: 677–686.G. AnderssonP. CarlbringA. HolmstromE. SparthanT. Furmark2006Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomized controlled trial.J Consult Clin Psychol74677686
- 26. Carlbring P, Gunnarsdóttir M, Hedensjö L, Andersson G, Ekselius L, et al. (2007) Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 190: 123–128.P. CarlbringM. GunnarsdóttirL. HedensjöG. AnderssonL. Ekselius2007Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support.Br J Psychiatry190123128
- 27. Titov N, Andrews G, Johnston L, Schwencke G, Choi I (2009) Shyness programme: longer term benefits, cost-effectiveness, and acceptability. Aust N Z J Psychiatry 43: 36–44.N. TitovG. AndrewsL. JohnstonG. SchwenckeI. Choi2009Shyness programme: longer term benefits, cost-effectiveness, and acceptability.Aust N Z J Psychiatry433644
- 28. Titov N, Andrews G, Choi I, Schwencke G, Johnston L (2009) Internet-based cognitive behavior therapy for social phobia without clinical input is effective: a pragmatic RCT of two types of reminders. Aust N Z J Psychiatry 43: 913–919.N. TitovG. AndrewsI. ChoiG. SchwenckeL. Johnston2009Internet-based cognitive behavior therapy for social phobia without clinical input is effective: a pragmatic RCT of two types of reminders.Aust N Z J Psychiatry43913919
- 29. Titov N, Andrews G, Schwencke G, Solley K, Johnston L, et al. (2009) An RCT comparing two types of support on severity of symptoms for people completing Internet-based cognitive behavior therapy for social phobia. Aust N Z J Psychiatry 43: 920–926.N. TitovG. AndrewsG. SchwenckeK. SolleyL. Johnston2009An RCT comparing two types of support on severity of symptoms for people completing Internet-based cognitive behavior therapy for social phobia.Aust N Z J Psychiatry43920926
- 30. Moher D, Schulz KF, Altman DG, for the CONSORT group (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. JAMA 285: 1987–1991.D. MoherKF SchulzDG Altmanfor the CONSORT group2001The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials.JAMA28519871991
- 31. American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (4th ed.). Washington, DC: American Psychiatric Association. 886 p.American Psychiatric Association1994Diagnostic and Statistical Manual of Mental Disorders (4th ed.).Washington, DCAmerican Psychiatric Association886
- 32. Kroenke K, Spitzer RL, Williams JBW (2001) The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16: 606–613.K. KroenkeRL SpitzerJBW Williams2001The PHQ-9: validity of a brief depression severity measure.J Gen Intern Med16606613
- 33. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, et al. (1998) The Mini-International Neuropsychiatric Interview (M.I.N.I.): the Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10. J Clin Psychiatry 59(S20): 22–33.DV SheehanY. LecrubierKH SheehanP. AmorimJ. Janavs1998The Mini-International Neuropsychiatric Interview (M.I.N.I.): the Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10.J Clin Psychiatry59(S20)2233
- 34. Beck AT, Steer RA, Brown GK (1996) BDI-II: Beck Depression Inventory Manual. 2nd ed. San Antonio, TX: Psychological Corp. AT BeckRA SteerGK Brown1996BDI-II: Beck Depression Inventory Manual. 2nd ed.San Antonio, TXPsychological Corp
- 35. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, et al. (2002) Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 32: 959–976.RC KesslerG. AndrewsLJ ColpeE. HiripiDK Mroczek2002Short screening scales to monitor population prevalences and trends in non-specific psychological distress.Psychol Med32959976
- 36. Sheehan DV (1983) The Anxiety Disease. New York: Scribner. 151 p.DV Sheehan1983The Anxiety Disease.New YorkScribner151
- 37. Devilly GJ, Borkovec TD (2000) Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 31: 73–86.GJ DevillyTD Borkovec2000Psychometric properties of the credibility/expectancy questionnaire.J Behav Ther Exp Psychiatry317386
- 38. Borkovec TD, Nau SD (1972) Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry 3: 257–260.TD BorkovecSD Nau1972Credibility of analogue therapy rationales.J Behav Ther Exp Psychiatry3257260
- 39. Löwe B, Unützer J, Callahan CM, Perkins AJ, Kroenke K (2004) Monitoring depression treatment outcomes with the Patient Health Questionnaire-9. Med Care 42: 1194–1201.B. LöweJ. UnützerCM CallahanAJ PerkinsK. Kroenke2004Monitoring depression treatment outcomes with the Patient Health Questionnaire-9.Med Care4211941201
- 40. Garb HN (2007) Computer-administered interviews and rating scales. Psychol Assess 19: 4–13.HN Garb2007Computer-administered interviews and rating scales.Psychol Assess19413
- 41. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, et al. (2007) Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput Human Behav 23: 1421–1434.P. CarlbringS. BruntS. BohmanD. AustinJ. Richards2007Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research.Comput Human Behav2314211434
- 42. Julious SA (2004) Tutorial in biostatistics: samples sizes for clinical trials with normal data. Stat Med 23: 1921–1986.SA Julious2004Tutorial in biostatistics: samples sizes for clinical trials with normal data.Stat Med2319211986
- 43. Vickers AJ (2005) Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med Res Methodol 5: 35.AJ Vickers2005Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data.BMC Med Res Methodol535
- 44. Vickers AJ (2005) Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom Med 67: 652–655.AJ Vickers2005Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches.Psychosom Med67652655
- 45. Richards DA, Suckling R (2009) Improving access to psychological therapies: Phase IV prospective cohort study. Br J Clin Psychol 48: 377–396.DA RichardsR. Suckling2009Improving access to psychological therapies: Phase IV prospective cohort study.Br J Clin Psychol48377396
- 46. Jacobson NS, Truax P (1999) Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 59: 12–19.NS JacobsonP. Truax1999Clinical significance: a statistical approach to defining meaningful change in psychotherapy research.J Consult Clin Psychol591219
- 47. Learmonth D, Trosh J, Rai S, Sewell J, Cavanagh K (2008) The role of computer-aided psychotherapy within an NHS CBT specialist service. Counsel Psychother Res 8: 117–123.D. LearmonthJ. TroshS. RaiJ. SewellK. Cavanagh2008The role of computer-aided psychotherapy within an NHS CBT specialist service.Counsel Psychother Res8117123
- 48. Haby MM, Donnelly M, Corry J, Vos T (2005) Cognitive behavioural therapy for depression, panic disorder, and generalized anxiety disorder: a meta-regression of factors that may predict outcome. Aust N Z J Psychiatry 40: 9–19.MM HabyM. DonnellyJ. CorryT. Vos2005Cognitive behavioural therapy for depression, panic disorder, and generalized anxiety disorder: a meta-regression of factors that may predict outcome.Aust N Z J Psychiatry40919
- 49. Clarke G, Reid Ed, Eubanks D, O'Connor E, DeBar LL, et al. (2002) Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 4(3): e14.G. ClarkeEd ReidD. EubanksE. O'ConnorLL DeBar2002Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program.J Med Internet Res4(3)e14Available: http://www.jmir.org/2002/3/e14/. Accessed 2009 Nov 30. Available: http://www.jmir.org/2002/3/e14/. Accessed 2009 Nov 30.
- 50. Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, et al. (2005) Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 7(2): e16.G. ClarkeD. EubanksE. ReidC. KelleherE. O'Connor2005Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders.J Med Internet Res7(2)e16Available: http://www.jmir.org/2005/2/e16/. Accessed 2009 Nov 30. Available: http://www.jmir.org/2005/2/e16/. Accessed 2009 Nov 30.