Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Direct versus Indirect Treatment for Preschool Children who Stutter: The RESTART Randomized Trial

  • Caroline de Sonneville-Koedoot ,

    desonneville@bmg.eur.nl

    Affiliations Institute of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, the Netherlands, Speech and Hearing Center, Department of Otorhinolaryngology, Sophia Children’s Hospital, Erasmus University Medical Center, Rotterdam, the Netherlands

  • Elly Stolk,

    Affiliation Institute of Health Policy and Management, Erasmus University Rotterdam, Rotterdam, the Netherlands

  • Toni Rietveld,

    Affiliation Department of Linguistics, Radboud University Nijmegen, Nijmegen, the Netherlands

  • Marie-Christine Franken

    Affiliation Speech and Hearing Center, Department of Otorhinolaryngology, Sophia Children’s Hospital, Erasmus University Medical Center, Rotterdam, the Netherlands

Abstract

Objective

Stuttering is a common childhood disorder. There is limited high quality evidence regarding options for best treatment. The aim of the study was to compare the effectiveness of direct treatment with indirect treatment in preschool children who stutter.

Methods

In this multicenter randomized controlled trial with an 18 month follow-up, preschool children who stutter who were referred for treatment were randomized to direct treatment (Lidcombe Program; n = 99) or indirect treatment (RESTART-DCM treatment; n = 100). Main inclusion criteria were age 3–6 years, ≥3% syllables stuttered (%SS), and time since onset ≥6 months. The primary outcome was the percentage of non-stuttering children at 18 months. Secondary outcomes included stuttering frequency (%SS), stuttering severity ratings by the parents and therapist, severity rating by the child, health-related quality of life, emotional and behavioral problems, and speech attitude.

Results

Percentage of non-stuttering children for direct treatment was 76.5% (65/85) versus 71.4% (65/91) for indirect treatment (Odds Ratio (OR), 0.6; 95% CI, 0.1–2.4, p = .42). At 3 months, children treated by direct treatment showed a greater decline in %SS (significant interaction time x therapy: β = -1.89; t(282.82) = -2.807, p = .005). At 18 months, stuttering frequency was 1.2% (SD 2.1) for direct treatment and 1.5% (SD 2.1) for indirect treatment. Direct treatment had slightly better scores on most other secondary outcome measures, but no differences between treatment approaches were significant.

Conclusions

Direct treatment decreased stuttering more quickly during the first three months of treatment. At 18 months, however, clinical outcomes for direct and indirect treatment were comparable. These results imply that at 18 months post treatment onset, both treatments are roughly equal in treating developmental stuttering in ways that surpass expectations of natural recovery. Follow-up data are needed to confirm these findings in the longer term.

Trial Registration

isrctn.org ISRCTN24362190

Introduction

Developmental stuttering is a prevalent childhood disorder. The incidence rate is 5 to 11% in preschool years [1,2]. The cause of stuttering is unknown, although recent research indicates that structural and functional brain anomalies underlie the disorder [36], with a strong genetic involvement [710]. Many children recover spontaneously; about 63% at 3 years post onset [1,11]. Knowledge of factors that favor the chance for recovery [11,12] can help pediatricians and speech-language pathologists (SLPs) to identify children at risk for chronic stuttering [13]. Nevertheless, the chance for recovery cannot be predicted for an individual child. Since chances for full recovery diminish when stuttering has been present for 15 months [14] and persistent stuttering in adolescents and adults can have a serious mental and social impact [1517], treatment is generally recommended to start before the age of 6 years [2,18]. However, the evidence base for the effectiveness of current therapies for preschool children who stutter is surprisingly weak as well as unbalanced in terms of published reports [19].

For about three decades, many preschool children who stutter around the world have been treated according to an indirect, multifactorial treatment approach, like treatment based on the Demands and Capacities Model (DCM) [20,21]. This approach aims to decrease demands set by the environment (e.g., parents are trained to slow down their habitual speech rate) and the child him- or herself (e.g., desensitization for disfluency), and increase the child’s capacities for speaking fluently (e.g., accurate and smooth speech motor movements that are age-appropriate) to arrive at a favorable balance between demands and capacities, eventually resulting in fluent speech. Since 2000, an increasing number of children have been treated according to a direct operant treatment approach: the Lidcombe Program (LP) for early intervention [22,23]. This direct approach teaches parents to give verbal contingencies after fluent and stuttered speech. With the limited data available at present, the direct LP offers the best evidence-based intervention for preschool children who stutter [19]. However, the long-term effectiveness of this treatment is still unclear [24]. More importantly, comparative effectiveness to current standard treatment has not yet been established; yet child health policy-makers, pediatricians and SLPs need this information to decide upon reimbursement and treatment choice. This is for instance illustrated by a recent proposal of the national speech-language pathology association of Australia (Speech Pathology Australia) to only fund treatment by the LP [25]. Therefore, the aim of the current study was to compare the effectiveness of direct versus indirect stuttering treatment in preschool children during an 18 month follow-up.

Methods

Study design, participants and setting

This parallel group randomized trial named RESTART (the Rotterdam Evaluation Study of Stuttering Therapy in preschool children- a Randomized Trial) included 199 preschool children who stutter, who were registered at one of the 20 participating speech clinics (including 24 SLPs) throughout the Netherlands. Eligible participants were children (1) aged 3.0–6.3 years, (2) with a stuttering severity rating ≥ 2 (‘mild’) on an 8-point scale [11] provided by the parent (3) and by the clinician, (4) who stuttered ≥ 3% of syllables and (5) for at least 6 months. The inclusion criterion of at least 3% syllables stuttered (SS) had replaced the original criterion of ‘at least 3.3% Stuttering Like Disfluencies (SLD)’ shortly before the start of the trial. This was based on critics on the SLD measure in literature and on the results of a study into the validity of the SLD measure that we conducted at our center. Exclusion criteria were a diagnosis of an emotional, behavioral, learning or neurological disorder, or a lack of proficiency in Dutch for children or parents. The exclusion criterion of having received treatment for stuttering during the past year was omitted after 5 months, since it was noticed that by excluding these children, the external validity would be restricted. All SLPs were trained and experienced in both treatments. DCM based treatment training is included in the regular clinical education in the Netherlands, and all but one SLP had additionally been trained in the assessment and treatment of children who stutter to become a certified fluency expert recognized by the Dutch association of stuttering therapy (NVST). To ensure a uniform application of DCM based treatment, a treatment manual was developed in collaboration with all participating clinicians prior to the start of the trial. In addition, all SLPs had gone through a three day LP course taught by a LP Consortium trainer and had been certified to provide LP therapy. They had on average 15 years of experience with DCM based treatment (range 7–21 years) and 3.7 years with the LP (range 1.5–7.6 years). Therapists’ fidelity to treatment was monitored in 3-monthly intervision meetings, regular telephone contacts with the research team, and by registration forms on the content and amount of treatment filled in by the SLPs and checked by the research team. The intervision meetings were chaired by a LP consortium trainer and a DCM trainer. The trial was approved by the Ethics Committee of the Erasmus MC and registered at isrctn.org (ISRCTN24362190). Written informed consent was obtained from all parents. The trial protocol and supporting CONSORT checklist are available as supporting information: see S1 CONSORT Checklist and S1 Protocol.

Interventions

Direct treatment: The Lidcombe Program.

The Lidcombe Program (LP) is a behavioral treatment based on the premise that stuttering is an operant behavior that can be targeted by contingencies. The LP is administered by parents under the direction of a clinician. Children allocated to the LP were treated according to the LP manual [22]. Parents were trained to deliver verbal contingencies in conversations with their child (e.g., “That was smooth” or “Were there any bumpy words?”) in a 5:1 ratio for stutter-free and stuttered speech. During the first stage of the program, the parent delivered contingencies during structured conversations of 10–15 minutes once or twice a day. The speech clinic was attended once a week. This continued until stuttering either disappeared or reached an extremely low level (≤1% of syllables stuttered). During the second stage, the use of verbal contingencies as well as the number of clinic visits was gradually reduced, provided that fluency was maintained.

Indirect treatment: The RESTART Demands and Capacities Model based treatment.

RESTART Demands and Capacities Model based treatment (RESTART-DCM) is premised on the idea that positive changes in the child’s functioning and/or in the environment will lead to a reduction of stuttering. Following the RESTART-DCM manual [26], parents were trained to decrease relevant motoric, linguistic, emotional or cognitive demands, thereby reduce communicative pressure on the child (e.g., parents slowing down their habitual speech rate). If deemed necessary, the child’s capacities for fluency were subsequently trained (e.g., improving the child’s speech motor movements or word-finding capacity). Parents were required to give their child their undivided attention and practice home assignments 15 minutes a day, for a minimum of 5 days a week. Treatment was gradually reduced if the child showed acceptable speech, parents had mastered implementing a fluency enhancing environment and knew what to do if a relapse occurred.

Randomization and blinding

A minimization software program (MINIM2) [27] was used by the principal investigator (CdeS) to allocate children to one of the treatment arms, according to factors known or thought of to be related to treatment outcome [28]: gender, stuttering severity in the clinic (based on the SSI-3 score) [29], time since onset (TSO; 6–12, 13–18, 19+ months), a first, second, or third degree relative with persistent stuttering (yes, no) and/or a history of recovered stuttering (yes, no), stuttering treatment during the past 12 months (yes, no), and SLP. Three stuttering severity categories were distinguished: (1) mild (SSI-3 score: 10–16); (2) moderate (SSI-3 score: 17–26); severe (SSI-3 score: 27+). For each participant, treatment allocation depended on the characteristics of the children already enrolled [28]. Judges of stuttering frequency were blinded to treatment allocation and measurement moment.

Outcome assessment

The primary outcome measure was the percentage of non-stuttering children at 18 months, operationalized as ≤1.5% syllables stuttered (SS). This criterion was obtained by applying a conversion ratio of 1.15 to the mean percentage of stuttered word disfluencies in children who do not stutter [30,31]. Parents were requested to make three audio recordings of 10–15 minutes each in a period of two weeks: one sample of their child speaking to a parent at home, one to a non-family member at home and one to a non-family member away from home [3234].

Secondary outcome measures assessed at baseline, and at 3, 6, 12 and 18 months after start of treatment, were the frequency of stuttering (%SS), a severity rating of stuttering by the parent on an 8-point scale[11], and parents´ valuation of their child´s health-related quality of life on a proxy version of the EuroQoL EQ-VAS [35] with anchor points 0 (worst imaginable health) and 100 (best imaginable health). Secondary outcome measures assessed at baseline and 18 months were the speech attitude of the child (KiddyCAT) [36] and emotional and behavioral problems measured by the Child Behavior Checklist (CBCL) [37]. The latter consists of the scales Internalizing (anxiety, depression, withdrawal, and somatic complaints), Externalizing (aggressive and delinquent behavior), and Total problem behavior [37]. At 18 months both the SLP and the child provided a stuttering severity rating: the SLP on an 8-point scale [11], the child on a 4-point scale where 1 = I do not stutter anymore and 4 = I stutter a lot.

Eight SLPs not involved in the study were trained to determine the %SS of the samples in real time with sufficient intrajudge reliability, using an electronic, button press counter. To ensure sufficient interjudge reliability, 64% of all samples were scored by at least two raters. Disagreements in ratings were discussed and a third, blinded senior rater was consulted in rare cases where no agreement could be reached (cf. Boberg & Kully [38]).

Statistical analysis

An a priori power calculation to detect a difference of 15% in percentage of non-stuttering children (80 versus 95%) with a power of 80% in a 2-tailed test at a significance level of .05 and allowing a 22% drop-out rate, resulted in a sample size of 98 in each group. Baseline factors were characterized as medians, means and standard deviations for continuous variables and as frequency distributions for categorical variables. Baseline comparisons between treatment groups and between survivors and drop-outs were assessed using χ2 tests and independent t-tests. Participants were analyzed in the group to which they were randomized.

The effect of treatment on the primary outcome measure was analyzed by χ2 tests and logistic regression analysis (ENTER method). The regression analysis included the main effect of therapy and the interaction terms therapy*age in years, therapy*stuttering severity (SSI-3 score), and therapy*TSO. Confidence intervals around the obtained percentages of children classified as non-stuttering were calculated according to the method of Wilson [39,40], using a website calculator (http://www.vassarstats.net/prop1.html). In a sensitivity analysis, cut-off scores of 1% SS and 2% SS were applied to further assess the robustness of the primary outcome.

For the secondary outcomes assessed at all measurement moments (%SS, parental rating of stuttering severity, and EQ-VAS) and at baseline and 18 months (KiddyCAT and CBCL), we applied a longitudinal repeated-measures mixed effects model with random intercepts, assuming missing at random. Participant was included as a random predictor; fixed predictors were therapy, and 4 cross-products as interaction terms: time*therapy, and time*therapy*age, severity, and TSO, respectively. An unstructured covariance matrix was assumed for the error as a more plausible autoregressive covariance structure did not provide a better fit. This approach was also used at level 2 of the model. Since the data on %SS did not meet the assumptions needed to calculate CIs for the intraclass correlation coefficient (ICC), interjudge reliability of the speech samples was assessed using Krippendorff's alpha [41] with the option 'interval data' for the macro developed by Hayes (2013) [42]. For the outcome %SS, an additional analysis was conducted into the progression in the first 3 months. CBCL outcomes at 18 months were analyzed separately using ANOVA-analysis. Secondary outcome measures only assessed at 18 months (severity ratings by clinician and child) were compared by independent t-tests. For all secondary outcomes, unadjusted and Holm-adjusted [43] p-values are presented, using an overall level of significance of α = .05 (2-sided). The Holm’s correction is generally considered a good alternative to the conservative Bonferroni approach [44]. Each pj is compared to α/(n-j+1); that is: the smallest pj (j = 1) is compared to α/n, the next smallest to α/(n-1) etc.

Treatment intensity was compared by independent t-test, and a χ2 test was conducted to compare the number of children on treatment at the endpoint of the trial. For analysis of the questionnaires, instructions offered in the manuals were followed. All analyses were carried out in SPSS 20 (Armonk, NY: IBM Corp.).

Results

Participants

Children were enrolled between September 2007 and June 2010. Of 615 children referred for treatment, 416 were not eligible for various reasons (Fig 1). In total 199 children met the inclusion criteria. One child was found ineligible after inclusion and therefore excluded from all analyses (Fig 1). Baseline characteristics did not differ between treatment groups (Table 1). In the LP group 12 children were lost to follow-up as compared to 9 children in the RESTART-DCM group (n = 21, 11% drop out rate). Children who were lost to follow-up did not significantly differ on any baseline characteristics (age, gender, ethnicity, educational level of parent, SSI-3 score, %SS, TSO, parental ratings, stuttering in family, prior treatment for stuttering) from children who completed the trial (p-values ranging from .11 to .91). For 191 children, at least one outcome measurement after the start of treatment was available.

thumbnail
Table 1. Baseline Characteristics of Participants by Treatment Group.

https://doi.org/10.1371/journal.pone.0133758.t001

Speech samples

The mean number of available audio samples for a child at a measurement moment was 2.9 (range 1–6). At least 85% of all samples had a length of ≥300 syllables. The mean intrajudge reliability [45] of measurement of %SS was 83%. Krippendorff’s alpha for samples with 2 ratings at baseline and after 3, 6, 12 and 18 months, respectively, was 0.849, 0.896, 0.817, 0.795, and 0.830; all significant, with significance obtained by bootstrapping. All scores represent good reliability [46].

Primary outcome

At 18 months, audiotapes were available for 173 children. For 1 child in the LP and 2 in the RESTART-DCM group audiotapes were missing and replaced by videotapes made in the clinic. For 1 child in the LP group, both audio and videotapes were lacking. Thus, the final analysis at 18 months was based on 176 children. In total, 76.5% (65/85; 95%CI: 66.4–84.2) of children in the LP group were classified as non-stuttering at 18 months compared to 71.4% (65/91; 95%CI: 61.4–79.7) of children in the RESTART-DCM group. This difference was statistically non-significant (χ2(1) = 0.579, p = .45). Nor did logistic regression analysis indicate therapy or other factors as significant predictors of being classified as non-stuttering (therapy: OR, 0.6; 95% CI, 0.1–2.4; p = .42; Table 2). Applying cut-off criteria of 1% SS and 2% SS did not significantly affect the results.

thumbnail
Table 2. Primary and Secondary Outcomes at Baseline and 18-month Follow Up.

https://doi.org/10.1371/journal.pone.0133758.t002

Secondary outcomes

The results for all secondary outcome measures at baseline and 18 months and the results for the mixed model analyses are presented in Table 2. For the outcome %SS, the effect of therapy type was non-significant. However, a significant interaction between time and therapy type was detected (adjusted p = .008), indicating that the %SS differed for therapy groups at different time points. The effect of time was also significant (adjusted p = .002), indicating that in both treatment groups the average %SS decreased significantly over time. Effect sizes were small (Table 2).

Fig 2 shows that in both groups most improvement in %SS occurred in the first 3 months of therapy. For this interval, an effect of therapy type was found (β = 2.30; t(217.38) = 2.10, p = .04), as well as a significant interaction between time and therapy type (β = -1.89; t(282.82) = -2.81, p = .005). Compared to the RESTART-DCM group, the LP group had a slightly higher mean %SS at baseline and showed a greater decline, resulting in a lower %SS at 3 months. Significant interactions with very small effect sizes were also present between time, therapy type, and stuttering severity (β = 0.25; t(173.94) = 2.51, adjusted p = .01) and time, therapy type, and TSO (β = -0.21; t(172.85) = 2.40, adjusted p = .02) (Fig 3).

thumbnail
Fig 2. Change in Three Secondary Outcome Measures During 18-month Follow Up.

https://doi.org/10.1371/journal.pone.0133758.g002

For the outcome parental rating of stuttering severity, a significant effect of time (adjusted p< .001) as well as a significant interaction between time and therapy type (adjusted p< .001) was detected. Fig 2 shows a slightly greater decline in scores for the LP group over the period of 18 months. The interaction between time, therapy type and age was significant (adjusted p< .001) but showed a very small effect size (Table 2). For the outcomes EQ-VAS and KiddyCAT, no significant effect of therapy type or any other factor was found (Table 2; Fig 2).

For all CBCL scale scores, the factor therapy type was significant (Table 2), but this effect was attributable to significantly higher scores for the LP group at baseline. At 18 months, no significant differences were found (Internal scale: F(1,196) = -1.04, unadjusted p = .32, partial eta squared = .006; External scale: F(1,196) = 1.04, unadjusted p = .31, partial eta squared = .006; Total problem scale: F(1,196) = 1.12, unadjusted p = .29, partial eta squared = .006). For the CBCL External scale, a significant interaction with a small effect size was established between time, therapy type and age: older children showed a greater decline in score, particularly in the LP group.

For the severity rating by the clinician as well as by the child at 18 months, significant interactions between therapy type and age were established (Clinician: adjusted p = .01; Child: adjusted p = .01). However the small eta-squared values (0.079 and 0.088, respectively) suggest that these differences are negligible.

Treatment intensity

The number of treatment sessions and treatment hours did not differ significantly between groups (Table 3). At 18 months, 27.6% (27/99) children in the LP group compared to 35.0% (35/100) children in the RESTART-DCM group were still on treatment, a difference that was also not statistically significant (χ2(1) = 1.277, p = .26).

Discussion

The RESTART-trial found that both direct and indirect treatment for preschool children who stutter reduced stuttering during 18 months of follow-up. The direct approach reduced stuttering frequency more quickly during the first three months of treatment, however, the difference was not significant anymore by 18 months. Most outcome measures were slightly in favor of the direct approach (LP), but the few significant interaction terms were deemed negligible due to their small effect sizes. For most children, stuttering frequency plateaued after three months, while about 30% of children were still on treatment after 18 months.

The direct LP and indirect RESTART-DCM treatment are based on different premises and assumptions regarding mechanisms underlying treatment effect (i.e., delivering verbal contingencies versus balancing demands and capacities for fluent speech, respectively). However, since results for both treatments were comparable, it could be hypothesized that their common components have a larger influence on recovery than their unique components (cf. Imel & Wampold [47]). In psychotherapy and counseling, this is known as the “dodo bird phenomenon” [48]. According to this hypothesis, treatments that are intended to be therapeutic are equally efficacious. Studies suggest that 30–70% of therapy outcome can be attributed to common factors, including good therapeutic relationships [47]. Unfortunately, little is known about the unique mechanisms that lead to change in stuttering behavior in both treatments [4951]. Common components of the LP and RESTART-DCM treatment may include consideration of maintaining factors, an increase in one-on-one time that parents spend with their child, a boost of encouragement and a reduction of linguistic demands for the child [52], and emotional support for the parents.

Our results do not enable us to distinguish the potential effect of treatment from spontaneous recovery. Spontaneous recovery in the general population at 36 months post onset has been estimated to be 63% or higher [11]. An estimate of the mean time since onset of stuttering at the endpoint in our study is 33 months. Thus, our percentages of children classified as non-stuttering exceed this estimate by about 10% (p = .02; based on statistical test for comparing two proportions from different populations). Furthermore, the chance of spontaneous recovery in our clinical study population is likely to be lower than in the general population, because this chance is known to diminish after 12 to 18 months [11,14] and 56% of children within our study stuttered for at least 12 months.

Strengths of our study are the large sample size with minimal loss to follow-up, the broad range of outcome measures, the large number of measurement moments, and the relatively long follow-up period (double the time in Jones et al. [33]). Participating therapists in the RESTART-study worked in usual-care centers throughout the Netherlands. Thus, the treatments were studied in a variety of regular clinical settings with therapists unconnected to the developers of the therapies [50,53], therefore employing a practical study design ensuring a high external validity. A limitation of our study is that a high number of children appeared ineligible for participation. Results may therefore not be fully generalizable to all preschool children presenting to a clinic with stuttering. Another limitation is that the applied follow-up time is insufficient to decide conclusively whether a child has recovered from stuttering. This requires a period of about 5 years [11,54], to account for the possibility of a relapse. Therefore, we intend to follow-up all children under study.

Conclusions

At 18 month post treatment onset, the evidence suggests that both direct and indirect treatment for stuttering can be recommended. However, direct treatment decreased stuttering more quickly during the first three months. Future research investigating the role of client and clinician factors, the effectiveness of a combined direct and indirect approach, and the cost-effectiveness of a limitation of treatment time or frequency may shed further light on the effectiveness of stuttering treatment in preschool children.

Supporting Information

S1 CONSORT Checklist. CONSORT 2010 Checklist of Information to Include When Reporting a Randomized Trial.

https://doi.org/10.1371/journal.pone.0133758.s001

(PDF)

S1 Protocol. Study protocol approved by the Medical Ethical Committee.

https://doi.org/10.1371/journal.pone.0133758.s002

(PDF)

Acknowledgments

We would like to thank all children and parents who participated in the RESTART-study. Furthermore, we would like to thank all participating SLPs: Jeanette van Baarsen, Patricia Blokker, Esther Bunschoten, Anneke Busser, Mary de Boer, Hannie Boevink, Karin Derks, Anne van Eupen, Alies Herweijer, Eeuwkje Kraak, Ellen Laroes, Caroline Nater, Brunette van der Neut, Mark Pertijs MA, Durdana Putker, Fine Schillevoort, Irma Uijterlinde, Lisette van der Velpen, Meina du Pui, Liesbeth van Wijngaarden, Corrie Witte, and Brenda Zwinkels. Lastly, we gratefully thank Marjon Bremer, Veronica Coppens, Wendy van Dongen, Maaike Elderhorst-Bas, Sandra van der Ende, Yoranique Klijnstra MA, Robine Stenders-van der Kuil, and Judith Polman MA, for scoring all audiosamples.

Author Contributions

Conceived and designed the experiments: MCF ES. Performed the experiments: CdeS MCF ES. Analyzed the data: CdeS TR. Wrote the paper: CdeS ES TR MCF.

References

  1. 1. Månsson H. Childhood stuttering: Incidence and development. J Fluency Disord 2000 0;25(1):47–57.
  2. 2. Reilly S, Onslow M, Packman A, Cini E, Conway L, Ukoumunne OC, et al. Natural history of stuttering to 4 years of age: a prospective community-based study. Pediatrics 2013 Sep;132(3):460–467. pmid:23979093
  3. 3. Chang SE, Zhu DC. Neural network connectivity differences in children who stutter. Brain 2013 Oct 16.
  4. 4. Sommer M, Koch MA, Paulus W, Weiller C, Buchel C. Disconnection of speech-relevant brain areas in persistent developmental stuttering. Lancet 2002 Aug 3;360(9330):380–383. pmid:12241779
  5. 5. Chang SE, Erickson KI, Ambrose NG, Hasegawa-Johnson MA, Ludlow CL. Brain anatomy differences in childhood stuttering. Neuroimage 2008 Feb 1;39(3):1333–1344. pmid:18023366
  6. 6. Cykowski MD, Fox PT, Ingham RJ, Ingham JC, Robin DA. A study of the reproducibility and etiology of diffusion anisotropy differences in developmental stuttering: a potential role for impaired myelination. Neuroimage 2010 Oct 1;52(4):1495–1504. pmid:20471482
  7. 7. Kang C, Riazuddin S, Mundorff J, Krasnewich D, Friedman P, Mullikin JC, et al. Mutations in the lysosomal enzyme-targeting pathway and persistent stuttering. N Engl J Med 2010 Feb 25;362(8):677–685. pmid:20147709
  8. 8. Rautakoski P, Hannus T, Simberg S, Sandnabba NK, Santtila P. Genetic and environmental effects on stuttering: a twin study from Finland. J Fluency Disord 2012 Sep;37(3):202–210. pmid:22682321
  9. 9. Yairi E, Ambrose N. Epidemiology of stuttering: 21st century advances. J Fluency Disord 2013 Jun;38(2):66–87. pmid:23773662
  10. 10. Felsenfeld S, Kirk KM, Zhu G, Statham DJ, Neale MC, Martin NG. A study of the genetic and environmental etiology of stuttering in a selected twin sample. Behav Genet 2000 Sep;30(5):359–366. pmid:11235981
  11. 11. Yairi E, Ambrose NG. Early Childhood Stuttering: For Clinicians by Clinicians (Austin, TX: Pro-Ed). 2005.
  12. 12. Yairi E, Ambrose NG, Paden EP, Throneburg RN. Predictive factors of persistence and recovery: pathways of childhood stuttering. J Commun Disord 1996 Jan-Feb;29(1):51–77. pmid:8722529
  13. 13. Guitar B, Conture EG. The child who stutters: to the pediatrician. 5th edition. Publication no. 0023 [Internet]. 2013 [cited 2014 November 16]. Memphis, TN: Stuttering Foundation of America. Available: http://www.stutteringhelp.org
  14. 14. Ingham RJ, Cordes AK. Treatment decisions for young children who stutter: Further concerns and complexities. Am J Speech Lang Pathol 1998;7(3):10.
  15. 15. Craig A, Blumgart E, Tran Y. The impact of stuttering on the quality of life in adults who stutter. J Fluency Disord 2009 Jun;34(2):61–71. pmid:19686883
  16. 16. Koedoot C, Bouwmans C, Franken MC, Stolk E. Quality of life in adults who stutter. J Commun Disord 2011 Mar 27.
  17. 17. Menzies RG, Onslow M, Packman A, O'Brian S. Cognitive behavior therapy for adults who stutter: a tutorial for speech-language pathologists. J Fluency Disord 2009 Sep;34(3):187–200. pmid:19948272
  18. 18. O'Brian S, Onslow M. Clinical management of stuttering in children and adults. BMJ 2011 Jun 24;342:d3742. pmid:21705407
  19. 19. Nye C, Vanryckeghem M, Schwartz JB, Herder C, Turner HM 3rd, Howard C. Behavioral stuttering interventions for children and adolescents: a systematic review and meta-analysis. J Speech Lang Hear Res 2013 Jun;56(3):921–932. pmid:23275413
  20. 20. Starkweather CW. The epigenesis of stuttering. J Fluency Disord 2002 Winter;27(4):269–87. pmid:12506446
  21. 21. Starkweather CW, Gottwald SR. The demands and capacities model II: Clinical applications. J Fluency Disord 1990;15(3):143–157.
  22. 22. Onslow M, Packman A, Harrison E. The Lidcombe Program of Early Stuttering Intervention: A Clinicians's Guide. Austin, Texas: Pro-ed; 2003.
  23. 23. Onslow M, Menzies RG, Packman A. An operant intervention for early stuttering. The development of the Lidcombe program. Behav Modif 2001 Jan;25(1):116–139. pmid:11151481
  24. 24. Jones M, Onslow M, Packman A, O'Brian S, Hearne A, Williams S, et al. Extended follow-up of a randomized controlled trial of the Lidcombe Program of Early Stuttering Intervention. Int J Lang Commun Disord 2008 Nov-Dec;43(6):649–661. pmid:18608610
  25. 25. Gore K. The Australian budget Lidcombe proposal debate: a primer in memes [Internet]. 2015 [cited 2015, June 19]. Available: http://www.speechirl.com/blog/the-australian-budget-lidcombe-proposal-debate-a-primer-in-memes
  26. 26. Franken MC, Putker-de Bruijn D. RESTART-DCM Method. Treatment protocol developed within the scope of the ZonMW project Cost-effectiveness of the Demands and Capacities Model based treatment compared to the Lidcombe programme of early stuttering intervention: Randomised trial [Internet]. 2014 [cited 2015, January 10]. Available: http://www.nedverstottertherapie.nl/
  27. 27. Evans S, Royston P, Day S. Minim: allocation by minimisation in clinical trials [Internet]. [cited 2013, February 8]. Available: http://www-users.york.ac.uk/~mb55/guide/minim.htm
  28. 28. O'Callaghan CA. OxMaR: open source free software for online minimization and randomization for clinical trials. PLoS One 2014 Oct 29;9(10):e110761. pmid:25353169
  29. 29. Riley GD. Stuttering severity instrument for children and adults. Austin: Pro-Ed; 1994.
  30. 30. Clark CE, Conture EG, Walden TA, Lambert WE. Speech sound articulation abilities of preschool-age children who stutter. J Fluency Disord 2013 Dec;38(4):325–341. pmid:24331241
  31. 31. Yaruss JS. Converting between word and syllable counts in children's conversational speech samples. J Fluency Disord 2000 0;25(4):305–316.
  32. 32. Ingham R. J. & Riley G. Guidelines for Documentation of Treatment Efficacy for Young Children who stutter. JSLHR 1998;41:753–770. pmid:9712124
  33. 33. Jones M, Onslow M, Packman A, Williams S, Ormond T, Schwarz I, et al. Randomised controlled trial of the Lidcombe programme of early stuttering intervention. BMJ (Clinical research ed) 2005 Sep 24;331(7518):659.
  34. 34. Sawyer J, Yairi E. The effect of sample size on the assessment of stuttering severity. Am J Speech Lang Pathol 2006 Feb;15(1):36–44. pmid:16533091
  35. 35. EuroQol Group. EQ-5D User Guide. Basic information on how to use EQ-5D. 2009; Available: http://www.euroqol.org
  36. 36. Vanryckeghem M, Brutten GJ, Hernandez LM. A comparative investigation of the speech-associated attitude of preschool and kindergarten children who do and do not stutter. J Fluency Disord 2005;30(4):307–318. pmid:16246410
  37. 37. Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2000.
  38. 38. Boberg E, Kully D. Long-term results of an intensive treatment program for adults and adolescents who stutter. JSLHR 1994 Oct;37(5):1050–9.
  39. 39. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med 1998 Apr 30;17(8):857–872. pmid:9595616
  40. 40. Wilson EB. Probable inference, the law of succession, and statistical inference. J. Am Statist Assoc 1927;22(158):209–212.
  41. 41. Hayes AF, Krippendorf K. Answering the call for a standard reliability measure for coding data. Commun Methods Meas 2007;1(1):77–89.
  42. 42. Hayes AF. Spss-macro for calculating Krippendorff’s alpha [Internet]. 2013 [cited 2014, September 20]. Available: http://www.afhayes.com
  43. 43. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat 1979:65–70.
  44. 44. Perneger TV. What's wrong with Bonferroni adjustments. BMJ 1998 Apr 18;316(7139):1236–1238. pmid:9553006
  45. 45. Sander EK. Reliability of the Iowa Speech Disfluency Test. J Speech Hear Disord 1961 Jun;(Suppl 7):21–30. pmid:13746132
  46. 46. Altman DG. Some common problems in medical research. Practical statistics for medical research 1991;1:396–403.
  47. 47. Imel Z, Wampold BE. The Importance of Treatment and the Science of Common Factors in Psychotherapy. In: Brown SD, Lent RW, editors. Handbook of counseling Psychology. 4th ed. New York: John Wiley & Sons Inc.; 2008. p. 249.
  48. 48. Luborsky L, Singer B, Luborsky L. Comparative studies of psychotherapies. Is it true that "everywon has one and all must have prizes"? Arch Gen Psychiatry 1975 Aug;32(8):995–1008. pmid:239666
  49. 49. Bernstein Ratner N. Evidence-based practice in stuttering: Some questions to consider. J Fluency Disord 2005;30(3):163–188. pmid:15961152
  50. 50. Hayhow R. Does it work? Why does it work? Reconciling difficult questions. Int J Lang Commun Disord 2011 Mar;46(2):155–168. pmid:21401814
  51. 51. Donaghy M, Harrison E, O'Brian S, Menzies R, Onslow M, Packman A, et al. An investigation of the role of parental request for self-correction of stuttering in the Lidcombe Program. Int J Speech Lang Pathol 2015 Mar 12:1–7.
  52. 52. Bernstein Ratner N, Guitar B. Treatment of Very Early Stuttering and Parent-Administered Therapy: the State of the Art. In: Bernstein Ratner N, Tetnowski JA, editors. Current Issues in Stuttering Research and Practice. 1st ed.: Psychology Press; 2006.
  53. 53. Robey RR. A five-phase model for clinical-outcome research. J Commun Disord 2004 Sep-Oct;37(5):401–411. pmid:15231420
  54. 54. Conture EG. Treatment efficacy: stuttering. J Speech Hear Res 1996 Oct;39(5):S18–26. pmid:8898263