Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Diagnostic utility of the amyotrophic lateral sclerosis Functional Rating Scale—Revised to detect pharyngeal dysphagia in individuals with amyotrophic lateral sclerosis

  • Jennifer L. Chapin,

    Roles Data curation, Formal analysis, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America

  • Lauren Tabor Gray,

    Roles Data curation, Formal analysis, Writing – review & editing

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Neurology, Phil Smith Neuroscience Institute, Holy Cross Hospital, Fort Lauderdale, FL, United States of America

  • Terrie Vasilopoulos,

    Roles Formal analysis

    Affiliation Department of Anesthesiology, College of Medicine, University of Florida, Gainesville, FL, United States of America

  • Amber Anderson,

    Roles Data curation, Writing – review & editing

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Speech, Language and Hearing Sciences, University of Florida, Gainesville, FL, United States of America

  • Lauren DiBiase,

    Roles Data curation, Writing – review & editing

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Speech, Language and Hearing Sciences, University of Florida, Gainesville, FL, United States of America

  • Justine Dallal York,

    Roles Formal analysis, Writing – review & editing

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Speech, Language and Hearing Sciences, University of Florida, Gainesville, FL, United States of America

  • Raele Robison,

    Roles Data curation, Formal analysis

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Speech, Language and Hearing Sciences, University of Florida, Gainesville, FL, United States of America

  • James Wymer,

    Roles Supervision, Writing – review & editing

    Affiliation Department of Neurology, College of Medicine, University of Florida, Gainesville, FL, United States of America

  • Emily K. Plowman

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Visualization, Writing – original draft, Writing – review & editing

    eplowman@phhp.ufl.edu

    Affiliations Aerodigestive Research Core, University of Florida, Gainesville, FL, United States of America, Department of Speech, Language and Hearing Sciences, University of Florida, Gainesville, FL, United States of America, Department of Neurology, College of Medicine, University of Florida, Gainesville, FL, United States of America, Department of Surgery, College of Medicine, University of Florida, Gainesville, FL, United States of America

Abstract

Objective

The ALS Functional Rating Scale–Revised (ALSFRS-R) is the most commonly utilized instrument to index bulbar function in both clinical and research settings. We therefore aimed to evaluate the diagnostic utility of the ALSFRS-R bulbar subscale and swallowing item to detect radiographically confirmed impairments in swallowing safety (penetration or aspiration) and global pharyngeal swallowing function in individuals with ALS.

Methods

Two-hundred and one individuals with ALS completed the ALSFRS-R and the gold standard videofluoroscopic swallowing exam (VFSE). Validated outcomes including the Penetration-Aspiration Scale (PAS) and Dynamic Imaging Grade of Swallowing Toxicity (DIGEST) were assessed in duplicate by independent and blinded raters. Receiver operator characteristic curve analyses were performed to assess accuracy of the ALSFRS-R bulbar subscale and swallowing item to detect radiographically confirmed unsafe swallowing (PAS > 3) and global pharyngeal dysphagia (DIGEST >1).

Results

Although below acceptable screening tool criterion, a score of ≤ 3 on the ALSFRS-R swallowing item optimized classification accuracy to detect global pharyngeal dysphagia (sensitivity: 68%, specificity: 64%, AUC: 0.68) and penetration/aspiration (sensitivity: 79%, specificity: 60%, AUC: 0.72). Depending on score selection, sensitivity and specificity of the ALSFRS-R bulbar subscale ranged between 34–94%. A score of < 9 optimized classification accuracy to detect global pharyngeal dysphagia (sensitivity: 68%, specificity: 68%, AUC: 0.76) and unsafe swallowing (sensitivity:78%, specificity:62%, AUC: 0.73).

Conclusions

The ALSFRS-R bulbar subscale or swallowing item did not demonstrate adequate diagnostic accuracy to detect radiographically confirmed swallowing impairment. These results suggest the need for alternate screens for dysphagia in ALS.

Introduction

Amyotrophic lateral sclerosis (ALS) is a progressive and fatal neurodegenerative disease affecting both upper and lower motor neurons within the cortex, brainstem and spinal cord [1]. Dysphagia, or swallowing impairment, occurs in a reported 85% of patients with ALS at some point during the disease process and is associated with malnutrition, weight loss, reduced quality of life, aspiration pneumonia and death [26]. Early detection and consistent monitoring of dysphagia provides the opportunity to mitigate associated risks and improve survival with timely interventions [7].

A universally accepted and validated clinical test battery to accurately assess and monitor bulbar disease progression is currently lacking [8]. A 2017 survey of Northeast ALS (NEALS) centers in the United States revealed highly variable practice patterns for the evaluation of bulbar function in patients with ALS [9]. Both clinical and instrumental swallow evaluations were found to be underutilized in multidisciplinary ALS clinics with less than 60% of respondents utilizing clinical swallow assessments and only 27% referring for the gold standard videofluoroscopic swallowing evaluation. Importantly, this survey revealed that the only clinical test routinely performed to evaluate bulbar function (>90% of sites) was the revised ALS Functional Rating Scale (ALSFRS-R).

The ALSFRS-R is a 12-item questionnaire with each question rated on a 5-point ordinal scale used to monitor progression of disability in patients with ALS. The scale currently represents the most widely used ALS outcome measure in Phase II and III clinical trials and longitudinal studies [10]. More recently, the psychometric properties of the ALSFRS-R have been evaluated with evidence suggesting multidimensionality, and the utilization of individual subscale scores rather than a total score has been recommended [1113]. These bulbar, motor and respiratory subscores are intended to provide more precise prognostic information, as the individual’s domain scores have been demonstrated to be more clinically robust when reported as subscores rather than a combined score [14]. Specifically, individuals with bulbar-onset disease demonstrated slower rate of decline on the motor subscore and hastened decline on the bulbar subscore compared to those with spinal onset disease [14]. One study of 18 individuals with motor neuron disease (MND) investigated the relationship between the ALSFRS-R bulbar subscore and radiographically confirmed airway invasion; however, individuals who aspirated during swallowing were not included in the study cohort, limiting the generalization of the results [15]. The discriminant ability of the bulbar subscore and swallowing item score to detect radiographically confirmed pharyngeal dysphagia in ALS has not yet been determined. We therefore sought to evaluate the discriminant ability of the ALSFRS-R bulbar subscale and swallowing item scores to classify early radiographically confirmed pharyngeal dysphagia in patients with ALS. Given the that the scale is a five-point ordinal scale that lacks linearity, we hypothesized that the ALSFRS-R would not demonstrate adequate sensitivity to detect mild changes in pharyngeal swallowing function in individuals with ALS.

Materials and methods

Participants

All eligible ALS patients who attended the University ALS clinic were informed of the study and invited to participate, representing a convenience sample. Two-hundred and one individuals were enrolled in this study. Inclusion criteria were: 1) confirmed diagnosis of ALS (Revised El Escorial criteria) by a neuromuscular neurology specialist; 2) not pregnant, 3) no allergies to barium, and 4) still consuming some form of foods and liquids by mouth. This study was approved by the University of Florida Institutional Review Board and conducted in accordance with the Declaration of Helsinki. All participants provided informed written consent. Participants attended a single testing session which included completion of the ALSFRS-R (index test) and a standardized videofluoroscopic swallowing examination (VFSE, gold standard reference test).

Index test.

The ALSFRS-R is a 12-item questionnaire validated to monitor functional disease progression across four subscales of activities of daily living that include bulbar, fine motor, gross motor and respiratory domains [10]. Three items assessing speech, salivation, and swallowing comprise the bulbar subscale with each item scored on a five-point ordinal scale (0–4) for a total of 12 points, with higher scores indicating better self-reported function (0 = total loss of function, 12 = normal functioning). A single question on swallowing is scored as follows 4—Normal eating habits, 3—Early eating problems; occasional choking, 2—Dietary consistency changes, 1—Needs supplemental tube feeding, 0—NPO (exclusively parenteral or enteral feeding) [10]. Participants completed the ALSFRS-R in interview fashion with an opportunity for input by their caregivers. All research personnel conducting these interviews completed training in the administration and scoring of the ALSFRS-R.

Reference standard.

VFSE was completed by a trained research speech-language pathologist (SLP) with participants comfortably seated in an upright lateral viewing plane using a properly collimated Phillips BV Endura fluoroscopic C-arm unit (GE 9900 OEC Elite Digital Mobile C-Arm system type 718074). Fluoroscopic images and synced audio were digitally recorded at 30 frames per second using a high resolution (1024 x 1024) TIMS DICOM system (Version 3.2, TIMS Medical, TM, Chelmsford, MA) for subsequent analysis. A standardized bolus presentation was administered utilizing a cued instruction to swallow and included: three 5 mL thin liquid barium, one comfortable cup sip of thin liquid barium, three 5 mL thin honey barium, two 5 mL pudding consistency barium, and a ¼ graham cracker square coated with pudding consistency barium (Varibar®, Bracco Diagnostics, Inc., Monroe Township, NJ). If the patient was unable to self-feed due to upper extremity weakness, clinician assistance or alternative methods (i.e., straw) were employed, consistent with the individual’s feeding methods routinely utilized at home. SLPs enforced standardized bailout criteria requiring administration of thicker consistencies following two instances of aspiration and discontinuation if an additional aspiration event occurred during the exam. VFSE recordings were saved to a secure server and blinded for subsequent analysis.

VFSE outcome measures

Each VFSE was rated in duplicate by two trained, blinded and independent raters. Complete agreement (100%) was required for all ratings, with a discrepancy meeting utilized to finalize any inconsistent ratings between raters.

Swallowing safety.

The Penetration-Aspiration Scale (PAS) was utilized to evaluate swallowing safety (Fig 1) [16]. This validated eight-point ordinal scale indexes the depth of contrast material entering the airway during swallowing events, the presence of a protective response, and if aspirate material was ejected from the airway [16]. All elicited swallows within a given bolus trial were ascribed a PAS score and the worst PAS score utilized for statistical analysis. Fig 1 denotes the PAS with established categorical levels of airway safety used.

thumbnail
Fig 1. Penetration Aspiration Scale (PAS) scores and corresponding binary classifications of swallowing safety status.

A) Representative videofluoroscopic images depicting safe swallowing with no contrast material entering the airway. B) Laryngeal penetration. C) Tracheal aspiration [16].

https://doi.org/10.1371/journal.pone.0236804.g001

Global pharyngeal swallowing.

The dynamic imaging grade of swallowing toxicity (DIGEST) is a validated five-point ordinal scale created to assess both efficiency and safety of bolus flow [17] and recently utilized in ALS [18]. The DIGEST (Fig 2) yields a global grade of pharyngeal dysphagia evaluated on bolus transport during the entirety of the videofluoroscopic swallow study to determine clinically relevant categories of overall pharyngeal dysphagia severity levels. DIGEST total scores are a composite of two subscores (scored 0–4) addressing: (i) swallowing efficiency based on degree of bolus clearance, and (ii) airway safety based on severity and frequency of PAS scores. DIGEST scores of zero indicate normal swallowing while total and subscore grades of 4 indicate life-threatening dysphagia.

thumbnail
Fig 2. Dynamic Imaging Grade of Swallowing Toxicity (DIGEST) outcome [17].

https://doi.org/10.1371/journal.pone.0236804.g002

Statistical analysis.

Descriptives were performed to summarize participant demographics and outcomes of interests. A receiver operator characteristic (ROC) curve analysis was then performed on the index test (ALSFRS-R bulbar subscale and swallowing item) to identify unsafe (PAS > 3) and global dysphagia (DIGEST > 1). Area under the curve (AUC) with bootstrapped 95% confidence intervals, sensitivity, specificity, positive predictive value (PPV), and negative predictive values (NPV) were calculated using JMP Pro Version 14.1.0 (SAS Institute Inc., Cary, NC). Optimal classification cutoffs for index test were determined by values that maximized both sensitivity and specificity.

Results

Participant demographics

Complete ALSFRS-R and VFSE data were missing in four participants resulting in 197 patients in the final analysis. Mean age was 62.9 (SD = 10.3) and average ALS disease duration was 26.6 months from symptom onset (SD = 23.6). Fifty-three percent were male (n = 106) and 58.1% presented with a spinal onset (n = 111). Mean ALSFRS-R score was 35.3 (SD = 7.4). Frequency data for the ALSFRS-R bulbar subscale and swallowing item scores are presented in histogram plots in Fig 3A and 3B respectively. Mean ALSFRS-R bulbar subscale score was 9.1 (SD = 2.4) and mean swallowing item score was 3.05 (SD = 0.79). Radiographically confirmed unsafe swallowing was identified in 38.9% of patients (n = 76, Fig 3C) and prevalence of global pharyngeal dysphagia was 58.9% (n = 116, Fig 3D).

thumbnail
Fig 3. Frequency distribution of outcomes of interest in 197 participants.

A) Amyotrophic Lateral Sclerosis Functional Rating Scale–Revised (ALSFRS-R) Bulbar Subscale scores. B) ALSFRS-R swallow item scores. C) Penetration aspiration scale scores (PAS). D) Dynamic Imaging Grade of Swallowing Toxicity scores (DIGEST).

https://doi.org/10.1371/journal.pone.0236804.g003

Discriminant ability of the ALSFRS-R to detect swallowing impairment

Scatterplots depicting relationships between the ALSFRS-R and swallowing outcomes of interest are shown in Fig 4. ROC curve results for the ALSFRS-R bulbar subscale and ALSFRS-R swallowing items to detect radiographically confirmed penetrators/aspirators are presented in Table 1 and Fig 5A and 5B. Classification accuracy for both ALSFRS-R outcomes to detect global pharyngeal dysphagia are presented in Table 2 and Fig 5C and 5D. Optimized classification cutoff of ALSFRS-R swallowing score of ≤ 3 and ALSFRS-R bulbar score of ≤ 9 were found for both outcomes.

thumbnail
Fig 4. Scatterplots for the amyotrophic lateral sclerosis Functional Rating Scale–Revised (ALSFRS-R) and radiographic swallowing outcomes of interest (dynamic imaging grade of swallowing toxicity–DIGEST and penetration-aspiration scale–PAS).

https://doi.org/10.1371/journal.pone.0236804.g004

thumbnail
Fig 5. Receiver operator characteristic curve results for the amyotrophic lateral sclerosis Functional Rating Scale–Revised (ALSFRS-R).

(A, B) ALSFRS-R bulbar subscale and swallowing item to detect unsafe swallowing. (C, D) Global pharyngeal dysphagia.

https://doi.org/10.1371/journal.pone.0236804.g005

thumbnail
Table 1. Summary of receiver operator characteristic curve results for the ALSFRS-R bulbar subscale and swallowing item to detect unsafe swallowing.

https://doi.org/10.1371/journal.pone.0236804.t001

thumbnail
Table 2. Summary of receiver operator characteristic results for the ALSFRS-R bulbar subscale and swallowing item to detect global pharyngeal dysphagia.

https://doi.org/10.1371/journal.pone.0236804.t002

Discussion

To our knowledge, this represents the first investigation to compare ALSFRS-R bulbar outcomes to those of the gold standard reference test for swallowing (VFSE). The ALSFRS-R bulbar subscale and swallowing item demonstrated poor to fair diagnostic accuracy to detect radiographically confirmed pharyngeal swallowing impairment in the 197 ALS patients examined (AUC: 0.68–0.76). No cut score emerged for ALSFRS-R outcome with an acceptable level of classification accuracy to distinguish normal versus disordered swallowing. Thus, the ALSFRS-R did not demonstrate adequate clinical utility as a screening tool to detect early pharyngeal dysphagia and demonstrated insufficient sensitivity as a marker of change in pharyngeal swallowing function for research clinical trials. These findings highlight the need for the development of sensitive tools to adequately screen relative risk of swallowing impairment for use in multidisciplinary ALS clinics and research settings alike.

Bulbar subscale

Classification accuracy of the ALSFRS-R bulbar subscale to detect global pharyngeal dysphagia was considered poor to fair when comparing our results to accepted screening tool criterion levels [19]. No clear score or threshold emerged that yielded an acceptable balance between sensitivity and specificity when examining ROC outcomes across bulbar subscale scores. An effective screening tool should accurately and quickly identify at risk individuals to triage for further comprehensive evaluation and ideally minimize false negatives (i.e., missing individuals with impairment) while at the same time avoiding over identification of individuals without the disorder being screened (i.e. false positives). While generally specificity is sacrificed at the cost of increased sensitivity; a screening tool with high sensitivity but very low specificity will create undue strain on health care workers, lead to overutilization of resources and unnecessary testing, and increase patient and caregiver burden. To this end, an ALSFRS-R bulbar subscale score of ≤11 correctly identified 87% of ALS patients with global pharyngeal dysphagia; however, misclassified two-thirds of patients as dysphagic who demonstrated normal swallowing on VFSE. Use of a lower cut-point of ≤10 decreased sensitivity to an unacceptable level without significant improvements in specificity, PPV or NPV. This cut point would miss one-quarter of patients with global dysphagia (i.e. false negatives) and would over-refer 52% of patients without dysphagia for additional testing. Finally, a bulbar subscale score of < 9 derived the most balanced degree of sensitivity and specificity of 68%, however would misclassify one-third of individuals with global pharyngeal dysphagia (false negatives).

Similarly, no cut score emerged for the ALSFRS-R bulbar subscale to detect penetration or aspiration. Although sensitivity of the bulbar subscale to detect unsafe swallowing was good-excellent for higher scores of 10 and 12 (> 86%) they were associated with low specificity (30% and 45%), high false positives, and ow PPVs. These findings are in agreement with observations the bulbar subscale is not sensitive to detect early speech impairment in ALS patients when compared to objective physiologic speech metrics [20].

Swallowing item

The ALSFRS-R swallowing item demonstrated poor overall screening accuracy to classify both global swallowing and safety status. Unlike the bulbar subscale, however, a clear score emerged to optimize obtained sensitivity and specificity. A swallowing item score of ≤ 3 accurately classified only 68% of individuals with confirmed global dysphagia missing approximately one-third of impaired individuals and representing a PPV that is not acceptable. Further, this score misclassified 36% of patients with confirmed normal swallowing as being dysphagic. Clearly this ‘optimal’ score is not acceptable for distinguishing global swallowing status in ALS.

Examination of the swallowing item’s classification accuracy to differentiate safe vs. unsafe ALS swallowers was noted to be higher, with a cut score of ≤3 yielding a sensitivity of 79% and a specificity of 60%. Although improved, diagnostic utility at this optimal score threshold remained suboptimal for a useful screening tool given that it would miss one in every five penetrator/aspirators and would over-refer 40% of patients without impairment for further evaluation. An important consideration when interpreting these data is the fact that individuals with ALS may not be fully aware of subtle dietary adaptations or modifications they may implement to compensate for a progressive decline in function [2123]. This is highly relevant given that the ALSFRS-R is a patient report outcome that asks patients to select the descriptor for a function being queried.

Given that the ALSFRS-R is commonly used in research as a baseline stratification tool or outcome to measure change in function over time; researchers are advised to consider these findings for future clinical trials. Further, clinical adoption of these scores as a dysphagia screen could create unnecessary burden for patients and their caregivers and facilitate inappropriately timed referrals for instrumental swallowing evaluations.

Although no published screening tool exists for dysphagia in ALS, two reports have examined the clinical utility of another patient-reported outcome measure (PROM) and of voluntary cough testing to detect aspiration. The Eating Assessment Tool (EAT-10) is a validated 10-item swallowing specific PROM that is available in 13 languages [24]. A cut score of >8 on the EAT-10 demonstrated a sensitivity, specificity and likelihood ratio of 86%, 72% and 3.1, respectively to detect radiographically confirmed aspiration [25]. However, the discriminant ability of the EAT-10 to detect global pharyngeal dysphagia in ALS, has not yet been established. In addition to PROMs, voluntary cough function is noted to significantly differ in individuals with ALS compared to healthy age and gender matched controls, contributing to the impaired ability to effectively expel tracheal aspirate and manage secretions in this population [26]. Given that peak expiratory flow is noted to be reduced by 50% in ALS patients with unsafe swallowing [27], voluntary cough peak expiratory flow (commonly known as peak cough flow testing) has been suggested as a screen to index one’s physiologic airway defense capacity [28, 29]. Future work is needed to identify additional sensitive clinical markers in order to develop and validate a pragmatic and accurate dysphagia screening tool for use in ALS clinics [8, 9, 29].

While this work represents the first attempt to examine the discriminant ability and clinical utility of the ALSFRS-R for detecting radiographically confirmed dysphagia, limitations need to be acknowledged. First, following typical analytic methods used in dysphagia research [30, 31], the worst PAS score was utilized to determine swallowing safety status, which may have skewed outcomes towards impairment [26]. Given that we were interested in catching early impairment however, we feel that any potential bias was warranted. Further, the global pharyngeal dysphagia metric (DIGEST scale) incorporates both the frequency and severity of penetration and aspiration and therefore mitigated potential bias for this specific outcome [17]. Second, the global dysphagia outcome only examines pharyngeal phase swallowing impairments. Therefore, our exam was specific to pharyngeal phase deficits. It is possible that a patient may have rated the ALSFRS-R swallowing item to reflect or communicate perceived impairment in the oral phase that were not detected in this study with use of the DIGEST or PAS scales. Third, given practical and ethical considerations and constraints, our sample represented individuals with mild-moderate ALS severity and bulbar dysfunction with only one patient 100% dependent on non-oral nutrition. Therefore, this cohort may not represent the complete spectrum of ALS swallowing severities. Fourth, other important non-physiologic aspects related to dysphagia such as mealtime enjoyment, mealtime duration, caregiver burden and fatigue were not indexed in this study. Finally, although these data represent the largest VFSE dataset presented to date, further work in additional patients is warranted to validate these findings.

Conclusion

Early detection of dysphagia is paramount to guide timely clinical management decisions to mitigate or delay development of known sequalae. Given the widespread use of the ALSFRS-R to index bulbar and pharyngeal swallowing function, we aimed to determine the discriminant ability of the bulbar subscale and swallowing item to detect radiographically confirmed impairments in swallowing safety and global pharyngeal swallowing function using the gold standard VFSE. Overall accuracy of the ALSFRS-R was poor to diagnostic accuracy for swallowing safety and global pharyngeal swallow function did not meet acceptable standards across any score criteria. We therefore do not recommend use of the ALSFRS-R in isolation to screen for pharyngeal swallowing function and encourage the development of a disease specific screening tool that can accurately triage high-risk individuals for instrumental swallowing evaluation.

Acknowledgments

We are grateful for the support of Dr. John Wilkins and Nancy Wilkins and the individuals with ALS who participated.

References

  1. 1. Brooks BR, Miller RG, Swash M, Munsat TL. El Escorial revisited: revised criteria for the diagnosis of amyotrophic lateral sclerosis. Amyotroph Lateral Scler Other Motor Neuron Disord. 2000;1:293–9. pmid:11464847
  2. 2. Chen A, Garrett CG. Otolaryngologic presentations of amyotrophic lateralsclerosis. Otolaryngol Head Neck Surg. 2005;132(3):500–4. pmid:15746870
  3. 3. Kuhnlein P, Gdynia HJ, Sperfeld AD, Lindner-Pfleghar B, Ludolph AC, Prosiegel M, et al. Diagnosis and treatment of bulbar symptoms in amyotrophic lateral sclerosis. Nat Clin Pract Neurol. 2008;4(7):366–74. pmid:18560390
  4. 4. Paris G, Martinaud O, Petit A, Cuvelier A, Hannequin D, Roppeneck P, et al. Oropharyngeal dysphagia in amyotrophic lateral sclerosis alters quality of life. J Oral Rehabil. 2013;40(3):199–204. pmid:23278936
  5. 5. Sorenson EJ, Crum B, Stevens JC. Incidence of aspiration pneumonia in ALS in Olmsted County, MN. Amyotroph Lateral Scler. 2007;8(2):87–9. pmid:17453635
  6. 6. Tabor L, Gaziano J, Watts S, Robison R, Plowman EK. Defining Swallowing-Related Quality of Life Profiles in Individuals with Amyotrophic Lateral Sclerosis. Dysphagia. 2016;31(3):376–82. pmid:26837611
  7. 7. Spataro R, Ficano L, Piccoli F, La Bella V. Percutaneous endoscopic gastrostomy in amyotrophic lateral sclerosis: effect on survival. J Neurol Sci. 2011;304(1–2):44–8. pmid:21371720
  8. 8. Pattee GL, Plowman EK, Brooks BR, Berry JD, Atassi N, Chapin JL, et al. Best practices protocol for the evaluation of bulbar dysfunction: summary recommendations from the NEALS bulbar subcommittee symposium. Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(3–4):311–2. pmid:29205054
  9. 9. Plowman EK, Tabor LC, Wymer J, Pattee G. The evaluation of bulbar dysfunction in amyotrophic lateral sclerosis: survey of clinical practice patterns in the United States. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(5–6):351–7. pmid:28425762
  10. 10. Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. Journal of the Neurological Sciences. 1999;169:13–21. pmid:10540002
  11. 11. Bacci ED, Staniewska D, Coyne KS, Boyer S, White LA, Zach N, et al. Item response theory analysis of the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised in the Pooled Resource Open-Access ALS Clinical Trials Database. Amyotroph Lateral Scler Frontotemporal Degener. 2016;17(3–4):157–67. pmid:26473473
  12. 12. Franchignoni F, Mandrioli J, Giordano A, Ferro S. A further Rasch study confirms that ALSFRS-R does not conform to fundamental measurement requirements. Amyotroph Lateral Scler Frontotemporal Degener. 2015;16(5–6):331–7. pmid:25916346
  13. 13. Franchignoni F, Mora G, Giordano A, Volanti P, Chio A. Evidence of multidimensionality in the ALSFRS-R Scale: a critical appraisal on its measurement properties using Rasch analysis. J Neurol Neurosurg Psychiatry. 2013;84(12):1340–5. pmid:23516308
  14. 14. Rooney J, Burke T, Vajda A, Heverin M, Hardiman O. What does the ALSFRS-R really measure? A longitudinal and survival analysis of functional dimension subscores in amyotrophic lateral sclerosis. J Neurol Neurosurg Psychiatry. 2017;88(5):381–5. pmid:27888187
  15. 15. Brandao BC, Galdino AS, Lourencao LG, Trindade GS, Silva M, Silva RGD. Correlation between bulbar functionality and laryngeal penetration and/or laryngotracheal aspiration on motor neuron disease. CoDAS. 2018;30(1):e20170056. pmid:29513871
  16. 16. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia. 1996;11(2):93–8. pmid:8721066
  17. 17. Hutcheson KA, Barrow MP, Barringer DA, Knott JK, Lin HY, Weber RS, et al. Dynamic Imaging Grade of Swallowing Toxicity (DIGEST): Scale development and validation. Cancer. 2017;123(1):62–70. pmid:27564246
  18. 18. Plowman E. K., Tabor-Gray L., Rosado K. M., Vasilopoulos T., Robison R., Chapin J. L., G, et al. Impact of expiratory strength training in amyotrophic lateral sclerosis: Results of a randomized, sham-controlled trial. Muscle Nerve. 2019;59(1):40–46. pmid:29981250
  19. 19. Safari S, Baratloo A, Elfil M, Negida A. Evidence based emergency medicine; part 5 receiver operating curve and area under the curve. Emergency. 2016;4(2):111–3. pmid:27274525
  20. 20. Allison KM, Yunusova Y, Campbell TF, Wang J, Berry JD, Green JR. The diagnostic utility of patient-report and speech-language pathologists' ratings for detecting the early onset of bulbar symptoms due to ALS. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(5–6):358–66. pmid:28355886
  21. 21. Onesti E, Schettino I, Gori MC, Frasca V, Ceccanti M, Cambieri C, et al. Dysphagia in amyotrophic lateral sclerosis: impact on patient behavior, diet adaptatin, and Riluzole management. Front in Neurol. 2017;8(94).
  22. 22. Goeleven A, Robberecht W, Sonies B, Carbonez A, Dejaeger E. Manofluorographic evaluation of swallowing in amyotrophic lateral sclerosis and its relationship with clinical evaluation of swallowing. Amyotroph Lateral Scler. 2006;7(4):235–40. pmid:17127562
  23. 23. Higo R, Tayama N, Nito T. Longitudinal analysis of progression of dysphagia in amyotrophic lateral sclerosis. Auris Nasus Larynx. 2004;31(3):247–54. pmid:15364359
  24. 24. Belafsky PC, Mouadeb DA, Rees CJ, Pryor JC, Postma GN, Allen J, et al. Validity and reliability of the Eating Assessment Tool (EAT-10). Ann Otol Rhinol Laryngol. 2008;117(12): 919–924. pmid:19140539
  25. 25. Plowman EK, Tabor LC, Robison R, Gaziano J, Dion C, Watts SA, et al. Discriminant ability of the Eating Assessment Tool-10 to detect aspiration in individuals with amyotrophic lateral sclerosis. Neurogastroenterology and motility. 2016;28(1):85–90. pmid:26510823
  26. 26. Tabor-Gray LC, Gallestagui A, Vasilopoulos T, Plowman EK. Characteristics of impaired voluntary cough function in individuals with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20(1–2):37‐42. pmid:30652513
  27. 27. Plowman EK, Watts SA, Robison R, et al. Voluntary Cough Airflow Differentiates Safe Versus Unsafe Swallowing in Amyotrophic Lateral Sclerosis. Dysphagia. 2016;31(3):383‐390. pmid:26803772
  28. 28. Yunusova Y, Plowman EK, Green JR, Barnett C, Bede P. Clinical Measures of Bulbar Dysfunction in ALS. Front Neurol. 2019;10:106. Published 2019 Feb 19. pmid:30837936
  29. 29. Pattee GL, Plowman EK, Focht Garand KL, et al. Provisional best practices guidelines for the evaluation of bulbar dysfunction in amyotrophic lateral sclerosis. Muscle Nerve. 2019;59(5):531‐536. pmid:30620104
  30. 30. Steele CM, Grace-Martin K. Reflections on Clinical and Statistical Use of the Penetration-Aspiration Scale. Dysphagia. 2017;32:601–16. pmid:28534064
  31. 31. Borders JC, Brates D. Use of the Penetration-Aspiration Scale in Dysphagia Research: A Systematic Review [published online ahead of print, 2019 Sep 19]. Dysphagia. 2019;10.