Ethical approval

No ethical approval was required for this systematic review with meta-analysis, as all data were already published in peer-reviewed journals. No patients were involved in the design, conduct or interpretation of our study.

Eligibility criteria

The Population, Index Test, Reference Test, Diagnosis (PIRD) framework was applied to contextualize the research question and conduct the systematic selection. Eligibility criteria were defined by PIRD as follows; P: tracheostomized patients, I: MEBDT, R: FEES, D: aspiration. Randomised controlled trials, observational, prospective, and retrospective studies were included, whereas reviews, case reports, case series, and simple descriptive studies were excluded. Pediatric populations (patients under 18 years of age), patients without tracheostomy tubes, and patients with contraindications to MEBDT and FEES were excluded from the analysis.

Information sources

The systematic search was conducted in three databases, PubMed, Cochrane Central, and Embase on 24^th November, 2023, and repeated it with the same search key and strategy on 25^th June, 2025.

Search strategy

The following search key was framed for the systematic search, with no restriction, no filters, and no automatic translations: (“FEES” OR (“fiberoptic” AND endoscop* AND “evaluation”) OR (“fiberoptic” AND endoscop* AND exam* AND swallow*) OR (“flexible” AND endoscop* AND evaluat*) OR (“Modified” AND “Evans” AND “blue” AND “dye” AND test*) OR “MEBDT” OR (“Evans” AND “blue” AND “dye” AND test*) OR (“blue” AND “dye” AND test*)) AND (tracheost* OR endotrach* OR (“artificial” AND “airway”) OR cannula* OR intubat* OR (“percutaneous” AND “dilatational” AND “tracheostomy”))

Two independent investigators (E.S. and I.S.) independently evaluated the articles from 15^th November to 24^th November 2023. Disagreements were resolved by a third independent reviewer (C.T.).

Selection process

The selection process was carried out by two independent reviewers (E.S and I.S), who first removed all duplicates, then screened articles by title and abstract, and finally selected relevant studies by full text. Publications were excluded based on predetermined selection criteria (see PROSPERO under CRD4202347992).

Data collection process

Data of eligible articles [8,10,11–13,14] were collected independently by two investigators (E.S and I.S), using a standardized data collection sheet based on consensus by study authors. A third independent reviewer (C.T.) resolved any disagreements. The following data items were extracted: title, first author, year of publication, study design, patient characteristics and clinical state, index test characteristics, reference test characteristics, and diagnostic outcomes, specifically sensitivity and specificity.

Risk of bias assessment

Two authors (E.S. and I.S.) performed the risk of bias assessment independently. On the basis of the recommendation of the Cochrane Collaboration, the QUADAS-2 risk of bias assessment tool was applied [15]. Two independent reviewers performed the assessment, and an independent third investigator resolved disagreements. Publication bias could not be assessed through visual inspection of Funnel-plots and Egger’s test as less than 10 articles were selected. The level of certainty of evidence of the studies included was assessed with the GRADE-Pro tool [16], based on the recommendation of the Cochrane Collaboration.

Synthesis methods

For the estimation of pooled specificity and sensitivity, the bivariate model of Chu et al (2006) [17] and Reitsma et al (2005) [18] was fitted. This approach considers the possible association between sensitivity and specificity. We plotted individual and pooled sensitivities and specificities of studies included, their summary estimates, and the corresponding 95% confidence and prediction regions. In these visualizations, the sizes of ellipsoids reflect the weights of the studies calculated according to the method described by Burke et al (2018) [19]. For PPV and NPV, separate univariate analyses were performed using the generalized mixed-effect approach (Stijten et al, 2010) [20]. Heterogeneity was assessed by calculating I² measure and its confidence interval arising from the separate univariate analyses.

PPV and NPV values for different prevalences (ranging from 0% to 100%) were also calculated from the pooled specificity and sensitivity values, to examine how the effect of prevalence changes the predictive values. These calculations were performed to illustrate the dependence of predictive values on population prevalence. For these calculations the following formulae were utilized:

where Sens_pooled and Spec_pooled are the the pooled sensitivity and specificity values derived from the bivariate meta-analysis of diagnostic accuracy, and P is the prevalence value.

Statistical analyses were carried out with R statistical software (version 4.1.2., R-core team, 2023) [21,22] using the meta (Balduzzi et al, 2019) [23] and the lme4 (Bates et al, 2015) [24] packages, and were partially based on the web-tool of Freeman et al (2019) [25]. The statistical analyses followed the advice of Harrer et al (2024 [26]).

Search and selection

The systematic search identified 2,227 records from the databases; 1,819 unique records remained after duplicate removal (Fig 1). After title-abstract selection, 61 records remained, of which six studies [8,10,11–13,14] met the inclusion criteria. All studies were observational studies.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. The PRISMA flowchart of systematic selection‌‌.

https://doi.org/10.1371/journal.pone.0349092.g001

Basic characteristics of studies included

Basic characteristics of the six articles included are presented in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Basic characteristics of studies included.

https://doi.org/10.1371/journal.pone.0349092.t001

Sensitivity and Specificity

On the basis of bivariate pooling of sensitivity and specificity, the overall sensitivity was 69.54% [95% CI: 37.68%, 89.60%] (Fig 2.). The overall pooled specificity derived from the six articles was 95.42% [67.38%, 99.53%] (Fig 2).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Random-effects bivariate diagnostic meta-analysis of sensitivity and specificity of the Modified Evan’s Blue Dye Test compared to the Fiberoscopic Evaluation of Swallowing in the diagnosis of aspiration.

Abbreviations: Crosses around dots represent confidence intervals in both dimensions. The confidence region of 2D 95% is marked with blue dashed line.

https://doi.org/10.1371/journal.pone.0349092.g002

Negative and positive predictive values

The prevalence of aspiration in the six articles ranged from 58.82% to 82.46% (Belafsky 73.33%, Filho 58.82%, Warnecke 60.98%, Fiorelli 82.46%, Munoz 70.73%, and Winklmaier 70%). Given this relatively wide prevalence range (23.64%), the pooled negative predictive value (NPV) was 59% [45%, 74%] (Fig 3).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Forest plot of the negative predictive value of the Modified Evan’s Blue Dye Test compared to the Fiberoscopic Evaluation of Swallowing in the diagnosis of aspiration.

Abbreviations: TP: true positive; FP: false positive; TN: true negative, FN: false negative; n: number of samples; CI: confidence interval.

https://doi.org/10.1371/journal.pone.0349092.g003

The pooled overall positive predictive value (PPV) was 95% [81%, 100%] (Fig 4), based on the six selected articles.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Forest plot of the positive predictive value of the Modified Evan’s Blue Dye Test compared to the Fiberoscopic Evaluation of Swallowing in the diagnosis of aspiration.

Abbreviations: TP: true positive; FP: false positive; TN: true negative, FN: false negative; n: number of samples; CI: confidence interval.

https://doi.org/10.1371/journal.pone.0349092.g004

Apart from the meta-analytic pooling of PPV and NPV values reported in the articles, these two metrics were also calculated from the above mentioned pooled specificity and sensitivity values. These calculated predictive values were then plotted against prevalence (Fig 5).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Linegraph of the dynamics of positive and negative predictive in relation to changing prevalences.

https://doi.org/10.1371/journal.pone.0349092.g005

Publication bias and heterogeneity

To assess heterogeneity of the bivariate sensitivity and specificity pool, we performed a leave-one-out (LOO) analysis (S3 Fig.). On the basis of this “sensitivity” analysis, specificity was not affected by the omission of any article (LOO specificity ranging from 93% to 95%), sensitivity was, however, somewhat affected by the omission of articles (LOO sensitivity ranges from 61% to 79%). Significant heterogeneity was identified among the studies in terms of univariate sensitivity (I² = 94%, p < 0.01) and specificity (I² = 72%, p < 0.01) (S1 and S2 Figs.).

Risk of bias and GRADE assessment

The methodological quality was assessed by determining the risk of bias across four domains of the QUADAS-2 evaluation tool [15] for diagnostic studies. During the analysis, the risk of bias was assessed as low in two of the articles, and high in four. In terms of GRADE assessment [27], the overall risk of bias was considered as serious due to the inclusion of a paper by Belafsky et al. [8] pooled for outcome, and the high risk of bias observed in the flow and timing domains of four articles [8,10,12,14] (Table 2).‌‌ We found that certainty of evidence was low, the most important issue was diagnostic accuracy and the effect of the test due to inconsistency and variability between different methods. GRADE results are shown in forest plots (S1 and S2 Figs.; Figs 3 and 4).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Summary of risk of bias and applicability concerns assessed by review authors for each domain in the included‌‌ studies.

https://doi.org/10.1371/journal.pone.0349092.t002

Comparison of FEES and MEBDT: Advantages and practical considerations

MEBDT has already been performed with food materials, such as ice, liquid, or mashed food, and blue dye. During the application of the MEBDT, the cuff of the tracheostomy tube is deflated. The patient takes blue-colored food or a drink of varying textures and amounts orally and is then asked to swallow. The MEBDT is positive if colored secretion appears when the trachea is suctioned through the tracheal cannula [10].

In contrast, FEES is an invasive procedure in which a thin fiberoptic endoscope is inserted transnasally to visualize the epiglottis, vocal cords, and the pharyngeal phase of swallowing in real-time. The procedure enables the detection of residue in the piriform sinuses and facilitates the assessment of movement disorders affecting the vocal cords and epiglottis. Local anesthesia is generally avoided, as it may compromise the sensory aspects of swallowing and increase the risk of aspiration. [28–30]. FEES is positive for aspiration if saliva, food, or any liquid is trapped under the vocal cords [31]. The blue coloration of the swallowed material can further enhance the assessment of aspiration. Both FEES and MEBDT can be performed at the bedside. One of the main advantages of FEES is that it is particularly useful in differentiating the pathophysiological background of dysphagia-related aspiration [32]. However, it may not be readily available in all medical settings, as it requires special instrument and skilled personnel trained in its use [33]. In contrast, MEBDT is much simpler and does not require specialized endoscopic skills or advanced professional training, [7,34] or special instruments. This can be a significant advantage in settings where endoscopy is not readily available.

Methodology and implication variability in MEBDT

The design of MEBDT in the studies included differed significantly (S1 Table). All six articles used different consistencies and amounts of food and liquid, and the time intervals between the index and the reference tests were also variable. Time frames for tracheal aspiration also varied widely. Without standardization, it is difficult to establish a proper protocol for MEBDT.

An important factor to consider during MEBDT is the temperature of the food or fluid being administered. Belafsky et al [8] used cold temperature bolus using ice chips. The sensation of cold may enhance swallowing reflexes toward improved motor control, and at the same time, induce rapid contraction of the esophageal muscles [35–38]. This condition may influence the swallowing test, and therefore room temperature fluid and food administration should be preferred in future studies.

MEBDT was performed with 100% PPV in three studies [10-12]. This suggests the application of different consistencies (water, saliva, or honey-like), different (5-10-20 ml) and repeated quantities of material to be swallowed. In this case, a PPV of 100% indicates that, in these studies, all patients who tested positive for aspiration using MEBDT were confirmed to have aspiration, meaning that all positive MEBDT findings represented true aspiration events across the tested bolus consistencies, reflecting high reliability for aspiration confirmation rather than overall diagnostic accuracy.

The time lag between MEBDT as an index and FEES as a reference test should be considered, as it might affect the reliable comparison of the two tests. Half of the articles did not perform the tests within one session [8,10,12,14] (S1 Table). In our opinion, a considerable delay between MEBDT and FEES could influence the diagnostic accuracy. This is because significant changes can be observed in the state of critically ill patients, even within a few hours. Therefore, the application of both tests in one session is recommended for future studies.

Only one study [8] included mechanically ventilated patients with minimal pressure support (10 out of 30 patients). Mechanical ventilation, even with minimized pressures could significantly affect diagnostic accuracy, as in the study by Belafsky et al [8], where the sensitivity of MEBDT was 100% in the case of mechanically ventilated patients. As the cuff was deflated, air leakage should be assumed with positive pressure support ventilation. Airflow from the trachea, to and through the vocal cords could influence aspiration. Therefore, we recommend the inclusion of patients weaned from mechanical ventilation in future studies, but further studies are also needed on the evaluation of aspiration of MEBDT with mechanical ventilation.

Among the articles, Filho et al [11] and Warnecke et al [13] reported the highest number of false negative cases relative to the total number, 9 and 17, respectively (Fig 4). However, the reasons are not discussed in the articles, but a few factors may be hypothesized to be behind these findings. One can be the limitations of the examination technique. When such a small amount of food or liquid is aspirated that is only visible by the fiberoptic examination, we speak of micro-aspiration [39]. In this case, the amount aspirated is so small that it cannot be suctioned from the trachea, even after repeated attempts.

Furthermore, there is no literature on the optimal amount of blue dye to add to food or liquid. Low concentrations may be too diluted by tracheal secretions, making aspiration less visible during tracheal suction.

Therefore, to improve the diagnostic accuracy of MEBDT, we recommend the use of high-concentration blue color and repeated attempts of tracheal suction. Blue dye should always be used, as other colors like red or green might be mistaken for blood or pus. Using a transparent suction tube with a white paper background helps make blue-colored tracheal secretions more visible during suction.

Effectiveness and limitations of MEBDT in aspiration detection: Findings from meta-analysis

Our meta-analysis has revealed that MEBDT, despite its lower sensitivity (69.5%), has a high positive predictive value (95%) that makes it a useful, adjunctive, bedside investigation for detecting aspiration in tracheostomized patients. This high PPV suggests that MEBDT is effective in confirming aspiration when the result is positive, particularly in a population with a higher prevalence of aspiration. However, although MEBDT has a high specificity (95%), its negative predictive value (NPV) is relatively low at 59%. This means that although a positive result is reliable, a negative result is less so for ruling out aspiration. Therefore, FEES is recommended as the next step following a negative MEBDT result for accurate detection.

It is important to emphasize that the calculated positive and negative predictive values (PPV and NPV) across different prevalence rates are presented to facilitate the interpretation of predictive performance. The PPV was consistently high throughout most of the plausible prevalence range. Even at a prevalence of 20%, the PPV was already around 80%, and for prevalences above 40%, the PPV consistently exceeded 90% (see Fig. 5). It should be emphasized that PPV and NPV, and the conclusions based on them, are only valid for populations with prevalence rates similar to those observed in the included studies (approximately 60–80%, which already represent a relatively heterogeneous range). Therefore, conclusions based on these predictive values cannot be directly applied to populations with substantially lower or higher prevalence.

Strengths and limitations

This is the first systematic review and meta-analysis comparing MEBDT with FEES, supplemented with RoB and GRADE assessments. This study directly informs bedside practice on the possibilities of using MEBDT in tracheostomized patients, and due to its simplicity, it can be immediately transferred into clinical practice. The primary limitation of this study is the relatively small total number of registered cases (n = 216). Additionally, only six articles were included, all of which were observational studies, resulting in substantial heterogeneity in clinical practice. A clearly defined study period was reported in only two of the six included studies, limiting methodological transparency and comparability (S1 Table). In addition, while FEES represents a standardized reference method, MEBDT lacks a uniformly standardized protocol, which may have contributed to methodological variability.

Implications for practice and research

Currently, due to methodological heterogeneity and limited evidence, MEBDT can serve as a bedside rule-in signal for aspiration in critically ill tracheostomized patients when positive; however, negative results require confirmation with an instrumental assessment. Additionally, there is insufficient data on the reliability of MEBDT in mechanically ventilated patients. The results from our systematic review and meta-analysis help inform clinical decision-making for dysphagia assessment in tracheostomized patients, highlighting areas where further research is needed. Future research should focus on prospective, standardized studies to further specify the sensitivity, specificity, and positive predictive value of MEBDT, taking into account the influence of aspiration prevalence and the possible heterogeneity of the investigating method. To establish MEBDT as a reliable standard screening test for aspiration, future research should aim to implement a standardized, reproducible protocol, and validate its performance prospectively across diverse patient populations against the gold standard FEES.

Key points‌‌

1. MEBDT can serve as a rule-in bedside signal for aspiration when positive.
2. MEBDT has high positive predictive value for confirming aspiration when positive, but negative results cannot reliably exclude aspiration.
3. Negative MEBDT result cannot reliably rule out aspiration requiring further diagnostic assessments.