Search methodology

We performed a systematic review in accordance with the preferred reporting for systematic reviews and meta-analysis (PRISMA-DTA) guidelines. The PRISMA-DTA checklist has been provided as a S1 Appendix) [14]. This review was registered with PROSPERO CRD42021268171; which is available at https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021268171. With the assistance of a health information specialist (QM), we searched CENTRAL, MEDLINE, EMBASE and Web of Science database from inception to August 10, 2022. We also searched ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Portal (ICTRP). The search strategy used MeSH headings and sensitive search terms. The search strategies are outlined in S2 Appendix. Finally, we hand searched reference lists of included articles and reviews for potentially eligible studies that might have been missed in the electronic database searches, conducted snowball searching, and contacted experts in the field.

Eligibility criteria, data extraction and risk of bias assessment

Patients of any age who were seen at an emergency department (ED) or hospital (any level of care) with suspected orbital infection (i.e., periorbital or orbital cellulitis) were eligible for inclusion. Studies were included if the study participants had undergone an ultrasound (index test) and a diagnostic reference standard (i.e., CT or MRI) within a reasonably close time interval (i.e., defined as within 48 hours). We included any type of ultrasound including those obtained by technologists and point-of-care ultrasounds (POCUS) obtained by healthcare professionals. We included all study types (i.e., case reports, case series, observational studies, randomized controlled trials. There were no restrictions based on country of origin or language of publication. We used Covidence to manage the selected studies [15].

Two review authors (MRA and TAO) independently screened titles and abstracts. The full text of any potentially relevant articles that met the inclusion criteria were then reviewed for eligibility. The review authors (MRA and TAO) independently screened the articles for inclusion and resolved any discrepancies through discussion. When needed, a third review author (PJG) was consulted. The review authors (MRA and TAO) independently extracted data using a custom designed data collection tool (S3 Appendix). Key aspects included in the data extraction form included general characteristics (e.g., year, country of study, time period, study design), population (e.g., age, sex, inclusion/exclusion criteria), imaging modality findings (e.g., signs of inflammation, abscess), timing of diagnostic tests, types of ultrasound probe/transducer, and outcomes (e.g., definitions of orbital cellulitis and complications). The imaging findings indicated the clinical diagnosis as per Chandler’s criteria [16]: I) periorbital (preseptal) cellulitis; II) orbital cellulitis; III) sub-periosteal abscess; IV) orbital abscess; and V) cavernous sinus thrombosis. These distinctions are anatomically based but provide a helpful schematic upon which to evaluate the diagnostic role of ultrasound. We also extracted data on ultrasound techniques that were employed to better visualize the orbit. Potentially relevant studies that were identified but for which the full text could not be obtained despite multiple attempts to contact the authors, were first labelled as “awaiting classification” and subsequently excluded.

Two reviewers (MRA, PJG) evaluated the methodological quality of individual studies using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool [17] and The Joanna Briggs Institute Critical Appraisal tools [18]. Discrepancies were resolved through discussion.

Statistical analysis

For the primary analysis, we a priori planned to use the individual patient as unit of analysis. For the secondary analysis, we used individual eyes as the unit of analysis. We planned to use Review Manager Web version (RevMan Web) to plot estimates of sensitivity and specificity from the studies in receiver operating characteristics (ROC) space and to build forest plots [19]. As the results of the ultrasound scan were expected to be binary, (i.e., presence or absence of orbital cellulitis), we intended to perform meta-analysis using a bivariate model to estimate summary sensitivity (i.e., proportion of individuals with orbital cellulitis who are correctly detected by ultrasound) and specificity (i.e., proportion of individuals without orbital cellulites who are correctly identified by ultrasound) [20]. However, due to insufficient data, we were unable to conduct a meta-analysis. We planned to conduct a meta-analysis if there were sufficient number of studies with low clinical heterogeneity. However, due to limited number of included studies, the results section is primarily descriptive.

Planned sensitivity analyses included the effect of excluding studies where older generations of ultrasound probes were used, and the effect of excluding studies with a high risk of bias for primary outcome. Planned subgroup analyses included the following characteristics: 1) Study type: experimental vs observational; 2) Age: <18 yrs (pediatric) vs. >18 yrs (adult); and 3) Type of ultrasound: standard ultrasound vs. POCUS.

Cohort studies

All three identified retrospective cohort studies were published before 2002, and none reported the sensitivity and specificity of ultrasound. Among the 46 patients reported in these studies, both ultrasound and CT scans were conducted on 18 (39%) cases. A summary of the cohort studies is provided in Table 1. Goodwin et al. was published in 1982 and reviewed medical records of 22 patients diagnosed with orbital cellulitis. Age of the patients ranged from 5 to 68 years. Orbital ultrasound was conducted by experienced technicians and CT scans were reviewed by a neuroradiologist. Findings were in agreement in five of the seven cases where both ultrasound and CT scans were conducted. In the remainder, ultrasound identified wooden foreign body and early abscess formation that were missed in the CT scans. Kaplan et. al. conducted a similar study published in 1999. Seven children between 2 and 13 years old hospitalized with sinus induced orbital infection were examined regularly by an otolaryngologist and ophthalmologist. All children underwent contrast enhanced CT scan and orbital ultrasound by an ophthalmologist within 24 and 48 hours of admission, respectively. Findings from the ultrasound and CT scan were in agreement in three. In two patients with orbital cellulitis (Chandler II) and two patients with Subperiosteal abscess (Chandler III), the authors reported that CT scan could not differentiate between orbital cellulitis and subperiosteal abscess, whereas ultrasound was able to do so. Mair et. al., published in 2002, reviewed clinical records of 17 children aged 1 to 10 years presenting with swelling and erythema of the eyelids. Orbital ultrasound and additional imaging (CT or MRI scan) were performed in only four patients. Although comparison of imaging findings for these patients were not reported, the authors reported that ultrasound identified patients with orbital infection. The authors also suggested that they were able to differentiate between subperiosteal abscess (Chandler III) and subperiosteal inflammatory infiltration (Chandler II) based on the sonographic findings.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Summary of cohort studies.

https://doi.org/10.1371/journal.pone.0288011.t001

QUADAS-2 assessment of the three cohort studies is summarized in Fig 2. For the patient selection domain, there was unclear risk of bias in two studies given the lack of detail on the process of patient selection. For the index test domain, all three studies had an unclear risk of bias as it was unclear from the study methods whether ultrasound (index test) was conducted before or after CT or MRI (reference standard). Risk of bias was low for the reference standard domain in two studies. The studies did not clearly state if the reference interpreters were aware of the index test. Risk of bias for ’flow and timing’ was judged to be high in one study and unclear in the other two. We did not notice any major applicability concerns in any of the studies.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Quality assessment of cohort studies.

https://doi.org/10.1371/journal.pone.0288011.g002

Case reports/series

There were 17 case reports/series including a total of 23 patients published between 1982 and 2021. Most reports were from hospitals in the USA (n = 8) and included patients ranging from birth to 83 years. A total of 11 males and 11 females were reported in the studies; the sex was not reported in one study. Although a direct comparison between ultrasound and other imaging modalities was not the objective of the studies, the authors report that most ultrasound findings were in agreement with cross-sectional imaging. The Chandler classification was not always specified; however, the authors did acknowledge that the ultrasound findings were sufficient to reach an immediate diagnosis. We have summarized the descriptive findings of case reports in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Summary of case reports.

https://doi.org/10.1371/journal.pone.0288011.t002

The critical appraisal of case reports revealed that most of the cases were well reported. The authors defined the patient demographics and described the diagnostic tests clearly. Only three of thirteen studies did not clearly describe the outcome of the patient post intervention. Critical appraisal of the four case-series identified major concerns. It was either unclear or the authors did not report if consecutive patients were included and case inclusion were unclear. However, patient demographics, measurement of the clinical condition, and follow up were clearly reported. The appraisal of the case reports and case-series are presented in Tables 3 and 4.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Joanna Briggs institute critical appraisal checklist for case reports.

https://doi.org/10.1371/journal.pone.0288011.t003

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Joanna Briggs institute critical appraisal checklist for case series.

https://doi.org/10.1371/journal.pone.0288011.t004

Guidance on ultrasound technique

Some of the studies reported the brand, model, and frequency of the transducer used to examine the patients, but it was difficult to compare or reach conclusions due to substantial variation across the ultrasound devices. Earlier studies do however suggest that a high-frequency (>10 Mhz) linear array transducer could be used to examine the orbital region [9, 10]. While the studies provided limited information on diagnostic performance, the authors report that most of the patients were examined in supine position without sedation. Some techniques were suggested that could guide better orbital assessment with ultrasound. For instance, the authors suggested to place the transducer on the closed upper eyelid using a non-irritating gel and to avoid any pressure on the eyeball and orbital tissues [10, 21, 22]. A transparent adhesive dressing (Tegaderm) could also be gently applied over the closed eye before applying the gel, ensuring no air is trapped between the plastic and the skin [22].

The medial orbital wall can be visualized by moving the transducer as far as possible toward the lateral canthus of the eye [10]. Another author pointed out that, while imaging the eye in the standard plane using POCUS, no obvious anomaly was detected. However, by rocking the probe medially and laterally and focusing on the nasal and temporal tissues additional views could be obtained. This alternating motion revealed prominence and edema of the nasal post septal tissue with echogenic, inflammatory fat in an adult female patient with orbital cellulitis [23]. For the transverse and longitudinal planes, the marker of the transducer can be oriented towards the right or head of the patient, respectively. The transducer can also be fanned back and forth for clear visibility of anterior and posterior anatomy of the eye. For assessing extraocular muscles, the patient can be asked to move their eye or fix their gaze on objects. The same study suggested that ocular POCUS could be performed while distracting the child by a video on the parent’s smartphone [22]. Studies have also suggested against use of color flow or doppler mode to limit the theoretical risk of damaging tissues [24, 25]. A table listing all the ultrasound techniques have been provided in the S5 Appendix.

Limitations

Our systematic review has several limitations. First, we had to exclude 37 studies that were awaiting classification as we could not obtain the full-text despite multiple attempts at contacting study authors and reference libraries. Most of these studies were completed several decades ago. Second, the exact timing of the reference standard test was often unclear given that most studies were not designed to compare ultrasound to other imaging modalities. Third, several included studies were decades old, and the imaging technologies might not be representative of those currently used. Fourth, many of the included studies were of low quality with a high risk of bias; no prospective, blinded, diagnostic accuracy studies were identified. However, such studies are feasible given similar studies in orbital ultrasound for retinal detachment in adult care [12, 40]. Studies on the application of ultrasound are susceptible to publication bias, as studies that report utility of a diagnostic tool are more likely to be published compared with studies that report it is not useful.

While the evidence generated from this review is limited, it is the first to systematically assess the benefit of ultrasound compared other imaging modalities to diagnose orbital cellulitis. Our findings support the need for future prospective observational studies or trials to evaluate the diagnostic test accuracy of ultrasound to diagnose and evaluate response to treatments in orbital cellulitis.