Introduction

Olfactory dysfunction has emerged as a common symptom in COVID-19. Persistent symptoms can result in a decline of quality of life, affecting nutritional, physical well-being and cognitive functioning [1–4]. Thus, assessing olfactory function accurately and efficiently is essential. Not only for clinical diagnosis and understanding the underlying mechanisms of olfactory dysfunction in COVID-19 patients, but it also enables healthcare professionals to provide patients with objective measurements of symptom severity, and guidance throughout their recovery.

Merely 10% of ENT-surgeons utilizes psychophysical tests for the evaluation of olfaction [5]. Most ENT-surgeons utilize subjective questionnaires, but this approach often yields inconsistent results and potentially leads to underestimation of the extent of the problem [5]. The most utilized psychophysical test in Europe is the validated Sniffin’ Sticks Test® (SST) [5, 6]. The SST provides a comprehensive evaluation of olfactory function, including the ability to identify specific odors, discriminate between different odors, and detect odor thresholds. These type of extended tests are the gold standard for diagnosing olfactory disorders [5, 7]. However, the time-consuming nature of the SST (around 30 to 60 minutes) [8, 9] and the need for sustained concentration from patients make it less suitable for routine clinical practice [10, 11].

In response to the need for more time-efficient olfactory tests, a screening version of the SST, known as the SST-12, has been developed [12]. The SST-12 serves as a diagnostic tool for screening olfaction in causes unrelated to COVID-19. As only 12 scents have to be identified, the test can be done in 5 minutes [13]. The SST-12 focuses solely on the identification subdomain, providing a quick assessment of an individual’s ability to identify twelve odors [14–16]. As the pathophysiology of smell loss varies between COVID-19 and other causes, particularly impacting the threshold domain [17, 18], we explored the SST-12’s ability to also detect smell loss in patients with COVID-19 as the plausible cause [13].

Methods

Patients and procedures

In order to assess the diagnostic accuracy of the SST-12, we included 113 consecutive patients. The cohort of participants included in this study originated from the COCOS trial [19]. This was a randomized controlled trial determining the possible benefit of an oral prednisolone treatment (10 days 40mg) on the olfactory function in patients with COVID-19 induced smell disorders. The institutional Review Board of the University Medical Center Utrecht approved the research protocol (21-635/G-D, October 2021). We obtained from all patients written informed consent in order to participate. Patients were recruited via the Dutch media and via the National patients association, between November 2021 and January 2022.

[20] Patients approached us by email and consecutive eligible patients were planned for inclusion by telephone. Inclusion criteria were adult patients with good understanding of the Dutch language, with a PCR confirmed COVID-19 diagnosis within 12 weeks before inclusion, and at least 4 weeks of smell loss since their diagnosis. Patients visit the outpatient clinic for Ear, Nose and Throat at baseline for the assessment of the SST and thereby objectify their smell loss. They were excluded when we objectified no smell loss at baseline with SST (TDI score >30.5), or when we found other causes for smell loss objectified by a nasendoscopy, such as nasal polyps or (rhino)sinusitis (Fig 1), or pre-existing smell loss before the COVID-19 diagnosis. These aforementioned criteria make loss of smell due to COVID-19 the most plausible cause. All patients signed informed consent in order to participate. They all performed the Sniffin’ Sticks test again at approximately 6 months after their diagnosis (Fig 1) [20]. Results showed no difference in olfactory function between patients who received prednisolone and those who received placebo [21]. More information about the inclusion- and exclusion criteria and the study procedures of the RCT are described elsewhere [20, 21]. In our analysis, we utilized the results of the SST during the visit approximately 6 months after diagnosis. The reason behind selecting this specific time point is that predictive values are influenced by the prevalence of the disease [22]. By choosing the second visit, we aimed to achieve a well-balanced representation of smell loss prevalence [23, 24].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Procedures and participant flow-chart.

https://doi.org/10.1371/journal.pone.0295911.g001

Results

Table 1 describes the characteristics and outcome measurements of the SST. Median time in days between conformed COVID-19 and the utilized tests for this analysis is 140 (IQR 128–154). Median TDI score on SST was 27.5 (IQR 23.63–30.0).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics and outcome measurements at the moment of performing the SST.

Data are presented as median (IQR) or n (%).

https://doi.org/10.1371/journal.pone.0295911.t001

Table 2 presents the outcomes of the 2 by 2 table analysis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Cross-tabulation.

https://doi.org/10.1371/journal.pone.0295911.t002

Sensitivity was 0.934 (CI 0.87–0.97). Among the 91 individuals with smell loss regarding the reference standard, 85 (93.4%) participants were detected with a positive test result on the SST-12. Specificity was 0.682 (CI 0.48–0.85). Among the 22 individuals without smell loss, 15 (68.2%) participants were ruled out to have smell loss. The PPV of the SST-12 was calculated as 0.924 (CI 0.86–0.97). Among the 92 individuals with a positive test result, 85 (92.4%) participants did have smell loss based on the reference standard. The NPV of the SST-12 was calculated as 0.714 (CI 0.50–0.88). Among the 21 individuals with a negative test result, 15 (71.4%) participants did not have smell loss based on the reference standard.

Discussion

The aim of this study was to identify the diagnostic accuracy of the SST-12 for COVID-19 induced loss of smell. We found a high PPV 92.4% (CI 0.86–0.97) and sensitivity 93.4% (CI 0.87–0.97), and an acceptable NPV 71.4% (CI 0.50–0.88) and specificity 68.2% (CI 0.48–0.85).

These findings were achieved by a comprehensive cohort design which included a large sample size of consecutive patients enrolled within a concise period. All patients had a confirmed COVID-19 diagnosis by PCR in the same timeframe since performing the SST. Besides we used a predetermined threshold, which contributes in the validity and generatability of the results. To the best of our knowledge, Vandersteen et al. performed the only study that included patients with post COVID-19 olfactory dysfunction to investigate the diagnostic accuracy of the SST-12, but their small sample size and wide confidence intervals raise uncertainty about the applicability of their results [13]. Hummel et al investigated the diagnostic accuracy of the SST-12 and of the Q-sticks (a three-odor test), but in non-COVID-19 related cases [14, 29]. Our results, however are comparable to theirs.

It is important to note that as we followed the typical sequence for all subdomains, from threshold to discrimination and identification, some patients may have experienced decreased concentration during the final identification test. In the SST-12, patients only perform the identification part. While this may yield different results compared to the extended SST’s identification section, as all patients underwent this procedure, we do not expect it to significantly impact our findings.

The high sensitivity in our study indicates that among patients with smell loss, a high number of patients will indeed receive a positive test result, resulting in few false negatives. If the SST-12 had been used in this study, six patients (6.6%) would have received a false-negative test result. The high sensitivity value suggests that the SST-12 is effective in correctly identifying individuals with smell loss and minimizing false-negative test outcomes. However, in the study of Sorokowska et al. there was a high number of false-negatives, but this contradiction in comparison with our study can be found in the fact that they used the Q-sticks, and because they did not use the SST as reference standard [6] (Sorokowska et al., 2019).

We found a moderate specificity, indicating that the SST-12 is amendable for obtaining false positives. In this study, if the SST-12 had been used instead of the SST, seven patients (32%) would have received a false-positive test result. Though, the interpretation and usefulness of any diagnostic test is dependent on the setting in which it is used. Most smell tests will be used in clinical settings, in patients in which more objective knowledge about their ability to smell can be crucial for provide guidance throughout their recovery trajectory. The consequences of the high sensitivity in combination with the relatively low specificity is a possibility of detecting a relative high number of false positives. There are however no harmful or expensive follow-up diagnostic tools or unnecessary treatment options for smell loss when patients test false-positive.

The moderate NPV we found, suggests that there is a possibility of missing the diagnosis. This could be attributed to the fact that the SST-12 only assesses the identification ability, while in COVID-19 patients the threshold domain seems most affected [17, 18]. The reason for this is the fact that of the olfactory threshold assessment primarily relies on the peripheral olfactory system, specifically the olfactory epithelium, which is the part most accessible to the SARS-CoV-2, while the identification and discrimination components are more closely associated with higher-level cognitive processes [30, 31]. These patients may perform well on the SST-12, but still have COVID-19-induced loss of smell, which can only be assessed by the SST. Since COVID-19 induced smell disorders can have a significant impact on individuals’ quality of life, it is advisable to combine the SST-12 with other clinical information to make an accurate diagnosis. In cases where the disease is suspected, even if the SST-12 is negative, additional testing may be necessary. Alternatively, healthcare workers can still provide guidance and support to help patients manage their complaints. The high PPV we found in combination with the high sensitivity makes the SST-12 especially helpful for identifying smell loss. Considering the diagnostic accuracy of the SST-12, it has the potential to aid in early detection and in monitoring disease progression. In general practice the SST-12 will mostly be used for counseling, and to follow the trajectory of the smell function since patients cannot objectify the vague improvement of their smell themselves. Understanding the limitations and potential false results of the test is relevant for managing patient expectations and ensuring appropriate counseling.

Conclusion

Our findings suggest that the SST-12 holds promise as a screenings tool in identifying smell loss, also in patients with COVID-19 as the most plausible cause.

Acknowledgments

The study team thanks and acknowledges all patients for their participation in this study. Special thanks to C. de Haas, K. Krommenhoek and N. Maagdenberg for their help with conducting the psychophysical tests.