Inclusivity protocol

The bioinformatic analysis of the potential FNs is determined by the following inclusivity workflow:

1. The collection of gene sequences and whole genomes for M. pneumoniae, C. pneumoniae, L. pneumophila, and B. pertussis was obtained from the non-redundant nucleotide (nr/nt) collection database at National Center for Biotechnology Information (NCBI). The search was conducted annually for the following periods: 1 April 2021–25 May 2022; 26 May 2022–31 March 2023; 1 April 2023–31 March 2024; and 1 April 2024–31 March 2025. These data periods correspond to the retrieval intervals used for internal annual PMS reporting of the QIAstat Respiratory assay, aligned with the product launch date. Sequences originating from non-human hosts were excluded from further analysis, except for L. pneumophila, based on its zoonotic condition and its ecological association with protozoa—particularly amoebae—that serve as natural hosts and reservoirs (reviewed in Newton et al. [23]). Human infection occurs through inhalation of contaminated water aerosols from environmental sources, representing an indirect zoonotic transmission pathway involving non-human hosts.
2. The selected sequences for all four targets were mapped to the reference genomes of Mycoplasma pneumoniae (CP008895.1), Chlamydophila pneumoniae (LN846995.1), Legionella pneumophila (CP021256.1), and Bordetella pertussis (CP017405.1) using a high-sensitivity mapping algorithm implemented in the Geneious Prime software version 2025.2 (Biomatters, Ltd., Auckland, New Zealand, Geneious Prime User Manual). This mapping included the annotated oligonucleotides of the QIAstat-Dx® Respiratory SARS-CoV-2 Panel. To facilitate mapping, sequences were divided based on their length, such as whole genome sequences and gene sequences [21].
3. The output assembly files were manually curated in each case to avoid accidental gaps in assignment and incorrectly aligned insertions or deletions. Only unique sequences were retained.
4. To detect mismatches in the oligonucleotide binding regions for all primers and labelled probe sequences of the target-assays in focus, FASTA files were extracted from the specific binding regions. These files were then processed using a custom-written Python script. The script was designed to systematically analyse nucleotide alignments in oligonucleotide binding regions in collected sequences. First, input FASTA files were parsed and filtered to remove duplicate sequences or those containing undetermined nucleotides represented by IUPAC ambiguity code characters. Each sequence was then compared to the corresponding reference sequence on a position-by-position basis to identify nucleotide mismatches within primer and probe binding sites. Detected mismatches were encoded into standardized patterns, which were subsequently grouped and quantified across the dataset. The script further enabled aggregation of mutation frequencies and generation of structured output files to support downstream interpretation within the PMS workflow.
5. Following the methodology described in Kosińska-Selbi et al. [21], the melting temperature of primer–target and probe–target sequences containing one or more mismatches was estimated using a custom script based on the melting package version 1.8.0 from the Bioconductor Software Packages version 3.16 (www.bioconductor.org). The script processed mismatch patterns by integrating sequence and positional information for forward primers, probes, and reverse primers, and generated corresponding oligonucleotide–target sequence variants. These sequences were subsequently used as input for melting temperature estimation under predefined reaction conditions. The specific concentrations of salts, oligonucleotides, and nucleotides were adjusted according to the QIAstat-Dx® Respiratory SARS-CoV-2 Panel cartridge configuration to ensure consistency with assay conditions.
6. The impact of mismatches was evaluated according to Table 1:

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Impact of mismatches on primers and probes.

https://doi.org/10.1371/journal.pone.0349967.t001

For the assessment of melting temperature (Tm), primers are considered acceptable when the predicted Tm is equal to or above the nominal annealing temperature (Ta) used in the PCR thermal profile. A lower Tm may negatively affect primer binding, amplification efficiency, and assay sensitivity, particularly near the Limit of Detection (LoD). Primers with a predicted Tm up to 2 °C below the nominal Ta may be accepted as a controlled tolerance, provided that no additional high-risk features are present and that the assay meets predefined performance requirements. This threshold reflects the uncertainty of Tm prediction and should not be considered an automatic rejection criterion. However, differences greater than 2 °C may be critical and require validation under laboratory conditions to ensure assay performance [24].

1. 7. The frequency of observed mismatches was calculated as the percentage of sequences that contained a mismatch relative to the total number of collected sequences covering the target region. Variations were considered high risk when located at the 3′ end of primers or the 5′ end of probes [18]. Threshold of three total mismatches or a single mismatch in first three bases was applied as a conservative criterion to identify potentially critical patterns.

Cross-reactivity protocol

The detection of potential FPs was conducted by evaluating the sequence homology using the Basic Local Alignment Search Tool (BLAST) against the NCBI database (https://blast.ncbi.nlm.nih.gov/Blast.cgi), optimized for somewhat similar sequences (blastn):

1. 1) The primer or probe sequence was used as the query in the blastn algorithm.
2. 2) The search was restricted to the following periods: 1 April 2021–25 May 2022; 26 May 2022–31 March 2023; 1 April 2023–31 March 2024; and 1 April 2024–31 March 2025“. Additionally, the search excluded target organisms (Mycoplasma pneumoniae: taxid 2104; Chlamydophila pneumoniae: taxid 83558 and 51291; Legionella pneumophila: taxid 446; and Bordetella pertussis: taxid 520), as well as vectors, uncultured and environmental samples sequences, and synthetic sequences.
3. 3) Algorithm parameters were set to a maximum of 5,000 hits using the default scoring parameters.
4. 4) Hits with less than 70% total homology were filtered out, where total homology refers to the combination of the Query Cover and % Identity parameters of the BLAST results.
5. 5) Potential false positives were identified by detecting unique sequences matching the forward and reverse primers, and the labelled probe from the same PCR assay.
6. 6) The following factors were considered in the assessment:
   • The likelihood of the organism’s DNA being present in human samples.
   • Potential amplicon size (<= 500 bp may indicate a potential false positive).
   • Occurrence of critical mismatches in the assay’s oligonucleotide binding sites. Sequences with critical mismatches (as defined in Table 1) were not considered potential FPs.

Literature review using Artificial Intelligence (AI) tool

Since 2024, an additional literature review has been conducted for pathogens with fewer than 100 newly added sequences covering the assay-specific target regions during a defined surveillance period, to ensure robust evaluation QIAstat-Dx® performance. In the current study, such limited sequence availability was observed for C. pneumoniae and L. pneumophila.

The Huma.ai Platform, a literature search framework jointly developed with Huma.ai, was employed for literature collection [22]. This advanced tool utilizes a multi-algorithmic framework that integrates sophisticated caching techniques with a natural language processing (NLP) system known as the Utterance Processor (UP). UP interprets user queries and dynamically constructs a pipeline of operations to retrieve and analyze relevant scientific literature. Unlike traditional keyword-based searches, the Huma.ai Platform performs full-text analysis and identifies specific regions of interest (ROIs) within articles, significantly improving the relevance and precision of the results.

The decision to use the Huma.ai Platform was based on its demonstrated superiority over manual search methods. The platform consistently outperformed manual approaches in both the number of relevant articles identified and the consistency of yearly precision rates. It also significantly reduced the time and effort required for literature screening and analysis [22].

Databases such as PubMed Central (PMC), PubMed, Embase, and ScienceDirect were searched to retrieve information relevant to product performance, including records on genetic variability in the gene target region and descriptions of newly identified strainsor genotypes of the pathogens. The resulting articles were compared across databases, and duplicate entries across databases were excluded from further assessment. ROIs were defined as sections of the publications that included descriptions of genetic variations within the assay’s target regions. These ROIs were extracted from the full text to capture information relevant for evaluating potential impact on assay performance. Sequences obtained from the selected literature were incorporated into the dataset used for the PMS analyses.

Inclusivity analysis

A bioinformatic analysis of oligonucleotide binding site integrity was conducted across four reporting periods. The results showed no major issues or significant concerns. Only a few genomic patterns involving single nucleotide changes were detected, and these occurred at non-critical positions that are not expected to affect PCR performance (see Table 2).

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Summary of potential FN results across reporting periods for atypical respiratory pathogens.

https://doi.org/10.1371/journal.pone.0349967.t002

One exception was observed during the 2021–2022 period: a critical single nucleotide mismatch was found in the binding region of the reverse primer of the B. pertussis PCR assay. This mismatch, associated with the MJBH01000027.1 sequence (https://www.ebi.ac.uk/ena/browser/view/PRJNA224116), had a frequency of 0.74%. However, the high copy number of IS481in the Bordetella genome likely mitigates the potential impact of this pattern. In addition, this mutation has not been observed in subsequent years and appears to be an isolated event that has not propagated over time. Between April 2021 and March 2025, the analysis of M. pneumoniae, C. pneumoniae, L. pneumophila, and B. pertussis confirmed the robust inclusivity of the QIAstat-Dx® Respiratory SARS-CoV-2 Panel. These findings support the continued reliability of the panel, with no significant impact on performance. A summary is provided in Table 2, with full details available in S1 Data.

Zero values in 3^rd column of Table 2 mean that, for the given organism and surveillance period, no publicly available sequences covering the assay target region were found using the predefined search and inclusion criteria. In such instances zero values in subsequent columns not mean that genetic variability was absent, but rather that suitable public sequence data were not available for that period. To reduce the potential bias caused by limited sequence availability, the sequence-based analysis was complemented with an AI-assisted literature review for targets with low sequence coverage. In addition, the surveillance analysis is repeated in consecutive PMS reporting periods to monitor newly available data over time.

Cross-reactivity analysis

The BLAST-based analyses conducted across four surveillance periods (April 2021–March 2025) to evaluate potential unexpected cross-reactivity of M. pneumoniae, C. pneumoniae, and L. pneumophila PCR assays included in the QIAstat-Dx® Respiratory SARS-CoV-2 Panel revealed no genetic changes producing potential false-positive results (Table 3). Although multiple sequences were initially identified during the homology analysis (S2 Data), the majority either exceeded the allowed amplicon size (500 bp) or belonged to viruses not associated with human infection. On the other hand, for Bordetella pertussis, no unexpected sequences were detected, except for two B. holmesii genomes identified in the 2024–2025 period – a known cross-reactive species.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Summary of potential false positive results across reporting periods for atypical respiratory pathogens.

https://doi.org/10.1371/journal.pone.0349967.t003

Enhancing inclusivity analysis through AI-driven literature review

To complement the sequence analysis for C. pneumoniae and L. pneumophila, a literature search was conducted using the Huma.ai Platform. Articles identified by the AI tool are provided in S3 Data.

For L. pneumophila, nine publications were retrieved: one from Embase, four from PMC, and four from ScienceDirect. Of these, six were considered potentially relevant based on the ROI extracted by the Huma.ai Platform, resulting in a precision rate of 66.67%. Full-text analysis revealed that all contained information about mutations in the assay target gene. However, none of the publications reported genomic variability that could impact QIAstat-Dx® performance.

For C. pneumoniae, 35 publications were retrieved: 33 from PMC and two from the ScienceDirect database. Of these, 22 were considered potentially relevant based on ROI, yielding a precision rate of 62.86%. Full-text analysis revealed that 18 publications contained information on genetic variability in the assay target gene. No publication reported a new C. pneumoniae strain or genotype, and none contained information that could impact QIAstat-Dx® assay performance.

In summary, the Huma.ai Platform successfully collected critical information to complement the sequence inquest inclusivity analysis – information that would not have been captured if the analysis had relied solely on database sequences. No additional findings indicating potential safety or performance concerns for the QIAstat-Dx® assay due to genetic variability in the target pathogens were identified, confirming its reliable performance.

Clinical utility and rationale for continuous PMS

As discussed in the introduction, the post-pandemic period has been associated with increased circulation of atypical bacterial respiratory pathogens, particularly Mycoplasma pneumoniae [8]. This broader epidemiological trend is also reflected in real-world diagnostic data generated by routine syndromic testing. Analysis of the internally monitored QIAstat-Dx® Respiratory SARS-CoV-2 Panel cartridge results collected via the QIAsphere® platform indicates a substantial increase in bacterial detections over time, rising from below 1% of positive cases in 2022 to over 10% by 2024 [7]. Together, these observations underline the growing clinical relevance of accurate molecular identification of atypical bacterial infections and further support the importance of continuous post-market surveillance to ensure sustained diagnostic performance as pathogen circulation patterns change.

Reliable detection of atypical bacterial respiratory pathogens has direct clinical and public health implications because these infections can present with non-specific symptoms and require targeted diagnostic confirmation to support timely management and appropriate antimicrobial selection. In the post-pandemic period, multiple respiratory pathogens have shown changes in circulation patterns, including re-emergence of M. pneumoniae and increased B. pertussis activity in several settings, reinforcing the need to ensure that widely used molecular diagnostics remain fit-for-purpose as pathogen epidemiology shifts [25,26]. In this context, a robust post-market surveillance system provides an evidence-based mechanism to detect early signals of potential assay vulnerability (e.g., emerging mismatches in oligonucleotide binding regions or novel cross-reactive sequences), thereby reducing the risk of undetected loss of sensitivity or specificity during periods of heightened clinical need.

Inclusivity analysis

Overall, the inclusivity assessment identified only limited binding-region variability and no evidence of a sustained, high-risk drift that would be expected to compromise assay detection. The most relevant signal was observed for Bordetella pertussis in the earliest surveillance window, where a single, low-frequency (<1%) mismatch pattern classified as potentially critical occurred in a reverse-primer binding region (MJBH01000027.1). Given its single occurrence, it remains unclear whether this represents a true biological variant or a sequencing artifact; moreover, even if genuine, the multi-copy nature of the IS481 target provides a plausible buffering context that may reduce the impact of isolated primer-site events on overall detection. Low-frequency variation in B. pertussis has been reported previously and contributes to genomic diversity that can be relevant for diagnostic performance [27].

For Mycoplasma pneumoniae, variability within the evaluated oligonucleotide binding region remained non-critical according to the applied criteria, including in the most recent dataset where a higher proportion of sequences carried a non-critical pattern. Such observations are compatible with ongoing genetic variability and align with prior reports discussing the diagnostic impact of point mutations in M. pneumoniae [28]. At the same time, fluctuations in publicly available sequence volumes and submission practices can influence apparent frequency changes; therefore, these observations reinforce the value of continuous monitoring rather than indicating a confirmed shift in assay vulnerability.

In the most recent window, B. pertussis also showed renewed non-critical variation in probe and primer binding regions. Continued monitoring is warranted, consistent with recent reports describing macrolide resistance and primer mismatches in circulating B. pertussis strains [29]. In contrast, no inclusivity-relevant signals were identified for Chlamydophila pneumoniae or Legionella pneumophila in the available datasets, and this absence of findings was corroborated through the complementary literature analysis using the Huma.ai platform. Collectively, these results emphasize that longitudinal PMS focused on oligonucleotide binding integrity remains essential to detect early signals should mutations emerge in critical positions and expand over time.

Literature review and AI integration

For pathogens with insufficient sequence data during the last surveillance period inclusivity analysis was supplemented by literature searches using the Huma.ai platform – jointly developed by QIAGEN and Huma.ai. The tool’s performance was previously validated by Reniewicz et al. [22], demonstrating superior results compared to manual searches in terms of article retrieval, precision, and time efficiency. In this study, platform access was expanded to include two additional databases: Embase and ScienceDirect. Precision rates for C. pneumoniae and L. pneumophila (62.86% and 66.67%, respectively) were comparable to those reported by Reniewicz et al. [22] (65.14% ± 2.07% for technical assessment of the NeuMoDx HIV-1 Assay). No safety or performance concerns for QIAstat-Dx® assays were identified, corroborating sequence-based findings. Although recent studies have explored AI applications in post-market surveillance for medical devices [30,31], the current study and Reniewicz et al. [22] remain, to our knowledge, the only reports demonstrating successful NLP-based literature review for in vitro diagnostics. Consequently, direct comparisons between Huma.ai and other tools are challenging due to the lack of equivalent studies.

Cross-reactivity analysis

Cross-reactivity was evaluated across four surveillance periods using BLAST-based analysis for each target organism. During the first three intervals (April 2021–March 2024), no potential false positives, defined as unexpected off‑target organisms, were detected, supporting high analytical specificity. In the final surveillance period (April 2024–March 2025), two B. holmesii sequences were identified for the B. pertussis target. These sequences contained the IS481 insertion sequence, which is shared among multiple Bordetella species, and this finding is consistent with the QIAstat‑Dx® specifications acknowledging potential IS481‑related cross‑reactivity (QIAstat‑Dx; QIAGEN® GmbH, Hilden, Germany). Because IS481 is not species‑specific, detection signals compatible with B. pertussis may also reflect other IS481‑carrying Bordetella species, such as B. holmesii, as previously reported in nasopharyngeal specimens from patients investigated for pertussis [12]. Large‑scale evaluations of IS481‑based PCR diagnostics have shown that, despite this lack of species‑level discrimination, overall clinical reliability remains high. In a U.S. multicenter study of nearly 4,000 IS481‑positive specimens, B. holmesii accounted for only approximately 1–2% of cases, with overall agreement between commercial assays and reference testing exceeding 85% [32]. While B. pertussis remains the primary etiologic agent of pertussis, occasional misclassification between B. pertussis and B. holmesii may have clinical and public health implications, including differences in expected antimicrobial susceptibility and downstream case‑management measures such as post‑exposure prophylaxis and infection‑control actions. Therefore, results for Bordetella targets should be interpreted in the context of known IS‑element sharing across species and, where appropriate, complemented by methods enabling species‑level discrimination. Overall, these findings confirm the assay’s specificity while highlighting the importance of continuous post‑market surveillance to monitor known and emerging cross‑reactivity patterns.