https://orcid.org/0000-0001-9932-5304
Abstract
Since the COVID-19 pandemic there has been a rapid uptake and utilisation of telemedicine in all aspects of healthcare. This presents a key opportunity in surgical site infection surveillance. Remote follow up methods have been used via telephone, with photographs and questionnaires for post-operative reviews with varying results. This review therefore aims to comprehensively synthesise available evidence for the diagnostic accuracy of all forms of SSI telemedicine monitoring. The protocol has been established as per both PRISMA-P (S1 Table) and the Cochrane handbook for reviews of diagnostic test accuracy. Medline, Embase, CENTRAL and CINAHL will be searched using a complete search strategy developed with librarian input, in addition to google scholar and hand searching. All study designs with patients over 18 and undergone a primarily closed surgical procedure will be eligible. Index tests will include all forms of telemedicine and a subgroup analysis performed for each of these. Comparative tests must include face to face review, and all reference standards will be included again for sub-group analyses. Search results will be screened by two investigators independently with a third providing consensus review on disagreements. Methodological quality will be assessed using the QUADAS-2 tool, first validated by two investigators as per the Cochrane handbook. Exploratory analysis will formulate summary receiver operating characteristic curves and forest plots with estimates of sensitivity and specificity of the included studies. Sources of heterogeneity will be identifying and investigated through further analysis. Potential benefits of telemedicine integration in surgical practice will reduce cost and travel time to patients in addition to avoiding wasted clinic appointments, important considerations in a peri-pandemic era. To avoid missed or further complications, there must be confidence in the ability to diagnose infection. This review will systematically determine whether telemedicine is accurate for surgical site infection diagnosis, which methods are well established and if further research is indicated.
Citation: Lathan R, Sidapra M, Yiasemidou M, Long J, Totty J, Smith G, et al. (2022) The diagnostic test accuracy of telemedicine for detection of surgical site infection: A systematic review protocol. PLoS ONE 17(11): e0263549. https://doi.org/10.1371/journal.pone.0263549
Editor: Vaikunthan Rajaratnam, Khoo Teck Puat Hospital, SINGAPORE
Received: January 20, 2022; Accepted: July 14, 2022; Published: November 17, 2022
Copyright: © 2022 Lathan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1. Background
1.1. Target condition being diagnosed
This review primarily aims to identify surgical site infections (SSI). The Centre for Disease Control and Prevention (CDC) defines SSI as an infection within 30 days of an operation or up to 90 days if an implant is left in place and the infection is related to an operative procedure [1]. SSI are further classified as ‘superficial incisional SSI’, ‘deep incisional SSI’, and ‘organ/space SSI’; further details of these definitions can be found in S1 File [2].
Surgical site infections complicate over 30% of operations, depending on the type of procedure [3, 4]. Up to 60% of these present after discharge and so accurate and timely diagnosis requires intensive follow-up and potentially significant travel distances on the patient’s behalf [5]. For patients, SSI may have a significant impact on morbidity and mortality with subsequent time and cost implications [6]. Unsurprisingly, the burden of infection encompasses healthcare providers too, with a recent UK study showing an association of SSI with a 92% increase in length of stay and an adjusted episode cost of £3040 [7]. Treatment for SSI can range from an oral course of antibiotics, to the need for reintervention (drainage or debridement), prolonged inpatient readmission and the subsequent risk of further morbidity (i.e. thrombotic and ischaemic events).
1.2. Index and gold standard tests
The current ‘gold standard’ for diagnosing SSI is a ‘face-to-face’ review using the US CDC criteria [2]. However, other scoring systems or criteria may be used during a face-to-face review. The ASEPSIS score uses weighted, objectively measurable criteria to identify surgical wounds as satisfactory healing, impaired wound healing or infected (S1 Table) [8]. A newly developed utility, the Bluebelle wound healing questionnaire (WHQ) has been validated in general surgery as a patient or clinician reported outcome measure to identify surgical site infection [9].
Diagnosis of SSI using remote, digitally based contact between patients and clinicians (telemedicine) is being investigated as the primary index test in this review. In this review we are focussing on digital remote follow-up. There are many ways of implementing this in practice. These include the use of photographic images and/or video, either in real time or deferred, telephone review and instant messaging. There may be other novel methods not listed here that are identified during the review process.
1.3. Clinical pathway
Patients undergoing surgery may present with SSI at one of three typical time points.
If an infection does occur, the first potential route of diagnosis is prior to discharge and within the first week. Infections at this point will depend on the surgical procedure, underlying comorbidities, and age, as these factors will influence typical length of stay (LOS) and wound healing. They are likely to be picked up on ward rounds or through dressing changes with the nursing team then highlighting issues to the surgeon. Face to face diagnosis using CDC criteria in this instance is straightforward as the patient has not left the department. The patient may have to undergo further observation, dressing changes and antibiotic course. Rarely, further imaging or intervention may be required.
The second point is typically patient initiated, after discharge but prior to any planned follow up, within 30 days of operation. Infections at this stage may not reach the surgeon’s knowledge as they are often managed by the patient’s primary care physician. However, without specialist input some infections progress in severity. Patients with evidence of deep incisional or organ/space infections may be referred to secondary care by the primary care team, may present to the emergency department or may contact the surgical team directly through an aftercare number. Only once seen can the gold standard assessment take place. Further imaging, microbiology culture and sensitivity testing are often implemented to ensure appropriate and specific management. Further surgical intervention may be required.
Finally, patients may not be contemporaneously identified as having SSI. Delayed or missed mild SSI diagnosis may present when the patient arrives in clinic for review (often after 30 days postoperatively, or if this is telephonic, it may be apparent through the history). Patients with more severe infection usually will have presented at the emergency department by this point, but if missed can impact on morbidity. Management will focus on ensuring no ongoing infection and alleviating further complications.
Index tests in this setting will likely be used as a comparative to face-to-face, gold standard review at planned review points. Potential implications of digital remote follow up are early and avoidance of missed diagnosis within the 30-day window for the CDC criteria.
1.4. Rationale
Widespread technological innovation and adoption have been exponential in the 21st century. Sophistication of the mobile phone now allows for instantaneous communication all over the planet. Users can even transfer image and video data in real time. In 2019, 88% of individuals in the UK were estimated to own a smartphone [10]. Naturally, the use of technology in healthcare has too progressed. Telemedicine is the remote diagnosis and treatment of patients by use of technology [11]. Coined in the 1970s, this concept has broadened with the advent of the smartphone and mobile data. Mobile health (mHealth) is a contemporary classification whereby healthcare is supported by the use of mobile devices [12]. The use of telemedicine has enabled patients in isolated centres access to specialist review through transfer of medical imaging, and teleconsultations are coming into practice [13]. In surgery, the process of postoperative care is changing with the introduction of telemedicine.
In 2020, the COVID-19 pandemic caused a countrywide lockdown in the UK, and many other countries all over the world. Elective operations were cancelled, expanding already lengthy waiting lists. This also posed a challenge for outpatient follow up, as patients requiring review would be at risk of COVID-19 in attending the hospital, and departments adapted to comply with social distancing regulations, limiting the number of people allowed in outpatient spaces at any one time.
Remote follow-up has the potential to reduce unnecessary clinic visits providing benefits for both patients and healthcare providers. The rationale of this review, therefore, is to synthesise the current available evidence for using telemedicine to diagnose or exclude SSI in the context of post-operative follow up.
1.5. Objectives
1.5.1. Research question.
Primary: Is digital remote follow-up accurate for the diagnosis of surgical site infection?
Secondary:
- What methods are used to facilitate SSI diagnosis?
- What are the limitations of telemedicine in SSI diagnosis?
This systematic review aims to assess the diagnostic accuracy of using telemedicine to identify SSI post-operatively. It will also aim to identify which methods are currently in use for this and any limitations of telemedicine methods in SSI diagnosis.
1.5.2. Objectives.
Primary: To determine the diagnostic accuracy of digital remote follow-up for the diagnosis of surgical site infection
Secondary objectives
- To determine what methods of digital remote follow-up have been used
- Evaluate the accuracy of different digital remote follow-up methods
- To determine limitations of digital remote follow-up
2. Methods
2.1. Protocol development
This protocol and review have been developed using the Cochrane handbook for reviews of diagnostic test accuracy and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements [14, 15]. In addition, this protocol is reported in line with the PRISMA statement for review protocols (PRISMA-P) which is attached in S1 Table [15].
2.2. Criteria for considering studies for this review
2.2.1. Types of studies.
There will be no restrictions on inclusion based upon prospective or retrospective study designs. Study designs which result in measures of test accuracy will be included, including randomised and observational studies. There will be no limitations on study sample sizes, or quality to thoroughly synthesise the available literature, but this will be accounted for in a quality of evidence assessment.
All study types will be included, and a sub group analysis performed for direct, fully paired and randomised studies.
Narrative and systematic review articles, letters and opinion pieces will be excluded from this review, however the reference lists of review articles will be hand searched for completeness.
2.2.2. Participants.
All patients over 18, who have undergone a procedure involving surgical incisions and followed up in a postoperative pathway will be eligible. Studies involving children under the age of 18 years and written in language other than English will be excluded from the review.
The study setting will vary depending on whether the index or reference test is being examined. The index tests can be performed anywhere as they are remote, the reference will be in a secondary care setting, likely clinics.
The index and reference tests will be applied on clinical assessment and on suspicion of surgical wound infection, no prior tests are required.
2.2.3. Index and comparative tests.
The index test to be reviewed is digital remote follow-up. Telemedicine is the remote diagnosis or treatment of patients using communications technology, which encompasses the term digital remote follow-up and will be used as a synonym in this review. Specifically, this entails the use of photographs or video, either deferred or in real time, telephone review and/or instant messaging. All index test methods will be evaluated through subgroup analyses.
The comparative tests are face to face review by a healthcare professional with the patient to directly observe the wound and obtain a history.
2.2.4. Target condition.
Surgical site infection as defined by CDC is the target condition for this review; infection within 30 days of surgery or within 90 days if an implant is left in place. This is further discussed in section 1.1. Some studies may categorise this further into superficial, deep and organ/space SSI (more details in S1 File).
2.2.5. Reference standards.
The same diagnostic criteria will be used for both tests. The gold standard for assessment of SSI is the US CDC criteria which clearly define indication of infection. Other possible methods are the ASEPSIS score with infection clearly define at a score of 21 or greater, and bluebelle WHQ. The latter has been suggested with an infection cut off score of 6–8. All methods of remote diagnosis will be extracted and a sub-group analysis performed for each type.
2.3. Search methods for identification of studies
2.3.1. Electronic searches.
Medline, Embase, CENTRAL and CINAHL databases will be searched from inception to the current date. Additional resources will be identified through google scholar. The search strategy has been developed in conjunction with an information specialist. An example search strategy for Medline or Embase is provided below;
(exp (Telemedicine)/ OR (remote consultation).mp OR (teleconsultation).mp OR (teleconsultation).mp OR (mobile health).mp OR (telehealth).mp OR (ehealth).mp OR (mhealth).mp OR (e*health).mp OR (e health).mp OR (m*health).mp OR (m health).mp OR (telephon*).mp OR (photograph*).mp OR (video*).mp OR (mobile app*).mp)
AND
(exp (surgical wound infection)/ OR (surgical wound dehiscence).mp OR surgical site infection).mp OR (postoperative infection).mp OR (SSI).mp OR (wound infection).mp OR (surgical wound complication).mp OR (post-surgical infection).mp OR (post operative infection).mp OR (post-operative infection).mp)
2.4. Data collection and analysis
2.4.1. Selection of studies.
Search results will be deduplicated and uploaded to the specialised online review tool, Rayyan. Study titles and abstracts will be screened by two investigators, RL and MS, independently. Any disagreement will be resolved by consensus decision from a third investigator. Articles included at this stage will be retrieved for full text screening, again by two authors acting independently. Those included after full text screening will go on to data extraction.
2.4.2. Data extraction and management.
Data extraction will be into a bespoke designed spreadsheet (Microsoft Excel), hosted remotely and updated in real-time. Information on study design, country of origin, participant age and gender, sample size, surgery performed, drop out rates, time to follow-up and type of remote follow-up (photograph / video / telephone / other) will be extracted. Details on infection rates for remote and face to face methods as well as sensitivity and specificity of diagnosis will also be extracted. If information is available on specifics of superficial, deep and organ/space SSI rates, these data will also be extracted.
2.4.3. Assessment of methodological quality.
Methodological quality will be assessed using the QUADAS-2 tool (S2 File and Table 1 within shows example tabulation for assessment of methodological quality.) [16]. Due to the nature of varying study design, systematic reviews of diagnostic test accuracy are often prone to heterogeneous results. In 2003, the quality assessment of diagnostic studies (QUADAS) tool was developed. This has since been revised as QUADAS-2 and is recommended for use in such reviews by the Agency for Healthcare Research and Quality, Cochrane Collaboration, and the UK National Institute for Health and Clinical Excellence [16].
Two authors will assess each manuscript as per the QADAS-2 tool, with a third independent author providing consensus review when discrepancies occur. Further details on the QUADAS-2 process can be found in S2 File, with an example of the data extraction process in Table 1.
2.4.4. Statistical analysis and data synthesis.
SSI diagnosed using any diagnostic criteria as part of a face-to-face review will be the reference standard. The patient will be the unit of analysis. Forest plots and receiver operating characteristic (ROC) curves will be produced as part of the initial, exploratory analysis and used to display estimates of sensitivity and specificity of the included studies.
Summary measures of sensitivity and specificity will be produced using a bivariate model for meta-analysis, if there are sufficient studies.
2.4.4.1 Receiver operator characteristic curves. Summary receiver operator characteristic (sROC) curves will be plotted using each study included as a data point. Confidence regions will also be calculated. Plots will be produced using metaDTA [17].
2.4.4.2 Sensitivity analysis. Sensitivity analysis based upon risk of bias will be evaluated. Studies with a high or unclear risk of bias identified by the QADAS-2 tool will be excluded in a separate analysis.
2.4.4.3 Assessment of heterogeneity. Several sources of potential heterogenetiy have been identified, and their effects will be investigated through the use of subgroup analysis and metaregression. The diagnostic criteria used to diagnose SSI (CDC, ASEPSIS, Bluebelle WHQ or others) may influence the test accuracy as different tools have been found to have poor correlation [18]. Other potential sources include the type of index test (subgroup analysis of photograph methods is planned) and the reviewer (i.e surgeon vs community healthcare worker).
3. Discussion
Integration of telemedicine has multiplied in recent years, exponentially so in response to the COVID-19 pandemic. One prospective application is remote diagnosis of SSI. Consolidation of telemedicine in surgical practice has the potential to reduce cost to both patient and care provider, as well as improved time implications for both parties. This would also reduce unnecessary visits to hospital clinic with a healthy surgical wound, an important consideration during a pandemic. To avoid further complications however and allow for confidence in diagnosis, telemedicine must be accurate in the detection of SSI. Previous studies have shown erythema detection to be difficult in review of wound images compare to face to face, which may influence a diagnosis of infection [19]. This review aims to comprehensively examine the accuracy for all methods of remote diagnosis of SSI thereby enabling evidence-based decision making on remote reviews of post-operative patients.
Supporting information
S1 Table. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: Recommended items to address in a systematic review protocol.
The PRISMA-P 2015 checklist for items to address in a systematic review protocol.
https://doi.org/10.1371/journal.pone.0263549.s001
(DOCX)
S2 Table. ASEPSIS score.
The ASEPSIS scoring method. A score of ≥21 equates to the presence of SSI.
https://doi.org/10.1371/journal.pone.0263549.s002
(DOCX)
S1 File. SSI classification.
SSI classification into superficial incisional, deep incisional, organ/space infection.
https://doi.org/10.1371/journal.pone.0263549.s003
(DOCX)
S2 File. QUADAS-2.
The QUADAS-2 tool for assessment of methodological quality of studies.
https://doi.org/10.1371/journal.pone.0263549.s004
(DOCX)
Acknowledgments
Identification: Protocol for systematic review of diagnostic test accuracy
Update: Primary review
Registration and no.: Prospero ID—CRD42021290610
References
- 1.
"Centres for Disease Control and Prevention (CDC). Surgical Site Infection (SSI) Event." https://www.cdc.gov/nhsn/pdfs/pscmanual/9pscssicurrent.pdf (accessed 21/10/2020.
- 2. Horan T. C., Gaynes R. P., Martone W. J., Jarvis W. R., and Emori T. G., "CDC definitions of nosocomial surgical site infections, 1992: a modification of CDC definitions of surgical wound infections," (in eng), Infect Control Hosp Epidemiol, vol. 13, no. 10, pp. 606–8, Oct 1992. pmid:1334988
- 3. GlobalSurg Collaborative, "Determining the worldwide epidemiology of surgical site infections after gastrointestinal resection surgery: protocol for a multicentre, international, prospective cohort study (GlobalSurg 2)," BMJ Open, vol. 7, e012150, p. e012150, 2017, pmid:28733294
- 4. Saeed MJ, Dubberke ER, Fraser VJ, and Olsen MA, "Procedure-specific Surgical Site Infection Incidence Varies Widely within Certain National Healthcare Safety Network Surgery Groups.," American Journal of Infection Control., vol. 43, no. (6), pp. 617–623, 2015, pmid:25818024
- 5. Woelber E, Schrick EJ, Gessner BD, and Evans HL, "Proportion of Surgical Site Infections Occuring after Hospital Discharge: A Systematic Review," Surgical Infections, vol. 17, no. 5, 2016, pmid:27463235
- 6. Astagneau P., Rioux C., Golliot F., and Brücker G., "Morbidity and mortality associated with surgical site infections: results from the 1997–1999 INCISO surveillance," (in eng), The Journal of hospital infection, vol. 48, no. 4, pp. 267–74, Aug 2001, pmid:11461127
- 7. Totty JP et al., "The impact of surgical site infection on hospitalisation, treatment costs, and health-related quality of life after vascular surgery," International Wound Journal, vol. 18, no. 3, pp. 261–268, 2020, pmid:33331066
- 8. Wilson A. P., Treasure T., Sturridge M. F., and Grüneberg R. N., "A scoring method (ASEPSIS) for postoperative wound infections for use in clinical trials of antibiotic prophylaxis," (in eng), Lancet, vol. 1, no. 8476, pp. 311–3, Feb 8 1986, pmid:2868173
- 9. "Validation of the Bluebelle Wound Healing Questionnaire for assessment of surgical-site infection in closed primary wounds after hospital discharge," (in eng), Br J Surg, vol. 106, no. 3, pp. 226–235, Feb 2019, pmid:30556594
- 10.
Deloitte. "Global Mobile Consumer Survey: UK cut." https://www2.deloitte.com/content/dam/Deloitte/uk/Documents/technology-media-telecommunications/deloitte-uk-plateauing-at-the-peak-the-state-of-the-smartphone.pdf (accessed 13/10/2020.
- 11.
WHO, "Telemedicine Opportunities and develpments in Member States," in "Global Observatory for eHealth series ", 2010, vol. 2. [Online]. Available: https://www.who.int/goe/publications/goe_telemedicine_2010.pdf.
- 12.
WHO, "mHealth: New horizons for health through mobile technologies.," in "Global Observatory for eHealth series," World Health Organisation, 2011, vol. 3. [Online]. Available: https://www.who.int/goe/publications/goe_mhealth_web.pdf?
- 13. Maani R, Camorlinga S, and Eskicioglu R, "A Remote Real-time PACS-based Platform for Medical Imaging Telemedicine.," Proceeding of SPIE, vol. 7264, 72640Q-1, 2009,
- 14.
Cochrane, "Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy," Cochrane Training, 2021. Accessed: 03/12/2022. [Online]. Available: https://training.cochrane.org/handbook-diagnostic-test-accuracy/PDF/v2.
- 15. Moher D. et al., "Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement," Systematic Reviews, vol. 4, no. 1, p. 1, 2015/01/01 2015, pmid:25554246
- 16. Whiting P. F. et al., "QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies," (in eng), Ann Intern Med, vol. 155, no. 8, pp. 529–36, Oct 18 2011, pmid:22007046
- 17. Freeman S. C., Kerby C. R., Patel A., Cooper N. J., Quinn T., and Sutton A. J., "Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA," BMC Medical Research Methodology, vol. 19, no. 1, p. 81, 2019/04/18 2019, pmid:30999861
- 18. A. P. Wilson et al., "Surgical wound infection as a performance indicator: agreement of common definitions of wound infection in 4773 patients," (in eng), Bmj, vol. 329, no. 7468, p. 720, Sep 25 2004, pmid:15367425
- 19. Totty J. P., Harwood A. E., Wallace T., Smith G. E., and Chetter I. C., "Use of photograph-based telemedicine in postoperative wound assessment to diagnose or exclude surgical site infection," Journal of wound care, vol. 27, no. 3, pp. 128–135, Mar 2018 2018. [Online]. Available: https://www.magonlinelibrary.com/doi/10.12968/jowc.2018.27.3.128 pmid:29509108