Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Estimating post-operative complication rates in patients with primary brain tumours from routine administrative data: A national cohort study

  • Radvile Mauricaite,

    Roles Formal analysis, Investigation, Writing – original draft, Writing – review & editing

    Affiliations Department of Radiotherapy, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, United Kingdom, Computational Oncology Laboratory, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

  • Alex Bottle,

    Roles Methodology, Writing – review & editing

    Affiliation School of Public Health, Imperial College London, London, United Kingdom

  • Andrew Brodbelt,

    Roles Conceptualization, Validation, Writing – review & editing

    Affiliation The Walton Centre NHS Foundation Trust, Lower Lane Liverpool, United Kingdom

  • Kerlann Le Calvez,

    Roles Conceptualization, Data curation, Methodology, Supervision, Writing – review & editing

    Affiliations Department of Radiotherapy, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, United Kingdom, Computational Oncology Laboratory, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

  • Peter Treasure,

    Roles Methodology, Writing – review & editing

    Affiliation Peter Treasure Statistical Services Ltd, King’s Lynn, United Kingdom

  • Stephen J. Price,

    Roles Validation, Writing – review & editing

    Affiliation Division of Neurosurgery, Department of Clinical Neurosciences, University of Cambridge, Cambridge, United Kingdom

  • Thomas C. Booth,

    Roles Conceptualization, Writing – original draft, Writing – review & editing

    Affiliations Department of Neuroradiology, Ruskin Wing, King’s College Hospital NHS Foundation Trust, London, United Kingdom, School of Biomedical Engineering & Imaging Sciences, King’s College London, London, United Kingdom

  • Seema Dadhania,

    Roles Validation

    Affiliations Department of Radiotherapy, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, United Kingdom, Computational Oncology Laboratory, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

  • Jonathan J. Gregory,

    Roles Conceptualization, Validation, Writing – review & editing

    Affiliation Computational Oncology Laboratory, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

  • Maureen Dumba,

    Roles Validation

    Affiliation Department of Neuroradiology, National Hospital for Neurology and Neurosurgery, University College London NHS Foundation Trust, London, United Kingdom

  • Joanne Droney,

    Roles Validation

    Affiliations Symptom Control and Palliative Care Team, The Royal Marsden NHS Foundation Trust, London, United Kingdom, Department of Surgery & Cancer (Honorary Clinical Senior Lecturer), Imperial College London, London, United Kingdom

  • Jawad Basharat,

    Roles Validation

    Affiliation Department of Clinical Coding, Imperial College Healthcare NHS Trust, London, United Kingdom

  • Matt Williams

    Roles Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing

    matthew.williams@imperial.ac.uk

    Affiliations Department of Radiotherapy, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, United Kingdom, Computational Oncology Laboratory, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

Abstract

Introduction

Neurosurgery is an important element of brain tumour treatment but carries with it the risk of complications. Previous work has defined a narrow set of general post-operative complications which are used as Patient Safety Indicators (PSIs), but these are not brain tumour specific and do not capture the full range of complications. As a result, there is no way of measuring post-operative complications in neurosurgery at a national level.

Methods

We conducted a retrospective, observational cohort study using a comprehensive national administrative dataset from England on adult brain or spinal tumour patients to better define post-operative complications. We generated and validated a new list of post-operative complications – ICL list. The ICL list contains codes selected specifically from our Gliocova dataset combined with general OECD-defined PSI list. The ICL list is novel as it can assess brain tumour patient complications using an administrative dataset at a national level and captures more specific brain tumour related complications.

Results

In our study, 30-day readmission after surgery was 12.7% and 30-day mortality was 2.3%. The ICL list of complications identified many more patients with complications (N = 3,274 (11.3%)) compared to OECD-defined PSI list (N = 568 (2.0%)) without reducing model performance. 30-day mortality was 6.5% in those with complications and 1.8% in those without.

Discussion

We have identified a much wider set of complications than the OECD-defined PSIs and shown that patients developing these have worse outcomes than those without complications. This enables us to estimate the risk of post-operative complications in brain tumour patients using national administrative data. It forms the basis for planned further work, allowing us to explore the predictors of and consequences of post-operative complications.

Introduction

Primary Brain and Central Nervous System (CNS) tumours are a group of rare, heterogenous diseases, including benign and malignant tumours, most of which occur in the brain rather than the spine. The commonest tumours are Glioblastoma (GBM; WHO Grade 4 glioma) and meningioma, which between them account for >50% of all adult brain tumours. GBM is an aggressive, malignant tumour, whereas meningiomas are largely benign. The treatment pathways and outcomes are thus very different.

Neurosurgery is a key treatment in brain tumour patients, but it is not without risk. Neurosurgery is very centralised in England, and England has a robust national cancer data infrastructure, which holds data on pathology, treatment, admissions and survival [1]. Post-surgical death is an obvious post-operative complication, but since mortality rates are low, it is not a sensitive indicator of quality [2]. Both the Organisation for Economic Co-operations and Development (OECD) and the US Agency for Healthcare Research and Quality (AHRQ) have developed Patient Safety Indicators (PSI), which define a small number of key diagnoses (e.g., post-operative Pulmonary Embolus) [3,4]. However, these are a very restricted set of complications, meant to be used as quality indicators across wide range of surgical practice, and to allow for trans-national comparisons of patient safety, rather than as a measure of the rate of complications. In addition, not all AHRQ PSI can be translated to the ICD-10-WHO system used in the UK [5], and the OECD-defined PSI list does not have any CNS specific indicators [6]. Research on patient safety and complications in brain tumour patients is limited, and most of it comes from the US [710]. Previous work from Europe and the UK includes the development and validation of a Post-operative Morbidity Survey for patients undergoing cranial neurosurgery [11] and complications after neurosurgery using either Landriel or Clavien-Dindo classifications [12,13]. However, these require review of clinical notes and patient examination or questionnaires, which limits the feasibility of conducting such work at scale. As a consequence, we currently have no way of measuring complications (outside of a very narrow set of PSI) at scale at a national level. This makes it impossible, for example, to even count the number of patients who have post-operative complications.

Large administrative datasets have been used to assess outcomes after procedures such as hip replacement and cardiac surgery [14]. Such datasets allow large scale assessment of outcomes and co-morbidities [15,16]. However, assessing the validity of the new definition of complications is challenging: by definition, we cannot conduct patient-level notes review. We are engaged in a trade-off: counting every post-operative diagnosis as a complication will capture too many patients; counting only the PSI indicators captures too few. One approach is examine plausible downstream effects of complications (such as mortality and length of stay). If the performance of statistical models stays the same as we expand the list of complications, then it suggests that the expanded list is valid. In this work, we use data from the GlioCova project to define a new set of post-operative complications (ICL list) and examine the relationship between our novel definitions of complications and 30-day mortality and length of stay in adult patients undergoing first surgical intervention for a primary CNS tumour [17]. Our key aim is to be able to estimate the number of complications; we validate this by assessing whether models constructed using the newly defined ICL set of complications have similar performance to models constructed using existing (PSI) definitions of complications. We use the ICL set of complications to estimate, for the first time, rates of complications in a large, complete, multi-year national cohort of primary brain tumour patients undergoing neurosurgery in England.

Methods

Data

The GlioCova project holds data on all adults with brain tumours diagnosed in England from 2013–2018, with treatment and survival data available until August 2020. Data was extracted on the 10th of August 2020, with mortality data (= survival) censored in October 2022. It contains linked pseudo-anonymised data on patients’ demographics, surgery, chemotherapy, radiotherapy, hospital admissions, imaging and death. Authors had no access to information that could identify individual participants during or after data collection. Data also includes deprivation status based on the Index of Multiple Deprivation (IMD). This is calculated based on postcode as a marker of socio-economic status at a small local area level across England and grouped into quintiles with IMD quintile 1 representing areas with lowest deprivation and quintile 5 areas with most deprivation [18]. Analysis is supported by the Expert Advisory Group (EAG), a multi-disciplinary group of data analysts, clinicians, patients, carers and charities. More details on ethical approval and coding systems used can be accessed in S1 Appendix.

Patient cohort

We identified all patients with a primary brain tumour who underwent surgical resection or biopsy (“surgical intervention”) (see S2 Table), and we excluded those who had a diagnosis of another cancer. Patients who were recorded as having a surgical resection and biopsy on the same day were considered to have a surgical resection. We only considered the first admission for surgical intervention for each patient (index admission). We excluded patients who had a clearly erroneous (> −5000 days) interval from diagnosis to procedure and removed duplicated admissions in 39 patients, retaining the admission with the shortest length of stay (LOS). The remaining patients were our analytical cohort. Although we considered the whole cohort, we explored rates of outcomes in patients with meningioma and glioblastoma as clinical exemplars.

Defining the ICL post-surgical complication list

We used the list of diagnosis codes from the 2018 OECD Patient Safety Questionnaire [6], excluding those related to obstetric trauma (see S3 Table). This is the OECD-defined PSI list. We separately identified all diagnoses from the GlioCova dataset during the first intervention, excluded brain/spine tumour diagnoses and those relating to chronic illnesses (e.g., hypertension, asthma), and selected the 100 most common remaining diagnoses after these exclusions (see S4 Table).

We used a modified Delphi [19] approach starting with the 100 most common diagnoses and three review rounds to refine the set of potential post-surgical complications as defined below (see S5 Fig). Diagnoses were reviewed by a group of five experienced attending clinicians (of whom at least 4 have more than 5-years of experience as attending consultants) who reviewed the diagnoses and selected potential complications. They were asked to select potential post-surgical complications and these were defined as `events and complications of care – even if recognised as being known complications’. As it is not clear whether a patient had a diagnosis code before admission or developed it afterwards, we asked to select only those codes that are definitely indicating a post-surgical complication by asking the following: ‘would you usually think this sign or symptom was a complication?’. The interrater variability for the first round was acceptable with Krippendorff’s Alpha = 0.7484, Fleiss’ Kappa = 0.7480 and pairwise agreement percentages varying from 0.786 to 0.954.

In the second round experts (4 clinicians and one clinical coder) voted on the codes that were not unanimously selected in the first round (selecting by one or more). The interrater agreement was very low indicating disagreement between voters with Krippendorff’s Alpha = −0.077, Fleiss’ Kappa = −0.084 and pairwise agreement percentage varying from 0.323 (poor agreement) to 0.645 (fair to good agreement). From this round that evaluated codes that did not have a unanimous agreement in the first round, we took all the codes selected either by the specialist clinical coder or by at least three out of four clinicians.

We then reviewed suggested diagnoses against a local dataset where we able to conduct manual notes review of 327 patients who had undergone neurosurgery, of whom 37 had both a brain tumour and a possible complication. Based on this, we retained only 11 (27.5%) out of 41 previously selected codes that were definitely post-operative complications. Finally, we combined the diagnosis codes selected from the GlioCova dataset with those used in the OECD Patient Safety Questionnaire into our new list of complications – ICL list. We then validated the ICL list by constructing statistical models using both the OECD-defined PSI list and ICL list of complications, and comparing performance of the models using OECD-defined complications and ICL complications.

Assessing co-morbidities

We measured co-morbidities using a modified Elixhauser index [15], removing brain/spinal tumour diagnoses, adding dementia [16] and derived weights using all admissions in our dataset and previously defined methods [20] (see S6 and S7 Tables).

Outcome measures

Our outcome measures were 30-day mortality rate and length of stay (in days). We defined 30-day mortality as those who had died within 30 days of first surgical intervention. LOS was defined as the interval in days between admission and discharge. We have defragmented admissions and combined LOS if patients were discharged to another hospital. We have removed patients that did not have a LOS available. In order to simplify model presentation, we dichotomised LOS by considering LOS of <= 3 days vs > 3 days based on data distribution as there was the highest number of patients that had a LOS of 3 days (N = 3,703). We have removed patients with very long LOS (>180 days) for LOS analysis as these were outliers. Since GBM and meningioma are the commonest primary brain tumours and have different rates of complications, we calculated rates of complications and LOS in patients with GBM vs. meningioma, and by broad age bands to illustrate the impact of these on complication rates and LOS.

Statistical approaches

To evaluate how the OECD-defined PSI list and ICL complication list were associated with 30-day mortality and LOS, logistic regression was performed in R (v. 4.1.2) using glm function [21,22]. Complications were defined as above, but for analysis of LOS we did not consider patients who were discharged from the neurosurgical centre to another hospital as this would complicate the statistical modelling. Both unadjusted and adjusted regression models were developed. Adjusted regression models also included other predictor variables such as deprivation status, tumour/intervention type, co-morbidities. Variable selection in the adjusted analyses was performed using backward elimination based on the Akaike information criterion (AIC). Age and sex were included in the models even if eliminated during AIC elimination due to their a priori importance. If there were differences in the models using OECD-defined PSI list compared to ICL list, we used all of the variables that were selected as significant in either of the two models. Discrimination was assessed by inspection of ROC curves and area under the curve (C-statistic). DeLong test was performed to calculate the p-value for the difference in the AUC of the ROC curves [23]. We compared the performance of models developed from the OECD-defined PSI list and the ICL list by comparing the AUC between the two models. Please note, the models were only used to validate the ICL list and not to predict outcomes.

Ethics statement

This data uses anonymised pooled data from multiple national datasets. In line with published guidance, such work does not require individual informed patient consent. We have research ethics (Yorkshire and the Humber – Sheffield Research Ethics Committee) approval for this work. All published data are anonymised and non-identifiable.

There is a data-sharing agreement (DSA) in place between NHS England/Digital and our team (“Computational Oncology Laboratory” hosted at Imperial College London) that specifies that we’re allowed to publish aggregated patients’ data in peer-reviewed journals. Further to the DSA, we also have Research Ethics Committee (REC) approvals (REC reference: 16/YH/0213) that gave us permission to publish aggregated patients’ data in peer-reviewed journals.

Results

Patient cohort and outcomes

We identified 29,467 patients with a primary CNS tumour who underwent a surgical intervention. We excluded 318 patients who underwent biopsy and surgery on different days (see S8 Fig), 81 patients with an incorrect interval from diagnosis to operation, and a further 7 patients who were duplicates. We further removed 18 patients that did not have an interval from diagnosis to vital status, 23 patients who had left the UK (and therefore vital status was unknown) and 2 patients that had incorrect intervals from diagnosis to vital status or surgical intervention (died before surgery date). Thus, the final analytical cohort consisted of 29,018 patients. The median age was 58 (Q1 = 18, Q2 = 46, Q3 = 58, Q4 = 68, Q5 = 94) and 50.5% were male. Patient characteristics can be found in Table 1. Most patients were discharged to ‘usual place of residence’ (8% to another hospital; 1% to care home or a hospice) (see S9 Table). The most common reasons for readmission were headache, further surgery, post-operative infection, CSF leak, and nausea and vomiting (see S10 Table).

Defining complications

The newly defined ICL list of complication codes consisted of 51 post-surgical complications (40 OECD-defined and 11 Gliocova-defined (see Table 2). Most patients who had a complication had one from the Gliocova-defined list (90.3%, n = 2,956) and a third of the 568 patients who had an OECD-defined complication also had a Gliocova-defined complication (44.0%, n = 250). See Fig 1 for the selection of complication and no-complication patient cohorts. Patients with complications from our ICL list had much higher rates of 30-day mortality compared to patients without complications – 6.5% (212 out of 3,274 patients) for patients with complications compared to 1.8% (465 out of 25,744 patients) in the no-complications group. They also had longer LOS (median LOS was 17 days compared to 5 days). Patients with complications from the OECD-defined PSI list only also had higher 30-day mortality (7.6%) and LOS (median LOS was 20 days). See Table 3.

thumbnail
Table 3. Outcomes (30-day mortality, median LOS) for patient subgroups based on complication status (ICL defined complications versus OECD-defined PSI list versus No complications).

https://doi.org/10.1371/journal.pone.0342011.t003

thumbnail
Fig 1. Flowchart for the selection of patients with and without complications from the patient cohort.

https://doi.org/10.1371/journal.pone.0342011.g001

Model performance using ICL list versus OECD only

Intervention type, ethnicity, age, comorbidities (ECI score), tumour type and income quintile were identified as significant predictors of 30DM and in addition sex was identified as significant predictor of LOS. In both adjusted and unadjusted analyses, regression models for 30-day mortality and longer LOS using the ICL complication list performed similarly or better than models using the OECD-defined PSI list only (Figs 2 and 3). In unadjusted analysis, ROC curves were slightly better for models using the ICL complication list. AUC for 30-day mortality model using the ICL list was 0.60 (95% CI 0.59–0.62) compared to 0.52 (95% CI 0.51–0.53) with a p-value = 2.7e-15 for the difference between the two AUCs. AUC in LOS model was 0.55 (95% CI 0.55–0.55) vs 0.51 (95% CI 0.51–0.51)) accordingly with a p-value = 2.2e −16. In the adjusted models the C-statistic (AUC) for the 30-day mortality model using ICL complications was 0.77 (95% CI 0.76–0.79) vs. 0.74 (95% CI 0.73–0.76) for the OECD-model with a p-value = 0.009, and for LOS it was 0.66 (95% CI 0.65–0.67) vs. 0.64 (0.63–0.65) accordingly with a p-value = 7.8e −05.

thumbnail
Fig 2. ROC curves and the c-statistic for unadjusted and adjusted models predicting 30-day mortality using ICL complication list versus OECD-defined PSI list only.

(A) ROC curves of unadjusted models. (B) ROC curves of the adjusted models. Adjusted models included intervention type, ethnicity, age, comorbidities (ECI score), tumour type and complications variables.

https://doi.org/10.1371/journal.pone.0342011.g002

thumbnail
Fig 3. ROC curves and the c-statistic for unadjusted and adjusted models predicting longer length of stay using ICL complication list versus OECD-defined PSI list only.

(A) ROC curves of unadjusted models. (B) ROC curves of the adjusted models. Adjusted models included intervention type, ethnicity, age, comorbidities (ECI score), tumour type, sex, income quintile and complications variables.

https://doi.org/10.1371/journal.pone.0342011.g003

Rates of outcomes by clinical group

30-day mortality was 2.3 (n = 677) and 12.7% (n = 3,701) of patients were re-admitted within 30 days. Using the ICL list of complications, the overall risk of complication was 11.3% (N = 3,274) (compared to 2.0% (N = 568) using the OECD-defined PSI list). The complications from the ICL list that were driving the increase in complication rate most were ‘Haemorrhage and hematoma complicating a procedure, not elsewhere classified’ (N = 953, 29%), ‘Urinary tract infection, site not specified’ (N = 673, 20.6%), ‘Lobar pneumonia, unspecified’ (N = 260, 7.9%), ‘Other postprocedural disorders of nervous system’ (N = 227, 6.9%) and ‘Cerebral infarction, unspecified’ (N = 211, 6.5%). Patients with glioblastoma had a higher risk of 30-day mortality (4.2% compared to 1.2%) and 30-day readmission (14.6% compared to 11.0%) compared to meningioma patients and the overall rates of these outcomes across all age categories (30-day mortality was 5.1%, 3.7% and 4.6% for the 18–40, 40–65 and the 65 + age categories accordingly compared to 0.5%, 0.6% and 2.4% and 30-day readmission was 17.5%, 14.5% and 14.4% compared to 11.1%, 11.3% and 10.4%). Nonetheless, risk of having post-surgical complications was higher in the meningioma patient group (13.6%, 12.0% and 20.0% for meningioma patients for the 3 age categories compared to 8.9%, 7.2%, 9.9% for the glioblastoma patients). See Table 4.

thumbnail
Table 4. Rates of outcomes (30-day mortality, complications, 30-day readmission, median LOS) for patient subgroups based on age and tumour type.

https://doi.org/10.1371/journal.pone.0342011.t004

Discussion

We have used a large, comprehensive cohort of all adults undergoing first surgical intervention for a primary brain tumour over a six-year period in England to define an expanded set of post-operative complications. We have shown the OECD-defined PSI capture a small proportion of patients with complications compared to the newly defined ICL list of complications. However, models developed using the ICL list of complications perform as well as those using the OECD-defined PSI list in predicting 30-day mortality and length of stay. Unlike previous approaches, this assessment of complications can be done entirely using routine healthcare data, rather than manually curated data. As a consequence, we were able to analyse a far larger dataset than previous work.

30-day mortality rates were low (2.3%), the complication rate was 11.3%, and approximately 1 in 8 patients were readmitted within 30 days of surgery. Patients who were older or had more co-morbidities, were more likely to have complications and more likely to die. While not surprising, over 40% of patients had at least one co-morbidity, and the median age was 58, with 18% being over 70. The need to understand neurosurgical outcomes in an older, co-morbid population is thus clear. Complication rates for patients with meningioma were higher than those for GBM; we think that this is due to clinical decision making around risk, given that surgery may be curative for patients with meningioma, and thus both patients and clinicians are willing to accept a higher risk of complications. However, we cannot prove this without detailed review of notes and interviews with clinicians and patients, which is not feasible in this type of work.

Our study is considerably larger than most previous work and post-operative outcome rates are similar to those found in previous research. Our reported 30-day mortality of 2.3% (4.2% in glioblastoma patients and 1.2% in meningioma patients) is similar to 3.3% after craniotomy for primary malignant brain tumours in a national US dataset, 3.6% in a small intra-axial brain tumour patient surgery cohort from India, 5.2% in a Dutch study of first-time glioblastoma surgery and 3% in our previous work using a slightly different time period and patient cohort [2,2426]. The 30-day readmission rate of 12.7% (14.6% in glioblastoma patients and 11.0% in meningioma patients) is also similar to previous work: 11.5% after craniotomy for primary malignant brain tumours in a US study using National Surgical Quality Improvement Program registry data (2005–2015), 13.2% readmission incidence after craniotomy for malignant supratentorial tumours in California (1995–2010), and 17.8% after craniotomy in brain metastasis patients in a US study [24,27,28].

Comparing post-surgical complication rates is challenging due to the varying sizes of datasets and different definitions used to define post-operative complications, and a review of methodological issues is given by Keltie et al. [29]. Nonetheless, the complication rate using our approach (11.3%) is similar to previous literature, but dependent on the definitions used. For example, one previous study quoted a very small complication rate of 3.4%, but this is due to them using a very short list of defined complications (retention of a foreign object, wrong side surgery, iatrogenic stroke, meningitis or haemorrhage) [30]. A French study had a 31% complication rate (25% with post-operative nausea and vomiting (PONV), 16% with neurologic complications) but was very small (N = 167) and only covered one hospital [31]. Another French study identified severe complications, which occurred in around 11% of patients [32]. Some large american studies that have used complications or adverse events as defined by the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) recorded complication rates of 8.2%, 14.3% and 25% in neurosurgery [3335]. Some other studies investigating neurosurgical procedures had complication rates varying from 7.8% to 25% [3638]. See Table 5 for comparison below.

thumbnail
Table 5. Complication rates in our study and in other studies.

https://doi.org/10.1371/journal.pone.0342011.t005

This is the first study to take a comprehensive national view of complications after brain tumour surgery. However, the use of administrative data does not allow us to provide detailed clinical information (e.g., on severity of complications) and some diagnoses might be miscoded or missed [3942]. There may be an underlying tendency to capture all complications (as it may increase the income for the hospital for that stay); equally, we know that many diagnoses are missed from routine electronic healthcare records, and while there is a risk of bias, it is not clear in which direction it might move our estimate. Equally, we have not conducted a formal sensitivity analysis, although 29% of the complications are due to haemorrhage and 20% due to urinary tract infections, and thus are likely to be robust. However, using administrative data does allow us to systematically explore predictors of outcomes and report both absolute and relative risks in a large, unselected cohort. The combination of data, analysts and our Expert Advisory Group has also allowed us to help refine our questions and analysis. Unlike much previous work, we start with an incident national cohort and have comprehensive tracking of admissions and death. In addition, the Delphi review process was difficult to conduct robustly, due to the distributed nature of the advisory group and their time commitments, as well as the limitations imposed by the COVID-19 pandemic.

The OECD-defined risk of complications is specific – it identifies a small group of patients who have a markedly increased risk of death (7.6% 30-day mortality compared to 6.5% using the ICL complications list and 2.2% in the no-complications group) – but not sensitive, as it misses a large number of patients who have post-operative complication. The vast majority of those who we found to have a post-operative complication were not identified by the OECD-defined PSI list, and the group defined using the combined set of complications had a risk of 30-day mortality that was still 5 times higher than the no-complications group.

Despite the comprehensive multi-year cohort, the number of patients undergoing a biopsy is still relatively small; it is therefore not possible to factors specific to biopsy patients vs. Patients undergoing resection. The C-statistic for all models is relatively low (below 0.8), suggesting that there are significant other factors that influence outcomes. It is therefore important to reinforce that the aim of this work is not to develop models that accurately predict complications. Instead, it is to develop and validate a novel definition of complications, and use that to estimate rates of complications. Model performance is used to justify our novel definition as it identifies many more patients than using the OECD definition, where model performance is similar to the much more restricted OECD definition, and where outcomes in patients with complications are clearly worse than in those without complications. For that reason, we did not perform internal model validation (e.g., with a test-train-validate split, or K-fold validation) because the aim was to validate the list of complications, rather than the predictive model itself. Therefore, for the first time, we have a definition of post-operative complications in primary brain tumour patients that is assessable at a national level, using routine electronic health records. The model built using the extended list performs as well as one constructed using a much smaller list of OECD-defined complications. However, this does not mean that this model can easily be used to predict post-operative complications, although it may help inform such models. Equally, this is likely to be most applicable to England, although it may act as a basis for work in other countries. The aim of this work is to unlock the ability to count post-operative complications in brain tumour patients undergoing neurosurgery. This then allows us to explore predictors and consequences of such complications in a robust, large-scale, unbiased fashion at a national level, and may allow for use as a quality standard or for benchmarking between centres, for example.

Previous work has shown that surgeon volumes predict post-operative mortality in this group of patients. However, post-operative mortality is low and there is much less work on post-operative complications, in part because there are few large datasets to work with. This paper shows that we can measure post-operative complications in patients undergoing neurosurgery for a primary brain tumour in England; that our list of complications captures many more patients than the OECD-defined PSI list, and yet maintains model performance. Future work will explore the mediators and implications of these complications in more detail. More information on the GlioCova project can be found here: https://blogs.imperial.ac.uk/gliocova/about-gliocova/.

Supporting information

S2 Table. List of OPCS 4 codes used to define major resection and biopsy.

https://doi.org/10.1371/journal.pone.0342011.s002

(DOCX)

S3 Table. OECD Patient safety (PS) questionnaire codes.

Codes were obtained from the OECD Health Care Quality and Outcomes (HCQO) 2018−19 Data Collection: https://www.oecd.org/statistics/data-collection/Health%20Care%20Quality%20Indicators_guidelines.pdf (* - codes excluded from our study).

https://doi.org/10.1371/journal.pone.0342011.s003

(DOCX)

S4 Table. The list of top 100 diagnosis codes extracted from index admission.

https://doi.org/10.1371/journal.pone.0342011.s004

(DOCX)

S5 Fig. Flowchart for the modified Delphi process.

https://doi.org/10.1371/journal.pone.0342011.s005

(PDF)

S6 Table. ECI comorbidity index variables and the corresponding ICD-10 codes used to identify them.

https://doi.org/10.1371/journal.pone.0342011.s006

(DOCX)

S7 Table. Scores for modified Elixhauer Comorbidity Index variables using all brain/spinal tumour patient admissions between 2012–2019 based on English HES data.

A higher score indicates that this comorbidity is linked with higher log odds of in-hospital mortality (e.g., a score of 2 is associated with a double increase in the log odds of mortality).

https://doi.org/10.1371/journal.pone.0342011.s007

(DOCX)

S8 Fig. Flowchart for the selection of patients having major resection or biopsy.

https://doi.org/10.1371/journal.pone.0342011.s008

(DOCX)

S9 Table. Distribution of the location of discharge for all patients undergoing first surgical intervention in the Gliocova dataset.

https://doi.org/10.1371/journal.pone.0342011.s009

(DOCX)

S10 Table. Most common ICD-10 diagnoses that could indicate reason of readmission.

https://doi.org/10.1371/journal.pone.0342011.s010

(DOCX)

Acknowledgments

This work uses data provided by patients and collected by the NHS as part of their care and support.

We want to thank Nicola Glover for collaboration regarding clinical coding.

References

  1. 1. NHS England. Cancer – NDRS. 2023. https://digital.nhs.uk/ndrs/about/ncras
  2. 2. Williams M, Treasure P, Greenberg D, Brodbelt A, Collins P. Surgeon volume and 30 day mortality for brain tumours in England. Br J Cancer. 2016;115(11):1379–82. pmid:27764843
  3. 3. Agency for Healthcare Research and Quality. AHRQ QI: PSI Technical Specifications Updates, 2021. 2021. https://qualityindicators.ahrq.gov/measures/PSI_TechSpec
  4. 4. Farquhar M. AHRQ Quality Indicators. Agency for Healthcare Research and Quality. 2008.
  5. 5. Quan H, Drösler S, Sundararajan V. Adaptation of AHRQ Patient Safety Indicators for Use in ICD-10 Administrative Data by an International Consortium. Advances in Patient Safety: New Directions and Alternative Approaches. 2008.
  6. 6. OECD. Health Care Quality and Outcomes (HCQO) 2018-19 Data Collection. 2023. https://www.oecd.org/statistics/data-collection/Health%20Care%20Quality%20Indicators_guidelines.pdf
  7. 7. Nuño M, Carico C, Mukherjee D, Ly D, Ortega A, Black KL, et al. Association between in-hospital adverse events and mortality for patients with brain tumors. J Neurosurg. 2015;123(5):1247–55. pmid:26053457
  8. 8. Trinh VT, Davies JM, Berger MS. Surgery for primary supratentorial brain tumors in the United States, 2000-2009: effect of provider and hospital caseload on complication rates. J Neurosurg. 2015;122(2):280–96. pmid:25397366
  9. 9. Neville IS, Ureña FM, Quadros DG, Solla DJF, Lima MF, Simões CM, et al. Safety and costs analysis of early hospital discharge after brain tumour surgery: a pilot study. BMC Surg. 2020;20(1):105. pmid:32410602
  10. 10. Clark PR, Dambrino RJ, Himel SM, Smalley ZS, Yimer WK, Washington CW. The impact of craniotomy for brain tumor case volume on patient safety indicators and in-hospital mortality. Clin Neurol Neurosurg. 2020;196:106043. pmid:32653799
  11. 11. Luoma AMV, Douglas DR, D’Antona L, Toma AK, Smith M. Using the Postoperative Morbidity Survey to Measure Morbidity After Cranial Neurosurgery. J Neurosurg Anesthesiol. 2022;34(2):201–8. pmid:35255015
  12. 12. Ferroli P, Vetrano IG, Schiavolin S. Brain tumor resection in elderly patients: potential factors of postoperative worsening in a predictive outcome model. Cancers. 2021;13(10).
  13. 13. Sarnthein J, Staartjes VE, Regli L, Neurosurgery-Registry consortium. Neurosurgery outcomes and complications in a monocentric 7-year patient registry. Brain Spine. 2022;2:100860. pmid:36248111
  14. 14. Bottle A, Aylin P. Intelligent information: a national system for monitoring clinical performance. Health Serv Res. 2008;43(1 Pt 1):10–31. pmid:18300370
  15. 15. Bottle A, Gaudoin R, Goudie R, Jones S, Aylin P. Can valid and practical risk-prediction or casemix adjustment models, including adjustment for comorbidity, be generated from English hospital administrative data (Hospital Episode Statistics)? A national observational study. Southampton (UK): NIHR Journals Library. 2014.
  16. 16. Bottle A, Aylin P. Comorbidity scores for administrative data benefited from adaptation to local coding and diagnostic practices. J Clin Epidemiol. 2011;64(12):1426–33. pmid:21764557
  17. 17. Gliocova. 2022. https://blogs.imperial.ac.uk/gliocova/
  18. 18. Ministry of Housing, Communities and Local Government. English indices of deprivation 2019: technical report. 2025. https://www.gov.uk/government/publications/english-indices-of-deprivation-2019-technical-report
  19. 19. Bowles N. The Delphi technique. Nurs Stand. 1999;13(45):32–6. pmid:10633705
  20. 20. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–9. pmid:16224307
  21. 21. glm function – RDocumentation. 2023. https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/glm
  22. 22. R-project. Index of/bin/windows/base/old/4.1.2. 2025. https://cran.r-project.org/bin/windows/base/old/4.1.2/
  23. 23. R. roc.test: Compare two ROC curves. 2023. https://rdrr.io/cran/pROC/man/roc.test.html
  24. 24. Senders JT, Muskens IS, Cote DJ, Goldhaber NH, Dawood HY, Gormley WB, et al. Thirty-Day Outcomes After Craniotomy for Primary Malignant Brain Tumors: A National Surgical Quality Improvement Program Analysis. Neurosurgery. 2018;83(6):1249–59. pmid:29481613
  25. 25. Moiyadi AV, Shetty PM. Perioperative outcomes following surgery for brain tumors: Objective assessment and risk factor evaluation. J Neurosci Rural Pract. 2012;3(1):28–35. pmid:22346187
  26. 26. De Witt Hamer PC, Ho VKY, Zwinderman AH. Between-hospital variation in mortality and survival after glioblastoma surgery in the Dutch Quality Registry for Neuro Surgery. Journal of Neuro-Oncology, 2019;144(2):313–23.
  27. 27. Marcus LP, McCutcheon BA, Noorbakhsh A, Parina RP, Gonda DD, Chen C, et al. Incidence and predictors of 30-day readmission for patients discharged home after craniotomy for malignant supratentorial tumors in California (1995-2010). J Neurosurg. 2014;120(5):1201–11. pmid:24605840
  28. 28. Tonse R, Townsend A, Rubens M, Siomin V, McDermott MW, Tom MC, et al. Factors associated with unplanned readmissions and costs following resection of brain metastases in the United States. Sci Rep. 2021;11(1):22152. pmid:34773051
  29. 29. Keltie K, Cole H, Arber M, Patrick H, Powell J, Campbell B, et al. Identifying complications of interventional procedures from UK routine healthcare databases: a systematic search for methods using clinical codes. BMC Med Res Methodol. 2014;14:126. pmid:25430568
  30. 30. De la Garza-Ramos R, Kerezoudis P, Tamargo RJ, Brem H, Huang J, Bydon M. Surgical complications following malignant brain tumor surgery: An analysis of 2002-2011 data. Clin Neurol Neurosurg. 2016;140:6–10. pmid:26615463
  31. 31. Lonjaret L, Guyonnet M, Berard E, Vironneau M, Peres F, Sacrista S, et al. Postoperative complications after craniotomy for brain tumor surgery. Anaesth Crit Care Pain Med. 2017;36(4):213–8. pmid:27717899
  32. 32. Cinotti R, Bruder N, Srairi M, Paugam-Burtz C, Beloeil H, Pottecher J, et al. Prediction Score for Postoperative Neurologic Complications after Brain Tumor Craniotomy: A Multicenter Observational Study. Anesthesiology. 2018;129(6):1111–20. pmid:30234580
  33. 33. Cote DJ, Karhade AV, Larsen AMG, Burke WT, Castlen JP, Smith TR. United States neurosurgery annual case type and complication trends between 2006 and 2013: An American College of Surgeons National Surgical Quality Improvement Program analysis. J Clin Neurosci. 2016;31:106–11. pmid:27183956
  34. 34. Rolston JD, Han SJ, Lau CY, Berger MS, Parsa AT. Frequency and predictors of complications in neurological surgery: national trends from 2006 to 2011. J Neurosurg. 2014;120(3):736–45. pmid:24266542
  35. 35. Meyer HS, Wagner A, Obermueller T, Negwer C, Wostrack M, Krieg S, et al. Assessment of the incidence and nature of adverse events and their association with human error in neurosurgery. A prospective observation. Brain Spine. 2021;2:100853. pmid:36248119
  36. 36. Boissonneau S, Tsiaremby M, Peyriere H, Graillon T, Farah K, Fuentes S, et al. Postoperative complications in cranial and spine neurosurgery: a prospective observational study. J Neurosurg Sci. 2023;67(2):157–67. pmid:33709657
  37. 37. Kulikov A, Krovko Y, Nikitin A, Shmigelsky A, Zagidullin T, Ershova O, et al. Severe Intraoperative Hyperglycemia and Infectious Complications After Elective Brain Neurosurgical Procedures: Prospective Observational Study. Anesth Analg. 2022;135(5):1082–8. pmid:35051950
  38. 38. Sahin B, Kilic M, Aydin L, Ayca Sahin S, Ozoner B, Akkaya E, et al. Analysis and Differences of the Neurosurgical Complications During COVID-19 Pandemic. Sisli Etfal Hastan Tip Bul. 2021;55(4):469–76. pmid:35317374
  39. 39. Mahbubani K, Georgiades F, Goh EL, Chidambaram S, Sivakumaran P, Rawson T, et al. Clinician-directed improvement in the accuracy of hospital clinical coding. Future Healthc J. 2018;5(1):47–51. pmid:31098532
  40. 40. Radhakrishnan ST, Perry R, Misra S, Ray S, Ruban A, Quayson BI, et al. Targeted education for clinicians and clinical coding staff improves the accuracy of clinical coding: A quality improvement project. Future Healthc J. 2024;11(1):100127. pmid:38689701
  41. 41. Toner E, Khaled A, Ramesh A, Qureshi MK, Al Suyyagh K, Dunkow P. Financial Impact of Inaccurate Coding Plus Cost-Effectiveness Analysis for Surgically Managed Patients With Periprosthetic Fractures. Cureus. 2021;13(2):e13060. pmid:33680602
  42. 42. Nouraei SAR, O’Hanlon S, Butler CR, Hadovsky A, Donald E, Benjamin E, et al. A multidisciplinary audit of clinical coding accuracy in otolaryngology: financial, managerial and clinical governance considerations under payment-by-results. Clin Otolaryngol. 2009;34(1):43–51. pmid:19260884