Validation of MTL30 as a quality indicator for colorectal surgery

Background Valid indicators are required to measure surgical quality. These ideally should be sensitive and selective while being easy to understand and adjust. We propose here the MTL30 quality indicator which takes into account 30-day mortality, transfer within 30 days, and a length of stay of 30 days as composite markers of an uneventful operative/postoperative course. Methods Patients documented in the StuDoQ|Colon and StuDoQ|Rectal carcinoma register of the German Society for General and Visceral Surgery (DGAV) were analyzed with regard to the effects of patient and tumor-related risk factors as well as postoperative complications on the MTL30. Results In univariate analysis, the MTL30 correlated significantly with patient and tumor-related risk factors such as ASA score (p<0.001), age (p<0.001), or UICC stage (p<0.001). There was a high sensitivity for the postoperative occurrence of complications such as re-operations (p<0.001) or subsequent bleeding (p<0.001), as well as a significant correlation with the CDC classification (p<0.001). In multivariate analysis, patient-related risk factors and postoperative complications significantly increased the odds ratio for a positive MTL30. A negative MTL30 showed a high specify for an uneventful operative and postoperative course. Conclusion The MTL30 is a valid indicator of colorectal surgical quality.


Results
In univariate analysis, the MTL30 correlated significantly with patient and tumor-related risk factors such as ASA score (p<0.001), age (p<0.001), or UICC stage (p<0.001). There was a high sensitivity for the postoperative occurrence of complications such as re-operations (p<0.001) or subsequent bleeding (p<0.001), as well as a significant correlation with the CDC classification (p<0.001). In multivariate analysis, patient-related risk factors and postoperative complications significantly increased the odds ratio for a positive MTL30. A negative MTL30 showed a high specify for an uneventful operative and postoperative course.

Conclusion
The MTL30 is a valid indicator of colorectal surgical quality. Background Surgery is often the best and only therapeutic option available for a large number of diseases but comes with specific complication rates which increase with procedural complexity [1]. Treatment quality and the adequate management of complications arising from surgery are therefore critical factors for therapeutic success [2]. Numerous studies have shown that complication rates largely depend on patient-specific factors such as age or the ASA score on the one hand, and on the infrastructure of the treating hospital and its expertise for specific procedures on the other [3][4][5][6][7]. The quality of a given medical therapy can broadly be divided into 3 parts: The quality of the structure, which reflects the available equipment at a medical facility; the quality of the process, which reflects the way a medical treatment is carried out; and the quality of the result, which reflects the actual outcome of the treatment [8]. Quality indicators (QI) in surgery are measuring units which in general enable a graded distinction between good and bad operative/postoperative outcomes. They do not measure quality directly, but serve as surrogate parameters. A large number of parameters for visceral surgery have been described including 30-or 90-day mortality, postoperative length of stay, minor or major complications, anastomotic leakage, wound healing disorders, number of removed lymph nodes, quality of mesocolic excision, intraoperative blood loss, postoperative ventilation duration, resumption rate, and quality of life indices, which all could be used hypothetically to evaluate quality [9][10][11]. However, these indicators are influenced by a variety of factors, both patientrelated (age, comorbidities) and the care system (availability of diagnostic and intervention options). They also require detailed and conscientious documentation. Ultimately, it is difficult to apply one of the QI listed above to a variety of operations in hospitals with different equipment.
An optimal QI should be applicable to all operations if possible, should be sensitive to the occurrence of complications, and should specifically depict comorbidities. In addition, it should be adjustable and take into account complex care structure across differently equipped hospitals.
We previously proposed the MTL30 as a general performance marker for surgical quality within the 30 day postoperative period [12]. In summary, if the patient died in inpatient care, MLT rated positive (M), whereas if they lived but are still in inpatient care at day 30, MLT rated positive (L). Further, if the patient was transferred to another acute care hospital (not for rehabilitation), MLT rated positive (T). Thus, the L and T designations reflect the occurrence of a major complication that either led to a significant extension of stay (L) or that could not be treated in the corresponding hospital (T). This allows adjustment between hospitals across different care levels. The time interval of 30 days was chosen for all designations to ensure comparability with the 30-day lethality. As it has been shown previously that daily documentation of complications is challenging, it is important to have access to pre-existing data [13]. The parameters that contribute to the MTL30 can be easily obtained from the hospital billing data which are universally documented and in contrast to the company's own documentations, are difficult or impossible to modify.
The German Society of General and Visceral Surgery (DGAV) have previously produced a large registry for patients to assess and to improve the quality of surgical treatment [14]. This includes the collection of data on the course of surgical treatment in hospitals in Germany. Importantly, all hospitals contribute to this registry, irrespective of their size or volume. This effectively reduces a potential selection bias, such as would occur if only high volume centers were included for instance, and enables the evaluation of quality markers over a wide range of patient groups and hospital categories.
Based on this registry, we conducted an analysis to characterize the MTL30 on colon and rectal cancer (StuDoQ|Colon Cancer; StuDoQ|Rectal Cancer registers) with respect to patientspecific factors and the sensitivity to complications.

Methods
The StuDoQ|Colon carcinoma and StuDoQ|Rectal carcinoma registers are prospectively documented databases for surgical interventions in colorectal carcinomas, which were set up by the DGAV in January 2010 (www.dgav.de/studoq, www.en.studoq.de). They were developed to facilitate the assessment of the quality and risk factors of colorectal cancer surgery in Germany. The declaration of consent, ethic approvment and the data security procedures were approved by the Society for Technology, Methods and Infrastructure for Networked Medical Research (http://www.tmf-ev.de). Written consent was obtained from all participants. The publication guidelines were determined by the DGAV (http://www.dgav.de/studoq/ datenschutzkonzept-und-publikations guidelines.html). The data from participating centers are entered prospectively in pseudonymized form using a browser-based tool and subjected to automatic plausibility checks. Validation by cross-checking with institutional medical control data is part of the annual certification process. For the present study, all cases that underwent curative resection due to colorectal cancer were identified in the StuDoQ|Colon cancer and StuDoQ|Rectal cancer registry, and relevant demographic data, comorbidities, as well as information on operations, histology and perioperative history for analysis extracted in anonymous form. Full wall excision, simple polypectomy, endoscopic mucosal resection, and other endoluminal procedures, as well as palliative interventions regardless of the size of the operation, were excluded. Basic registration structures are comparable to the StuDoQ|Pancreas registration [14].
Postoperative complications included anastomotic leakage (grade C) [15,16], infection of the surgical site [17], Clavien-Dindo classification (CDC) [18], burst abdomen, reoperation, and hospital mortality. They were defined as either present or not present. Additional postoperative parameters that were assessed were the need for unscheduled postoperative ventilation lasting more than 48 hours, pneumonia, length of stay (LOS), and readmission. Postoperative total morbidity was summarized as none (CDC 0), minor (CDC 1-2), severe (CDC 3a-4), and fatal (CDC 5) according to the CDC. Patients were counted as MTL30 positive if they had died within 30 days after index operation, the postoperative length of stay exceeded 30 days or if they had been transferred to another acute hospital or in hospital unit (e.g. transfer to a tertiary center due to surgical complications or to internal medicine due to postoperative pulmonary emboly). Transfer to a postoperative rehabilitation did not count as positive.
Statistical analysis was performed with a bilateral significance level of 0.05. Scale variables were expressed as median and range and categorical parameters as absolute frequency and percentage. Univariate analysis was performed using the chi-square test for categorical variables and the Mann-Whitney test for ratio variables.
All variables with a p-value <0.1 in the univariate analysis were included in the multivariate analysis. The multivariable analysis was carried out by logistic regression.
Focusing on a positive MTL30 as a function of postoperative complications, there was a significant correlation with both operative and nonoperative complications. The group of patients with postoperative burst abdomen, wound healing disorders, postoperative haemorrhaging, ileus, blood transfusion or re-operation each had a significantly higher percentage of positive MTL30 (Table 3). Similarly, patients with postoperative non-surgical complications, such as prolonged ventilation for more than 48 hours, renal failure, pneumonia, or thromboembolic events (pulmonary embolism, myocardial infarction, stroke) each showed a significantly increased percentage of positive MTL30. There was also a significant correlation between the percentage of positive MTL30 and the severity of the complication according to Clavien-Dindo (Tables 4).
The multivariable analysis for known patient-dependent factors as ASA or age as well as tumor-dependent factors as UICC-stage showed a significantly increased odds ratio (OR) for a positive MTL30. Similarly, the occurrence of postoperative complications increased the odds ratio for a positive MTL30. The 30-day lethality also showed a correlation with a number of risk factors and the occurrence of complications. But the 95% CI was smaller in all parameters arguing for a higher accuracy of the MTL30 (Table 5; Fig 1).

Discussion
Benchmarking is increasingly used within the healthcare sector as an approach to increase quality [19]. Nevertheless, valid assessment of surgical quality remains a major challenge [20].
Because morbidity and mortality decrease with the number of surgeries performed in hospitals [3,4], "minimum quantities" have been recommended internationally. This has led to centralization in high volume centers [21,22], but without necessarily improving surgical care. Indeed, it is the management of complications that appears to be better in high-volume hospitals. In addition, this volume-outcome relationship only applies to major interventions, since no volume-dependent differences have been shown for minor interventions [23]. A major problem with measuring the difference between surgical departments is that complications need to be documented in an unbiased manner, which necessitates an external audit. On the other hand, measured events need to be frequent enough to result in significant differencesparticularly when cases are low. With the MTL30, we introduced a composite marker that allows a comparison between multiple hospitals across different levels of care and complexity of patient groups [12]. The MTL30 is based on the idea that there are no/minor complications if the patient can be released alive within 30 days of the operation. On the other hand, the

PLOS ONE
Validation of MTL30 as a quality indicator for colorectal surgery operative/postoperative course must be regarded as conspicuous if the patient has died within 30 days, has been transferred to another acute care hospital, or is still in inpatient care after 30 days. Any serious medical intervention possesses a certain degree of morbidity and mortality, which explains why an MTL30 of 0% is extremely unlikely. The percentage of MTL30 positive patients is therefore dependent on the type of intervention performed (e.g. inguinal herniotomy vs. pancreatic resection), the patient's co-morbidity (e.g. ASA status), and the quality of the intervention. In addition, the index time-point of 30 days after operation was chosen for the MTL30 in accordance with the widely used 30-day mortality standard. Nevertheless, it can be adjusted to 7 or 14 days for other kinds of operations or other healthcare systems. As a proof of concept, we showed for almost 20,000 datasets from the StuDoQ register that a positive MTL30 has a high correlation with existing risk factors of the patient, and strongly correlated with the occurrence of postoperative complications. As a composite indicator, the MTL30 is also more specific and sensitive than individual indicators.
Correctly determining surgical quality using QIs is a complex task [24]. On the one hand, markers should be used which are easy to collect, for example on the basis of accounting data, and which are difficult to falsify. This is indeed the case for the MTL30, since both the postoperative length of stay and the type of discharge from inpatient care (deceased, relocated, or discharged home/transferred to rehabilitation) are recorded in the accounting system. On the other hand, markers should be adjustable to ensure a fair comparison between hospitals. This was confirmed in the present study, where we used data from the StuDoQ register. For a simple comparison between hospitals at different care levels, a global QI should ideally map all three pillars of quality to enable a fair comparison. If the outcome quality in terms of mortality is only considered, then the structural quality is overlooked. This includes e.g. the equipment of the hospital, the 24-hour preparedness for surgery, or the possibility of performing CT imaging. A low structural quality in a hospital therefore results in a high patient transfer rate, which is included in the recording of the "transfer" in the MTL30. In this way, it can be shown whether hospitals per se are appropriately equipped for the procedure performed. On the other hand, the length of stay (L) would be longer if hospitals were suitably equipped but are of worse quality due to disproportional complication rates.
Up to now, quality indicators are typically used to detect adverse events. By using the negative MTL30 as a quality indicator, we specifically tried to identify a "normal" postoperative course and could show a high degree of specificity (90%). Determining significant differences between facilities depends on two basic parameters: The first is the number of cases carried out, and the second is the relative frequency of an "adverse" event. In order to show a significant difference between hospitals with, for example, 10% vs. 5% mortality, 185 interventions would be required [9] which increases to as many as 1, 000 cases in order to detect a statistically valid difference between 1% and 2% adverse events. In addition, mortality is a rare event overall, which explains why it is difficult to map a significant difference due to the small number of cases. Furthermore, mortality increasingly occurs after 30 days due to the continual improvement of intensive care medicine. For this reason, 90-day mortality is increasingly considered to be more appropriate [25]. However, these patients are still inpatient on the 30th day, and are therefore rated as conspicuous in the MTL30. Since the MTL30 subsumes several adverse events, and is therefore higher per se, the number of cases required to detect significant differences decreases. Furthermore, different operations can be clustered into groups of similar complexities to further increase the number of cases. In this way, a higher number of cases per hospital can be achieved, even with relatively rare interventions such as esophagus, stomach, and pancreatic resections.
One shortcoming of the MTL30 is that it does not detect minor changes within the postoperative course that do not prolong the in-patient care or lead to transfer to another unit (e.g. superficial side infection). Also, it does not differentiate between an eventful operative course from an eventful postoperative course. The major goal for further validation will be to determine the optimal cut-off timepoint for different operative procedures according to the nationally used healthcare system.
It has been shown previously that in addition to directly comparing between hospitals, selfevaluation leads to an increase in quality in individual hospitals, which is referred to as the Hawthorne effect [26]. The morbidity and mortality in participating hospitals decreased significantly due to the introduction of NSQIP [27]. As the MTL30 is easy to obtain and does not need extra documentation, it will give each individual hospital a quick and unbiased overview of where it stands in comparison with others.
In summary, measuring quality is a fundamental task in medicine. The MTL30 is an easily obtained and adjustable QI. A positive MTL30 shows high sensitivity for an eventful postoperative course, whereas a negative MTL30 has high specificity for an uneventful postoperative course.
Supporting information S1