Identifying cirrhosis, decompensated cirrhosis and hepatocellular carcinoma in health administrative data: A validation study

Background To evaluate screening and treatment strategies, large-scale real-world data on liver disease-related outcomes are needed. We sought to validate health administrative data for identification of cirrhosis, decompensated cirrhosis and hepatocellular carcinoma among patients with known liver disease. Methods Primary patient data were abstracted from patients of the Toronto Center for Liver Disease with viral hepatitis (2006–2014), and all patients with liver disease from the Kingston Health Sciences Centre Hepatology Clinic (2013). We linked clinical information to health administrative data and tested a range of coding algorithms against the clinical reference standard. Results A total of 6,714 patients had primary chart data abstracted. A single physician visit code for cirrhosis was sensitive (98–99%), and a single hospital diagnostic code for cirrhosis was specific (91–96%). The most sensitive algorithm for decompensated cirrhosis was one cirrhosis code with any of: a hospital diagnostic code, death code, or procedure code for decompensation (range 88–99% across groups). The most specific was one cirrhosis code and one hospital diagnostic code (range 89–98% across groups). Two physician visit codes or a single hospital diagnostic code, death code, or procedure code combined with a code for cirrhosis were sensitive and specific for hepatocellular carcinoma (sensitivity 94–96%, specificity 93–98%). Conclusion These sensitive and specific algorithms can be used to define patient cohorts or detect clinical outcomes using health administrative data. Our results will facilitate research into the adequacy of screening and treatment for patients with chronic viral hepatitis or other liver diseases.


Introduction
In 2016, 1.26 million people worldwide died of cirrhosis and chronic liver diseases, and their complications. [1] Hepatocellular carcinoma (HCC) mortality rates are rising faster than those from any other malignancy. [2] Globally, viral hepatitis secondary to hepatitis B and C virus infection underlies 55% of cirrhosis-related deaths and 61% of deaths from HCC. [1] Yet, many patients with liver disease remain undiagnosed, largely because they remain asymptomatic until a late stage. [3,4] Of late, much progress has been made in the prevention and treatment of viral hepatitis. Many jurisdictions have advanced the timing of immunization against hepatitis B virus (HBV) from early adolescence to infancy. [5,6] Further, new treatments for chronic hepatitis C virus (HCV) infection have enabled large numbers of patients to achieve sustained virologic response, a marker of long-term clinical cure. [7][8][9][10] Finally, our understanding of the epidemic of non-alcoholic fatty liver disease (NAFLD) in North America is just beginning and the natural history of this disease is still not completely defined.
To evaluate the epidemiology and the long-term clinical effectiveness of treatments and screening programs for chronic liver diseases, it is essential to accurately detect clinical outcomes such as cirrhosis, decompensated cirrhosis, and HCC. Although a registry of patients could be used to measure long-term treatment effects, it is ill-suited to study the overall burden or healthcare utilization related to liver disease since liver-related outcomes can take decades to occur. Longitudinal, systematically collected information from large cohorts of patients with chronic liver disease is needed.
Routinely collected health administrative data enable efficient research on real-world outcomes of patients with liver disease, while offering objective evidence of past healthcare utilization. Although several studies have validated data algorithms for identifying cirrhosis, decompensated cirrhosis, and HCC, these have been limited to International Classification of Diseases 9 th revision (ICD-9) codes in the healthcare system of the United States. [11][12][13][14][15][16][17] In Canada, all hospitalization data have been coded using the ICD-10 system since 2002 and as of as of October 2015, all hospital discharge information in the United States have also been coded using the ICD-10 system. [18,19] While administrative data codes have been used in outcomes research [20,21], there are no existing validation studies for liver-related outcomes in Canadian patients and no validation studies using ICD-10 codes in the United States.
The primary objective of this study was to measure the validity of combined ICD-9 and ICD-10 health administrative data codes for detecting cirrhosis, decompensated cirrhosis, and HCC in patients with known chronic liver disease. This will facilitate the study of the longterm effects of antiviral treatment or policy changes relating to this patient population. Furthermore, as the epidemiology of chronic liver diseases such as NAFLD and alcohol-related disease have a different natural history than viral hepatitis, we also aimed to assess their broader validity in a group of patients with liver diseases of all causes. any underlying individuals is low. While data sharing agreements and privacy legislation for the province of Ontario prohibit ICES from making the data set publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/ DAS.

Setting
The validation cohorts consisted of patients from two different university-affiliated tertiary care hepatology clinics in Ontario, Canada: The Toronto Centre for Liver Disease (TCLD) at the University Health Network (UHN) located in Toronto and the Liver Disease Clinic at the Kingston Health Sciences Centre (KHSC) located in Kingston. Both clinics are staffed by subspecialty trained academic hepatologists and receive patient referrals for patients with acute or chronic liver diseases. At both sites, clinicians employ a standardized computerized form for clinical data entry. Patient status, including test results and treatments, are updated at every encounter. UHN and KHSC clinical records include information on patient demographics, most responsible diagnosis, laboratory data, imaging data, endoscopic reports, pathology data, non-invasive fibrosis assessment tests and results, and any hepatic decompensation events.

Administrative databases
The Institute for Clinical Evaluative Sciences holds health administrative data for all Ontario residents with provincial health insurance. Data on demographics, physician visits, emergency department visits, hospital admissions and procedures are linked using an encrypted patient identifier. [22] The Ontario Health Insurance Program (OHIP) database contains all billing claims made by physicians. [23] The Canadian Institute for Health Information's Discharge Abstract Database contains information for all admissions to acute care hospitals in Ontario [24], and the National Ambulatory Care Reporting System database contains information on emergency department and day surgery visits. [22] The Office of the Registrar General Death Database contains the cause of death for all deaths in the province. [25] The Ontario Cancer Registry includes detailed clinical information on malignancies such as anatomical site and tumour histology. [26]

Study population
We identified patients for inclusion in the chronic HBV and HCV cohorts using a two-step process. First, any patient followed at the TCLD with an HBV or HCV treatment status, positive HBV or HCV serology, positive HBV DNA, or positive HCV RNA, with a clinic visit between April 2006 and March 2014 were selected for further review. We then reviewed the charts to confirm chronic HBV or HCV status rather than resolved prior infection (patients with the latter were not included in the study cohorts). Patients who had evidence of ongoing infection with HBV or HCV at any time from their first clinic visit to March 31 st 2014 were included in the study. The cohort of patients from KHSC comprised consecutive patients seen at the KHSC Liver Clinic from May through August 2013. Patients in the KHSC cohort had liver disease of viral and non-viral etiology. Across all groups, any patient that could not be linked to administrative data holdings was excluded from the study.

Reference standard: Clinical outcomes
Outcomes evaluated in this study included cirrhosis, decompensated cirrhosis, and hepatocellular carcinoma. Two trained medical graduates (FG at TCLD and DC at KHSC) reviewed the charts to identify clinical outcomes.
Decompensated cirrhosis was identified based on the presence of any of the following in the clinical record: ascites, bleeding varices, encephalopathy, use of spironolactone without alternative indication, or explicit mention of decompensated cirrhosis. Cirrhosis was identified based on any of the decompensated cirrhosis criteria, or explicit mention of cirrhosis, non-bleeding varices, or use of nadolol without alternative indication. In addition, cirrhotic appearance on ultrasound, a liver biopsy result of F4 fibrosis, or a non-invasive test result consistent with F4 fibrosis were also considered diagnostic of cirrhosis. Hepatocellular carcinoma was identified based on explicit mention anywhere in the clinical note. Uncertain cases were reviewed and classified by a hepatologist (JJF at UHN or JAF at KHSC).
A 5% random sample of charts was re-abstracted by a general internist at TCLD (LLS) and a hepatologist at KHSC (JAF). Agreement beyond chance on the outcome ascertainment by both abstracters was measured using Cohen's kappa.

Administrative data outcomes
The primary outcomes of cirrhosis, decompensated cirrhosis and HCC were defined using relevant physician visit, emergency department visit, hospital diagnosis, procedure, death and pathology codes (Table 1). A secondary outcome of 2-year all-cause mortality following last clinic visit was reported as an overall measure of patient severity of illness. We built a series of diagnostic algorithms for each outcome ranging from simple (e.g. a single physician visit code) to more complex. This was done to identify the most parsimonious algorithms that were also highly sensitive and/or specific. We aimed to utilize all available health administrative data, and include both hospital-based and outpatient physician visits in our algorithms. Data sources were searched for relevant codes ten years prior to two years following the date of last clinical assessment (up to March 31 st , 2014 for TCLD and August 31 st , 2013 for KHSC).
Cirrhosis algorithms ranged from cirrhosis codes only to combinations with codes for chronic liver disease or any complication (decompensation events, HCC or liver transplant). Algorithms were combined in such a way as to make them more sensitive ("or" combinations) or specific ("and" combinations). As physician visit codes were noted to be less specific, we aimed to increase specificity by combining two or more such codes with hospitalization codes.
Decompensation algorithms included hospital diagnostic and death codes for portal hypertension, hepatorenal syndrome, jaundice, hepatic coma, hepatic failure, bleeding esophageal varices, gastric varices (bleeding not specified), and ascites. There were no physician visit codes available for decompensation events. We included procedure codes for endoscopy or insertion of Sengstaken tube for upper gastrointestinal bleeding, transjugular intrahepatic portosystemic shunt, and paracentesis. Since several of these procedures could occur for reasons other than decompensated cirrhosis (such as bleeding from an ulcer, or ascites secondary to an extra-hepatic malignancy), we tested combinations of procedure codes with a cirrhosis code from a physician visit.
Hepatocellular carcinoma algorithms ranged from simple (a single physician visit code) to more complex. We tested several combinations in order to optimise both sensitivity and specificity. We combined physician visit codes, hospital diagnostic codes, and cause of death codes. Further, we included procedure codes for radiofrequency ablation. Since this procedure can also be used to ablate tumours outside the liver (e.g., renal tumours), we combined ablation codes with an outpatient code for cirrhosis. Finally, we tested our results with and without anatomical and pathology codes from the Ontario Cancer Registry.

Analysis
Characteristics of patients in each validation cohort (HBV and HCV patients from TCLD, patients from KHSC) and 2-year mortality were described using univariate statistics.
We tested the performance of administrative data algorithms for cirrhosis, decompensated cirrhosis, and HCC against the clinical reference standard. Each algorithm was evaluated for sensitivity, specificity, and overall accuracy. Performance measures were reported with their 95% confidence intervals. We did not report positive and negative predictive values as these parameters are highly dependent on prevalence in the reference population, making them poorly generalizable.
Measurement of algorithm performance was performed using SAS software, version 9.4 (SAS Institute Inc., Carey, NC). This project was approved by the Research Ethics Boards of UHN and KHSC.
The characteristics of TCLD patients with HBV or HCV and all KHSC patients are presented in Table 2. The patients in the validation cohorts were, on average, middle-aged (median age 48-57 years), more likely to be male (57-60%), frequently low-income (26-28%), and mostly urban (74-99%). Many TCLD patients were either still consuming alcohol or had done so regularly in the past (37% of HBV patients, 64% of HCV patients). Few patients had been hospitalized in the previous year (1-16%), however many KHSC patients had visited the emergency department (44%). Most patients had not undergone a liver biopsy, but had been assessed clinically, including using non-invasive fibrosis testing (44-75%).
At re-abstraction of charts belonging to HBV patients (n = 176) from TCLD, Cohen's Kappa was 0.94 for cirrhosis, 1 for decompensated cirrhosis and 1 for hepatocellular carcinoma. For HCV patients (n = 148), Cohen's kappa was 0.99 for cirrhosis, 0.92 for decompensated cirrhosis, and 1 for hepatocellular carcinoma. For KHSC patients (n = 20), there was complete agreement (kappa = 1) for the presence of cirrhosis, decompensated cirrhosis and hepatocellular carcinoma.

Algorithms for decompensated cirrhosis
The algorithm with the greatest sensitivity (88% in HBV, 92% in HCV, 99% in KHSC group), was one physician visit code for cirrhosis and any of the following: one hospital diagnostic  code, death code, or procedure code (Table 4). For the HBV and HCV groups, this algorithm also had the highest overall accuracy (95%, CI 94-96% in HBV group; 88%, CI 87-90% in HCV group). The most specific algorithm was one physician visit code for cirrhosis and one hospital diagnostic code for decompensation (95% in HBV, 88% in HCV, 79% in the KHSC group); this was also the algorithm with the greatest overall accuracy in the KHSC group (83%, 95% CI 80-87%). After excluding a single hospital diagnostic code which accounted for many false positives (ICD-10 code for portal hypertension K766), the specificity of this algorithm was further improved to 98% in HBV, 95% in HCV, and 89% in KHSC groups.

Algorithms for hepatocellular carcinoma
The most sensitive algorithm for HCC was any of: two or more physician visit codes, a hospital diagnostic code, a death code or a procedure code with a physician visit code for cirrhosis (algorthim 9: sensitivity 96% in HBV, 97% in HCV, 96% in KHSC group, Table 5). The most specific algorithm not employing pathological data was one physician visit cirrhosis code and one procedure code (specificity 100% in both HBV and HCV, 97% in KHSC group). This compared favourably to the specificity of using the Ontario Cancer Registry (which includes pathology data), which had a specificity of 99% in HBV and HCV groups, 97% in KHSC group. The algorithm with the highest overall accuracy in the HBV (99%, CI 98-99), HCV (98%, CI 98-99%) and all-cause liver disease (96%, CI 94-98%) groups was a diagnosis in the Ontario Cancer Registry or a hospital diagnostic or death code for HCC.

Discussion
We identified sensitive and specific algorithms for the identification of cirrhosis, decompensated cirrhosis and HCC in patients with HBV or HCV infection, and confirmed these findings in patients with known liver disease of other causes. While identifying an algorithm that combines optimal sensitivity and specificity is desired, this is not always possible, and thus different criteria can be used depending on the study purpose. If the goal is to identify a cohort of individuals with a condition, highly specific criteria are preferable as this maximizes the likelihood that included individuals indeed have the target condition. In contrast, when the condition is the outcome variable in a study, a more sensitive definition can be used to prevent underestimation. In our study, the tradeoff between sensitivity and specificity was most notable for cirrhosis. In particular, outpatient physician claims for cirrhosis were very sensitive but not specific. We believe this is because, until recently, physicians in Ontario could receive additional payment for a visit with a diagnosis of cirrhosis. [27,28] On this basis, physicians may have been more likely to correctly list this diagnosis on their billing claim, or even to "up-code" patients with borderline clinical features consistent with cirrhosis. Although algorithms for cirrhosis and decompensated cirrhosis demonstrated inverse relationships between sensitivity and specificity, we were able to identify an algorithm for HCC that was both highly sensitive and specific.
Hospitalization diagnostic data are entered by trained chart abstractors and may be more reliable than outpatient physician billing claims. Hospitalization codes were more specific than physician visit codes for the target conditions in our study. The only exception was the in- hospital diagnostic code for portal hypertension, which falsely identified several patients as having decompensated cirrhosis. Non-cirrhotic portal hypertension is covered by this diagnostic code, as is portal hypertensive gastropathy, a condition diagnosed based on non-specific endoscopic findings, raising the possibility that it might be over-diagnosed. These conditions may explain this diagnostic code's lack of specificity. Overall, algorithms to identify complications in HBV-infected patients tended to be less sensitive but more specific than the same algorithms, when used in the HCV or KHSC groups. The difference was largest for algorithms identifying decompensated cirrhosis. We suspect that this can be explained by lower rates of hospital-based care for HBV-infected patients, who had fewer previous hospitalizations and emergency department visits than the other two groups. Substance use and mental health issues may contribute to greater healthcare usage by patients with HCV infection. [29] The range in results obtained in our study underscores the importance of testing administrative algorithms in several different patient populations. Although sensitivity, specificity and accuracy varied across patient groups, the most sensitive, specific and accurate algorithms in each group remained the same or very similar.
Previous studies have validated administrative data algorithms for cirrhosis, decompensation and HCC in U.S. populations. As we have done, others have also included codes for decompensation events as part of their administrative data definition of cirrhosis, or have combined HCC codes with cirrhosis codes to improve algorithm performance. [11,12] In one validation study of cirrhosis definitions, sensitivity using multiple codes was achieved at the expense of low specificity, similar to what we observed. [15] While prior studies were limited to ICD-9 codes, ours is the first study to test the performance of ICD-10 codes for the detection of liver disease outcomes. U.S. hospital data have been coded using ICD-10 since 2015. [19] The algorithms and codes we provide can now be used to identify liver disease outcomes using Canadian, U.S. and European health administrative data, as well as any data coded in the widely-used ICD-9 or ICD-10 systems.
Strengths of our study are the inclusion of inpatient and outpatient data, as well as procedures, cause of death and pathology results, all tested against a physician-confirmed reference standard. Further, we included a large number of individuals in three patient cohorts from two institutions, with different etiologies of liver disease. The consistency of the relative ranking of algorithms across patient groups suggests that the most sensitive or specific algorithms can be applied broadly to all patients with known liver disease. Finally, our study results are comprehensive as they can be used to identify all three important clinical outcomes in patients with chronic liver disease.
Our study has several limitations. First, the existence of a premium payment for physician visits in Ontario may indicate that cirrhosis billing codes were claimed more often in our study setting than they would be elsewhere. Without a premium code, one might expect that a single physician visit code for cirrhosis would have greater specificity and lower sensitivity than measured in our study. Second, the study of hepatology clinic patients has advantages and disadvantages. One advantage is that specialist physicians with expertise in evaluating patients for liver outcomes can be expected to have greater diagnostic accuracy than generalist physicians. However, patients seen in a specialist clinic are likely to have more severe disease, which can lead to spectrum bias. [30] We would expect this to be most relevant for cirrhosis, where there is a clear spectrum of disease ranging from asymptomatic to severely symptomatic, decompensated cirrhosis.
If patients in a hepatology clinic are sicker than the general population, we would expect our measured sensitivity to be higher, and specificity to be lower, than they would be in the general population. Our results are valid for patients like those seen in a hepatology clinic: that is, patients with diagnosed liver diseases. Therefore, studies validating health administrative data codes against patient data abstracted from community hospitals, primary care clinics, or the general population would be an important contribution to the literature. In the general population, our specific algorithms can be used to define cohorts, however they may not be sensitive enough to identify all cases, and as such may underestimate outcome rates. An additional caveat is that some decompensation codes (e.g. ascites) can occur without liver disease. In order to avoid misclassification when applied to the general population, these codes should be combined with liver disease or cirrhosis codes.

Conclusions
Liver diseases such as chronic HBV and HCV infection are a major cause of mortality worldwide. Administrative data can be used to identify large groups of patients with complications of chronic liver disease. We have reported the operating characteristics of algorithms for cirrhosis, decompensated cirrhosis, and hepatocellular carcinoma, using ICD-9 and ICD-10 health administrative data. We have identified sensitive and specific algorithms for each of these clinical conditions, which can be used to define patient cohorts or detect clinical outcomes. Our results will facilitate research into the adequacy of screening and treatment outcomes for patients with chronic HBV, HCV, or other liver diseases. Future research should test the performance of health administrative data codes in the general population.