https://orcid.org/0000-0002-8367-5294
Figures
Abstract
Target trial emulation applies design principles from randomised controlled trials to the analysis of observational data for causal inference and is increasingly used within human epidemiology. Veterinary electronic clinical records represent a potentially valuable source of information to estimate real-world causal effects for companion animal species. This study employed the target trial framework to evaluate the usefulness on veterinary observational data. Acute diarrhoea in dogs was used as a clinical exemplar. Inclusion required dogs aged ≥ 3 months and < 10 years, presenting for veterinary primary care with acute diarrhoea during 2019. Treatment strategies were: 1. antimicrobial prescription compared to no antimicrobial prescription and 2. gastrointestinal nutraceutical prescription compared to no gastrointestinal nutraceutical prescription. The primary outcome was clinical resolution (defined as no revisit with ongoing diarrhoea within 30 days from the date of first presentation). Informed from a directed acyclic graph, data on the following covariates were collected: age, breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, pyrexia, duration, additional treatment prescription and veterinary group. Inverse probability of treatment weighting was used to balance covariates between the treatment groups for each of the two target trials. The risk difference (RD) of 0.4% (95% CI -4.5% to 5.3%) was non-significant for clinical resolution in dogs treated with antimicrobials compared with dogs not treated with antimicrobials. The risk difference (RD) of 0.3% (95% CI -4.5% to 5.0%) was non-significant for clinical resolution in dogs treated with gastrointestinal nutraceuticals compared with dogs not treated with gastrointestinal nutraceuticals. This study successfully applied the target trial framework to veterinary observational data. The findings show that antimicrobial or gastrointestinal prescription at first presentation of acute diarrhoea in dogs causes no difference in clinical resolution. The findings support the recommendation for veterinary professionals to limit antimicrobial use for acute diarrhoea in dogs.
Citation: Pegram C, Diaz-Ordaz K, Brodbelt DC, Chang Y-M, Tayler S, Allerton F, et al. (2023) Target trial emulation: Do antimicrobials or gastrointestinal nutraceuticals prescribed at first presentation for acute diarrhoea cause a better clinical outcome in dogs under primary veterinary care in the UK? PLoS ONE 18(10): e0291057. https://doi.org/10.1371/journal.pone.0291057
Editor: Jan S. Suchodolski, Texas A&M University College Station, UNITED STATES
Received: June 6, 2023; Accepted: August 21, 2023; Published: October 4, 2023
Copyright: © 2023 Pegram et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data relevant to this study are available from figshare at DOI: 10.6084/m9.figshare.24065352.
Funding: CP is supported at the RVC by an award from the Dogs Trust Canine Welfare Grants (number 5654). URL: https://www.dogstrust.org.uk/how-we-help/the-future/research The funders reviewed the manuscript and were involved in the decision to publish, but did not play a role in study design, data collection or analysis.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: AIC, Akaike information criterion; AMR, antimicrobial resistance; ASGs, antimicrobial stewardship guidelines; BSAVA, British Small Animal Veterinary Association; CI, confidence interval; DAG, directed acyclic graph; EPR, electronic patient record; IP, inverse probability; IPTW, inverse probability of treatment weighting; IQR, interquartile range; RCT, randomised controlled trial; RD, risk difference; SMD, standardised mean difference
Introduction
Causal research methods are increasingly used within human epidemiological studies [1]. In basic terms, causal inference may refer to the process of drawing a conclusion that a specific treatment or exposure (i.e., intervention) was the “cause” of the effect (or outcome) that was observed [2]. Observational data (e.g. veterinary clinical records) are increasingly recognised as a valuable source of information to estimate real-world causal effects, especially in the absence of available randomised experiments (or in complement), and may provide a more realistic representation of a clinically relevant environment [3].
Causal inference from large observational databases (Big Data) can be viewed as an attempt to emulate the randomised experiment—the target experiment or target trial—that one would have liked to run to answer a specific question of interest [4]. Hernán and Robins (2016) have described a framework for research into comparative effectiveness using Big Data that aims to make the target trial explicit. In broad terms, the framework explicitly defines the “target trial” as the trial one would conduct if it were feasible and then describes how to emulate this target trial using observational data [4].
The VetCompass database contains large volumes of anonymised veterinary clinical data shared by UK veterinary practices that have the potential for analysis to answer a wide range of clinical causal questions. Such questions should be both clinically important and based on common conditions relevant to veterinarians in primary-care practice [5]. Diarrhoea is a commonly diagnosed condition in dogs, with one study reporting diarrhoea as the sixth most common disorder diagnosed in dogs under primary veterinary care in the UK in a one year period [5], with most cases considered mild and self-limiting [6]. Acute diarrhoea is commonly defined as lasting 7–14 days s [6–9], however, the majority (78.0%) of acute diarrhoea cases in dogs are reported to last for two days or shorter [10].
Dietary indiscretion is incriminated in the majority of acute diarrhoea cases in dogs, with affected animals generally responding well to supportive treatment [11]. Such treatment may include nutritional management, gastrointestinal nutraceutical and fluid therapy, with antimicrobials only recommended for dogs showing signs, or at high risk, of sepsis [6, 11–15]. Indeed, there is limited evidence for the clinical effectiveness of antimicrobials compared to placebo or gastrointestinal nutraceuticals for treatment of acute, uncomplicated diarrhoea [16, 17]. A large observational study including 2,429 dogs identified no association between prescription of antimicrobials and clinical resolution [18]. Although a large sample size doesn’t offset the limitations of observational studies, which generally report associations rather than causality [19], the absence of an improved response following antimicrobial therapy is in line with results from 5 prospective (randomised controlled) treatment trials [20–24].
Despite the lack of evidence for clinical effectiveness, antimicrobial prescription has been reported in over 50% of dogs with acute uncomplicated diarrhoea within 10 days of first veterinary presentation [18]. In these antimicrobial-treated cases, metronidazole was the most commonly prescribed systemic antimicrobial (47.0% of antimicrobial prescribing cases), followed by clavulanic acid potentiated amoxicillin (22.7%) [18]. Antimicrobial resistance (AMR) is considered one of the most serious and imminent health-related problems worldwide [15, 25], with indiscriminate antimicrobial use reportedly exacerbating the issue [26, 27]. As well as intra-species concerns with AMR, companion animals may act as a source of antimicrobial resistant enteric bacteria or resistance genes for their owners, given common closely shared owner-animal environments [26, 28–30]. Metronidazole specifically can also have long-term detrimental implications for the gut microbiome and metabolome, particularly in relation to bile acid dysfunction [17, 22, 31, 32]. Therefore, given the current available evidence, high prescription rates of antimicrobials for acute diarrhoea is concerning [17].
Studies evaluating the effectiveness of antimicrobials in the treatment of acute diarrhoea in dogs have also variably included gastrointestinal nutraceuticals (including prebiotics, probiotics, synbiotics, adsorbents, and motility modifiers) as a treatment group [18, 20, 21] or evaluated gastrointestinal nutraceuticals versus placebo only [33–36]. Some studies reported possible or minor acceleration in clinical resolution of canine acute diarrhoea with gastrointestinal nutraceutical treatment [33–36]. Singleton et al. (2019) found that prescription of gastrointestinal nutraceuticals, in combination with dietary modification, were positively associated with resolution of signs (OR 2.8, 95% CI 1.3–6.1), whereas Shmalberg et al. (2019) identified no significant difference in clinical resolution of diarrhoea in dogs treated with a gastrointestinal nutraceutical compared with placebo [18, 20]. This variability was highlighted in a systematic review, which pointed toward a very limited and possibly clinically unimportant effect for gastrointestinal nutraceuticals in the prevention or treatment of acute gastrointestinal disease [37]. Singleton et al. (2019) additionally evaluated the effect of anti-inflammatories, gastrointestinal agents, endoparasiticides and endectocides and dietary modification advice on diarrhoea resolution, with no significant associations identified for any of these treatments [18].
Using anonymised veterinary clinical data from the VetCompass Programme [38], the current study used the target trial framework on veterinary primary care observational data. Specifically, this study aimed to separately compare the effects on clinical resolution (defined as no revisit with ongoing diarrhoea within 30 days of first presentation) and time to treatment escalation for two common clinical management approaches: 1. antimicrobial prescription compared to no antimicrobial prescription and (separately) 2. gastrointestinal nutraceutical prescription compared to no gastrointestinal nutraceutical prescription for acute diarrhoea in dogs. This study aims to address the research gap for the clinical effectiveness of antimicrobials for acute diarrhoea in dogs [16, 17]. Given the high rates of antimicrobial prescription within treatment plans for acute diarrhoea in dogs [18] and growing concerns over AMR [15, 25], this study will provide causal evidence to guide prescribing practices by veterinary professionals. Additionally, the evidence for the clinical effectiveness of gastrointestinal nutraceuticals in the treatment of acute, uncomplicated diarrhoea in dogs is conflicting [37]. Therefore, this study can provide further evidence based on primary-care data as to whether gastrointestinal nutraceuticals have a clinical benefit or not. We hypothesized that there is no difference in “clinical resolution” or time to treatment escalation in dogs prescribed antimicrobials versus no antimicrobials or in dogs prescribed gastrointestinal nutraceuticals versus no gastrointestinal nutraceutical. Since the study utilises observational data, target trial emulation and causal inference analyses were adopted to attenuate as many sources of bias as possible.
Materials and methods
Data source and power calculation
The Veterinary Companion Animal Surveillance System (VetCompass), established in the UK in 2010, focuses on generating an improved evidence base from analysis of veterinary clinical data to support improved companion animal welfare [39]. VetCompass shares de-identified electronic patient record (EPR) data from over 1800 primary-care, referral, charity and emergency-care veterinary practices distributed throughout the UK for epidemiological research [38]. EPR data are extracted from practice management systems using integrated clinical queries [40] and uploaded to a secure VetCompass structured query language (SQL) database [41]. The acute diarrhoea study population included all available dogs under primary veterinary care at clinics participating in the VetCompass Programme during 2019. Dogs under veterinary care were defined as those with at least one EPR (free-text clinical note, treatment or bodyweight) recorded during 2019. Available data fields included a unique animal identifier along with species, breed, date of birth, sex, neuter status, insurance status and bodyweight, and also clinical information from free-form text clinical notes and treatment with relevant dates.
Since the study used the target trial emulation method [4], power calculations were based on an equivalence trial designed to demonstrate that a) antimicrobial therapy is equivalent to but not superior or inferior to no antibiotic therapy and b) gastrointestinal nutraceutical therapy is equivalent to but not superior or inferior to no gastrointestinal nutraceutical therapy for the treatment of acute diarrhoea in dogs [42]. This necessitated a sample size of 79 dogs in each arm. The calculation assumed 95% of cases in each treatment group resolved [18], 80% power, a 1:1 ratio of dogs prescribed antimicrobials or gastrointestinal nutraceuticals to dogs not prescribed antimicrobials or gastrointestinal nutraceuticals, a drop-out rate of 25% and a margin on the risk difference scale of 0.1 [18, 43], using risk calc Sample Size Calculator (https://riskcalc.org/samplesize/). Ethics approval was obtained from the RVC Social Science Ethical Review Board (reference number SR2018-1652).
Case definition, case finding and covariates
Only incident cases of diarrhoea were included in the study, with these cases defined as dogs that were diagnosed with acute diarrhoea between January 1, 2019 and December 31, 2019 without any recorded history of diarrhoea in the preceding 30 days within the available EPRs. In line with VetCompass methods used in several previous studies [5, 44–46], candidate diarrhoea cases were identified applying search terms relevant to the diagnosis and management of acute diarrhoea in the clinical notes (including “acute d+”, diarr*, D+, Protexin, Prokolin, loose stools and gastroenteritis). The search findings were merged, and a random subset of these candidate cases had their clinical notes examined manually in detail to identify dogs that met the case definition. The case definition comprised dogs aged ≥ 3 months and < 10 years first diagnosed with diarrhoea by the attending veterinarian during 2019. Ages to enrol based on consultation with internal medicine specialists and previous clinical trial reports [20, 21, 23]. Exclusion criteria comprised: diarrhoea due to acute haemorrhagic diarrhoea syndrome, parvovirus, endocrinopathy (including an acute flare-up of a chronic condition), liver disease, kidney disease, immune-mediated disease, pancreatitis, parasitic cause or non-steroidal anti-inflammatory drug (NSAID) use; antimicrobial or gastrointestinal nutraceutical prescription within 30 days prior to date of acute diarrhoea diagnosis; diarrhoea for greater than 7 days prior to first diagnosis; clinical dehydration (defined by admission for intravenous fluids); hospitalisation at first presentation for acute diarrhoea and death or euthanasia at first presentation for acute diarrhoea.
Based on existing evidence and expert knowledge, a directed acyclic graph (DAG) was constructed using DAGitty version 3.0, Fig 1, that encapsulated prior beliefs by the research team about the causal relationships relevant to the two questions of interest. The same covariate sets were believed to apply for both “Antimicrobial Prescription” and “Gastrointestinal Nutraceutical Prescription” as alternative exposures and “Clinical Resolution” and “Time to Treatment Escalation” as outcomes, therefore these variables replaced each other in the modelling. The DAG was used to identify which variables should be controlled for in the modelling [47], and therefore data on the following variables were collected: age, breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, pyrexia, duration, additional treatment prescription and veterinary group (Fig 1). Clinical resolution (defined as no revisit with ongoing diarrhoea within 30 days of first presentation) was evaluated as the primary outcome, with time to treatment escalation a secondary outcome (with the same covariate set applied).
For dogs meeting the diarrhoea case definition, demographic data were extracted automatically from the VetCompass database, with further data relating to clinical management extracted manually from the EPR (Table 1).
Target trial specification and emulation
Separate target trials were specified and emulated using EPR data to answer the two research questions of interest: 1. “Does antimicrobial prescription relative to no antimicrobial prescription at first presentation with acute diarrhoea in dogs cause a difference in clinical resolution and time to treatment escalation?” and 2. “Does gastrointestinal nutraceutical prescription relative to no gastrointestinal nutraceutical prescription at first presentation with acute diarrhoea in dogs cause a difference in clinical resolution and time to treatment escalation?” [4, 49, 50]. The protocols of the target trials, and the trial emulation, are summarised in Table 2.
Descriptive analysis
Demographic data were described. Continuous variables were assessed graphically for their distribution and summarised using median, interquartile range (IQR) and range given most of them were non-normally distributed. Chi-square test was used to compare categorical variables and the Student’s t-test or Mann–Whitney U test for univariable comparison of continuous variables between groups as appropriate [51].
Statistical analysis of the emulated trials.
Clinical resolution and time to treatment escalation were compared in a) dogs prescribed antimicrobials and dogs not prescribed antimicrobials for acute diarrhoea and b) dogs prescribed gastrointestinal nutraceuticals and dogs not prescribed gastrointestinal nutraceuticals for acute diarrhoea. For the analysis to be causal, the four assumptions of consistency, no interference, positivity and no unobserved confounding should hold [50]. The consistency assumption implies that an individual’s potential outcome under their observed exposure is the outcome that will actually be observed for that individual [52] i.e. the values of treatment under comparison correspond to well-defined interventions that correspond to the versions of treatment in the data [50]. No interference refers to the assumption that the potential outcomes of one individual are unaffected by the treatment assignment of other individuals [53]. Positivity refers to the assumption that the probability of receiving each treatment conditional on measured covariates is greater than zero [50]. Positivity violations occur when certain subgroups (defined by a combination of covariates) in a sample rarely or never receive some treatments of interest [54].
To emulate randomisation at baseline (diagnosis with acute diarrhoea), the following variables needed to be balanced between treatment groups: age, breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, pyrexia, duration, additional treatment prescription and veterinary group attended (as defined and categorised in Table 1 and based on the DAG in Fig 1). We assume that we have adjusted for enough variables to control for confounding. This is a strong assumption and there may be some confounders in a study that are unknown or not measured and hence unobserved.
To evaluate clinical resolution as an outcome, inverse probability of treatment weighting (IPTW) was used. IPTW is a propensity-score based method, with the propensity-score defined as the probability of treatment assignment conditional on observed baseline characteristics [55]. For IPTW, a pseudo-population is created by weighting each individual in the population by the inverse of the conditional probability of receiving the treatment level they indeed received [50]. The goal is to balance covariates between the two treatment groups [56].
To derive the weights using IPTW, a binary logistic regression model was fitted for each trial, with treatment (antimicrobial versus no antimicrobial prescription and gastrointestinal nutraceutical versus no gastrointestinal nutraceutical prescription) as the respective outcome regressed on the confounding variables described above (Table 1). Biologically plausible interaction terms were added to the model and their effect on the standardised mean differences (defined below) were assessed for inclusion, with interaction terms included if they improved covariate balance [57]. The linearity to the logit assumption was assessed by visually inspecting the scatter plot between the continuous predictor (age) and the logit values [58]. The model generated predicted probabilities for each dog of receiving either treatment, which were then used to calculate stabilised IP weights [59]. Extreme weights result when a treated patient has an extremely low or high propensity score but extreme weights can lead to potentially biased results by increasing the variability of the estimated treatment effect. The use of stabilised weights can be achieved by replacing the numerator (which is 1 in the unstabilised weights) with the crude probability of exposure (i.e. given by the propensity score model without covariates) [60, 61]. These weights were then used to weight each dog’s contribution to binary logistic regression outcome models for the antimicrobial and gastrointestinal nutraceutical target trials (Hernán and Robins, 2020 [50], Jiménez-Moro and Gómez, 2014 [62], Robins and Finkelstein, 2000 [63]), with clinical resolution (i.e. no revisit with ongoing diarrhoea within 30 days) as the outcome. The robust (or sandwich) variance estimator was used to obtain valid standard errors [64].
Standardised mean difference (SMD) examines the balance of covariate distribution between treatment groups [65]. For each covariate, SMD between pre- and post-IPTW were calculated, with SMD < 0.1 indicating good covariate balance between the two treatment arms. If SMD is > 0.1 for a variable, the propensity score model should first be re-assessed for improvement e.g. inclusion of further interaction terms, or the variable adjusted for in both the propensity score model and the outcome model to help further adjust for residual confounding [66, 67]. Effect modification was assessed by adding biologically plausible interaction terms to the outcome models and evaluating their effect on the confidence intervals, Akaike information criterion (AIC) and covariate balance (see S1 and S2 Tables in S1 File for results).
To evaluate time to treatment escalation in a) dogs prescribed antimicrobials versus dogs not prescribed antimicrobials for acute diarrhoea and b) dogs prescribed gastrointestinal nutraceuticals and dogs not prescribed gastrointestinal nutraceuticals for acute diarrhoea, unadjusted nonparametric estimation of time-to-event curves was firstly performed. Subsequently, we estimated time-to-event curves via IP weighted hazards models and calculated time-to-event probabilities (and associated 95% Cis) i.e. probability of no treatment escalation at 5-day time intervals [50].
Missing data were handled using the missing-indicator method, which uses a dummy variable in the statistical model to indicate whether the value for that variable is missing [68, 69]. The missing-indicator method assumes that the confounder variable is only a confounder (simultaneously associated with treatment and outcome) when observed, and not when missing. Additionally, we assumed that there is no interaction between the missing indicator and the fully observed confounder in the true propensity score [70]. Data were checked for internal validity and cleaned in Excel (Microsoft Office Excel 2013, Microsoft Corp.), with analyses conducted using R version 4.0.2 (R Core Team, Vienna, Austria). The “IPW” package was used to generate IP weights (and validated manually) [71] (2020). The “survey” package was used for binary logistic regression outcome modelling [72]. The “survival”, “ggplot2” and “survminer” packages were used for the causal time-to-event analysis and associated plots [73–75].
A flowchart summarising the study design and data analysis process is displayed in Fig 2.
Results
The study population consisted of 2,250,741 dogs under primary veterinary care in the VetCompass database during 2019. Diarrhoea search terms yielded 808,298 candidate cases, of which 8,080 (1.00%) were manually reviewed. Of these, 894 (11.1%) met the eligibility criteria for the emulated trial. The first emulated trial included: 355 (39.7%) dogs prescribed antimicrobials and 539 (60.3%) dogs not prescribed antimicrobials at first presentation for acute diarrhoea, whilst the second included 597 (66.8%) dogs prescribed gastrointestinal nutraceuticals and 297 (33.2%) dogs not prescribed gastrointestinal nutraceuticals at first presentation for acute diarrhoea. There were 207/894 (23.2%) dogs prescribed both antimicrobials and gastrointestinal nutraceuticals at first presentation for acute diarrhoea.
Demography of dogs in the emulated trials
Acute diarrhoea cases prescribed antimicrobials had a median age of 3.2 years (IQR 1.4–6.5, range 0.3–9.9) which was older than the median age of cases not prescribed antimicrobials (2.7 years, IQR 1.0–5.7, range 0.3–9.9) (p = 0.005). The median bodyweight of cases prescribed antimicrobials (12.3kg, IQR 7.6–24.6, range 1.1–56.0) did not differ significantly to the median bodyweight of cases not prescribed antimicrobials (11.7kg, IQR 7.3–22.6, range 1.7–70.6) (p = 0.437). The most prevalent breeds among cases prescribed antimicrobials were the Labrador Retriever (7.6% of cases prescribed antimicrobials; 27), German Shepherd Dog (5.9%; 21) and Cockapoo (5.1%; 18) in addition to 77 (21.7%) crossbreds. The most prevalent breeds among cases not prescribed antimicrobials were the Labrador Retriever (7.6%; 41), Cockapoo (5.9%; 32) and Staffordshire Bull Terrier (3.2%; 17) in addition to 130 (24.1%) crossbreds (Table 3).
Acute diarrhoea cases prescribed gastrointestinal nutraceuticals had a median age of 2.7 years (IQR 1.0–6.0, range 0.3–9.9), which did not differ significantly to the median age of cases not prescribed gastrointestinal nutraceuticals (3.2 years, IQR 1.2–6.2, range 0.3–9.9) (p = 0.172). The median bodyweight of cases prescribed gastrointestinal nutraceuticals (11.8kg, IQR 7.3–24.4, range 1.1–70.6) did not differ significantly to the median bodyweight of cases not prescribed gastrointestinal nutraceuticals (12.6kg, IQR 8.0–22.7, range 2.0–52.0) (p = 0.512). The most prevalent breeds among cases prescribed gastrointestinal nutraceuticals were the Labrador Retriever (7.7% of cases prescribed gastrointestinal nutraceuticals; n = 46), Cockapoo (6.5%; 39) and German Shepherd Dog (3.9%; 23) in addition to 138 (23.1%) crossbreds. The most prevalent breeds among cases not prescribed gastrointestinal nutraceuticals were the Labrador Retriever (7.4%; 22), Shih-tzu (4.4%; 13) and German Shepherd Dog (4.0%; 12) in addition to 69 (23.2%) crossbreds (Table 4).
Descriptive analysis of outcomes
The following descriptive analysis of clinical outcomes reports confounded estimates i.e., prior to adjusting for confounding. A similar proportion of dogs prescribed antimicrobials clinically resolved compared with dogs not prescribed antimicrobials (88.5% versus 87.4%, risk difference (RD) 1.1%). Likewise, a similar proportion of dogs prescribed gastrointestinal nutraceuticals clinically resolved compared with dogs not prescribed gastrointestinal nutraceuticals (88.3% versus 86.9%, RD 1.4%) (Table 5).
The RD estimates represent the risk in dogs treated compared with dogs not treated.
A similar proportion of dogs prescribed antimicrobials at first presentation for acute diarrhoea (38; 10.7%) had a treatment escalation compared with dogs not prescribed antimicrobials at first presentation (62; 11.5%). Likewise, a similar proportion of dogs prescribed gastrointestinal nutraceuticals had a treatment escalation (68; 11.4%) compared with dogs not prescribed gastrointestinal nutraceuticals (32; 10.8%). There were 9/894 (1.0%) dogs in both treatment trials with evidence of non-resolution, but these dogs were not prescribed further treatment. The median time (days) to treatment escalation in dogs prescribed antimicrobials (2.0, IQR 1.0–3.8, range 0.0–22.0) did not differ significantly to the median time to treatment escalation in dogs not prescribed antimicrobials (2.5, IQR 1.0–4.8, range 0.0–22.0) (p = 0.280). Conversely, the median time (days) to treatment escalation in dogs prescribed gastrointestinal nutraceuticals (3.0, IQR 1.0–5.0, range 0.0–22.0) was significantly longer than the time to treatment escalation in dogs not prescribed gastrointestinal nutraceuticals (1.0, IQR 0.8–2.3, range 0.0–22.0) (p = 0.004). There were 3/894 (0.3%) dogs that had a revisit for diarrhoea within 12 hours of their initial appointment.
Comparing unadjusted, and therefore potentially confounded, time-to-event from date of first presentation for acute diarrhoea to date of treatment escalation identified no significant difference in time-to-event in antimicrobial cases compared with no antimicrobial cases (log-rank test, P = 0.821) (Fig 3).
Comparing unadjusted time-to-event from date of first presentation for acute diarrhoea to date of treatment escalation identified no significant difference in time-to-event in gastrointestinal nutraceutical cases compared with no gastrointestinal nutraceutical cases (log-rank test, P = 0.493) (Fig 4).
Emulated trial results
The final model in the antimicrobial prescription trial included the following covariates to generate propensity scores: age (including a quadratic term), breed, bodyweight, insurance status, comorbidities, vomiting, age*vomiting (* indicating interaction), reduced appetite, haematochezia, pyrexia, haematochezia*pyrexia, duration, veterinary group, gastrointestinal nutraceutical prescription, dietary modification advice, gastrointestinal nutraceutical prescription*dietary modification advice, antiparasitic and gastrointestinal agent. The final model in the gastrointestinal nutraceutical prescription trial included the following covariates to generate propensity scores: age (including a quadratic term), breed, bodyweight, insurance status, comorbidities, vomiting, reduced appetite, haematochezia, age*haematochezia, pyrexia, duration, veterinary group, antimicrobial prescription, dietary modification advice, antiparasitic and gastrointestinal agent.
After balancing covariates between the antimicrobial and no antimicrobial prescribed dogs using IPTW, antimicrobial prescription caused no significant difference in clinical resolution compared with no antimicrobial prescription. Specifically, the risk difference for clinical resolution in dogs prescribed antimicrobials versus no antimicrobials was 0.4% (95% CI -4.5 to 5.3). Likewise, after balancing covariates between the gastrointestinal nutraceutical and non-gastrointestinal nutraceutical prescribed dogs using IPTW, gastrointestinal nutraceutical prescription caused no significant difference in clinical resolution compared with non-gastrointestinal nutraceutical prescription. Specifically, the risk difference for clinical resolution in dogs prescribed gastrointestinal nutraceuticals versus non-gastrointestinal nutraceuticals was 0.3% (95% CI -4.5 to 5.0). (Table 6).
The estimates represent the risk in dogs prescribed antimicrobials or gastrointestinal nutraceuticals using inverse probability of treatment weighting to adjust for confounding.
Time-to-event analysis for treatment escalation
Antimicrobial trial.
Following adjustment for confounding using an IP weighted hazards model, time-to-event curves (and corresponding 95% CIs) were estimated (Fig 5) i.e., the probability of not having a treatment escalation over time.
Time-to-event probabilities, i.e., the probability of no treatment escalation, were calculated from day 1 to 30 at regular time intervals (Table 7). The probabilities as if all dogs were treated with antimicrobials at first presentation for acute diarrhoea and the probabilities as if all dogs were not treated with antimicrobials were computed. The time-to-event probabilities marginally decreased over time in both groups, with a narrow difference in time-to-event probability (range 0.11–0.74%), with overlap in 95% confidence intervals at all time points.
Time-to-event probabilities were estimated via IP weighted hazards model.
Gastrointestinal nutraceutical trial.
Following adjustment for covariates using an IP weighted hazards model, time-to-event curves (and corresponding 95% CIs) were estimated (Fig 6) i.e., the probability of not having a treatment escalation over time.
Time-to-event probabilities and difference in probability were calculated from day 1 to 30 at regular time intervals (Table 8). The probabilities as if all dogs were treated with gastrointestinal nutraceuticals at first presentation for acute diarrhoea and the probabilities as if all dogs were not treated with gastrointestinal nutraceuticals were computed. The time-to-event probabilities marginally decreased over time in both groups, with a narrow difference in time-to-event probability, ranging from -0.42 to -3.50%. The confidence intervals were all below 0 from day 10 onwards, indicating that prescribing gastrointestinal nutraceuticals to all dogs at the first presentation for acute diarrhoea, it would cause an increase in treatment escalation from day 10 onwards compared with dogs not prescribed gastrointestinal nutraceuticals at first presentation.
Time-to-event probabilities were estimated via IP weighted hazards model.
Discussion
Interest in the target trial emulation framework is gaining momentum in human epidemiology [76], with the current study being one of the pioneering applications of target trial emulation in the veterinary field. The framework aims to make the ‘target trial’ explicit, which helps avoid common design flaws and methodological pitfalls in the analysis of non-randomised studies, keeping each step transparent and accessible [4]. Indeed, studies in the human medical literature have demonstrated that it is possible to approximate the results from well conducted RCTs using the target trial framework, combined with using suitable analytical methods [77–82]. Therefore, this is a powerful approach, which the current study has demonstrated can be successfully applied to veterinary EHR data.
After balancing covariates between the antimicrobial and no antimicrobial groups using IPTW, antimicrobial prescription at first presentation of acute diarrhoea in dogs did not cause a significant change (ie neither an increase or decrease) in proportional clinical resolution (no revisit with ongoing diarrhoea within 30 days of first presentation) (RD 0.4%, 95% CI -4.5% to 5.3%) compared with no antimicrobial prescription. Time-to-event analysis similarly identified no significant difference in the probability of treatment escalation between dogs prescribed antimicrobials and dogs not prescribed antimicrobials at first presentation of acute diarrhoea. Therefore, these results provide no evidence supporting a difference in “clinical resolution” risk or time to treatment escalation in dogs prescribed antimicrobials versus not prescribed antimicrobials at first presentation of acute diarrhoea.
These findings are in line with those of a large observational study [18] and previous clinical trial reports, which also did not identify a significant difference in clinical resolution between dogs prescribed antimicrobials and dogs not prescribed antimicrobials [20–23]. The current results provide further evidence to support the British Small Animal Veterinary Association (BSAVA) “PROTECT ME” guidelines, which state that dogs presenting with acute gastrointestinal signs, including dogs with haemorrhagic diarrhoea that are systemically well, do not require antimicrobial therapy [83]. Likewise, the results support European antimicrobial stewardship guidelines (ASGs), as reported in a recent review, where all evaluated ASGs did not recommend antimicrobials for treatment of acute diarrhoea in dogs [14]. Given a relatively high proportion of dogs were prescribed antimicrobials at first presentation of acute diarrhoea in the current study (39.7%), there still appears to be a disconnect between guidance and clinical practice. That said, the proportion treated in this 2019 cohort is lower than a previous report of 52.5% dogs with acute uncomplicated diarrhoea prescribed antimicrobials within 10 days of first veterinary presentation from 2014–2017 [18]. Therefore, this could represent moves within the veterinary profession towards decreasing prescription of antimicrobials for acute diarrhoea over time, although different study designs and case definitions may in part explain this variation.
After balancing covariates between the gastrointestinal nutraceutical and no gastrointestinal nutraceutical treatment groups using IPTW, gastrointestinal nutraceutical prescription at first presentation of acute diarrhoea in dogs did not cause a significant increase or decrease in clinical resolution compared with no gastrointestinal nutraceutical prescription. Time-to-event analysis identified that if all dogs were prescribed gastrointestinal nutraceuticals at first presentation for acute diarrhoea, it would cause an increase in treatment escalation from day 10 onwards compared with dogs not prescribed gastrointestinal nutraceuticals at first presentation. However, although significant, the difference in time-to-event probability was minor.
The current study findings of no significant difference in “clinical resolution” of acute diarrhoea between dogs prescribed gastrointestinal nutraceuticals and dogs not prescribed gastrointestinal nutraceuticals fits with a previous systematic review. That systematic review pointed toward a very limited and possibly clinically unimportant effect for gastrointestinal nutraceuticals in the prevention or treatment of acute gastrointestinal disease [37]. A previous large-scale observational study based on primary-care data found that prescription of gastrointestinal nutraceuticals, in combination with dietary modification, were positively associated with resolution of signs [18]. It is possible that dietary modification is the overriding factor for clinical resolution, rather than gastrointestinal nutraceuticals. The current study collected data on “dietary modification advice” as an adjustment variable, however the specific diet fed will vary and it is unknown to what extent the advice was heeded. Further prospective studies evaluating diet as a primary exposure might help to clarify the contribution of dietary modification in clinical resolution of acute diarrhoea.
Although minimal, dogs prescribed gastrointestinal nutraceuticals at first presentation of acute diarrhoea were more likely to have a treatment escalation than dogs not prescribed gastrointestinal nutraceuticals from day 10 onwards. Gastrointestinal nutraceuticals are often prescribed as a first-line treatment for acute diarrhoea [18], therefore, this might indicate that owners of dogs prescribed gastrointestinal nutraceuticals at first presentation exhibit health-seeking behaviour and are more invested in a clinical response. Further studies, including qualitative studies exploring owner expectations for acute diarrhoea treatment and resolution, might help to clarify the effect identified in the current study.
The limitations of this study are largely based on the nature of retrospective analysis of electronic patient record data, including issues related to unobserved confounding, missing and misclassified data and application of a case definition to the data available [84]. Although analysis of VetCompass EPR data can generate evidence generalisable to the overall dog population, case definitions, diagnosis recording and history-taking relies on the clinical acumen and note-taking of attending practitioners, with no attempts made by the researchers to second-guess diagnoses [84]. “Clinical resolution” is difficult to define within retrospective primary-care veterinary EPR data, especially for an acute and often self-limiting condition. Clinical resolution defined as no revisit with ongoing diarrhoea within 30 days of first presentation was reasoned as an appropriate outcome, in line with a previous study based on primary-care EPR data [18]. Dogs might have been prescribed treatments other than antimicrobials and gastrointestinal nutraceuticals which could affect outcome, however concurrent treatments were included as covariates to help account for these differences. Expert opinion was sought in construction of the DAG, however it is possible unmeasured confounders could influence the risk differences and time-to-event probabilities calculated.
Causal inference methods, and specifically the target trial emulation framework presented in this study, focus on the relationship between exposure and outcome [85]. We described the variables collected as covariates (as identified in the DAG) for clarity, however we did not specifically analyse confounder effects to avoid “Table 2 Fallacy” [86]. This being that presenting multiple estimated effect measures from the same model encourages the reader to interpret all these estimates in the same way, however the interpretation of a confounder effect estimate may be different than for the exposure effect estimate, due to underlying differences in the causal model [86].
Conclusions
Overall, this study demonstrated successful application of the target trial framework to veterinary observational data. Acute diarrhoea was used as the condition of interest, with the findings showing that antimicrobial or gastrointestinal prescription at first presentation of acute diarrhoea in dogs causes no difference in proportional clinical resolution (i.e., no revisit with ongoing diarrhoea within 30 days of first presentation). Time-to-event analysis identified no significant difference in the probability of treatment escalation between dogs prescribed antimicrobials and dogs not prescribed antimicrobials at first presentation of acute diarrhoea. Dogs prescribed gastrointestinal nutraceuticals at first presentation of acute diarrhoea were more likely to have a treatment escalation than dogs not prescribed gastrointestinal nutraceuticals from day 10 onwards, however the difference was minimal. Overall, the majority of dogs did not have a revisit for the diarrhoea episode, or a treatment escalation, whether they were prescribed antimicrobials or gastrointestinal nutraceuticals at first presentation or not. These findings support the recommendation to restrict the use of antimicrobials for acute diarrhoea in dogs within the veterinary profession.
Acknowledgments
Thanks to Noel Kennedy (RVC) for VetCompass™ software and programming development. We acknowledge the Medivet Veterinary Partnership, Vets4Pets/Companion Care, Goddard Veterinary Group, CVS Group, IVC Evidensia, Linnaeus Group, Beaumont Sainsbury Animal Hospital, PDSA, Blue Cross, Dogs Trust, Vets Now and the other UK practices who collaborate in VetCompass™. We are grateful to The Kennel Club Charitable Trust, Agria Pet Insurance and The Kennel Club for supporting VetCompass™.
References
- 1. Musci RJ, Stuart E. Ensuring causal, not casual, inference. Prevention Science. 2019;20(3):452–6. pmid:30613853
- 2. Frey BB. The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. 2018.
- 3. Maringe C, Benitez Majano S, Exarchakou A, Smith M, Rachet B, Belot A, et al. Reflection on modern methods: trial emulation in the presence of immortal-time bias. Assessing the benefit of major surgery for elderly lung cancer patients using observational data. International Journal of Epidemiology. 2020;49(5):1719–29. pmid:32386426
- 4. Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American journal of epidemiology. 2016;183(8):758–64. pmid:26994063
- 5. O’Neill DG, James H, Brodbelt DC, Church DB, Pegram C. Prevalence of commonly diagnosed disorders in UK dogs under primary veterinary care: results and applications. BMC Veterinary Research. 2021;17(1):1–14.
- 6. Armstrong J. Approach to diagnosis and therapy of the patient with acute diarrhea. Todays Vet Prac. 2013:20–56.
- 7. Brandt KG, Antunes MMdC, Silva GAPd. Acute diarrhea: evidence-based management. Jornal de pediatria. 2015;91:S36–S43. pmid:26351768
- 8. Guarino A, Vecchio AL, Dias JA, Berkley JA, Boey C, Bruzzese D, et al. Universal recommendations for the management of acute diarrhea in nonmalnourished children. Journal of pediatric gastroenterology and nutrition. 2018;67(5):586. pmid:29901556
- 9. Candellone A, Cerquetella M, Girolami F, Badino P, Odore R. Acute Diarrhea in Dogs: Current Management and Potential Role of Dietary Polyphenols Supplementation. Antioxidants (Basel). 2020;9(8). pmid:32784917
- 10. Hubbard K, Skelly B, McKelvie J, Wood J. Risk of vomiting and diarrhoea in dogs. Veterinary record. 2007;161(22):755. pmid:18056013
- 11. Battersby I, Harvey A. Differential diagnosis and treatment of acute diarrhoea in the dog and cat. In Practice. 2006;28(8):480–8.
- 12. Marks S, Rankin S, Byrne B, Weese J. Enteropathogenic bacteria in dogs and cats: diagnosis, epidemiology, treatment, and control. Journal of veterinary internal medicine. 2011;25(6):1195–208. pmid:22092607
- 13. Unterer S, Busch K. Acute hemorrhagic diarrhea syndrome in dogs. Veterinary Clinics: Small Animal Practice. 2021;51(1):79–92. pmid:33131918
- 14. Allerton F, Prior C, Bagcigil AF, Broens E, Callens B, Damborg P, et al. Overview and Evaluation of Existing Guidelines for Rational Antimicrobial Use in Small-Animal Veterinary Practice in Europe. Antibiotics. 2021;10(4):409. pmid:33918617
- 15. Cerquetella M, Rossi G, Suchodolski J, Schmitz SS, Allenspach K, Rodríguez‐Franco F, et al. Proposal for rational antibacterial use in the diagnosis and treatment of dogs with chronic diarrhoea. Journal of Small Animal Practice. 2020;61(4):211–5. pmid:32065388
- 16. Moore E, Gordon-Evans W. Should we prescribe oral metronidazole or probiotics for acute gastroenteritis in dogs? Veterinary Evidence. 2022;7(2).
- 17. Rogers-Smith E. The use of metronidazole in adult dogs with acute onset, uncomplicated, diarrhoea. Veterinary Evidence. 2021;6(4).
- 18. Singleton DA, Noble P, Sánchez-Vizcaíno F, Dawson S, Pinchbeck GL, Williams NJ, et al. Pharmaceutical prescription in canine acute diarrhoea: a longitudinal electronic health record analysis of first opinion veterinary practices. Frontiers in veterinary science. 2019:218. pmid:31334254
- 19.
Dohoo I, Martin S, Stryhn H. Veterinary epidemiologic research. VER. Inc, Charlottetown, PE, Canada. 2009.
- 20. Shmalberg J, Montalbano C, Morelli G, Buckley GJ. A randomized double blinded placebo-controlled clinical trial of a probiotic or metronidazole for acute canine diarrhea. Frontiers in veterinary science. 2019;6:163. pmid:31275948
- 21. Pignataro G, Di Prinzio R, Crisi PE, Belà B, Fusaro I, Trevisan C, et al. Comparison of the Therapeutic Effect of Treatment with Antibiotics or Nutraceuticals on Clinical Activity and the Fecal Microbiome of Dogs with Acute Diarrhea. Animals. 2021;11(6):1484. pmid:34063855
- 22. Chaitman J, Ziese A-L, Pilla R, Minamoto Y, Blake AB, Guard BC, et al. Fecal microbial and metabolic profiles in dogs with acute diarrhea receiving either fecal microbiota transplantation or oral metronidazole. Frontiers in veterinary science. 2020;7:192. pmid:32363202
- 23. Werner M, Suchodolski JS, Straubinger RK, Wolf G, Steiner JM, Lidbury JA, et al. Effect of amoxicillin‐clavulanic acid on clinical scores, intestinal microbiome, and amoxicillin‐resistant Escherichia coli in dogs with uncomplicated acute diarrhea. Journal of veterinary internal medicine. 2020;34(3):1166–76. pmid:32324947
- 24. Rudinsky AJ, Parker VJ, Winston J, Cooper E, Mathie T, Howard JP, et al. Randomized controlled trial demonstrates nutritional management is superior to metronidazole for treatment of acute colitis in dogs. Journal of the American Veterinary Medical Association. 2022;260(S3):S23–S32. pmid:36191142
- 25.
World Health Organization. Global antimicrobial resistance surveillance system (GLASS) report: early implementation 2017–2018. 2018.
- 26. Anholt RM, Berezowski J, Ribble CS, Russell ML, Stephen C. Using informatics and the electronic medical record to describe antimicrobial use in the clinical management of diarrhea cases at 12 companion animal practices. PLoS One. 2014;9(7):e103190. pmid:25057893
- 27. Morley PS, Apley MD, Besser TE, Burney DP, Fedorka‐Cray PJ, Papich MG, et al. Antimicrobial drug use in veterinary medicine. Journal of veterinary internal medicine. 2005;19(4):617–29. pmid:16095186
- 28. Weese JS. Antimicrobial resistance in companion animals. Animal Health Research Reviews. 2008;9(2):169–76. pmid:18983722
- 29. Guardabassi L, Schwarz S, Lloyd DH. Pet animals as reservoirs of antimicrobial-resistant bacteria. Journal of antimicrobial chemotherapy. 2004;54(2):321–32. pmid:15254022
- 30. Prescott JF, Hanna WB, Reid-Smith R, Drost K. Antimicrobial drug use and resistance in dogs. The Canadian veterinary journal. 2002;43(2):107. pmid:11842592
- 31. Pilla R, Suchodolski JS. The Role of the Canine Gut Microbiome and Metabolome in Health and Gastrointestinal Disease. Front Vet Sci. 2019;6:498. pmid:31993446
- 32. Igarashi H, Maeda S, Ohno K, Horigome A, Odamaki T, Tsujimoto H. Effect of oral administration of metronidazole or prednisolone on fecal microbiota in dogs. PLoS One. 2014;9(9):e107909. pmid:25229475
- 33. Gómez-Gallego C, Junnila J, Männikkö S, Hämeenoja P, Valtonen E, Salminen S, et al. A canine-specific probiotic product in treating acute or intermittent diarrhea in dogs: A double-blind placebo-controlled efficacy study. Vet Microbiol. 2016;197:122–8. pmid:27938673
- 34. Herstad H, Nesheim B, L’Abée‐Lund T, Larsen S, Skancke E. Effects of a probiotic intervention in acute canine gastroenteritis–a controlled clinical trial. Journal of small animal practice. 2010;51(1):34–8. pmid:20137007
- 35. Ziese AL, Suchodolski JS, Hartmann K, Busch K, Anderson A, Sarwar F, et al. Effect of probiotic treatment on the clinical course, intestinal microbiome, and toxigenic Clostridium perfringens in dogs with acute hemorrhagic diarrhea. PLoS One. 2018;13(9):e0204691. pmid:30261077
- 36. Nixon SL, Rose L, Muller AT. Efficacy of an orally administered anti‐diarrheal probiotic paste (Pro‐Kolin Advanced) in dogs with acute diarrhea: A randomized, placebo‐controlled, double‐blinded clinical study. Journal of veterinary internal medicine. 2019;33(3):1286–94. pmid:30882953
- 37. Jensen AP, Bjørnvad CR. Clinical effect of probiotics in prevention or treatment of gastrointestinal disease in dogs: A systematic review. J Vet Intern Med. 2019;33(5):1849–64. pmid:31313372
- 38.
VetCompass. VetCompass™ Programme London: RVC Electronic Media Unit; 2019 [Available from: http://www.rvc.ac.uk/VetCOMPASS/.
- 39.
RVC. VetCompass overview 2019 [Available from: https://www.rvc.ac.uk/vetcompass/about.
- 40. Kearsley-Fleet L, O’neill D, Volk H, Church D, Brodbelt D. Prevalence and risk factors for canine epilepsy of unknown origin in the UK. Veterinary Record. 2013:vetrec-2012-101133.
- 41. O’Neill DG, Church DB, McGreevy PD, Thomson PC, Brodbelt DC. Longevity and mortality of owned dogs in England. Vet J. 2013;198(3):638–43. pmid:24206631
- 42. Christensen E. Methodology of superiority vs. equivalence trials and non-inferiority trials. Journal of Hepatology. 2007;46(5):947–54. pmid:17412447
- 43. Althunian TA, de Boer A, Groenwold RH, Klungel OH. Defining the noninferiority margin and analysing noninferiority: an overview. British journal of clinical pharmacology. 2017;83(8):1636–42. pmid:28252213
- 44. O’Neill DG, Rowe D, Brodbelt DC, Pegram C, Hendricks A. Ironing out the wrinkles and folds in the epidemiology of skin fold dermatitis in dog breeds in the UK. Scientific Reports. 2022;12(1):10553. pmid:35794173
- 45. O’Neill DG, Yin Y, Tetas Pont R, Brodbelt DC, Church DB, Pegram C, et al. Breed and conformational predispositions for prolapsed nictitating membrane gland (PNMG) in dogs in the UK: A VetCompass study. Plos one. 2022;17(1):e0260538. pmid:35081121
- 46. Pegram C, Brodbelt DC, Diaz-Ordaz K, Chang Y, von Hekkel AF, Church DB, et al. Risk factors for unilateral cranial cruciate ligament rupture diagnosis and for clinical management in dogs under primary veterinary care in the UK. The Veterinary Journal. 2023;292:105952. pmid:36708945
- 47. Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research. Epidemiology. 1999:37–48. pmid:9888278
- 48.
The VeNom Coding Group. VeNom Veterinary Nomenclature: VeNom Coding Group; 2019 [Available from: http://venomcoding.org.
- 49. Danaei G, Rodríguez LAG, Cantero OF, Logan R, Hernán MA. Observational data for comparative effectiveness research: an emulation of randomised trials of statins and primary prevention of coronary heart disease. Statistical methods in medical research. 2013;22(1):70–96. pmid:22016461
- 50.
Hernán MA, Robins JM. Causal inference: what if. Boca Raton: Chapman & Hall/CRC; 2020.
- 51.
Kirkwood Kirkwood B., BR Sterne, JAC (Eds.), Essential medical statistics. Blackwell Science, Malden, Mass. Oxford; 2003.
- 52. Rehkopf DH, Glymour MM, Osypuk TL. The consistency assumption for causal inference in social epidemiology: when a rose is not a rose. Current epidemiology reports. 2016;3(1):63–71. pmid:27326386
- 53. Hudgens MG, Halloran ME. Toward causal inference with interference. Journal of the American Statistical Association. 2008;103(482):832–42. pmid:19081744
- 54. Petersen ML, Porter KE, Gruber S, Wang Y, Van Der Laan MJ. Diagnosing and responding to violations in the positivity assumption. Statistical methods in medical research. 2012;21(1):31–54. pmid:21030422
- 55. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399–424. pmid:21818162
- 56.
Barter R. 2017. Available from: https://www.rebeccabarter.com/blog/2017-07-05-ip-weighting/.
- 57. Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661–79. pmid:26238958
- 58. Osborne JW, Waters E. Four assumptions of multiple regression that researchers should always test. Practical assessment, research, and evaluation. 2002;8(1):2.
- 59. Xu S, Ross C, Raebel MA, Shetterly S, Blanchette C, Smith D. Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals. Value in Health. 2010;13(2):273–7. pmid:19912596
- 60. Allan V, Ramagopalan SV, Mardekian J, Jenkins A, Li X, Pan X, et al. Propensity score matching and inverse probability of treatment weighting to address confounding by indication in comparative effectiveness research of oral anticoagulants. Journal of comparative effectiveness research. 2020;9(9):603–14. pmid:32186922
- 61. Chesnaye NC, Stel VS, Tripepi G, Dekker FW, Fu EL, Zoccali C, et al. An introduction to inverse probability of treatment weighting in observational research. Clinical Kidney Journal. 2022;15(1):14–20. pmid:35035932
- 62. Jiménez-Moro JL, Gómez J. Inverse probability of censoring weighting for selective crossover in oncology clinical trials. Paper SP02. 2014.
- 63. Robins JM, Finkelstein DM. Correcting for noncompliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted (IPCW) log‐rank tests. Biometrics. 2000;56(3):779–88. pmid:10985216
- 64. Zou G. A modified poisson regression approach to prospective studies with binary data. American journal of epidemiology. 2004;159(7):702–6. pmid:15033648
- 65. Zhang Z, Kim HJ, Lonjon G, Zhu Y. Balance diagnostics after propensity score matching. Annals of translational medicine. 2019;7(1). pmid:30788363
- 66. Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Statistics in medicine. 2009;28(25):3083–107. pmid:19757444
- 67. Yang D, Dalton J. A unified approach to measuring the effect size between two groups using SAS®. SAS Global Forum; 2012. Contract No: Paper. 2014:335–2012.
- 68. Burton A, Altman D. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines. British journal of cancer. 2004;91(1):4–8. pmid:15188004
- 69. Miettinen OS. Theoretical epidemiology: principles of occurrence research in medicine. Theoretical epidemiology: principles of occurrence research in medicine2012. p. 359–.
- 70. Blake HA, Leyrat C, Mansfield KE, Seaman S, Tomlinson LA, Carpenter J, et al. Propensity scores using missingness pattern information: a practical guide. Statistics in medicine. 2020;39(11):1641–57. pmid:32103533
- 71. van der Wal WM, Geskus RB. ipw: an R package for inverse probability weighting. Journal of Statistical Software. 2011;43:1–23.
- 72. Oberski D. lavaan. survey: An R package for complex survey analysis of structural equation models. Journal of statistical software. 2014;57:1–27.
- 73.
Therneau T, Lumley T. R survival package. R Core Team. 2013.
- 74. Wickham H, Chang W, Wickham MH. Package ‘ggplot2’. Create elegant data visualisations using the grammar of graphics Version. 2016;2(1):1–189.
- 75. Kassambara A, Kosinski M, Biecek P, Fabian S. Package ‘survminer’. Drawing Survival Curves using ‘ggplot2’(R package version 03 1). 2017.
- 76. Gomes M, Latimer N, Soares M, Dias S, Baio G, Freemantle N, et al. Target trial emulation for transparent and robust estimation of treatment effects for health technology assessment using real-world data: opportunities and challenges. Pharmacoeconomics. 2022;40(6):577–86. pmid:35332434
- 77. Lodi S, Phillips A, Lundgren J, Logan R, Sharma S, Cole SR, et al. Effect estimates in randomized trials and observational studies: comparing apples with apples. American journal of epidemiology. 2019;188(8):1569–77. pmid:31063192
- 78. Hernán MA, Alonso A, Logan R, Grodstein F, Michels KB, Stampfer MJ, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology (Cambridge, Mass). 2008;19(6):766. pmid:18854702
- 79. Admon AJ, Donnelly JP, Casey JD, Janz DR, Russell DW, Joffe AM, et al. Emulating a Novel Clinical Trial Using Existing Observational Data. Predicting Results of the PreVent Study. Ann Am Thorac Soc. 2019;16(8):998–1007. pmid:31038996
- 80. Boyne DJ, Cheung WY, Hilsden RJ, Sajobi TT, Batra A, Friedenreich CM, et al. Association of a Shortened Duration of Adjuvant Chemotherapy With Overall Survival Among Individuals With Stage III Colon Cancer. JAMA Netw Open. 2021;4(3):e213587. pmid:33783516
- 81. Dickerman BA, García-Albéniz X, Logan RW, Denaxas S, Hernán MA. Avoidable flaws in observational analyses: an application to statins and cancer. Nature medicine. 2019;25(10):1601–6. pmid:31591592
- 82. Emilsson L, García-Albéniz X, Logan RW, Caniglia EC, Kalager M, Hernán MA. Examining bias in studies of statin treatment and survival in patients with cancer. JAMA oncology. 2018;4(1):63–70. pmid:28822996
- 83.
BSAVA. BSAVA/SAMSoc Guide to responsible use of antibacterials: protect me: British Small Animal Veterinary Association; 2018.
- 84. O’Neill DG, Church DB, McGreevy PD, Thomson PC, Brodbelt DC. Prevalence of disorders recorded in dogs attending primary-care veterinary practices in England. PloS one. 2014;9(3):e90501. pmid:24594665
- 85. Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. Journal of clinical epidemiology. 2016;79:70–5. pmid:27237061
- 86. Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. American journal of epidemiology. 2013;177(4):292–8. pmid:23371353