https://orcid.org/0000-0002-9588-2944
Figures
Abstract
Valproate is the most effective treatment for idiopathic generalised epilepsy. Currently, its use is restricted in women of childbearing potential owing to high teratogenicity. Recent evidence extended this risk to men’s offspring, prompting recommendations to restrict use in everybody aged <55 years. This study will evaluate mortality and morbidity risks associated with valproate withdrawal by emulating a hypothetical randomised-controlled trial (called a “target trial”) using retrospective observational data. The data will be drawn from ~250m mainly US patients in the TriNetX repository and ~60m UK patients in Clinical Practice Research Datalink (CPRD). These will be scanned for individuals aged 16–54 years with epilepsy and on valproate who either continued, switched to lamotrigine or levetiracetam, or discontinued valproate between 2014–2024, creating four groups. Randomisation to these groups will be emulated by baseline confounder adjustment using g-methods. Mortality and morbidity outcomes will be assessed and compared between groups over 1–10 years, employing time-to-first-event and recurrent events analyses. A causal prediction model will be developed from these data to aid in predicting the safest alternative antiseizure medications. Together, these findings will optimise informed decision-making about valproate withdrawal and alternative treatment selection, providing immediate and vital information for patients, clinicians and regulators.
Citation: Mbizvo GK, Martin GP, Sperrin M, Bonnett LJ, Schofield P, Buchan I, et al. (2024) An international study to investigate and optimise the safety of discontinuing valproate in young men and women with epilepsy: Protocol. PLoS ONE 19(8): e0306226. https://doi.org/10.1371/journal.pone.0306226
Editor: Johanna Pruller, PLOS: Public Library of Science, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: June 25, 2024; Accepted: July 2, 2024; Published: August 29, 2024
Copyright: © 2024 Mbizvo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.
Funding: This study is funded by an Academy of Medical Sciences (AMS) Starter Grant for Clinical Lecturers (REF: SGL030\1029), and an Epilepsy Research Institute Emerging Leader Fellowship (F2401) awarded to G.KM. G.K.M. is also supported by a National Institute for Health and Care Research (NIHR) Clinical Lectureship (CL-2022-07-002). AGM is supported by the NIHR Applied Research Collaboration North West Coast (ARC NWC). The funders had no role in the study design, data collection, analysis and interpretation, or writing of this manuscript. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, AMS, the Epilepsy Research Institute, or the Department of Health and Social Care.
Competing interests: A.G.M. declares i) a UCB Pharma grant paid to University of Liverpool for the National Audit of Seizure Management in Hospitals (NASH) study, which is unrelated to the submitted work; ii) an Angelini grant to be paid to University of Liverpool as co-applicant for A multi-method PRoject to maximise efficient and equitable pathways tO suPport from a rEgional epiLepsy centre (PROPEL), which is unrelated to the submitted work; iii) Honoraria paid to University of Liverpool for lectures unrelated to the submitted work given at educational events sponsored by Sanofi, Eiasi, and GSK; iii) Support from Angelini for attendance unrelated to the submitted work at the 2024 International League Against Epilepsy (ILAE) congress. G.K.M declares an Angelini grant to be paid to University of Liverpool as co-applicant on the PROPEL study, which is unrelated to the submitted work; ii) Honoraria to be paid to University of Liverpool for delivering a lecture at an educational event sponsored by Angelini which was unrelated to the submitted work. The remaining authors declare no competing interests. This does not alter our adherence to PLOS ONE policies on sharing data and materials. All authors confirm there are no patents, products in development or marketed products associated with this research to declare.
Introduction
Epilepsy impacts 70 million people worldwide [1]. Valproate is one of the most effective drugs for treating epilepsy, outperforming any other in idiopathic generalised epilepsy, which affects a third of people with epilepsy (PWE) [2,3]. It was the most prescribed antiseizure medication (ASM) globally in the early 2000s [4]. However, widespread evidence demonstrating foetal congenital malformations (11%) and neurodevelopmental disorders (30–40%) associated with prenatal valproate exposure led to guidance changes between 2015–2018 [5]. These prohibited valproate prescribing in women of childbearing potential unless other ASMs had failed, subsequently halving prescriptions in women [4,5].
Valproate remains the first-line ASM for men with newly diagnosed generalised epilepsy in most countries [2,3]. In the UK, regulators recently considered valproate’s known risks of impaired male fertility, emerging pre-clinical evidence of transgenerational risks with prenatal exposure, and animal studies suggesting adverse effects on testes [5,6]. In response, they announced a phased implementation (from January 2024) of recommendations restricting valproate prescribing for both men and women aged <55 years unless other ASMs have failed [5,6]. Similar restrictions are being considered in other countries [7–9]. Together, they are likely to result in withdrawal of valproate from a substantial number of men already taking it, as has been the case for women [4,5].
To support informed and safe decision-making [10], it is vital to provide these young men and women and their clinicians with comprehensive information on the safety of valproate withdrawal. Whether there are any potential risks of personal harm or morbidity (including hospital admissions, injuries, or death) associated with valproate withdrawal has not yet been quantified for either men or women on valproate [10–17]. Existing studies have focused on the group of PWE in remission from seizures for ≥2 years, usually showing seizure relapse in generalised epilepsy. They have tended to be single-centre or small, looking at seizures alone or assessing women alone [11–17]. A valproate decision support tool is available from NHS England for patients and clinicians (www.t.ly/NFi5m), designed by our centre [10]. Figures for the risks of teratogenicity on valproate are provided on page 4 of the tool as 8–37 per 100 babies [10]. However, on the same page, the “risks to you” section from withdrawing from valproate and switching to another ASM remains devoid of any figures [10], illustrating the insufficiency of current evidence to address this question.
The rise of digital care records has transformed treatment evidence generation by enabling both immediate and extensive analysis of national and international cohorts [18]. This approach overcomes limitations in sample size, cost, and time associated with completing prospective studies while also facilitating exploration of previously unexamined patient outcomes. TriNetX [19], the world’s largest real-world electronic health data platform, harbours 70 billion date- and patient-indexed electronic clinical observations from ~250m patients across 19 countries, predominantly in America but also Europe, Middle East, Africa, and Asia [18,19].
We recently published a study of 8,991 men and 5,243 women aged 16–54 with epilepsy taking valproate in TriNetX [20]. This was a survival analysis assessing 5-year mortality and morbidity in those withdrawn from valproate between 01/06/2015–01/06/2018 (regardless of whether switched to another ASM), with propensity-matched comparisons to those who remained on valproate. Results suggested substantially increased risks of emergency department (ED) attendances, hospital admissions, falls, injuries, burns and new-onset depression in both men and women withdrawn from valproate [20]. However, these were aggregate data designed to scope potential trends and generate hypotheses, not to probe causal links.
To progress the field, we propose a comprehensive study examining mortality and morbidity outcomes in individuals who either continued valproate, switched to lamotrigine or levetiracetam, or discontinued valproate. Employing a novel approach known as target trial emulation [21,22], this study will replicate a hypothetical randomised-controlled trial (RCT) using person-level retrospective observational data from both TriNetX and Clinical Practice Research Datalink (CPRD, which encompasses 60m UK patients) [23]. Target trial emulation minimises confounder and selection biases in observational studies by statistically emulating randomisation at baseline (time 0) [21]. Individuals are then analysed based on the treatment strategy aligned with their data. Results are available quickly and shown to mirror those of an RCT [21]. This offers a cost-effective and immediate alternative to conducting an RCT [21]; crucial given the urgency for information on valproate withdrawal safety amidst ongoing regulatory changes [24,25]. Additionally, the emulation facilitates creation of causal prediction models [26]. These are developed using observed data alongside causal theory and assumptions to calculate predicted risk under different hypothetical treatment conditions, allowing “what if” questions to be addressed for each potential (counterfactual) outcome [26]. This facilitates actionable prevention [26].
Aims
- Identify whether there are any differences in mortality or morbidity outcomes between remaining on valproate compared with valproate withdrawal and switch to lamotrigine, levetiracetam, or no new ASM.
- Develop and validate a causal prediction model that clinicians and patients could use when planning to withdraw from valproate to predict which of these alternative ASMs would have the lowest mortality and morbidity, taking baseline clinical and demographic characteristics into account.
Hypotheses
- Some morbidity or mortality outcomes will be increased when valproate is withdrawn, regardless of whether switched to lamotrigine, levetiracetam, or no new ASM;
- The causal prediction model developed will be able to predict which alternative ASMs are safest.
Benefits to PWE
This study will benefit PWE as substantial numbers of young men and women may soon be invited by their clinicians to come off valproate owing to both current and new regulatory changes [5,6]. Whilst this is justified in terms of preventing teratogenic harm, the patients will wish to know what the risks of withdrawing from valproate are for their own health [24,25]. This study will provide immediate evidence to help doctors answer that question [10]. It will also help patients make safer decisions over switching to alternative ASMs by generating the first ever causal prediction tool for this purpose [26]. This will be the largest study of valproate withdrawal to date, the first to assess risks in men and first to assess multiple outcomes beyond seizures alone for both men and women [11–17]. These outcomes will be collaboratively chosen with PWE through public engagement strategies, ensuring their relevance and impact on those most affected. Finally, mortality prediction is highlighted as a key recommendation for epilepsy research by NICE [27], meaning the causal prediction model generated by this study also addresses an established research need for PWE.
Methods
Public involvement
This study protocol (including outcome selection) was co-developed with a members of the public with lived experience of epilepsy, caring for a person with epilepsy, or taking valproate.
Addressing Aim 1: Target trial emulation
Target trial design.
A hypothetical target trial will be emulated in which young men and women taking valproate would be randomised to either continue valproate, switch to lamotrigine or levetiracetam, or discontinue valproate [21]. The following mortality and morbidity outcomes would be assessed over the subsequent five years:
- Deaths (all-cause/epilepsy-related);
- ED/hospital admissions (all-cause/epilepsy-related);
- Seizures (symptom codes);
- Falls;
- Injuries;
- Burns;
- Aspiration pneumonia;
- Manic/bipolar episodes;
- New-onset depression;
- Self-harm/suicide;
- A composite morbidity index incorporating seizures, injuries, admissions, depression.
Intention-to-treat (ITT) and per-protocol effects would be assessed between groups [21]. Analysis would take a time-to-first-event approach (Cox-proportional hazards model) for deaths, new-onset depression and composite morbidity. A recurrent events approach (e.g., Prentice-Williams-Peterson Total Time Model) would be taken for the remaining outcomes.
Target trial emulation using observational data from TriNetX and CPRD.
Eligibility criteria. Anonymised, person-level, electronic health data will be extracted from TriNetX and CPRD, linked across primary care, ED, inpatients, outpatients, and mortality subsets. The study period will be from 01/01/2014 to 01/01/2024. Included will be men and women aged 16–54 years. The study will take an open cohort design with patients entering the cohort when the following criteria are met:
Either: ≥1 epilepsy disease or symptom code recorded during the study period and two separate valproate prescription instances over the preceding 12 months before at least one of the epilepsy disease or symptom codes;
Or: ≥1 epilepsy disease or symptom code recorded any time before the study period with at least two separate valproate prescription instances over 12 months during the study period.
TriNetX and CPRD use ICD-10-CM and SNOMED codes, respectively. The coding strategy combining disease and symptom codes with valproate, as used our published study successfully [20], was shown to accurately identify epilepsy cases in our prior work [28,29].
Treatment arms:
- Withdrawn valproate and commenced lamotrigine
- Withdrawn valproate and commenced levetiracetam
- Withdrawn valproate and no new ASM commenced
- Continues valproate
Treatment assignment. Baseline (time 0) will be defined as when valproate dose reduction commences (treatment arms 1–3) or prescription of valproate at a matched time (arm 4). Eligible individuals are assigned at baseline to the treatment strategy that their data are consistent with [21]. To emulate randomisation, adjustment is made for baseline confounders using Robins’ generalised methods (g-methods, e.g., inverse probability weighting) [21], considering age, sex, ethnicity, deprivation, type of epilepsy (using G40 codes), baseline seizure frequency (symptom codes recorded over preceding 12 months), alcohol excess, developmental delay, Cambridge Multimorbidity Score, length of valproate treatment, valproate dose, and ASMs used other than valproate. The final set of covariates required to adjust for confounding will be chosen using a causal directed acyclic graph [21].
Outcomes. These are listed in the target trial design section and derived from our prior work [1,20,30,31], and public consultation. All coded events will be recorded for flexible categorical or continuous analyses, as needed.
Causal estimand: Assesses impact of treatment arms 1–3 vs. 4 on outcome (per-protocol assumption as observational data limits ITT assessment) [21].
Start and end of follow-up. The start of follow-up (Time 0 or index date) will be defined as when valproate treatment discontinued (treatment arms 1–3) or prescription of valproate at a matched time (arm 4). For time-to-first-event analyses, the end of follow-up will be defined as when the outcome occurs, censoring (including deviation from assigned treatment or leaving the study catchment area), or 10 years. For recurrent events analyses, alongside censoring as above, the end of follow-up will be 10 years, allowing for all events to be assessed throughout follow-up.
Statistical analysis: mirrors target trial section.
Addressing Aim 2: Causal prediction modelling
A causal prediction model will be created using Cox-proportional hazards of time-to-first-event in a causal framework [26]. The predicted outcome will be a composite personal harms risk combining epilepsy-related deaths, seizures, injuries, admissions, and depression. Predictors for definite inclusion in the final model will be treatment arms 1–4. Candidate predictors considered in addition will be taken from the list of baseline confounders adjusted in the target trial emulation, alongside any additional variables considered to influence outcome but not treatment assignment. A priori clinical expertise and data-driven variable selection will refine the predictor variable list for modelling [30]. Penalisation methods (e.g., lasso), will be used to combine variable selection with shrinkage (reducing overfitting risks).
The model will be developed in TriNetX to help ensure it is internationally generalisable [32]. External validation will be undertaken in CPRD, facilitating subsequent clinical implementation [32]. Internal validation within TriNetX will involve 1,000 bootstrap analysis samples. Nagelkerke’s R2 and Brier Scores will estimate overall model performance. Calibration will be estimated with calibration-in-the-large, calibration slope, and flexible calibration plots, and discrimination with C-index. Calibration and discrimination will be reassessed within CRPD to externally validate model performance [32]. Counterfactual prediction will be embedded into the model development [26], allowing patients and clinicians to answer clinically meaningful questions such as “at the point of proposed valproate withdrawal, accounting for baseline characteristics, what is the risk of epilepsy-related death or composite morbidity for levetiracetam? And what is it for lamotrigine?”
Contingency planning
Studying both TriNetX and CPRD allows external cross-validation of findings between the two databases, enhancing generalisability whilst providing a contingency study cohort should one dataset encounter limitations.
Pathways to impact
The study will be published open-access in peer-reviewed journals and shared with regulators, charities, and PWE. Following our guide [34], we will explore various presentation formats for the externally validated prediction model including points-based systems, graphical score charts, and nomograms, as informed by PWE at public engagement workshops and clinicians at scientific conferences. An interactive version of the final causal prediction model will be deposited online in a public repository (as was did for the mortality prediction tool we developed previously: https://seds-tool.github.io/seds), making it globally accessible. We did similar with the CHA₂DS₂-VASc score we created and deposited on MDCalc.com (www.t.ly/poRmb): now the most widely incorporated prediction tool for stroke prevention in atrial fibrillation globally, embedded in NICE guidelines (www.t.ly/d0d3W). We will also add the model to the Civic Data Cooperative (https://civicdatacooperative.com/about) Digital Commons (https://github.com/civicdatacoop) to facilitate integration into health system core infrastructure.
Colleagues at our centre who created NHS England’s valproate decision support tool (www.t.ly/NFi5m) are in support of figures generated in addressing Aim 1 being added to the medication change “risks to you” section on page 4, alongside including a link to the online version of the causal prediction model generated in addressing Aim 2 [10].
Ethics and dissemination
The University of Liverpool Research Ethics Committee has provided formal ethics review exemption for this study as it does not involve human participants, human tissue or personal data (REC Ref. 14455). Dissemination is outlined in the pathways to impact section.
Data management plan
Data will be curated through University of Liverpool’s (UoL) Active Data Storage -a centralised, secure, supported data storage facility with multiple layers of protection. Data are replicated between two secure physical locations and backed up regularly. A regular tape backup is made to a third physical location, and segregated from the public network both physically and logically. Data are encrypted in transit using SSL.
We will use a public repository (www.github.com) to make all diagnostic and outcome coding algorithms, metadata, and R analysis scripts used publicly available, facilitating external replication and adaptation.
All data storage and use will comply with legal obligations (including GDPR) and UoL’s Research Data Management Policy.
Study status and timeline
Funding has been received from the Epilepsy Research Institute and the Academy of Medical Sciences to proceed with this study. Permissions to access study data are being sought and PhD students to help complete the study are being recruited. We plan to complete the study within 36 months of protocol submission/publication
Acknowledgments
We thank our Public Advisors (including Sean Barnes and Steph Tomlinson) for support with this study, and NIHR Applied Research Collaboration North West Coast for providing access to Public Advisors.
References
- 1. Mbizvo GK, Bennett K, Simpson CR, Duncan SE, Chin RFM. Epilepsy-related and other causes of mortality in people with epilepsy: A systematic review of systematic reviews. Epilepsy Res. 2019;157:106192. Epub 2019/09/19. pmid:31526975.
- 2. Marson A, Burnside G, Appleton R, Smith D, Leach JP, Sills G, et al. The SANAD II study of the effectiveness and cost-effectiveness of valproate versus levetiracetam for newly diagnosed generalised and unclassifiable epilepsy: an open-label, non-inferiority, multicentre, phase 4, randomised controlled trial. Lancet. 2021;397(10282):1375–86. Epub 2021/04/12. pmid:33838758; PubMed Central PMCID: PMC8047813.
- 3. Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, et al. The SANAD study of effectiveness of valproate, lamotrigine, or topiramate for generalised and unclassifiable epilepsy: an unblinded randomised controlled trial. Lancet. 2007;369(9566):1016–26. Epub 2007/03/27. pmid:17382828; PubMed Central PMCID: PMC2039891.
- 4. Thomas RH. Valproate: life-saving, life-changing. Clin Med (Lond). 2018;18(Suppl 2):s1–s8. Epub 2018/04/28. pmid:29700085; PubMed Central PMCID: PMC6334031.
- 5. Medicines and Healthcare products Regulatory Agency. Valproate: reminder of current Pregnancy Prevention Programme requirements; information on new safety measures to be introduced in the coming months 2022 17/06/2023. Available from: https://tinyurl.com/z8v8bb8e.
- 6. Medicines and Healthcare products Regulatory Agency. National Patient Safety Alert: Valproate: organisations to prepare for new regulatory measures for oversight of prescribing to new patients and existing female patients (NatPSA/2023/013/MHRA)2023 07/12/2023. Available from: https://tinyurl.com/55j6cu8b.
- 7. Health Sciences Authority. Epilim® (Valproate)—Risk of neurodevelopmental disorders including autism spectrum disorders in children after paternal exposure2023 08/08/2023. Available from: https://tinyurl.com/4xyp2hdd.
- 8. New Zealand Medicines and Medical Devices Authority. Sodium valproate (Epilim) use in people who can father children: important new safety information2023 08/08/2023. Available from: https://www.medsafe.govt.nz/safety/Alerts/Sodium_valproate_Epilim_use_in_people_who_can_father_children.asp.
- 9. Swanborough N. Sanofi reports issues around safety study into valproate2023 08/08/2023. Available from: https://epilepsysociety.org.uk/news/valproate-safety-study-concerns.
- 10. The Walton Centre. Decision support tool: is valproate the right epilepsy treatment for me?2023 07/12/2023. Available from: https://www.england.nhs.uk/wp-content/uploads/2023/05/decision-support-tool-valproate-epilepsy-v1.1.pdf.
- 11. Atalar AC, Sirin NG, Bebek N, Baykan B. Predictors of successful valproate withdrawal in women with epilepsy. Epilepsy Behav. 2021;119:107980. Epub 2021/05/07. pmid:33957390.
- 12. Angus-Leppan H, Moghim MM, Cock H, Kinton L, Synnott Wells M, Shankar R. Valproate risk form-Surveying 215 clinicians involving 4775 encounters. Acta Neurol Scand. 2020;141(6):483–90. Epub 2020/02/20. pmid:32072612.
- 13. Cerulli Irelli E, Morano A, Cocchi E, Casciato S, Fanella M, Albini M, et al. Doing without valproate in women of childbearing potential with idiopathic generalized epilepsy: Implications on seizure outcome. Epilepsia. 2020;61(1):107–14. Epub 2019/12/13. pmid:31828782.
- 14. Quilez A, Baraldes-Rovira M, Gallego Y, Sanahuja J, Mauri-Capdevila G, Purroy F. Risk-benefit assessment of treatment of epileptic women of childbearing age with valproic acid. Seizure. 2020;82:27–30. Epub 2020/09/27. pmid:32979602.
- 15. Tomson T, Battino D, Bonizzoni E, Craig J, Lindhout D, Perucca E, et al. Withdrawal of valproic acid treatment during pregnancy and seizure outcome: Observations from EURAP. Epilepsia. 2016;57(8):e173–7. Epub 2016/06/21. pmid:27319360.
- 16. Vajda FJE, O’Brien TJ, Graham JE, Hitchcock AA, Lander CM, Eadie MJ. Valproate-associated foetal malformations-Rates of occurrence, risks in attempted avoidance. Acta Neurol Scand. 2019;139(1):42–8. Epub 2018/08/16. pmid:30109700.
- 17. Randomised study of antiepileptic drug withdrawal in patients in remission. Medical Research Council Antiepileptic Drug Withdrawal Study Group. Lancet. 1991;337(8751):1175–80. Epub 1991/05/18.
- 18. Taquet M, Sillett R, Zhu L, Mendel J, Camplisson I, Dercon Q, et al. Neurological and psychiatric risk trajectories after SARS-CoV-2 infection: an analysis of 2-year retrospective cohort studies including 1 284 437 patients. Lancet Psychiatry. 2022;9(10):815–27. Epub 2022/08/21. pmid:35987197; PubMed Central PMCID: PMC9385200.
- 19. TriNetX. Welcome to the world’s largest, living ecosystem of real-world data and evidence for the life sciences and healthcare industries. Global data, for global health 2023 25/08/2022. Available from: www.trinetx.com.
- 20. Mbizvo GK, Bucci T, Lip GYH, Marson AG. Morbidity and mortality risks associated with valproate withdrawal in young men and women with epilepsy. Brain. 2024. Epub 2024/04/24. pmid:38657115.
- 21. Hernan MA, Wang W, Leaf DE. Target Trial Emulation: A Framework for Causal Inference From Observational Data. JAMA. 2022;328(24):2446–7. Epub 2022/12/13. pmid:36508210.
- 22. Fu EL. Target Trial Emulation to Improve Causal Inference from Observational Data: What, Why, and How? J Am Soc Nephrol. 2023;34(8):1305–14. Epub 2023/05/03. pmid:37131279; PubMed Central PMCID: PMC10400102.
- 23. CPRD. Clinical Practice Research Datalink. National Institute for Health and Care Research [Internet]. 2023 14/08/2023. Available from: https://cprd.com.
- 24. Watkins LV, Cock HR, Angus-Leppan H, Shankar R. Valproate and the Pregnancy Prevention Programme: exceptional circumstances. Br J Gen Pract. 2019;69(681):166–7. Epub 2019/03/30. pmid:30923139; PubMed Central PMCID: PMC6428457.
- 25. Mole TB, Appleton R, Marson A. Withholding the choice of sodium valproate to young women with generalised epilepsy: Are we causing more harm than good? Seizure. 2015;24:127–30. Epub 2014/09/17. pmid:25223837.
- 26. Lin L, Sperrin M, Jenkins DA, Martin GP, Peek N. A scoping review of causal methods enabling predictions under hypothetical interventions. Diagn Progn Res. 2021;5(1):3. Epub 2021/02/05. pmid:33536082; PubMed Central PMCID: PMC7860039.
- 27. National Institute of Health and Care Excellence (NICE). Epilepsies in children, young people and adults: recommendations for research. NICE guideline [NG217] [Internet]. 2022 16/10/2023. Available from: https://tinyurl.com/mr44t5jf.
- 28. Mbizvo GK, Bennett KH, Schnier C, Simpson CR, Duncan SE, Chin RFM. The accuracy of using administrative healthcare data to identify epilepsy cases: A systematic review of validation studies. Epilepsia. 2020;61(7):1319–35. Epub 2020/06/01. pmid:32474909.
- 29. Mbizvo GK, Schnier C, Simpson CR, Duncan SE, Chin RFM. Validating the accuracy of administrative healthcare data identifying epilepsy in deceased adults: A Scottish data linkage study. Epilepsy Res. 2020;167:106462. Epub 2020/09/23. pmid:32961444.
- 30. Mbizvo GK, Schnier C, Simpson CR, Duncan SE, Chin RFM. Case-control study developing Scottish Epilepsy Deaths Study score to predict epilepsy-related death. Brain. 2022:awac463. Epub 2022/12/09. pmid:36477471.
- 31. Mbizvo GK, Schnier C, Simpson CR, Chin RFM, Duncan SE. A national study of epilepsy-related deaths in Scotland: Trends, mechanisms, and avoidable deaths. Epilepsia. 2021;62(11):2667–84. Epub 2021/09/20. pmid:34537957.
- 32. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. Epub 2013/02/09. pmid:23393430; PubMed Central PMCID: PMC3564751.
- 33. Riley RD, Ensor J, Snell KIE, Harrell FE Jr, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441. Epub 2020/03/20. pmid:32188600.
- 34. Bonnett LJ, Snell KIE, Collins GS, Riley RD. Guide to presenting clinical prediction models for use in clinical settings. BMJ. 2019;365:l737. Epub 2019/04/19. pmid:30995987.