http://orcid.org/0000-0002-2034-8376
Figures
Abstract
Objectives
We wished to validate our recently devised 16-item ENLIST ENL Severity Scale, a clinical tool for measuring the severity of the serious leprosy associated complication of erythema nodosum leprosum (ENL). We also wished to assess the responsiveness of the ENLIST ENL Severity Scale in detecting clinical change in patients with ENL.
Methods
Participants, recruited from seven centres in six leprosy endemic countries, were assessed using the ENLIST ENL Severity Scale by two researchers, one of whom categorised the severity of ENL. At a subsequent visit a further assessment using the scale was made and both participant and physician rated the change in ENL using the subjective categories of “Much better”, “somewhat better”, “somewhat worse” and “much worse” compared with “No change” or “about the same”.
Results
447 participants were assessed with the ENLIST ENL Severity Scale. The Cronbach alpha of the scale and each item was calculated to determine the internal consistency of the scale. The ENLIST ENL Severity Scale had good internal consistency and this improved following removal of six items to give a Cronbach’s alpha of 0.77. The cut off between mild ENL and more severe disease was 9 determined using ROC curves. The minimal important difference of the scale was determined to be 5 using both participant and physician ratings of change.
Conclusions
The 10-item ENLIST ENL Severity Scale is the first valid, reliable and responsive measure of ENL severity and improves our ability to assess and compare patients and their treatments in this severe and difficult to manage complication of leprosy.
The ENLIST ENL Severity Scale will assist physicians in the monitoring and treatment of patients with ENL. The ENLIST ENL Severity Scale is easy to apply and will be useful as an outcome measure in treatment studies and enable the standardisation of other clinical and laboratory ENL research.
Author summary
Erythema nodosum leprosum (ENL) is a severe, painful complication of leprosy, which can occur before, during or after successful treatment of the infection. ENL is characterised by severe pain and the development of new painful skin lesions. Other organ systems are often affected. ENL may continue to affect people for many years leading to disability, significant loss of income and sometimes death. ENL often requires prolonged treatment with corticosteroids, thalidomide or other drugs that modulate the immune system. Thalidomide is the most effective treatment but is not widely available or affordable in many leprosy endemic settings. This results in many people taking high doses of corticosteroids for prolonged periods and being at risk of severe adverse effects. In order to assess the efficacy of different treatments for ENL it is important to be able to compare individuals before and after treatment. This is difficult to do in a complex, multi-system disorder such as ENL. To overcome this problem we have developed a scale, which is easy to use, to measure the severity of ENL. In this article, we describe the validation of this scale, which ensures that it is a useful measure of ENL severity.
Citation: Walker SL, Sales AM, Butlin CR, Shah M, Maghanoy A, Lambert SM, et al. (2017) A leprosy clinical severity scale for erythema nodosum leprosum: An international, multicentre validation study of the ENLIST ENL Severity Scale. PLoS Negl Trop Dis 11(7): e0005716. https://doi.org/10.1371/journal.pntd.0005716
Editor: Stephen Baker, Oxford University Clinical Research Unit, VIET NAM
Received: March 8, 2017; Accepted: June 16, 2017; Published: July 3, 2017
Copyright: © 2017 Walker et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The study was funded by the Austrian Leprosy Relief Association / AHWÖ Aussätzigen-Hilfswerk Österreich (http://www.aussaetzigen-hilfswerk.at/) and Leprosy Research Initiative (https://www.leprosyresearch.org/), grant number: 703.15.15. SLW was supported in part by a grant (ITCRBH21) from the Hospital and Homes of St. Giles. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Erythema nodosum leprosum (ENL) is a severe inflammatory complication of borderline lepromatous (BL) leprosy and lepromatous leprosy (LL). ENL affects up to 50% of individuals with LL and 5–10% of BL leprosy patients [1, 2]. A bacterial index of four or more is also a risk factor for developing ENL. ENL may occur before, during or after successful completion of anti-mycobacterial multi-drug therapy (MDT)[2].
ENL causes inflammation in many systems and is characterised by severe pain, tender cutaneous skin lesions, fever, joint and bone pain, iritis, orchitis, lymphadenopathy and neuritis [3]. Most patients have multiple episodes of painful inflammation extending over several years [2, 3].
ENL is associated with a deleterious impact on health related quality of life (HRQoL)[4], increased mortality[5] and severe economic hardship for affected individuals and their families[6].
The Erythema Nodosum Leprosum International STudy (ENLIST) Group[7] aims to improve the understanding of the mechanisms which cause ENL, improve the evidence to guide treatment decisions of individuals with ENL and improve access to effective treatments. The ENLIST Group includes clinicians and laboratory scientists with extensive experience in the treatment and investigation of the causes of ENL based at institutions in eight countries.
The cause of ENL is unclear. It is associated with a complex array of immune activation and consequent inflammation, which requires immunosuppression. ENL skin lesions may show features of vasculitis and there is evidence of neutrophil and lymphocyte activation. The role of immune complexes in ENL remains unproven. Patients are treated with corticosteroids, clofazimine and thalidomide either alone or in combination, and less commonly other immunomodulatory agents, which are used for prolonged periods of many months or years [3]. Many patients require high doses of corticosteroids to control their disease and this leads to complications and deaths associated with long-term use of these drugs[8]. Thalidomide is usually effective but is not available in many countries or it is severely restricted because of the risk of teratogenicity. Other adverse effects occur with thalidomide and these have been reported to occur in 25–68% of individuals [9–12]. The neuropathy caused by thalidomide during its use to treat other conditions is approximately 20% but there are no good data for the frequency of thalidomide-induced neuropathy in patients with ENL[13]. The identification of other agents for controlling ENL has been identified as a priority for patients in countries where thalidomide is prohibited or highly restricted, unaffordable, ineffective or poorly tolerated [14].
The evidence for choosing the appropriate treatment for ENL is limited. There have been eight small, randomised treatment studies of ENL since the introduction of MDT [9–12, 15–17]. Only 269 patients were enrolled into these studies and just three studies with a total of 53 participants reported allocation concealment and blinding [15, 16]. Determining outcome measures for clinical studies in complex, multisystem disorders such as ENL is difficult. Quantitative severity scoring systems provide one possible outcome measure.There have been several scoring systems employed in studies of ENL however none have been validated [9, 18–21]. Unpublished (non-validated) scales have been shown to be a source of bias in randomised controlled trials [22, 23]. A Cochrane review highlighted the difficulty in comparing treatment studies in ENL and recommended the development of validated severity scales[14].
We developed a 16 item scale, the ENLIST ENL Severity Scale (EESS), for measuring the severity of ENL[24]. We applied and critically appraised three previously published scales for ENL that had not been validated and used regression analysis of data from our cross-sectional study of the clinical features of ENL[3] to enable us to develop the EESS.
The scale incorporates assessments of pain and wellbeing using visual analogue scales (VAS), fever, skin signs, oedema, orchitis, ocular inflammation, joint and bone involvement, nerve assessments and urinalysis. We wished to validate the EESS and determine the minimal important difference (MID). MID is a concept used to determine whether an outcome is clinically relevant and relates to the smallest difference in score which is perceived as beneficial[25].
Methods
Participants who gave written, informed consent were recruited from seven centres: DBLM Hospital, Nilphamari, The Leprosy Mission Bangladesh; Oswaldo Cruz Institute, Rio de Janeiro, Brazil; the ALERT Center, Addis Ababa, Ethiopia; Bombay Leprosy Project, Mumbai, India; The Leprosy Mission Hospital, Purulia, India; Anandaban Hospital, The Leprosy Mission Nepal and the Cebu Skin Clinic, the Leonard Wood Memorial Center, Cebu, Philippines.
Ethical approval was obtained from the Ethics Committee of the London School of Hygiene and Tropical Medicine (Reference 10370), The Leprosy Mission International Bangladesh Institutional Research Board, Brazilian National Ethical Review Board (CAAE: 12834613.4.0000.5248), AHRI-ALERT Ethical Review Committee (PO12/16), Ethics Committee of the Managing Committee of the Bombay Leprosy Project (BLP/OO/TRC/76A/2015), The Leprosy Mission Trust India Ethics Committee, the Nepal Health and Research Council (106/2011), Institutional Ethical Committee/Institutional Ethical Regulatory Board of Cebu Leprosy and Tuberculosis Research Foundation, Inc. (CLTRFI/LWM-IEC 2015–005).
Individuals were eligible to participate in the study if they met any of the following criteria:
- Diagnosed with BL leprosy or LL within 24 months of enrolment (and who did not have ENL or a history of ENL)
- History of ENL with no evidence of active ENL and not requiring treatment for ENL
- Diagnosed with or receiving treatment for ENL
For the purposes of the study ENL was defined as a patient with leprosy who had crops of tender cutaneous or subcutaneous lesions or was on treatment for ENL (whether it was active or not).
The type of ENL was categorised as acute, recurrent or chronic which were defined as follows:
- acute for a single episode lasting less than 24 weeks,
- recurrent if a patient experienced a second or subsequent episode of ENL occurring 28 days or more after stopping treatment for ENL
- chronic if occurring for 24 weeks or more during which a patient has required ENL treatment either continuously or where any treatment free period had been 27 days or less[5].
Individuals who did not wish to give consent or were diagnosed with leprosy Type 1 reactions were excluded.
Each participant was examined independently by a health worker (usually a doctor but sometimes a physiotherapist and in one centre an experienced leprosy research scientist) who had been trained to use the EESS and by an experienced leprologist who also applied the scale and categorised the ENL as “inactive” or “mild” or “moderate” or “severe”. We did not attempt to standardise the categorisation of ENL by the experienced leprologists. Neither assessor (nor the participant) was aware of the result of the other assessor’s examination. The time interval between the two assessments was kept as short as practicable.
At a subsequent visit, the MID of the EESS was determined by applying the scale to individuals after treatment and asking both the participant and the examining leprologist to independently categorise the change as: “much better”, “somewhat better”, “no change” (or “about the same” for physicians), “somewhat worse” or “much worse”. The leprologist had performed one of the original assessments at the first visit but did not apply the EESS on the second occasion and was blinded to the result (as was the participant). The EESS, on this occasion, was applied by the same health worker as at the initial visit whenever possible. The MID methodology was only used for participants who had been categorised as having mild or moderate or severe ENL at the first set of assessments.
All data including demographic, clinical and EESS were collected on data collection forms specifically designed for the study. The anonymised data were entered into a password protected Access database at each centre and subsequently merged. The data were analysed using Stata 14 (StataCorp. 2015 Stata Statistical Software: Release 14. College Station, TX: StataCorp LP).
Statistical methods
We wished to recruit 300 participants as this would provide more than 10 study subjects per scale item [26].
The internal consistency or reliability was assessed using Cronbach’s alpha. An alpha between 0.7 and 0.9 is considered acceptable[27]. The contribution of each item in the scale was assessed by calculating Cronbach’s alpha for the scale if that item were removed.
The ability of the scale to discriminate between patients with different clinical severity categories was determined using analysis of variance. The threshold for accepting statistical significance was p<0.05.
Inter-observer reliability was evaluated using Intra-Class Correlation of the total score of each examiner using a two-way analysis of variation (5% level of significance) and the strength of agreement criteria of Landis and Koch[28]. A Bland Altman plot of the difference between pairs of observations and the mean of those pairs was used to highlight any potential systematic differences between assessors
Receiver operator characteristic curves were used to determine cut off points for mild, moderate and severe reactions.
The ability of the scale to reflect the change in ENL was calculated as the mean of the change in severity associated with each of the reported outcomes “Much better”, “somewhat better”, “somewhat worse” and “much worse” compared with “No change” or “about the same” (for physician rated change).
Results
447 individuals were recruited between 13th May 2015 and 16th July 2016. 336 (75.3%) were male and 110 (24.7%) female. 19 physicians classified the severity of the 210 participants with active ENL. The demographic and clinical features are summarised in Table 1.
54.3% of the 210 individuals with ENL were receiving treatment for their ENL at the time of enrolment. Of the nine drug regimes used, prednisolone (57.2%), prednisolone and clofazimine (24.6%), thalidomide (6.5%), thalidomide and prednisolone (4.3%) and clofazimine alone (2.9%) were the most common.
Scale testing
Initially only 14 of the 16 items were considered for inclusion in the severity scale. The VAS Wellbeing item was excluded because we wished to maintain a strictly clinical focus. Orchitis was also omitted as we wished to try and produce a gender-neutral scale. The items showing the lowest levels of correlation were inflammation of the eyes due to ENL, urinalysis and the items related to sensory and motor nerve function.
The internal consistency of the 14 potential items for inclusion in the scale was assessed using Cronbach's alpha producing an initial value of 0.7413. The series of analyses reported were based on the data from those individuals who had been classified as having mild or moderate or severe ENL (n = 210). Removing eye inflammation due to ENL and urinalysis increased the value of alpha to 0.7633. Removing the count of nerves with sensory NFI due to ENL and the count of nerves with motor NFI due to ENL further increased alpha to 0.7672. Removal of any further items brought a reduction in alpha, confirming the inclusion of the remaining 10 items in the scale.
The derived 10 item scale was analysed separately for men to see if the addition of orchitis significantly altered the internal consistency. Using the data for men alone the 10 item scale has a Cronbach alpha of 0.7633 which increases to 0.7645 with the addition of orchitis. The increase in alpha did not result in greater internal consistency for men compared to men and women combined.
Unidimensionality and discrimination of the 10 item scale
Principal component analysis showed a general factor to which all 10 items contributed accounting for 33.4% of the total variance. A second “pain” factor contributed 16.0% of the total variance. It contrasted VAS pain, bone pain, inflammation of joints and nerve tenderness with items describing the number, extent and inflammation of skin lesions and lymphadenopathy.
The 10 item scale discriminated well between patients with active ENL and those without. Fig 1 shows the distribution
The difference in scores between the non-ENL group and those categorised as having mild ENL were significant (p<0.001, parametric and non-parametric test).
A threshold value differentiating between those who were classified as having “moderate” ENL and those with “severe” ENL was not identified by either score or ROC curve. However, the difference in mean severity scores was statistically significant (p<0.001, parametric and non-parametric test).
The EESS scores of participants with acute or recurrent or chronic ENL were not significantly different.
The intra-class correlation coefficient assuming random effect for both patients and assessors and individual assessors is 0.797 (95% CI 0.742, 0.843). The strength of agreement is good[28].
A Bland-Altman plot (Fig 2) showed good agreement between the two assessors of each patient with no evidence of a systematic difference in terms of larger differences for higher severity scores. 15 (7.4%) of the 204 paired assessments fell outside the confidence limits but these were evenly distributed between positive and negative differences.
To determine the cut off score between “mild” and more severe categories of ENL the ROC curve was plotted for patients classified as having “mild” ENL and those with either moderate or severe ENL (Fig 3).
Mild ENL was determined to be an EESS score of 8 or less and more severe forms scoring 9 or more (Fig 4). The area under the curve for mild and combined moderate and severe ENL is 0.8372. This value indicates that the final scale is a good discriminator between the mild and more severe categories of ENL traditionally used by clinicians.
The line in red indicates the cut off between mild ENL and more severe disease.
152 participants with ENL completed the two sets of assessments. The median interval between these two sets of assessments was 28 days, range 0 to 185.The changes in EESS scores and the participant rated and physician rated improvement are shown in Figs 5 and 6 respectively.
Table 2 shows the mean change in scores for each participant-rated or physician-rated change and the difference from “no change” or “about the same” respectively. The MID was 4.9 for both participants and physicians which equates to a change in EESS score of 5. The change in mean EESS score from baseline to “much better” which resulted in an “important difference” was 8.4 for participants and 7.0 for physicians. This equates to a change in EESS score of 9.
Discussion
Leprosy reactions present a major challenge to the successful management of the disease. The adapted version of the EESS is a valid and reliable measure of the severity of ENL. We have been able to show that it discriminates between patients with mild ENL and more severe disease. The scale and accompanying guide (S1 Appendix) are easy for clinicians to use.
A clinical tool to measure the severity of leprosy Type 1 reactions was designed and validated by some members of the ENLIST Group [29]. Type 1 reactions are a major cause of nerve damage in leprosy. The Type 1 reaction severity scale was first validated in Brazil and Bangladesh and in a subsequent study from Ethiopia[30]. It has been shown to reflect change in severity following treatment and has been used in clinical trials of corticosteroids, azathioprine and ciclosporin [31–34]. We believe that the EESS has the potential to be equally important in ENL.
The EESS is gender-neutral following removal of the orchitis item which did not materially affect the internal consistency of the scale. The two items relating to NFI were removed from the final version of the EESS but clinicians should remain vigilant to new NFI occurring in the context of ENL, which we have previously shown to have a prevalence of 22.9% in individuals with ENL[3]. Nerve tenderness is a component of the scale but nerve tenderness does not always accompany NFI. Eye involvement due to ENL is not included in the scale and clinicians will need to be cognizant of this uncommon feature when using the EESS.
We were able to demonstrate that the EESS discriminates between moderate and severe ENL but could not determine a cut off score for the two categories. We did not attempt to standardise physician assessment of the severity of ENL which is a limitation of the study. The overlap of scores in the group categorised as having “moderate” ENL and those with “severe” ENL is likely to have occurred due to variation in physician perception of the construct of ENL severity. The multisystem nature of ENL may mean that different physicians attach different weight to different symptoms or signs when categorising ENL severity. Variation between physician global assessments has also been reported to occur during comparison of different methods for the assessment of flares in systemic lupus erythematosus [35].
The MID of the EESS was determined using both participant and physician reported change. There are no agreed criteria for determining which group should be used for determining MID. It has been argued that physicians are the best judges of change in measures of disease activity or damage, whereas in functional or HRQoL measures it is the affected individual [36]. We felt it was important to assess the responsiveness of the EESS using both groups. The results were concordant with an MID of 5 for both groups. The greatest discordance, of two scale units, between the ratings of participants and physicians occurred for the change of “much better”.
The EESS is the first validated, published scale of ENL severity and is responsive to change in ENL. We plan to use the EESS in future double-blind randomised controlled treatment studies of ENL and believe it will be an important tool for other clinical researchers. The scale will also be useful in providing a standardised way of describing the severity of ENL in patients who participate in immunological and genetic studies. It is equally important that the EESS be incorporated into routine clinical practice where we believe it will help physicians to assess, monitor and treat patients.
Supporting information
S1 Appendix. ENLIST ENL Severity Scale.
Validated ENLIST ENL Severity Scale and User Guide.
https://doi.org/10.1371/journal.pntd.0005716.s001
(PDF)
Acknowledgments
We are grateful to the participants who took part in the study and the organisations where the work was conducted. The membership of the ENLIST Group also includes: Nanda Ajayan, Medhi Alinda, Sylvia Anggraeni, Annamma John, M. Yulianto Listiawan, Milton Moraes, Lilian Pinheiro Rodrigues Nogueira and Euzenir Sarno.
In Bangladesh we would like to acknowledge Shamsun Naher, Virginia Sukla Biswas, Kiron S. Chokroborty, Kalpona Roy and Rehana Begum. In Nepal, we would like to acknowledge the contribution of the Mycobacterial Research Laboratories staff: Pawan Parajuli, Ruby Thapa, Divya RSJB Rana, Preeti Maharjan and Jeni Maharjan. In Ethiopia, we would like to acknowledge the assistance provided by Yilma Tesfay.
References
- 1. Kumar B, Dogra S, Kaur I. Epidemiological characteristics of leprosy reactions: 15 years experience from north India. Int J Lepr Other Mycobact Dis. 2004;72(2):125–33. pmid:15301592.
- 2. Pocaterra L, Jain S, Reddy R, Muzaffarullah S, Torres O, Suneetha S, et al. Clinical course of erythema nodosum leprosum: an 11-year cohort study in Hyderabad, India. Am J Trop Med Hyg. 2006;74(5):868–79. pmid:16687695.
- 3. Walker SL, Balagon M, Darlong J, Doni SN, Hagge DA, Halwai V, et al. ENLIST 1: An International Multi-centre Cross-sectional Study of the Clinical Features of Erythema Nodosum Leprosum. PLoS Negl Trop Dis. 2015;9(9):e0004065. pmid:26351858.
- 4. Yap FB, Kiung ST, Yap JB. Quality of life in patients with erythema nodosum leprosum in Kuala Lumpur, Malaysia. Indian Dermatol Online J. 2016;7(4):255–8. pmid:27559497;
- 5. Walker SL, Lebas E, Doni SN, Lockwood DN, Lambert SM. The mortality associated with erythema nodosum leprosum in Ethiopia: a retrospective hospital-based study. PLoS Negl Trop Dis. 2014;8(3):e2690. Epub 2014/03/15. pmid:24625394;
- 6. Chandler DJ, Hansen KS, Mahato B, Darlong J, John A, Lockwood DN. Household costs of leprosy reactions (ENL) in rural India. PLoS Negl Trop Dis. 2015;9(1):e0003431. pmid:25590638;
- 7. Walker SL, Saunderson P, Kahawita IP, Lockwood DN. International workshop on erythema nodosum leprosum (ENL)—consensus report; the formation of ENLIST, the ENL international study group. Lepr Rev. 2012;83(4):396–407. Epub 2013/04/26. pmid:23614260.
- 8. Walker SL, Waters MF, Lockwood DN. The role of thalidomide in the management of erythema nodosum leprosum. Lepr Rev. 2007;78(3):197–215. pmid:18035771.
- 9. Kaur I, Dogra S, Narang T, De D. Comparative efficacy of thalidomide and prednisolone in the treatment of moderate to severe erythema nodosum leprosum: a randomized study. Australas J Dermatol. 2009;50(3):181–5. Epub 2009/08/08. pmid:19659979.
- 10. Kar HK, Gupta L. Comparative efficacy of four treatment regimens in Type 2 leprosy reactions (prednisolone alone, thalidomide alone. prednisolone plus thalidomide and prednisolone plus clofazimine). Indian Journal of Leprosy. 2016;88:29–38.
- 11. Sales AM, Matos HJ, Nery JA, Duppre NC, Sampaio EP, Sarno EN. Double-blind trial of the efficacy of pentoxifylline vs thalidomide for the treatment of type II reaction in leprosy. Braz J Med Biol Res. 2007;40(2):243–8. pmid:17273661.
- 12. Villahermosa LG, Fajardo TT Jr., Abalos RM, Balagon MV, Tan EV, Cellona RV, et al. A randomized, double-blind, double-dummy, controlled dose comparison of thalidomide for treatment of erythema nodosum leprosum. Am J Trop Med Hyg. 2005;72(5):518–26. pmid:15891124.
- 13. Bastuji-Garin S, Ochonisky S, Bouche P, Gherardi RK, Duguet C, Djerradine Z, et al. Incidence and risk factors for thalidomide neuropathy: a prospective study of 135 dermatologic patients. J Invest Dermatol. 2002;119(5):1020–6. pmid:12445187.
- 14. Van Veen NH, Lockwood DN, van Brakel WH, Ramirez J Jr., Richardus JH. Interventions for erythema nodosum leprosum. Cochrane Database Syst Rev. 2009;(3):CD006949. Epub 2009/07/10. pmid:19588412.
- 15. Roy K, Sil A, Das NK, Bandyopadhyay D. Effectiveness and safety of clofazimine and pentoxifylline in type 2 lepra reaction: a double-blind, randomized, controlled study. Int J Dermatol. 2015. pmid:26094723.
- 16. Lambert SM, Nigusse SD, Alembo DT, Walker SL, Nicholls PG, Idriss MH, et al. Comparison of Efficacy and Safety of Ciclosporin to Prednisolone in the Treatment of Erythema Nodosum Leprosum: Two Randomised, Double Blind, Controlled Pilot Studies in Ethiopia. PLoS Negl Trop Dis. 2016;10(2):e0004149. pmid:26919207;
- 17. Girdhar A, Chakma JK, Girdhar BK. Pulsed corticosteroid therapy in patients with chronic recurrent ENL: a pilot study. Indian J Lepr. 2002;74(3):233–6. pmid:12708702.
- 18. Karat AB, Thomas G, Rao PS. Indomethacin in the management of erythema nodosum leprosum—a double-blind controlled trial. Lepr Rev. 1969;40(3):153–8. pmid:4900710.
- 19. Waters MF. An internally-controlled double blind trial of thalidomide in severe erythema nodosum leprosum. Lepr Rev. 1971;42(1):26–42. pmid:4338720.
- 20. Haslett PA, Roche P, Butlin CR, Macdonald M, Shrestha N, Manandhar R, et al. Effective treatment of erythema nodosum leprosum with thalidomide is associated with immune stimulation. J Infect Dis. 2005;192(12):2045–53. pmid:16288366.
- 21. Feuth M, Brandsma JW, Faber WR, Bhattarai B, Feuth T, Anderson AM. Erythema nodosum leprosum in Nepal: a retrospective study of clinical features and response to treatment with prednisolone or thalidomide. Lepr Rev. 2008;79(3):254–69. pmid:19009975.
- 22. Marshall M, Lockwood A, Bradley C, Adams C, Joy C, Fenton M. Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000;176:249–52. pmid:10755072.
- 23. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. pmid:20169472;
- 24. Walker SL, Knight KL, Pai VV, Nicholls PG, Alinda M, Butlin CR, et al. The development of a severity scale for Erythema Nodosum Leprosum—the ENLIST ENL severity scale. Leposy Review. 2016;87:332–46.
- 25. King MT. A point of minimal important difference (MID): a critique of terminology and methods. Expert review of pharmacoeconomics & outcomes research. 2011;11(2):171–84. pmid:21476819.
- 26. Cabrera-Nguyen P. Author Guidelines for Reporting Scale Development and Validation Results in the Journal of the Society for Social Work and Research. Journal of the Society for Social Work and Research. 2010;1(2):99–103.
- 27.
Streiner DL, Norman GR. Health Measurement Scales:A practical guide to their development and use. 3rd ed. Oxford: Oxford University Press; 2003.
- 28. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. pmid:843571.
- 29. Walker SL, Nicholls PG, Butlin CR, Nery JA, Roy HK, Rangel E, et al. Development and validation of a severity scale for leprosy type 1 reactions. PLoS Negl Trop Dis. 2008;2(12):e351. Epub 2008/12/24. pmid:19104651;
- 30. Lambert SM, Haroun O, Lockwood DNJ. Validation of the Leprosy Type 1 Reaction Severity Scale in Ethiopia. Leprosy Review. 2016;87:113–7. pmid:27255066
- 31. Walker SL, Nicholls PG, Dhakal S, Hawksworth RA, Macdonald M, Mahat K, et al. A comparison of the change in clinical severity scale score and a retrospective physician assessment of neurological outcome in individuals with leprosy associated nerve function impairment after treatment with corticosteroids. Lepr Rev. 2012;83(2):154–63. Epub 2012/09/25. pmid:22997691.
- 32. Walker SL, Nicholls PG, Dhakal S, Hawksworth RA, Macdonald M, Mahat K, et al. A Phase Two Randomised Controlled Double Blind Trial of High Dose Intravenous Methylprednisolone and Oral Prednisolone versus Intravenous Normal Saline and Oral Prednisolone in Individuals with Leprosy Type 1 Reactions and/or Nerve Function Impairment. PLoS Negl Trop Dis. 2011;5(4):e1041. Epub 2011/05/03. pmid:21532737;
- 33. Lambert SM, Alembo DT, Nigusse SD, Yamuah LK, Walker SL, Lockwood DN. A Randomized Controlled Double Blind Trial of Ciclosporin versus Prednisolone in the Management of Leprosy Patients with New Type 1 Reaction, in Ethiopia. PLoS Negl Trop Dis. 2016;10(4):e0004502. pmid:27046330;
- 34. Wagenaar I, Brandsma W, Post E, van Brakel W, Lockwood D, Nicholls P, et al. Two randomized controlled clinical trials to study the effectiveness of prednisolone treatment in preventing and restoring clinical nerve function loss in leprosy: the TENLEP study protocols. BMC Neurol. 2012;12:159. pmid:23249098;
- 35. Isenberg DA, Allen E, Farewell V, D'Cruz D, Alarcon GS, Aranow C, et al. An assessment of disease flare in patients with systemic lupus erythematosus: a comparison of BILAG 2004 and the flare version of SELENA. Ann Rheum Dis. 2011;70(1):54–9. pmid:20833737.
- 36. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Medical care. 2000;38(9 Suppl):II84–90. pmid:10982093.