Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Development and Validation of a New Clinical Scale for Infants with Acute Respiratory Infection: The ReSVinet Scale

  • Antonio José Justicia-Grande ,

    Contributed equally to this work with: Antonio José Justicia-Grande, Federico Martinón-Torres

    Affiliations Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain, Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Jacobo Pardo-Seco,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Miriam Cebey-López,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Lucía Vilanova-Trillo,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Alberto Gómez-Carballa,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Irene Rivero-Calle,

    Affiliations Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain, Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • María Puente-Puig,

    Affiliation Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain

  • Carmen Curros-Novo,

    Affiliation Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain

  • José Gómez-Rial,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Antonio Salas,

    Affiliations Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain, Unidade de Xenética, Departamento de Anatomía Patolóxica e Ciencias Forenses, and Instituto de Ciencias Forenses, Grupo de Medicina Xenómica (GMX), Facultade de Medicina, Universidade de Santiago de Compostela, Galicia, Spain

  • José María Martinón-Sánchez,

    Affiliations Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain, Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Lorenzo Redondo-Collazo,

    Affiliations Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain, Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Carmen Rodríguez-Tenreiro,

    Affiliation Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Federico Martinón-Torres ,

    Contributed equally to this work with: Antonio José Justicia-Grande, Federico Martinón-Torres

    federico.martinon.torres@sergas.es

    Affiliations Translational Pediatrics and Infectious Diseases Section, Pediatrics Department, Hospital Clínico Universitario de Santiago de Compostela, Santiago de Compostela, Spain, Genetics, Vaccines, Infections, and Pediatrics Research Group (GENVIP), Healthcare Research Institute of Santiago de Compostela, Santiago de Compostela, Spain

  • Respiratory Syncytial Virus network (ReSVinet)

    A complete list of members of the ReSVinet network is provided in the Acknowledgments.

Development and Validation of a New Clinical Scale for Infants with Acute Respiratory Infection: The ReSVinet Scale

  • Antonio José Justicia-Grande, 
  • Jacobo Pardo-Seco, 
  • Miriam Cebey-López, 
  • Lucía Vilanova-Trillo, 
  • Alberto Gómez-Carballa, 
  • Irene Rivero-Calle, 
  • María Puente-Puig, 
  • Carmen Curros-Novo, 
  • José Gómez-Rial, 
  • Antonio Salas
PLOS
x

Abstract

Background and Aims

A properly validated scoring system allowing objective categorization of infants with acute respiratory infections (ARIs), avoiding the need for in-person assessment and that could also be used by non-health professionals is currently not available. We aimed to develop a new clinical assessment scale meeting these specifications.

Methods

We designed a clinical scale (ReSVinet scale) based on seven parameters (feeding intolerance, medical intervention, respiratory difficulty, respiratory frequency, apnoea, general condition, fever) that were assigned different values (from 0 to 3) for a total of 20 points.170 children under two years of age with ARI were assessed independently by three pediatricians using this scale. Parents also evaluated their offspring with an adapted version of the scale in a subset of 61 cases. The scale was tested for internal consistency (Cronbach’s alpha), Pearson correlation coefficient for the items in the scale, inter-observer reliability (kappa index) and floor-ceiling effect.

Results

Internal consistency was good for all the observers, with the lowest Cronbach’s alpha being 0.72. There was a strong correlation between the investigators (r-value ranged 0.76–0.83) and also between the results obtained by the parents and the investigators(r = 0.73). Light’s kappa for the observations of the three investigators was 0.74. Weighted kappa in the group evaluated by the parents was 0.73. The final score was correlated with length of hospital stay, PICU admission and Wood-Downes Score.

Conclusions

The ReSVinet scale may be useful and reliable in the evaluation of infants with ARI, particularly acute bronchiolitis, even with data obtained from medical records and when employed by parents. Although further studies are necessary, ReSVinet scale already complies with more score validation criteria than the vast majority of the alternatives currently available and used in the clinical practice.

Background and Aims

Acute Respiratory Infections (ARI), and more specifically bronchiolitis, constitute a major cause of morbidity and mortality in infants throughout the world [1,2,3]. Even though the clinical characteristics, pathophysiology and course of ARI in infancy are well known, there is no consensus with regards to how best to measure the severity of disease and the impact of therapeutic or preventive interventions in clinical trials [4, 5].

Many clinical scores have been published and even validated for the evaluation of infants with ARI and/or bronchiolitis, although none of them has been sufficiently validated to allow meaningful use in children [5]. These scales incorporate a variety of end points, and they all have important limitations. These include the following: they are designed for medical personnel, they rely on in-person assessment of the patient by a physician (neglecting clinical records and parental opinion), many of them are focused only in the respiratory status of the patient, and in most cases they require parameters that are not universally available (such as oxygen saturation or invasive procedures) [6,7].

No score has been validated as a research tool in the setting of pediatric respiratory infections; many of them are focused only in the respiratory status of the patient, and most, if not all, require in-person evaluation, which may be particularly difficult to ensure in outpatient clinical trials [5, 7]. We also currently lack a scale that could be used by parents and physicians alike. Clinicians in emergency departments and primary settings, as well as professionals involved in outpatient clinical trials, could all benefit greatly from such a scoring system. Doctors could have a reliable tool, allowing parents to monitor the evolution of children at home, establishing objective thresholds for patient referral or to seek of medical assistance. In addition, in the setting of clinical trials, having a validated score that parents can use would allow objective follow-up after hospital discharge or in case of mild diseases.

The goal of this project was to develop and evaluate a global clinical severity scale (ReSVinet scale) that would be amenable to be used in outpatient and inpatient studies without the need of in-person assessment. Internal reliability, construct validity, and ease of applicability of the scale were assessed. Additionally, the validity and reproducibility of an adapted version of this score in non-medical language for parental use was evaluated.

Methods

All necessary ethical permissions were obtained from the regional ethics committee in Galicia (Spain) prior to the beginning of this study, which was approved by the aforementioned Institution on 23 March 2010 (registration number 2010/015). Written informed consent was obtained from all participants’ legal guardians before each patient was included in the study.

a- Score design

The main investigators, with substantial research experience in pediatric respiratory infections, systematically reviewed studies that reported on the use of a respiratory score in children with ARI. The following symptoms and clinical findings were initially considered as indicative of disease severity: general condition, respiratory difficulty, oxygen saturation by pulse oxymeter, cyanosis, apnea, food tolerance, tachypnea, and tachycardia. Finally some of these symptoms were either included as part of a single component of the score, or excluded for being considered redundant or for limiting the applicability of the score in non-hospital settings and/or by non-medical personnel.

The proposed score was submitted for evaluation to 90 pediatricians of outpatient, hospital care and pediatric critical setting from the Galician Pediatric Research Network (REGALIP) and from the FIVE research group [8]. They agreed on the ease of recording and value of the parameters included. The final ReSVinet scale is detailed in Table 1, consisting of seven different items with different value ranges assigned, which add up to a total score that spans from 0 to 20.

thumbnail
Table 1. ReSVinet scale.

This table presents the original scale, and was the one used by the three investigators.

https://doi.org/10.1371/journal.pone.0157665.t001

We aimed to validate this scale using information obtained from patients’ medical records. Our secondary aim was to provide the parents with a simple tool for evaluating the severity of their children’s disease. The scale was thus developed to cover an important limitation of the scoring systems currently available: none of them was designed for use by non-medical personnel. All the components as explained in the scale should be easily evaluated by a parent/caregiver with no previous training. This resulted in an adapted version of the ReSVinet scale written in plain Spanish language (Table 2).

b- Study Population

Patients were prospectively recruited among those admitted during two consecutive seasons in three hospitals in Spain. Although the ReSVinet scale was initially conceived as a tool for the entire pediatric age scope, a homogeneous group was defined in order to minimize biases: only previously healthy children under two years of age with the diagnosis upon admission of acute respiratory infection were included in the study. Infants with relevant underlying diseases (including cardiac disease, chronic pulmonary problems, anatomic malformations, neurologic diseases, severe malnutrition, immunodeficiency or hematologic malignancies) were excluded. The parents of those fulfilling the above criteria were asked to participate in the study right after their children had been admitted and, if informed consent was given, infants were allocated to one of the following two cohorts:

  • In group 1 (professional; G1), the clinical records of 170 children were retrospectively evaluated using the new scale by three physicians (Observer 1- O1-, Observer 2 –O2-, Observer 3 –O3-) with expertise in the emergency setting.
  • In group 2 (parental; G2), the clinical status of 61 patients was assessed by their caregivers at the moment of discharge using an adapted version of the scale. Their results were compared to those obtained by a physician blinded to the parental score (O1) who reviewed the medical history of patients from G2 upon discharge. When parents consented to take part in the study, simple directions on how to complete the scale were offered by a study nurse, who outlined the different items reading them to the parents to make sure that they were perfectly understood, and offered them the possibility of asking their doubts about the study. Parents should answer those questions using their own conception about severity in each item. They had no access to medical records. Respiratory rate references were avoided in the parental version as their inclusion could require standardized training. Whenever possible, infants were always fed by parents. Children were clinically assessed on a daily basis by a doctor not involved in the evaluation of the score. The same doctor then offered the legal tutors information on the evolution of the illness, treatment options and test to be performed as a part of routine care. Routine care was the same for all infants, whether they were taking part in the study or not. Physicians in charge of scoring the severity of the illness did not take part in the everyday clinical care of those children, and made no decisions on the tests a patient should undergo or on treatment changes. At the moment of discharge, the caregivers completed their version of the ReSVinet scale as if the child was in his or her worst moment during admission.

In both sets of patients, clinical information and demographic data were registered in standardized forms by either investigators or parents. A descriptive analysis of the recorded variables was performed.

c- Score assessment

Our scale was tested in terms of internal reliability, construct validity, and ease of applicability:

c.1.) Validity.

Construct validity was assessed by examining the correlation between the score and other measures of impact of the disease: modified Wood-Downes Score [WDS] (as the most widely used score in our setting) [9], length of stay, PICU internment, treatments received and duration of admission. Optimal cut-points methods were used to establish a severity threshold for the total score [10]. Pearson’s and Spearman’s Rho correlation coefficients were used to analyze correlation with quantitative variables. In order to compare the distribution of the score among groups of patients, Wilcoxon test and Kruskal-Wallis test were employed.

c.2) Reliability.

Inter-observer reliability was assessed for the total score using a weighed kappa for each paired couple of investigators in G1, and for O1 and parents in the second cohort. For the first subset of patients, we also obtained the Light’s kappa for more than two observers (i.e, the mean of the kappa values obtained comparing each pair of investigators). Intra-class correlation coefficients were also used to assess inter-observer reliability. Internal consistency was evaluated using Cronbach’s alpha for each of the observers (O1, O2, and O3) and for the parental cohort. Scales with Cronbach’s alpha values >0.7 are considered to have good internal consistency [5, 11,12,13].

c.3) Utility.

Individual items were reviewed for missing data and floor and/or ceiling effects. Percentages of missing data of less than 10% and floor/ceiling effects <15% are generally considered acceptable.

All statistical calculations were carried out using the packages stats, psy and multilevel from R Software, version 3.1.1. (https://www.r-project.org). Level of significance was set to P-value < 0.05.

Results

The three investigators completed all of the scales, whereas 60 of the 61 parents who agreed to do so completed all the items of the score (one of the patients was discharged without having his parents filled in their version of the score). Demographics and clinical aspects of “professional” and “parental” cohorts are summarized in Tables 3 and 4, respectively. Median age (interquartilic range) was 6.5 (2.0–9.5) months for the “professional” cohort while it was 2.3 (1.8–6.3) months for the “parental” cohort. Score results for each subject can be seen in Fig 1.

thumbnail
Fig 1.

Comparison of results obtained (A) by the three observers, and (B) by the parents and one of the observers. In graph A: O1- Results obtained by observer 1. O2- Results obtained by observer 2. O3- Results obtained by observer 3. In graph B: O1- Results obtained by observer 1. P—Results obtained by the parents.

https://doi.org/10.1371/journal.pone.0157665.g001

thumbnail
Table 3. Results of the ReSVinet scale in the professional cohort (Group 1).

This table categorizes patients according to clinical, epidemiological and outcome variables in the cohort evaluated by three pediatricians.

https://doi.org/10.1371/journal.pone.0157665.t003

thumbnail
Table 4. Results of the ReSVinet scale in the parental cohort (group 2).

This table reflects the results of the score obtained by a physician (observer 1) and parents and the relation of the values obtained with clinical, epidemiological and outcome variables.

https://doi.org/10.1371/journal.pone.0157665.t004

Cronbach’s alpha for the results obtained by each of the observers and for parental subgroup was greater than 0.70 in all cases (r-value ranged from 0.72 to 0.79) for the professional cohort (G1), 0.73 for the parental cohort (G2).

Weighed kappa values for each paired couple of investigators was 0.77 for O1-O2, 0.71 for O1-O3 and 0.74 for O2-O3 in the first cohort. Light’s kappa for more than two observers was 0.74. The weighed kappa for the group evaluated by parents and O1 was 0.73. There was a strong correlation between the results obtained by investigators (r-value ranged from 0.76 to 0.83) and also between the results obtained by the parents and the investigators (r = 0.73).

O1 allocated 2.4% of the patients evaluated to the extreme endpoints of the scale (Total score 0 or 20). Rates for O2 and O3 were 0.59% and 0.0%, respectively. Only 1.7% of children reviewed by their parents were given an extreme value.

Changes in the score depending on several clinical conditions were sought. Only those patients who were admitted to a Paediatric Intensive Care Unit (p<0.001 for both cohorts) or those receiving Heliox (a fixed mixture of 30% oxygen and 70% helium) (P-value < 0.001 for G1 and P-value = 0.002 for G2) were found to have a statistically significant higher score in both cohorts. A previous episode of wheezing could be related to a more severe disease in the sample assessed by professionals (P-value = 0.012) but not in the parental cohort (P-value = 0.672). RSV was identified as the agent causing the disease in 77.1% (G1) and 81% (G2) of the cases, but could not be definitively linked to a more aggressive course of infection (i.e, achieving higher scores in the scale). No differences were found in the score when comparing younger patients to older ones, regardless of the person carrying out the assessment, but this observation lacked statistical significance (P-value = 0.790 for G1 and P-value = 0.880 for G2). Interventions such as nebulized epinephrine and inhaled or oral corticosteroids could not be related with significant changes in the final score. Children on antibiotics did receive a higher score value in the paediatrician-assessed cohort G1 (mean difference range: 2.2–2.4), and this difference could be attributable to endovenous prescriptions (P-value < 0.001), but not to oral ones (P-value = 0.980). Similarly, scores of children with suspected bacterial super-infection presented also a greater score than those discharged without that suspicion (mean difference range: 1.7–1.9, P-value = 0.001).

Severity was defined by admission to PICU. Those infants who required intensive care presented a significantly higher score than those admitted to wards (15.7 ± 2.6 vs. 10.2 ± 2.5 when evaluated by O1, 15.4 ± 2.7 vs. 9.2 ± 2.4 for O2, 14 ± 2.6 vs. 9.4 ± 2.4 for O3 in the professional cohort, P-value < 0.001 for all of these values). These results could be replicated in the parental subgroup (15.9 ± 2.5 for vs. 10.6 ± 2.7 for those children assessed by their parents, P-value < 0.001). Cut-points analysis shows that a “severe episode” can be defined when a child scores ≥ 14 points in our scale. If severity of the episode was defined instead by obtaining a value in the “severe” range of the WDS (face value), we also found that values ≥ 14 in the ReSVinet scale were associated with values of ≥ 7 in the WDS; this association was found to bear statistical significance when comparing these patients to those who presented a “moderate” severity (defined by obtaining a 4, 5 or 6 in the WDS): in the first cohort results were 15.1 ± 3.1 vs. 11.1 ± 2.7 for O1, 14.7 ± 3.0 vs. 10.2 ± 2.7 for O2, 13.7 ± 2.9 vs. 10.2 ± 2.5 for O3; and parental values (G2) were 15.9 ± 2.5 vs. 11.7 ± 2.0 (P-value < 0.001 for all of these observations). Mean values and standard deviation from O1 for patients in parental cohort (G2) were 16.8 ± 3.1 vs. 11.4 ± 2.6. In both cohorts, higher results meant a higher WDS (P-value < 0.001 for G1 and G2).

Length of stay had a significant positive correlation with the total value of the scale (r: 0.48–0.60 for G1, P-value < 0.001, and r: 0.33–0.35 for G2, P-value = 0.027).

Parents found the scale easy to understand, and they could complete it in less than two minutes. For investigators, all the information required in the scale was accessible through electronic clinical records. Investigators needed about five minutes to review the clinical record of a patient.

Discussion

The ReSVinet scale fulfills most of the requirements of an acceptable scoring system (Table 5) [5, 12,13,14] with the added value of avoiding the need for in-person evaluation and allowing an adapted version that is useful for non-professional assessment of the patient status.

thumbnail
Table 5. Check-list of the characteristics of the ReSVinet scale according to the desired properties of a clinical scale validated for use in infants with acute respiratory infections*.

The “considerations” column explains whether the ReSVinet scale meets the requirement. N.A. = not assessed.

https://doi.org/10.1371/journal.pone.0157665.t005

Our scale showed a good internal consistence and an acceptable inter-observer reliability even when tested by non-medical personnel. Most of the literature concerning scale validation deals with information obtained by only two investigators [5, 15], so having an acceptable weighted kappa index for more than two investigators is a relevant asset [5]. Intra-observer reliability or test-retest procedures can be assumed to be good if the inter-observer reliability is acceptable [12]. Hypotheses chosen as outcomes for testing face validity were correct for the WDS and discriminating the severity of the illness, and length of hospital stay also correlated positively with higher values of the scale in both groups.

So far, none of the many scores used for assessing the degree of dyspnea in patients with acute respiratory infections has been sufficiently validated to allow clinically meaningful use in children [5]. Our scale performed positively in 9 of the 15 items required. To the best of our knowledge, there are only a few scales whose validation criteria have been tested so thoroughly, and even fewer meeting more quality criteria than the ReSVinet scale [5]. The exclusion of different parameters by an expert panel is debatable, as there is ample evidence that some of them (such as peripheral oxygen saturation) are associated with longer admission and invasive measures [7]. The main points of disagreement concerned the inclusion of oxygen saturation and heart frequency, as both of them are frequently used in other scoring systems [16]. In the end, both were intentionally left out of the scale. Oxygen saturation is not always available in an outpatient setting. When used, recent studies show that its value in the decision-making process when facing a child with bronchiolitis may have been overestimated, leading to a higher rate of admissions [17]; physicians tend to abuse this “objective” measurement even when many do not understand its basis, placing great value on a difference of 2%, which is the accuracy most manufacturers report for their machines [18]. Parents also rely greatly on peripheral oxygen saturation when their child is admitted [19], and they could be prone to underestimate the true severity of the disease if this particular parameter is good enough for them. This variable was definitively discarded as its inclusion in the ReSVinet scale would hamper its usability by non-medical personnel–given the need of a device and training in probe placement and data interpretation–as well as its applicability in an outpatient setting. The need of oxygen supplementation (the intervention normally undertaken in patients with low oxygen saturation) is in any case covered in Q2. The “heart rate” parameter is featured in the most widely used scoring system, WDS [9], but its cut-off (120 beats per minute) is poorly discriminative in children under two years as it can be secondary to fever, irritability, pain, excitement, bronchodilators, or even a physiological heart rate in youngest ones. Another scores have defined higher cut-off values for this parameter. In the score designed by Marlais et al., the heart rate was the best of the five items chosen as predictors for admission[16], but since this was a retrospective study, this parameter was not compared with the heart rate of children being discharged directly from the emergency department; furthermore a higher heart rate may in any case be related to a more aggressive course of bronchodilators prior to admission. Wheezing is another item frequently included in respiratory assessment scales [4,6,9], but it can be difficult to explain, medical material (stethoscope) could be required for its assessment, and case definition of bronchiolitis in several countries does not require wheezing for the diagnosis [20, 21]. We chose instead to address the “respiratory noise” in Q3, as this concept is more understandable by non-medical personnel. The ReSVinet scale covers feeding difficulties (Q1), which have been related in several studies with greater risk of admission and invasive medical interventions [22,23,24] but are neglected by many of the scores already in use in clinical trials [4,5,7]. What is more, some of the most widely used scales deal only with the respiratory impact of the disease, like the Respiratory Disease Assessment Instrument (RDAI) or the Respiratory Assessment Change Score (RACS), both of which are widely used but have shown poor to moderate construct validity [4]; they even neglect findings such as hypoventilation and episodes of apnea. This particular concern is also found in scales more directed to asthmatic patients, like the Pulmonary Score [25]. Last, general condition is another parameter of our clinical scoring system which is usually not included in most of the existing scales, despite the fact that most clinicians seek and value the parental opinion on how severely affected they think their offspring are.

Our results suggest that this score has a reliable performance when calculating the respiratory difficulty from clinical records, making in-person evaluation not necessary. This feature could be of extraordinary value for clinical trials, where frequent in-person assessment and close follow-up is usually required when a participant presents with an adverse event such as bronchiolitis. Moreover, this could also be an asset in everyday clinical practice, allowing the primary care physician to obtain an impression of the condition of the infant at the moment of discharge from the emergency department, or their previous condition in a center that has just referred the patient for interment in another institution. In cases where medical records are not available or are not sufficient to allow the correct use of this score by the physician, parents could evaluate their children with results that can be compared to those of a physician. To the best of our knowledge, there is no other scale which shares these two advantages with the ReSVinet scale.

Limitations of our study include the lack of responsiveness in our currently gathered data, that the score was tested only on hospitalized patients younger than two years of age, and the inexistence of a gold standard for the evaluation of children with acute lower respiratory infection. Most of our patients suffered from acute bronchiolitis which, although the most frequent ARI during infancy, may not represent the whole spectrum of respiratory infections in the first 2 years. A memory bias was a possible confounding factor for the parents who filled in the score (as they completed it at the moment of discharge with the sole help of what they remembered), whereas quality and accuracy of clinical records could influence the results of the observers. Another reservation about the results presented in this study is that the professionals reviewing the clinical records were experienced pediatricians. Further testing may be required to ensure investigators with a different profile conduct the evaluation of clinical records with comparable precision. It could also be argued that clinicians may also bias the results as medical interventions influence the final outcome of the scoring system. However, the opposite could be argued too, as many decisions of pediatricians could be affected by the final score of any scale they use. Despite this potential risk of doctors’ decisions impacting the outcome of the scale, it was not possible to demonstrate that use of drugs, such as nebulized epinephrine or oral medication (corticosteroids or antibiotics), was associated with a significant increase in the final score in any of the two cohorts. In fact, severity range in the ReSVinet Scale correlated well with the definition of severity in the WDS, where medical interventions are not considered. It must be noted that there was a significant overlap between values that would correspond to “mild” and “moderate” episodes in the WDS, a finding that could be explained because the scale has only been applied to hospitalized children.

The final score value for patients in our setting did not show any difference dependent on the pathogen infecting them. In studies concerning other scoring systems, patients suffering from RSV had a higher total score [26]. Our scale failed to demonstrate any statistically significant impact of prematurity in the score, even though those children are at risk of worse episodes of bronchiolitis and many of them should receive prophylaxis against RSV [27]. Children of younger age in our study had a similar mean score to that obtained in older patients, although this finding lacked statistical significance. This can be explained by an insufficient number of subjects in both cohorts or the fact that we excluded children with chronic pulmonary conditions (which is a frequent problem in prematurely born babies); however, this outcome has already been described in medical literature [26], probably explained by the widely accepted measure of preemptively interning prematurely born children and younger infants. Aspects such as distance to a health provider, parental stress, or need for endovenous rehydration influence the decision of discharging a patient from the Emergency Department, and they are usually not gathered in respiratory scales nor clinical records. At the same time, a previous episode of wheezing was linked to a more severe disease in the cohort assessed by professionals but not in the parental cohort, which may be attributed to the significant lower mean age of the latter group. Neither the impact of maternal smoking during pregnancy nor breastfeeding were addressed in this study, although there is ample evidence of their influence in the severity of the disease [2].

The next steps for completing the validation of this score should include the application of this scale in outpatient settings, expansion of the age span of the scale to those children older than two years, and the inclusion of other pathologies which primarily manifest with respiratory difficulty in older children (asthma, atypical causes of pneumonia…) or in patients with underlying conditions. More importantly, the responsiveness of the ReSVinet scale should also be assessed. If the ReSVinet scale’s performance regarding responsiveness is acceptable, this scale will prove to be an important tool for the evaluation of infants suffering from respiratory infections.

Conclusions

Our results indicate that the ReSVinet severity score has adequate criterion validity, adequate construct validity, adequate inter-rater reliability, and appropriate usability for infants under two years of age hospitalized for acute respiratory infections, even when used by non medical-personnel or when evaluating patients using clinical records and therefore avoiding the need for in-person evaluation of the subject by a physician. Although this scale still does not meet all the criteria for a perfectly validated scale, it complies with far more of them than the vast majority of the alternatives currently available and used in the clinical practice.

Acknowledgments

Our most sincere thanks to the members of the ReSVinet group (Louis Bont1, Anne Greenough2, Harish Nair3, Paolo Manzoni4, Nikolaos Papadopoulos5, Eugenio Baraldi6, Brigitte Fauroux7, Terho Heikkinen8, Asunción Mejías9, Fernando Polack10, Mike Sharland11, Octavio Ramilo12, Renato Stein13) and to colleagues working in the FIVE research group (S. Alfayate Miguélez, C. Alvarez Alvarez, F. Álvez González, J.V. Arcos Machancoses, J. Arístegui Fernández, F. Baquero Artigao, E. Bernaola Iturbe, S. Bueno Pardo, MJ. Cabero Pérez, G. Cabrera Roca, C. Calvo, I.M. Ceballos Rodríguez, S. Cerdán Oncala, L.M. Ciria Calavia, J.A. Couceiro Gianzo, J.M. de Cea, F. de Juan Martín, M. Delgado Cardoso, G.M. Escudero Bueno, M. Fernández de Sevilla, L. Fernández Silveira, M. García Barreiro, M.J. García de Miguel, M.L. García-García, F. Giménez Sánchez, A. Gimeno Díaz de Atauri, M. González González, M.I. González Sánchez, M.I. González Tomé, B. González García, S. Guillén Martín, X. Hernández Fernández, J.F. Hurtado Díaz, M. Lillo Lillo, M. López Sousa, S. Martínez Megías, A.I. Menasalvas Ruiz, D. Moreno-Pérez, C. Otero Reigada, M.J. Peláez Cantero, R. Pérez Gorricho, M.E. Pérez Gutiérrez, R. Piñeiro Pérez, R. Rodríguez Fernández, P. Ruiz del Arbol Sánchez, M. Ruiz Jiménez, J.M. Rumbao Aguirre, M. Sánchez Forte, M.A. Tejero Hernández, M. Tobeña Rué, M. Triviño Rodríguez, A. Urda Cardona, N. Viguria Sánchez). We would also like to acknowledge all the members of the REGALIP research network (Miguel Ángel Fernández Pérez, José Manuel Fernández Bustillo, Mercedes del Río Garma, Josefina Pena Nieto, María Elena Álvarez Garnelo, Ángel López-Silvarrey Varela, Consuelo Alonso Alonso, Carmen Murias Taboada, Inés del Río Pastoriza, Julio Regueiro Martínez, Natalia García Sánchez, María Cristina López del Olmo, Lucía Castro Paz, Juan Manuel Sánchez Lastres, María Teresa Valls Durán, Manuel Sampedro Campos, Miguel Angel San José González, Patricia Vilas Rodríguez). Last and not least, we would like to express our admiration to those parents who cooperated with us in the development of this score while they took care of the most precious thing they have.

Contributors

Members of the Respiratory Syncytial Virus network (ReSVinet): 1. Utrech RSV Research group. University Medical Center Utrech, Utrech. The Netherlands.

He is considered the leading author of this group for notification purposes. E-mail: L.Bont@umcutrecht.nl. 2. Center for Vaccines and Immunity. Nationwide Children’s Hospital. Columbus. Ohio. United States of America. 3. Center for Population Health Sciences, The University of Edinburgh Medical School. Edinburgh. United Kingdom. 4. Neonatal Intensive Care Unit. S. Anna Hospital. Torino, Italy. 5. Department of Allergy. Second Pediatric Clinic, University of Athens, Greece. 6. Women’s and Children’s Health Department. Unit of Respiratory Medicine and Allergy. Padova. Italy. 7. Non-invasive Ventilation, Sleep, and Respiratory Physiology Unit. Necker University Hospital, Paris, France. 8. Department of Paediatrics, University of Turku and Turku University Hospital, Turku, Finland. 9. Center for Vaccines and Immunity. Nationwide Children’s Hospital. Columbus. Ohio. United States of America. 10. Division of Infectious Diseases. St. George’s Healthcare NHS Trust. United Kingdom. 11. Division of Infectious Diseases. St. George’s Healthcare NHS Trust. United Kingdom. 12. Center for Vaccines and Immunity. Nationwide Children’s Hospital. Columbus. Ohio. United States of America. 13. Department of Pediatrics, School of Medicine, Pontifica Universidade Católica, Porto Alegre, Brazil.

Author Contributions

Conceived and designed the experiments: FMT AJJ JPS AS MPP CCN LRC JMMS CRT. Performed the experiments: AJJ IRC LVT CCN MPP. Analyzed the data: JPS MCL JGR AJJ. Contributed reagents/materials/analysis tools: AGC JGR. Wrote the paper: AJJ AS FMT. Obtained permission from ethics committee: CRT LVT.

References

  1. 1. Piedimonte G, Perez MK. Respiratory syncytial virus infection and bronchiolitis. Pediatrics in review / American Academy of Pediatrics. 2014;35(12):519–30.
  2. 2. Ralston SL, Lieberthal AS, Meissner HC, Alverson BK, Baley JE, Gadomski AM, et al. Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis. Pediatrics. 2014;134(5):e1474–502. pmid:25349312
  3. 3. Gil-Prieto R, Gonzalez-Escalada A, Marin-Garcia P, Gallardo-Pino C, Gil-de-Miguel A. Respiratory Syncytial Virus Bronchiolitis in Children up to 5 Years of Age in Spain: Epidemiology and Comorbidities: An Observational Study. Medicine. 2015;94(21):e831. pmid:26020386
  4. 4. Fernandes RM, Plint AC, Terwee CB, Sampaio C, Klassen TP, Offringa M, et al. Validity of bronchiolitis outcome measures. Pediatrics. 2015;135(6):e1399–408. pmid:25986025
  5. 5. Bekhof J, Reimink R, Brand PL. Systematic review: insufficient validation of clinical scores for the assessment of acute dyspnoea in wheezing children. Paediatric respiratory reviews. 2014;15(1):98–112. pmid:24120749
  6. 6. Ramos Fernandez JM, Cordon Martinez A, Galindo Zavala R, Urda Cardona A. Validation of an acute bronchiolitis severity scale. Anales de pediatria. 2014;81(1):3–8. pmid:24103250
  7. 7. Destino L, Weisgerber MC, Soung P, Bakalarski D, Yan K, Rehborg R, et al. Validity of respiratory scores in bronchiolitis. Hospital pediatrics. 2012;2(4):202–9. pmid:24313026
  8. 8. Moreno-Perez D, Calvo C, Five Study G. Epidemiological and clinical data of hospitalizations associated with respiratory syncytial virus infection in children under 5 years of age in Spain: FIVE multicenter study. Influenza and other respiratory viruses. 2014;8(2):209–16. pmid:24382343
  9. 9. Ferres Mataro J, MBMFRR , Julia Brugues A, Bonal de Falgas J. Adrenalina subcutánea versus salbutamol inhalado en el tratamiento de la crisis asmática infantil. Anales espanoles de pediatria. 1987;27:37–40.
  10. 10. Lopez-Raton M, Rodriguez-Alvarez MX, Cadarso-Suarez C, Gude-Sampedro F. OptimalCutpoints: An R Package for Selecting Optimal Cutpoints in Diagnostic Tests. Journal of Statistical Software 2014;61(8):1–36.
  11. 11. Carmines EG, Zeller RA. Reliability and validity assessment. Beverly Hills, Calif.: Sage Publications; 1979.
  12. 12. Argimon Pallás JM, Jiménez Villa J. Métodos de investigación clínica y epidemiológica. Madrid: Elsevier; 2013.
  13. 13. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 4th ed. Oxford; New York: Oxford University Press; 2008. xvii, 431 p. p.
  14. 14. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of clinical epidemiology. 2010;63(7):737–45. pmid:20494804
  15. 15. Duarte-Dorado DM, Madero-Orostegui DS, Rodriguez-Martinez CE, Nino G. Validation of a scale to assess the severity of bronchiolitis in a population of hospitalized infants. The Journal of asthma: official journal of the Association for the Care of Asthma. 2013;50(10):1056–61.
  16. 16. Marlais M, Evans J, Abrahamson E. Clinical predictors of admission in infants with acute bronchiolitis. Archives of disease in childhood. 2011;96(7):648–52. pmid:21339199
  17. 17. Schuh S, Freedman S, Coates A, Allen U, Parkin PC, Stephens D, et al. Effect of oximetry on hospitalization in bronchiolitis: a randomized clinical trial. Jama. 2014;312(7):712–8. pmid:25138332
  18. 18. Mallory MD, Shay DK, Garrett J, Bordley WC. Bronchiolitis management preferences and the influence of pulse oximetry and respiratory rate on the decision to admit. Pediatrics. 2003;111(1):e45–51. pmid:12509594
  19. 19. Bergman AB. Pulse oximetry: good technology misapplied. Archives of pediatrics & adolescent medicine. 2004;158(6):594–5.
  20. 20. Scottish Intercollegiate Guidelines Network. Bronchiolitis in Children: A National Clinical Guideline. Scotland:NHS. November 2006.
  21. 21. Gonzalez Caballero D, Gonzalez Perez-Yarza E. Acute bronchiolitis: fundamentals of a rational protocol. Anales espanoles de pediatria. 2001;55(4):355–64. pmid:11578545
  22. 22. Mansbach JM, Clark S, Christopher NC, LoVecchio F, Kunz S, Acholonu U, et al. Prospective multicenter study of bronchiolitis: predicting safe discharges from the emergency department. Pediatrics. 2008;121(4):680–8. pmid:18381531
  23. 23. Mansbach JM, Piedra PA, Stevenson MD, Sullivan AF, Forgey TF, Clark S, et al. Prospective multicenter study of children with bronchiolitis requiring mechanical ventilation. Pediatrics. 2012;130(3):e492–500. pmid:22869823
  24. 24. Parker MJ, Allen U, Stephens D, Lalani A, Schuh S. Predictors of major intervention in infants with bronchiolitis. Pediatric pulmonology. 2009;44(4):358–63. pmid:19283838
  25. 25. Smith SR, Baty JD, Hodge D III. Validation of the pulmonary score: an asthma severity score for children. Acad Emerg Med 2002; 9: 99–104. pmid:11825832
  26. 26. Rodl S, Resch B, Hofer N, Marschitz I, Madler G, Eber E, et al. Prospective evaluation of clinical scoring systems in infants with bronchiolitis admitted to the intensive care unit. European journal of clinical microbiology & infectious diseases: official publication of the European Society of Clinical Microbiology. 2012;31(10):2667–72.
  27. 27. Hasegawa K, Pate BM, Mansbach JM, Macias CG, Fisher ES, Piedra PA, et al. Risk factors for requiring intensive care among children admitted to ward with bronchiolitis. Academic pediatrics. 2015;15(1):77–81. pmid:25528126