A Delphi Process to Optimize Quality and Performance of Drug Evaluation in Neonates

Background Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. The objective of this work was to identify the major criteria considered necessary for selecting neonatal intensive care units that are able to perform drug evaluations competently. Methodology and Main Findings This Delphi process was conducted with an international multidisciplinary panel of 25 experts from 13 countries, selected to be part of two committees (a scientific committee and an expert committee), in order to validate criteria required to perform drug evaluation in neonates. Eighty six items were initially selected and classified under 7 headings: “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6). Sixty-one items were retained and headings were rearranged after the first round, 34 were selected after the second round. A third round was required to validate 13 additional items. The final set includes 47 items divided under 5 headings. Conclusion A set of 47 relevant criteria will help to NICUs that want to implement, conduct or participate in drug trials within a neonatal network identify important issues to be aware of. Summary Points 1) Neonatal trials remain difficult to conduct for several reasons: in particular the need for study sites to have an existing infrastructure in place, with trained investigators and validated quality procedures to ensure good clinical, laboratory practices and a respect for high ethical standards. 2) The present Delphi study was conducted with an international multidisciplinary panel of 25 experts from 13 countries and aims to identify the major criteria considered necessary for selecting neonatal intensive care units (NICUs) that are able to perform drug evaluations competently. 3) Of the 86 items initially selected and classified under 7 headings - “NICUs description - Level of care” (21), “Ability to perform drug trials: NICU organization and processes (15), “Research Experience” (12), “Scientific competencies and area of expertise” (8), “Quality Management” (16), “Training and educational capacity” (8) and “Public involvement” (6) - 47 items were selected following a three rounds Delphi process. 4) The present consensus will help NICUs to implement, conduct or participate in drug trials within a neonatal network.


Introduction
In the neonatal population, more than 90% of products are used unauthorised or off-label, especially when neonates are treated in Neonatal Intensive Care Units (NICUs). New treatments are often introduced in the neonatal therapeutic arena without specific evaluation, on the basis that they have proven efficacy in adults and older children. Altogether, many drugs, even when authorised for use in neonates, would benefit from further validated data and a consensus among neonatologists to ensure the most optimal use. In order to improve this situation, a few initiatives were undertaken starting in the USA with the Food and Drug Administration Modernization Act and Best Pharmaceuticals for Children Act respectively in 1997 and 2002, and the American Paediatric Research Equity Act and Newborn Drug Initiative respectively in 2003 and 2006. Similarly in Europe, the European Paediatric Regulation entered in force in June 2007 to increase the development of medicines for all paediatric age groups, including neonates.
There are many major and already well-known practical and ethical issues in conducting drug evaluation in neonates [1,2]. Although neonates represent only a small part of the population, they have specific diseases and high variations in disease presentations with a major risk of unfavorable long-term outcome. Evaluation of drug pharmacokinetics, efficacy and safety are required in the different neonatal age groups from 24 to 44 weeks' gestational age, characterised by differences in physiological and pharmacological maturation affecting drug disposition and effects [3,4]. Suitable approaches, adapted to neonates are recommended but need to be more widely used: population pharmacokinetic and bridging studies [5,6], adapted designs and other methodologies recognised as pertinent to evaluate efficacy when randomized controlled trials are not possible [7][8][9].
Many of these issues can be solved by bringing together scattered expertise within a neonatal network dedicated to drug evaluation in neonates, in collaboration with the European Society of Developmental, Perinatal and Paediatric Pharmacology (ESDPPP). According to the request of the European Commission and following the EnprEMA network initiative (European Network of Paediatric Research at the European Medicines Agency), we conducted a Delphi process [10][11][12][13][14] to identify the criteria that would help neonatologists to organise a NICU research infrastructure in order to conduct drug evaluation trials and be part of a European network for drug evaluation in neonates.

Ethics Statement
The study did not need institutional review board approval as it did not affect patient care, and the information that it generated was used for consensual quality criteria only.
The objectives of the Delphi study were presented to all participants, their agreement and availability to participate were obtained (their written consent consisted of replying positively by email to the invitation sent by the organisers), and their independency verified.

Organization of the Delphi Study
The objective of the Delphi study was to obtain a consensus on the prerequisites that NICUs need to fulfil in order to perform clinical drug trials. Two experts groups conducted the Delphi study by email between February 2012 and February 2013.

Composition of the Scientific and Expert Committees
Two expert groups conducted the Delphi study: the Scientific Committee consisted of 10 experts responsible for organising the Delphi device, selecting and/or approving the members of the Expert Committee, drafting the successive versions of the questionnaire, and analysing the data after each round (questions and answers, results). All members of the Expert Committee (the panellists) agreed to answer the questionnaire during each round and to criticize the answers if necessary.
All the participants ( Table 1) were initially selected by the organisers (FL & EJA) in order to ensure that they will represent all potential differences in background, occupational environment, clinical approaches or practices. They were contacted because of their recognised expertise in the different areas of knowledge required for drug evaluation in neonates (neonatology, pharmacology, clinical research), after evaluation of teaching functions, publications, participations to international scientific societies, scientific and ethic boards or networks). The composition of the Scientific and Expert committees included clinical investigators (paediatricians or neonatologists), pharmacologists or pharmacists, researchers, regulators, and employed by industry, academia or regulatory agencies and in most cases members of scientific societies and clinical networks. They were from various countries/ continents, had a broad range of ages and different levels of expertise. All the members of the Scientific committee agreed on the composition of the groups, and accepted to participate. The expert selection was then submitted for approval to the Scientific Committee, and finally two additional members were added upon their suggestions.

Questionnaire preparation
The Scientific Committee drafted the first version of the questionnaire, composed of questions and simple items. Each member proposed recognition criteria and a total of 105 items were listed. They were then invited to confirm and/or refine all the items to allow them to be rated during the Delphi process. The resulting questionnaire was submitted a third time by the organisers, for validation by each member of the Scientific Committee. Finally, the elaborated questionnaire included 86 items divided over 7 headings: ( Table 2,

Rounds
As required in a Delphi study, each item is required to be assessed twice, using a 1 st and a 2 nd level of consensus to be finally selected.

First round
The first round was performed from February to July 2012. The panellists (Expert Committee) received the first questionnaire by electronic mail and they were invited to rate their agreement about each item on a 9-point scale, where 1 meant definitely not agree and 9 definitely agree. They were also invited to comment on each item using a dedicated ''comment box'', and/or to add items considered as important. Items were included in the second round if a consensus was reached based on two selection criteria: a median score in the top tertile (7)(8)(9) and at least 65% of panel ratings in the top tertile (1 st level of consensus). At the end of the first round, the questionnaire was slightly modified by the scientific committee and a few items were added to take into account comments and suggestions of the panellists.

Second round/Third round
The second round was conducted between November and December 2012. All panellists who had participated in the first round were sent the second-round questionnaire by email, with the results of the first round including median panel rating, frequency distribution rating as well as their individual ratings from the first round. They were asked to re-score each item based on their own opinion, and the panel responses obtained during the first round. The second round also included a limited number of ''new'' items, not evaluated in the first round and that had to send out from a second evaluation during what has been called a ''third round'' in order to evaluate each item twice. This third round took place between January and February 2013. To be included in the final list, the items were selected by the level of median rating in the top tertile (7-9) and a 75% agreement among panellists that the rating was in the top tertile (7-9) (2 nd level of consensus).

Results
The characteristics of the two experts groups (n = 25) are presented Table 1 and they are all listed in alphabetic order in the ''Acknowledgment'' section. They were selected because they had at least 10 years' experience and a well-known international recognition in their field of expertise.
The members of the Scientific Committee were selected and contacted by the organisers, and all of them agreed (10/10, 100%) to participate.
Of the twenty panellists appointed by the Scientific Committee to participate to the Delphi rounds, 15 (79%) accepted: 9 were neonatologists, 2 were pharmacologists, 2 were institutional and industrial project managers and 1 was a neonatologist/scientific officer in a regulatory authority. All of them participated in the three rounds of the process.

First round
The 7 headings comprising a total of 86 initial items tested during the first round are listed in Table 2, column A ( Figure 1).
Adjustments at the end of the first round. Adjustments were made prior to the second round, taking into account the suggestions and comments of the Expert Committee and the Scientific Committee review ( Table 2, column B): rearrangements of headings and redefinition of a few items were made, such as H3: ''Research Experience'' and H6: ''Training and educational capacity'' that were re-organized and included under the first heading H1: ''NICUs Description''; a few items were better defined (H1: i6; and i40; H2: i33 to i35; H3: I49; H4: I65, i66 and i72), while others were compiled into new items (H1: i69and i759; H2: i269; H4: i609; and H5: i829); and a new ''Access to electronic record'' was added (H2: i369).
After these adjustments, the questionnaire included 61 items under 5 headings ( Table 2, column B) as: from 57 items selected in this first round, 53 items remained; similarly, among the 29 discarded items in the previous round, 7 items were maintained to be re-assessed in the second round; and one additional item was added.

Second round
At this step of the process, 61 items were therefore evaluated, including modified, retained or additional items (27 items), which were considered as ''new items'' and were rated for the first time through this second round. Results are presented Table 2.
47 items were selected (47/61, 77%). Among them, 34 items were definitively retained based on the second level of consensus, and 13 items required to be re-assessed (second assessment) through a third round ( Figure 1). The remaining 14 items were, for their part, definitively discarded.

Third round
The previous 13 items that were re-assessed reached the second level of consensus and were definitively selected.

Final results
At the end of the process, 47 items were finally selected according to the second level of consensus and divided under 5 headings ( Table 3 and Figure 1).

Discussion
Drug trials should always be conducted in sites that can guaranty quality, performance and high ethical standards and this is obviously of even greater importance in neonatal drug evaluations. In this context, the present Delphi study was conducted to define criteria that neonatologists should consider to optimise organisation of the NICU, trial management and conduct. Starting from 86 items divided in 7 headings, a consensus among 15 multidisciplinary experts with a wide range of experience in neonatology, experimental or regulatory practices identified 47 items in 5 re-organised headings ( Figure 1). These criteria should help to set up a network of NICUs specialised in drug evaluation in neonates.
As previously stated, drug research in neonates is difficult and, consequently, the number of neonatal drug trials is limited and sometimes of poor quality [7]. Therefore, collaborative, multicenter and multinational studies are essential to recruit neonates with similar diseases from various regions or countries in order to obtain a sample size of sufficient magnitude and to conduct scientific sound studies. In addition to increasing recruitment capacities, such specialised centers will ''combine and share'' competences in order to build a network to guaranty trial quality and performances, but also to develop investigator and nurse training. This approach is in agreement with the request of the European Commission and is currently supported by the European Medical Agency that developed the EnprEMA network, ''a network of networks'' [15].
Most of the paediatric networks that we are aware of are so called ''Paediatric improvement networks'', collaborating to reduce the gaps and disparities in health care quality and improve outcomes by accelerating the translation of evidence into practice [16][17][18]. They are mostly organised in paediatric subspecialties such as the Children's Oncology Group (COG) in the USA, the Paediatric Rheumatology INternational Trials Organization (Pinto) and the Paediatric European Network for Treatment of AIDS (Penta) in Europe [15]. Some of them are dedicated to perinatal or neonatal care such as the California Perinatal Quality Care Collaborative (CPQCC) [16,19] but many other national or      regional initiatives do exist [17]. All of them underline the importance of a close link with research although limited networks of NICUs were set up for research purposes. In the USA, the Paediatric Pharmacology Research Units network (PPRU) was a cooperation of clinical centers participating in the cooperative agreement with NICHD and represented academic institutions with experience in multi-center clinical research. They agreed to abide by the study protocols and have comparable staff, facilities, and equipment. In Europe, the European Neonatal Network (EuroNeoNet) primarily aims to give European neonatologists a tool to perform their own quality assurance and benchmarking. Additional neonatal networks do exist at the national level in most countries, including the German Neonatal Network (GNN), the National Institute for Health Research Medecines for clinical research network (NIHR) in the UK, and the Paediatric Clinical Investigation Centers in France (CIC). However, the criteria that might facilitate initial adhesion and follow-up of the NICUs that are members of a neonatal network are not easily identified. The present Delphi study was conducted in order to combine opinions into group consensus on this topic and is reported here according to published recommendations [20].
The first heading ''NICUs description'' includes subdivisions focussing on ''Level of care, Research experience and Training -Education capacities'' with description of the size, organization, and teaching hospital status. All the items related to ''highly specialised care technics'' (such ECMO, or cooling) were put together as the centers providing them are highly specialised and always identified at the regional or national level. Therefore, the corresponding NICUs will be identified within a network and contacted at the initial step of study feasibility if such technics are required by the protocol.
The term ''feasibility'' is currently used but may cover different issues: for industry, the question behind feasibility is: do the patients corresponding to the inclusion criteria really exist and in addition, where do we find them? Indeed, when industry-driven, the protocol is developed according to regulatory guidelines for drug evaluation in neonates [3] and follows a Paediatric Investigation Plan that is binding. There are concerns that these may not always feasible in the neonatal population however, as per current experience with TINN (Treat Infections iN Neonates) projects (TINN1 and TINN2) -very few indications of the drug limiting inclusions for a RCT; registry considered as noninformative. In contrast, for clinicians, feasibility primarily means: is this acceptable for the patient and his parents? Do we have time and staff for this? Indeed, analysis of feasibility by a local Scientific Review Board with the medical and nurse staff is essential to evaluate recruitment capacities within predefined calendars and in our experience, the best response may sometimes be NO [21,22].
Research activities will be reported annually in centers already conducting drug trials, underlining the major importance to be given to performances evaluated in terms of adherence, number of patients included, queries. Continuous re-evaluations of quality and performances are now currently performed in neonatal intensive care and with significant results in terms of long-term quality-adjusted survival [23]. They are also required to evaluate research activities, ensure that all organisational and training efforts are maintained, allowing to reach positive results in terms of recruitment, adherence to timelines, quality and Ethics. Although discussed, additional criteria such as success in grant applications and/or publications were thought to reflect more networking activities than individual NICU activities and performances.
Three headings ''NICUs organisation, Scientific competencies, Quality management'' include all the items required to guaranty quality and performances of clinical research. Among key items, a  [24]. Similarly, their role in clinical research should be better acknowledged and recognised. As neonates hospitalised in intensive care are the most nurse-intensive patients, conducting clinical drug research requires additional dedicated research nurses. In such context, the required neonate -research/nurse ratio in a given NICU obviously depends upon research workload related to the trials and should also include time for training care nurses, availability for parents' information and all additional tasks related to quality and ethics [25].
Although research in neonates is almost always preceded by research in adults and children, diseases in neonates are different in terms of clinical presentation, evolution and risks. It is therefore essential for researchers and pharmaceutical industry people to get advice from experts in the NICU and its surrounding [26], as involvement in pharmacokinetic and pharmacodynamics drug evaluation, trial designs adapted to low numbers, is expected. The quality management is based on Standard Operating Procedures, i.e documents with detailed instructions, written to describe steps to follow in all activities under defined conditions. They are derived from knowledge of Good Clinical and Laboratory Practices from International Conference on Harmonisation (ICH guideline Q11); and Good Laboratory Practice (GLP) from the Organisation for Economic Cooperation and Development (OECD). Such knowledge is required for all health professionals participating in a clinical trial. In particular, the roles, obligations and responsibilities of the sponsor and all health professionals, respect of ethical standards focussing on parental information and consent are clearly defined. The major ''Quality SOPs'' available in any research center and that have been used and adapted to the TINN2 multicenter European drug trial in neonates are listed Table 4.
Among them, ''Trial Pharmacy Procedures'' including procedures related to ''preparation and administration of the IMPs'' are essential as dilutions, sometimes multiple, of a concentrated product, low volumes of infusion, potential physico-chemical interactions are key issues [27,28]. In addition, in this heading, adherence to trial dependent -standard operating procedures should be evaluated by monitors on a regular basis, and documentation and report of all adverse drug reactions.
The last heading is related to ''Public involvement''. The philosophy of ''Family-centered care'' referring to a partnership between parents and the medical staff [29] focuses on involving them in all ethical and medical decisions related to their newborn's care [30]. Similarly, decisions regarding neonatal research enrollment need to be made conjointly with parents and health care professionals and many studies are now available to improve the consent process [31][32][33]. Researchers are also concerned by involving parents in an earlier phase, in order to help drafting the information and consent documents and make them more accessible to parents, as reflected in the selected items kept in the heading. One item ''Involvement of parents and their representatives or organisations in trial design'' was not selected, probably because ''Trial design'' refers to specific methodologies to optimize drug trials taking into account neonatal specificities. In contrast, trials dealing with safety evaluation or long-term follow up are to be discussed extensively with parents in order to evaluate their view on how the child and the whole family will be involve, on how to face open questions on outcome and potential sequelae or neurodevelopmental disorders.
In conclusion, this is to our knowledge, the first consensus obtained by experts and aiming to list the items that should be considered to organize a neonatal network of specialised intensive care units dedicated to drug evaluation. Some of these items may not be required in all units but should be identified to optimize trial design, conduct highly specialised evaluations and train health professionals to trial conduct for the benefit of neonates and their parents.