Browse Subject Areas

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

POPI (Pediatrics: Omission of Prescriptions and Inappropriate Prescriptions): Development of a Tool to Identify Inappropriate Prescribing

  • Sonia Prot-Labarthe ,

    These authors are joint senior authors on this work

    Affiliation Pharmacie, AP-HP Hôpital Robert-Debré, Paris, France

  • Thomas Weil ,

    These authors are joint senior authors on this work

    Affiliations Pharmacie, AP-HP Hôpital Robert-Debré, Paris, France, Pharmacie Clinique, Université Paris Descartes, Paris, France

  • François Angoulvant,

    Affiliation Service d’Accueil des Urgences, AP-HP Hôpital Robert-Debré, Paris, France

  • Rym Boulkedid,

    Affiliations Unité d’Epidémiologie Clinique, AP-HP Hôpital Robert Debré, Paris, France, Inserm U 1123 et CIC 1426, Paris, France

  • Corinne Alberti,

    Affiliations Unité d’Epidémiologie Clinique, AP-HP Hôpital Robert Debré, Paris, France, Inserm U 1123 et CIC 1426, Paris, France, Sorbonne Paris Cité UMRS 1123, Université Paris Diderot, Paris, France

  • Olivier Bourdon

    Affiliations Pharmacie, AP-HP Hôpital Robert-Debré, Paris, France, Pharmacie Clinique, Université Paris Descartes, Paris, France, Laboratoire Educations et Pratiques de Santé, Université Paris XIII, Bobigny, France

POPI (Pediatrics: Omission of Prescriptions and Inappropriate Prescriptions): Development of a Tool to Identify Inappropriate Prescribing

  • Sonia Prot-Labarthe, 
  • Thomas Weil, 
  • François Angoulvant, 
  • Rym Boulkedid, 
  • Corinne Alberti, 
  • Olivier Bourdon


16 Sep 2014: The PLOS ONE Staff (2014) Correction: POPI (Pediatrics: Omission of Prescriptions and Inappropriate Prescriptions): Development of a Tool to Identify Inappropriate Prescribing. PLOS ONE 9(9): e108007. View correction



Rational prescribing for children is an issue for all countries and has been inadequately studied. Inappropriate prescriptions, including drug omissions, are one of the main causes of medication errors in this population. Our aim is to develop a screening tool to identify omissions and inappropriate prescriptions in pediatrics based on French and international guidelines.


A selection of diseases was included in the tool using data from social security and hospital statistics. A literature review was done to obtain criteria which could be included in the tool called POPI. A 2-round-Delphi consensus technique was used to establish the content validity of POPI; panelists were asked to rate their level of agreement with each proposition on a 9-point Likert scale and add suggestions if necessary.


108 explicit criteria (80 inappropriate prescriptions and 28 omissions) were obtained and submitted to a 16-member expert panel (8 pharmacists, 8 pediatricians hospital-based −50%- or working in community −50%-). Criteria were categorized according to the main physiological systems (gastroenterology, respiratory infections, pain, neurology, dermatology and miscellaneous). Each criterion was accompanied by a concise explanation as to why the practice is potentially inappropriate in pediatrics (including references). Two round of Delphi process were completed via an online questionnaire. 104 out of the 108 criteria submitted to experts were selected after 2 Delphi rounds (79 inappropriate prescriptions and 25 omissions).

Discussion Conclusion

POPI is the first screening-tool develop to detect inappropriate prescriptions and omissions in pediatrics based on explicit criteria. Inter-user reliability study is necessary before using the tool, and prospective study to assess the effectiveness of POPI is also necessary.


Rational use of medicines refers to the correct, proper and appropriate use of medicines. The WHO estimates that over 50% of medications are prescribed, dispensed or sold inappropriately and that more than 50% of all countries do not implement basic policies to promote rational use of medicines [1]. In developing countries, less than 40% of patients in the public sector and 30% in the private sector are treated according to clinical guidelines [1]. The use of medication in pediatrics should be based on established recommendations from well-conducted clinical trials, however in the absence of such trials, recommendations are often based on clinical experience. Rational prescribing for children is an issue for all countries and has been inadequately studied [2], [3].

The Medical Subject Headings (MeSH) tool is a thesaurus integrated into the PubMed search engine that allows access to the MEDLINE database. In 2011, it introduced the term ‘Inappropriate Prescribing[4]. The use of a medication for which the associated risks outweigh the expected benefits can be considered as inappropriate, especially if an alternative treatment has been shown to be safer and more effective. According to a report published by the French National Authority for Health, both prescription of medication for excessively long periods and the failure to prescribe recommended medications can be classified as inappropriate prescribing [5]. In addition, the prescription of medications that have a high risk to interact with other drugs, or with the disease can also be considered as inappropriate. All of these examples will be herein described as inappropriate prescription (IP).

Many tools have been developed to detect IP in the elderly. This is largely due to the susceptibility of the elderly to disease and the prevalence of polypharmacy in this population. The Beers Criteria for Potentially Inappropriate Medication Use in Older Adults [6] were the first criteria to be proposed and are also the most well-known. However, one major disadvantage of this tool is that it includes many medications that are not sold in Europe. In 2008, Gallagher et al. developed a tool called STOPP/START (Screening Tool of Older Person’s Prescriptions/Screening Tool to Alert doctors to Right Treatment) that comprises two medication lists [7]. The ‘STOPP’ list includes prescriptions that should be stopped and the ‘START’ list includes prescriptions that should be initiated, in the absence of any contra-indication. This system is particularly useful because it classifies drugs according to various medical conditions that are commonly found in the elderly. In a study in 2008, the use of the STOPP list identified IPs in 35% of a cohort of elderly patients and one third of these IPs were associated with an adverse drug event [8]. Another study involving randomized hospitalized patients showed that the occurrence of IP was 35% lower in patients who were prescribed drugs according to STOPP/START criteria than in patients for who usual pharmaceutical criteria were used [9]. However, so far no tool has been created to the pediatric population.

Our objective was to create the first IP tool in pediatrics, which we called POPI (Pediatrics: Omission of Prescriptions and Inappropriate prescriptions) [10]. Our objective was to raise awareness about this tool and to validate its content through a network of medical professionals working in pediatrics.

Materials and Methods

POPI should contain around 100 propositions that were classified according to biological system and classified according to whether they involve an omitted or an inappropriate prescription. The propositions were further divided within these two lists according to the major biological systems (as this was done for other geriatric tools [6], [8]). We decided to include around 100 propositions: this was a good compromise between the number of major biological systems to explore, the number of items in the geriatric lists and the maximum number of items compatible with a tool easy use.

This project began in the Robert-Debré University Hospital, AP-HP (Assistance Publique-Hôpitaux de Paris) in Paris, France. POPI is comprised of a list of health problems frequently encountered in pediatrics. These problems were chosen in 2010 according to the following criteria, as concerns pediatrics: their frequency in the general population, the reasons for hospitalization (listed in the French hospital system’s medico-administrative database in 2011 ‘programme de médicalisation des systèmes d’information’ [PMSI] at the Robert-Debré University Hospital), and their prevalence according to data from the French National Health Insurance Fund for Employees (la caisse nationale de l’assurance maladie des travailleurs salariés [CNAMTS]) of long-term illnesses [11]. According to these criteria, we selected health problems requiring either drug intervention, or no pharmacological intervention whatsoever (i.e. treatment in such cases would be considered as inappropriate).

For each disease, we considered the recommended pharmacological treatments, the risks of errors, contra-indications, drug-drug interactions, drug-disease interactions, and issues associated with dose and route.

For each of the chosen themes (or diseases), we established a literature search strategy to retrieve management recommendations. We selected only recommendations that were both backed up by evidence and were published after 2000. Recommendations were weighted according to their publication date. Data was obtained from learned or professional societies or agencies in France, the United States, or Great Britain: the French Health Products Safety Agency (ANSM or Agence Française de Sécurité Sanitaire des Produits de Santé), the French National Authority for Health (Haute Autorité de Santé Française), the French Society for Pediatricians (Société Française de Pédiatrie), the American Academy of Pediatrics (National Guideline Clearing House), and finally the National Institute for Health and Clinical Evidence, Cochrane Library (UK). We used the following databases for the origin of pharmacological agents, the commercially available forms, and potential drug-drug interactions: Thériaque [12], Micromedex [13], Lexi-Comp’s Pediatric & Neonatal Dosage Handbook [14], and the French medical journal ‘La Revue Prescrire’ [15]. We also used the MEDLINE database to search for examples of medication error and inappropriate prescription.

We validated the propositions included in POPI by a two round Delphi method [16], [17]. The aim of the Delphi method is to achieve a convergence of opinion and a general consensus on a particular topic, by questioning experts through successive questionnaires. The experts were chosen according to their area of expertise, and included pediatricians most of who are members of the French Society of Pediatricians, and pharmacists mostly members of the French Society of Clinical Pharmacy. Each expert has disclosed his conflicts of interest.

The fisrt round questionnaire comprised all of the propositions included in POPI draft, which were graded according to a nine-point Likert scale for agreement. A score of 1 indicates ‘total disagreement’ whereas a score of 9 ‘total agreement’, with intermediate values indicating degrees of agreement between these two extremes. The experts were also encouraged to make suggestions about the dose, the frequency, and the duration of treatment, provided that they could cite appropriate references to back up these suggestions. The experts could also comment on the propositions. The questionnaire was available online via the website ‘SurveyMonkey’, which is a tool designed to conduct web-based surveys [18].

Each of the panelists who had participated in the first round was sent the second-round questionnaire. These panelists were also given feedback on the results of the first round (their own previous individual ratings, median panel rating, and frequency distribution of the agreement rating). The panelists were then asked to re-rate each proposition based on both their own opinion and the group response to the previous round.

Only the propositions that obtained a median score in the upper tertile (between 7 and 9) with an agreement of more than 65% of participants in the first round of Delphi were retained. These propositions were modified according to the experts’ comments, and were subjected to a second round of questioning. Only the propositions that obtained a median score between 7 and 9 with an agreement of more than 75% of participants in this second round were retained. The experts had two weeks to reply to the questionnaire. For both the first and second questionnaire, a reminder was sent out one and two weeks before the deadline.

Experts characteristics were also noted, including their age, their place of work, and their number of years of experience.

Qualitative data are expressed as numbers (percentages) and quantitative data as median (quartiles) and minima, maxima. SAS software (VERSION 9.3) was used for statistical analysis.

The study was reviewed and approved by the Robert-Debré institutional review board.


The first draft of POPI contained 108 propositions: 80 propositions of Inappropriate Prescription (IP) and 28 propositions of Omission of Prescription (OP). These propositions were classified into five broad categories: digestive problems (n = 15); Ear, Nose and Throat (ENT) problems or pulmonary problems (n = 23); dermatological problems (n = 30); neuropsychiatric disorders (n = 16); and diverse illnesses (n = 24). Each category was further divided into several medical conditions. We contacted 33 experts between June and September 2012. Sixteen experts agreed to participate in the development of the POPI tool. The median expert age was 49 years, range [32–66 years] and their median number of years of experience was 25 years, range [3–40 years]. The ratio of pediatricians to pharmacists was 1∶1. Half were working in a hospital environment and the other half were working in the community. Each physician working within a hospital environment was specialized in a particular medical domain: endocrinology, hematology, nephrology, cancerology, or pulmonology.

Figure 1 shows the workflow of the study. The first round questionnaire was sent to the 16 experts at the start of December 2012 and the replies were collected by the start of January 2013; 14 (14/16, 87.5%) participants responded to the first round of questions. Two propositions received 13 replies because one expert did not use the answer grid properly. More than 65% of the panelists gave top-tertile (7–9) agreement to 93 propositions. Ten propositions were modified according to the experts’ comments during this first round of questions producing 93 propositions for the second round.

Figure 1. Workflow for the validation of POPI.

*An item involving codeine was removed subsequent to the validation of the propositions included in POPI, following the revelation of new contraindications for this drug in children under 12 years old [22]. N: Number of items; n: number of panelists.

The second questionnaire was submitted at the end of March 2013 and the replies were collected within one month. During this second round of questions, 85.5% (12/14) of participants replied. More than 75% of the panelists gave top-tertile agreement to all the 93 propositions submitted.

The propositions involving the category ‘digestive problems’ (n = 15) were submitted separately in April 2013 for 2 rounds rating. All of these propositions were unanimously accepted during two rounds of questions that took place between April and May 2013. Ten experts participated in these rounds of questions (i.e. 71.5% of the 14 experts who replied in the initial survey carried out between December 2012 and January 2013.

Table 1 shows the 102 propositions that were validated for use in POPI. A proposition involving codeine was removed subsequent to the validation of POPI, following the revelation of new contraindications for this drug in children under 12 years old [19]. Another proposition about the use of permethrin for lice was removed because of new recommendation to use dimeticone first (lack of resistance) [20]. Table 2 summarizes the references justification for each table 1 pathology.


POPI (Pediatrics: Omission of Prescriptions and Inappropriate prescriptions) is the first tool that has been designed to detect the omission of prescriptions or inappropriate prescriptions specifically in pediatric patients [21]. If polymedication is unusual for children, there are however multiple health care professional who prescribe or counsel drug for children: general practitioner, paediatricians, pharmacists, nurses, midwives etc.

The POPI criteria are based on the same classification system as the STOPP/START criteria, (i.e. according to the major biological systems [8]). We selected this form because such lists have been successfully used to detect preventable adverse drug events [8], [9], [22]. The Beers criteria were updated in 2012 to incorporate this classification system [6]. Our tool, which was developed using a Delphi method, was validated by 14 health care professionals. The Delphi method is one of the main method used for the development of tools designed to detect inappropriate prescriptions in geriatric patients [6], [8], [22][26]. The number of experts to develop geriatric tools vary between 11 and 32 and their specialties include pharmacy, psychopharmacology, pharmacology, pharmacoepidemiology, internal medicine or geriatrics [6], [7], [23], [24], [26]. For the validation of POPI, the number of experts in each category was equal so as to ensure that hospital and community environments were equally represented. There is currently no consensus regarding the composition of such panels of experts; there are no recommendations about the numbers or qualifications of experts to be included. More pharmacists were involved in the validation of the POPI criteria than in the validation of similar criteria that were developed for geriatrics. This strong representation is partly because the initial project was developed by hospital pharmacists. One limitation of our study in the absence of general practitioners from our panel of experts. Indeed, these doctors regularly deliver health care to children in the community and hence could greatly benefit from the use of POPI.

Few data about inappropriate prescriptions have been published in pediatric patients. Although studies have investigated medication errors [27][29], not one study has examined the link between the rate of medication errors and the rate of adverse drug events in pediatrics. In adults, it is estimated that around one adverse drug occurs for every 100 medication errors [30], [31]. There is increasing recognition that rational prescribing is an important issue in children [2].

The different propositions included in POPI were based on recommendations from recognized learned and academic societies and were preselected by the initial working group. Of the 108 propositions, 104 were validated by experts in the first round of Delphi, and all of the propositions submitted in the second round were subsequently validated. The final version of the POPI criteria contains 79 examples of inappropriate prescription and 25 examples of omission of prescription. The modifications that were made during the first and second rounds of Delphi involved refinements in the phrasing and exact details of the propositions. Overall, the experts were very responsive, and we collected around 80% of replies within three weeks of sending the questionnaires. The feedback of the experts was very positive and many of them commented that they were very interested in the development of POPI. The STOPP/START criteria contained as many propositions to validate as the POPI criteria. For STOPP/START, a consensus was obtained for 77 out of 80 propositions that were submitted in the first round [7]. For the criteria developed by Laroche et al. a consensus was reached for 33 out of 37 criteria during the first round [26]. This illustrates the importance of preselecting the propositions prior to their submission to experts, to ensure that a consensus will be reached on the largest possible number of propositions. The time that experts were given to reply to questionnaires during the development of criteria similar to POPI is often not stated, with the exception of STOPP/START, in which all answers were obtained within two months [7]. We estimated that one month (a minimum of two weeks with two reminders) was a reasonable amount of time for the completion of the questionnaire. This time constraint was applied to both rounds of questions.

Our criteria contain more propositions than the STOPP/START criteria (83 propositions vs. 102 for POPI) and more than the updated 2012 Beers criteria (85 propositions). The classification of these propositions by biological system makes the POPI criteria fast to use, and POPI considers only those medical conditions that require prescriptions. The categories that we used are not the same as those in the STOPP/START criteria or the updated 2012 Beers criteria because diseases that affect children are not the same as those that affect the elderly. Indeed, in most criteria designed for use in geriatrics, psychiatry and cardiology constitute major categories [6], [7], [26], whereas the categories that contain the most propositions in POPI are respiratory problems, gastroenterology, and dermatology.

The POPI criteria have not yet been tested in the setting of routine prescriptions and needs validating clinically. Two studies will be carried out with this objective in mind. One study will examine the degree of inter-rater agreement of the various propositions of POPI, by assessing the percentage of concordance corrected for chance agreement, termed κ (Kappa). This will provide a measure of the precision of the POPI criteria. A second study will examine the capacity of the POPI criteria to identify medication errors and evaluate the safety of drug used (involved drugs, indication) prospectively.


We created the first set of criteria for the detection of inappropriate prescriptions and the omission of prescriptions in pediatrics. The resulting tool, named POPI, is available to all medical professionals (clinicians, pharmacists, in hospital or community working environment) liable to prescribe or dispense medication to children.


Thanks to our panelists: F Amouroux, R Assathiany, JP Blanc, V Breant, D Cau, L Cret, N Davoust, M Detavernier, N Duval-Ehrenfeld, A Lecoeur, F Netzer, L Priqueler, H Sarda, E Séror, B Virey, C Wehrle.

Thanks to the PMSI unit at the Robert-Debré University Hospital for the data concerning the patients’ reasons for hospitalizations.

Thanks to S Auvin, E Bourrat, A Hubert, MF Le Heuzey, C Madre.

Author Contributions

Conceived and designed the experiments: SPL TW FA OB. Performed the experiments: SPL TW FA OB. Analyzed the data: TW RB CA. Contributed reagents/materials/analysis tools: SPL TW FA RB CA OB. Contributed to the writing of the manuscript: SPL TW FA RB CA OB.


  1. 1. WHO | Medicines: rational use of medicines (n.d.). WHO. Available: Accessed 2014 May 12.
  2. 2. Choonara I (2013) Rational prescribing is important in all settings. Arch Dis Child 98: 720–720
  3. 3. Risk R, Naismith H, Burnett A, Moore SE, Cham M, et al. (2013) Rational prescribing in paediatrics in a resource-limited setting. Arch Dis Child 98: 503–509
  4. 4. Inappropriate Prescribing - MeSH - NCBI (n.d.). Available: Accessed 2012 June 18.
  5. 5. Legrain S, others (2005) Consommation médicamenteuse chez le sujet âgé. Consomm Prescr Iatrogénie Obs. Available: Accessed 2012 June 17.
  6. 6. Fick D, Semla T, Beizer J, Brandt N (2012) American Geriatrics Society Updated Beers Criteria for Potentially Inappropriate Medication Use in Older Adults - J Am Geriatrics Society - 2012.pdf. J Am Geriatr Soc 60: 616–631
  7. 7. Gallagher P, Ryan C, Kennedy J, O’Mahony D (2008) STOPP (Screening Tool of Older Person’s Prescriptions) and START (Screening Tool to Alert doctors to Right Treatment). Consensus validation. Int J Clin Pharmacol Ther 46: 72–83.
  8. 8. Gallagher P, O’Mahony D (2008) STOPP (Screening Tool of Older Persons’ potentially inappropriate Prescriptions): application to acutely ill elderly patients and comparison with Beers’ criteria. Age Ageing 37: 673–679
  9. 9. Gallagher P, O’Connor M, O’Mahony D (2011) Prevention of Potentially Inappropriate Prescribing for Elderly Patients: A Randomized Controlled Trial Using STOPP/START Criteria. Clin Pharmacol Ther 89: 845–854
  10. 10. Prot-Labarthe S, Vercheval C, Angoulvant F, Brion F, Bourdon O (2011) «POPI; pédiatrie: omissions et prescriptions inappropriées». Outil d’identification des prescriptions inappropriées chez l’enfant. Arch Pédiatrie Organe Off Sociéte Fr Pédiatrie 18: 1231–1232
  11. 11. - Affection de longue durée (ALD) (n.d.). Available: Accessed 2012 June 18.
  12. 12. Thériaque (n.d.). Available: Accessed: 2012 July 3.
  13. 13. Thomson Healthcare Products (n.d.). Available: Accessed: 2012 July 3.
  14. 14. Taketomo CK, Hodding JH, Kraus DM (2011) Pediatric & neonatal dosage handbook: a comprehensive resource for all clinicians treating pediatric and neonatal patients. Hudson, Ohio; [United States]: Lexi-Comp; American Pharmacists Association.
  15. 15. La Revue Prescrire (n.d.). Rev Prescrire. Available: Accessed Accessed: 2012 July 12.
  16. 16. Bourrée F, Michel P, Salmi LR (2008) Methodes de consensus: Revue des méthodes originales et de leurs grandes variantes utilisées en santé publique. Rev Epidémiologie Santé Publique 56.
  17. 17. Hsu CC, Sandford BA (2007) The Delphi technique: Making sense of consensus. Pract Assess Res Eval 12: 1–8.
  18. 18. SurveyMonkey (n.d.) SurveyMonkey. SurveyMonkey. Available: Accessed: 2012 Sept 12.
  19. 19. ANSM (2013) Médicaments à base de tétrazépam, d’almitrine, de ranélate de strontium et de codéine (chez l’enfant) - Retour d’information sur le PRAC - ANSM. Médicam À Base Tétrazépam Almitrine Ranélate Strontium Codéine Chez Enfant - Retour Inf Sur Pr - ANSM. Available: Accessed 2013 June 23.
  20. 20. La Revue Prescrire (2014) Poux du cuir chevelu - Diméticone, substance pédiculicide de premier choix. Rev Prescrire 34: 198–202.
  21. 21. Kaufmann CP, Tremp R, Hersberger KE, Lampert ML (2013) Inappropriate prescribing: a systematic overview of published assessment tools. Eur J Clin Pharmacol. Available: Accessed 2013 Dec 16.
  22. 22. Levy HB, Marcus EL, Christen C (2010) Beyond the beers criteria: A comparative overview of explicit criteria. Ann Pharmacother 44: 1968–1975.
  23. 23. Beers MH, Ouslander JG, Rollingher I, Reuben DB, Brooks J, et al. (1991) Explicit Criteria for Determining Inappropriate Medication Use in Nursing Home Residents. Arch Intern Med 151: 18255–32.
  24. 24. McLeod PJ, Huang AR, Tamblyn RM, Gayton DC (1997) Defining inappropriate practices in prescribing for elderly people: a national consensus panel. Can Med Assoc J 156: 385.
  25. 25. Waller JL, Maclean JR (2003) Updating the Beers Criteria for potentially inappropriate medication use in older adults. Arch Intern Med 163: 2716–2724.
  26. 26. Laroche M-L, Charmes J-P, Merle L (2007) Potentially inappropriate medications in the elderly: a French consensus panel list. Eur J Clin Pharmacol 63: 725–731
  27. 27. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, et al. (2001) Medication errors and adverse drug events in pediatric inpatients. JAMA J Am Med Assoc 285: 2114.
  28. 28. Kaushal R, Goldmann DA, Keohane CA, Abramson EL, Woolf S, et al. (2010) Medication errors in paediatric outpatients. Qual Saf Health Care 19: 1–6
  29. 29. Davis T (2010) Paediatric prescribing errors. Arch Dis Child 96: 489–491
  30. 30. Bates DW, Boyle DL, Vliet MBV, Schneider J, Leape L (1995) Relationship between medication errors and adverse drug events. J Gen Intern Med 10: 199–205.
  31. 31. Schmitt E (1999) Le risque médicamenteux nosocomial: circuit hospitalier du médicament et qualité des soins. Masson. IX–287 p.