This study aimed to generate prioritised goals for oral health services for people with disabilities as a first step in meeting the need for evidence based oral health services for people with disabilities in Ireland.
The study used a three round modified e-Delphi method, involving dental service professionals and people with disabilities or their representatives, in Ireland. Three rounds were completed online using SurveyMonkey. Round 1 asked: “List what you think dental services for people with disabilities in Ireland should be like.” Items for subsequent rounds were generated from responses to Round 1. Round 2 and Round 3 used 5 point Likert scales to rank these items by priority: from No Priority (1) to Top Priority (5). Consensus was achieved on each item where at least 80% of respondents considered an item either High or Top Priority. A consensus meeting concluded the process.
Sixty-one panelists started and 48 completed the survey. The Delphi panel agreed on level of priority for 69 items and generated 16 consensus statements. These statements covered a range of topics such as access to care, availability of information and training, quality of care, dental treatment and cost. A recurrent theme relating to the appropriateness of care to individual need arose across topics suggesting a need to match service delivery according to the individual's needs, wants and expectations rather than the disability type/diagnosis based service which predominates today.
This process produced a list of prioritised goals for dental services for people with disabilities. This creates a foundation for building evidence-based service models for people with disabilities in Ireland.
Citation: Mac Giolla Phadraig C, Nunn J, Dougall A, O'Neill E, McLoughlin J, Guerin S (2014) What Should Dental Services for People with Disabilities Be Like? Results of an Irish Delphi Panel Survey. PLoS ONE 9(11): e113393. doi:10.1371/journal.pone.0113393
Editor: Peter M. Milgrom, University of Washington, United States of America
Received: July 7, 2014; Accepted: October 23, 2014; Published: November 24, 2014
Copyright: © 2014 Mac Giolla Phadraig et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. All relevant data are within the paper and its Supporting Information files.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
For many years research in Ireland has described poorer outcomes from oral diseases and their treatment among people with disabilities –. Examples of poor outcomes include extraction rather than filling of decayed teeth, increased severity and extent of periodontal disease and a lack of functional replacement of extracted teeth when people lose some teeth or become edentulous. Some possible contributing factors are evidenced in the literature, both published and gray. These include a lack of general dental practitioners who are appropriately trained and confident to meet the needs and expectations of this growing group at primary care level ; limited access to general anaesthesia , which leads to longer waiting times and poorer outcomes ,; a reported lack of resources for dental services  and a lack of appropriately designed service models . A review of primary care dental services found inconsistent targeting of “Special Needs” groups across Ireland with great variability nationally. This report suggested a need for evidence-based service models for people with disabilities across Ireland .
Given the restrictions placed on healthcare systems, priorities must be selected as a first step to achieving these evidence-based service models. There is no simple, universally accepted approach to priority setting in healthcare, although it is advised that this process should be inclusive and transparent . In this article we report the use of a Delphi Panel composed of representatives of stakeholders to agree on priorities for dental services for people with disabilities in Ireland.
The term Delphi Method refers to the Oracle of Delphi. It is described as a structured group communication process that allows the production of information for decision-making . This allows an expert group to resolve complex problems with the goal of producing useful guidance and opinions for decision makers . The Delphi method has been used in oral healthcare research for some time and is a popular and acceptable means of answering questions of clinical, educational and policy issues . According to Hsu and Sanford the Delphi technique is often designed for the purpose of goal setting, policy investigation, or predicting the occurrence of future events, attempting to address “what could/should be” . While Jones et al. developed oral health related goals for services through a Delphi process in a residential care setting  and Prabhu and colleagues  used a similar process to develop a clinical decision making tool for adults with disabilities, the Delphi method has not been used to identify broader goals of oral health services for people with disabilities.
This study aimed to develop a prioritised set of goals for oral health services for people with disabilities in Ireland.
This study used a three round modified e-Delphi. Following the three online rounds, a consensus conference was held to finalise the process.
All participants received written information regarding their participation and provided informed consent for this study. Ethical approval was received from the Faculty of Health Sciences Research Ethics Committee of Trinity College Dublin.
File S1 contains an Excel file of summary statistics used for this analysis.
Sampling and participation.
The sampling frame was generated using convenience sampling and snowballing. Participants were primarily invited based on their recognised standing in the fields of Public Dental Services and Disability Advocacy in Ireland. Others joined via online open registration. Participants could also recommend colleagues, thus allowing an element of snowballing. This non-anonymised registration process provided a sampling frame from which a stratified, purposive sample was selected with similar proportions of “dental” and “disability” experts to ensure a suitable range of experience and expertise within the panel. Identifiers were removed prior to analysis in successive rounds. Participants were thus “quasi-anonymised”.
Inclusion and exclusion criteria.
The researchers felt that a broad inclusive panel would be of value and as such both dental and non-dental experts were included in the research process. The term expert refers to those who plan and provide dental services for people with disabilities as well as those who receive those dental services (and their advocates). To be considered an expert, prospective panel members had to fulfill one or more of the inclusion criteria listed in Table 1.
Data collection and analysis
An online secure data collection portal was used to collect responses in all three online rounds (SurveyMonkey). All responses from Round 1 to Round 3 were anonymised. A consensus conference concluded the data collection process. Details on each stage follow.
We asked all panelists: “List what you think dental services for people with disabilities in Ireland should be like.” Items for subsequent rounds were generated from responses to Round 1. Three researchers (CM, AD, SG) then reduced these items using pre-defined principles: firstly by combining duplicate responses, secondly by coalescing responses that had the same meaning, but were expressed differently and lastly by combining low frequency items into broader topics, which encompassed multiple related items, while ensuring that their meaning was not lost.
In Round 2 panelists used 5 point Likert scales to rank emergent items by priority: “Rank (the following) statements according to the level of priority that you think each should hold for dental services for people with disabilities, using the following 5 point scale”. The options included 1.No Priority; 2.Low Priority; 3.Not Sure; 4.High Priority and 5.Top Priority. The ranked mean was used to rank items by relative priority. Consensus was accepted when more than 80% of respondents gave either Top Priority or High Priority responses for an item. In order to permit continuing consideration of all items emerging from Round 1, items which failed to reach this consensus level were noted by the researchers but not removed from the materials given to panel participants until after the final consensus meeting.
Statistics for all items including the median rank and level of consensus from Round 2 were fed back in a controlled manner at Round 3. Where little consensus existed on items at this stage, qualitative responses were also fed back to allow participants to consider both group statistics and qualitative responses. Panelists then rescored each item using the same Likert scale. If they felt that their score was outside of the group consensus they were encouraged to include a reason for their continuing disagreement in their Round 3 response.
A subgroup analysis was undertaken for results to each item from Round 3 based on gender, urban/rural location and expert role (dental/disability). Association between category and distribution of scores on each item was tested using Pearson's chi-square statistic. Given the number of tests for each variable (n = 83), level of significance was reduced (p<0.025), as per the rough false discovery rate .
All participants were invited to a consensus conference. Two researchers (CM and AD) independently sorted all items arising from the e-Delphi into topics, prior to the consensus conference . Next, a single statement was generated for each topic, where the meaning of constituent items were in agreement. For example, the following Consensus Statement was generated from the grouping of two items (A–B).
Consensus Statement: Oral health services should raise awareness of oral health among people with disabilities, their families, carers and non-dental health professionals (see items A and B below)
Constituent Delphi items: A. People with disabilities and their carers should be aware of the importance of oral health
B. All relevant professionals should be aware of the importance of oral health for people with disabilities
Where constituent items within a topic were contradictory or not concordant Questions were generated instead to encourage discussion, which ultimately led to consensus and reframing as statements. For example, the following question was generated from the grouping of two items (C–F):
Consensus Question: How often should people (with disabilities) access dental care? (see items C–F below)
Constituent Delphi items: C. Services should be structured to enable appropriate follow up
D. Services should be structured to enable regular screening and review
E. Services should be structured to enable yearly review
F. Services should be structured to enable twice yearly review
Statements and questions such as these were discussed at the consensus conference, leading to the production of 16 Consensus Statements as an output of this process. These were ranked by priority according to mean priority ranking of constituent items from Round 3 of the Delphi process. The number of constituent items and their mean level of agreement were also reported.
Participant flow and sample
Figure 1 demonstrates the flow of participants in this study. Six people included in the sampling frame were purposively excluded: Three of these were felt not to have sufficient experience regarding dental services for people with disabilities and therefore did not meet inclusion criteria, and three others were dental professionals who were removed to reduce an over-representation of people in the category of General Dentist (mainly public) within the panel. Eleven panelists did not return a completed Round 1 questionnaire and were therefore eliminated from further rounds. A further three panelists, invited onto Round 2 did not respond to this survey and were also removed from further analysis.
Legend: N = sample completing round; % = % participants having completed the prior round, to complete the following round.
This resulted in 48 panelists (36 females and 12 males) who completed the survey: twenty-five participants (52.1%) were dental professionals and twenty-three (47.9%) represented people with disabilities or their advocates (Table 2). Those representing dental services consisted of frontline service providers such as dental nurses, dental hygienists, private and public dentists, who often see a high number of people with disabilities, Public Dental Service managers/policy makers and specialist dental service providers such as dental public heath, paediatric and special care dentists.
The group representing people with disabilities included five people with disabilities and four parents of people with disabilities. Of the fourteen disability professionals remaining, seven were disability service providers, such as care workers and allied health professionals, four were advocates for people with disabilities (three professional and one voluntary), and two were policy makers and one a disability academic.
The panel represented a broad range of ages. Dental services panelists tended to be younger (median = 30–39 years) than those representing people with disabilities (median = 50–59 years; p>0.01), but the groups were otherwise similar. Geographically, participants from all four provinces of Ireland were included, although most were from the eastern seaboard (Leinster: n = 33, 68.8%). Most participants were from urban settings (n = 37, 77.1%). The panel was associated with a broad range of disability types (Figure 2).
27 (23 disability respondents and 4 dental) respondents reported a median of two disability types (IQR = 5) with which they were associated. Multiple categories allowed.
Data Collection and analysis
A total of 351 open-ended responses were returned from 51 respondents. Following the process described above, 83 distinct items were sent back to the group for ranking in Round 2.
Ninety four percent of experts completing Round 1 completed Round 2 (Figure 1). Consensus was achieved on many items at this stage: only 22 items had agreement at less than 80%. No items were considered Low or No Priority by the group, after Round 2. Forty-eight items had a median score of 5 (Top Priority), 33 a median score of 4 (High Priority) and two had a median score of 3 (Unsure). Initial rankings are presented in Tables 3 and 4.
All those who completed Round 2 also completed Round 3 (Figure 1). The presence of tied ranking meant that the 83 items were ranked in 43 places. Thirty one items had 100% agreement, while 11 failed to achieve consensus at the 80% level. No items were considered Low or No Priority. Forty-eight items had a median score of 5 (Top Priority), 33 a median score of 4 (High Priority) and two had a median score of 3 (Unsure). The ten highest and lowest ranked items following Round 3 are presented in Tables 3 and 4. These tables present the ranking of specific items at Round 2 and 3 as well as the level of agreement at each stage. This gives a measure of consistency over rounds for the group as a whole. Reasons for disagreement between panelists are summarized in Table 4.
Expert role and urban/rural location did not have a statistically significant association with scores on any item. The proportion of top priority scores was higher (p<0.025) among females on a number of items (8.5); (7.1); (13.3) and (11.5) (see Table S1 which gives further detail on all retained items and how they relate to the consensus statements.). Given the low number of participants contingency tables for the first three items listed above had more than two expected cell counts lower than 5 which limits our interpretation of these tests.
All 83 items generated by the Delphi panel were grouped by topic as in the process described above, generating 18 statements/questions. These were reviewed by 19 panelists (7 disability and 12 dental experts) at a face-to-face conference (including 6 members of the panel who contributed remotely using online conferencing software). Following this meeting, the total number of items reduced from 83 to 69, as 11 items that had failed to reach level of consensus (80%) were now removed and 3 other items were coalesced by the consensus group. This process also reduced the number of statements/questions from 18 to 16 Consensus Statements as two statements were coalesced and one statement failed to reach the 80% cut off agreed for consensus. Table 5 summarises the final Consensus Statements generated by this round, along with their final ranking, based on mean ranking of constituent items. Included are the number of items included within each statement and their mean level of agreement.
Structured engagement with service users is important when setting priorities for health services . This article reports a novel means of identifying priorities using structured engagement – in this case with a single broadly constituted sample. These priorities were generated in response to an identified need for evidence-based service models . The study met its aim by generating a list of 16 consensus statements regarding oral health services for people with disabilities, as well as a complementary ranked list of 69 priority items.
The topics, which arose in this study, were diverse encompassing access to care, quality of care, dental treatment, information and cost. Many items were ranked as top or high priority by the Delphi Panel. This must be kept in mind when considering the ranking outcome. There was very little difference in mean rank between items or statements. Although their final ranking may suggest an ordinal relationship, often the difference between ranked items was minimal. Meaningful ranking is therefore difficult, as previously reported . Nevertheless, it was perhaps surprising that the highest priority statement to arise from this process related to health awareness. This highlights the importance of “thinking outside of the box” when considering goals for dental services, especially where resources may limit potential for achieving alternative priorities.
Many items mirrored a multidimensional model of access to care as described by Penchansky , though an added dimension relating to appropiateness of care to individual need arose. Previous research found this additional dimension specific to dental services for people with disabilities ,. Many statements focused on meeting individual need rather than focusing on diagnosis. This element of care delivery recognised the importance of focusing on the individual, their expectations, desires and needs when accessing and receiving care and prevention, rather than a disability or disease specific focus. This is in line with the World Health Orgnaisation's International Classification of Functioning, Disability and Health (ICF) model of disability  which is increasingly applied to understand, measure and improve awareness of oral health for people with disabilities ,,. This focus was evident across Delphi panel statements suggesting that individual need should drive the individual's frequency of contact with oral health services, the choice of healthcare setting, the availability of care pathways, and types of service available and accessible (such as general or specialist, community or hospital). Individual need should also decide optimal outcomes from interactions with oral health services for the individual.
Traditional Delphi approaches emphasise the importance of group interaction on the development of consensus, yet this study relied mainly on interaction online. An approach to address the potential lack of group interaction is to incorporate formal feedback within the e-Delphi Process . The authors were also conscious to avoid undue coercion for participants to cede to group pressure. Therefore responses were anonymous to other participants with a focus on statistical group responses. Controlled feedback was also provided at each stage . In Round 2 participants received the list of items grouped by topic for their ranking exercise and in Round 3, feedback was provided to participants as the median level of priority and level of consensus for each item following Round 2. This allowed participants to consider the group scoring before final ranking without undue pressure to conform . Participants could also provide a reason for disagreeing with the group consensus at this stage if they wished.
Despite the broad representation on this panel, we should not misinterpret the panel as being “representative” in the true sense. Sampling was non-probabilistic and therefore we were not surprised that the group does not reflect the natural occurrence of particpants seen in society. For example this panel contained twice as many public dentists than private practitioners while in reality approximately 80% of Irish Dentists are based in private practice. This has implications for how we intepret the results.
The concept of disability used in this process was broad. This was considered favourable to accommodate a broad range of views across perpectives rather than the views of a specific disability subgroup. An outcome of this inclusive approach is that the general concepts which were prioritised by this group may be different to the specifc priorities of groups of people with specific disabilities (e.g. people with communication disabilities compared to people with physical disabilities or bleeding disorders). Therefore the results of this Delphi panel could be used as a starting point for further validation with specific groups, where indicated.
Despite reasonable accommodation, the nature of this study meant that some people with communication and or cognitive difficulties could not be included. Additional processes were initiated for those with communication or cognitive issues and these are reported elsewhere. People with visual impairment were included using Braille documents with appropriate formatting of questionnaires and associated documentation.
The retention rate in this suvey was good at each stage. Most losses occurred in the initial round where 17.7% of those selected did not respond. Only a further 5.8% did not complete the following two rounds. This is a pleasing result as retention over rounds is a recognised problem in Delphi Panel research. Importantly, this means that no participant dropped out of later rounds due to disagreement with the group consensus. It is felt that the use of email, and online data collection was a large part of this success. Additionally, the short timeframe between rounds was also felt to benefit the response rate. One challenge the team encountered was the need to generate Braille versions of questionnaires. Given the quick turnaround, this addition challenged both researchers and respondents, but doubtlessly enriched the data collected.
Delphi panel results are often improved by including diverse perspectives on a problem . Therefore this study included experts from a broad range of backgrounds, most importantly, people with disabilities and their advocates. This is in acknowledgement of their “expertise by virtue of having experienced the impact of a condition or intervention” (, p. 379). A larger panel is obviously needed to incorporate sufficient numbers of people with these diverse opinions, which are known to increase the number of items generated and also reliability . In the absence of clearly accepted guidance on panel sizes , this study aimed to include 60 panelists, giving equal representation of disability and dental experts. Forty eight panellists completed all rounds, giving equal representation of disability and dental experts. Interestingly, the subgroup analysis did not find any statistically significant difference between dentists and disability experts. Only gender showed significant difference on a small number of items, the reason for which is unclear.
The consensus conference led to the development of common topics among clustered items, a useful technique, to make it conceptually easier for participants (who were at this stage also knowledge users) to consider large sets of items concisely. This clustering by topic has previously been used for similar purposes . The consensus conference enabled the group to clarify and agree meaning within statements and items. The research team felt that this step was beneficial. While it was well attended, a self-selecting subgroup of the Delphi Panel attended this step and this may have influenced the resultant output. Online attendance was evidently favoured by some participants who could not attend, thus allowing broader attendance.
This study reveals a clear focus on placing the indvidual at the centre of services. The clear, recurrent emphasis on delivering services according to individual needs and expectations rather than based on diagnosis is poignant. This means that services should be planned and delivered to meet the needs of the individual with disabilities in an acceptable manner. This may present challenges to those who plan and deliver services. Further research is needed to conceptually understand individual need among this group, and operationally, how this is presently and can be met.
We have used the outcomes of this Delphi to produce research tools for application with people with specific disability types to help understand their health care priorities and adapt the results of the current study. These results will be reported seperately. The research team will use the results of these initial studies to develop local priorities for people with disabilities and oral health services in Ireland. This will hopefully lead to theory development upon which to design service models.
Previous research has questioned whether priorities produced by public opinion surveys are relevant or useful for decision-makers  and the results of this Delphi survey may similarly be questioned, especially if one attempts to infer from an Irish context to other areas. While poor oral health among people with disabilities appears to be a global phenomenon ,, it is unlikely that the priorities for dental services will be the same internationally. Similar processes could be undertaken in other countries within the policy, practice and cultural norms of other societies.
Lastly, it is hoped that policy makers and commissioners can also learn from our experiences when planning services for this group of patients. The intention is that policy makers can consider the consensus statement of interest with the option of mining the constituent items that may give guidance as to the goals of the services which they may commission. A measure of agreement and priority will enrich the utility of these data. A full list of items is available in Table S1.
This study places a clear focus on the indvidual at the centre of oral health services. Services should be planned and delivered to meet the needs of the indivdual with disabilities in a way that is accpetable to that person. The study represents a first step in the development of future policy, service development and research in this area of public health and special care dentistry. As the items and statements summarised in this article represent a set of priorities as agreed by one panel of experts, there is a need to incorporate other opinions to truly achieve consensus on the question: What should dental services for people with disabilities be like? As such, the research team has undertaken qualitative work with specific groups of people with disabilities to develop broader consensus using the outcomes described here as a foundation. The research team aim to ultimately use this process to develop interventions for evaluation in Ireland. The pursuit of these goals will surely engage those who wish to rise to this challenge.
Project SMILE Ireland Delphi Panel consensus statements and constituent items. Consensus statements are presented in order of priority along with mean rank and level of agreement. Constituent items are similarly presented with rank and level of agreement and agreed priority. Statements relate specifically to oral health services for people with disabilities. This is implied in most statements to reduce burden except where this phrase is needed for clarity. The statement: Services should be accessible locally (initial ranking 8, n = 3, mean rank of included items = 17, mean level of agreement = 94%) was removed during this consensus meeting and amalgamated with the Statement 2 as contributors felt that this represented needless repetition; The statement Novel funding models of oral health service for people with disabilities should be examined (initial ranking 17, n = 3, mean rank of included items = 41, mean level of agreement = 52%) was removed at the end of the consensus conference as all constituent items failed to achieve the agreed level of consensus of 80%.
Excel file of summary statistics used for this analysis.
The research team would like to thank all those who agreed to take part in this research programme including: Angela Connolly, RDN; Agnes Mooney, Epilepsy Ireland; Cecilia Mc Cooey, HSE Disability Services; Brega Howley, Dentistry for Kids; Maria Caples, UCC; Marion Meany, Social Care Division, HSE; Fedelma O'Farrell, Parent; Liz Ellard, Childrens University Hospital, Temple St; Siobhan Doran, Longford Dental Centre; Ruth Daisley Martin, RDN and mother of autistic child; Yvonne Rooney, King's College NHS Foundation Trust; Carolyn Akintola, Self advocate; Anne Lawlor, 22q11 Ireland Support Group; Joe Mooney, Muscular Dystrophy Ireland; Sharon Mahon, HSE Dental Services; Jim Sweeney, St. Joseph's Intellectual Disability Service; John Ryan, Kare; Jacqui Browne, Self advocate; Ann Kennedy, Self advocate; Aidan Larkin, Multiple Sclerosis Ireland; Donie O'Shea, National Disability Association; Ceara Cleary, HSE Dental Services; David Suitor, Creative Smiles; Sarah Cronin, Ace Communication Ireland; Jill Smith, The Dental Suite. We also acknowledge the Scientific Advisory Group: Prof Ruth Freeman, University of Dundee; Prof Colman McGrath, University of Hong Kong; Prof Jimmy Steele, Newcastle University.
Conceived and designed the experiments: CMGP JN JM AD EON. Performed the experiments: CMGP. Analyzed the data: CMGP AD SG. Wrote the paper: CMGP JN AD JM SG EON.
- 1. Disability Federation Ireland (2012) Disability in Ireland: Some Facts and Figures. Disability Federation Ireland. Available: http://www.disability-federation.ie/userfiles/file/Disability%20FactSheet.pdf. Accessed 2014 Feb 20.
- 2. National Disability Authority (2006) Proportion of disabled people set to rise as population ages. Available: http://www.nda.ie/cntmgmtnew.nsf/0/06881F423C2416948025710D0047E5EF?OpenDocument. Accessed 2014 Feb 26.
- 3. Crowley E, Whelton H, Murphy A, Kelleher V, Cronin M, et al.. (2005) Oral Health of Adults with an Intellectual Disability in Residential Care in Ireland 2003. Department of Health and Children, Galway.
- 4. Costello PJ (1990) Dental health status of mentally and physically handicapped children and adults in the Galway Community Care Area of the Western Health Board. J Ir Dent Assoc 36:99–101.
- 5. Holland TJ, O'Mullane DM (1990) The organisation of dental care for groups of mentally handicapped persons. Community Dent Health 7:285–293.
- 6. Elliot I, Nunn J, Sadlier D (2005) Oral Health & Disability: The way forward. National Disability Authority, Dublin.
- 7. Mac Giolla Phadraig C, McCallion P, Cleary E, McGlinchey E, Burke E, et al.. (2014) Total tooth loss and complete denture use in older adults with intellectual disabilities in Ireland. J Public Health Dent. doi: 10.1111/jphd.12077
- 8. Smith G, Rooney S, Nunn J (2010) Provision of dental care for special care patients: the view of Irish dentists in the Republic of Ireland. J Ir Dent Assoc. 56:80–84.
- 9. Irish Society of Disability and Oral Health (2012) Access To Special Care General Anaesthetic Facilities: Results Of A National Informal Information Gathering Exercise. ISDH, Dublin.
- 10. Holland TJ, Lucey S, Kavanagh C, O'Mullane DM (1997) Costs in providing facilities for treatment of handicapped patients under general anaesthesia. J Ir Dent Assoc 43:72–75.
- 11. McGeown D, Mac Giolla Phadraig C, Nunn J (2012) The effects of waiting lists on oral health changes for a special care population. Does being placed on a waiting list lead to harm for Special Care Patients? J Disab Oral Health 13 (3); 101. Available: http://www.shancocksltd.co.uk/cart/download.php?op=view_article&article_id=1132. Accessed 2014 Oct 20.
- 12. PA Consulting Group (2010) Strategic Review of the Delivery and Management of HSE Dental Services. Dublin: Health Services Executive.
- 13. Holm S (1998) The second phase of priority setting. Goodbye to the simple solutions: the second phase of priority setting in health care. BMJ 317:1000–1002. doi: 10.1136/bmj.317.7164.1000
- 14. Turoff M, Hiltz SR (1996) Computer based Delphi process. In Adler M, Ziglio E (1996) Gazing into the oracle: the Delphi method and its application to social policy and public health. (pp. 56–88). London: Jessica Kinsley Publishers Ltd.
- 15. Cramer CK, Klasser GD, Epstein JB, Sheps SB (2008) The Delphi process in dental research. J Evid Based Dent Pract 8:211–220. doi: 10.1016/j.jebdp.2008.09.002
- 16. Hsu CC, Sandford B (2007) The Delphi Technique: Making Sense Of Consensus Practical Assessment, Research & Evaluation. 12:1–8 Available: www.pareonline.net/pdf/v12n10.pdf Accessed 2014 Jan 04..
- 17. Jones JA, Brown EJ, Volicer L (2000) Target outcomes for long-term oral health care in dementia: a Delphi approach. J Public Health Dent 60:330–334. doi: 10.1111/j.1752-7325.2000.tb03344.x
- 18. Prabhu NT, Nunn JH, Evans DJ, Girdler NM (2008) Development of a screening tool to assess the suitability of people with a disability for oral care under sedation or general anesthesia. Spec Care Dentist 28:145–158. doi: 10.1111/j.1754-4505.2008.00034.x
- 19. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29(4):1165–188. doi: 10.1214/aos/1013699998
- 20. Efstathiou N, Coll AM, Ameen J, Daly W (2011) Do Greek healthcare users and healthcare providers share cancer care priorities? Analysing the results from two Delphi studies. Eur J Cancer Care 20:179–186. doi: 10.1111/j.1365-2354.2009.01176.x
- 21. Penchansky R, Thomas JW (1981) The concept of access: definition and relationship to consumer satisfaction. Medical Care 19:127–140. doi: 10.1097/00005650-198102000-00001
- 22. Owens J, Dyer TA, Mistry K (2010) People with learning disabilities and specialist services. Br Dent J 208:203–205. doi: 10.1038/sj.bdj.2010.204
- 23. Sagheri D, McLoughlin J, Nunn JH (2013) Dental caries experience and barriers to care in young children with disabilities in Ireland. Quintessence Int 44:159–169.
- 24. World Health Organisation (2001) International Classification of Functioning, Disability and Health ICF. Geneva: WHO.
- 25. Faulks D, Norderyd J, Molina G, Mac Giolla Phadraig C, Scagnet G, et al. (2013) Using the International Classification of Functioning, Disability and Health (ICF) to describe children referred to special care or paediatric dental services. PloS one 8:e61993. doi: 10.1371/journal.pone.0061993
- 26. Faulks D, Freedman L, Thompson S, Sagheri D, Dougall A (2012) The value of education in special care dentistry as a means of reducing inequalities in oral health. European journal of dental education: official journal of the Association for Dental Education in Europe 16:195–201. doi: 10.1111/j.1600-0579.2012.00736.x
- 27. Dougall A, Pani SC, Thompson S, Faulks D, Romer M, et al. (2013) Developing an undergraduate curriculum in Special Care Dentistry - by consensus. Eur J Dent Educ 17:46–56. doi: 10.1111/eje.12011
- 28. Dalkey NC (1969) The Delphi Method: An experimental study of group opinion Santa Monica, CA1969. Available: http://www.rand.org/content/dam/rand/pubs/research_memoranda/2005/RM5888.pdf. Accessed 2014 Sep 25.
- 29. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41:376–382. doi: 10.1046/j.1365-2648.2003.02537.x
- 30. Ludwig B (1997) Predicting the future: Have you considered using the Delphi methodology? Journal of Extension (On-line) 35: 5; Tools of the Trade: TOT2. Available: http://www.joe.org/joe/1997october/tt2.php. Accessed 2014 Jan 14.
- 31. Keeney S, Hasson F, McKenna H (2006) Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 53:205–212. doi: 10.1111/j.1365-2648.2006.03716.x
- 32. Kaplan G, Baron-Epel O (2013) The public's priorities in health services. Health Expect. DOI: 10.1111/hex.12064 Available: http://onlinelibrary.wiley.com/doi/10.1111/hex.12064/full. Accessed 2014 Feb 21.
- 33. Anders PL, Davis EL (2010) Oral health of patients with intellectual disabilities: a systematic review. Spec Care Dentist 30:110–117. doi: 10.1111/j.1754-4505.2010.00136.x
- 34. Glassman P, Miller C (2003) Dental disease prevention and people with special needs. J Calif Dent Assoc 31:149–160.