A Quantitative Approach to the Prioritization of Zoonotic Diseases in North America: A Health Professionals’ Perspective

Background Currently, zoonoses account for 58% to 61% of all communicable diseases causing illness in humans globally and up to 75% of emerging human pathogens. Although the impact of zoonoses on animal health and public health in North America is significant, there has been no published research involving health professionals on the prioritization of zoonoses in this region. Methodology/Principal Findings We used conjoint analysis (CA), a well-established quantitative method in market research, to identify the relative importance of 21 key characteristics of zoonotic diseases for their prioritization in Canada and the US. Relative importance weights from the CA were used to develop a point-scoring system to derive a recommended list of zoonoses for prioritization in Canada and the US. Study participants with a background in epidemiology, public health, medical sciences, veterinary sciences and infectious disease research were recruited to complete the online survey (707 from Canada and 764 from the US). Hierarchical Bayes models were fitted to the survey data to derive CA-weighted scores for disease criteria. Scores were applied to 62 zoonotic diseases to rank diseases in order of priority. Conclusions/Significance We present the first zoonoses prioritization exercise involving health professionals in North America. Our previous study indicated individuals with no prior knowledge in infectious diseases were capable of producing meaningful results with acceptable model fits (79.4%). This study suggests health professionals with some knowledge in infectious diseases were capable of producing meaningful results with better-fitted models than the general public (83.7% and 84.2%). Despite more similarities in demographics and model fit between the combined public and combined professional groups, there was more uniformity across priority lists between the Canadian public and Canadian professionals and between the US public and US professionals. Our study suggests that CA can be used as a potential tool for the prioritization of zoonoses.


Introduction
Zoonotic diseases are diseases that are naturally transmitted between vertebrate animals and humans [1,2]. Currently, it is estimated that zoonoses account for 58% to 61% of all communicable diseases causing illness in humans globally [3,4] and up to 75% of emerging human pathogens [3][4][5]. Although zoonotic outbreaks are a significant burden of disease in North America, for example, outbreaks of West Nile virus, SARS, H1N1 influenza and Lyme disease in the past decade [6][7][8][9], there are limited resources available for their control and prevention making it necessary to prioritize diseases in order to allocate resources to those with the greatest impact. While there is consensus on the need to prioritize zoonoses, there are numerous challenges to the process. These include, the difficulty in comparing the overall public health impact of zoonoses when they vary greatly in incidence, clinical manifestations, control measures, transmission potential and socio-economic impact in humans and animals [10,11]; the multiple stakeholders involved each with their own prioritization objectives and beliefs [12]; and the lack of agreement in prioritization methodologies [13][14][15]. These challenges limit the ability to establish a universally accepted priority list for zoonoses. Nonetheless, a number of studies have attempted to prioritize communicable diseases [13,[16][17][18][19][20][21] and more recently, zoonotic diseases [22][23][24][25][26], though the studies on zoonotic diseases have all been conducted in Europe.
Progress has been made towards the use of quantitative approaches to address the complexity of disease prioritization and to overcome constraints in traditional methods [13,14,21,24,25]. Although methodological approaches differ, prioritization typically follows a series of steps: (1) selecting a group of diseases/pathogens to prioritize; (2) identifying a list of measurable criteria to assess diseases/pathogens; (3) defining a range of levels for each criterion; (4) determining the relative importance for each level by assigning a weight or score; (5) assigning weights and/or scores by matching the level of each disease criterion to the select group of diseases/pathogens; (6) aggregating weights/scores to produce an overall score for each disease/pathogen; and (7) ranking diseases/pathogens by their overall score to derive a priority list. Current prioritization methods, including recent quantitative approaches, are limited by the requirement to produce arbitrary scores and subjective weights for disease criteria and their levels (step 4), these are typically derived from a simplified linear point-scoring system without weights [16][17][18][19][20] or by an expert Delphi panel who assign weights that are applied to a linear point-scoring system [13,14,21,24,26]. This self-explicated approach to deriving scores and weights can introduce subjective bias into the prioritization exercise. Further, the matrix approach to assigning scores and weights to each disease criterion separately makes the assumption that disease criteria are independent.
A novel quantitative approach to overcoming these specific limitations is Conjoint Analysis (CA). CA is a market research technique used in exploring consumer preferences [27]. It is gaining recognition in the last decade for its use in eliciting preferences in the healthcare setting [28][29][30][31]. The principle behind CA is that a product (goods or service) can be described by a set of characteristics and the extent to which an individual values a product is determined by the level of each of those characteristics and the combination of those characteristics together [28,29,32]. A CA study presents individuals with competing products containing both desirable and undesirable characteristics and forces the individual to state a preference, usually as a choice between products. In doing so, individuals make a trade-off between the desirable and undesirable characteristics in the products through their choices revealing the true value of each characteristic relative to each other. This preference elicitation method overcomes the need to assign arbitrary scores and subjective weights as relative weighted scores for each characteristic and their levels are derived from the choice data. Additionally, this approach forces individuals to consider multiple characteristics together; thus, criteria are not assumed to be independent from each other.
In the context of zoonoses, a disease can be treated as a product described by a set of disease criteria (characteristics), and the value of the disease can be determined by the level of each criteria and the combination of those criteria. While similar methods to CA such as Maximum Difference Scaling (MaxDiff) are available [32], zoonotic diseases are often complex requiring an understanding of preferences under a range of multiple characteristics and levels, thus the use of CA, that allows for the exploration of interrelationships across numerous characteristics and levels, was considered a more appropriate tool. An additional benefit in using CA in disease prioritization is that by presenting zoonoses as a set of disease characteristics without identifying diseases, individuals are forced to prioritize based on science, eliminating potential biases associated with disease names. These can include biases arising from prioritizing diseases on the basis of professional awareness and/or personal gain, or from the potential fear of a disease name compared to a lesser-known disease. Finally, by presenting respondents with all the information to prioritize diseases, CA allows for wide social participation; this can include experts who may not be familiar with the full range of diseases. CA is similar to the methodology used in a recent study on the prioritization of emerging zoonoses in The Netherlands [25], although a different mathematical approach was used.
Previously, we presented on the novel use of CA to develop a point scoring system for disease criteria considered important in determining priority amongst individuals from the general public [33]. The current study describes the results of individuals identified as 'health professionals' -individuals with a background in medical and veterinary sciences, public health, epidemiology and infectious diseases. This paper will also compare the results between the public groups and the professional groups. The primary objective of this study is to present on the methodological approach used to prioritize zoonoses in North America, a secondary objective is to define the most important zoonoses in North America as identified using this approach.

Study Participants
The Research Ethics Board at the University of Guelph approved all aspects of this study. The target study participants were: epidemiologists, public health practitioners and policymakers in the human and animal health disciplines at the local, provincial/state and national level, academic and practicing physicians and veterinarians, infectious disease researchers, human and animal health laboratory microbiologists, pathologists and technicians and registered nurses. Thus, the study group included individuals in relevant professional disciplines that would provide some prior knowledge of infectious diseases.
Participants from both Canada and the US were recruited through email invitation. Email searches were conducted for academic and practicing physicians and veterinarians and public health representatives and policymakers at the local, provincial/ state and federal level. Direct email invitations were endorsed and sent by professional associations. Web advertisements were placed in relevant publications and some individuals were recruited in person at provincial and national conferences (see S1 for full listing). An additional 125 Canadian physicians were recruited online using a healthcare panel through Research Now TM [34]; these were groups of pre-screened Canadian physicians who had expressed a willingness to participate in online surveys. Reminder emails were sent a week after the initial email invitation and three weeks after the initial email invitation. Recruitment commenced in November 2010 and was completed in January 2012. Sample size calculations were made using Sawtooth Software SSI Web v7 [35]; a minimum of 500 professionals per country was needed for this study.
Surveys were collected online and anonymously. All participants acknowledged an informed consent assuring confidentiality and the option to withdraw from participation without penalty. Sawtooth Software SSI Web v7 [35] was used to screen participants through a series of demographic questions prior to survey commencement. Participants were disqualified if they did not reside in North America or were not employed in one of the following fields: epidemiology, public health, medical sciences, veterinary sciences, infectious disease research, laboratory technician or nursing. emerging zoonotic and enteric diseases was selected for the study on the basis of being nationally or internationally notifiable or identified as a priority [33]. Diseases exhibiting multiple forms (for example, acute/chronic, latent/active) were divided into separate syndromes and approximate proportions were assigned to each syndrome as informed by the literature. There were 117 separate disease syndromes identified from the 62 diseases. A literature search for each criterion for each disease syndrome was conducted, searches included websites of reputable human and animal health organizations, reference textbooks and PubMed catalogued peer-reviewed publications [33]. Criterion levels were defined according to the range exhibited in the literature with three or four levels assigned to each criterion.

Survey Instrument
Due to the large number of criteria, a partial-profile choicebased conjoint (CBC) survey was developed comprising 14 choice tasks [36,37]; each choice task presented participants with five disease combinations containing varying levels of 5 of the 21 criteria using an orthogonal experimental design [33]. Participants were asked to select one zoonosis to prioritize for their control and prevention in either Canada or the US (Figure 1). Definitions for the technical terms 'case-fatality' (proportion of deaths over the total number of cases) and 'disease incidence' (number of new cases of diseases over the last five years in the population) were provided to ensure study participants understood the presented disease criteria. Disease criteria and levels varied between choice tasks and the ordering of the presentation of disease criteria within each choice task was randomized to reduce ordering bias. Two additional fixed choice tasks were included to test the reliability of responses by identifying respondents who did not understand the choice task process and/or fatigue responders. Fixed choice task 1 presented one zoonosis with the highest incidence in humans (10,000 cases), most severe illness in humans (severe clinical symptoms), highest transmission potential between humans (high), highest case-fatality in humans (80%) and the most costly economic burden in humans ($10,000 per sick individual). In comparison, the remaining four zoonoses contained a combination of lower and less severe criteria levels. Fixed choice task 2 presented one zoonosis with the most severe illness in animals (severe clinical symptoms), highest case-fatality in animals (80%), most costly socioeconomic burden in trade in animals (high cost such as culling of herds or destroying infected crops/produce), longest duration of illness in animals (chronic illness or permanent deficits) and rapid change in disease trend in the human population (new emerging disease, rapid increase over the last five years). In comparison, the remaining four zoonoses contained a combination of lower and less severe criteria levels. The fixed choice tasks were also randomized to reduce ordering bias and as an additional measure of reliability. Sawtooth Software CBC module v7 [38] was used to create 300 survey versions, each version presented an efficient experiment design using a balance overlap approach with balanced levels across choice tasks and an orthogonal design [39]. The D-efficiency of the experimental design was 908.13326 relative to a full-orthogonal design with a standard error of ,0.05 for each criterion level. Sawtooth Software SSI Web v7 [35] was used to randomly assign a survey version to each study participant. Surveys were offered in English, French and Spanish.

Data Analysis
Hierarchical Bayes (HB) was used to compute parameter estimates (weighted scores) from the CA survey choice data [40]. Sawtooth Software CBC/HB v5.2.8 [2] was used to estimate individual-level parameter estimates (b). The program combined Bayes theorem with a Monte Carlo Markov Chain (MCMC) procedure and the Metropolis/Hasting algorithm to iteratively update parameter estimates drawn from an upper-level model (prior) and a lower-level model (posterior) [40]. The HB algorithm estimates the average parameter estimates for the entire population (prior) and then uses the respondent's individual data (posterior) to determine how each respondent differed from the population mean. The algorithm will then adjust each respondent's parameter estimates so that they reflect an optimal mix of the individual respondent choices and the population mean. The optimal mix is determined by the amount of data provided by each respondent (posterior) and the amount of variance in the population mean (prior). The greater the prior variance (the distribution or differences amongst the population), the less Bayesian shrinkage is applied to the mean and the more individuals are allowed to vary such that their parameter estimates provide better individual-level fit to the individual-level responses) [41]. A total of 30,000 preliminary iterations were computed before convergence was observed (graphically and on observing stable goodness-of-fit measures including root-likelihood and variance) and an additional 30,000 iterations were computed per respondent to estimate final parameters. Final parameter estimates (b) are presented as zero-centered standardized utility values by setting the average range of the parameter values of all disease criteria to 100.
Part-worth utility values (b) represent the relative influence each criterion level had on respondent choices with higher values indicating a stronger influence on choice [32]. As part-worth utility values were calculated from choice task sets presenting disease combinations containing varying levels of multiple disease criteria, interaction effects between disease criteria and levels were accounted for in the calculation of part-worth utility values. Twenty-one part-worth utility values, one for each disease criterion, were assigned to the 117 separate disease syndromes by matching the level of each disease criterion to those of disease syndromes [33]. Part-worth utility values were summed up in proportion to the relative frequency of each syndrome within a disease to derive an overall score for each of the 62 diseases. The overall scores were used to rank-order diseases; the higher the score, the higher the ranking on the priority list. As part-worth utility values are interval data, the overall scores cannot be directly compared both within and between countries [32]. Instead, we compare the difference in disease ranking as an overall measure of proximity of diseases both within and between countries. The summed part-worth values approach was used instead of market simulations to apply CA-derived scores to a set of diseases to explore a method comparable to current traditional prioritization methods [14,[19][20][21][24][25][26].
To estimate the influence of each criterion collectively, importance scores were calculated, for each respondent, as a percentage by dividing the difference in range between the highest and lowest level part-worth utility values by the sum of all partworth utility value ranges across all criteria, and then averaging those importance scores across respondents. The larger the difference between the levels in a criterion, the higher the importance score and the stronger the influence the criterion had on the decision to prioritize [32]. Individual-level part-worth utility values and importance scores were calculated directly using Sawtooth Software SMRT v4.22.0 [42]. The standard error across individual part-worth utility values was used to quantify uncertainty in the part-worth utility values. T-statistics were derived by dividing the mean difference in range in part-worth utility values across each criterion by the standard error of the differences to test each disease criterion for statistical significance in the final model. Overall scores for diseases were calculated as the summation of the mean (population-level) part-worth utility values across the 21 disease criteria. As the overall scores were derived from population-level utility values, uncertainty in the overall scores were quantified by incorporating the standard error across individual part-worth utility values and calculated as the summation of the upper and lower confidence interval part-worth utility values, respectively, across the 21 disease criteria. As part-worth utility values are interval data, we compare changes in disease ranks rather than changes in the overall scores to assess uncertainty.
We used Sawtooth Software CBC/HB v5.2.8 [2] to generate goodness-of-fit measures for the individual-level HB models; these included a percent certainty fit and a root likelihood (RLH). Both of these measures are calculated from the probability of each respondent choosing as they did on each choice task using a logit model estimated with each respondent's part-worth utility values [40]. The percent certainty and RLH both reflect how well the current model fit is in comparison to a chance model and a perfect model.
Chi Square and Fisher's exact tests were used to compare the demographic, professional background and survey characteristics of study participants to their respective national populations and to make comparisons by country. The Mann-Whitney test was used to explore differences in completion time by country. National population data for gender, age, region and education were obtained from Statistics Canada [43,44] and the US Census Bureau [45,46]. Unpaired t-tests, F-tests and Welch's t-tests were used to explore differences in standardized importance scores and part-worth utility values between Canada and the US. We used ttests to compare differences in importance scores between the public groups presented in our previous paper [33] and the professional groups presented in this paper, and ANOVA to compare differences between the four groups (Canadian public, Canadian professionals, US public and US professionals). Spearman rank correlation was used to compare disease priority ranks between groups. Friedman's ANOVA was used to assess uncertainty in the final models on disease priority ranks.

Survey and Demographic Characteristics
The response rate for survey completion could not be calculated because the total number of respondents approached at conferences and through advertisement in publications could not be Figure 1. Example of one choice task set completed by each study participant. As multiple survey versions were administered randomly to each person, a different combination of disease criteria and levels was presented to study participants. The ordering of the presentation of disease criteria within each choice task was randomized to reduce ordering bias. (*Canadian participants were asked to prioritize for Canada while US participants were asked to prioritize for the US.). doi:10.1371/journal.pone.0072172.g001 ascertained. Approximately 12,076 Canadians and 33,992 Americans were recruited via direct email invitation, however, as individuals may have been affiliated with one or more professional associations, duplicate posting likely occurred and the exact number of participants approached is unknown. As surveys were collected anonymously, we could not determine survey responses by recruitment mode.
A total of 928 Canadian and 998 US surveys were completed in 13 months and 3 months, respectively, of these, 62 Canadian and 55 US surveys were removed due to not fulfilling the eligibility requirements of being human health or animal health professionals. The majority of Canadian surveys were completed in English (95.5%) while the remaining surveys were completed in French (4.5%). All US surveys were completed in English despite an option to complete the survey in Spanish. Participants passed the survey if all 14 choice task sets were completed and the correct diseases were selected for both fixed choice task set. The fixed choice task questions were designed to present an obvious ''best answer'' to serve as a screening tool to identify participants who did not understand the choice task process. The Canadian pass rate was 81.6% (707) while the US pass rate was 81.0% (764); there was no significant difference in the pass rate between countries (x 2 = 0.1148, p = 0.735), nor was there significant difference in the pass rate between Canadian surveys completed in English or French (x 2 = 0.1262, p = 0.722). Although the fixed choice task sets were introduced to eliminate bias from unreliable respondents, these results suggest that respondents were no more likely to pass or fail (and thus introduce bias) from a particular country or language of completion. The median completion time for passed surveys was 26.9 minutes in Canada and 28.1 minutes in the US; there was no significant difference in the completion time between countries (p = 0.515). There were 1,471 completed and passed surveys available for analysis in this study.
There were minor differences between the study populations and their respective national populations by gender, age and geography (Table 1). The most notable difference was a higher educated population in both study populations compared to their respective national populations (p,0.001 for both countries). This was expected as the study populations were recruited for their professional background rather than to obtain a representative sample of the national populations. Differences were also observed between the Canadian and US study populations by gender and education ( Table 2); there were more female study participants in the US than in Canada (61.2% vs. 54.0%), more high school graduates or less (3.5% vs. 0.6%), Bachelor's degrees (14.7% vs. 13.5%), Master's degrees (18.6% vs. 13.6%) and Doctorate degrees (22.7% vs. 20.1%) in the US than in Canada, and more Diplomas, trade or college degrees (3.6% vs. 0.5%) and Professional degrees (48.6% vs. 40.0%) in Canada than in the US.
The Canadian and US study populations differed significantly in professional disciplines, individuals identifying as human health or animal health professionals and by workplace of employment ( Table 3). The Canadian study population comprised of more physicians and professionals in the medical sciences (19.9% vs. 10.1%), veterinarians and professionals in the veterinary sciences (34.1% vs. 29.2%) and animal health laboratory technicians (2.0% vs. 0.8%) while the US study population comprised of more epidemiologists (16.0% vs. 10.5%), public health professionals (20.9% vs. 16.4%), infectious disease researchers (8.9% vs. 5.7%), human disease laboratory technicians (1.8% vs. 0.7%) and nurses (4.7% vs. 2.5%). There were more individuals who self-identified as animal health professionals in the Canadian study population, conversely, more individuals self-identified as human health professionals or both human and animal health professionals in the US study population. The Canadian study population included a higher number of professionals working for the government or at hospitals/clinics while the US study population included a higher number of professionals working in academia and from industry. Despite differences in professional background characteristics between Canada and the US, the study populations each reflected a good representation of professional disciplines, animal and human health professionals and workplace of employment within country (Table 3). Consistent between countries was the years in employment with the majority of study participants indicating over 10 years of work experience in their profession (62.5% in Canada, 62.8% in the US).

Model Fit
The Canadian model had a percent certainty fit of 83.7% and a root likelihood (RLH) of 0.77, the US model had a percent certainty fit of 84.2% and a root likelihood (RLH) of 0.78. The expected percent certainty for a chance model is 0% and a perfect model is 100% while the expected RLH for a chance model is 0.2 (one divided by five disease combinations per task) and a perfect model is 1.0 [40]. While the models in this study do not represent a perfect model, the models are certainly above satisfactory producing robust part-worth utility values.

Disease Criteria Importance Scores and Part-worth Utility Values
The importance scores for disease criteria indicate the degree to which each criterion contributed to the decision to prioritize (Table 4). Human-related criteria contributed more to the decision to prioritize than corresponding animal-related criteria with each of the eight matching criteria exhibiting this trend in both countries. The four transmission potential criteria contributed in the following order of preference in both countries: animalhuman, human-human, animal-animal and human-animal; thus also revealing a stronger preference for human-related criteria over animal-related criteria. While the contribution of each disease criterion in the decision to prioritize differed with varying degree of importance, ranging from 1.32% to 9.06% (Table 4), each criterion as a whole was statistically significant (p,0.05) in the final model for both countries (Table 5).
Although differences were observed between countries, both groups considered incidence of the disease in the last five years in humans and case-fatality in humans to be the most influential criteria in the decision to prioritize zoonoses (Table 4). Similarly, both groups considered high-risk groups in animals to be the least influential criteria in the decision to prioritize zoonoses. An additional eight disease criteria were ranked equally between the countries (transmission potential from animals to humans, case-fatality in animals, economic and social burden on trade in animals, transmission potential between humans, transmission potential between animals, efficacy of control measures in humans, severity of disease in animals and duration of illness in animals) while the remaining criteria differed only by a maximum of two ranked positions (severity of the disease in humans, disease trend in the last five years in humans, disease incidence in the last five years in animals, economic burden in humans, duration of illness in humans, disease trend in the last five years in animals, transmission potential from humans to animals, efficacy of control measures in animals, how much is known scientifically about the disease and high risk groups in humans) indicating a general consensus between the two countries on the contribution of the 21 disease criteria in the decision to prioritize zoonoses.
The part-worth utility values (b) indicate the relative influence each level had on respondent choices with higher values representing a stronger degree of influence on choice. The mean part-worth utility values and the upper and lower uncertainty estimates of mean part-worth utilities values are presented in Table 5. The wider the range in part-worth utility values between the lowest levels and highest levels within each criterion, the more influence the criterion had on the decision to prioritize. There were nine disease criteria in which the importance score differed significantly between countries (Table 4); the utility trends for these criteria can be broadly summarized as follows: Canadian professionals were more strongly influenced by severity of disease in humans, economic burden in humans, duration of illness in humans, case-fatality in animals and high-risk groups in animals (p,0.0024 for all, Table 4). US professionals were more strongly influenced by incidence of the disease in the last five years in humans, incidence of the disease in the last five years in animals, disease trend in the last five years and economic and social burden on trade in animals (p,0.0024 for all, Table 4). Despite the differences in importance scores, there was agreement between countries on the levels of least importance (lowest partworth utility values) and levels of highest importance (highest part- 2006 education data for individuals 20 years and over in Canada (most current and available data) [43]. 5 2010 education data for individuals 18 years and over in the US [45]. *Significant at p,0.01. + An additional Fisher's exact test was conducted to take into account of the small sample size in some Provinces and Territories; the results remain unchanged with no statistically significant relationship between the study population and the national population distributions (p = 1.000). worth utility values) for each of these nine disease criteria, with incremental increases in the part-worth utility values for the levels in between (Table 5). While the strength of preference in disease criteria importance scores and part-worth utility values differed between countries (Tables 4 and 5), there was general agreement in the contribution of disease criteria in the decision to prioritize zoonoses. The difference may be due to a difference in the perceived threat of disease and disease characteristics by country. Table 6 presents the final ranking of diseases derived from their overall CA scores. The range in the overall scores by diseases differed between Canada and the US and correlates with the partworth utility values derived from the country-specific models ( Table 5). Canadians considered rabies to be the most important zoonoses to prioritize, followed by Nipah virus encephalitis, H1N1 influenza, variant Creutzfeldt-Jakob disease and listeriosis. These were also the top five priority diseases in the US, ranked in different order. There was also consensus between the bottom five diseases on the priority list with three of the five least important diseases appearing in both priority lists. Although differences were observed in disease rankings between countries, the majority of diseases (76%) were within ten ranked positions of each other indicating general agreement in disease ranks between countries (Spearman's rho = 0.8356, p = 0.000).

Disease Priority Lists
As we included a broad group of zoonotic diseases, many of these diseases may not be relevant for specific stakeholder groups, for example, there may be groups only interested in vector-borne diseases, food-borne and enteric diseases, exotic diseases, endemic diseases or diseases affecting certain commodity groups. The priority list can be further broken down into these sub-groups and the priority diseases by subgroup include: vector-borne diseases (leishmaniasis, Chagas disease and the plague in Canada; babesiosis, anaplasmosis and the plague in the US), food-borne  Table 6). Diseases of high priority exhibited high incidence (H/A), high case-fatality (H/A), severe symptoms in humans, an increasing or emerging trend (H/A), high socioeconomic burden (H/A), prolonged duration of illness in humans and high transmission potential from animals to humans. However, it was not necessary to exhibit each of these characteristics to be identified as a priority (for example, H1N1 influenza and babesiosis both have low casefatality rates while Nipah virus encephalitis, Ebola virus hemor-rhagic fever and Marburg hemorrhagic fever do not occur naturally in North America). Diseases of low priority generally included rare diseases or diseases with a large proportion of asymptomatic cases (H/A), low case-fatality (H/A), mild symptoms in humans, stable or decreasing trend (H/A), low socioeconomic burden (H/A), short duration of illness in humans and low transmission potential from animals to humans.
Canadians considered leishmaniasis, Hendra virus, salmonellosis, cryptosporidiosis and Crimean-Congo hemorrhagic fever of higher priority than Americans (difference of 12 ranked positions or more, Table 6). Conversely, Americans considered babesiosis, anaplasmosis, paralytic shellfish poisoning, coccidioidomycosis, cysticercosis and Rocky Mountain spotted fever of higher priority than Canadians (difference of 12 ranked positions or more, Table 6). This can be explained by regional differences in both human and animal disease incidence; for example, babesiosis does Includes other medical and science related disciplines such as health education, travel medicine, wildlife and aquatic biologists, environmental and ecosystem health, occupational and environmental health and safety, medical entomologists, food inspection and risk assessment, regulatory medicine and policy. 2 This group consisted of five individuals who selected the 'I prefer not to answer' response for professional discipline but who identified themselves as either animal health or human health professionals with at least one year of work experience and working in academia, industry or a hospital/clinic. 3 Includes non-government organizations, private consultancy, small businesses, aquariums and zoos, farms, and medical and veterinary associations. *Significant at p,0.001. doi:10.1371/journal.pone.0072172.t003 not occur naturally in Canada but is endemic in the US [47], anaplasmosis is extremely rare in Canada but endemic in the US [48] and Rocky Mountain spotted fever is found only in Western Canada but is distributed throughout the US [49]. Regional differences in disease trend also likely contributed to different rankings, for example, paralytic shellfish poisoning has been increasing in the US [50] but is stable in Canada. Other differences can be explained by differences in part-worth utility values in disease criteria by country (Table 5).
To test the uncertainty in the part-worth utility values derived from the statistical models, we applied the 95% lower and upper confidence intervals of the mean part-worth utility values (Table 5) to compare the ranking of diseases. There was no statistical differences in the ranking of the diseases when uncertainty estimates were applied with 58 of 62 disease ranks unchanged and 4 diseases changing by one rank in Canada (Friedman's x 2 = 0.000, p = 1.000) and 53 of 62 disease ranks unchanged, 8 diseases changing by one rank and one disease changing by two ranks in the US (Friedman's x 2 = 0.008, p = 0.996) (S2). There were also no statistical differences in the ranking of the diseases when uncertainty estimates were applied to cluster of similar diseases (vector-borne diseases, food-borne and enteric diseases, exotic diseases and endemic diseases) (p.0.993 for all disease groups for both Canada and the US).

Comparison between Public and Professional Groups
Results from the same survey administered to the public were published previously [33]. The Canadian and US public groups completed the survey faster with a lower pass rate compared to their respective professional groups (Table 7). There was no difference in the gender and age distribution between the Canadian public and Canadian professional groups, but there were a higher number of study participants from Alberta, Manitoba, Ontario and Saskatchewan in the professional group. There was no difference in the geographic distribution between the US public and the US professionals, but there were a higher number of middle to older age females amongst the study participants. This may reflect the target population in the professional group. A highly educated population was observed in both professional groups compared to the public groups despite the public groups already representing a highly educated population in comparison to the national populations [33]. (H) = human-related characteristic, for example, case-fatality in humans.

2
(A) = animal-related characteristic, for example, case-fatality in animals. 3 Relative rank of disease criteria by importance scores for Canadian participants; the Table is presented in order of importance for Canadian participants. 4 Mean importance score across respondents. 5 Standard deviation of importance scores across respondents. 6 Relative rank of disease criteria by importance scores for US participants.   The percent certainty for the Canadian and US public models were 79.4% each [33]. The models presented here for the Canadian and US professionals indicate the professional models were better fitted than the public models (83.7% and 84.2%, respectively), though models of percent certainty of 70% or higher are regarded as models of good fit [51].
The mean disease criteria importance scores by country and by groups are presented in Figure 2. Similarities were observed between the combined public groups and combined professional groups while differences were observed between the public and professional groups. We identified differences in mean scores between the public and the professional groups for each disease criterion (p,0.0024) with the exception of disease incidence in humans, disease incidence in animals, animal-animal transmission potential and how much is known scientifically about the disease. The public groups considered criteria relating to the individual-level of disease burden such as case-fatality (H/A), human-to-human transmission and duration of illness (H/A) more important in the decision to prioritize (p,0.0024) while the professional groups deemed criteria relating to the societal and population-level of disease burden such as socioeconomic burden (H/A), disease trend (H/A) and efficacy of control measures (H/A) to be more important (p,0.0024).
A unique priority list was derived for each group in each country (Tables 6 and 8). Although differences were observed, similarities were also apparent in diseases in the top 10 and in the bottom 10. Seven of the top ten diseases were common across all groups (rabies, Nipah virus encephalitis, Ebola virus hemorrhagic fever, Marburg hemorrhagic fever, H1N1 influenza, variant Creutzfeldt-Jakob disease and listeriosis) while five of the bottom ten diseases were common across all groups (bovine tuberculosis, cyclosporiasis, Dengue fever, La Crosse encephalitis and St. Louis encephalitis). Despite more similarities in demographics, model fit, disease criteria importance scores and part-worth utility values between the two public and the two professional groups, there was more uniformity across priority lists between the Canadian public and Canadian professionals (Spearman's rho = 0.9707, p = 0.000) and between the US public and US professionals (Spearman's rho = 0.9774, p = 0.000) than between the Canadian and US public groups (Spearman's rho = 0.8497, p = 0.000) and between the Canadian and US professional groups (Spearman's rho = 0.8356, p = 0.000).

Discussion
Zoonotic diseases are a significant public health burden in North America [6][7][8][9]. As there are limited resources available for their control and prevention, a scientifically driven framework for the prioritization of zoonoses is essential. An increasing number of recent studies have focused on quantitative methods for disease prioritization [13,14,21,[24][25][26], though there is currently no agreeable standard. In this study, we present on the novel use of the quantitative method, CA, for prioritizing zoonoses in North America. Our method fits a statistical model to choice data under a robust experimental design to generate relative weighted scores for disease criteria and levels. This overcomes the primary limitation of current methods that rely on simplified linear point-scoring systems without weights [16][17][18][19][20] or an expert Delphi panel to assign subjective weights that are applied to linear point-scoring systems [13,14,21,24,26]. Further, CA allows for disease criteria to be considered jointly, acknowledging dependence between criteria. CA forces individuals to prioritize on the basis of scientific information, eliminating biases associated with disease names and as individuals are provided with all the information necessary to prioritize, the method allows for the inclusion of individuals with no prior knowledge of the diseases (public) or health professionals who may not be knowledgeable in the full range of diseases.
In comparison to other prioritization studies, this study included a larger number of disease criteria than previously used [14,[16][17][18][19][20][21][22][23][24][25], although many of these studies involved diseases exclusively in humans hence our animal-related criteria would not have been appropriate in these studies. We also used a broad objective for  disease prioritization, asking our respondents to prioritize diseases for their control and prevention without specifying the method of control and prevention (for example, regulation, management, vaccination, laboratory diagnosis, research and surveillance). Many studies focused on a specific aspect of disease prioritization, for example, prioritizing solely for surveillance [19,20] or prioritizing a specialized group of diseases, for example, emerging or food-borne zoonoses only [24,25]; these studies would require fewer disease criteria to assess diseases. Our study also included the largest number of professional participants of any disease prioritization study [14,[16][17][18][19][20][21][22][23][24][25][26]; this was feasible due to the novel CA method used in which respondents were presented with all the information necessary to prioritize diseases. This allowed for wide participation including health professionals who may not be familiar with the full range of diseases being prioritized, but could add much value to the decision to prioritize due to their specific professional training and experience. While our method involves more work and a larger study population than traditional methods, the use of CA overcomes traditional limitations in prioritization methods including simplified linear scores, subjective weights, the assumption of independence between disease criteria, biases associated with disease names and limited expert participation. Our previous study indicated individuals with no prior knowledge of infectious diseases were capable of producing meaningful results with satisfactory model fits [33]. This study suggests professionals with knowledge or experience in prioritizing zoonoses were capable of producing meaningful results with better-fitted models than the general public. Disease criteria importance scores were realistic and sensible, consistent with findings from previous prioritization studies [13,14,21,[24][25][26].
Part-worth utility values demonstrated face-validity with higher preferences given to higher levels and lower preferences to lower levels. The disease priority lists generated from applying CAderived part-worth utility values to diseases produced a list of diseases that are reasonable for prioritization, particularly when further divided into subgroups such as vector-borne diseases, foodborne and enteric diseases, exotic diseases and endemic diseases. Analysis of the uncertainty estimates in the part-worth utility values showed that disease priority lists did not change substan-tially; indicating good model fits producing robust part-worth utility values.
Our study found that the public groups placed more importance on disease criteria relating to the individual-level of disease burden while the professional groups placed more importance on disease criteria relating to the population-level of disease burden. Our previous study on the focus groups conducted to inform this current study affirms this observation with similar findings between the public and professional focus groups [12]. Despite closer resemblances in demographics, model fit and disease criteria importance and part-worth utility values between the combined public and combined professional groups, there was more unity in the disease priority lists between the Canadian public and Canadian professionals and between the US public and US professionals than the combined respective public and professional groups. This suggests that regional differences in country-specific disease criteria (disease incidence and trend) contributed more to disease priority rankings than differences in demographics and disease criteria preference. This finding is not surprising given study participants were asked to prioritize diseases for their control and prevention in their respective countries, moreover, if disease incidence was high and disease trend was emerging in the US but the disease was not found in Canada, the part-worth utility values assigned to the disease in Canada would reflect the lowest levels despite disease incidence and disease trend having high importance to Canadians, while the part-worth utility values assigned in the US would reflect the highest levels. This would explain why diseases such as anaplasmosis, babesiosis, paralytic shellfish poisoning and Rocky Mountain spotted fever were ranked much higher by the US groups than their respective groups in Canada.
Limitations related to the study were outlined in the previous paper [33]. Additional limitations associated with the study presented in this paper relate to the multiple recruitment methods used to recruit professionals. It is unknown whether some recruitment methods resulted in better-quality responses than others; we assume study participants responded to the survey in the same manner regardless of their mode of recruitment. However, if a particular recruitment method resulted in lowerquality survey responses, inconsistent responders would have been screened out with the two fixed choice task questions. Multiple recruitment methods may have also resulted in a different group of qualified professionals within the study population; nonetheless, the demographic data collected suggest the two professional groups were representative of individuals with some prior knowledge of infectious diseases. Further, in comparison to the public groups, the two professional groups were distinct populations with higher education and a background in an eligible professional discipline. Although the survey was offered in three languages (English, French and Spanish) across the two countries, surveys were only completed in English and French. While we found no significant difference in the pass rate between the surveys completed in English and French, it is unknown whether translation bias may have resulted in differences in responses between languages or whether bias was introduced as a result of having no surveys completed in Spanish. An additional uncertainty that cannot be account for in this study is the quality and/or the lack of data in the literature used to assign levels of criteria to diseases [33]. It is unknown to what degree this type of uncertainty may have affect the presented results, however, the best available data available at the time of analysis was used [33] and as better quality data and more scientific information about the diseases becomes available, the priority lists presented in this study can only be improved upon.
As noted in our previous paper, there are multiple objectives for prioritizing zoonoses (for example, research, regulation, control, prevention, management, vaccination, diagnosis, cost-effective and surveillance), although study participants were asked to prioritize for policy implementation for the control and prevention of zoonoses, they may have prioritized with another objective in mind. This would be even more apparent in the professional groups who are actively involved in public health and zoonoses. There is no way to measure this type of bias and we assume participants were consistent in their prioritization objectives. Additionally, our professional study population included a wide range of professional disciplines, yet not all disciplines can be treated equally. For example, medical doctors and veterinarians are more likely to have better knowledge and experience with zoonotic diseases compared to laboratory technicians, thus, treating our professional study population uniformly may weaken differences across different groups of professionals. An argument could also be made that some professional disciplines in our study may not truly reflect experts in zoonotic diseases, for example public health nurses, despite the fact that the professionals selected in our study were distinct from individuals in the general public and that as nurses usually represent the front line of defense during an outbreak, their experience would add much value to the decision to prioritize. The same argument could be made for laboratory technicians, who may not be experts in zoonotic diseases, but have a strong understanding on the identification and diagnosis of disease-causing pathogens. This raises the question of which experts should be engaged in prioritizing zoonotic diseases for public health. While our study results represent the collective opinion of a broad group of experts across multiple disciplines, we did identify important differences between our public and professional groups. Our future research goal will be to investigate how individuals from different professional disciplines may prioritize zoonotic diseases differently and explore the issue of which individuals should be responsible for making such decisions.
We present the first zoonoses prioritization exercise involving public health, veterinary and medical professionals in North America. Our novel quantitative approach is an established method in other disciplines. Our study results were validated with satisfactory model fits and reasonable disease criteria scores and part-worth utility values. These results illustrate that CA can be used as a potential tool for the prioritization of zoonoses, particularly as a method to overcome subjective weighting and scoring of disease criteria. Other limitations that can be addressed by CA include assuming independence between disease criteria; biases associated with disease names; and limited expert participation. Although this approach involves more work and a larger study population, disease priority lists can be revised on a regular basis by updating criteria levels to match the most current disease trends, thus, a large-scale CA study can be conducted to establish the baseline disease criteria scores with minor year-to-year updates of disease priority lists. This type of scientifically driven framework for disease prioritization would be of value in North America.

Supporting Information
File S1 Professional associations, publications and conferences targeted for professional study participants recruitment.