Introduction

Trachoma, caused by the bacterium Chlamydia trachomatis, leads to more cases of preventable blindness than any other infectious disease [1]. Through Surgery for trachomatous trichiasis (TT), Antibiotic treatment, Facial cleanliness, and Environmental improvement (the SAFE strategy), national programs are reducing the prevalence of trachoma in many countries worldwide. As of December 2018, the World Health Organization (WHO) had validated the elimination of trachoma as a public health problem in eight countries: Cambodia, Ghana, the Islamic Republic of Iran, Lao People’s Democratic Republic, Mexico, Morocco, Nepal, and Oman [2]. To eliminate trachoma as a public health problem, a country must meet three requirements: (1) a trachomatous inflammation—follicular (TF) prevalence sustained below 5% for at least two years in 1–9-year-olds without antibiotic mass drug administration in each formerly-endemic evaluation unit (EU), (2) a TT prevalence of less than 0.2% in people 15 years of age or older in each formerly-endemic EU, and (3) a system in place for identifying and treating new cases of TT [3]. In this definition, an EU usually contains 100,000–250,000 people.

Before validating elimination, WHO requires evidence that the prevalence of TF and TT very likely falls below the elimination thresholds. To generate this evidence, national programs often use surveys. Trachoma baseline surveys assess the prevalence of TF and TT before any intervention so that countries can decide which aspects of the SAFE strategy to implement, if any [4]. After national programs implement these interventions, they can conduct trachoma impact surveys (TIS) to assess TF and TT prevalence to see if a district can stop implementing the SAFE strategy at a population level. At least 24 months after stopping interventions, trachoma surveillance surveys (TSS) are conducted to ensure that trachoma has not re-emerged as a public health problem [5]. These surveys all use population-proportional-to-size cluster sampling methods from very similar standard methodologies [6]. In addition to these surveys, a TT-only survey methodology has been developed for use in certain epidemiological contexts where programs wish to assess the prevalence of TT separately from assessing the prevalence of TF [7]. National programs must factor the costs of all surveys into their annual and long-term budgets.

A few costing studies on trachoma prevalence surveys exist. One study examined the costs of a mix of baseline and impact surveys in eight countries in Africa [8], and another examined the costs of baseline surveys conducted by the Global Trachoma Mapping Project [9]. Neither of these, however, specifically examined TIS, TSS, or TT-only surveys. One (to-date-non-peer-reviewed) analysis of TIS costs does exist, but it only examines costs within the Amhara region of Ethiopia [10]. To add to the body of cost data available to national programs as their countries progress toward elimination, this study examines the costs of TIS and TSS in 11 countries and of TT-only surveys in four countries. To assist national programs with applying the findings to their unique contexts, the study further analyzes the cost components of these surveys and tests several hypotheses regarding factors that might drive differences in survey costs, such as country, calendar year, and years of implementation within a country. With these data, national programs will be better prepared to budget for the elimination of trachoma as a public health problem.

Methods

Data collection

The methods used in this paper draw on those of previous investigators, particularly those of Chen et al. [8]. We developed and piloted a standardized data extraction form in Excel that requested information on survey type, EU characteristics, start and end dates, and costs by line item. An EU represents the level at which a survey is powered to show results; in most settings, it corresponds with an implementation unit such as a district. We gathered costs using the ingredients approach [11]: each line item was associated with a unit count and a unit cost, which were multiplied to calculate the cost per line item. In addition, each line item was tagged with exactly one activity and one category, shown in Table 1. Since we assessed only the incremental cost of the surveys—that is, only additional costs directly related to undertaking the surveys beyond the usual costs of the national program—we excluded any costs that would have been incurred had the survey not taken place. This restriction excluded salaries and other remuneration of staff time at both central and field offices except for any per diems received by a staff member for survey-related activities; it also excluded the development and maintenance of the Tropical Data system. As we collected only financial costs to the program, we did not include economic costs or costs to society, such as the value of subjects’ time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Line item categories and activities.

https://doi.org/10.1371/journal.pntd.0007605.t001

Each national program often conducted multiple surveys of a given type in a single year. As all costs for a given survey type in each fiscal year were tracked as a block, we could not gather unique costs by EU. We therefore calculated the cost per survey for each line item by dividing its cost by the reported number of EUs for the country, fiscal year, and survey type [9]. Throughout this paper, a “survey” refers to one survey of a given type (TIS, TSS, or TT-only) implemented in an EU. (A fiscal year stretches from the October of the previous numeric calendar year through the September of the current calendar year; for example, fiscal year 2017 includes dates from October 1, 2016 through September 30, 2017. Since we gathered data on the actual calendar dates of the survey, we report years below in terms of calendar years for ease of interpretation.)

Local finance teams from each national program completed the data extraction form. Costs were recorded in the currency used in the national team’s records, either United States dollars (USD) or the local currency. Wherever possible, country teams provided actual expenditure data. Where surveys are funded through fixed obligation grants (FOGs), in which programs receive a pre-negotiated amount of money upon the completion of pre-arranged milestones, national programs often did not track their actual expenditures with the granularity we required for our analysis. Therefore, for surveys funded through FOGs, we used the line item amounts agreed upon in the FOG budgets, as those amounts reflect the amount disbursed for the surveys.

Results

TIS and TSS

We gathered data from 228 TIS and 94 TSS, or 322 surveys in total. Surveys were conducted between 2011 and 2018, with the first TSS starting in 2014 (Fig 1). They took place in 11 countries: Burkina Faso, Cameroon, Ethiopia, Guinea, Mozambique, Nepal, Nigeria, Senegal, Tanzania, Uganda, and Vietnam (Fig 2). 250 surveys were funded by reimbursement of expenditures and 72 by FOGs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Number of TIS and TSS conducted per year.

Each bar shows the number of each type of survey conducted in each year. As each EU must conduct TIS before TSS, TSS began being conducted in 2014, 3 years after the first TIS in our study.

https://doi.org/10.1371/journal.pntd.0007605.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Number of TIS and TSS conducted in each country.

The size of the bubble centered on each country represents the number of TIS and TSS included in the study from each country. Figure created by authors using R package rworldmap [17].

https://doi.org/10.1371/journal.pntd.0007605.g002

Fig 3 summarizes the overall costs per survey. (A more detailed breakdown of the data by country, survey type, year, cost category, and activity appears in S1 Data; the Q-Q plot of the distribution is shown in S1 Fig) As is common with cost data, a few high outliers created a skewed distribution of costs, which makes the median and interquartile range (IQR) better measures of their central tendency and spread than the mean and standard deviation. TIS and TSS cost a median of $8,298 (IQR: $6,532 –$10,111). Costs did not differ significantly by survey type (Mann-Whitney p = 0.68, 95% confidence interval (95% CI) for cost difference = -$620 –$788). Costs also did not differ significantly by the source of the data, FOG budgets versus expenditure records (Mann-Whitney p = 0.58, 95% CI = -$1,339 –$483).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. TIS and TSS cost distributions.

The histogram displays the cost per TIS and TSS with bins $1,000 (2017 USD) wide. The boxplot, which shares its x axis with the histogram, further summarizes the data. The bar in the middle of the boxplot shows the median, and the edges of the box represent the interquartile range (IQR), or the 25^th and 75^th percentiles. The whiskers display the spread of the rest of the data, except for the outliers, defined as observations that fall more than 1.5 times the IQR away from the median, which appear as dots.

https://doi.org/10.1371/journal.pntd.0007605.g003

Initially, in a simple linear regression, the cost per survey appeared to decline significantly by $407 per year of implementation in a country (95% CI = -$814 –-$46, adjusted R² = 0.007). After controlling for country by adding each country as an independent fixed variable, however, the association between years of implementation and cost per survey was no longer significant (95% CI: -$222 –$607, model adjusted R² = 0.69). In fact, differences between countries account for 69% of the variation in survey costs (adjusted R² from a linear regression with each country as an independent dummy variable, bootstrapped 95% CI = 56%– 77%). In addition, the calendar year of implementation did not appear to be significantly associated with the cost per survey (95% CI: -$194 –$359, adjusted R² = -0.002).

In a simple linear regression, the cost per survey declined by $98 per survey implemented in a single round (95% CI: -$125 –-$70, adjusted R² = 0.06). After adding each country to the model as an independent fixed variable, this change shrank to an $83 reduction per survey implemented in a single round, but it remained statistically significant. (95% CI: -$108 –-$63, adjusted R² = 0.71). See S2 Fig for histograms and Q-Q plots of these bootstrapped distributions.

The cost breakdown (Table 2) highlights the cost categories and activities that drive differing costs of TIS and TSS. By activity, survey fieldwork accounted for 80% of the total costs. Of those fieldwork costs, 58% went towards per diems, including lodging, and 38% went towards travel. Training accounted for the next largest proportion of costs by activity, at 13% of all costs. By cost category, “other” costs, mostly hall rentals and communications costs, accounted for 11% of costs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. TIS and TSS costs by category and activity.

https://doi.org/10.1371/journal.pntd.0007605.t002

Discussion

As in previous costing studies of trachoma prevalence surveys, costs per survey varied substantially by country and were driven primarily by the costs of per diems and travel for survey fieldwork [8–10]. Our estimated costs per survey [for TIS and TSS $8,298 (IQR: $6,532 –$10,111)] were substantially lower than those reported by Trotignon et al. for the Global Trachoma Mapping Project [median $15,839 (IQR: $10,773 –$19,915), 2015 USD; inflated to $16,316 (IQR $11,098 - $20,515), 2017 USD] [9]. They were higher than the Chen et al. costs [median $4,784 (IQR: $3,508 –$6,650), average of 2007–2009 USD; inflated to $5,498 (IQR $4,031 –$7,643), 2017 USD)]. The costs found in our study do align well with the costs found in a study which used very similar methods and inclusion criteria as ours to examine transmission assessment surveys for lymphatic filariasis, another cluster-based prevalence survey [median $9,540, 2012 USD; inflated to $10,298, 2017 USD; excludes rapid diagnostic tests] [18].

As noted in the discussion of the costing study conducted by Trotignon et al., the inclusion and exclusion criteria for costs can create substantial variation in the overall estimates [9]. A key difference in the inclusion/exclusion criteria of studies published to date was whether the data included international technical assistance—Trotignon et al. did, but Chen et al. and we did not. Furthermore, Trotignon et al. estimated the value of in-kind support from government partners in terms of transportation. As we estimated the financial costs to the national program itself, we excluded this support from our costing; given the high proportion of costs dedicated to transportation in both studies, this difference explains much of the discrepancy in expected survey costs. Other cost differences between the studies could derive from differences in the survey settings based on between-country and within-country features. For example, although Chen et al. reported a lower median cost per survey than we found, even within countries, the highest costs per survey found in our studies were similar ($28,954 in our study, $29,203 in Chen et al., both in 2017 USD). These highest costs in Chen et al. were associated with physical and logistical barriers that drove up transportation and personnel costs, even when compared with other districts in the same country [8].

Given the need to conduct surveys at a high level of quality [19] and according to standardized protocol templates [6,7], many of the component costs for TIS, TSS, and TT-only surveys are inflexible. Within countries, we found that the costs per survey did not decline over years of implementation or over calendar years; that is, even implementation experience did not help national programs find efficiencies that significantly affected their total costs. We did, however, find evidence of small economies of scale within a given country and round of survey implementation. Costs such as training costs, which do not increase linearly with the number of surveys as the per diems and supplies required for survey field work do, drive this difference. Inherent features of the area to be surveyed can keep costs fixed: external conditions of geography, population density, and seasonal effects affect the operations and therefore the costs of the survey [9]. The prevailing rates of per diems, vehicle hire, and fuel, in particular, dictate the costs of per diems and travel, which together account for 85% of the cost of the surveys in this study. In our study, countries whose surveys required a higher per diem for surveyors or greater expenditures on fuel due to higher prices or greater distances reported higher overall survey costs. Small efficiency gains found within a country over time might be offset by increases in the unit costs of these inputs or in resource needs for harder-to-reach survey locations [20]. In addition, the standard implementation methods used in these surveys already account for years of lessons learned in diverse international settings, which further decreases the likelihood of massive improvements in efficiency over time, though without rendering such improvements impossible.

TT-only surveys appear to cost about the same as or slightly more than TIS and TSS in the same country. This similarity makes sense due to the close resemblance of their methods. The slightly higher costs per TT-only survey could be attributed to the fact that TIS and TSS may include between 20–30 clusters while TT-only surveys almost always require 30 clusters [6,7]. We do note, however, that the small number of TT-only surveys analyzed in this paper limits the generalizability of these findings. We included these surveys in this paper to provide some insight into the potential costs of TT-only surveys, but a more robust future study of TT-only survey costs, once more are performed, would be useful.

The costs of surveys funded through FOGs and the costs of surveys funded directly did not differ significantly. As countries move toward self-reliance in their neglected tropical disease and other development programs, donor organizations have turned to FOGs and similar milestone-based grants (e.g., fixed amount awards) as a means of improving the management capacity of local health systems. For example, a recent evaluation of USAID’s Neglected Tropical Disease Program found that the use of FOGs improved communication and collaboration between the local governments that signed the grants and the local health service units that bore responsibility for achieving the milestones, which in turn increased the local governments’ interest and involvement in the health concerns of their communities [21]. Given this interest in FOGs, it is useful to note that they do not appear to change the cost of survey activities compared to methods that rely on tracking actual expenditures. Brady et al. found a similar lack of evidence of difference in costs by funding mechanism in their analysis of transmission assessment surveys for lymphatic filariasis [18].

The data and methods of this analysis, though in line with standards for costing analyses of this sort, do have several limitations. First and foremost, our choice of perspective of incremental financial costs to the national program excludes several key cost elements, including remuneration of full-time staff—except when they received per diems for survey activities—and the value of in-kind contributions, including vehicles and training space. It also excludes international training-of-trainer events. Although these events are a key piece of the quality assurance process for trachoma prevalence surveys [19], they are budgeted by a different mechanism and may support many more trachoma elimination projects than those accounted for here. Second, we calculated costs per survey by dividing the total costs of each type of survey in a country each year by the number of each type of survey conducted. This lack of EU-level costing obscures differences within a country, which prevents the analysis of factors such as distance between clusters that might drive differences in survey costs beyond country-level effects. Third, we assumed that all materials purchased for the surveys have a useful life of a single year and therefore did not amortize costs over multiple years, which could make early surveys within a country appear more expensive than later surveys even if they used the same amount of resources. In our analysis, however, we found no statistically significant change in within-country survey costs over time, which suggests that this assumption was appropriate. Finally, although this paper evaluates data for all surveys available to the authors, it is possible that our findings in this paper might not apply equally to surveys in other environments. For example, as all of the surveys examined used Tropical Data for data capture, this study cannot estimate the expected costs of a paper-based survey method or of surveys not using the standardized Tropical Data methodology. WHO, however, does now recommend the use of electronic data capture for trachoma prevalence surveys [6]. We hope that our analyses provide sufficient context for national programs to adapt the findings for use elsewhere.

Although these objective, standardized, population-based surveys require substantial investment, they provide national programs with the data necessary to plan their trachoma intervention activities. Furthermore, they improve confidence that the monitoring of progress toward WHO elimination prevalence thresholds is taking place with as much care as baseline mapping was performed [19,22]. Costing studies like this one therefore provide crucial planning data as countries advance toward the elimination of trachoma as a public health problem.

Supporting information

Acknowledgments

We thank the country finance and operations team members who completed our data extraction forms and answered our many follow-up questions. We also acknowledge with gratitude the hard work of the field teams who carried out the surveys we examine in this manuscript. Finally, we thank the Ministries of Health of Burkina Faso, Cameroon, Ethiopia, Guinea, Mozambique, Nepal, Nigeria, Senegal, Tanzania, Uganda, and Vietnam, which lead the national programs discussed in this paper.