Ethics statement

The study protocol was reviewed and approved by the Ethiopian Public Health Institute Scientific and Ethical Review Office (reference number SERO-128-4-2005) and the Institutional Review Board of Imperial College London (reference number ICREC_8_2_2) [25]. Official letters were written by the Ethiopian Public Health Institute (EPHI) to the regional Health and Education Bureaus to obtain regional consent. The school director provided written consent on behalf of the student families or guardians as the survey falls under the mandate of the Ministry of Health. Teachers and students were informed about the study in their language including the purpose, the benefits, the potential risks and the operational procedure. Before sample collection, students provided verbal consent to be included in the survey.

Overview

We assessed the minimal requirements for four attributes (sensitivity, specificity, throughput, and cost per test) for three hypothetical serological tests in three steps. In the first step, we applied a previously developed simulation framework [26] to determine those combinations of sensitivity and specificity that minimize the risk of incorrect program decisions in the presence of logistic constraints (the number of schools and the number of children per school that can be sampled). In the second step, we determined those combinations of sensitivity and specificity that also fulfilled budget constraints, where the total survey costs should not exceed that of a survey based on the Baermann method. In the third step, we determined the minimal requirements for sample throughput and cost per test of new diagnostics, along with their diagnostic accuracy (sensitivity and specificity), that meet both the logistic and budget constraints.

In addition, we determined the most cost-efficient survey designs when considering operating in an Ecuadorian setting and using two currently available Ab-based assays, including the the commercially available Strongyloides ratti IgG ELISA (Bordier Affinity Products, Crissier, Switzerland), and a prototype point-of-care LFA developed by the Institute for Research in Molecular Medicine of the Universiti Sains Malaysia [9].

Required diagnostic performance in the presence of logistic constraints

To determine those combinations of sensitivity and specificity that minimize the risk of incorrect program decisions in the presence of logistic constraints, we applied a previously developed simulation framework [26]. This framework is based on a two-stage lot-quality assurance sampling (LQAS) strategy and allows to determine the risk of incorrect program decisions (risks of prematurely stopping or unnecessarily continuing with the program) when imperfect tests are deployed. The two-stage refers to the aspect that the design first involves randomly sampling clusters (e.g., schools or community) from an implementation unit afterwhich participants within each cluster are randomly sampled (e.g., children in the sampled schools). To this end, this framework captures spatial heterogeneity of infections across clusters. Note that the simulation model considered a beta-binomial distribution, where the beta-distribution represents the variation in true cluster-level prevalence within an implementation unit, while the binomial distribution is the variation in the number of positive tests results (both true and false positive) across all clusters. Using this framework, we previously assessed the required performance of diagnostics when implemented for M&E of STH (Ascaris, Trichuris and hookworms) control programs [26]. Since the details of this framework have been described elsewhere, we limit ourselves here to justifying the values for the eight input parameters that are required to determine the risk of incorrect decision-making (see S1 Table). These eight input parameters can be divided into three groups. The first group represents the parameters of primary interest: test sensitivity () and specificity (). Next, we have setting-specific parameters: the program prevalence decision threshold () and the intracluster correlation coefficient (), which represents the degree of geographical variation in prevalences. Finally, we have the framework constraints, which consist of logistic constraints and several parameters related to the risk of incorrect decisions. For the logistic constraints, we capped the maximum number of sampled schools () and the number of children per school (). With regard to risk of incorrect decisions, we defined the maximum allowed probability of unnecessary continuing STH control () when the true prevalence is at some value (lower limit or ) under the prevalence decision threshold. Similarly, we defined the maximum allowed risk of prematurely stopping () the program when the true prevalence is at some value above the decision thresholds (upper limit or ).

We let the diagnostic test sensitivity and specificity vary from 50% to 100% (with increments of 2 percentage points), resulting in 676 theoretical diagnostic tests with different diagnostic performance (26 values for sensitivity x 26 values for specificity). The program prevalence decision threshold represents the true underlying prevalence that triggers distribution of ivermectin and was set at 10%, assuming that school-age children will be sampled [7]. The chosen value for the intra-cluster correlation () reflects the spatial heterogeneity in strongyloidiasis measured across 6,846 school-aged children from 64 schools in Ethiopia who were screened for the presence of anti-Strongyloides antibodies in their plasma (see S1 Info). We quantified the using a generalized linear mixed model using these data. We further let the number of schools visited ( vary from 5 to 10 with increments of 1. Assuming that at most 100 children can be sampled per school, for each combination of sensitivity and specificity we determined the minimum number of schools that would have to be sampled to achieve a sufficiently low risk of incorrect program decisions (details below). To calculate the risk of incorrect program decisions, the prevalence limits ( and ) surrounding were arbitrarily set at () [26]. Then, for each combination of , , and , we determined the risk of overtreating and undertreating using Monte Carlo simulation (10,000 iterations). We kept all the combinations of sensitivity, specificity, and number of schools that satisfied the condition of ideal decision-making ( and ) or adequate decision-making ( and ), as in the previous study [26].

Required diagnostic performance in the presence of budget constraints

After having considered logistic constraints, we assessed which combinations of performance of hypothetical serological diagnostic tests (sensitivity, specificity), number of schools, and number of children of schools allow for ideal or adequate decision-making while remaining within a target budget. Here, we restricted the target budget to the total cost of a Baermann-based survey that was sufficiently powered for ideal or adequate decision-making. For each combination of diagnostic performance and sample size, we calculated the survey cost considering two alternative diagnostic assay formats for the detection of anti-Strongyloides antibodies (Ab): an LFA that allows for point-of-care testing of whole blood and an Ab-ELISA that can be used in a central laboratory testing approach. For the Ab-ELISA, we considered the possibility of testing either plasma or eluate from dry blood spot (DBS) samples, resulting in three implementation choices: LFA, Ab-ELISA/plasma, and Ab-ELISA/DBS. We distinguish these three implementation choices as they have different impacts on survey costs (details below).

Simulating the impact of diagnostic test performance and survey design

To determine the required performance of serological diagnostics, we used our simulation framework as described above, but we now allowed the number of sampled children per school to vary from 10 to 100, by increments of 2. This is because with improved diagnostic performance, generally, fewer children might be required per school. For each combination of , , and and , we performed 10,000 repeated Monte Carlo simulations and again determined the risk of overtreating and undertreating. We kept all the combinations of diagnostic performance and survey design that satisfied the conditions of ideal decision-making ( and = 5%) or adequate decision-making ( and ), as in the previous study [26].

For the Baermann method, we only determined which combinations of number of schools and number of children per school allowed for adequate and ideal decision-making, fixing the sensitivity and specificity at 50.0% and 98.0% as reported in literature [9]. However, we note that such high specificity can only be attained by well-trained microscopists. Since less optimal specificity requires a larger samples size (and thus more expensive surveys) [26,27], we also assessed the outcome of survey design when the specificity of the Baermann method was reduced arbitrarily from 98% to 93%, while keeping the sensitivity at 50%. This somewhat arbitrary reduction was applied because identification of larvae in human stool is not trival as these larvae can also represent other parasites (e.g., the hookworms Necator americanus and Ancylostoma duodenale). The possibility of having other worm species is often overlooked or not known to technicians. Moreover, the skills to distinguish larvae at species level (is based on first identifying the larval stage and then differentaing larvae based on stage specific features) requires training and acquisition of experience over time. Thus, reduced specificity of techniques relying on microscopy identification of larvae does occur, especially when personnel is new to the diagnosis of strongylodiasis.

Estimate the total survey costs

For each diagnostic test (Baermann and hypothetical serological tests) and survey design we calculated the total survey costs, which comprised four cost items: (i) the operational cost to inform all schools about the survey (), (ii) the cost to collect and prepare all individual samples for further analysis (), (iii) the cost to transport all samples to a central laboratory (; Ab-ELISA only), and (iv) the cost to analyze all samples (). To estimate the total survey cost for any given survey design, we assumed that a working day consists of 8 hours. Consequently, the number of samples that can be collected and further analyzed daily is limited and varies between deployed diagnostic assay. Therefore, for each survey design and deployed diagnostic assay, we determined the number of working days required to collect and further analyze all samples from the recruited children. This required number of days depends on the number of laboratory technicians in the mobile team, the number of working hours per day, the number of samples collected per hour, and the sample throughput per hour that one person can analyze (see S2 Info). We considered that 4 laboratory technicians would be needed per day when deploying the Baermann method, while only 2 technicians would be needed if using the Ab-based assays (LFA, Ab-ELISA/plasma, and Ab-ELISA/DBS). We further included the costs of various required items (see S3 Info) based on an itemized cost-assessment representing the real costs of a school-based survey carried out in Ecuador [9]. This itemized cost-assessment included the cost of consumables to collect one sample (), the cost to prepare one sample (), the cost to test one sample (), the daily salary for every technician and nurse in the mobile team (), the daily salary for local helper (, only required when deploying the Baermann method) and the daily cost of car rental, petrol, and driver wages (). Note that we did not consider costs for establishing and maintaining laboratory infrastructures. We further assumed that all work takes place on regular working days and that the team does not take any breaks during processing. Further details on how we estimated the total survey costs can be found in S2 Info.

Required diagnostic performance to not exceed the total cost of Baermann-based survey

After we determined the total survey cost for each combination of diagnostic performance and survey design, we set a benchmark for the total survey cost for new hypothetical serological tests, using Baermann as the reference. For this, we considered those combinations of and that allowed a Baermann-based survey to result in adequate and ideal decision-making for the lowest total survey cost. Given this benchmark cost, we identified for each implementation scenario (LFA, Ab-ELISA/plasma, and Ab-ELISA/DBS) those combinations of diagnostic sensitivity and specificity that were logistically feasible and resulted in a total survey cost that did not exceed the benchmark cost.

Minimal requirements for sample throughput and cost per test

We previously showed that higher diagnostic performance allows for a reduction of sample size for the same level of program decision-making (adequate or ideal), thus allowing for less stringent requirements in terms of sample throughput and cost per test [26]. We therefore explored the minimal requirements for sample throughput and cost per test such that the total survey cost does not exceed that of a survey based on the Baermann method (benchmark cost). To this end, we repeated the total survey cost analysis for the three implementation scenarios while varying the sample throughput (5, 10, 20, and 40 samples per hour) and cost per test (0.20 EUR to 15 EUR, with 0.20 EUR increment). Finally, we determined those combinations of diagnostic performance, sample throughput and cost per test that met both the logistic and budget constraints.

Most cost-efficient survey design for existing serological assays

We determined the most cost-efficient survey designs for two existing serological assays, including a commercially available Ab-ELISA and a prototype LFA. For the Ab-ELISA, we based our assumptions about sensitivity (79%) and specificity (93%) on the commercially available Strongyloides ratti IgG ELISA (Bordier Ab-ELISA, Bordier Affinity Products, Crissier, Switzerland), based on crude antigens of the parasite Strongyloides ratti. For the prototype LFA, we considered a cassette-format immunochromatographic rapid diagnostic test developed by the Institute for Research in Molecular Medicine of the Universiti Sains Malaysia. This LFA is based on the recombinant antigen NIE (NIE-LFA) [28]. To determine the most cost-efficient survey design for these assays, we calculated the total survey costs for each combination of survey design () as described in S2 Info, while fixing the sensitivity and specificity at 79.0% and 93.0%, respectively [9]. These tests are assumed to only differ in terms of cost to test one sample, cost to prepare sample, cost to test one sample and the number of samples that can be analyzed by one person in one hour (see S2 Info). The most cost-efficient survey design is the one () that minimized the total survey cost while attaining an adequate and an ideal program decision-making.

Required diagnostic performance in the presence of logistic constraints

Given the presence of logistic constraints, we investigated the combinations of sensitivity and specificity that minimize the risk of incorrect programs decisions. Fig 1 represents the required diagnostic performance (minimum sensitivity and specificity) for adequate (Fig 1A) and ideal (Fig 1B) decision-making when sampling 100 children per school (the assumed maximum logistically feasible number) across five to 10 schools. Three important points can be noted in this figure. First, requirements are more stringent for specificity than for sensitivity. For example, specificity cannot be below 72% for adequate decision-making (Fig 1A) and 84% for ideal decision-making (Fig 1B), whereas sensitivity could be as low as 50% in both cases. Second, the requirements for sensitivity and specificity are inversely correlated. For example, when sampling seven schools with a theoretical test with a specificity of 98%, the sensitivity should be at least 50% to allow adequate decision-making, while when deploying a theoretical test with a specificity of 84%, the sensitivity should be at least 92%. Third, the requirements for diagnostic performance become less stringent as a function of an increasing number of schools (thus more children are screened) and increasing risk of incorrect decision-making. When focusing on adequate program decision-making and fixing the specificity at 96% (Fig 1A), the required sensitivity was 58% when 7 schools were sampled, while it was 50% when 10 schools were included. When now focusing on ideal program decision-making (Fig 1B) while fixing the specificity at 96% and the number of schools at 7 (Fig 1B), the required sensitivity was 78% (instead of 58% for adequate decision making).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Required sensitivity and specificity when making adequate and ideal program decisions in the presence of logistic constraints.

The figure indicates the required sensitivity and specificity for adequate ( and ; Panel A) and ideal decision-making ( and ; Panel B) when sampling 100 children per school for different number of schools (colored lines). The black dot represents the diagnostic performance of the Baermann method (sensitivity = 50.0% and specificity = 98.0%).

https://doi.org/10.1371/journal.pntd.0012774.g001

Required diagnostic performance in the presence of budget constraints

While we have considered logistic constraints so far, we further explored those combinations of sensitivity and specificity that allowed for adequate and ideal decision-making in the presence of budget constraints. For this, we explored three implementation choices (LFA, Ab-ELISA/plasma and Ab-ELISA/DBS) while not allowing the total survey cost to exceed the cost of a Baermann-based survey. For adequate decision-making, the most cost-efficient survey design for Baermann-based survey involved a total of 644 children (), resulting in a total survey cost of 9,410 EUR. As soon as 41 children test positive (), one could already stop screening and initiate a strongyloidiasis control program. For ideal decision-making, the most cost-efficient design involved a total of 980 children (), resulting in an estimated total survey cost of 13,968 EUR. For this level of decision-making, 65 positive test results would trigger large-scale deworming. If the assumed specificity of Baermann was 93% instead of 98% because of insufficiently trained laboratory technicians, the required sample size was beyond what was considered logistically feasible (adequate: 13 schools x 96 children per school; ideal: 20 schools x 96 children), resulting in a total cost of 17,625 EUR (adequate) and 27,103 EUR (ideal). We will therefore base the benchmark for total survey costs on the assumption that the Baermann method is performed by well-trained technicians (i.e., specificity of 98%).

After defining the cost benchmark, we continued identifying those combinations of sensitivity and specificity of hypothetical new serological tests that allowed for a survey that was not more expensive than the most cost-efficient Baermann-based survey design. Table 1 presents the required diagnostic performance (sensitivity and specificity) for adequate and ideal decision-making when deploying an LFA and Ab-ELISA/plasma in the presence of both logistic and budget constraints. When focusing on LFA, two crucial points can be made. First, this table indicates that when an LFA format is applied, the test performance can take a wide range of values for sensitivity and specificity, the two parameters again being inversely correlated. Second, when looking more closely into the absolute values, they represent all possible combinations of sensitivity and specificity illustrated in Fig 1 (all combinations above the purple line). In other words, even when the sampling effort is maximized (10 schools and 100 children per school), the total survey costs never exceeded that of a Baermann-based survey, highlighting that the budget constraints had no impact on the requirements for the diagnostic performance of LFA.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Required specificity and sensitivity for adequate and ideal decision-making when deploying LFA and Ab-ELISA/plasma in the presence of both logistic and budget constraints. The table summarizes the sensitivity and specificity for the LFA and Ab-ELISA/plasma to achieve adequate ( and ) and ideal program decision-making ( = 5% and = 10%). Each of these combinations result in a total survey cost that does not exceed that of a Baermann-based survey (assuming a specificity of 98% and sensitivity of 50% for the Baermann method). Ab-ELISA/plasma: Ab-ELISA assays that are based on plasma. Note that the blank cells indicate that there were no combinations of sensitivity and specificity that resulted in sufficiently low risk of incorrect decision-making.

https://doi.org/10.1371/journal.pntd.0012774.t001

For Ab-ELISA-based surveys, which are more expensive (see S2 Info), the impact of budget constraints on the requirements for diagnostic performance was more pronounced, resulting in fewer and more stringent diagnostic performance options. For an Ab-ELISA-based survey to be as cost-efficient as a Baermann-based survey for adequate decision-making, its required sensitivity should be at least 80%, while its specificity cannot be below 98% (Table 1). Additionally, the required number of schools and children that need to be sampled was lower for an Ab-ELISA-based surveys compared to LFA (5 schools vs. 10 schools and 100 children per school).

Minimal requirements for sample throughput and cost per test

We then looked at how better diagnostic performance allows for less stringent sample throughput and cost per test for all logistically feasible surveys. When focusing on LFA (Fig 2A and 2B), we note two important aspects. First, results confirm that improving the sensitivity and specificity of a test allowed for higher costs per test (shift from dark green to red colored tiles), with the highest possible cost per test for a LFA with 100% sensitivity and specificity. As illustrated in S1 Fig, improving the diagnostic performance required smaller sample sizes for the same level of program decision-making, thus allowing to have higher cost per test for the same total survey cost. Second, by comparing Fig 2A and 2B, we can see that the maximum cost per test was very similar for tests with lower (5 samples per hour) and higher throughput (40 samples per hour). For example, when fixing diagnostic performance at 92% specificity and 74% sensitivity, the maximum cost per test equaled 6.4 EUR when five samples per hour were processed, and 7.6 EUR when 40 samples per hour can be processed. As such, improving the diagnostic performance allows for a relatively higher maximum cost per test than improving the sample throughput.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The maximum cost per test for an LFA and Ab-ELISA/plasma for adequate decision making.

This figure presents the maximum cost per test for all possible combinations of sensitivity and specificity, sample throughput (number of samples that can be analyzed by one technician in one hour) that allowed for adequate ( and ) decision-making for an LFA (Panel A: sample throughput of five samples per hour; Panel B: sample throughput of 40 samples per hour) and Ab-ELISA format (Panel C: sample throughput of five samples per hour; Panel D: sample throughput of 40 samples per hour). Note that the maximum cost per test is the true maximum cost per test that could work in at least one logistically feasible survey design that is not more expensive than the benchmark cost based on the Baermann method. Absence of tiles indicates that the combination of sensitivity and specificity was inadequate for decision-making, given the logistic and budget constraints.

https://doi.org/10.1371/journal.pntd.0012774.g002

When comparing these findings for LFA (Fig 2A and 2B) with those for Ab-ELISA/plasma (Fig 2C and 2D), two important differences can be noticed. First, as expected from the performance options in presence of budget constraints (Table 1; LFA: Fig 2A vs. Ab-ELSA: Fig 2C), there are fewer and more stringent diagnostic performance options for Ab-ELISA (fewer colored tiles in the figure). Second, the requirements for the cost per test are stricter for Ab-ELISA (fewer red tiles), indicating that the cost per test should be relatively cheap. For example, if we focus on a throughput of five samples per hour, the maximum cost for test with a specificity of 100% and sensitivity of 52% was 13.6 EUR when deployed as LFA format, while this was 1.0 EUR when deployed as an Ab-ELSA format.

We next focus on the maximum cost per test for an LFA and Ab-ELISA/plasma across different combinations of diagnostic performance and sample throughput for ideal (instead of adequate) decision-making (Fig 3). Overall, we noted two aspects. First, for ideal decision-making (Fig 3), patterns in maximum cost per test (as function of sensitivity, specificity and throughput) are essentially the same as for adequate decision-making (Fig 2). However, diagnostic performance requirements are more stringent, meaning there are fewer options. Next, given that the benchmark cost for ideal decision-making was set to be higher than for adequate decision-making, the maximum cost per test for a given diagnostic performance was actually very similar between the two levels of certainty in decision-making.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The maximum cost per test for an LFA and Ab-ELISA/plasma for ideal decision making.

This figure represents all possible combinations of sensitivity and specificity, sample throughput (number of samples that can be analyzed by one technician in one hour), and the maximum cost per test (in EUR) that allowed for ideal ( and ) decision-making for an LFA (Panel A: sample throughput of five samples per hour; Panel B: sample throughput of 40 samples per hour) and Ab-ELISA format (Panel C: sample throughput of five samples per hour; Panel D: sample throughput of 40 samples per hour). Note that the maximum cost per test is the true maximum cost per test that could work in at least one logistically feasible survey design that is not more expensive than the benchmark cost based on the Baermann method. Absence of tiles indicates that the combination of sensitivity and specificity was inadequate for decision-making, given the logistic and budget constraints.

https://doi.org/10.1371/journal.pntd.0012774.g003

Most cost-efficient survey design for existing Ab-based assays

We finally determined the most cost-efficient survey designs when deploying two currently available Ab-based assays. Table 2 a summarizes the most cost-efficient survey designs that allowed for adequate and ideal decision-making for these two tests when assuming the diagnostic performance reported by Tamarozzi et al. [9] (sensitivity = 79% and specificity = 93%). Compared to the Baermann method, two important differences can be noted for these Ab-based assays. First, the required sample size for the survey is slightly smaller when deploying Ab-based assays. For instance, when considering adequate decision-making, a total of 644 children () need to be screened when using the Baermann method. This required sample size decreased to 564 children () when deploying Ab-based assays. Second, despite this smaller sample size, the surveys were more expensive when deploying Ab-based assays compared to the cost of a cost-efficient Baermann-based survey. For instance, when deploying the Bordier Ab-ELISA, the total survey costs were 81% times higher when applied on plasma (Bordier Ab-ELISA/Plasma) and 82% times higher when applied on DBS (Bordier Ab-ELISA/DBS). In contrast, deploying LFA resulted in cheaper surveys compared to the cost of a cost-efficient Baermann-based survey. For example, for adequate decision-making, the total survey costs for NIE-LFA were 52% times lower than that of surveys based on the Baermann method. If these Ab-based tests are implemented (6 schools and 94 children per school), 76 positive test results would trigger large-scale deworming programs for adequate decision making and 123 for ideal decision-making (9 schools and 98 children per school).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The most cost-efficient survey design to initiate large-scale deworming against strongyloidiasis when deploying Baermann method and existing Ab-based assays. This table represents the required sample size (), the decision cutoff , and the total survey cost to assess whether the true prevalence of strongyloidiasis is under or above 10% for adequate and ideal decision-making. Bordier Ab-ELISA represents a commercially available Strongyloides ratti IgG ELISA (Bordier Affinity Products, Crissier, Switzerland) applied on either plasma (Bordier Ab-ELISA/plasma) or dried blood samples (Bordier Ab-ELISA/DBS). NIE-LFA is a prototype (research use only) lateral flow assay (LFA) developed by the Institute for Research in Molecular Medicine of the Universiti Sains Malaysia. This LFA is based on the recombinant antigen NIE.

https://doi.org/10.1371/journal.pntd.0012774.t002