Technical efficiency of national HIV/AIDS spending in 78 countries between 2010 and 2018: A data envelopment analysis

HIV/AIDS remains a leading global cause of disease burden, especially in low- and middle-income countries (LMICs). In 2020, more than 80% of all people living with HIV (PLHIV) lived in LMICs. While progress has been made in extending coverage of HIV/AIDS services, only 66% of all PLHIV were virally suppressed at the end of 2020. In addition to more resources, the efficiency of spending is key to accelerating progress towards global 2030 targets for HIV/AIDs, including viral load suppression. This study aims to estimate the efficiency of HIV/AIDS spending across 78 countries. We employed a data envelopment analysis (DEA) and a truncated regression to estimate the technical efficiency of 78 countries, mostly low- and middle-income, in delivering HIV/AIDS services from 2010 to 2018. Publicly available data informed the model. We considered national HIV/AIDS spending as the DEA input, and prevention of mother to child transmission (PMTCT) and antiretroviral treatment (ART) as outputs. The model was adjusted by independent variables to account for country characteristics and investigate associations with technical efficiency. On average, there has been substantial improvement in technical efficiency over time. Spending was converted into outputs almost twice as efficiently in 2018 (81.8%; 95% CI = 77.64, 85.99) compared with 2010 (47.5%; 95% CI = 43.4, 51.6). Average technical efficiency was 66.9% between 2010 and 2018, in other words 33.1% more outputs could have been produced relative to existing levels for the same amount of spending. There is also some variation between WHO/UNAIDS regions. European and Eastern and Southern Africa regions converted spending into outputs most efficiently between 2010 and 2018. Rule of Law, Gross National Income, Human Development Index, HIV prevalence and out-of-pocket expenditures were all significantly associated with efficiency scores. The technical efficiency of HIV investments has improved over time. However, there remains scope to substantially increase HIV/AIDS spending efficiency and improve progress towards 2030 global targets for HIV/AIDS. Given that many of the most efficient countries did not meet 2020 global HIV targets, our study supports the WHO call for additional investment in HIV/AIDS prevention and control to meet the 2030 HIV/AIDS and eradication of the AIDS epidemic.

Introduction of inefficiency previously reported. To address this gap, this study answers the following two questions: 1. What is the technical efficiency of national HIV spending in 78 countries from 2010 until 2018, and how does efficiency differ by region and income level? 2. What country-level independent variables are associated with technical efficiency across the 78 countries?

Data envelopment analysis
Data envelopment analysis (DEA), also known as frontier analysis, was first introduced by Charnes, Cooper and Rhodes in 1978 [23]. It is a performance measurement technique used for evaluating the relative efficiency of decision-making units (DMU's). It assesses how efficiently multiple inputs are able to produce multiple outputs through a maximisation process (maximising outputs from a given set of inputs) or minimisation process (minimising the inputs needed to produce a set of outputs) [24]. The model output is obtained from a nonparametric estimation for the optimum Pareto frontiers through which the efficiency of organisations can be determined. In other words, efficiency is a rank ordering of DMUs compared to a frontier of fully efficient DMUs. The radial distance of a DMU towards its frontier provides the measurement of its efficiency. Inefficient DMUs are enveloped by their efficient counterparts. DMUs can therefore improve their outputs without increasing inputs by maximising the ratio of the weighted sum of the outputs to the sum of the inputs. For the present study purposes, countries and their national HIV programmes are considered DMUs. They may be more efficient (higher efficiency scores) and closer to the estimated efficiency frontier or less efficient (lower efficiency scores) and further away from the frontier. DEA analyses can use output-(maximisation) or input-oriented (minimisation) approaches to compute the efficiency frontier [19,24]. We use an output-oriented approach because national HIV programmes aim to maximise outputs rather than minimise their inputs. Also, rapid structural changes on inputs (related to the level of spending on HIV by country) are unlikely to occur in the short-term [25,26] and national HIV programmes arguably have less control over how many inputs they receive compared with the outputs they produce. We assumed variable returns to scale. This means an increase or decrease in any of our inputs or outputs is not translated into a proportional change in the outputs or inputs, respectively [27], given that national HIV programmes use multiple inputs to produce multiple outputs [28].

Sample and main data sources
We collated data for an initial sample of 131 countries from publicly available sources. Input and output data were sourced from UNAIDS [29]. Data for independent variables were sourced from the World Bank (WB) [30], WHO Global Health Observatory [31], Global Burden of Disease (GBD) study by the Institute of Health Metrics and Evaluation [32] and the United Nations (UN). In total, 78 countries had information available on the main input and output variables (see Fig 1 for details and Table A of Section B in S1 Text). Countries were classified by WB income group [33] and WHO and UNAIDs regions [34,35], as detailed elsewhere. Specifically, for WB income groups, low-income economies were defined as having a gross national income (GNI) per capita �$1,045 in 2020; lower middle-income economies a GNI per capita >$1,046 and �$4,095; upper middle-income economies a GNI per capita >$4,096 and �$12,695; and high-income economies a GNI per capita �$12,696 or more. The final 78 countries included in the analysis represent every WHO and UNAIDS regions and WB income group (Table 1), with an average number of 7 data points for input and output variables between 2010 and 2018. This generated 581 observations in total. We used a different set of country-level characteristics as inputs, outputs, and independent variables for our main analysis following recent literature [19,22,[36][37][38] that we describe below. Finally, some auxiliary variables were only used for a posterior sensitivity analysis.

Input and output variables
Annual national HIV spending per person living with HIV (including new and existing cases) was used as an input [29]-which represents spending per prevalent case of HIV/AIDS. Data on total annual national HIV spending were extracted from UNAIDS for the years 2010 until 2018. Values are expressed in constant US$ 2019. Total annual HIV spending includes all spending, domestic and external, except for private and out-of-pocket spending. To generate the DEA input, annual national spending on HIV was divided by the annual number of PLHIV, or spending per person notified, to enable comparison across DMUs.
Two variables were used as annual outputs in the DEA model: (1) the percentage of PLHIV receiving Antiretroviral Therapy (ART) and (2) the percentage of pregnant women living with HIV receiving Antiretrovirals (ARV) for the prevention of mother-to-child transmission (PMTCT). We chose these variables because of data availability favouring cross-country comparability, and ART and PMTCT services accounted for more than 50% of total HIV spending (on average) in the countries with available information. The relationship between output and input variables was positive (ρ = 0.17-0.27) and satisfied the need for isotonicity.

Independent variables
A series of independent variables were selected to investigate associations with technical efficiency. These variables were chosen based on the literature, interaction with HIV treatment, and data availability. A total of 10 variables were used in our base model and four additional variables were included as auxiliary variables in the sensitivity analysis. The correlation between selected independent variables (using Pearson's coefficient) was investigated to avoid multicollinearity (see Table B of Section B in S1 Text). Our selected variables were not highly correlated between them (medium and low correlation levels encountered, Pearson coefficient <0.7). Multicollinearity tests using the variance inflator factor (VIF) for the adjusted truncated models were employed. We did not find high multicollinearity (mean VIF = 3.28, Table F of Section B in S1 Text) for our adjusted truncated regression. Details on the variables used, justification for inclusion and expected direction of their relationship with efficiency are included in Table 2. Additional variables were tested including a wide range of countries' characteristics (see S1 Data); however, most variables were removed due to collinearity, high correlation or null evidence provided within the literature.

Missing data
We used a two-step procedure to address missing values from collated data (15% of the independent variables were missing; density of health posts, number of nurses, and HIV prevalence accounted for most). First, we imputed missing data points between years by using the average of the observable variables, following which we carried back or forward values from the earliest or latest observable data-points respectively. Third, for countries without data on a specific independent variable, we employed missing value imputation using multivariate normal regression methods, in which 50 imputations were performed, and then the final independent variable was generated using an averaged value (details on the model used and imputation diagnostics are found in Section A in S1 Text).  Table 2. Expected effect of independent variables on technical efficiency and justification for inclusion in the analysis.

Independent variable Definition and Justification [Expected effect for efficiency � ] Source
Rule of Law Rule of law is a continuous variable ranging from -2.5 to 2.5 which indicates how well a country does in regard to their police force and crime rates, quality of contract enforcement, property rights, violence rates, among others. This has been previously included by Zeng and co-authors in their technical efficiency analysis of HIV spending [22]. Rule of law was included amongst a different set of government indicators having the highest association with the outcomes analysed. [+] WB Database [30] Antenatal Care Coverage (ACC), at least 4 visits (%) It is an indicator of access and use of health care by pregnant women during their pregnancy. Antenatal care is crucial for pregnant women and also their infants.
Receiving care increases the probability of receiving adequate and effective interventions to improve their health, including PMTCT which is one of the outputs in the main model. Development assistance and financial aid from external funding for HIV may enhance the efficiency of the health systems through the prevention of treatable health conditions, investment on hospital infrastructure, more technical support and appropriate staff, among other improvements in terms of drugs, medicines, and treatments. Even though higher aid may be associated with clear enhancements in healthcare, it might be the case that higher levels of aid is also a reflection of dependence on external money due to a lack of infrastructure and health system resources which can lead to greater inefficiencies. [+ or -] IHME Financing Global Health [39] Government spending per total HIV spending ratio Ratio indicating the proportion between public spending and HIV-specific expenditures. Government public spending includes subsidies, property income, compensations of employees, education, and social benefits. If the overall ratio is substantially high, then the country has other priorities than HIV or simply the HIV prevalence is too small. Efficiency will therefore depend on contextual variation. [+ or -] IHME Financing Global Health [39] (Continued )

Statistical analysis
The two-stage double-bootstrap DEA approach, developed by Simar-Wilson [40,41], is used in this paper to explore how different country-level independent variables are associated with higher or lower efficiency scores. The Simar-Wilson double-bootstrap approach is increasingly being used to investigate technical efficiency in the health sector [19,28,37,38]. In the first stage, this linear programming approach adjusts initial efficiency scores by the potential biases caused by other independent variables. In the second stage, the scores are bias corrected from the previous step and then used in a truncated regression model that controls for independent variables which may affect outputs. This approach attempts to improve the estimated technical efficiency scores by eliminating potential biases caused by existing measurement errors or serial correlations [19]. Full descriptions of the DEA algorithms used can be found elsewhere [37,40,41] and in Section A in S1 Text. We estimated bias-corrected efficiency scores using an output-oriented DEA model with variable returns to scale and adjusted to independent variables [40,41]. We used 1,000 bootstrap replications in the first loop of Simar and Wilson's (2007) approach and 3,000 bootstrap replications in the second loop for bias-correction of technical efficiency scores. 95% CIs computations were built based on the distance function, i.e., the reciprocal of efficiency score, ranging between one to infinity. Then, we computed the reciprocal of this value (inverse), to generate bias-corrected efficiency scores between 0 (least efficient) and 1 (most efficient). All analyses were carried out in RStudio version 3.3 using the rDEA package (dea.env.robust command, available at https://github.com/jaak-s/rDEA). As per the software programme, truncated multivariate regression was employed using the reciprocal of the efficiency scores as dependent variable. Therefore, the direction and size of the estimates reflect their impact on the inverse of the efficiency scores (i.e., inefficiency).

Sensitivity analysis
To investigate the robustness of our main model technical inefficiency scores (reciprocal of efficiency), we carried out two sensitivity analyses. In the first analysis, we tested three different models by including auxiliary variables that have been relevant in the literature [22,36,38] but contained substantial missing data that had to be imputed. First, we added the number of nurses and health posts to the base model (A). Second, we incorporated the percentage of HIV spending from government sources and the percentage of current health expenditure accounted for by external sources (development assistance for health spending) to the base model (B). Third, we combined models A and B into a single model. In the second analysis, we removed the lowest and highest 5% outliers of our outputs and inputs (separately and jointly) given that the DEA method is highly sensitive to outliers [42].  2).

Descriptive statistics
HIV prevalence is 2.56% across the countries (SD = 5.23) with the highest levels observed in Africa, specifically in Southern, Central, and Western Africa (Fig 2).  (Fig 3). In other words, while 52.5% more outputs could have been produced in 2010 for the same amount of spending, in 2018 an additional 18.2% more outputs could have been produced for the same amount of spending. Constant improvement in technical efficiency is observed between 2010-2018 across countries, with an average annual improvement of 5 percentage points (pp). The average bias-corrected efficiency score across country years was 67.0%, with a 1.6% (average bias calculated = -0.011) correction of the initial computed efficiency scores. Initial DEA, and bias-corrected DEA efficiency scores by country and year are presented in Table C of Section B in S1 Text. The distributions of efficiency scores by country and across countries are displayed in Figs B and C of Section B in S1 Text. The correlation coefficient (Pearson's) between initial and bias-corrected efficiency scores was ρ = 0.996. As shown in Fig 3, technical efficiency scores increased most between 2010 and 2015, following which there is a more gradual increase from 2015 to 2018. However, there is a substantial spread of efficiency scores between 2010 and 2013. From 2014 onwards, there is a narrowed range for efficiency scores (concentrated around the mean)-until technical efficiency reached its highest level in 2018. High income countries are the most efficient between income groups, with the narrowest 95% CIs and a large concentration of data around their median (Fig 4). High-income countries performed better than low-income counterparts (efficiency scores of 97% and 60%, respectively), followed by upper and lower middle-income countries which had median efficiency levels of 77% and 63%, respectively. Across WHO regions, the Eastern Mediterranean region (EMRO) is the least efficient (33%), whereas the European region is the most efficient (83%). Covering UNAIDS regions, we can see that Eastern and Southern Africa (78%) had one of the highest efficiency levels following Western and Central European and North American countries (99%). However, results by income groups and regions vary over time, as seen in Fig 5, and Fig E of Section B in S1 Text. The largest change in technical efficiency over time is observed for Eastern Mediterranean countries, which has been the region presenting most inefficient countries over time, but their efficiency levels have increased by 48% between 2010 and 2018. Even though European countries outperform other countries, the Western Pacific region exhibited the second highest efficiency scores in the most recent year (2018). Similarly, while high-income countries outperformed other income groups over time, low-income countries surpassed lower-middle income countries, becoming more efficient from 2016 onwards. Overall, sampled countries could improve their efficiency by up to 20% (approximately) in 2018, although room for improvement varies by income and region group. Top performers by income group are Chile, Spain and Portugal (high income), Cuba, Dominican Republic, Romania and Suriname (upper-middle income), Bolivia, Cambodia and Cameroon (lower-middle income) and Benin, Gambia, Mozambique and Rwanda (low income).
While differences between income groups and regions are noteworthy, the technical efficiency of HIV spending differs most between countries from 2010 to 2018 (Fig 6). Efficiency scores range between 22% and 98%, a difference of 76pp between the least efficient (Indonesia and Sudan) and most efficient (Romania) countries. The average technical efficiency across country-years was 67% over time, and most countries (90% of observations) were between 26% and 98% efficient. Most of our sampled countries (83%) reporting the highest HIV-burden (adults' prevalence above 5%) lied above the average technical efficiency across countryyears (e.g. Malawi, Mozambique, Lesotho, Zimbabwe, Zambia) except for Equatorial Guinea.   Initial efficiency scores, bias estimates and bias-corrected efficiency scores are included in full by country-year in Table C of Section B in S1 Text).

Independent variables associated with technical inefficiency of national HIV spending
Eight of the ten independent variables investigated are significantly associated with average bias-corrected efficiency scores in the truncated multivariate regression (Table 4). Three variables are negatively associated with average inefficiency (but positively with efficiency). In decreasing order of coefficient size these are: Rule of Law (Coeff = -15.49, p-value<0.001), HIV prevalence (Coeff = -3.17, p-value<0.001), and HDI (Coeff = -1.55, p-value<0.001). These variables have an inverse association with technical inefficiency, in other words a oneunit increase in these variables is associated with an increase in average efficiency score (decrease for average inefficiency). For example, a 1% increase in HIV prevalence is associated with a 3.2% decrease (increase) in average inefficiency (efficiency).
In contrast, five variables were positively associated with technical inefficiency. These are, in decreasing order of their coefficient size: GNI per capita (Coeff = 9.31, p-value<0.001), CHE per capita (Coeff = 7.14, p-value = 0.001), out-of-pocket expenditures as percentage of total HIV spending (Coeff = 0.60, p-value<0.001), DAHS as a percentage of total HIV spending (Coeff = 0.13, p-value = 0.026), and population density (Coeff = 0.02, p-value = 0.046). A rise in one of these variables is associated with an increase (decrease) in technical inefficiency  Table C of Section B in S1 Text for further details on scores estimated per country).
Pearson's correlation and a visual relationship between all our independent variables and technical efficiency can be found in Table D and Fig A of Section B in S1 Text, respectively. The univariate (unadjusted) truncated models were consistent with our main adjusted truncated results (Table E of Section B in S1 Text). The direction of our estimates, which include imputed missing data, was similar with those using only non-imputed data (Table G of Section B in S1 Text).

Sensitivity analysis
For the first sensitivity analysis (a), three alternative models were used to test the base model bias-corrected efficiency scores. Overall, scores varied by less than 1% compared with base model estimates (Fig 7). Model A in Table 5, which includes the number of nurses and health posts, is similar to the base model but finds a significant negative association between the number of nurses and average inefficiency (Coeff = -0.55, p-value<0.001), i.e. positive association with average efficiency. Model B finds that the two-additional sources of spending included, i.e. government spending as a percentage of total HIV and external expenditures as a percentage of CHE, are not significant predictors of technical inefficiency. However, average efficiency scores between models are almost identical. Last, the third model tested combined the independent variables separately added in Models A and B. The third model finds no other significant association in the additional independent variables with average efficiency, aside from number of nurses and density of health posts (Fig 7 and Table 5).
For the second analysis (b), three models were tested excluding the 5% upper, lower and both upper and lower outliers in our main model (Table H of Section B in S1 Text). The lower and upper 5%, outliers for our inputs and output, excluded from the whole sample, did not influence the reciprocal of the efficiency scores (inefficiency). On average, our main model scores varied by one percentage point after excluding either the upper or the lower outliers. The variation was not significant after removing all the outliers from the base model (Model G, Fig 7). Furthermore, there were no differences in the average efficiency score after comparing each model with the base model (t-test p-value>0.1), (Table H of Section B in S1 Text, panel B).

Discussion
This study provides an updated estimate of the technical efficiency of HIV spending, between 2010 and 2018, in 78 countries accounting for approximately 50% of the global HIV burden. Country-level factors associated with average efficiency are also investigated. We used a double-bootstrap truncated regression DEA approach, which has not yet been applied to investigate the technical efficiency of HIV spending across countries. Our findings showed that the global technical efficiency of HIV spending was 81.8% in 2018. In other words, 18.2% more outputs could have been produced globally for the same amount of spending. However, variation was observed in average efficiency between WHO regions and WB income groups, with higher income countries and the EURO, SEARO and WPRO performing especially well. Although global and regional efficiency improved substantially over time, by 34.3 percentage point since 2010, there remains scope to further reduce inefficiency. For example, 36.5% more outputs could have been produced for the same level of spending in the EMRO region in 2018. Even larger differences are observed between countries,  Table 5. Results of the sensitivity analyses for the reciprocal of the efficiency scores (i.e. inefficiency) (N = 78). with technical efficiency ranging from 22% to 98%, which suggests that some countries can substantially increase (by up to 78%) levels of output for the same amount of spending. Countries with higher HIV-burdens did not appear much more likely to have higher spending on HIV (Pearson's correlation coefficient = 0.19). Also, high HIV-burden countries had the highest levels of inputs and output in our analyses which is driving the efficiency scores resulting in a better distribution of the resources (Fig F of Section B in S1 Text). Some of the least efficient countries identified here, such as Guinea-Bissau or the Central African Republic, were also found to be among the least efficient in recent DEA analyses of TB spending [38] and spending for UHC [37]-both of which use the Simar-Wilson approach. Similarly, some of the most efficient countries identified in this analysis, such as Rwanda or Zimbabwe, are also among the most efficient in the analyses of TB and UHC spending. Rwanda and Zimbabwe are also two of the 14 countries that have achieved the target of 73% of PLHIV having suppressed viral loads [5], and Rwanda is among the countries singled out as most efficient by the previous analysis of the technical efficiency of HIV spending in 68 LMICs between 2002-2007 carried out by Zeng and colleagues [22].

A) Exploring by number of nurses and density of health posts
Overall, our results therefore highlight that despite improvements over time there is substantial variation in technical efficiency and there is still room to enhance performance-especially in countries with high-HIV burden. By and large, these findings are comparable to those in the existing literature [13,14,17,22,36]. The key paper by Zeng and colleagues found that the efficiency of HIV spending in 68 LMICs increased from 13.3% to 47.7% between 2002 to 2007 [22]. This is in line with our results, which showed a continued improvement in global technical efficiency between 2010 and 2018 by similar levels. However, there are noteworthy differences, strengths and limitations of our study compared with Zeng and colleagues. First, our sample includes a larger number of countries, years and high-income countries, which were found to be among the most efficient and therefore may have further reduced the estimated efficiency of the lowest performing countries. The smaller sample of countries and years available to Zeng and colleagues at the time may have inflated estimated efficiency. Second, Zeng and colleagues use a traditional two-stage DEA approach, while the double-bootstrap method used in this analysis corrects for bias which reduces estimated levels of efficiency. Third, we only include two of the three outputs used by Zeng and colleagues as voluntary counselling and testing was not considered in this analysis due to data not available.
We find independent variables that consistently affect country efficiency estimates across all models tested. These include Rule of law, HDI, CHE per capita, GNI, OOP spending as % of the GDP, and HIV prevalence, which is in line with previous evidence [13,22,36]. However, these associations should not be interpreted as causal. Indeed, CHE per capita is found to have a negative association with efficiency of HIV spending. This is likely reflective of the high performance of low-and low-middle and upper-middle income countries, and large differences in CHE per capita between WB income country groups for similar levels of achieved efficiency ( Fig G of Section B in S1 Text). That said, CHE as a percentage of GDP is positively associated with higher efficiency. This is likely because high and upper-middle income countries invested more as a percentage of their GDP in health than lower-and lower-middle income countries. Another noteworthy negative association is between the number of health posts and average efficiency, which is weak but to our knowledge has not yet been investigated in other similar analyses. Other studies that measured efficiency of HIV spending using cost-effectiveness analyses reported that overall efficiency may depend on the funding available as well as factors such as epidemic response, national targets, societal and development indicators [14,17,43]. On the whole, our results indicate that countries with higher HIV prevalence and more nurses are less and more efficient, respectively. In turn, however, the association driven by the health posts might be an indication of high levels of inpatient care and hospitalisation, both of which result in lower efficiency scores. Moreover, the number of nurses was negatively correlated with health posts (Pearson's correlation coefficient = -0.15).
In terms of policy implications, improvements at a country-level may be addressed through different ways of expanding HIV/AIDS services and program coverage. First, countries could increase their budgets towards universal coverage of ART for PLHIV and ARV for pregnant women in need. This is the case in countries with the highest efficiency scores (e.g., Cameroon, Cuba, Dominican Republic, Mozambique, Suriname). They achieved this through the development of national monitoring and evaluation systems, as well as the corresponding local support for resource mobilization and programme implementation to scale-up HIV/AIDS services [44]. Second, countries need to improve the administration of heath resources and routine tracking of technical efficiency. As estimated by the WHO, there is a large degree of inefficiency in the health sector at a global level, which ranges between 20-40% of total healthspending [45]. Our estimates are in line with this and indicate an average inefficiency of 27-31%. Therefore, measures on the capacity of public health systems to deliver good diagnostic, prevention, and treatment, are crucial for a prolonged sustainability. This is especially important for less efficient countries with high HIV-burden that need to address their immediate needs through the re-allocation of their resources (e.g., Democratic Republic of the Congo, Equatorial Guinea, Guinea-Bissau). While efficiency gains might help to expand fiscal and budgetary space for HIV/AIDS, they are insufficient to address the gap between current spending and projected resource needs to achieve the end of the AIDS epidemic by 2030 [46]. Third, improvements in other areas enabling better population health, such as universal healthcare access and stewardship/counselling programs, may enhance technical efficiency by promoting awareness of HIV and equitable access to healthcare [47].
While the sensitivity analysis indicated that findings are robust, some limitations must be considered when interpreting the results. First, we removed a large number of countries (mainly high income) due to substantial missing data in input and output variables. It is possible that including these countries in a future analysis changes our estimates. Second, the DEA method only utilises a single frontier approach (calculated from pooled DMUs) without incorporating a multiple frontier perspective to account for group heterogeneity within DMUs (divergent frontiers). Also, DEA uses an hypothetical comparator rather than an existing (reallife) DMU [48], which might again mask the estimates. However, our analysis is based on publicly available sources and efficiency scores were corrected by accounting for potential biasesinitial efficiency scores were overestimated by 1.6% before correction for bias. Also, the DEA approach is more flexible to account for risk biases due to its non-parametric feature. DEA does not require, as deterministic and stochastic parametric methods do, the specification of a functional form for the production frontier [42]. Third, some potential independent variables were not included due to insufficient data, such as information on HIV policy, laws or budgeting processes as well as broader indicators on key characteristics like progress toward Sustainable Development Goals such as Universal Health Coverage. For instance, the ease of applying Trade-related Aspects of Intellectual Property (TRIPS) flexibilities should be accounted for because both ART and PMTCT programs have a large component of expenditure devoted to commodities, so they are largely constrained by how easy it is to apply TRIPS flexibilities to ensure access to medicines for all the country population [49]. Future iterations of technical efficiency analyses should consider such variables as more data becomes available. Fourth, a measure of quality of care is not included in this analysis, which is a common shortfall of efficiency analyses.
Moving forward, future research may include a larger sample of countries and use additional data as this becomes available on other inputs and outputs such as HIV testing services, and HIV spending by activity or program to obtain more informative estimates. The efficiency scores from this analysis can also be used to undertake a resource needs analysis by decomposing the performance gap into efficiency and resource gaps, as done by the follow-up study to the HIV efficiency analysis previously carried out by Zeng and colleagues [36]. In addition, future analyses should aim to analyse how efficient countries are when considering TB and HIV spending and outcomes combined-the importance of which is clear in emerging literature [50]. For instance, TB patients tested positive for HIV (and-or receiving ART), and joint attributed mortality, are also a concern in high HIV-burden countries (e.g., Southern and Central African countries) which will likely affect their technical efficiency. Nonetheless, the results of this study can help governments and donors by providing a benchmark to facilitate improvements in the efficiency of converting HIV spending into service coverage to accelerate progress towards 2030 targets for eradication [5]. However, even some of the most efficient countries in this analysis such as Romania and Ethiopia have not made sufficient progress toward global HIV targets. Combined with the negative impact of the COVID-19 pandemic on progress toward global HIV targets, this study highlights the need for additional investment to develop new approaches in addressing HIV programs as well as broader investment in health and social protection. Optimal and timely investment and distribution of goods and services for the general population, but specifically those at risk, are crucial for better epidemiological and financial sustainability.

Conclusion
The present study used the double-bootstrap DEA to examine the technical efficiency of national HIV/AIDS spending and independent variables associated with efficiency in 78, mostly low-and-middle income, countries between 2010 and 2018. Our findings suggested that, on average, outputs could have increased by 18.2% in 2018 for the same amount of national HIV/AIDS spending. Efficiency scores have varied by income group, and geographical region, but exhibit sustained improvement over the years. Rule of Law, GNI per capita, reduced out-of-pocket expenditure as a % of the total HIV spending, and HDI were associated with technical efficiency. Our sensitivity analyses showed that our predicted efficiency scores did not vary (< 1 percentage point), which suggests our results are robust. Given that even the most efficient countries did not meet global 2020 HIV targets, our study supports the WHO call for additional investment in HIV/AIDS prevention and control to meet the 2030 global targets for viral load suppression and eradication of the AIDS epidemic.