Notation and the three statistical approaches

The statistical approach in a regression setting seeks to identify and employ the most suitable function for mapping inputs to the service delivery outcomes [17]. Before modelling, it is essential to acknowledge the various data treatments derived from univariate and bivariate studies and their implications, as early data exploration is frequently necessary to determine the best appropriate model [18]. The quantitative outcome/the composite indicator score is denoted as Y and p different inputs/predictors. For some input values X= (X₁, X₂, X₃, …, X_p), the function as in equation (1) is modeled on the assumption that there is a relationship between Y and X.

(1)

where f is some fixed but unspecified function of X and is an i.i.d random variable that describes observation noise and perhaps un-modeled processes. Primarily, statistical modelling is to find a function (either parametrically or otherwise) based on the observed data, that is close to the unknown function f that generated the data and demonstrates effective generalization [17]. Parametric statistical modelling involves two steps: making of assumptions about the functional form of f, which facilitates model selection, and using training data to fit or train the model [19]. In contrast, non-parametric approaches do not impose explicit assumptions regarding the functional form of f, instead they seek an estimate of in a manner that closely approximates the data points while maintaining smoothness. The advantage is that by avoiding the assumption of a particular functional form for f, they have the potential to accurately fit a broader spectrum of feasible shapes for f. However, this can be detrimental as it typically results in over-fitting and produces low accuracy on data not utilized for model training.

In our view, a significant challenging phase in a study is selecting an appropriate model or approach that fits a set of predictors to the dependent variable. This is because, no single method surpasses all others over all potential data sets [20,21]. As such, selecting the appropriate family of models is essential, necessitating multiple options to address various research inquiries [22]. Moreover, statistical approaches differ from decision analysis since statistical analyses may end with an indication of the probabilities and related uncertainties while the latter culminates in the selection of a single decision [23]. Considering this position, we provide a concise overview focusing on the prerequisites, possibilities and limitations of a selection of potential models applicable to an investigation of this kind. We commence with the linear regression model, according to Hastie et al. (2019) the regression function E(Y|X) is linear in the inputs X= (X₁, X₂, X₃,…,X_p) [20]. Furthermore, linear regression is as well linear in the parameters [17,18]. It predicts Y as a weighted aggregate of the inputs, typically employing the ordinary least squares approach for estimation. Besides the linearity assumption, the linear model is premised on other assumptions, such that Ɛ ~ N (0, δ²) and the absence of multi-collinearity among the predictors. The strength of the linear model lies not only in its user-friendly simplicity [24] and the clarity it provides in interpreting the link between predictors and outcomes, but it also serves as a robust foundation for extensions or generalizations to other models. The disadvantage with the linear regression model is that it works well for regression but not for classification. In addition, accuracy and simplicity (interpretability) are at odds, as the accuracy of a linear model typically falls short compared to that of the less interpretable neural networks [21]. Again, if the assumptions are breached, the confidence intervals of the coefficients become unreliable. To achieve sparsity, model regularization techniques like lasso and ridge regressions extend multiple linear regressions by shrinking coefficients, resulting in a final model where the outcome is associated with just a limited subset of predictors.

This study aligns with the assertion that statistical modeling necessitates a choice between parametric and non-parametric approaches [25]. Accordingly, we select models categorized into three types: parametric (beta regression), semi-parametric (generalized additive model), and machine learning (random forest). The parametric approach employs beta regression, which is effective for modeling doubly bounded response variables while ensuring that fitted values () remain within specified intervals. Parametric modeling, while simpler and more interpretable, has faced criticism when its strict assumptions are violated. For instance, if the true distribution is more complex than the assumed parametric models, this can result in potential under-fitting. In contrast, non-parametric modeling enables the data to be analyzed without strict assumptions, making it more suitable in cases where assumptions are violated and when outliers are present. A semi-parametric model is intermediate between parametric and nonparametric models and contains finite-dimensional and infinite-dimensional components. This study applies the generalized additive model- a semi-parametric model- in elucidating non-linear covariate effects. Recent advancements in artificial intelligence and machine learning have led to the application of various regression algorithms, offering alternative methods for achieving more objective and accurate regression modeling. Random Forest (RF) is a machine learning algorithm that mitigates over-fitting and offers a method for assessing feature importance in the prediction process. Within the domain of machine learning algorithms, we evaluated the application of RF due to its effectiveness in modeling both classification and regression tasks, its ability to mitigate over-fitting, and its proficiency in assessing important features that can be readily communicated to stakeholders. This study demonstrates that employing multiple modeling approaches enhances robustness and mitigates the risk of findings being artifacts of a single method’s assumptions, particularly when these approaches yield similar conclusions. Furthermore, due to the strengths and weaknesses inherent in each approach, a comparative analysis aids in balancing accuracy and generalizability, thereby establishing a foundation for selecting the most suitable approach for the analytical objective.

Beta regression

Since its introduction by Ferrari and Cribari-Neto (2004), beta regression models have become prominent in modelling doubly bounded response variables inside the interval (0,1) or [0,1], that are related to the predictors through a regression structure [26]. Like our developed composite indicator, other variables in that characterization of (0,1) encompass proportions, percentages, and rates. Beta regression has previously been used to model an evenness index and a straightness index- which are composite indicators [27]. Before its development, continuous doubly bound data were exclusively modeled using transformations and a normal error distribution [28]. Linear regression, however, is unsuitable for this data: i) it may generate fitted values for the response variable that surpasses its lower and upper limits; ii) as proportions or composite indicators, they are often asymmetrical, rendering inferences based on the normality assumption potentially misleading. Furthermore, due to the data modifications especially due to angular transformation, the model parameters are not readily interpretable in relation to the original response and, angular transformation performs poorly and should be avoided in cases where the denominators or sample sizes for the variables are different [29].

Beta regression accounts for heteroscedasticity by positing that the dependent variable follows a beta-distribution characterized by the mean and the dispersion parameters. The mean is associated with a set of covariates via a linear predictor with unspecified coefficients and a link function whilst the dispersion parameter may either be fixed or vary, contingent upon the set of covariates through a link function as well. Estimation in a beta regression model is conducted via maximum likelihood, with model parameters interpretable in terms of the mean of the response and, when the logit link is used, of an odds ratio [26]. Although there are no significant differences in model diagnostics compared to the previously utilized angular transformation models, beta regression yields more reliable parameter estimates, particularly when effect sizes are regarded as equally important as hypothesis testing [28].

With y₁, …, y_n representing a random sample such that y_i~ Beta (µ_i, ɸ), the beta regression model is articulated as delineated in equation (2):

(2)

where, β = (β₁, …, β_k)^T represents a k x 1 vector of unknown regression parameters (k < n), x_i= (x_i1,…, x_ik)^T denotes the vector of k explanatory variables and, the mean is presumed to be a linear function of the predictors [30]. Also, g(µ_i) is a link function g: (0,1) → ℝ, which is strictly monotonic and twice differentiable. Consequently, it is essential to acknowledge that beta regressions encompass two sub-models: one for the mean response and another one for the precision. As such, efficient parameter estimation in beta regression model depends on the correct modeling of the dispersion [31]. Cognizant of this requirement, it is important to model beta regression with varying and fixed dispersion and compare their performances. Along these lines, the merits of parsimony- where a model has to be selected as a subset of parameters gathered during the investigation, come into play. Model selection is crucial in regression analysis, as neglecting a significant predictor may result in biased outcomes, whilst the inclusion of irrelevant predictors can significantly impair estimation efficiency [32]. In this regard, three decisions during model selection are important, namely; i) assumption of the response distribution; (ii) selection of the appropriate link functions, and (iii) the determination of which linear predictor(s) to include in the regressions [30]. The appropriate specification of the link function can be evaluated by the mis-specification test, whereas the predictors can be examined using goodness of fit tests like the Akaike Information Criterion (AIC) or visual assessments of model fits for smaller datasets [27].

Notwithstanding the predictive strengths, beta regression is limited under the frequentist framework where the response variable y ∊ [0,1] [33]. To estimate observations existing at 0 or 1, or ordered beta, an inflated beta regression under the Bayesian framework is applied with four parameters [34]. The inflated beta regression models encompass a mixture of models where values of 0 and 1 are modeled using the logistic regression while the values in between are modeled using the beta regression.

Random forest regression

Within the machine learning framework, a random forest represents a specific category of ensemble learning algorithm [35] with numerous randomly generated trees, each casting a vote for a class [21]. Random forest (RF) regression consistently forecasts a continuous output by employing an ensemble of many decision trees [36]. The RF algorithm constructs a substantial ensemble of de-correlated decision trees by utilizing random selections of data and predictors, with each tree independently forecasting a result. The randomness in selecting training instances and input features introduces diversity among decision trees, facilitating the effective capture of relationships between input features and target variables [35]. This process de-correlates the trees, enhancing the robustness of the random forest against noisy data. In addition to effectively handling collinear data for final predictions, random forests aggregate the predictions of individual trees, thereby mitigating over-fitting and enhancing accuracy [37]. The strength of a RF process, is in the assessment of the importance between an input feature on the dependent variable, specifically its contribution to the overall model fit [38]. In ensemble approaches, variable importance is computed based on the mean square error reduction associated with it. An input feature that exerts influence on the output variable dwindles the model’s predictive accuracy. Mean decrease accuracy as an indicator of variable importance demonstrates the extent to which the predictive accuracy reduces with the exclusion of a variable from the decision trees [36]. Nonetheless, the more reduction in the prediction accuracy, the stronger the relationship between the feature input and dependent variable. Furthermore, random forests possess the advantage that decision trees can be utilized for both regression and classification tasks [19].

Given that its slowly gaining traction in data science, its inherent strengths in reduced over-fitting risk, and enhancing predictive power by returning important variables based on reduction in mean square error, we tested the application of RF regression to have a picture of which important factors influence the dependent variable.

Generalized additive models

Generalized additive models (GAMs) come in handy cognizant of the need to identify and characterize non-linear covariate effects. The linear predictor includes a sum of smooth functions of covariates, where the shape of the functions to be estimated is as shown in equation (3):

(3)

where the s_p‘s are the unspecified smooth nonparametric functions, and is the link function that links the expected value E(Y) to the covariates. Each function is fitted using scatterplot smoother and then an algorithm for simultaneously estimating all p functions is provided [20]. This flexibility implies that the predictive functions are unknown a priori but are uncovered by the modelling process of the underlying patterns in the data, thereby allowing an analyst to choose a fitting method appropriate for each explanatory variable. While other methods generally lack at least one of the three strengths, GAMs offer altogether, i) better interpretability, ii) flexibility-predictor functions can cover hidden patterns in data, and iii) regularization-helps to reduce over-fitting [39]. From equation (3), it is important to note that smoothers (local regression, smoothing and regression splines) are the defining aspects of GAMs for they allow balancing of the bias-variance trade-off.

Data, sources and management

Firstly, to obtain the dependent variable, this study constructed a composite indicator of service delivery using administrative data from the education, health and water sector for 136 districts/local governments in Uganda, including Kampala. The list of elementary indicators used to build the composite indicator is shown in S1 Table. Data points for the elementary indicators corresponding to the year 2021 were sourced from the Education Management Information System (EMIS), Annual Health Sector Performance Report-2022, Ministry of Water and Environment- Natural Resources, Environment, Climate Change, Land and Water Management Program Performance Report 2022, and the Uganda Bureau of Statistics website. For the process of CI building we refer to [40]. After a synthesis of literature on district local governments such as The Uganda Local Government Act 1997, Sector Service Delivery Assessments [11], third National Development Plan [41], and the Discretionary Development Equalization Grant (DDEG) Grant, Budget and Implementation Guidelines [42], we extracted potential predictors. This aligns with Chowdhury and Turin (2020), who posited that candidate variables for a specific topic can be selected based on subject matter knowledge or by reviewing the existing literature on the topic [43]. It is our position that these predictors would impact the performance of district local governments regarding service delivery. A brief description of the predictor variables is shown in Table 1. Data points for the predictors corresponding to the year 2021 were obtained from various sources such as budget dashboard [48], government finance statistics, and administrative geography file. These were already published data by mandated authorities. Because of deficiencies in data supply for all higher local government levels the data for the municipalities or cities were added to the district data for easier comparison. Descriptive statistics in form of measures of central tendency and of dispersion were performed to reveal the pattern and structure of the data [44]. This aided in identification and rectification of outliers and cases with missing values prior to bivariate and further analyses.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Description of the variables used in this study.

https://doi.org/10.1371/journal.pone.0338264.t001

Building of the composite indicator of service delivery

To obtain reliable differentials, it was important that a tested approach is adopted to construct a composite indicator of service delivery in line with the OECD recommendations [45]. Fig 1 is a flow chart detailing the general steps followed to build the composite indicator.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Steps followed to construct a composite indicator of service delivery, as synthesized from OECD guidelines.

https://doi.org/10.1371/journal.pone.0338264.g001

As shown in Fig 1, the construction of the CI started with the collation of data points for the selected elementary indicators by dimension. Exploratory data analysis was performed as a part of preliminary data analysis to determine whether a variable was symmetrical or not. In case of non-symmetry, the variable was winsored. Correlation analysis was performed as a data dimension reduction approach to signal variable redundancy within the dimension. Next, was the data normalization phase followed by first-stage aggregation to obtain the dimension sub-indices. The weighting of the sub-indices by testing the four methods was done in combination with either additive or multiplicative aggregation to obtain the CI scores that were ranked. Robustness tests using average shifts in ranks for the approach as well as the district local governments were performed, upon which the scores from the most stable approach were adopted as the values for the dependent variable.

Theoretical framework for public service delivery assessment

In the building of composite indicators, a robust theoretical framework delineates the multi-dimensional phenomena under measurement, its dimensions, pillars, justifies the selection of elementary indicators, and, in certain instances, the weights to be applied [45]. This paradigm is derived from a combination of two theoretical perspectives: the World Bank’s public administration production function [46] and the methodology for assessing public service delivery by Amin and Chaudhury (2008) [47]. According to Amin and Chaudhury, improved measurement and evaluation of service delivery occurs along the intended results chain from the governance intervention to the quality of services. The World Bank’s service production function highlights the importance of culture as a pivotal element around which service inputs and outcomes revolve. This is important to contextualize service performance measurements in terms of elementary indicators, as whatever is considered normal fluctuates by national settings and stage of development. Accordingly, we construe our theoretical framework as follows: Service delivery originates from triggers such as development frameworks, mandates, and policies that inform governance decisions made by policymakers at both central and local government levels. Consequently, experts in both state and non-state institutions execute these decisions by transforming service inputs into outputs provided to users at front-line service delivery points, including schools, health centers, roadways, and water points. Resultantly, through the contact between providers and seekers at delivery points, service-seekers cultivate an experience that manifests as an evaluation of the services’ fitness for purpose. This framework conceptualizes service delivery as a results chain, illustrating the logical links of cause and effect from inputs to processes, outputs, and ultimately outcomes. Under this theoretical framework, i) elementary indicators assessing service delivery across all levels of the results chain were incorporated into the CI, classified under three sectors/dimensions, ii) the dimensions are mutually exclusive, facilitating the initial aggregation to derive sub-indices, iii) global and national service delivery standards were incorporated using distance-to-reference point data normalization, iv) the objective of service delivery is to optimize supply to satisfy demand; thus, the service-seeking behaviors and experiences of active recipients were taken into account through the selection of elementary indicators, and v) the elementary indicators selected largely reflect to the context of a developing country. Where available, the service delivery standards are presented as ideal/utopian values for the elementary indicators (see S1 Table).

The primary stages of composite indexing are data normalization, weighting, aggregation, and robustness testing. The identification of utopian values, which would ideally serve as global service standards, is connected to the initial phase (development framework and mandate) of service delivery within our proposed theoretical framework. The distance-to-reference point approach of data normalization used these utopian/idealistic values for data transformation and facilitates worldwide comparisons in service performance. This study evaluated four weighing methods classified as data-derived and externally-derived. The data-derived weighting approaches include equal weights and principal component analysis, which are more empirical and typically yield robust CI rankings. Furthermore, the application of additive aggregation weights entails value judgments; thus, employing equal weights signifies equal significance of the elementary indicators within the dimension, as well as among the dimensions that constitute the overall CI. This is an important concern in contexts where service centers acknowledge all sectors as equally significant to service recipients, so expressing equality as a service objective. The externally sourced weights acknowledge the significance of context in influencing service delivery results. The financial analysis weights were determined by the proportion of the national budget assigned to each of the three sectors, acknowledging that the assessment of service performance should fluctuate with the level of financial inputs, as funding, ceteris paribus, is a significant factor in enhancing service outputs. The budget allocation weights were derived from experts (local government planning officers) to indicate the areas they deem crucial for enhanced service performance. This participatory approach facilitates the gathering of insights from specialists in the CI construction process [49] while ensuring equitable acknowledgment of service contributions.

Exploratory data analysis

We categorized the 20 elementary indicators according to three dimensions (see S1 Table) and thereafter analyzed the corresponding variables for descriptive statistics- mean, median, variance, skewness, kurtosis, and coefficient of variation. Variables that were asymmetrical (skewness>2 or kurtosis>3) were winsored. The Pearson correlation coefficient between any two variables within the dimension was obtained to guide in data dimension reduction using a threshold of less than −0.3 for highly negative, above 0.7 for highly positive correlation. Only two variables surpassed the thresholds, leading to the exclusion of one; thus, 19 variables were incorporated into the final construction model. Three dimensions- education, health and water- were selected for assessment because according to part two of the second schedule of the Uganda Local Government Act, 1997 (as amended) the three are the topmost mandated service sectors under the jurisdiction of the district local governments [13]. Moreover, the three sectors consume over 20% of the national budget [48]; thus, it is logical for service delivery assessments to prioritize outcomes in these sectors. Nonetheless, the composite indicator would have been enhanced by the incorporation of additional recognized service sectors, including justice, transportation, and electricity, along with a varied array of elementary indicators that thoroughly address service quality, quantity, infrastructure, access, and equity; however, this was constrained by the lack of district-level data for these themes and sectors.

Data normalization stage

The selected variables were normalized at the subsequent stage using any of the three methods- ranking, minimax or distance to reference point. Minimax normalization involves subtraction of the value from the minimum and dividing by the variable range. This method does not reshape the distribution of the data. Distance to reference point involves dividing the value with the reference value/point. For some variables, the reference point was obtained exogenously from service delivery standards or development framework targets, while for the rest it was the maximum value possible for that variable. The ranking method transforms the data into ranks by taking the position of the value relative to the highest performing value for that variable. This method reshapes the distribution of the variable. These three methods were applied as plausible scenarios in composite indexing to aid in testing the robustness of the composite indicator, but also to offer basis for comparison and selection of the most reliable combination of approaches.

Aggregation

At the first stage of aggregation, normalized variables were aggregated additively and with equal weighting within each dimension to obtain dimension sub-indices. The additive aggregation at this stage enabled compatibility of the values obtained through minimax and distance to reference point with multiplicative aggregation at the second stage.

At the second stage, dimension sub-indices with the independent application of four weighting schemes (equal weighting, principal component analysis, financial allocation, and budget allocation process) were aggregated additively and multiplicatively. Twenty-four sets of composite indicator scores and associated ranks for each district local government were obtained through a combination of methods as illustrated in the Table 2. The ranks allowed for the computation of the average shifts in ranks as a test of uncertainty/ robustness. The results showed that the constructed composite indicator is stable, but also provided an opportunity to compare the 24 sets of approaches to enable the selection of the most robust. The set that produced the lowest average absolute shift in ranks (minimax transformation- equal weighting-multiplicative aggregation) was considered the most stable, and therefore its scores were adopted as the composite indicator of service delivery for district local governments in Uganda [49]. It is this composite indicator scores that were used as the outcome variable for this study.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Sets of methods leading to the composite indicator scores.

https://doi.org/10.1371/journal.pone.0338264.t002

To determine the relationship between the outcome variable (CI) and the predictors, that is the variables that could explain the service delivery differentials in Ugandan local governments the three statistical approaches narrated in the preceding section – Beta, random forest regression and generalized additive model – were applied.

Some measures of central tendency and dispersion were performed for each of the independent variables to show patterns in the performance of the district local governments as shown in Table 2. Missing cases were identified and addressed by assigning median values for that variable. Outliers were determined using box plots and addressed in two variables (population size and central government funding selected arbitrarily) using the inter-quartile range method leading to a 124X17 dataset. Two-way tables were produced for categorical variables in addition to histograms for the distribution of continuous variables.

Considering that the composite indicator scores are akin to percentages in the range, y ∈ (0,1), beta regression, as reviewed from literature, is the appropriate statistical approach as shown by equation (2). We began by running the full model with all the sixteen predictors a fixed precision parameter and equally the same model but with a varying precision parameter in respect to some continuous variables. On the basis of their Information Criteria (IC), one model was selected however, this further necessitated regularization using step Beta function to select the important variables in pursuit of parsimony. The selected approach was used to explain the differentials in service delivery aided by the level of significance. The coefficients were corrected for bias in pursuit of efficient estimates [50]. Model diagnostics using tools such as Cook distance, standardized weighted residual plot, half-normal plots and local influence method were performed through residual plots. We tested the predictive accuracy of the selected model by learning from it with a training (70% of the observations) and testing datasets.

To perform the random forest algorithm, five (corresponding to m_try = p/3) randomly selected variables were used to split each node from the bootstrapped datasets to generate a decision tree in a random forest training. The minimum node size was 5. Mean decrease accuracy (MDA), was used to evaluate the importance of each variable in its impact on predicting service delivery in Ugandan local governments. A comparison of the fitted values for random forest regression and Beta regression regularized model was performed and it showed significant correlation. Whereas, the out-of-bag (OOB) error rate can conclusively be used to evaluate the predictive accuracy of a RF model with especially small sample sizes as it uses bootstrapping, in this study we as well used the training and testing datasets upon which 500 decision trees were built and an ensemble of results aggregated. This was intended to provide a consistent procedure for testing the predictive accuracy applied to the Beta regression and GAM in this study.

The dataset was subjected to a GA model through specification of the smooth functions for the continuous variables. Model regularization was performed in pursuit of parsimony. The selected variables/model were run against the training and testing dataset to evaluate the predictive accuracy. The functional forms for the covariates were observed through the plots and provided basis for reporting their trajectory and relationship with service delivery at local government level.

Beta regression model

The results of the Beta regression model with fixed and varying dispersion parameter are shown in Table 5 below. The model with fixed dispersion parameter generates four significant predictors that are the number of sub-counties, number of town councils, distance of the district headquarters from the capital city and land area. The model with varying dispersion parameter generates five significant predictors for the mean model namely, number of sub-counties, number of town councils, land area, central government and other funding from central government agencies. For the dispersion model, four significant factors are generated namely; the number of sub-counties, central government funding, local revenue mobilized, and distance of the district headquarters from the capital city. The signs of the coefficients are similar for the mean models in both cases. The Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and Pseudo R² show no significant differences in fit for the two models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Results of application of Beta regression.

https://doi.org/10.1371/journal.pone.0338264.t005

Variable selection is crucial as it enhances estimation accuracy by effectively identifying a subset of significant predictors, resulting in a parsimonious representation [32]. In terms of variable selection as shown in Table 6, the model with a fixed dispersion parameter identifies five variables: the number of sub-counties, the number of town councils, the distance of the district from the capital city, additional central government funding, and expenditure across the three sectors. The model with varying dispersion parameter identifies six variables (four are like the fixed dispersion model) namely- number of sub-counties, number of town councils, land area, age of the district, central government funding to the district and other central government funding.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Variables selected through model regularization.

https://doi.org/10.1371/journal.pone.0338264.t006

The differences are rooted in the methodological approaches of these selection methods. Stepwise regression employs information criterion as shown in equation (4) to penalize the number of predictors included in the stepwise regression so as to maximize model fit with the fewest predictors [51]. AIC and BIC have different penalty levels, for AIC has a fixed penalty of 2, while BIC’s penalty level- ln(N), that increases with the sample size.

(4)

where penalty(p) quantifies the penalty assigned to the model incorporating p parameters. This quantity may be interpreted as a measure of divergence between the distribution of future data generated by random variable Y and that predicted by the model. The choice of the penalty identifies the type of information criterion where for AIC it is −2log (y; ) + 2p while for BIC it is −2log (y; )+p log n.

The R² is one of the metrics used to evaluate regression models, as it offers insights into the proportion of variability in the data explained by the model. By comparing Tables 5 and 6, the R² is low and reduces with model regularization, which is its typical limitation in that it increases with the number of predictors even if they do not necessarily contribute meaningful improvements in the model. The results show the parsimonious model obtained a pseudo-R² value of 0.237 which is comparable to Cribari-Neto (2023) who in his study on beta regression analysis of COVID-19 mortality rates in Brazil, obtained a pseudo-R² value of 0.347 [52]. Nonetheless, he utilized the residual half normal plot to test for beta-regression model fitness. This is reasonable, because the R² does not pass as the absolute panacea for model fitness/quality as it is sensitive to model complexity (over-fitting and under-fitting), and ignores residual analyses. To this end, other fit statistics such as information criteria, predictive abilities [53,54] and visual diagnostic plots [28] which may be applied singularly or in combination are invaluable for understanding model fitness/quality.

Consequently, we based goodness of fit evaluation on the AIC index, and we choose the model characterized by varying dispersion and regularized via stepwise beta regression. To obtain efficient coefficients in beta regression, bias correction or reduction is applied. The coefficients for the selected model are shown in Table 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Results of final model’s efficient parameter coefficients chosen after correction of bias.

https://doi.org/10.1371/journal.pone.0338264.t007

From Table 7 and by using bias corrected coefficients, the number of sub-counties and land area have a negative relationship while Central Government funding from the treasury, funding from other agencies, number of town councils, age of the district have a positive relationship with service delivery. The intercept shows that when all predictors are zero the odds of service delivery outcomes (index) are reduced by about 27%. This implies that regardless of inclusion of the identified predictors, the service performance effect is below by 27%, suggesting the need for identification of more predictors to include in the model. The creation of an additional lower administrative units in the form of sub-counties, and keeping other predictors constant, reduces the odds of service delivery index by 3%. This could be ascribed to the increased cost of administering the sub-counties in the midst of low budgetary allocations. The financing for service delivery is diverted to cover running costs such as wages for staff, utilities, office space, travels, etc. The creation of an additional town council while keeping other factors constant increases the odds of service delivery outcomes (index) by 1%. Town councils are urban areas with higher population density making it easier for Government to provide services in a smaller geographical scope. Additionally, town dwellers are capable of mobilizing voices to demand for services than the rural counterparts. A unit increase in land size of a district reduces the odds of service delivery outcomes by 1%. Increased land area implies increased costs of service delivery supervision and monitoring, which may not be effectively realised. Predictors such as central government funding, other central government transfers and age of the district, though positive have very low odds ratios suggesting that a unit increase in their quantities changes the odds of service delivery outcomes by less than 1%. Of course, the funding element is an important input in service delivery especially for start-up of service points such as schools in areas where they do not exist.

The dual structure of beta regression modelling facilitates intricate analysis of data exhibiting heteroscedasticity, wherein both the mean and variability of the response are influenced by predictors. The precision model predicts the variability or precision of the response (service delivery composite indicator) utilizing logarithmic link. Two predictors- distance from the capital city and Central Government funding – have a positive relationship with the precision parameter while the number of town councils has a negative relationship with the precision parameter. A unit increase in the number of sub-counties decreases the log of the precision parameter by 0.006 units, thereby increasing dispersion (increasing variability) in the service delivery composite indicator. Conversely a unit increase in the distance of the district from the capital city increases the log of the precision parameter by 0.003 units, reducing dispersion (reducing variability) in the service delivery composite indicator. The central government funding has increased log of the precision parameter thereby reducing variability in the service delivery composite indicator.

Statistical analysis typically utilizes methods to assess whether a fitted model accurately represents reality. Accordingly, diagnostic tools utilizing residuals or influence metrics may be employed for that purpose [55]. The goodness of fit is assessed using different types of diagnostic displays shown in Fig 3a-3d.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Diagnostic plots for beta regression of service delivery composite indicator.

https://doi.org/10.1371/journal.pone.0338264.g003

Fig 3a and 3d employ standardized weighted residuals to account for leverages of the different observations and show observation randomly scattered, signaling absence of unusual features in the data which is a first indication that the Beta regression model with varying dispersion provides an appropriate representation for the data. From Cook’s distance plot in Fig 3b, it is evident the stability of the model and its coefficients. Observation 105 is characterized by a significant Cook’s distance and substantial residual. Therefore, to improve parameter accuracy, it may be excluded from the model in future studies. By employing the deviance, the plots in Fig 4a-4d further confirm the non-presence of substantial influential observations as shown by the half normal plot of residuals. Most of the residuals vary between 0 and 1.5; yet they all remain within the confidence bands as shown in Fig 4c, indicating a lack of definitive evidence for mis-specification of the Beta regression model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Diagnostic plots for the beta regression model based on deviance tests.

https://doi.org/10.1371/journal.pone.0338264.g004

In summary, the beta regression model with varying dispersion parameter has a proper fit for the data used to model the predictors of service delivery.

Selection of important variables using random forest regression

A random forest regression model was run using the composite score as the outcome variable and five as the number of variables tried at each split. 500 hundred trees were grown, the mean of squared residuals at 0.0029 and the percentage of variance explained at 20%. The error rate is minimal and stabilizes from about 50 trees grown as shown in S1 Fig, an indication that growing about 50 trees could have as well yielded similar results.

RF regression assesses the importance of the predictors regarding the contribution to model accuracy. Higher importance means more influence on predictions. According to Table 8 and S2 Fig, the top five important variables in explaining service delivery composite indicator are: number of sub-counties with a percentage increase in mean square error at 12%, followed by central government funding, district expenditure on the three sectors, the district population and whether the district is a classified as a high disaster-risk.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Variable importance as returned by random forest regression.

https://doi.org/10.1371/journal.pone.0338264.t008

Accessed literature suggests that when the increase in mean squared error (MSE) and the increase in node purity yield slightly divergent outcomes as seen in Table 8, the researcher should prioritize the results associated with the increase in MSE. This is because MSE directly relates to a model's predictive accuracy thus offering a more direct interpretation than increase in node purity.

A plot followed by correlation analysis of the fitted variables from the Beta regression model and predicted values of the random forest regression was significant at 0.05% level (ρ = 0.69, p-value = 0.00). Fig 5 shows a positive linear relationship hence suggesting that both approaches to some extent yield similar results, thus meriting their applicability in modelling service delivery in Ugandan local governments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Plot of the beta regression fitted values and random forest predicted values.

https://doi.org/10.1371/journal.pone.0338264.g005

Generalized additive model outputs

The coefficients of the GAM, after variable selection, are shown in S4 Table presenting significant effects for the predictors of number of town council, sub-county and central government funding. Notwithstanding the results and due to the underlying objective of GAM, it is often advised to make conclusions based on the plots.

Looking at the coefficients, sub-county and central government funding are the basis function weights that control the shape of these splines. For enhanced interpretability, these coefficients require a full understanding of what these basis functions look like and how they act collectively to form the spline, hence justifying the visualizing of the output using partial effects plots (Fig 6).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Plots from application of a generalized additive model.

https://doi.org/10.1371/journal.pone.0338264.g006

The plots in Fig 6 are necessary to visualize the individual predictor effects (one predictor when all other remaining predictors are dropped). The smooth plots for Central Government funding and number of sub-counties are slightly wiggly, which is uncovered through the GAM. Further, the diagnostic plots in S3 Fig confirmed that it is a good fit for this data.

As shown in Fig 6, the plot for the Central Government funding confirms that the effect of this predictor is slightly negative for lower levels of funding but quickly becomes positive (above zero) for intermediate amounts of funding (about UGX 2.5 billion). It increases gently and then plateaus at less than 0.05 (service delivery outcome/indicator) with increased funding. Refugee hosting districts have on average higher service delivery outcomes than their non-refugee hosting counterparts. This may be attributed to the additional external financing provided by mainly development partners to start-up and maintain services in refugee hosting communities. Low numbers of sub-counties are characterized by high service delivery outcomes, which quickly reduce with increased administrative units(sub-counties) up to about 20, and thereafter the service delivery composite indicator slightly increases with the number of sub-counties. The number of town councils have a gently increasing effect on the service delivery outcomes.

Comparison of the predictive accuracy of the three approaches

Comparison of the three approaches was achieved by learning from a consistent training dataset using the subset of variables returned by each approach, and then evaluated using the testing dataset. The results are reported in Table 9.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Comparison of predictive performance of the four approaches.

https://doi.org/10.1371/journal.pone.0338264.t009

Table 9 shows that the predictive accuracy as presented by the mean square (MSE) or mean absolute error (MAE) for the four models is almost similar. This shows that, regardless of slightly different variables selected through regularization, any choice of the approach yields comparable predictive accuracy.

Study limitations

The primary constraints of this study pertain to the insufficiency of elementary indicators and data points for evaluation. Due to the lack of accurate data at the sub-national level, fewer service sectors than required were incorporated into the model. The composite indicator would be more representative if it incorporated data from all tiers of the established theoretical framework, particularly regarding service processes. The dynamic nature of service delivery may introduce biases in the study, as it relied on annual data series, perhaps overlooking crucial service characteristics that fluctuate more frequently.