Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Exploring the impact of a personalised disability reform on people with disability and their primary carers: Evidence from the Australian national disability insurance scheme

  • Bernice Hua Ma ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing

    bernice.ma@monash.edu

    Affiliation Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia

  • Samia Badji,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliation Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia

  • Gang Chen,

    Roles Conceptualization, Supervision, Writing – review & editing

    Affiliations Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia

  • Dennis Petrie

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation Centre for Health Economics, Monash University, Caulfield East, Victoria, Australia

Abstract

Australia introduced the National Disability Insurance Scheme (NDIS) in 2013 to provide personalised formal care to individuals under 65 with significant and likely permanent disability. However, many ineligible individuals now face challenges accessing care. Against the backdrop of the introduction of NDIS funding and the simultaneous defunding of other disability services due to the NDIS, this research investigates the short-term impacts of NDIS on the formal service utilisation and carer outcomes for people with profound or severe disability, irrespective of their NDIS status. Using the staggered NDIS rollout, we analyse data from the 2015 and 2018 Survey of Disability, Ageing and Carers. We compare outcomes between primary carers and care recipients in NDIS-available areas (n = 736) and NDIS-not-yet-available areas (n = 318). Results show no short-term impact of NDIS availability on formal service utilisation or frequency, or primary carer outcomes. While some individuals benefit from the NDIS, others may lose access to care. Policymakers should address NDIS equity concerns and consider targeted measures to improve carer outcomes.

Introduction

There is an international trend towards the personalisation of funding for care services to enable more appropriate and higher quality care. Under this approach, funds are given to people with disability so they can purchase those services that best suit their needs. Australia joined the trend in 2013 by launching the National Disability Insurance Scheme (NDIS) to provide more personalised services to people under 65 with a significant and likely permanent disability. While focusing on supporting people with disability, the NDIS also recognises the essential role of carers and includes them in discussions about support.

The cost of the NDIS is substantial. The projected total Scheme expenses in 2023–24 are AU$41.4, representing 1.6% of the national Gross Domestic Product [1]. The NDIS was particularly targeted at supporting those with high on ongoing care needs. Implementing the NDIS meant not only additional funds to support people with a disability but also a reallocation of public funds away from some national and state or territory-based disability programs that had supported the wider population of people with disability. This included programs like Partners in Recovery, Personal Helpers and Mentors and Day to Day Living [2,3]. Many local governments also ceased providing Home and Community Care programs for adults with disability [4]. As these programs have been scaled back or defunded [5], people not on the NDIS have faced a lack of support, leaving their families as their main source of support. As shown in Fig 1, by August 2018, the NDIS had enrolled a total of 183,965 Australians, which is only one-fourth of the estimated population of people with severe and profound disability under 65 [6,7].

thumbnail
Fig 1. The composition of people with disability in Australia as of August 2018.

https://doi.org/10.1371/journal.pone.0321377.g001

Previous evidence suggested that the NDIS improved outcomes for its participants and carers. For example, participants reported using more formal services to support domestic tasks and seeing specialists more [8]. Their carers also reported taking more frequent breaks [9]. The employment rate also increased for carers of participants under 25 years who had been in the scheme for over a year, with the 15–24-year-old participant group having the most significant improvement [10,11].

The impact of the NDIS on people with severe and profound disability and their carers remains uncertain. This raises significant equity concerns, particularly for those who have not been able to access the scheme. Addressing this evidence gap, this paper estimated whether providing personalised formal services to eligible people with disability in place of access to state-funded organised services for a wider disability group increased formal care use and improved carer outcomes in the short run.

The study uses repeated cross-sectional data (waves 2015 and 2018) of the Survey of Disability, Ageing and Carers (SDAC) [12] to perform an intention-to-treat study. In the first stage, we assessed the changes of the utilisation of formal services for people with disability, as this is a major pathway to change caring hours thus relevant carer outcomes. Then we consider the further downstream impacts on carer outcomes themselves where we also examine the average changes in informal caring hours, employment, and social participation for primary carers. SDAC is useful because it is the only Australian data that was being regularly collected on those with a disability and their carers both prior to and post the NDIS implementation. However, utilising SDAC data to evaluation the NDIS could be challenging especially because of its sample size, which may constrain our ability to interpret the results as being causal. Therefore, we also perform a sample size power calculation to determine what would have been the necessary sample size to detect various possible magnitudes of change.

We first discuss the literature and describe the possible mechanisms through which the NDIS may impact formal service use and carer outcomes. We then lay out the empirical framework, including the data, the identification strategy, and model specification. Finally, we present the results of our analysis and discuss the policy implication.

Background

The NDIS

The NDIS provides support to people with permanent and significant disability who enter the scheme before age 65. People with disability usually need their health care provider to prepare evidence stating that their disability is permeant and significant. After gathering the information, they need to submit the evidence to the NDIA. It is ultimately up to the NDIA to decide if an applicant’s disability is permeant and significant and thus meet the eligibility criteria of entering the NDIS. Unlike the previous system, which assumes a one-size-fits-all approach and funds services providers directly, the NDIS considers each individual’s personal goals and preferences, then tailors and provides funds for each individual to spend on the services they need. Three types of support budgets can be funded in one’s NDIS plan: the core supports budget, the capacity building supports budget, and the capital supports budget. The core supports budget, in particular, funds assistance such as self-care activities, household tasks, therapeutic supports not covered by the mainstream health system, and transport if one cannot use public transportation. In addition to the government-funded programs available to support carers, an NDIS participant might use funding in their plan to facilitate respite care, which gives carers short breaks from caring responsibilities. All these services are referred to as “formal services” in this paper, services that are professional, regulated, and typically involve monetary transactions.

There have been several stages of the implementation of the NDIS. It was first tested in trial sites in five states/territories in 2013. Prioritisation was given based on age or residential location. The trial ended in July 2016, and since then, the NDIS began to roll out across Australia (except for some early roll-out areas in NSW). ACT was the first state or territory to complete the roll-out, followed by other states and territories, with Western Australia being the last state to reach the full scheme from July 2023. Details regarding the trial and roll-out dates of the NDIS are listed in Appendix S1Table.

There are several steps to receive a support budget from the NDIS. First, one has to apply for the NDIS with the evidence to prove that they meet the eligibility criteria of entry. Then the National Disability Insurance Agency (NDIA) assesses of the minimum eligibility requirements, and will notify the applicant if they are granted access to the NDIS. Planning meetings between an NDIA planner and the participant will take place if access is granted to discuss how much support the participant is likely to need and a plan will be made accordingly. The NDIA then reviews the plan. Once the plan is signed off by the NDIA, participants can arrange their services [13]. The NDIS participants could choose whether their NDIS plan is managed by themselves, by the NDIA or by a plan manager. About 70% of people with disability choose to use a plan manager who helps them with reimbursement and fund monitoring but does not support coordination [14]. About 40% of NDIS participants also have support coordination funded, which helps them coordinate support and services; however, research showed that only about 20% of the participants were able to use at least 80% of their support coordination funding [15]. For those NDIS eligible, the waiting time to get an approved plan could be long, especially during the early stages of the NDIS roll-out. It was estimated that it took more than seven months for a participant to get an approved plan before July 2019 [11], not to mention the time needed to arrange services.

Mechanisms

Fig 2 illustrates the potential mechanisms through which the NDIS roll-out may impact carer outcomes through its impact on formal service utilisation, as the latter is a core funding area of the NDIS. Our study examined the average effect of the NDIS by comparing two key groups – people with profound or severe disability living in areas where the NDIS has been rolled out, and those living in areas where the NDIS has not been rolled out. Given that formal services are a core area funded by NDIS, eligible people living in areas where the NDIS is available could receive more and higher quality formal services and, consequently, the average formal service use in the NDIS available areas may increase (Path A, Fig 2). The literature has competing views on whether increased formal caring hours substitute, complement or are independent of informal caring hours. According to the substitution theory, an increased supply of formal care may decrease informal care [1620] (Path A1, Fig 2). However, if formal care is provided for tasks that informal carers cannot perform [21] or if it is provided only when the informal care has been exhausted [22], the informal care may remain unchanged even if the formal care increased (Path A2, Fig 2). As highlighted in the NDIS guidelines, the NDIS was not intended to replace informal care; in such circumstances, formal care and informal care may be independent. It is also possible that NDIS increased the informal caring hours as the complementary theory postulates (Path A3, Fig 2). There is not much evidence about the reinforcing effects of formal care. However, more informal care may be needed to permit access to the formal care being provided (e.g., if one needs help to undertake formal care activities). Also, at the early stages of the NDIS roll-out, carers may have needed to spend more time planning and coordinating the approved services for the recipients; therefore, their caring time may have increased.

thumbnail
Fig 2. The potential mechanism through which NDIS may affect caring hours, employment and social participation of primary carers.

https://doi.org/10.1371/journal.pone.0321377.g002

Those ruled ineligible for the NDIS but who may have previously accessed state/territory-based services may now face difficulties in accessing formal services. As a result, the average use of formal services in the area may not change (Path B, Fig 2) or decrease (Path C, Fig 2) even if the NDIS has become available. The caring hours are only expected to change if there is a change in the use of formal services. However, if the formal service use decreases, the caring hours may need to increase to fill this gap.

Given the changes in caring hours, the NDIS may or may not affect labour force and social participation outcomes through the change of caring hours (Path A1 and A3, Fig 2). It is possible that the decreased informal caring hours free up some of the informal carer’s time so that informal carers can work or participate in social events because previous research found that caregiving is associated with higher rates of part-time work and fewer hours of work [23,24]. However, if the NDIS increased informal caring hours, the carers may be less likely to be employed or enjoy the same leisure time simply because they have no free time. There is also a possibility that the NDIS influences employment and social participation outcomes through other channels that are independent of the caring hours. For example, family support and counselling may help the family of people with a disability to manage the demands the disability brings. If mental pressure prevents carers from participating in economic or social activities, then the NDIS may indirectly support carers in this way as well.

This study examines changes occurring within a relatively brief period of 2.5 years maximum ---- a duration considerably shorter compared to the 10-year milestone reached by the NDIS. The impact of the NDIS may be different in the short term compared to the long term. While the impact on caring hours and social participation may show relatively quickly after getting formal care, the effects on employment may take longer to manifest. Also, the impact on employment may depend on the flexibility of available jobs — for example, the number of part-time jobs in the labour market.

Methods

This study estimates the impact of the NDIS on the use of formal services of people with disability who have a profound or severe disability (or needing aids) under the age of 65, as well as the caring hours, employment and social participation of their primary carers before and after the NDIS has become available in their areas. To minimise bias from factors other than the intervention, we compare people with profound or severe disability (or needing aids) under the age of 65 who lived in areas where the NDIS are available to a similar control group but who were living in areas where the NDIS was not yet available to them. We also control for the characteristics of the primary carer and recipient dyad and area-level fixed effects. The analysis is run on a repeated Australian cross-sectional survey—Survey of Disability, Ageing and Carers (SDAC), the most detailed and comprehensive source for examining people with disability and their carers in Australia.

Data

Household level dataset: survey of disability, ageing and carers (SDAC).

This study uses mainly the cross-sectional data from SDAC waves 2015 and 2018 which each include approximately 20,000 responding households. These are the only SDAC waves that contain information about an individual’s local government area (LGA)—the third level of government in Australia at the time of analysis.

Self-constructed data on NDIS availability at the LGA level.

We use the LGA to determine if the individual is in an area where the NDIS has become available. The NDIS availability date for each LGA is extracted from the agreements the states and territories signed with the Commonwealth government.

Local unemployment rate.

We also control for the local unemployment rate in our analysis and use data from the Australian National Skills Commission to extract this information (National skills commission, 2022).

Ethics was approved by the Monash University Human Research Ethics Committee (MUHREC) (Project ID: 21758). The authors gained access to the data on 10th May, 2020.

Sample

Identification of the primary carers and recipient(s) dyad.

We match each primary carer to their care recipient. Some primary carers in SDAC cared for more than one recipient, and some recipients had multiple primary carers. In this situation, we kept only one primary carer and recipient pair, prioritising those identified as in a primary carer and main recipient relationship (85%), followed by the primary carer and non-main recipient relationship. We also restricted the sample to those recipients potentially eligible for the NDIS who have a profound or severe disability or need an aid and are under the age of 65. Our final sample comprises 1,054 primary carers of recipients with profound or severe disability (or needing aids) under 65. Further details of the sample size can be found in the Appendix S1Fig flowchart.

Treatment and control groups.

The NDIS trials and roll-out targeted different population groups. Some states or territories prioritised certain ages, while some were geographical.

Both the treatment and control groups consist of people with profound and severe disability or who need an aid/s who meet the age criteria and their primary carers. The primary carers are aged over 14 years and live in the same private dwelling with their recipient of care. The treatment group are people who lived in the LGAs where the NDIS became available between the two waves of SDAC interviews, i.e., 1 January 2016 and 30 June 2018 (“the treatment period”). In the control group, people lived in the LGAs where the NDIS became available only after the 2018 SDAC interview, i.e., after 1 July 2018 (“the control period”). We exclude people living in the LGAs that received the NDIS before 31 December 2015 as we do not have relevant data before 2015 (i.e., they were always treated). We also excluded people from Western Australia (WA) because WA ran its own trial, which extended into our analysis period, and the NDIS delivery model in the trial differed from the other states and territories.

Details of the timeline are presented in Fig 3, and the details of the trial and roll-out criteria can be found in Appendix S2 Table. Characteristics of the treatment and control groups are presented in the following subsection.

Outcome of interest

As the carer’s outcomes can be changed through the NDIS’ impact on formal services utilisation of the care recipient, this paper focuses on two sets of outcomes: the first set encompasses formal service use by the care recipients and the second set, the outcomes of the primary carers.

For the first set of outcomes at the care recipient level, we use questions about the frequency of assistance received from organised services because of their disability. Health care services are excluded as we cannot distinguish from the survey which ones are covered by the health system (Medicare) versus funded by other means. The survey participants were asked: “On average, how many times per [day/week/month/year]?” if they stated they needed organised care. The frequencies are transformed into ‘per week’ amounts if the original answer was provided in times per day/month/year assuming there are 52 weeks in a year. For example, if the answer is three times per month. Then we assume that it is 36 times (12months *3) per year. This is then converted to 0.69 = 36/52 times/week. We consider the overall numbers of formal services along with the extensive margin (i.e., whether they used any formal services) and the intensive margin (i.e., how many formal services they used when they had used at least some).

In the second set of outcomes at the carer level, our main outcome of interest is the weekly caring hours of primary carers for recipients. We also explore employment and social participation as those will likely change through the change in caring hours. We focus on co-inhabiting primary carers because they tend to care for longer hours, and the NDIS may have the greatest impact on them. Primary carers consist of 35% of all carers in our data, and they were asked: “On average, how many hours do you spend each week providing care to (recipient X)” This variable is converted into a continuous variable from a 6-level categorical variable (<9, 10–19, 20–29, 30–39, 40–69, 60 + hours) using mid-point transformation.

We also analyse the impact on employment and social participation of the primary carers. Our employment indicator is equal to one if the respondent reported they were employed full-time or part-time (and zero if they were unemployed, looking for work or not in the labour force). For social participation, we look at both social participation without the recipient (one if the primary carer had been to certain places or events in the last 12 months without the recipient and zero otherwise) and social participation with or without the recipient.

Further details of how the variables are constructed are listed in Appendix S3Table.

Covariates

We control for several covariates likely to be associated with our outcome of interest but unrelated to NDIS access to increase our estimation precision. The covariates include two sets. The set of carer characteristics includes age, sex, education level and the number of recipients of care. The second set is recipient-level characteristics and includes age, sex, marital status, education level, whether born in Australia, disability severity levels, disability types, the Accessibility Remoteness Index of Australia (ARIA) to reflect the ease or difficulty people face accessing services, the number of adults (aged 15 or more) without disability living in the household, and the number of bedrooms in the household. The last one is regarded as a proxy for the income level of the household, as the income variable has 20% of missing values. Unemployment rates by LGA one year before the start of each wave of survey (September 2014 to June 2015; September 2017 to June 2018) are also controlled for to account for local labour market conditions.

Estimation strategy

To start with, we compare the impact of the NDIS on formal service use of people with profound or severe disability (or needing aids) under 65 living in areas where the NDIS became available between January 2015 to June 2018 and the caring hours, employment and social participation for their primary carers to those with the same characteristics but living where the NDIS was not available until after July 2018 (i.e., after when the 2018 SDAC interview took place). Between the first quarter of 2015 and the second quarter of 2018, the NDIS covers approximately one-fourth of the total population of individuals with severe and profound disabilities in those specific areas. Given that our data does not include information on actual NDIS enrolment, the results represent intent-to-treat (ITT) estimates. ITT estimates are informative for understanding the population-level impact of the policy change. In our case, the results show the change of outcome attributed to the NDIS roll-out (and withdrawal of other services) on all the people with profound or severe disability or needing an aid (aids) who meet the age criteria in the NDIS available areas.

Our identification of the NDIS impact relies on LGA variation in the timing when the NDIS becomes available. Therefore, we need to ensure that the impact of the NDIS on our outcomes is not attributed to other LGA-level factors associated with the availability of the NDIS. We use LGA fixed effects to control for the potential correlation between the NDIS roll-out and time-invariant unobservable LGA characteristics. Simple pre- and post-treatment comparisons could also be influenced by the time trends in the outcome variables and other national policy changes or events; therefore, we then estimate the effect by applying the difference-in-difference (DiD) method using OLS regression. Our specification is as follows:

(1)

For the formal service outcomes, represents the outcome for the overall number of formal services used, the intensive and extensive margin of the formal services of recipient j living in LGA 𝑙 during wave 𝑙. For the carer outcomes, the dependent variable y corresponds to outcomes of carer i of recipient j living in LGA 𝑙; during wave 𝑙. The outcomes include weekly caring hours, employment and social participation. 𝑙𝑙𝑙𝑙𝑙 equals one if the outcome is measured in 2018 and zero otherwise. Til equals one if people are in the treatment group and zero if they are in the control group.

The parameter of interest is 𝑙1, representing the impact of the NDIS on the treatment group relative to the control group after the NDIS became available. This assumes that the differences in the outcomes are attributed to the random-like roll-out of the NDIS participation at the LGA level, not other factors.

We also control for the characteristics of the recipients 𝑙 and the primary carers 𝑙 and the unemployment rate, which are included in vector Z𝑙𝑙𝑙. Time-fixed effects 𝑙𝑙 control for common nationwide policy changes such as changes in welfare payments, the availability of new medicines and other nationwide trends between the two waves. The LGA fixed effect 𝑙𝑙; controls for any time-invariant differences across LGAs that may be correlated with our outcomes, such as demographic composition, supplier market, disability coordination facilities, cultural values, or economic conditions which are crucial factors that may impact our outcomes. 𝑙𝑙𝑙𝑙𝑙 is the error term. We do not further match our sample based on characteristics such as age and gender within each LGA as suggested in other literature [25] because there are, on average, only 5 observations per LGA.

The previous evaluation reports on NDIS found different effects for different age groups, and positive employment changes were observed in the carers of participants aged 15–24 [10]. Therefore, we also perform a subgroup analysis for recipients aged 15–24 versus those aged 0–14 and 25–64, who are within the NDIS eligible age criteria.

A key assumption in DiD analysis is that in the absence of the treatment, the trends in the outcomes between the treatment and the control groups would have been the same (the parallel trend assumption). In our study, we cannot test this assumption, given that only two waves of data are available. In the following section, we examine the robustness of the results to a different control group—primary carers of recipients who are not eligible for the NDIS (e.g., the disability status of the recipients is not profound or severe, or they are aged 65 years and older) and they live in same LGAs where the NDIS has not been rolled out.

Given that repeated cross-sectional data is used, we need to check that the time-invariant characteristics of the participants in our sample did not change dramatically between the two waves to perform a valid DiD analysis. Two-sided T-tests and Chi-square tests are performed to compare the characteristics of the primary carers and recipients in our treatment and control group between the waves.

We perform multiple robustness tests for our carer outcomes, including sensitivity to the region-level fixed effects, cut-off dates and different sample restrictions. In order to minimise the impact of different characteristics of the sample on the results, in one of the robustness checks, we use propensity score weighting to weight the entire sample from Wave 15 and the control group from Wave 18 against the treatment group of Wave 18 for characteristics including age, sex of the carers, and the sex, disability status, disability type, marital status, education level, whether born in Australia, number of bedrooms, number of adults without disability in the household, and the rurality (Accessibility/Remoteness Index of Australia or ARIA) of the recipients of care’s residence. We bootstrapped the sample 1,000 times to account for the uncertainty around the weights and the estimates of the weighted sample. We also interacted individual characteristics with the treatment variable (𝑙𝑙𝑙𝑙𝑙 #T𝑙𝑙) and estimated the average effect to account for potential heterogenous treatment effects.

We also ran a placebo analysis with primary carers of people with disability who are not eligible for the NDIS because they are older than age 65 or who have mild disability using our treated and control group LGA. This estimates whether there is any effect for those not targeted by the NDIS.

Minimum sample size estimation

Given that the SDAC survey was the most comprehensive source of information available to understand different effects of the NDIS, we wanted to understand how big the SDAC sample frame needed to be in order to detect economically meaningful effect of the NDIS.

We therefore hypothesized several scenarios of different impact sizes and computed the minimum cluster size needed to detect these potential impacts.

SDAC’s sampling is clustered at the LGA level. We therefore computed cluster size conditional on the maximum number of LGA possible. There were 407 LGAs in Australia in 2018 - excluding the state of Western Australia which had its own NDIS trial at different periods. We also excluded three LGAs in other states because they were also the early trial sites (Hunter NSW, Barwon VIC and Barkly NT) and therefore 404 LGAs remained.

We assumed an equal number of LGAs in the control and treatment groups to compute the minimum number of people per LGA under different assumptions of intraclass correlations that would be needed to detect meaningful changes in outcomes. We used the means and standard deviations of the control group in the post treatment period, as a reference point for all computations - last column in Table 1. All results are for an 80% power test using a 5% two-sided test.

Results

Descriptive statistics

Table 1 presents the summary statistics of the sample by the NDIS participation group and wave, and Appendix S4 Table shows the results of our comparison tests. Of the 1,054 primary carers and recipient dyads, 439 are in the treatment group observed in wave 2015, 297 are observed in wave 2018, 182 are in the control group observed in wave 2015, and 136 are observed in wave 2018. The proportion of male recipients is higher in the control group observed in 2018 than in the other groups. There are differences in the number of bedrooms among the groups, although the effect size is small (max 0.3 rooms). People are more likely to live in major cities if they are in the treatment group. The average age of the primary carer was two years older for both the treatment and the control groups in 2015 than in 2018. There are fewer adults without disability in the household in the treatment group observed in 2015 than in other groups.

Main results

The estimated impact of the NDIS roll-out on the primary carers’ outcomes and the formal service used by people with disability are shown in Table 2. The coefficient for the variable NDIS available area# Wave18 reflects the absolute additional change in the outcomes between 2015 and 2018 for those who live in the LGAs where the NDIS had been rolled out between 2015 and 2018 (the treatment group) versus where the NDIS had not yet been rolled out (the control group). The outcomes include (1) overall number of formal services, (2) the extensive margin of formal services, (3) the intensive margin of the formal services, (4) weekly caring hours, (5) employment status, (6) social participation without recipients, (7) social participation with or without recipients. The complete list of covariates and results are provided in Appendix S5 Table.

thumbnail
Table 2. The association between NDIS availability in an LGA and changes in the outcomes of interest.

https://doi.org/10.1371/journal.pone.0321377.t002

We did not detect statistically significant NDIS impacts on any of the outcomes. The results show that the people with a profound or severe disability (or needing aids) who live in areas where the NDIS became available are, on average, nearly 10% less likely to use formal services compared to their counterparts in areas where the NDIS had not yet been implemented after the reform; however, the result is insignificant. For those who use formal services at least once per year, the NDIS reform did not lead to a statistically significant difference in the frequency of formal service use between the treatment and the control group.

For the carer outcomes, the weekly caring hours of primary carers increased by 4.8 hours between 2015 and 2018 for the carers in the NDIS area group relative to the control group, though it was not statistically significant. We find that primary carers in the treatment group are five percentage points less likely to have part-time or full-time work after the NDIS roll-out (β= -0.053, SE = 0.081) compared to the control group, but this result is not statistically significant. We find no statistically significant impact on participation in any social activity (β = -0.067, SE = 0.079) or participation in social activity without the recipient (β = -0.097, SE = 0.063). We then explore if the decrease in social participation is driven by the change in the social participation of recipients of care. This may reduce the chance of the primary carer accompanying the recipient to those events. The results (Appendix S6 Table) do not show a statistically significant effect (β = 0.101, SE = 0.115).

While previous reports have suggested possible gains for carers of young people, our subgroup analysis shows no significant heterogeneity in results for the 15-to-24-year-old age group (Appendix S7 Table).

Robustness checks and placebo test

We conduct multiple analyses to explore the sensitivity of our results on the carer outcomes to various levels of fixed effects, cut-off dates, sample restrictions and sample weightings. A summary of the robustness test results is presented in Table 3, and fully separated results are shown in the appendix. Generally speaking, all the results consistently show no significant effects. Although some of the results changed sign, as they stay statistically insignificant, it could be driven by random variation rather than meaningful effects because of the limitation of small sample size. First, due to the concern that the characteristics of our 2018 sample may differ from the characteristics of our 2015 sample, we weighted our sample based on the characteristics of the NDIS participants in the 2018 sample as described in Section 3.6 and bootstrapped the sample 1,000 times. The results are similar to our main analysis using the unweighted sample. We do not find statistically significant changes across the seven outcomes. (Appendix S8 Table). We then explore if the fixed effect at the regional level instead of the LGA level would impact the statistical power of our analysis. Since the NDIS roll-out is based on the regional level information, then broken down to the LGA level in our main analysis, the treatment and control groups remain identical. With the regional fixed effects, we find similar trends with a negative coefficient for employment and social participation and an increase in caring hours. All the results are statistically insignificant (Appendix S9 Table).

thumbnail
Table 3. Robustness checks and placebo test: estimated association between NDIS availability and outcomes under alternative assumptions.

https://doi.org/10.1371/journal.pone.0321377.t003

The primary carer may suffer short-term negative impact when the NDIS is being coordinated. Therefore, we also explore if the results differ for those whose areas that had NDIS available for at least one year. We vary the cut-off date for those considered to be treated from 30 June 2018–30 June 2017 when constructing T𝑙𝑙, and keep the same control group to ensure that the differences in the results are not due to the change in the control groups. The results in Appendix S10 show that people living in areas where NDIS was available before June 2017 were 14% less likely to participate in social activities with or without the recipients of care, and the result is significant at the 10% level (β = -0.135, SE = 0.071) (Appendix S10 Table)

Primary carers may spend more time caring for the main recipients, and therefore, the effect size may become larger if the sample is restricted to main recipients only. We restrict our sample to the primary carer and main recipient relationship by excluding the non-main recipients in our sample (7.0%). The results are similar to our main analysis except that the possibility of participating in social activities with or without the recipients becomes significant at the 10% level (β = -0.115, SE = 0.065) (Appendix S11 Table).

About 10% of our primary carers in the sample are themselves eligible for NDIS and live in LGAs where the NDIS was available between 2015 and 2018. Therefore, there is a concern about whether the carer’s potential participation in the NDIS would also impact caring hours. We ran a sensitivity analysis restricting our sample to carers not eligible for the NDIS or living in LGAs where the NDIS is unavailable during the treatment period. Our main results were robust to this approach (Appendix S12 Table).

We also examine the robustness of the results with a different control group—primary carers of recipients who are unlikely to be eligible for the NDIS (because of age and disability severity level) and live in the same areas as the treatment group in our main analysis (areas with NDIS). This was done because we could not test for the parallel trend assumption, and we expected the results to be similar because the NDIS should also not affect this alternative control group. All the results remain statistically insignificant, although the outcomes of employment, social activities, and the frequency of using formal services are slightly above zero (Appendix Table S13).

Lastly, we examine the heterogeneous treatment effects by interacting the carer and recipient characteristics with our treatment term. We find no significant change to our main results (Appendix Table S14)

The results of the placebo analysis with the population of people with mild or moderate disability or older than 65 show no statistical changes in the outcomes. It indicates that the NDIS reform has no or minimal effects on the population, which is not targeted, consistent with our expectations (Appendix Table S15).

Minimum sample size estimation

We wanted to understand for different possible impacts, how big SDAC would have needed to be to detect meaning changes in the outcomes. The sample means in Appendix Figs S2aS2e provide a graphical representation of the minimum sample size for three possible impacts, and considers a range of values of intraclass correlations for all outcomes. For example, taking S2a Fig, in blue we see that the minimum sample size needed to detect a difference of one hour of formal services per week. This effect size cannot be substantiated as significant when rho is larger than 0.1 as can be seen in the figure. For smaller rhos there may still need over 100 people per cluster to detect a statistically significant one-hour difference. Given that we have on average five observations per cluster, the SDAC survey would need to be much larger to detect a difference of one hour per week. S2c Fig shows that to detect a 10-percentage-point increase in employment we would only need a sample size of approximately 50 observations in each group even for large intraclass correlations. Generally speaking, the SDAC survey which is currently the only survey capable of evaluating the impact of the NDIS on carers would have benefited from having a substantial increase in sample size prior to the NDIS roll out sot that this multi-billion-dollar scheme could have been properly evaluated.

Discussion

While one of the goals of the NDIS is to increase personalised formal service use and the wellbeing of the carers, the actual effects on the population level of those with profound or severe disability under the age of 65 are less certain given the replacement of one system with another. Addressing this evidence gap, this study used the staggered roll-out of the NDIS to examine changes in primary carers’ outcomes, including their weekly caring hours and employment and social participation, and investigated if the carer outcome changed due to the change in the use of formal services. It’s important to note that our discussions below were constrained by the limited evidence available due to the small sample size of the SDAC.

We find that formal service use for those people with a severe or profound disability (or needing aids) is similar before and after the NDIS becomes available. As discussed in the introduction, state or territory disability support programs were scaled back when the NDIS became available in the area. This has created challenges for the people who still need to be added to the NDIS but are living in NDIS-available areas and who may have been able to previously access state-based services [5]. In particular, at the early stages of the NDIS implementation, the process of getting an approved plan for the NDIS was so long that even eligible people who had applied would wait for more than seven months [7]. In addition, early research also reported that the supply of disability support services had not developed adequately to satisfy the extra needs created by the NDIS [9], especially in remote areas where the market for formal caring services is thin. Research also shows that NDIS participants with lower socioeconomic status or psychosocial disability, who are Aboriginal and Torres Islanders or who belong to a Cultural and Linguistic Diverse group have lower utilisation rates of the NDIS plan than their counterparts [15,26,27]. This extra barrier of finding and accessing formal care services may result in people with NDIS funding remaining unable to access the services they need, and enhancing cultural awareness among service providers might increase access.

We found no evidence that NDIS availability in an area had short-term impacts on the weekly average caring hours for primary carers of those with a profound or severe disability under 65. The lack of change in informal caring hours may imply that the NDIS has not freed up the carer’s time in the short-term of those who managed to get access to the NDIS; for example, carers may need to spend significant time helping recipient access services for which they have funding. The result could also suggest that any improvements are counteracted by negative impacts for those whose recipients did not get access to the NDIS.

Our study also found no impact of NDIS availability on the employment of primary carers. This is consistent with an early evaluation of the NDIS, which found that the percentage of carers who reported paid work did not change for NDIS participants [10]. The report also showed that the percentage of carers who would like to work more has not changed after their recipients enrolled in NDIS, with the lack of flexibility at work being the key barrier. Our research only examined the effect within two years of NDIS being available, and many primary carers may have already been disconnected from the labour market for many years, making re-engagement difficult. The impact on employment outcomes of primary carers for the NDIS may take many years to be fully realised.

The impact on social participation was expected more immediately than employment. We found no association between NDIS and social participation with or without the care recipient. The lack of change may reflect additional barriers for carers to participate in social events. For example, if the carers do not care for their recipient full time, they may lose their carer payment or receive a reduced amount of carer allowance. The financial pressure may prevent them from joining an event. Additionally, our study only looked at social participation, such as going to a zoo or library or attending a concert or a sports event. Therefore, other common social activities, such as catching up with friends for a coffee or dinner, are not included.

Even though our study focuses on the short-term impact of the NDIS on those potentially eligible NDIS participants, it provides a reference for a broader policy issue. As of March 2021, about 450,000 Australians are on the NDIS, representing just 10% of the people with disability in Australia. These “ineligible” people with disability are not able to access the scheme for various reasons. For instance, people with mental health problems may find it difficult to get into the NDIS because their symptoms are episodic and may or may not be long-term, and it is unclear how their needs during episodes should be met [28]. The evidence required to obtain access to the NDIS is also unclear, and people from a culturally or linguistically diverse background may have less understanding of the system and be less able to advocate for themselves. While aiming to provide appropriate services to the people on the NDIS, it is also critical to consider how to best assist those on the margin of NDIS access and their carers.

Limitation

Our research results are robust to different sample restrictions. However, there are several limitations. Even with some of the best available data on those with a disability, we suffer from the lack of statistical power in our analysis, which makes it susceptible to Type II error, or we are less likely to detect small changes in the outcomes accurately. Our sample size power calculation indicates that a much larger sample size would have been required to evaluate the NDIS using the same strategy. Future evaluations might usefully consider linked administrative data to be able to detect meaningful differences due to the NDIS implementations. However, such analyse will not be able to cover as broad a range of outcomes as the SDAC and ideally SDAC would have been increased prior to the NDIS implementation. Second, due to data limitations, we have only two time points and, therefore, cannot test the ‘parallel trend’ assumption in the pre-treatment period. However, our results are still robust when using alternative control groups. Third, we used a repeated cross-sectional dataset in our analysis, recruiting new respondents in each wave of the survey, which could impact the results if the composition of the samples changed. Not following up with the same population over time also means we cannot apply individual fixed effects in the model to control for time-invariant individual characteristics. However, we compared the characteristics of primary carers and their recipients of care in these two waves and controlled for the potential differences in these characteristics in our analysis. We have also conducted robustness checks with a matched sample and found similar results. We also explored the potential heterogeneity of the impact of NDIS, but we did not find any effect. Lastly, we do not have the NDIS utilisation data available, so we cannot account for the different impacts the utilisation rate may bring. But since the sample comprises people with disability on and not on the NDIS, we have applied the LGA fixed effects to control for areas characteristics that are stable over time, including factors on the supply side.

Conclusion

While these limitations should be addressed in future studies, our current analysis of the roll-out of the NDIS between 2015 and 2018 suggests that we lack sufficient evidence to confirm that the new support system is making positive changes for primary carers of individuals under age 65 with profound or severe disability. The use of formal care services at a population level has not increased in the short term. While previous evaluations found positive impacts of the NDIS on the enrolled individual [11], we do not see positive spill-over effects on the carer’s caring time, social participation and employment. Our findings suggest that there may be a need not only to continue improving the support for people with disability on the NDIS but also to provide support for those on the margin of the NDIS. Specific consideration should also be given to the carers, for example, to provide them with more work flexibility so they can balance work and caring responsibilities.

Further research

Future research should be conducted to evaluate the long-term effect of the NDIS when data becomes available. It is worth exploring further how we should support people with disability not on the NDIS in parallel with the NDIS. Access to disability services, thin markets and cultural awareness among service providers in Australia and also in international context are some topics that are worth exploring in future. Longitudinal data, and administrative data connecting with survey data capturing a wider range of outcomes should also be employed. The recent development of the National Disability Data Asset (NDDA) in Australia provides a unique opportunity to have a more complete picture of the programs and services people with disability use and thereby help better understand the overall impact of the NDIS as well as other policies aim at better supporting people with disability and their carers. Future research could also incorporate qualitative methods, such as interviews and focus groups, to gain deeper insights into the personal experiences and challenges faced by individuals accessing disability services, thereby complementing and enriching the quantitative findings and offering more insights into potential policy implications.

Supporting information

S2 Table. Eligibility Criteria for the trial, treatment period, and control period.

https://doi.org/10.1371/journal.pone.0321377.s002

(DOCX)

S3 Table. Description of the constructed variables.

https://doi.org/10.1371/journal.pone.0321377.s003

(DOCX)

S4 Table. Comparison of outcome and characteristics among the participants and non-participants observed in 2015 and 2018.

https://doi.org/10.1371/journal.pone.0321377.s004

(DOCX)

S6 Table. The impact of NDIS on social participation of recipients.

https://doi.org/10.1371/journal.pone.0321377.s006

(DOCX)

S7 Table. Subgroup Analysis (15–24 vs other ages).

https://doi.org/10.1371/journal.pone.0321377.s007

(DOCX)

S8 Table. Sensitivity analysis: weighted sample.

https://doi.org/10.1371/journal.pone.0321377.s008

(DOCX)

S9 Table. Sensitivity analysis: Regional fixed effects.

https://doi.org/10.1371/journal.pone.0321377.s009

(DOCX)

S10 Table. Sensitivity analysis: Cut-off date at 30 June 2017.

https://doi.org/10.1371/journal.pone.0321377.s010

(DOCX)

S11 Table. Sensitivity analysis: Main recipients only.

https://doi.org/10.1371/journal.pone.0321377.s011

(DOCX)

S12 Table. Sensitivity Analysis: Excluding carers eligible for NDIS.

https://doi.org/10.1371/journal.pone.0321377.s012

(DOCX)

S13 Table. Sensitivity Analysis—Different control group (not eligible and live in NDIS available areas).

https://doi.org/10.1371/journal.pone.0321377.s013

(DOCX)

S14 Table. Sensitivity Analysis—Margins of Treatment * Post (accounting for characteristics interaction with the treatment* post variable).

https://doi.org/10.1371/journal.pone.0321377.s014

(DOCX)

S15 Table. Placebo analysis on primary carers of non-NDIS-eligible caring recipients.

https://doi.org/10.1371/journal.pone.0321377.s015

(DOCX)

S1 Fig. The flowchart of sample restriction and sample size.

https://doi.org/10.1371/journal.pone.0321377.s016

(DOCX)

S2a-S2e Fig. Results of sample size estimation by outcomes of interest.

https://doi.org/10.1371/journal.pone.0321377.s017

(DOCX)

Acknowledgments

The authors thank the participants at Bernice’s milestone and at the Australian Health Economics Society conference who provided valuable comments on this paper. We also thank Dr. Duncan Mortimer, Dr. Rohan Sweeney and Dr. Maame Esi Woode for their insightful feedback.

References

  1. 1. National Disability Insurance Agency. Annual Report 2022-2023. 2023.
  2. 2. Hamilton D, Hancock N, Scanlan JN, Banfield M. The National Disability Insurance Scheme and people with severe and persistent mental illness/psychosocial disability: A review, analysis and synthesis of published literature. Aust N Z J Psychiatry. 2020;54(12):1162–72. pmid:33111539
  3. 3. Hancock N, Bresnan A, Smith-Merry J, Gilroy J, Yen I, Llewellyn G. NDIS and psychosocial disability–the Victorian story: Insights and policy recommendations from expert stakeholders. 2018.
  4. 4. Olney S, Mills A, Fallon L. The Tier 2 tipping point: access to support for working-age Australians without individual NDIS funding. 2022.
  5. 5. Carey G, Malbon E, Reeders D, Kavanagh A, Llewellyn G. Redressing or entrenching social and health inequities through policy implementation? Examining personalised budgets through the Australian National Disability Insurance Scheme. Int J Equity Health. 2017;16(1):1-12. pmid:29110663
  6. 6. Australian Institute of Health and Welfare. People with disability in Australia 2019: in brief 2019 [Available from: https://www.aihw.gov.au/reports/dis/74/people-with-disability-in-australia-in-brief/contents/how-many-people-have-disability
  7. 7. National Disability Insurance Agency. The NDIS Annual Report 2019-20. 2020.
  8. 8. National Disability Insurance Agency. Participant outcomes report. 2020.
  9. 9. Mavromaras K, Moskos M, Mahuteau S, Isherwood L, Goode A, Walton H, et al. Evaluation of the NDIS. Final report Adelaide: National Institute of Labour Studies, Flinders University; 2018.
  10. 10. National Disability Insurance Agency. Employment Outcomes for families and carers of NDIS participants. 2018.
  11. 11. National Disability Insurance Agency. NDIS Annual Report 2018-19. 2019.
  12. 12. Australian Bureau of Statistics. Microdata: Survey of Disability, Ageing and Carers [Datalab], accessed 15 December 2022. [Available from: https://www.abs.gov.au/statistics/microdata-tablebuilder/available-microdata-tablebuilder/disability-ageing-and-carers-australia
  13. 13. National Disability Insurance Agency. Planning Operational Guideline - Deciding to include supports in a participant’s plan [Available from: https://www.ndis.gov.au/about-us/operational-guidelines/planning-operational-guideline/planning-operational-guideline-statement-participant-supports
  14. 14. National Disability Insurance Agency. National Dashboard as at 31 March 2022. 2022.
  15. 15. Disney G, Yang Y, Summers P, Gupta A, Byars S, Bonyhady B, et al. NDIS Plan Utilisation Project: describing understanding and explaining inequalities in plan utilisation. 2021.
  16. 16. Bauer JM, Sousa-Poza A. Impacts of informal caregiving on caregiver employment, health, and family. J Population Ageing. 2015; 8(3):113–45.
  17. 17. Bolin K, Lindgren B, Lundborg P. Informal and formal care among single-living elderly in Europe. Health Econ. 2008;17(3):393–409. pmid:17768700
  18. 18. Mentzakis E, McNamee P, Ryan M. Who cares and how much: exploring the determinants of co-residential informal care. Review of Economics of the Household. 2009;7(3):283-303.
  19. 19. Pickard L Substitution between formal and informal care: a ‘natural experiment’ in social policy in Britain between 1985 and 2000. Ageing and Society. 2012;32(7):1147-75
  20. 20. Van Houtven CH, Norton EC. Informal care and health care use of older adults. J Health Econ. 2004;23(6):1159–80. pmid:15556241
  21. 21. Litwak E, Messeri P, Silverstein M. The role of formal and informal groups in providing help to older people. Marriage Family Review. 1990;15(1-2):171-93.
  22. 22. Chappell N, Blandford A. Informal and formal care: exploring the complementarity. Ageing & Society. 1991;11(3):299-317.
  23. 23. Costa-Font J. How to support adult caregivers? IZA World of Labor. 2023.
  24. 24. Van Houtven CH, Coe NB, Skira MM. The effect of informal care on work and wages. J Health Econ. 2013;32(1):240–52. pmid:23220459
  25. 25. Ribas RP. Using pseudo-panels to analyse labour market transitions. 2022.
  26. 26. Devine A, Dickinson H, Rangi M, Huska M, Disney G, Yang Y, et al. ‘Nearly gave up on it to be honest’: Utilisation of individualised budgets by people with psychosocial disability within Australia’s National Disability Insurance Scheme. Social Policy & Administration. 2022; 56(7):1056-73.
  27. 27. Malbon E, Weier M, Carey G, Writer T. How personalisation programs can exacerbate socio-economic inequities: findings from budget utilisation in the Australian National Disability Insurance Scheme. BMC Public Health. 2022;22(1):1-12. pmid:35501795
  28. 28. Williams T, Smith G. Mental Health and the NDIS: Making It Work for People with Psychosocial Disability. The National Disability Insurance Scheme: Springer; 2021. p. 161-91.