Approvals and permissions

The authors obtained prospective ethical approval for this study from the Royal Children’s Hospital and Health Services District Ethics Committee (ethics number EC00175 and 2006/66), and from the University of Queensland Human Research Ethics Committee (ethics number 2009000278). The Office for Research at Griffith University (the corresponding author’s current institution) confirmed with the journal office that appropriate ethics approvals as required at the institutional level were in place at the time of the study. The authors also obtained letters of support in advance of the study from multiple parties, including representatives from Queensland Government and the Aboriginal and Torres Strait Islander Ear Health Program (the predecessor of the Deadly Ears Program).

Children’s Health Queensland Hospital and Health Service (CHQ HHS) have clarified with the journal office that CHQ was established as a statutory body in 2012 and that the use of Deadly Ears Program data currently requires release and authorization by CHQ; this system was not yet in place when the authors conducted the study, although ethical approval and letters of support were required at that time.

Deadly Ears Program services

Questions were raised as to whether the frequency of Deadly Ears Program visits to Cherbourg was accurately reported. The authors clarified that staff from the Deadly Ears Program (formerly called the Aboriginal and Torres Strait Islander Ear Health Program) travelled to Cherbourg at least three times per year during the years 2006–2008 and conducted hearing tests, among other activities; this service was used as the “comparator” in the cost-effectiveness analysis, and was compared to this same service supplemented by mobile telemedicine-enabled screening and surveillance (MTESS).

By screening services, the authors referred to assessments by Deadly Ear Program staff to identify individuals who required surgery. These included an initial assessment by a clinical staff member to identify children who may have potential hearing problems, and subsequent hearing tests for such individuals conducted by an audiologist on the team.

Concerns were raised that the Deadly Ears Program services were not accurately described in the article, specifically that this program did not provide screening and surveillance service in the Cherbourg community during the study period but rather only provided Ear, Nose, and Throat (ENT) surgical services.

Data sources and updated cost estimates

The data used for the cost-effectiveness analysis included estimates derived from personal communications with Deadly Ears staff, high-level summary data provided by Queensland Health or summarized from the Centre for Online Health database, data obtained from the published literature, and expert opinions. Tables 2 and 3 are updated here to include data source information and to clarify which values are based on assumptions. The revised version of Table 3 reports results of a reanalysis performed with updated information for Variable staff costs, Consumables, Variable travel and accommodation costs, Total annual cost, and Cost per surgery. The original analysis of costs, with data source information added, is included with this notice in the supporting information S1 Table. Fixed and variable costs in Table 3 and S1 Table are the annual average costs for outreach surgical services, and do not represent costs to the Cherbourg community or the Deadly Ears program.

Costings for the Deadly Ears Program ENT surgical services in S1 Table were based on estimates of resource use, staffing requirements, and the unit costs of those resources following discussion with staff at Cherbourg and running the MTESS and the Deadly Ears programs. These were not actual costings, but cost estimates, and were not provided by the Deadly Ears Program. Furthermore, since the analysis was conducted from a health system perspective, not all cost items included in S1 Table were incurred by the Deadly Ears program; they were actual costs incurred by other entities of the health care sector, including the State or Commonwealth Government.

An additional cost analysis was undertaken with early (2008) data previously supplied as expert opinions from the Deadly Ears staff (Table 3). The total cost for the MTESS was $236,200 vs. $78,243 for Deadly Ears (Table 2). Furthermore, the cost of employees who operate the programs were incurred to the health care system, but not specifically to the Deadly Ears or the MTESS programs. These include (i) the Aboriginal health worker employed full-time to conduct the mobile health screening program, and (ii) nurse and nurse unit manager employed to run the surgical service. The Deadly Ears Program did not employ any full-time staff in Cherbourg during the study period.

Travel costs included in the analysis were based on the assumptions that there would be 4 visits annually, each trip took 4 days, and assumed 8 working hours per day (4×4×8 = 128 hours per year). This is an error, per updated information the Deadly Ears Program made biannual visits. This is equivalent to Deadly Ears Program providing approximately 32 hours of ENT staff support in Cherbourg per year. The average cost per case was used in the analysis; if the number of visits and trips decrease, and there is a proportionate reduction in the number of cases, then the average cost per case would be similar to the estimate made by the authors. Fuel was estimated at $0.75 per km, in line with the Australian Tax Office claimable tax offset for the running costs of an average motor vehicle at that time.

Updated sensitivity analyses

While costs in an economic evaluation are subject to variation, including due to temporal change and geographical variation, the impacts of variation in costing and related factors are addressed in sensitivity analyses. Reducing the staff costs for 25% (i.e., to the 32 annual hours, and 0.25 FTE for the Clinical Nurse and the Nurse Manager), plus adjusting the travel costs to reflect the two trips per year gives a total cost of $2,261 per surgery. The cost of surgery affects the total costs of both the Deadly Ears and the MTESS programs but remains relative. The additional sensitivity analysis (S2 Table) showed that at the cost of $2,261 per child surgery in the outreach clinic the MTESS remained cost-effective and provided better health outcomes (ICER = AU $507, compared to the original published ICER of AU $656 presented in Table 6 of [1]).

Additional sensitivity analyses were conducted after publication of [1] to further understand the impact of the cost of surgery on the conclusions pertaining to cost-effectiveness; this includes a table of results for the sensitivity analyses (S2 Table) and a new Tornado diagram (S1 Fig). The authors tested a claim that the actual cost was three to six times less than the cost presented in S2 Table (the surgical cost per case was $2,369). The authors replaced this number ($2,369 = $379,023 / 160 cases) with a cost value of $340 (which is more than six times smaller). They found that the (average) total costs (per strategy) for MTESS vs. Deadly Ears were $6,031 vs. $6,120 and QALYs were 15.644 vs. 15.902. Please note that these average total costs were not the one-off cost of providing surgeries or per screening occasion, but are the accumulated costs per individual (children) over a life time (of the model), which include screening cost, surgeries and treatment if it occurred, cochlear implants or other hearing aids and education cost for hearing loss children (if required).

Further probabilistic sensitivity analyses (PSA) were undertaken. In the original model, the authors specified the beta distributions using the base case value as mean, and variance as mean*0.2. For the revised model, the authors imposed new parameters to ensure all the beta distributions had a range as close to (0, 1) as practically possible. Since the authors specified the alpha and beta parameters from the base case value (as mean), they tested a range of variance values (for each beta distribution) to find the maximum variance values that still produce positive values for both alpha and beta parameters. The PSA results are presented on the willingness-to-pay spectrum in S2 Fig (original values) and S3 Fig (updated values).

With this Correction, the authors provide a revised version of Table 1 with additional information on the distributions used for the sensitivity analyses. In the probabilistic sensitivity analyses, the value for each parameter was drawn randomly from the respective distribution, then the model calculated the costs and outcomes for each strategy and the resulting incremental cost effectiveness ratio (ICER). The number of draws was 10,000 (same specification as the original analysis). The scatter plot of 10,000 draws is presented in S4 Fig.

Discussion

There are a number of limitations made more explicit here.

The authors relied on expert opinion for some parameters. These parameters are subject to potential opinion bias, and the analysis of uncertainty may not fully reflect the variation that might be expected from expert opinion or assumptions used. The experts providing the information were experts in screening for hearing loss, ENT surgery, and indigenous health. In Tables 1 and 7, the authors specified the expert opinions and the sensitivity range based on their opinions. The range of some of the parameter values for the sensitivity analysis, such as “treatment failure” and “progression”, while not specified in Table 1, could be found in Tables 7 and 8 (and also available in the full model that is publicly available). Those parameters specified with a beta distribution, such as the “probabilities” are bounded within the range of 0 and 1. An updated sensitivity analysis is provided to address these concerns.

A point to highlight here is that there are more parameters derived from real-world data than assumptions used in the analysis. Like all economic evaluation models, it is informed by both actual data (trials) and literature; and this is the norm in economic evaluations. Program costs were calculated from the data provided within the studies, sourced from the literature, or where these sources did not contain the data required, expert opinions were sought. The scatterplot demonstrates the likely spread of the cost-effectiveness of MTESS compared with the service delivery model that was used prior to the introduction of MTESS (i.e., Deadly Ears Program, formerly called the Aboriginal and Torres Strait Islander Ear Health Program, during the years 2006–2008).

The limitation that this is a single site study is noted; evaluation of additional sites would be advantageous to confirm these results.

Conclusions

Concerns were raised during the post-publication assessment that the conclusions reported in the article overstate what could be supported by the results, given the reliance of several parameters on expert opinion and/or assumptions, and in light of the PSA analysis results. This relates to the following:

• Tenth sentence of the Abstract: “We concluded that the MTESS service is a cost-effective strategy.”
• First and second sentences in the second paragraph of the Discussion section: “This model-based analysis shows that, compared to the Deadly Ears Program, the MTESS service is cost effective, with an average 98% probability of an acceptable ICER at the $50,000/QALY threshold. The cost effectiveness arises from preventing hearing loss in the given population and subsequent reductions in associated educational support costs and hearing aids and equipment costs.”
• First sentence of the Conclusion section: “…from a health service perspective, the supplemental mobile telemedicine-enabled screening and surveillance (MTESS) service is cost effective compared to the current practice alternative alone.”

The conclusions are hereby amended to state that the results of this study suggest that MTESS is likely to be a cost-effective strategy, but in light of the limitations discussed above and in [1] due caution should be exercised when making policy decisions.

Tables

Please see the updated Tables 1–3 here.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Transition probabilities.

https://doi.org/10.1371/journal.pone.0234021.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Costs of screening, both strategies.

https://doi.org/10.1371/journal.pone.0234021.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Costs for surgical treatment in outpatient clinic.

https://doi.org/10.1371/journal.pone.0234021.t003

Supporting information