Notions

We choose dengue as the infectious disease to implement our methodology. Our choice is motivated by the availability and the length of the temporal window of the dataset. Dengue is transmitted when an infectious female vector, primarily Aedes aegypti, bites a susceptible person, formally known as time of exposure. Symptoms of the infection start to appear within 3 to 10 days [11], a period of time known as the intrinsic incubation period (IIP). The latent period (LP) ends up to 2 days prior to the onset of symptoms [19] and can only be observed through a lab test. A person is capable of providing an infectious blood meal after the end of the latent period, and retains this capability for up to 7 days after the symptom onset [20]. This period is referred to as the infectious period. A susceptible mosquito undergoes an extrinsic incubation period (EIP) after having an infectious blood meal. The duration of the EIP is influenced by a range of factors including the ambient temperature, and has been reported to be as short as 3 days [11]. In unfavourable conditions (e.g., when temperatures drop below 20°Celsius), the EIP is estimated to be longer than a typical mosquito’s lifetime [11] (i.e., the mosquito dies before the end of its incubation period). In order to facilitate our analysis, we introduce the notions of transmission period. We know that, once infected, a mosquito remains infectious for life. We also know that it can take multiple blood meals over its lifetime [21]. Accordingly, we define the period that begins at the end of the EIP and persists until the death of the infectious mosquito as the vector transmission period. Since an infectious person may infect multiple mosquitoes, the transmission period for a particular dengue case (called the case transmission period) is an aggregation of the vector transmission periods. It starts with the earliest concluding EIP and ends with the death of the last infected mosquito (S1 Text). See Fig 1 for a visual depiction of the dengue infection timeline. For the sake of brevity, we some time refer to the case transmission period simply as the transmission period.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Dengue infection timeline.

The blue, yellow and green arrows depict the dengue infection timeline in a human, while the two red arrows depict the same for a mosquito. A human undergoes a latent, intrinsic incubation (IIP) and infectious period indicating the time span during which it is infected: but not infectious, but not showing symptoms, and infectious, respectively. A mosquito undergoes an extrinsic incubation (EIP) and transmission period during which it is infected: but not infectious, and infectious, respectively. A case transmission period is an aggregation of all vector transmission periods associated with the case that starts with the earliest concluding EIP and ends with the death of the last infected mosquito.

https://doi.org/10.1371/journal.pone.0258332.g001

We evaluate the timeliness of surveillance through its impact on disease incidence using four epidemiological indicators: case fertility, reproduction, outbreak size and generation. We define a case to be fertile if it leads to a secondary case (i.e., it is the parent of at least one case that we call its child). The reproduction of a case indicates the number of child cases it has caused. An outbreak is a set of cases connected by the chain of infections (through parent child relations). The size of an outbreak denotes the number of cases in an outbreak. Let the first case of an outbreak be called root. An outbreak generation is a set of cases that are at the same distance (by number of parent child relations) from root. The number of generations in an outbreak are called the depth of the outbreak.

Dengue in Australia

To evaluate our methodology, we take the reported occurrence of dengue in Australia, between 2002 and 2018, as a case study. Dengue is not endemic in Australia. However, regular annual local outbreaks are reported, triggered by infected travellers arriving from endemic countries [22]. Clearly, the transmission of dengue is subject to the presence of the vector. In Australia, the vector presence is limited to parts of the state of Queensland. The primary vector, Aedes aegypti, is prevalent in northern coastal communities and has also been recorded in several areas of central and southern Queensland [22, 23]. The presence of Aedes albopictus, a secondary vector capable of transmitting dengue infection, has also been recorded in the northern parts (Torres Strait Islands) of the state [24]. Queensland Health has a passive surveillance system in place for identifying dengue cases. The system relies on general practitioners, emergency departments and laboratories notifying Queensland Health of suspected and laboratory confirmed cases. The treating doctors are advised to report a case upon clinical suspicion under the provision of the Public Health Act 2005 [25]. Since dengue is not endemic in Australia, particular attention is paid to people who exhibit dengue symptoms and have recently travelled to an endemic country. Queensland Health defines a dengue outbreak as at least one locally acquired case [22]. All laboratory-confirmed and suspected cases are notified to the local Public Health Unit (PHU). Upon notification the PHU communicates with the patient as well as the treating doctor and laboratory to collect case details. This includes the spatio-temporal aspects of the case(see Data Collection section for details). Vector elimination is the first line of defence for Queensland Health to prevent the wider spread of dengue. According to the Queensland Dengue Management Plan [22], when a dengue case is notified it is referred to a medical entomologist by the concerned PHU. If the reported case is in a region with known vector presence, the medical entomologist instigates appropriate vector control measures. The measures consist of indoor residual spraying and treating water containers using insect growth regulators or pesticides at the addresses identified through the travel and contact history of the patient. The aim is to eradicate the virus by eliminating the vector within a radius of about 200 metres (the estimated distance a vector can travel over its lifetime) [26]. Queensland Health works collaboratively with local governments and dedicated Dengue Action Response Teams to manage the efforts.

Since vector control in Queensland is a reactive exercise, it is more effective when conducted closer in time to the start of the infectious period. Theoretically, if conducted within the EIP (the beginning of EIP and infectious period can coincide as a susceptible mosquito may bite the infectious person on their first day of infectiousness), the efforts can potentially prevent any subsequent local transmission. However, prompt reporting of a case may not always be possible due to various factors, including awareness and willingness of a patient to seek medical advice, training and diagnostic ability of the treating doctor, capability and capacity of the testing laboratory and timeliness and complexity of the disease surveillance mechanism. We call the time elapsed (in days) since a case has become infectious until it has been notified to Queensland Health the notification delay.

Data collection

Dengue is a notifiable disease in Australia under the Public Health Act 2005 and laboratory confirmed cases are required to be notified to health departments within each state and territory [25]. Since the impact of disease response can only be measured in the regions with local transmission, we limit the scope of our analysis to these regions. Data on the occurrence of dengue was obtained from the Communicable Disease Branch (CDB) of Queensland Health for the period of October 2001 to September 2018. During this period 5,272 cases were reported, of which 2,808 (53%) were acquired locally. The data records cases on an individual level and covers their spatial, temporaland epidemiological aspects. Spatial information includes the place of residence and the place of acquisition on locality/suburb and postcode levels, respectively. For cases acquired overseas the highest resolution for the place of acquisition is the country. Temporal information consists of the date of symptom onset, specimen collection, diagnosis and notification. The epidemiological data contains information about the serotype of the infection.

Traditionally, temporal analysis of the dengue cases is performed with an observation window set for each calendar year, where cases are assigned to a window with respect to their date of symptom onset. Since the majority of locally acquired dengue cases in Queensland occur between October and May, we move the observation window to align it with this seasonal fluctuation. For each year, our observation window starts at the beginning of September of the previous year and concludes at the end of August of the current year. This shift allows us to incorporate the outbreaks that span across calendar years (see Table 1 for a year-wise breakdown of the occurrence). For the ease of analysis, spatial statistics are aggregated by Statistical Area Level 4 (SA4). The SA4s are sub-State regions defined by the Australian Statistical Geography Standard (ASGS) with a population range between 100,000 and 500,000 individuals. Queensland has 19 SA4 regions in total, of which 7, namely Cairns, Darling Downs, Fitzroy, Mackay, Queensland-Outback, Townsville and Wide Bay, have recorded the presence of the vector.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Statistics on dengue occurrence and dynamics of its spread in Queensland, Australia.

The Fertility, Parent and Outbreak columns present the average (value outside the parentheses) and confidence interval values that were computed by applying the Monte Carlo’s method to the construction of the disease spread network. A case is fertile if it has resulted in at least one secondary case and the reproduction indicates the average number of secondary cases per fertile case. The parent of a locally acquired case is unknown if no suitable parent case was found during the construction of the disease spread network and known otherwise.An outbreak is a set of cases connected by the chain of infections (through parent child relations). The size of an outbreak denotes the number of cases in an outbreak. Let the first case of an outbreak be called root. An outbreak generation is a set of cases that are at the same distance (by number of parent child relations) from root. The number of generations in an outbreak are called the depth of the outbreak.

https://doi.org/10.1371/journal.pone.0258332.t001

In order to infer relationships between the cases, the occurrence data is utilized in combination with weather and population mobility data. Daily weather data includes minimum and maximum temperatures for the period of October 2001 to September 2018, recorded at 216 weather stations across Queensland and was obtained from the Australian Bureau of Meteorology. For human mobility data, we constructed yearly origin-destination matrices for the studied period from National Visitor Surveys (NVS), International Visitor Surveys (IVS) and Twitter data. The surveys of national and international visitors were conducted by Tourism Research Australia and record the different SA4 regions that individuals visited. For national visitors the SA4 region of residence is also recorded. NVS data is available between 1998 and 2015, IVS data is available between 2005 and 2015. The Twitter data supplements the IVS and NVS datasets as a source of intra-region mobility. It was collected in 2015 and contains 925,945 trips from 79,271 individuals [27]. For the missing years, we apply an auto-regressive moving average model [28], a common method for forecasting time-series to estimate movement between regions.

Disease spread network

Utilizing the dengue occurrence data along with the population mobility and meteorology datasets, we construct the most likely disease spread network (DSN) that depicts the spatio-temporal spread of dengue in Queensland. The DSN is a directed acyclic graph, where nodes represent dengue cases and arcs represent probable relationships between them. The construction of the network is a two stage process. In the first stage, we draw arcs by applying the epidemiological soundness criteria and in the second stage, we prune the network using the LIPP model [10] to identify the most likely transmission paths. The epidemiological soundness of an arc depends on the spatio-temporal and clinical characteristics of the cases that are being considered to be connected by the arc. More specifically, in order for two cases p and c to be connected via an arc (p → c), they must fulfill the following criteria;

• the exposure period of c must overlap with the transmission period of p,
• the region of acquisition of c must coincide with region of residence of p (applied on SA4 level),
• p and c must not be infected by different serotypes of dengue.

We describe the relationship between p and c as that of a parent and child, where p is the parent case and c is the child case. To fulfil the first criterion, we estimate the IIP and average EIP for each case using a Gamma and log-normal time-to-event model respectively, where the latter incorporates temperature sensitivity through a co-variate (see S2 Text for more details) [11]. The LP is estimated by decreasing the estimated IIP value by 2 [19]. The remaining criteria can be tested using the information provided in the dengue occurrence dataset. Since IIP is randomly drawn from a gamma distribution its randomness may impact the structure of the network. To account for this variability in our analysis we repeat the network construction process through Monte Carlo’s simulation and present disagreement of measures across various instances with a 95% confidence interval wherever applicable. The simulation converges when there is 95% probability that the relative error of the sampling distribution of means is at most 1%. The EIP on the other hand, is drawn from a log-normal distribution that generates the mean of all possible EIPs for a given temperature. The mean EIP remains unchanged across independent draws for the same temperature.

At the end of the first stage of network construction we obtain a probable DSN that logically aligns with the epidemiological characteristics of dengue. Yet it is not an acceptable representation of disease spread. In nature, we expect each dengue case to have exactly one parent case. However, the above criteria do not prevent a node from having multiple in-coming arcs, implying a practical impossibility that it has been caused by more than one case. The second stage is required to address this problem.

In the second stage, LIPP model is used to prune the excess arcs. The model estimates pair-wise causation probabilities for dengue cases using a multivariate Hawkes process [29]. It models disease spread across a heterogeneous social system by incorporating three major counterbalancing factors: (i) exogenous influence covering environmental heterogeneity, (ii) endogenous influence attributed to macro level interactions between meta-populations, and (iii) a time decay kernel. In this context, the exogenous influence incorporates the importation of the infection from abroad and the availability and density of the vector in the regions (given that dengue is non-endemic in Australia, importation of infections play a crucial role in the local spread of the disease). The endogenous influence captures the impact of inter-region human mobility. An infected individual gradually recovers from the infection through a natural immune response, decreasing their infectivity over time, this effect is integrated by the the time decay kernel. The estimates are computed using the disease occurrence and population mobility data (see S2 Text for more details). The estimates generated by the model are used to break ties between multiple parents of a child resulting in the most likely parent-child arc being preserved.

In the context of the disease spread network, we define a case to be fertile if the node representing the case has at least one out-going arc and infertile otherwise. Similarly the reproduction of a case denotes the number of out-going arcs of the node representing the case.

Statistical analysis

Utilizing the disease spread network, we enquire whether morbidity is related to delay in disease response. Furthermore, we test varying response delay thresholds for what may represent timely surveillance, and gauge their impact on various parameters associated with the incidence of dengue. In particular, we considered the impact of calibrated disease response on the occurrence and reproduction of the cases as well as the size and depth of the outbreaks. The thresholds are enforced by removing the child cases (and their descendants) whose exposure period most likely occurred after the response efforts have been conducted for the parent case. The cases removed through this process are denoted as prevented cases. The removal of these cases effects all four indicators described above. We note that it is not possible to prevent a case with an unknown parent by reducing response time. Similarly, a childless case has zero reproduction, which cannot contract further through reduction in response time. Therefore, we restricted our analysis of case occurrence and reproduction to the cases with a known parent and to the cases that has at least one child, respectively. We also measure the impact of variation in response delay in terms of outbreak size and number of generations. Since an outbreak consisting of a single case cannot be reduced in size by limiting response delay, we restrict our analysis to the outbreaks of size at least 2. Similarly, when considering the reduction in number of generations, we limit our analysis to the outbreaks with at least two generations as the number of generations cannot be reduced for a singleton. All results are reported in comparison to the baseline established in the original data.

We conduct a spatio-temporal analysis of the notification delay components by dividing it into three parts namely, patient delay, laboratory delay and clerical delay. The patient delay is the time elapsed (in days) since the onset of symptoms to the date of specimen collection, which can be interpreted as a combination of the delay on part of a patient before they seek medical attention and the delay in clinical diagnosis by the treating doctor. The time taken by a laboratory from the date of specimen collection to the date of diagnosis is called the laboratory delay. Lastly, the time elapsed between the date of diagnosis and the date of notification is called clerical delay. For further insights we apply the spatial analysis of the lab and patient delays on a finer (locality) level. Additionally, we classify the reported cases by their seasonal and demographic attributes to identify any impact these attributes may have on the lab and patient delays. For seasonality analysis we classify the cases by month of onset while the demographic analysis is conducted through classification into age (9 classes containing patients of age 1 to 90 with each class consisting of ten consecutive discrete values (e.g., 1–10, 11–20)) and gender groups. The outcome of theses analyses can prove useful in providing guidelines on targeted resource allocation for public awareness as well as to improve diagnostic abilities and turn-around times for doctors and laboratories involved.

Case fertility and response delay

Of the 5,272 reported cases, 537 (10%) were estimated to be fertile. Moreover, of the 2,808 locally acquired case, a parent was found for 2,360 (84%) while no suitable candidates were found for 448 (16%) (see Table 1 for details on fertility and parent-child relationship estimates). The years with the highest number of fertile cases mostly correspond to years with most number of reported cases. However, in terms of proportion of reported cases, 2005 was estimated to have the highest proportion of fertile cases (18%) followed by 2006 and 2004 with 17% fertile cases each. In terms of region-wise distribution, Cairns was estimated to have the highest number of fertile cases (350) followed by Townsville (112) and Queensland-Outback (44). Proportionally, Townsville led Cairns with 22% fertile cases compared to 15%, with Queensland-Outback (10%) at the third spot.

We observed a positive relation between the number of fertile cases and the length of transmission period. A longer transmission period allows time for the infected mosquitoes to bite more people, which leads to a higher chance of fertility. We note that the transmission period can be cut-short through the deployment of disease response. Therefore, a longer transmission period is a consequence of delayed response. Our estimation showed that the fertile cases had a mean transmission period of 10 days. Moreover, the first, second (median) and third quartile transmission periods were observed to be of 5, 8 and 13 days, respectively. On the other hand, only 5% fertile cases had a transmission period of 2 days or less.

A year-wise breakdown of the fertile cases (see Fig 2A) presented a similar pattern. With the exception of 2017(4 days), overall the fertile cases reported each year had an estimated median transmission period of at least 5 days (IQR = 5–13). Spatially (Fig 2B), fertile cases in Townsville were estimated to have the longest transmission periods with the mean, first, second and third quartile of 13, 7, 11 and 21 days, respectively. Overall, the median transmission period exceeded 5 days across all regions. For Cairns and Queensland-Outback in particular, we observed a median transmission period of at least 7 and 8 days, respectively. The distributions of the length of transmission periods of imported and locally acquired fertile cases show that the descriptive statistics are not skewed by a particular group or a small number of very long transmission periods.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Transmission periods of fertile cases.

The violin charts in (A) and (B) depict the length of transmission periods in fertile cases by year and region (SA4) of case occurrence. Each element of the violin chart contains a box and whiskers plot sandwiched by the distribution curves of the transmission periods of imported and locally acquired fertile cases.

https://doi.org/10.1371/journal.pone.0258332.g002

Temporal calibration of response

We tested 6 response delay thresholds between 5 and 30 days (inclusive) with an interval of 5 days. In general, the highest reduction was noticed across all parameters for the response delay threshold of 5 days. Another commonality was the large difference between the response times of 5 and 10 days (see Fig 3). Particularly, there was an average reduction of 87% [95% CI, (86, 88)] and 83% [95% CI, (83, 84)] in case occurrence and reproduction respectively, when a response time of at most 5 days was enforced. A sharp decline was observed in the number of prevented cases, which contracted by 41% (to 46% [95% CI, (43, 48)]) and a further 15% (to 31% [95% CI, (28, 33)]) when the response delay was extended to 10 and 15 days. The difference was not as significant for higher response delay thresholds.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Temporal calibration of disease response.

The impact of varying response delay thresholds on the dynamics of dengue occurrence. (A) plots percentage of cases prevented while (B), (C) and (D) plot percentage reduction in reproduction, outbreak size and number of generations against varying response delay thresholds. The reproduction denotes the number of secondary cases per fertile case, the size and generations of an outbreak denotes the number of cases and number of generations in the infection chain of the outbreak, respectively.

https://doi.org/10.1371/journal.pone.0258332.g003

The average reduction in case reproduction also declined with increasing response time, though not as dramatically, ranging between 71% [95% CI, (71, 71)] and 83% [95% CI, (83, 84)]. In terms of year-wise classification (Fig 3A), the years that recorded more than 100 locally acquired cases(2003, 2004, 2009, 2011, 2013 and 2014) saw a significant reduction in incidence across all response delay thresholds. On the other hand, for the years with less than 100 locally acquired cases, the proportion of prevented cases fell to less than 20% when the response delay was set to 15 days and above. The analysis of the annual breakdown of the case reproduction (Fig 3B) showed that the overall averages were skewed by the 3 years with the highest local incidence where the reduction consistently remained between 80% and 90% across all 6 response thresholds. However, this was not the case for the years with lower occurrence where in most cases the reproduction declined between 30% to 50% when the response time was extended from 5 to 10 days.

The outbreak size followed a trend that was overall similar to that of prevented cases. An 80% [95% CI, (79, 81)] reduction was estimated in average outbreak size when the response delay was fixed at 5 days. Once again, a sharp decline (38%) was observed when the response delay was extended to 10 days with outbreaks shrinking by 42% [95% CI, (39, 45)] on average. It shrunk by another 18% (to 24% [95% CI, (21, 28)]) before flattening, when a response delay of at most 15 days was applied. The year-wise analysis of the outbreak sizes (Fig 3C) revealed trends that were quite similar to that of prevented cases. The years with less than 100 locally acquired cases showed little reduction at response thresholds of 15 days and above, while the outbreaks in years with the highest occurrence shrunk significantly at both small and large response delay thresholds. The number of generations was the least affected parameter in our analysis, with a 47% [95% CI, (45, 48)] average reduction for a response delay threshold of at most 5 days. It dropped swiftly to 19% [95% CI, (16, 21)] and then 7% [95% CI, (5, 9)] when the response delay was extended to 10 and 15 days before somewhat flattening to under 5% at the response delay of at most 25 and 30 days. Analysis of the year-wise classification (see Fig 3D) showed that a response delay of 5 days resulted in a 30 to 50% contraction in the number of generations for most years with the smallest reduction noted at 18%. This fell to 10% and below for most years when the response was delayed by 15 days. Furthermore, little to no reduction was noted at the response thresholds of 25 and 30 days.

Reducing response delay

A temporal plot of the notification delay and its components is provided in Fig 4A. The patient delay showed no clear temporal pattern. Though a couple of dips below three days in the years 2008 and 2012 coincided with a drop in case fertility to under 5%. This pattern was not noticed for other year (2002, 2016, 2017 and 2018) with under 5% fertile cases. The lab and clerical delays gradually declined as the years progressed. The average lab delay decreased from nearly 7.5 days to just over 3 days between 2003 and 2018 with minor hikes in 2006 and 2012. Similarly, the average clerical delay mostly fluctuated between 1 and 3 days for years between 2002 and 2007 (except 2004), fell to 0 in 2008 and consistently remained under 0.5 days for the following years. Overall, the variation in notification delay from 2008 onward were largely influenced by lab and patient delays as the clerical delays flattened.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Notification delay and its components.

A temporal (A) and spatial (B) depiction of notification delay and its components for the dengue cases reported in the vector present regions of Queensland, Australia. The cases are grouped by the year and region (SA4) of occurrence, respectively. The patient delay is the time elapsed (in days) since the onset of symptoms to the date of specimen collection. The time taken by a laboratory from the date of specimen collection to the date of diagnosis is called the laboratory delay. And, the time elapsed between the date of diagnosis and the date of notification is called clerical delay.

https://doi.org/10.1371/journal.pone.0258332.g004

Spatially (Fig 4B), an average patient delay of 4 days or above was observed for all regions with Townsville and Fitzroy reporting an average delay of 5 and 6 days respectively. Similarly, An average lab delay of at least 4 days was reported for all regions, except Mackay. Townsville saw the longest clerical delay (1.1 days) on average across all regions followed by Cairns (0.9 days), Wide Bay (0.6 days) and Queensland—Outback (0.4 days). Overall, an average notification delay of at least 9 days was noted in all regions except Wide Bay (8.1 days). Of the 3 regions with the highest incidence (Cairns, Queensland—Outback and Townsville), Townsville encountered the longest notification delay (11.1 days) on average. For a finer spatial analysis, a geo-spatial depiction the boundaries for Cairns and its localities are rendered using shape files obtained from Australian Bureau of Statistics [30] that are publicly available under the Creative Commons licence of the average lab and patient delay in the localities of Cairns is provided in Fig 5 (see S1 Fig, for similar results on other vector present regions of Queensland).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Spatial analysis of the notification delay components.

Geo-spatial depiction of the average lab and patient delays in the localities of Cairns.

https://doi.org/10.1371/journal.pone.0258332.g005

To analyze the seasonal and demographic trends in patient and lab delays, we classified the reported cases by their month of occurrence, age and gender. We noticed that the longest patient delays on average were recorded during the spring season between August and October (see Fig 6B). A regional (SA4 level) breakdown of this analysis is presented in S3 Fig. On the other hand, the lab delays peaked during the fourth quarter with the median delay of 4 days, while the shortest delays occurred in the third quarter (Fig 6A). Applying seasonal analysis on SA4 level (see S2 Fig), we noticed similar trends in Cairns and Townsville. In Cairns, the longest lab delays on average were recorded during the months of April (6 days), October and December (4 days). In Townsville, the average delays peaked during September and October (5 and 6 days, respectively). The remaining regions showed no clear pattern. Lastly, in order to identify demographic groups of patient for targeted awareness campaigns, we performed an analysis of the patient delay trends through classification of the reported cases by age and gender (S4 Fig). We found that in male patients, most cases with a patient delay of 4 to 5 days were observed in the age group 30 to 35 years. Moreover, the largest number of cases with a delay of 10 or more days were observed in patients of age 45 to 60. On the other hand, a delay of 4 to 5 days was most frequently observed in female patient of age groups 20–35 and 45–50 years, the same groups also had the highest frequency of cases with delay of 10 and more days.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Impact of seasonality on lab and patient delays.

The box and whisker plots depict monthly statistics of the lab (A) and patient (B) delays where cases are grouped by month of occurrence.

https://doi.org/10.1371/journal.pone.0258332.g006