https://orcid.org/0000-0002-3426-5520
Figures
Abstract
This article presents an approach to stochastic analysis of disease dynamics. We develop an explicit semi-Markovian model that accounts for spatial dependence, operating in discrete time over a finite state space. The model allowed us to have a propagation model conditioned by neighboring states and quantifies two key characteristics : spatial propagation timescales and propagation law in a region dependent on neighboring states. The model is inferred from data collected on the spread of Covid’19 in Madagascar’s 22 regions, using the Bayesian approach to get a better idea of model parameter values. The result has demonstrated the effect of neighborhoods on the propagation dynamics of diseases. We conclude with a discussion of potential future theoretical developments.
Citation: Raherinirina A, Tsilefa ST, Nirilanto T, Manou-Abi SM (2025) Bayesian inference of a spatially dependent semi-Markovian model with application to Madagascar Covid’19 data. PLoS One 20(7): e0326264. https://doi.org/10.1371/journal.pone.0326264
Editor: Pablo Martin Rodriguez, Federal University of Pernambuco: Universidade Federal de Pernambuco, BRAZIL
Received: January 30, 2025; Accepted: May 27, 2025; Published: July 7, 2025
Copyright: © 2025 Raherinirina et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data and all the source code used in this article is publicly available on the following github page: https://github.com/Jlingz14/Markov_chain_with_spatial_dependency.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
Following the emergence of Covid’19 and its spread around the world, many countries have implemented multiple interventions, including non-pharmaceutical and pharmaceutical interventions. The Malagasy government has launched awareness, protection and intervention programmes and campaigns. The observed epidemic data studied in this paper relate to the analysis of the spread of the Covid’19 epidemic during the first wave of the epidemic, which began in March 2020 and ended in October 2020, and was reported in all the R = 22 regions of the island of Madagascar. Many mathematical models of the Covid’19 coronavirus epidemic have been developed over many years [1,2]. To model phenomena that evolve over time, such as the evolution of a small epidemic, it is often appropriate to represent it by a stochastic process. The Markov model is widely used in epidemiology when working in a discrete state space [3–5] and also in environmental and control analysis for land use dynamics [6–11]. The power and efficiency of the Markovian approach in analysing and modelling complex systems has been known for decades [12,13]. In its various forms and extensions, the use of Markov models has become intuitive and natural for dealing with sequence data, including in epidemiology [1,14], the work of Linda J.S. Allen being one of the best references on this point [15,16]. The most natural model is that of a classical Markov chain, which imposes that the sojourn times in each state are geometrically or exponential distributed [10,11,17]. The adequacy tests performed in this study indicate that this assumption does not always hold for the observed data. A semi-Markov model may be appropriate when the sojourn times laws in different states are not geometric or exponential [7,18,19]. The results of preliminary tests on sojourn times allow us to consider other alternatives such as discrete Weibull and Poisson distributions. In addition to suitability tests on the sojourn times, we also perform spatial dependence tests to check the spatial proximity of the observed state transitions [20]. In this study, we will operate on this aspect by integrating a spatial parameter for the state transition according to the neighbouring observations. These models were introduced by [21] in order to propose new strategies for analysing the evolution of the spatial (joint) distributions of revenues in time and space, using this spatialised Markovian extension to measure the spatial dependence of a given dynamic. The semi-Markov hypothesis makes the models more flexible and tractable [22,23]. In this paper, we try to model the dynamics of Covid’19 from its arrival in Antananarivo (the capital) in March 2020 to its appearance in the other regions of Madagascar. By plotting the intensity rates of the appearance of Covid’19 in each region, we can study the dynamics of its spread in a given region according to the neighbouring regions. In this approach, we propose a classification of the intensity rates into three levels: low, medium and high. Based on the adequacy tests conducted for these three states, the classical Markov assumption appears not to always hold. This motivates the use of semi-Markov models, which provide a natural extension of classical Markov models by allowing greater flexibility in the modeling of sojourn times. We will also address a spatial extension of the semi-Markov model. The remainder of the paper is organized as follows. First, the data-driven model selection process by performing statistical tests on the fit of the data to the sojourn time laws and the spatial dependence of the observations. The results of the statistical tests then lead us to the spatialisation of the semi-Markovian model. Secondly, we perform statistical inference on the model parameters. To do this, we use a Bayesian approach to overcome the data size constraint using expert knowledge. Finally, we discuss the estimation and simulation results of the model and highlight future theoretical and future work.
2 Material and methods
The methods outlined below are based on the daily number of Covid’19 cases during the first wave of the pandemic, spanning March to October 2020, across the 22 regions of Madagascar presented in Fig 1.
2.1 Data
The daily number of new cases reported during the above study period is shown in Fig 2.
The spatio-temporal evolution of confirmed cases across these regions is presented in Fig 3. To compute the frequency distribution of the cases for each region, we will also consider the population size as well as the total number of cases cumulated.
2.2 Markov models and spatial dependency
Markov models are the most popular form of analytical framework for statistical models used to describe systems transition from one state to another in a state space(finite). These models are widely used in various fields, such as epidemiology, engineering, decision-making processes and so on. They are simple, well-suited to systems with clear state transitions and computationally efficient, with sound theoretical foundations and well-established analysis methodology. Preliminary results presented in [20] used transition matrices with spatial dependency and neighborhood-conditioned sojourn time distribution to test whether there is a significant difference in spatial lag, determined by the states of neighboring regions. This earlier study highlighted the importance of incorporating spatial dependence to accurately model propagation dynamics, with the difference of sojourn time laws and independence of neighborhood in sojourn times concluding the relevance of a non-Markovian model. On this basis, the present paper adopts semi-Markovian models with spatial dependency in state transition. Maximum likelihood estimation (MLE) is a point estimator used to estimate the appropriate parameters given the observed data. In contrast, the Bayesian method [24] incorporates prior information about the parameters by estimating their distribution. It is widely recognized that the Bayesian approach often provides more accurate results than MLE. The following outlines some key methodological considerations.
For a given region , let nrj denote the number of Covid’19 cases recorded during the j-th observation week (
, with 21 weeks of observation in our dataset). The intensity of Covid’19, denoted by
, for region r during week j is defined as (see [20,25]) :
We then highlight three distinct state profile observations as follows. The choice of the threshold, as described in Table 1, corresponds to the statistical position parameter, ensuring an approximately homogeneous distribution of states. This approach allows for the classification of regions into comparable categories based on their epidemic intensity.
Consider a stochastic process with value in this state space, denoted by E. For a state
, Ti defines the sojourn time in state i and is given by
. After identifying the different types of sojourn time distributions, as detailed in the next section, we assume that the distribution of sojourn times in the LowLow state depends on the nature of the transition originating from that state. More precisely, we assume that for transitions from the LowLow state to the HighHigh state, the sojourn times follow a discrete Weibull distribution, whereas for transitions from the LowLow state to the MediumMedium state, the sojourn times follow a Poisson distribution.
2.2.1 Spatial dependency analysis.
We are concerned here with spatial homogeneity tests, which consist in identifying some geographical areas of Madagascar according to the intensities of their spatial neighbors. The given k nearest neighbors of the structure in terms of intensity [20,26] is given in Fig 4 for k = 3. The Markov transition matrix can be modified so that the transition probabilities of a given region depend on the intensity class of its spatial neighborhood. Rey [21] developed a theoretical framework that expanded the conventional Markov chain to include the influence of neighbors (spatial dependence) on the progression of the distribution and the probability of transition, incorporating a spatial delay. The first column of Table 2 displays the state of the spatial lag (space neighbor), i.e. the state in which the average Covid’19 intensity per capita of the neighbors of a region belongs to it. The transition probability from the neighboring class i to the neighbouring class j, given that the neighbors are in class v, is denoted by . This results in a three-dimensional probability hypermatrix
corresponding to the transition matrices of the process conditioned by the neighborhood states [21].
The spatial homogeneity test is conducted using the previous k = 3 nearest neighbours, with spatial autocorrelation indices measuring the spatial dependence between values at different locations in space. The Moran diagram facilitates the interpretation of the spatial structure see [27]. The Local Moran’s I statistic quantifies the degree of spatial autocorrelation around each spatial unit within the study area. It serves as a Local Indicator of Spatial Association (LISA), capturing localized clustering patterns of similar or dissimilar values. Formally, it is defined as:
where:
- R is the total number of spatial units (regions),
- i and j are indices corresponding to two different spatial units
,
- Ai and Aj denote the observed values at locations i and j, respectively,
is the mean of the observed values across all locations,
- We transform the link between points i and j into the element wij of the weight matrix W.
- The weight matrix quantifies the spatial correlation between geographical areas through the neighbourhood matrix denoted W.
The numerator measures the spatial cross-product of deviations from the mean, weighted by spatial proximity, while the denominator normalizes this by the overall variance of the observations. The value of the global Moran’s I statistic lies between –1 and 1. A value close to 1 indicates strong positive spatial autocorrelation (similar values cluster together). A value close to –1 indicates strong negative spatial autocorrelation (dissimilar values are neighbours) and a value near 0 suggests spatial randomness.
2.2.2 A semi-Markovian model with spatially dependence.
We denote by the set of real matrices on
and by
the set of matrix-valued functions defined on
, with values in
. Consider a discrete-time stochastic process
on the probability space
with value in E that describes the state of the system through the following tools:
- The discrete process
with state space E measures the n-th jump time.
- The discrete process
represents the successive time points at which state transitions occur in
.
- The chain
with state space
,
describes the sojourn time in state Jn−1.
The matrix-valued function q= is a discrete-time semi-Markov kernel define by :
It denotes the probability of transitioning from state from a state i to another state j after a stay of k time units in i. We refer to [28,29] for more details on the construction of Semi-Markov chain. The discrete time process is the semi-Markovian Chain associated with the Markovian renewal chain
if :
where X0:n−1 denotes . When Equation is independent of n, the Markov renewal chain (J,S) is said to be homogeneous. Generally, semi-Markov chains are not Markov processes, as they do not satisfy the Markov property when the sojourn time distribution deviates from a geometric distribution. Furthermore, a semi-Markov process allows arbitrarily distributed sojourn times within any state while preserving the Markov property for the embedded (discrete-time) Markov chain. For more details on the construction of semi-Markov chains, refer to [28,29]. We denote by
the transition matrix of (Jn), incorporating the spatial conditionality as defined in [20,30]:
We will consider the following assumption for all :
- The conditional distribution fij(.) of
is independent of the neighborhood state:
- The conditional cumulative distribution Fij(.) of
is given by
Note that, for alland for all
, the following relation holds:
(1)
- The sojourn distribution time, hi(.), in state i is given by :
- The sojourn cumulative distribution time, Hi(.), in state i is given by the following formula:
Note that, from Eq (1), the semi-Markov kernel satisfies the following relation:
for all and
such that
. It is clear that the changing state is controlled by the following hypermatrix, according to the dependence of the neighborhood region.
for and
and then, the sojourn time distribution is as follows :
where is the parameter of the Poisson distribution,
represents the parameters of the discrete Weibull distribution, and
denotes the parameters of the geometric distributions considered.
2.3 Bayesian statistical inference for parameter estimation
We consider a Bayesian approach to estimate the parameters described above. The posterior distributions will be computed using the Monte Carlo Markov Chain (MCMC) approximation method, employing algorithms that have proven effective in similar contexts [18]. Specifically, we aim to estimate the elements characterizing the semi-Markov process with spatial effects, including the transition hypermatrix governing state transitions and the conditional sojourn time distributions. To this end, we employ statistical estimation within a Bayesian framework [31,32]. As previously mentioned, the transition dynamics of a semi-Markov process are fully characterized by the transition matrix of the embedded chain P and the conditional sojourn time distribution f [11]. Let nr denote the length of the observations associated with region r and the observations of the Covid-19 intensity in region r. The neighborhood effect can be represented using a hypermatrix as described above. After recording the sojourn times
for a given region r, we can compute the times of state changes and the censored sojourn time for the last state visited, denoted by
. Now, denote by
the number of state changes observed for region r. We introduce the following notations:
- The value nij(k) or
- is defined as the transition number from i to j (assuming that there is a total of k transitions), where Jl is the included chain that govern the succession states of X.
- The value
- is defined as the transition number from i to j, knowing that the region in the neighbourhood is v, where
is the included chain that govern the succession states of X and
the states of the process in the neighboring regions of r.
- The quantity
- is the number of transitions from i to any other state, knowing that the neighbouring region is v and set
.
The empirical estimator of the transition matrix and the conditional sojourn time distribution fij are defined as follows:
We have
where is the initial law of
and
corresponds to the survival function associated with the time spent in the last visited state [11]. We assume in the following lines that such survival quantities will be neglected (censored data) [33]. The likelihood function associated with the model is given by
Using the expressions for P and f in the equations (2) and (3), the likelihood can be written as:
where is the parameter value of the transition hypermatrix.
2.3.1 Bayesian approach with informative prior distributions for estimating sojourn time laws.
Now, let us proceed to estimate the parameters described above using a Bayesian approach. We assume that the conditional sojourn time distributions are spatially homogeneous but characterized by different parameters, as follows:
where represents the parameter of the Poisson distribution,
denotes the parameters of the discrete Weibull distribution, and
corresponds to the parameters of the geometric distributions under consideration. In the following, we choose to estimate only the parameters
and
using a Bayesian approach. This decision stems from the fact that estimating the parameters of the discrete Weibull distribution within this framework would incur significant computational costs. For the geometric distribution parameter, we will use a Beta distribution as an informative prior, since it is defined over the interval [0,1]. For the Poisson distribution parameter, we will adopt a Gaussian distribution as the prior. We can then decompose (4) as follows:
where
Under the classical framework, the Maximum Likelihood estimators of the parameters are as follows [6]:
Now, based on the Bayesian estimation framework, the distribution of an entire parameter vector , conditional on the observations, referred to as the posterior distribution
is derived using Bayes’ theorem:
where is the likelihood function, which is simply the distribution of the observations conditional on
.
2.3.2 Bayesian approach with non-informative prior distributions for estimating the transition matrix.
This approach leverages Bayesian inference to incorporate uncertainty in disease transmission dynamics, particularly useful when prior knowledge is limited or data is sparse. For the parameters of the transition hypermatrix of the embedded chain, denoted as , we adopt the non-informative Jeffreys prior [31]. This choice ensures invariance under reparameterization and avoids introducing subjective bias, which is critical in epidemiological applications where unbiased estimation is paramount [34]. The Jeffreys prior is derived from the Fisher information matrix
, which quantifies the expected curvature of the log-likelihood function:
and is the log-likelihood of the observed transitions. This prior is particularly advantageous for multinomial transition models, as it automatically adapts to the scale of the parameters.
Applying Bayes’ theorem, the posterior distribution combines the likelihood with the Jeffreys prior:
The Bayesian estimator is the posterior mean, which minimizes the expected quadratic loss:
3 Results and discussion
This section begins with a statistical analysis of the Covid’19 epidemic curve on the island of Madagascar using a simple Markov model. The above data seems to reveal that the epidemic exhibited distinct dynamic behaviors across Madagascar’s 22 regions. In our study of COVID-19 intensity, each region is classified into one of three distinct states reflecting increasing levels of epidemic severity: Low , Medium
, and High
At any given time t, the Covid’19 intensity level variable takes one of these three values, corresponding to a particular state, see Fig 5 for the observed weekly intensity sequence. To provide an overview of the disease’s progression across the country, Fig 6 shows the general trend in the 22 regions during the observation period.
Since the transition between two successive states depends only on the Covid’19 intensity level at the start time, a first-order time-dependent 3-state Markov chain can be constructed. We now turn to justify the choice of our modelling approach based on Markovian testing hypotheses [35].
Note that in the classical Markov assumption it remains to assume that the sojourn-time laws are geometrically distributed [18]. To evaluate such hypotheses, we propose somme Statistical test on the sojourn times laws. For instance, one can use the classical Kolmogorv-Smirnov test [36] to check the adequacy of the geometric law for the sojourn times. The geometric distribution on of parameter
,
is defined by:
Fig 7 shows the superposition with a geometric distribution and the results of the goodness-of-fit test in Table 3.
The above result leads us to examine the following laws for the sejour times laws of the intensity rate. In the case of the state we studied the sejourn time on the states separately according to the type of transition. For the transitions
and
, we suggest performing goodness-of-fit tests with other distributions, such as the Poisson distribution and the discrete Weibull distribution as shown in Fig 8. We also present the associated goodness-of-fit tests in Table 4. Recall that the discrete-time Weibull distribution with parameter vector
is defined as follows:
where and
In the following lines, we’ll analyse the spatial effect in the transition probability. Building on the previous statement, it is also insightful to consider the use of non-geometric laws for the sojourn times within the states. Therefore, for the rest we will consider a semi-Markovian model in which the transition probabilities of a region depend on its spatial lag. Such spatial dependence will also be analysed. The Moran statistical test, as outlined in Fig 9 for the global spatial autocorrelation and Fig 10 for the Local indicators of spatial association (LISA) has been employed to validate the neighbourhood zones. It demonstrates a p-value of 0.1 when evaluated against the criterion of local spatial autocorrelation.
We can also assess the spatial effect by comparing the elements in Table 2. The influence of neighboring regions becomes evident, for instance, when . This indicates that the probability of regions in class 2 (medium intensity) transitioning to class 3 (high intensity) is higher without considering their neighbors is higher than when these regions have low-intensity neighbors. Similarly, if
, the probability of regions transitioning from class 1 (low intensity) to class 2 (medium intensity) is higher when they have neighbors in class 3 (high intensity) compared to the average transition probability. Conversely, if neighbors exert no influence, the transition probabilities would satisfy the condition
, indicating uniformity of the neighboring regions’ classes [20,37]. The above data analysis show the following. Firstly, the law of the sojourn times on a given state does not ollow a geometric distribution, depending on the nature of the transitions observed. Secondly, the evolution of the intensity of the Covid’19 in a given region depends on the values of the Covid’19 intensity in neighbouring regions.
This means that a semi-Markovian model that includes a spatial effect on the transition matrix could be insightfull. Applying log-likelihood maximization of (5) like that [12,18,38], we obtain :
The Bayesian estimator of
is the mean of the posterior distribution, and it is given by:
We will use the Markov Chain Monte Carlo (MCMC) method [31,32]. This methods consist in simulating a target probability distribution from an ergodic Markov chain. These techniques were introduced by Metropolis et al. [39], they enable numerical sampling according to the a posteriori law. This approach has been previously applied to agrarian dynamics problems [18].
Algorithm 1. Metropolis-Hastings algorithm with Gaussian kernel: the target distribution is and the proposal distribution is
, where g is the density associated with the centered normal distribution
.
Applying the bayesian approach combined with the MCMC method, Fig 11 shows the samples and the evolution of the estimators as a function of iteration. The sojourn time distributions are estimated as follows:
The count of transitions in the dataset is presented in Table 5.
We displayed in Fig 12, the Bayesian estimation using Markov Chain Monte Carlo (MCMC) method as well as the Maximum Likelihood estimation.
Bayesian estimators of transition matrixes conditioned on neighborhood states are shown in Fig 13.
3.1 Limit laws, simulation and time scale of dynamics
We will first examine the law of limits as conditioned by neighbourhood. With the above estimators, the included Markov chains are all irreducible, and the state sojourn times have finite means. Therefore, the spatial semi-Markov chain admits limit laws conditionally on the neighbors.
where is the invariant law of the included Markov chain
and
the vector containing the mean of the sojour times on the states [33,38,40]. After calculation, we have determined that:
and
The limit laws represent the equilibrium state of the pandemic, conditional on the neighboring regions (see Fig 14).
From such results, we observe that once the transition laws stabilize, the Covid’19 cases will predominantly remain in the Medium state, regardless of the state of neighboring regions
3.1.1 Numerical simulation.
Using the numerical simulation method, we can simulate the model over a long trajectory. As the simulated trajectory progresses, the time-scale laws will converge towards a distribution. The simulation method is based on the following algorithm.
Algorithm 2. Simulation algorithm for the semi-Markov model with spatial dependence.
This algorithm simulates semi-Markov dynamics with spatial dependency by generating a state trajectory X0:Horizon over a given horizon for the different regions. Starting from a state i, it identifies the state of influence neighbors v depending on the neighboring states , then updates the next state j according to a transition law q conditioned by v; and finally it simulates the sojourn time S on i towards j according to a law fij(k). This approach makes it possible to simulate several model trajectories for numerical study after a few evolution time distributions. Fig 15 shows an example of the trajectory of this model.
Now, we can compute the law of the expected return time for instance to the low state, i.e. the recovery time, and the time to return to the high state, knowing the state of the neighbors based on several simulation samples (see Figs 16 and 17).
For the recovery time following contamination in a region with “low” intensity neighbors, the expected return time is 5.45 weeks. In contrast, for a region with “Medium” intensity neighbors, the expected recovery time is 10.95 weeks, and for a region with “High” intensity neighbors, the the expected recovery time is 19.66 weeks.
For the return to epidemic state after recovery in a region with “low” intensity neighbors, the expected time is 62.95 weeks. For a region with “Medium” intensity neighbors, the expected return time is 19.47 weeks, while for a region neighboring “High” intensity areas, the return time is 9.03 weeks. To conclude this section we conducted a simulation of the spread of the epidemic in space.
The Covid’19 began in Antananarivo, the capital city and from the international airport and then spread across Madagascar. Using simulation methods, we estimated the expected duration for each phase of propagation. The results are shown in Fig 18. It was observed that, after the epidemic reached the capital, the province of Atsinanana would transition to the “High” state most rapidly, with an expected duration of 6.7 weeks. In contrast, the duration in the other provinces showed little variation: Antsiranana reached this state in 7.87 weeks, Mahajanga in 7.53 weeks, Matsiatra Ambony in 7.96 weeks, and Toliary in 7.81 weeks.
The line represents the kernel estimate of the empirical distribution.
4 Conclusion and perspectives
This study focuses on spatializing a semi-Markovian dynamic, with particular attention to the intensity conditions of neighboring regions. In our model, the spatial component is integrated by considering the states of neighboring areas, with a specific application to the dynamics of Covid’19 in Madagascar. We began by conducting statistical tests to assess our hypotheses. The semi-Markovian nature of the dynamics was confirmed through a goodness-of-fit test for the geometric distribution, which was rejected for certain state residence times. Regarding the influence of neighboring regions, statistical tests based on the global and local Moran’s index confirmed the dependence of several regions on their three nearest neighbors. The parameters of the spatially dependent semi-Markovian model were estimated using both maximum likelihood maximization and a Bayesian approach. These methodologies provided precise estimates, reinforcing the reliability of our model to reflect the epidemiological reality in Madagascar. By estimating the spatial transition matrix conditioned on neighbors, where the state of a region is determined by the maximum intensity of its three nearest neighbors, and confirming this spatial dependence through a homogeneity test, we constructed a model characterized by the hypermatrix of the embedded chain and the laws of residence times. The model’s analysis led to key insights regarding the spread of Covid’19 in Madagascar, particularly the temporal scales of propagation and spatial diffusion across the country. In conclusion, our study introduces an innovative approach to modeling dynamic systems using a spatially dependent semi-Markovian model. The statistical tests confirmed the validity of our model from both the semi-Markovian and spatial perspectives. The results offer valuable insights into the spread of Covid’19, with both temporal and spatial applications in Madagascar. These findings can guide public health policies and help implement targeted strategies in regions that are highly influenced by their neighbors. Looking ahead, we plan to investigate a deep understanding of a kind of spatial Markov model including a mathematical analysis of their asymptotic behavior. In addition, more realistic distance measures to improve the accuracy of our modeling approach. These future developments could contribute to a deeper understanding of spatial Markov models in decision-making to manage transitional dynamics in various contexts.
References
- 1. Dietz K, Heesterbeek JAP. Daniel Bernoulli’s epidemiological model revisited. Math Biosci. 2002;180:1–21. pmid:12387913
- 2. Kermack WO, McKendrick AG, Walker GT. A contribution to the mathematical theory of epidemics. Proc R Soc Lond A. 1927;115(772):700–21.
- 3.
Bouzalmat I, de Saporta B, Manou-Abi SM. Parameter estimation for a hidden linear birth and death process with immigration. In: 2023. https://arxiv.org/abs/2303.00531v2
- 4.
Tsilefa SF, Manou-Abi S, Raherinirina A. Numerical convergence of some stochastic compartmental models in epidemiology. 2022. https://inria.hal.science/hal-03715840
- 5. Raherinirina A, Fandresena TS, Hajalalaina AR, Rabetafika H, Rakotoarivelo RA, Rafamatanantsoa F. Probabilistic Modelling of COVID-19 Dynamic in the Context of Madagascar. OJMSi. 2021;09(03):211–30.
- 6. Campillo F, Hervé D, Raherinirina A, Rakotozafy R. Markov analysis of land use dynamics - a case study in Madagascar. Revue Africaine de Recherche en Informatique et Mathématiques Appliquées. 2014;Volume 17 - 2014 Special issue CARI’12:1–22
- 7. Kharchenko V, Ponochovnyi Y, Ivanchenko O, Fesenko H, Illiashenko O. Combining Markov and semi-Markov modelling for assessing availability and cybersecurity of cloud and IoT systems. Cryptography. 2022;6(3):44.
- 8. Bickenbach F, Bode E. Evaluating the Markov property in studies of economic convergence. Int Reg Sci Rev. 2003;26(3):363–92.
- 9. Claudio D, Moyce S, Albano T, Ibe E, Miller N, O’Leary M. A Markov chain model for mental health interventions. Int J Environ Res Public Health. 2023;20(4):3525. pmid:36834220
- 10.
Privault N. Discrete-time Markov chains. Springer Undergraduate Mathematics Series. Springer Singapore; 2018. p. 89–113. https://doi.org/10.1007/978-981-13-0659-4_4
- 11.
Raherinirina A. Modélisation markovienne des dynamiques d’usage des sols. Cas des parcelles situées sur le bord du corridor forestier Ranomafana-Andringitra; 2018. https://tel.archives-ouvertes.fr/tel-00870679
- 12. Anderson TW, Goodman LA. Statistical inference about Markov chains. Ann Math Statist. 1957;28(1):89–110.
- 13. Yao Q, Wang Y, Xiong X, Wang P, Li Y. Adversarial decision-making for moving target defense: a multi-agent Markov game and reinforcement learning approach. Entropy (Basel). 2023;25(4):605. pmid:37190393
- 14. Gani J, Jerwood D. Markov chain methods in chain binomial epidemic models. Biometrics. 1971;27(3):591–603. pmid:5116573
- 15.
Allen LJS. An introduction to stochastic epidemic models. Lecture notes in mathematics. Springer Berlin Heidelberg; 2008. p. 81–130. https://doi.org/10.1007/978-3-540-78911-6_3
- 16. Allen EJ, Allen LJS, Arciniega A, Greenwood PE. Construction of equivalent stochastic differential equation models. Stoch Anal Appl. 2008;26(2):274–97.
- 17. Smith WL, Stoneley R. Regenerative stochastic processes. Proc R Soc Lond A. 1955;232(1188):6–31.
- 18.
Raherinirina A, Hervé D, Campillo F. Inferring land use dynamics by semi-Markov model. In: 2014. 1. https://hal.inria.fr/hal-01100087
- 19. Gibbon J. Scalar timing and semi-markov chains in free-operant avoidance. J Math Psychol. 1971;8(1):109–38.
- 20. Tabera Tsilefa S, Raherinirina A. Spatial Markov matrices for measuring the spatial dependencies of an epidemiological spread: case Covid’19 Madagascar. BMC Public Health. 2024;24(1):2243. pmid:39160542
- 21. Rey SJ. Spatial empirics for economic growth and convergence. Geogr Anal. 2001;33(3):195–214.
- 22.
Janssen J, Limnios N. Semi-Markov models and applications. 2011. Available from: https://api.semanticscholar.org/CorpusID:117813955
- 23. Vassiliou P-CG, Georgiou AC. Markov and semi-Markov chains, processes, systems, and emerging related fields. Mathematics. 2021;9(19):2490.
- 24.
Christian R. The Bayesian choice. Springer texts in statistics. New York, NY: Springer; 2007. Available from: http://link.springer.com/10.1007/0-387-71599-1
- 25. Wang Y, Zhao C, Liu Z, Gao D. Spatiotemporal analysis of AIDS incidence and its influencing factors on the Chinese Mainland, 2005-2017. Int J Environ Res Public Health. 2021;18(3):1043. pmid:33503938
- 26. Can A. The measurement of neighborhood dynamics in urban house prices. Econ Geogr. 1990;66(3):254.
- 27.
Getis A. Spatial autocorrelation. Handbook of applied spatial analysis: software tools, methods and applications. Springer; 2009. p. 255–78.
- 28.
Limnios N, Oprisan G. An introduction to semi-Markov processes with application to reliability. In: Shanbhag DN, Rao CR, editors. Stochastic processes: modelling and simulation. Elsevier; 2003. p. 515–56.
- 29.
Barbu V, Limnios N. Semi-Markov chains. Lecture notes in statistics. Springer New York; 2008. p. 1–32. https://doi.org/10.1007/978-0-387-73173-5_3
- 30. Silver D, Silva TH. A Markov model of urban evolution: neighbourhood change as a complex process. PLoS One. 2021;16(1):e0245357. pmid:33449942
- 31.
Marin JM, Robert C. Bayesian core: a practical approach to computational Bayesian statistics. Springer Science & Business Media; 2007.
- 32. Alyousifi Y, Ibrahim K, Kang W, Zin WZW. Robust empirical Bayes approach for Markov chain modeling of air pollution index. J Environ Health Sci Eng. 2021;19(1):343–56. pmid:34150239
- 33.
Barbu VS, Limnios N. Reliability of semi-Markov systems in discrete time: modeling and estimation. Handbook of performability engineering. Springer London; 2008. p. 369–80. https://doi.org/10.1007/978-1-84800-131-2_24
- 34. Kass RE, Wasserman L. The selection of prior distributions by formal rules. J Am Stat Assoc. 1996;91(435):1343–70.
- 35. Solow AR, Smith WK. Using Markov chain successional models backwards. J Appl Ecol. 2006;43(1):185–8.
- 36. Massey FJ Jr. The Kolmogorov-Smirnov test for goodness of fit. J Am Stat Assoc. 1951;46(253):68–78.
- 37. Dehghan Shabani Z, Shahnazi R. Spatial distribution dynamics and prediction of COVID‐19 in Asian countries: spatial Markov chain approach. Reg Sci Policy Pract. 2020;12(6):1005–26.
- 38.
Barbu VS, Bulla J, Maruotti A. Estimation of the stationary distribution of a semi-Markov chain; 2012. p. 15--26.
- 39. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E. Equation of state calculations by fast computing machines. J Chem Phys. 1953;21(6):1087–92.
- 40. Moore AD. The semi-Markov process: a useful tool in the analysis of vegetation dynamics for management. J Environ Manag. 1990;30(2):111–30.