Markov chain.

Since anomaly is defined based on the sequence of activities, it is necessary to profile an individual’s daily routine using a probability-based model where the probability of each step (or activity) depends on the previous step [21]. The simplest such model is a conventional Markov chain in which a stochastic process x = {x₁, x₂, ⋯, x_l} consists of a finite number of states S = {s_a, s_b, ⋯, s_f} (where x_i ∈ S) and the probability of each step x_i only depends on the value of the preceding step x_i−1 and not on the entire sequence [21–23], i.e. (1) in which P_i−1,i is the transition probability of x_i−1 → x_i. Each Markov chain can be represented as a Markov model with a corresponding transition matrix. For instance, in Fig 2, a Markov chain x with four states (a, b, c, d) is represented with the transition matrix T_x.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. A sample Markov model with 4 states.

https://doi.org/10.1371/journal.pone.0195605.g002

Entropy rate.

Generally, the day-to-day activities in home can be seen to be random or regular in short or long period of time, as daily activities do not necessarily follow a specific pattern in short time spans but over long-term observations some specific patterns could be observed. It can be assumed that each person has a ‘ground-state’ of typical routines which are mixed with some fluctuations. We use an information theory based parameter (i.e., entropy) to evaluate these fluctuations [24].

Entropy is a well-known measure for quantifying information and indicating the degree of random occurrence in a succession (in our case the Markov chain) and it has been extensively studied for pattern detection [23, 24]. Given the transition matrix T_x, the entropy value is calculated as [22, 25, 26]: (2) in which ϵ is the entropy value; P_αβ are the transition probabilities for a transition from state α to state β in a Markov chain. Larger ϵ values indicate higher uncertainty level (randomness) and the entropy value reaches its maximum if all the states occur with equal probabilities. The entropy defined in Eq (2) only depends on the ‘transition probabilities’ between states and it ignores the state probabilities. To incorporate the state probabilities, we use the entropy rate value which is defined as [27, 28]: (3) in which ξ is the entropy rate. The entropy rate depends on both the state probability (P_α) and the transition probability (P_αβ) which enhances the discrimination ability of the algorithm. For instance, if P_αβ = P_βα and P_α ≠ P_β, the calculated entropy rate differentiates between transition patterns from α to β and β to α, while the entropy value assigns the same value to both patterns.

The proposed algorithm.

We have used a Markov chain model and entropy rate to develop an algorithm that identifies the daily routines and detects the unusual patterns of people with dementia. We first construct a Markov chain model based on an individual’s historical data to identify their regular daily routine or sequence of events. Although an individual’s activities can be quite complicated, they usually follow a sequence of events in specific locations and time ranges within each day [24]. In other words, the proposed algorithm observes the participants’ day-to-day activities in a quantitative fashion and detects anomalies by analysing the deviation and differences in the distribution of the Markov model. The entropy rate of the historical data is used to define the ‘deviation boundaries’ and any test data exceeding these boundaries is detected as an anomaly. In order to select the deviation boundaries (i.e. defining how much deviation is permitted), the historical data is divided into two sets as ‘training and verification’. The training set is used to construct the Markov model by learning the state transition probabilities while the verification set is used to define a confidence threshold for the deviation range.

Data pre-processing.

The environmental data used in this study is collected from a subset of passive sensors installed in different locations within residential homes. We have deployed 9 sensors in total: 2 PIR sensors (installed in the hallway and living room), 3 motion sensors (one in the kitchen and 2 on the bedroom and bathroom doors), 2 pressure sensors (installed on the chair and the bed), 1 main entrance door sensor and 1 central energy consumption monitoring device. The data is therefore collected from multiple sources (provided by 7 different companies) in uncontrolled real-world environments, i.e. residential homes. This means that data may be noisy and incomplete (containing missing values) with various sampling rates for different source. We first apply pre-processing techniques to only include those participants who have at least three sensors reporting continuously for the last three months. We then generate the training and verification sets for each of these participants. The raw data is filtered and aggregated over 10-minute intervals; i.e. each day is represented by a 24 × 6 window of 10-minute intervals and each window is an array of n elements where n is the number of active nodes for one participant. In addition, to perform feature scaling, the aggregated data is normalised such that the activity level of each sensor is ranged between 0 to 10. This ensures that all sensors are represented within the same range, regardless of their sampling rates and type of measurements, where they are equally taken into consideration for further analysis, such as K-means algorithm [29].

Prior to constructing the trained Markov model, the integrated data is clustered into two categories using a K-means algorithm [29]. This is particularly important as individuals typically exhibit different patterns of daily activities on high-active days (e.g. being home most of the day) compared to low-active days. These clusters are sorted based on their total number of observations, so that categories D1 and D2 represent low-active and high-active days, respectively. In the next step, a second K-means algorithm (with n centroids) is used to map each n-dimensional window to a single state. These states are then used to construct the Markov Model. The transition probabilities between the states are subsequently learned using the training set observations. The learned model is then used to infer an individual’s routine and detect anomalies, as described below. Notice that clusters are estimated only based on the training dataset and the same model has been then re-used for the verification and test sets, meaning that centroids of the K-means clusters have been fixed throughout the analysis.

Anomaly detection.

To analyse the level of variation/randomness for each participant, the pre-processed dataset, including D1 and D2 categories, is divided into the training and verification sets. The training sets are used to construct two Markov models with transition matrices T_D1 and T_D2. This results in two separate models for high activity and low activity days. The number of clusters and models representing them can increase based on a need for more detailed and specific models. In this paper, we demonstrate our proposed solution based on 2 clusters.

In the next step, we quantify the information and analyse the randomness based on the consecutive state transitions. To achieve this, the transition probabilities for the Markov chain are learned and the entropy rate is calculated based on the chain. For instance, the transition probabilities for a Markov chain {a, b, a, a, c, b} are represented with {P_aa, P_ab, P_ba, P_aa, P_ac, P_cb} and the entropy rate is calculated as (4) where P_aa represents the state probability and P_ab represents the state transition probability. After calculating the entropy rate for the training data (ξ_T), this process is repeated to calculate an entropy rate for each day from the verification set, i.e. generating ξ_V = {ξ₁, ⋯, ξ_v} in which v is the number of verification days. These values are used to determine the expected deviation boundaries by calculating the confidence interval Δ as: (5) in which μ is the confidence coefficient. Given the above, anomalies are identified as days in which entropy rates exceed the expected boundaries (ξ > Δ), i.e., days that their sequence distribution highly deviates from the trained Markov model.

The proposed method is summarised in S1 Algorithm (see S1–S3 Algorithms). The proposed macro assessment algorithm identifies unusual patterns over a longer time scale (i.e., daily and weekly analysis). The next section describes the micro assessment which is focused on detecting patterns on a short-time span.

Sensing layer.

The sensing layer consists of multiple sensing modalities which are categorised into subgroups G₁, G₂ and G₃ based on the context of the collected data. For example, the physiological monitoring sensors belong to G₁ and are used to collect blood pressure, heart rate and body temperature of the participant. These sensors are wirelessly connected to a gateway, enabling them to automatically relay the measurements to a server upon activation. On the other hand, G₂ and G₃ consists of environmental sensory devices, see Fig 3. As shown, G₂ includes motion activity patterns collected from the chair, pillbox and bed of the participant, whereas G₃ contains the motion data that is collected from sensors deployed in the living room, hallway and kitchen. It can be seen from Fig 3 that the deployment of G₂ sensors are closely associated with the areas of participant’s movement, whereas G₃ sensors record the multi-occupancy movement pattern within a home. The formation of groups G₂ and G₃ not only helps us to contextualise movement activity (i.e., allowing us to decouple participant specific motion activity in comparison to the aggregated movement indicator) but also allows us to learn the relationship between AIA events and specific activity patterns. Each subgroup in the sensing layer pre-processes the collected observations, aggregates it and forwards it to the respective processing modules in decision layer I.

Decision layer I & II.

The decision layer I receives the measurements from the sensing layer and processes them further using expert system decision scoring and categorisation algorithms as detailed in S2 and S3 Algorithms, respectively (see S1–S3 Algorithms). The expert decision scoring algorithm takes an input from G₁ that consists of physiological symptom monitoring sensors CS_i, where i ∈ G₁. As shown in S2 Algorithm, the decision score D_CS is calculated based on comparing the observations O_i belonging to CS_i against the standard clinical thresholds and (provided by the clinical experts). For each CS_i, a threshold comparison is performed and subsequently the value of D_CS is incremented only if the O_i is outside the threshold values. The decision score is then forwarded to the fusion layer for final inference as shown in Fig 3.

On the other hand, the categorisation algorithm is applied independently to the measurements obtained from G₂ and G₃. At first, a mapping procedure is performed in which hourly aggregated measurements P_x from each of the respective sensors within a group is mapped to a unique alphabetical label. Once the label assignment for each sensor is complete, the algorithm outputs a string (S) by assigning the generated labels as detailed in S3 Algorithm. The label assignment step is performed as follows. Firstly, a label set A_x corresponding to each sensor within a target group is pre-stored in the database. A_x consists of labels that represent two states: normal and abnormal. Each label is unique such that intersection amongst any label set would result in an empty set. The mapping of P_x to a specific label from A_x is determined by threshold comparison. This requires estimation of appropriate upper and lower threshold boundaries and for each target sensor within G₂ and G₃.

To estimate the threshold values for label assignment, we employed an adjusted box-plot method as proposed by Hubert and Vandervieren [32]. We used three months of historical motion sensor data for each target participant as an input to the box-plot method. A standard box-plot gives information regarding the shape, variability, and centre (or median) of a data set and is often used to detect outliers. However, an advantage of using adjusted box-plot in comparison to box-plot is the ability to handle skewed data distributions. Since our data is noisy, we selected the adjusted box-plot method to compute the threshold values of and . The label assignment makes use of these boundary values to select a label from A_x. For example, if the observed measurement falls within the boundary values, a label corresponding to a normal state would be selected or vice-versa. Once the mapping is finished the categorisation block of G₂ and G₃ generates the sequence of labels S_G2 and S_G3, respectively, and forwards it to the decision layer II for further processing.

The label sequences S_G2 and S_G3 are passed on to their respective Hidden Markov Models (HMM) as shown in Fig 3. HMM [33] is well known to model sequential data and it addresses the problem of finding hidden states that might have generated the observations. HMM is based on stochastic processes and computes the joint probability of a collection of hidden states and observed variables. In our case, we are interested to find if the observed movement patterns exhibit participants’ normal routine or symptoms of agitation based on the hourly sequences S_G2 and S_G3 fed in by decision layer I. To be able to identify anomalies in the participant’s specific movement pattern as well as the multi-occupancy movement pattern, we have developed independent HMM models for G₂ and G₃. A person can stay in one state for several hours and then potentially change to a different state. The HMM works on the principle that a transition matrix describing the probabilities of transitioning between states can be learned from the observation sequence and that prior probabilities of starting in a particular state are known, so that the probability of sequence of states can be computed. In this work, participant-specific and multi-occupancy HMM models learn the joint probability of observing a sequence of labels and the hidden state of the participant (i.e., AIA or normal).

Our HMM model can be characterised by λ = {T, B, π}, where T is a transition matrix representing the possible state transitions within a model; B is matrix of observation probabilities (i.e., the probability of an observation at time t given a particular state) and π is a vector of initial state probabilities. We have used lab experiments, and the clinical team’s expertise to initialise the values of T and π. Note that applied techniques, including the Baum-Welch algorithm [33] and the Viterbi algorithm [33], can also be used to train the model and find the most probable series of hidden states; however, in this work we have used the data provided by our clinical monitoring team. For this purpose, the clinical monitoring team generated a set of data that was representative of probable AIA and normal state of the participant based on the observations. In conjunction to this, we used lab experiments to generate seed data for our training sets. Given S_G2 and S_G3, the participant-specific and multi-occupancy HMM models learn if an individual’s state is AIA or not and subsequently generate an output score D_pf and D_mo, indicating the state of the participant. Subsequently, these score are forwarded to the decision fusion layer for final inference.

Decision fusion.

The objective of the decision fusion layer is to optimally fuse the decision scores D_CS, D_pf and D_mo, such that the final decision is less sensitive to bias and variance. To achieve this, we define a reliability score for each group. To compute , the system should have the prior knowledge about the precision of individual groups for predicting class labels. A class-specific confidence estimate for each group can be computed as follows: (6) where p_i is the class-specific precision of the group, is the number of class labels (i.e., which in our case is AIA or normal) and TP and FP are true and false positive rates, respectively. The reliability scores for each group is computed as: (7) where for G₁, G₂ and G₃ is learned during the cross-validation phase. The cross-validation is a common approach in machine learning for model optimisation and performance estimation. The idea is to randomly partition the data into k-folds, and use the k − 1 fold as training data and the rest as validation data for testing the model. The cross-validation process is then repeated k times (the folds), with each of the k subsamples used exactly once as the validation data. Performance scores are averaged over k-folds and are used as a criterion for model selection. In our experiments, we chose a standard value of k equal to 10 for cross-validation. The decision score output by the respective group is then multiplied with to weigh it accordingly and subsequently summed to generate final scores for each class, as given by: (8) where g is the number of groups. Finally, the class label with the highest score is reported as the label of the test event. Performance of the proposed algorithms are evaluated using the data collected from the participants in the TIHM project, which is discussed below.