Computerized ecological momentary assessment (EMA) is widely accepted as a “gold standard” method for capturing momentary symptoms repeatedly experienced in daily life. Although many studies have addressed the within-individual temporal variations in momentary symptoms compared with simultaneously measured external criteria, their concurrent associations, specifically with continuous physiological measures, have not been rigorously examined. Therefore, in the present study, we first examined the variations in momentary symptoms by validating the associations among self-reported symptoms measured simultaneously (depressive mood, anxious mood, and fatigue) and then investigated covariant properties between the symptoms (especially, depressive mood) and local statistics of locomotor activity as the external objective criteria obtained continuously. Healthy subjects (N = 85) from three different populations (adolescents, undergraduates, and office workers) wore a watch-type computer device equipped with EMA software for recording the momentary symptoms experienced by the subjects. Locomotor activity data were also continuously obtained by using an actigraph built into the device. Multilevel modeling analysis confirmed convergent associations by showing positive correlations among momentary symptoms. The increased intermittency of locomotor activity, characterized by a combination of reduced activity with occasional bursts, appeared concurrently with the worsening of depressive mood. Further, this association remained statistically unchanged across groups regardless of group differences in age, lifestyle, and occupation. These results indicate that the temporal variations in the momentary symptoms are not random but reflect the underlying changes in psychophysiological variables in daily life. In addition, our findings on the concurrent changes in depressive mood and locomotor activity may contribute to the continuous estimation of changes in depressive mood and early detection of depressive disorders.
Citation: Kim J, Nakamura T, Kikuchi H, Sasaki T, Yamamoto Y (2013) Co-Variation of Depressive Mood and Locomotor Dynamics Evaluated by Ecological Momentary Assessment in Healthy Humans. PLoS ONE 8(9): e74979. https://doi.org/10.1371/journal.pone.0074979
Editor: Marianna Mazza, Catholic University of Sacred Heart of Rome, Italy
Received: February 23, 2013; Accepted: August 13, 2013; Published: September 13, 2013
Copyright: © 2013 Kim et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was partly supported by Grants-in-Aid for Exploratory Research , Scientific Research (A)  (to Y.Y.), and Young Scientists (B)  (to T.N.) from the Ministry of Education, Culture, Sports, Science and Technology and by Core Research for Evolutional Science and Technology from the Japan Science and Technology Agency (to Y.Y.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Ecological momentary assessment (EMA) is a method of acquiring self-reported information about a person’s subjective symptoms at the moment of recording and is used continuously in most cases to record instantaneous states of feeling without recall bias [1,2]. A paper-and-pencil diary was initially used for EMA, but it has been reported that the diary has the disadvantage of delayed data entry or even forward entry of data (i.e., “faked compliance”) . Therefore, computerized EMA, which employs a computer as an electronic diary (ED), has been developed to avoid faked compliance; in this method, the input time is automatically registered in the device. Symptom diaries derived from computerized EMA are now generally regarded as the “gold standard” in the fields of psychiatric/psychosomatic medicine and has recently attracted increasing attention as an essential component for healthcare monitoring systems based on the information and communication technology (ICT) [4,5].
EMA scores for, e.g., depressive mood and fatigue, which are the key symptoms of depression  studied mainly in this paper, inevitably fluctuate because of their repetitive nature. Characteristics of these fluctuations have been studied by examining the relationships among simultaneously measured self-reported symptoms [7,8,9]. Many studies have also investigated that the fluctuations in psychological states are associated with those in objective measures sampled momentarily. For example, psychological stress is reported to be concurrently associated with cardiovascular parameters from blood pressures [10,11,12] and salivary cortisol [13,14,15,16]. Also, the co-variations between pulmonary functions tested by a spirometer and positive/negative affect in patients with asthma were reported . In addition, some studies demonstrated that addictive behaviors such as smoking [18,19] and alcohol consumption [20,21] are related to the fluctuations in psychological states, e.g., positive/ negative affect and craving. Furthermore, the associations of momentary psychological states with self-reported physical activity were demonstrated [22,23].
To our knowledge, however, whether EMA scores co-vary with other symptoms evaluated simultaneously and with locomotor activity (habitual physical activity) measured continuously and un-obstructively, which is an important hypothesis for the interpretation of fluctuations of EMA scores, has not been rigorously tested. Therefore, the primary aim of this study was to test this hypothesis for groups of healthy subjects in two ways: evaluating (null) convergent associations among self-reported mood and fatigue scores and concurrent associations of the self-reported scores with external criteria provided by objective measures of locomotor activity, as described below.
It is considered that momentary micro-fluctuations in behavioral data, specifically those in locomotor activity capturing bodily acceleration counts in a continuous fashion, reflect the dynamics of systems organizing human behavior and can be used to probe behavioral disorders, including mental illnesses [24,25,26,27,28,29,30]. Indeed, altered locomotor activity is one of the cardinal signs of psychiatric disorders and included in their diagnostic criteria . For example, major depressive disorders are known to be characterized by the presence of symptoms associated with behavioral alterations, such as diminished physical activity, loss of energy, psychomotor retardation or agitation, and sleep disturbances . In order to quantitatively evaluate such alterations, several actigraphic studies have been conducted in depression patients by showing the significant decrease in activity levels during daytime and disruption of the circadian rhythm [24,25,26]. Furthermore, we recently found robust statistical properties of long-term (>7 days) locomotor activity altered in patients with major depressive disorders but not in healthy controls, reflecting increased intermittent bursts in the activity counts, characterized by reduced activity levels associated with occasional bursts of locomotor activity [31,32]. Therefore, the quantitative and objective evaluation of the levels and/or the intermittency of locomotor activity could provide appropriate behavioral measures capable of probing alterations in patients’ depressive mood and/or physical symptoms in daily life.
While our recent finding is thought to be important because it may lead to an objective evaluation of depressive disorders, whether changes in depressive mood are “momentarily” associated with such behavioral alterations probed by statistical properties of locomotor (or habitual physical) activity remains unknown. Thus, the secondary aim of this study was to search for the robust underlying associations between momentary depressive mood using EMA and local statistical properties of locomotor activity in healthy humans, regardless of group differences in age, lifestyle, and occupation by evaluating them with cross validation among various groups of subjects. Showing such associations is considered to be important because locomotor activity can be measured in a truly continuous manner, and finding objective correlates of momentary mood scores can compensate for a known drawback of EMA that the data based on sparse sampling may not provide contiguous data despite high levels of respondent load [33,34,35]. The continuous estimation of changes in depressive mood from locomotor activity might also help to detect the worsening of depressive mood even in healthy individuals at the beginning of pathogenic processes to depressive disorders.
The mean compliance rate, which is defined as the percentage of the number of completed recordings within 30 min from the first beep signal against the total number of time-based EMA during the study period, was 94.54% (SD = 5.96), and the mean number of recordings was 5.40 (SD = 1.58) per day for the adolescents [the mean number of event-based EMA recordings was 1.19 (SD = 0.88) per day for this group]; for the undergraduates, they were 92.03% (SD = 8.57) and 8.98 (SD = 1.43), respectively, and for the office workers, they were 95.40% (SD = 4.54) and 6.05 (SD = 4.49), respectively. The compliance rate was considerably high for all groups without significant differences (ANOVA, F2,82 = 1.92, p = 0.15). The mean scores of the three symptoms and the three local statistics of locomotor activity are summarized in Table 1.
|All||Adolescents (G1)||Undergraduates (G2)||Office workers (G3)||Multiple comparison|
|Depressive mood||39.93 (10.57)||40.00 (12.09)||40.31 (9.62)||39.34 (10.13)|
|Anxious mood||31.54 (19.25)||33.81 (21.67)||31.40 (18.09)||28.88 (17.90)|
|Fatigue||44.47 (18.43)||45.54 (19.04)||47.42 (18.18)||39.33 (17.65)|
|Local statistics of LA|
|Mean||144.44 (22.31)**||152.76 (17.97)||149.59 (22.00)||127.38 (18.72)||G3 < G1, G2|
|Skewness||0.04 (0.27)**||−0.11 (0.22)||0.11 (0.28)||0.14 (0.22)||G1 < G2, G3|
|Detrended skewness||0.06 (0.23)*||−0.03 (0.20)||0.08 (0.25)||0.14 (0.21)||G1 < G3|
Convergent associations among symptoms
Depressive mood showed significant positive correlation with fatigue (r = 0.26, p < 0.01) and anxious mood (r = 0.39, p < 0.01) for the data sets sampled from all groups. The correlation between fatigue and anxious mood was also positive and significant (r = 0.18, p < 0.01). These relations were consistent across groups (Table 2), indicating that all subjects properly performed the EMA recordings. The results also suggest the convergent validity of self-reported symptoms by EMA.
Concurrent associations of depressive mood with local statistics of locomotor activity
Univariate multilevel modeling.
The univariate multilevel analysis revealed simple correlations between the depressive mood scores and the local statistics (Figure 1). The mean and skewness as well as the detrended skewness showed significant correlations (FDR, q = 0.05). However, such relations depended significantly on the size and location of the time frame, especially for skewness (Figure 1). The mean levels of locomotor activity were negatively associated with depressive mood over a wide range of locations (60 min before to 55 min after) around the EMA recordings, and the associations were not significantly affected by the size of the time frame (data length 120–10 points) [Figure 1(a)]. This indicates that activity levels tend to be low with a higher level of depressive mood, and this relationship is consistent and robust over a long period. On the other hand, skewness (also detrended skewness) exhibited positive correlations in a limited range centered around the EMA recordings (e.g., from location −15 to 55 min with size 60 points for skewness, and −25 to 60 min for detrended skewness) [Figure 1(b) (c)]. This implies that depressive mood is associated with intermittent patterns of locomotor activity even in healthy populations, and this relationship appears instantaneously with the worsening of depressive mood. However, such a relationship disappeared with a small size of the time frame, which is partly attributed to the lack of statistical stability of skewness. Considering these results, we used the time frame centered around the EMA recordings with a size of 60 min while constructing the best-fitting model for depressive mood and examined the effects of the choice of time frame. For the univariate modeling, the coefficient of each predictor was fixed across the subjects because the inclusion of the random effect did not have a large effect on the results.
The estimated values of the univariate model coefficient for depressive mood scores are shown in a colored matrix form consisting of 25 rows (different locations) and 12 columns (different sizes). Each grid cell indicates some specific location and size of a time frame used for calculating the local statistics of locomotor activity. A color in each cell represents the value of the model coefficient (γ10) of the predictors: (a) Mean, (b) Skewness, and (c) Detrended skewness. False discovery rate with a q value of 0.05 was used as the multiple comparison adjustment. Only significant cases are shown by colors. Color bars indicate the value of the model coefficient. Note that the univariate model used for the analysis is as follows; Depressive mood scoreij = γ00 + γ10 (Local statistics of locomotor activityij) + ζ0i + εij (see Results for details).
Determination of best-fitting model.
Of all multilevel models evaluated, a model consisting of local mean and detrended skewness as the predictors was selected as the best-fitting model to describe the variations of depressive mood scores, where the predictor “mean” and intercept had a random effect. The best-fitting model is as follows: Depressive mood scoresij = γ00 + γ10 (Meanij) + γ20 (Detrended skewnessij) + ζ0i + ζ1i (Meanij) + εij, where the Depressive mood scoresij indicates a score of depressive mood at the jth recording for the ith subject; Meanij and Detrended skewnessij are the local mean and detrended skewness of locomotor activity corresponding to the jth EMA recording (depressive mood score) for the ith subject, respectively; γ00 is the average intercept across all subjects; γ10 and γ20 are the average regression slopes across all subjects for each predictor; the random terms, ζ0i and ζ1i, are the between-person residuals; and εij is the within-person residual. As expected from the results of univariate multilevel analysis, the coefficient of both local statistics was significant (Table 3), with a negative value (−0.04 ± 0.01, p < 0.01) for the mean and a positive value (1.40 ± 0.69, p = 0.04) for the detrended skewness.
|Coefficient(SE)||F value||p value|
|All data set|
|Intercept (γ00)||45.74 (2.40)||<0.01|
|Coefficient of the local mean (γ10)||−0.04 (0.01)||F1,1386 = 9.35||<0.01|
|Coefficient of the local detrended skewness (γ20)||1.40 (0.69)||F1,1386 = 4.18||0.04|
|Cross validation across the three groups|
|Intercept (γ00)||F1,82 = 0.08||0.92|
|Office workers||45.24 (4.20)||<0.01|
|Coefficient of the local mean (γ10)||F1,1382 = 0.07||0.94|
|Office workers||−0.04 (0.02)||0.07|
|Coefficient of the local detrended skewness (γ20)||F1,1382 = 2.88||0.06|
|Office workers||−1.24 (1.29)||0.34|
Although the best-fitting model marked the minimum AIC value, a model incorporating local mean with a random effect (F1,1386 = 9.80, p < 0.01) and skewness with a fixed effect (F1,1386 = 3.20, p = 0.07) was equally well-fitted to the data without any significant difference from the best-fitting model (see Table 4 for details). The common feature represented in these models is the intermittency of locomotor activity, characterized by a combination of reduced activity with occasional bursts of locomotor activity.
|Coefficient(SE)||F value||p value||AIC||p value*|
|Coefficient of local statistics of locomotor activity|
|Mean||−0.04 (0.01)||F1,1386 = 9.35||<0.01|
|Detrended skewness||1.40 (0.69)||F1,1386 = 4.18||0.04|
|Coefficient of local statistics of locomotor activity|
|Mean||−0.04 (0.01)||F1,1386 = 9.80||<0.01|
|Skewness||1.21 (0.68)||F1,1386 = 3.20||0.07|
Dependency on size and location.
We demonstrate the robustness of our model with regard to the choice of the size and location of the time frame by showing the dependency of the model coefficients on this choice (Figure 2). The coefficient of the “mean” was consistently significant with negative values (between −0.06 and −0.04) over a broader range (from location −30 to 20 min with size 60 points) [Figure 2 (a)]. Furthermore, this association was also robust against variation in sizes (120–10 points) [Figure 2 (b)]. In the case of “detrended skewness,” the significant positive coefficients were localized in a limited area around the EMA recordings (from 0 to 30 min with 60 points) [Figure 2 (c)], and this association was confirmed only at a larger size of the frame (data length > 50 points) [Figure 2 (d)].
The estimate of the model coefficient of the predictor: (a) Mean (γ10) and (c) Detrended skewness (γ20) of the best-fitting model for depressive mood scores (Depressive mood scoresij = γ00 + γ10 (Meanij) + γ20 (Detrended skewnessij) + ζ0i + ζ1i (Meanij) + εij) as a function of the location. In panels (a) and (c), the size of the time frame was fixed at 60 min. The same is shown in panels (b) and (d), except for the dependency on the size of the time frame. For evaluation, the location of the time frame was fixed at 0 min; thus, the frame was centered around the EMA recordings. Error bars indicate standard error of the coefficients. The asterisk and dagger indicate significant cases at the 0.05 and 0.10, respectively.
Cross validation for association of depressive mood with local statistics of locomotor activity
To examine the cross validation for the associations of depressive mood scores with local statistics of locomotor activity across the three groups, we considered one categorical variable representing “group.” There was no significant difference in the coefficient for either predictor (Table 3) among the groups, which supports the cross validity of our results. However, careful investigation of this model demonstrated that the inclusion of the categorical variable and the separate evaluation of the best-fitting model for each group led to insignificant model coefficients, except for the adolescents. This is caused by the difference in the sample size.
In this study, we investigated the nature of within-individual temporal variations in self-reported symptoms assessed using the EMA technique. EMA is known to be a powerful tool to overcome retrospective biases existing in recall-based self-reports and thus has recently attracted massive attention [1,2]. Many studies have demonstrated the strength of EMA in terms of the minimization of recall biases [36,37,38] and the correlation with other objective measures [10,11,12,13,14,15,16,17]. However, the concurrent associations of self-reported symptoms by EMA with objective real-time measures were not rigorously examined. Therefore, we investigated such associations between self-reported symptoms, especially depressive mood, and locomotor activity data, which can be measured in a truly continuous fashion. We then found the robust (against differences in groups of healthy subjects and the methods of EMA) within-individual temporal associations between fluctuations in depressive mood in daily life and the patterns of locomotor activity. We believe this finding could provide a novel insight into the psychophysiological correlates of “variations” of EMA recordings and a possibility to develop a continuous monitor for changes in depressive mood from locomotor activity, as discussed below.
Validity of EMA
We confirmed the concurrent association of depressive mood with alterations in locomotor activity and the convergent associations among self-reported symptoms recorded simultaneously. In addition, we demonstrated the consistency and robustness of such associations across populations with different age groups, lifestyles, occupations, and so on and proved the existence of cross validity. These verifications provide strong evidence that the within-individual temporal variations in self-reported symptoms measured by EMA are not merely random but may partially originate from and reflect the alterations in psychophysiological states in daily life.
Our systematic approach to modeling the temporal variations in depressive mood scores using the local statistics of locomotor activity for the first time revealed the “momentary” associations of the depressive mood with the levels and patterns of locomotor activity. The intermittent bursts of locomotor activity were important for describing depressive mood even in healthy subjects, which is consistent with previous studies on major depressive disorders [31,32]. Considering the existence of the cross validity in these associations, we suggest that the present relations would generally hold for a wider range of populations. Whether this could be further extrapolated to patients suffering from depressive disorders would be the next important challenge.
The associations between depressive mood and alterations in the levels and patterns of locomotor activity have been shown in epidemiological samples [39,40,41,42,43] and clinical depression [31,32], but the underlying mechanism is unknown. Although the reasons for the covariant associations between depressive mood scores and behavioral alternations are still unclear, the present analyses of data with high temporal resolution clarify the mechanism, at least partially. As shown in Figure 1, there is less or virtually no association in the case of skewness measures reflecting intermittent activity patterns (shown in the upper left corner of the size vs. location plots). This implies that the association of the higher order statistics of locomotor activity data, even when recorded just before the EMA recordings, with the subsequent depressive mood scores is weak. Thus, it is most unlikely that behavioral alteration precedes momentary mood change, which is supported by higher coefficients for the detrended skewness of the best-fitting model at positive or posterior temporal locations [Figure 2 (c)]. The temporal characteristics of the covariant associations indicate that the behavioral alteration begins to be concurrent with mood change. This concurrency suggests that in addition to a possible effect of variations in depressive mood (or more instantaneous feeling) on the subsequent behavior, there is a more direct neurophysiological link between the changes in body state and (depressive) feeling, as proposed by Damasio . On the other hand, the mean activity levels were associated with depressive mood over relatively longer time scales.
Prospective for continuous monitoring of symptoms
Revealing the psychophysiological origin of the temporal variations in self-reported symptoms in terms of behavioral alterations is considered to be highly significant. EMA is known to have the critical drawback of sparse sampling over time; the number of data points recorded in a day is limited. In contrast, locomotor activity can be measured in a truly continuous fashion using a simple device. Therefore, our findings concerning the existence of psychobehavioral correlates between EMA recordings and behavioral alterations might be able to compensate for this drawback and contribute to the continuous estimation of changes in depressive mood in daily life, leading to the early detection or novel treatment of depressive disorders.
Several limitations should be noted. Our populations were limited to healthy subjects with different gender and age distributions. Another limitation is the inconsistency in the EMA protocols among groups leading to differences in the number of recordings and the time of day. Therefore, careful consideration is needed for generalizing our findings to other populations, especially patients with psychiatric disorders such as depression. Also, the model coefficients of the local statistics for the depressive mood scores were not consistently significant in all groups, which is considered to be caused by the small sample size. Therefore, further studies with a large number of subjects and data points would be necessary to rigorously examine cross validity. Finally, other local statistics of locomotor activity that can capture the intermittency in locomotor activity more robustly, such as entropy-type nonlinear statistics, should be considered.
This is the first study providing an insight into the psychophysiological origin of within-individual temporal variations in momentary symptoms assessed by EMA by evaluating co-variant properties with continuous locomotor activity as the external criteria. We confirmed the robust concurrent associations between depressive mood and behavioral alterations and the convergent associations among self-reported symptoms. These findings indicate that temporal variations in momentary symptoms are not random but reflect changes in the underlying psychophysiological variables, represented as changes in the statistical properties of locomotor activity. We believe that our finding on the concurrent changes in depressive mood and locomotor activity may contribute to the continuous estimation of changes in depressive mood and early detection of depressive disorders.
All subjects were given a thorough explanation of the purposes and potential risks of the study by well-trained researchers; then, the subjects signed an institutionally approved informed consent form. A written consent was also obtained from the adolescent subjects’ parents. The study was approved by the research ethics committee of the Graduate School of Education, the University of Tokyo.
A total of 113 healthy subjects were enrolled in this study; 35 were adolescents, 44 were undergraduates, and 34 were office workers. The adolescents were junior high school students in the secondary school attached to the Faculty of Education of the University of Tokyo. The undergraduates were recruited from the Department of Physical and Health Education at the University of Tokyo. Full-time employees at the University of Tokyo were recruited as the office worker subjects.
Of the 113 subjects, 28 were excluded from our analysis because their data were not measured properly owing to faults and/or subject error encountered while using the devices. We finally obtained data from 85 subjects consisting of 30 adolescents [4 males (M)/26 females (F), 13.6 ± 0.5 years of age (yrs)], 31 undergraduates (25M/6F, 21.6 ± 2.3 yrs), and 24 office workers (24M/0F, 41.0 ± 9.2 yrs).
Materials, procedures, and measurements
Ecological momentary assessment.
In this study, we used the EMA technique to examine momentary symptoms. The approach allows researchers to address subjects’ behavior, psychological states, and physiological reactions at multiple time points as they are experienced in daily life. Collecting data in natural settings can enhance the validity of measurements, thus avoiding the pitfalls of retrospective recall, which highly distort self-report data collection . In the present study, we measured momentary symptoms during the study period using the EMA technique with the following procedures.
A watch-type computer (Ruputer, ECOLOG, 42 g; Seiko Instruments Inc., Tokyo, Japan)  was used as the ED to record momentary symptoms. This device has a 20 mm × 30 mm screen and a joystick and buttons as input devices (Figure 3). The subjects were provided all appropriate instructions and also given manuals on how to use the device before the beginning of the study period; in addition, the subjects manipulated the device as a practice exercise until they became accustomed to it.
Filled circles indicate depressive mood scores (y-axis on the left side) recorded by ecological momentary assessment (EMA) over seven consecutive days. The inset in the upper left represents the watch-type activity monitor equipped with EMA software. The locomotor activity measured simultaneously (zero-crossing counts within every 1 min time interval) is also shown (y-axis on the right side). The periods shaded in gray are times during which the subject was sleeping or had taken off the device.
Adolescents: This group was instructed to complete the EMA questionnaires at randomly selected times within plus or minus 10 min of the predefined times (12:50, 16:30, and 20: 00) over seven consecutive days. The subjects were prompted to complete the questionnaires by a beep signal. If they could not enter the recordings when the device beeped, they were allowed to postpone the recordings twice for 15 min each time. If the recordings were not performed within 30 min, the corresponding EMA questionnaires were canceled.
Undergraduates: The EMA questionnaires were recorded over two consecutive days. The subjects were prompted by a beep signal to complete the EMA questionnaires on average every two hours during their waking period. The signals were programmed to occur randomly within intervals ±12 min of the predefined times (8: 00, 10: 00, 12: 00,…). The subjects were allowed to postpone the recordings twice for 10 min each time. If the recordings were not performed within 20 min, the corresponding questionnaires were canceled. The first experimental day of the week was randomly chosen, regardless of weekdays and weekends, so that the study periods can cover all possible day-conditions.
Office workers: EMA questionnaires were recorded on average every four hours during their waking period, over seven consecutive days. The signals were programmed to occur randomly within intervals ± 24 min of the predefined times (8: 00, 12: 00, 16: 00,…). The subjects were allowed to postpone the recordings twice for 10 min each time. If the recordings were not made within 20 min, they were canceled.
In addition, all subjects were asked to record the EMA questionnaires when they woke up and before they went to bed by choosing “waking up” or “going to bed,” from the menu. The selection of the latter item suspended the prompt for recordings until the former item was selected so that to the subjects’ sleep is not disturbed. Furthermore, they were instructed to remove the device before bathing, showering, or any other activity likely to damage the device. When they removed or wore the device, they were requested to select “taking off” or “putting on,” respectively, from the menu as well as the reason for removing the device. The adolescents were asked to conduct EMA recordings (i.e., event-based EMA ) in addition to the scheduled EMA (i.e., time-based EMA ) when they felt bad or had physical symptoms, such as severe fatigue, headache, or stomachache. Note that these data were analyzed without distinguishing between the two types of EMA because we obtained the consistent results whether such data points were included or not.
The EMA questionnaires assessed fatigue and mood states. The recording times were automatically registered in the device. Fatigue intensity and mood states were rated using a visual analog scale from 0 to 100, which was displayed on the device screen. The EMA questionnaires of momentary symptoms were randomly ordered, and each of them was displayed one at a time using a visual analog scale; the anchor words “none” and “most intense” were displayed at the respective ends of the scale. The subjects adjusted the length of the scale by manipulating the joystick of the device so that it corresponded to their subjective symptoms. Owing to the limited display resolution, the 0–100 scale for the intensity of subjective symptoms increased at 5-point intervals. Fatigue intensity was assessed by instructing the subjects to score their current fatigue level by presenting a single question with the adjective “fatigued.” Mood states were rated using the Depression and Anxiety Mood Scale (DAMS) , which was developed to measure anxious and depressive moods as separately as possible. The DAMS comprises nine adjectives representing different mood states: “vigorous,” “gloomy,” “concerned,” “happy,” “unpleasant,” “anxious,” “cheerful,” “depressed,” and “worried.” From these items, anxious mood (sum of “concerned,” “anxious,” and “worried” scores), positive mood (sum of “vigorous,” “happy,” and “cheerful” scores), and negative mood (sum of “gloomy,” “unpleasant,” and “depressed” scores) were assessed. The depressive mood scores were obtained by combining the last two mood scores [(300 − positive mood) + negative mood]. These mood scores were rescaled in the range 0–100.
Assessment of locomotor activity.
This watch-type device is also equipped with an activity monitor, which is analogous in performance to the commercial actigraph (Ambulatory Monitors Inc., Ardsley, NY, USA), widely used in clinical fields [24,31,32,47]. The sensor for assessing locomotor activity is a uni-axial piezo-electric accelerometer capable of detecting small changes in wrist acceleration (≥0.01 G/rad/s), which makes it possible to register even slight movements. All subjects were instructed to wear this device on the wrist of their respective non-dominant hand throughout the study period, except while bathing or performing rigorous exercises. Zero-crossing counts were accumulated for every 1-min epoch for the undergraduates and office workers. For adolescents, acceleration counts were accumulated for every 10 seconds; these activity data were aggregated to make 1-min values to ensure compatibility with the other groups. Locomotor activity data for periods in which the subjects were not wearing the device or sleeping were excluded from the analysis. An example of locomotor activity data is shown in Figure 3.
Local statistics of locomotor activity.
We calculated the local statistics of locomotor activity data up to the third order moment, but used only the mean and skewness because the first and third order moments are considered to be sufficient to characterize the observed data. While the standard deviation (i.e., the second moment) is a standard measure characterizing variability of given data, it is not appropriate for the case where the data do not obey a normal distribution; the distribution of locomotor activity has nonnegative values, leading to a positively-skewed distribution. Indeed, the standard deviation did not play major roles in predicting depressive mood scores when it was included in the model (results not shown). In contrast, the skewness, as a measure of asymmetry of a distribution, is thought to be more appropriate to characterize the observed asymmetry. Lower or higher mean activity levels could quantify the states related to psychomotor retardation or agitation, respectively [24,26]. Higher skewness could characterize the intermittent bursts of locomotor activity [31,32,48].
In addition, we calculated the skewness from the detrended locomotor activity data, where a trend of activity data is subtracted by fitting the first order polynomial (i.e., linear) function before calculation, This detrending was aimed at eliminating effects of non-stationarity due to, e.g., daily activities; we examined the effects up the third order polynomials without finding systematically different results (results not shown). Thus, we obtained three local statistics of locomotor activity, the mean, skewness, and (linearly) detrended skewness for evaluating the associations with the scores of EMA recordings.
Choice of size and location of time frame.
The local statistics were calculated from the neighboring data of each EMA within a certain time frame with size N and location K. The value of K (min) represents the central position (location) of the time frame against the timing of EMA recordings, and the size N (min) indicates the data points used to calculate the statistics. Thus, the combination of N and K indicates the time frame starting from N/2 − K points (min) prior to each EMA recording and ending at N/2 + K points (min) after the recording. Therefore, the case where N = 60 and K = −30 indicates the time frame staring from 60 min prior to an EMA recording to the timing of the recording.
The choice of the size of the time frame may be important because they could have a significant impact on the robustness of the statistics and their temporal coincidence with the symptoms. Theoretically, the larger the size of the time frame, the greater is the stability and reliability of the estimates. However, the choice of larger time frames with less localized temporal positions may obscure the information on the momentary associations between locomotor activity and self-reported symptoms. In addition, the choice of the location plays an important role to investigate the casual association, whether locomotor activity precedes or follows changes in momentary symptoms. Thus, careful consideration of such a trade-off is required, while the optimal choice of combination of the size and location is in advance unknown. Therefore, we varied the size N and location K of the time frame to examine their effects.
The data set in the present study has a hierarchical structure, in which EMA recordings were conducted at multiple times during the study period; thus, the recordings and the corresponding local statistics of locomotor activity were nested within the subjects. Therefore, we used multilevel modeling, which is an extension of traditional regression models and has been recommended for the analysis of data with a hierarchical structure including the within- and between-person levels (e.g., individual differences such as age, gender and occupation) , using SAS Proc Mixed (SAS 9.2, SAS Institute Inc., Cary, NC). Multilevel modeling can also handle unbalanced data in which the number of the EMA recordings is different across subjects and can include both random and fixed effects together in the same model. A p value less than 0.05 was considered significant. For the univariate multilevel modeling, a false discovery rate (FDR) procedure  was performed at a q value of 0.05 to correct for the multiple comparisons. All models were estimated from the data set sampled from the three groups.
Convergent associations among symptoms.
To evaluate the convergent associations among self-reported moods and fatigue, simultaneously measured by EMA, we calculated correlations among their scores using 2-level mixed MANOVA models with EMA scores as the dependent variables and the type of symptoms (i.e., depressive mood, anxious mood, and fatigue) as the predictor. Various studies demonstrate the co-variation between depressive and anxious moods, suggesting the possibility to categorize them into the same class of a broader negative affect dimension [51,52,53]. Indeed, the validation study of DAMS confirms the significant positive correlation between these two mood scores . Fatigue is known to have an extremely high comorbidity rate with depressive as well as anxious moods. For example, fatigue is a common complaint in depressive patients [54,55,56,57]. In chronic fatigue syndrome, the high rate of its comorbidity with both moods has been reported . Co-variations of fatigue with these moods have also been reported in patients with cancer . Therefore, these three symptoms would be expected to co-vary with each other if the subjects properly performed the EMA recordings; otherwise, such correlations should be absent.
Concurrent associations of symptoms with local statistics of locomotor activity.
To investigate the concurrent associations between self-reported symptoms and simultaneous locomotor activity data, the following multilevel models were estimated. In all models, the scores of EMA recordings were treated as the dependent variables, and the corresponding local statistics of locomotor activity were modeled as the predictors. Firstly, we estimated univariate multilevel models to test the simple correlations between the symptoms and local statistics. In addition, we also examined the effects of the choice of time frame to find an optimal combination of size and location for evaluating the local statistics. With regard to the size of the time frame, we considered 12 different conditions: N = 120, 110, 100, …, 20, 10 min. We also considered 25 different locations: K = −60, −55, −50, …, 55, 60 min. We considered a total of 300 combinations (12 sizes × 25 locations) for each symptom and statistic. The FDR with the q value of 0.05 was used as the multiple comparison adjustment.
Secondly, all possible multivariate multilevel models were evaluated by considering the three predictors and their interaction terms. However, the terms consisting of both skewness and detrended skewness were not considered; the interaction terms we considered were: mean × skewness and mean × detrended skewness. Furthermore, we considered both fixed (i.e., the slope of the predictor does not vary across individuals) and random (i.e., the slope varies across individual) effects for all predictors. In this report, we focus on the results for depressive mood because we could not find a significant association of locomotor activity with fatigue or anxious mood.
We adopted the following procedures to evaluate the goodness-of-fit of the models. If the inclusion of a random effect into a predictor significantly improved the residual variance of the data explained by the model, we considered the random effect for that term . After determining the type of effect of all predictors for all models, we compared the goodness-of-fit by using two types of tests depending on the structure of the models. While comparing the models with nested or inclusive relations, where all predictors used in one model were a subset of the predictors used in another model, we tested the deviance statistic . This approach tests the difference in the −2 log-likelihood on the basis of the chi-square test with degrees of freedom equal to the difference in the number of model parameters to be estimated. If the difference in the −2 log-likelihood was not significant, the model with fewer parameters was selected as the preferable one on the basis of the principle of parsimony. For the non-nested models, we used a statistical test based on the Akaike information criteria (AIC), in which the difference of AIC between models is evaluated using the z-test .
To determine the best-fitting model for describing the variations of depressive mood scores, we used a time frame of 60 min centered around the EMA recording (i.e., N = 60, K = 0) on the basis of the investigation of univariate multilevel models (see Results). After determining the best-fitting model, we also considered the effects of size and location by varying them. We considered 12 different sizes (N = 120, 110, 100, …, 20, 10) keeping the location of the frame (K = 0). We also considered 13 different locations (K = −30, −25, −20, …, 25, 30) with the size set at 60 min.
Cross validation for association of depressive mood with local statistics of locomotor activity.
To cross validate the associations of depressive mood scores with local statistics of locomotor activity among the three groups, an ad hoc analysis was performed for the best-fitting model. We added a categorical variable representing the groups (adolescents, undergraduates, and office workers) to the best-fitting model in order to test whether the concurrent associations differed among categories.
We would like to thank Dr. Naoko Aoyagi for data collection from adolescent subjects.
Conceived and designed the experiments: YY JK HK TS. Performed the experiments: JK HK. Analyzed the data: JK TN HK. Contributed reagents/materials/analysis tools: YY JK HK TN. Wrote the manuscript: JK TN YY HK TS.
- 1. Stone AA, Shiffman S (1994) Ecological momentary assessment (EMA) in behavioral medicine. Ann Behav Med 16: 199-202.
- 2. Shiffman S, Stone AA, Hufford MR (2008) Ecological momentary assessment. Annu Rev Clin Psychol 4: 1-32. doi:10.1146/annurev.clinpsy.3.022806.091415. PubMed: 18509902.
- 3. Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR (2002) Patient non-compliance with paper diaries. BMJ 324: 1193-1194. doi:10.1136/bmj.324.7347.1193. PubMed: 12016186.
- 4. Conner TS, Barrett LF (2012) Trends in ambulatory self-report: the role of momentary experience in psychosomatic medicine. Psychosom Med 74: 327-337. doi:10.1097/PSY.0b013e3182546f18. PubMed: 22582330.
- 5. Kumar S, Nilsen W, Pavel M, Srivastava M (2013) Mobile health: revolutionizing healthcare through transdisciplinary research. IEEE Comput 46: 28-35. doi:10.1109/MC.2012.392.
- 6. American Psychiatric Association (2000) Diagnostic and statistical manual of mental disorders. Washington, D.C.: American Psychiatric Press.
- 7. Thewissen V, Bentall RP, Lecomte T, van Os J, Myin-Germeys I (2008) Fluctuations in self-esteem and paranoia in the context of daily life. J Abnorm Psychol 117: 143-153. doi:10.1037/0021-843X.117.1.143. PubMed: 18266492.
- 8. Steptoe A, Leigh ES, Kumari M (2011) Positive affect and distressed affect over the day in older people. Psychol Aging 26: 956-965. doi:10.1037/a0023303. PubMed: 21517182.
- 9. Kimhy D, Delespaul P, Corcoran C, Ahn H, Yale S et al. (2006) Computerized experience sampling method (ESMc): assessing feasibility and validity among individuals with schizophrenia. J Psychiatr Res 40: 221-230. doi:10.1016/j.jpsychires.2005.09.007. PubMed: 16300791.
- 10. Kamarck TW, Schwartz JE, Shiffman S, Muldoon MF, Sutton-Tyrrell K et al. (2005) Psychosocial stress and cardiovascular risk: what is the role of daily experience? J Pers 73: 1749-1774. doi:10.1111/j.0022-3506.2005.00365.x. PubMed: 16274452.
- 11. Kamarck TW, Shiffman SM, Smithline L, Goodie JL, Paty JA et al. (1998) Effects of task strain, social conflict, and emotional activation on ambulatory cardiovascular activity: daily life consequences of recurring stress in a multiethnic adult sample. Health Psychol 17: 17-29. doi:10.1037/0278-622.214.171.124. PubMed: 9459066.
- 12. Smith TW, Birmingham W, Uchino BN (2012) Evaluative threat and ambulatory blood pressure: cardiovascular effects of social stress in daily experience. Health Psychol 31: 763-766. doi:10.1037/a0026947. PubMed: 22251220.
- 13. Robles TF, Shetty V, Zigler CM, Glover DA, Elashoff D et al. (2011) The feasibility of ambulatory biosensor measurement of salivary alpha amylase: relationships with self-reported and naturalistic psychological stress. Biol Psychol 86: 50-56. doi:10.1016/j.biopsycho.2010.10.006. PubMed: 20955759.
- 14. van Eck M, Berkhof H, Nicolson N, Sulon J (1996) The effects of perceived stress, traits, mood states, and stressful daily events on salivary cortisol. Psychosom Med 58: 447-458. PubMed: 8902896.
- 15. Smyth J, Ockenfels MC, Porter L, Kirschbaum C, Hellhammer DH et al. (1998) Stressors and mood measured on a momentary basis are associated with salivary cortisol secretion. Psychoneuroendocrinology 23: 353-370. doi:10.1016/S0306-4530(98)00008-0. PubMed: 9695136.
- 16. Steptoe A, Gibson EL, Hamer M, Wardle J (2007) Neuroendocrine and cardiovascular correlates of positive affect measured by ecological momentary assessment and by questionnaire. Psychoneuroendocrinology 32: 56-64. doi:10.1016/j.psyneuen.2006.10.001. PubMed: 17157442.
- 17. Ritz T, Rosenfield D, Dewilde S, Steptoe A (2010) Daily mood, shortness of breath, and lung function in asthma: concurrent and prospective associations. J Psychosom Res 69: 341-351. doi:10.1016/j.jpsychores.2010.05.004. PubMed: 20846535.
- 18. Chandra S, Scharf D, Shiffman S (2011) Within-day temporal patterns of smoking, withdrawal symptoms, and craving. Drug Alcohol Depend 117: 118-125. doi:10.1016/j.drugalcdep.2010.12.027. PubMed: 21324611.
- 19. Shiffman S, Balabanis MH, Gwaltney CJ, Paty JA, Gnys M et al. (2007) Prediction of lapse from associations between smoking and situational antecedents assessed by ecological momentary assessment. Drug Alcohol Depend 91: 159-168. doi:10.1016/j.drugalcdep.2007.05.017. PubMed: 17628353.
- 20. Muraven M, Collins RL, Shiffman S, Paty JA (2005) Daily fluctuations in self-control demands and alcohol intake. Psychol Addict Behav 19: 140-147. doi:10.1037/0893-164X.19.2.140. PubMed: 16011384.
- 21. Jahng S, Solhan MB, Tomko RL, Wood PK, Piasecki TM et al. (2011) Affect and alcohol use: an ecological momentary assessment study of outpatients with borderline personality disorder. J Abnorm Psychol 120: 572-584. doi:10.1037/a0024686. PubMed: 21823761.
- 22. Dunton GF, Atienza AA, Castro CM, King AC (2009) Using ecological momentary assessment to examine antecedents and correlates of physical activity bouts in adults age 50+ years: a pilot study. Ann Behav Med 38: 249-255. doi:10.1007/s12160-009-9141-4. PubMed: 20052568.
- 23. Wichers M, Peeters F, Rutten BP, Jacobs N, Derom C et al. (2012) A time-lagged momentary assessment study on daily life physical activity and affect. Health Psychol 31: 135-144. doi:10.1037/a0025688. PubMed: 21988094.
- 24. Teicher MH (1995) Actigraphy and motion analysis: new tools for psychiatry. Harv Rev Psychiatry 3: 18-35. doi:10.3109/10673229509017161. PubMed: 9384925.
- 25. Teicher MH, Glod CA, Magnus E, Harper D, Benson G et al. (1997) Circadian rest-activity disturbances in seasonal affective disorder. Arch Gen Psychiatry 54: 124-130. doi:10.1001/archpsyc.1997.01830140034007. PubMed: 9040280.
- 26. Burton C, McKinstry B, Szentagotai Tătar A, Serrano-Blanco A, Pagliari C et al. (2013) Activity monitoring in patients with depression: a systematic review. J Affect Disord 145: 21-28. doi:10.1016/j.jad.2012.07.001. PubMed: 22868056.
- 27. Indic P, Murray G, Maggini C, Amore M, Meschi T, et al. (2012) Multi-scale motility amplitude associated with suicidal thoughts in major depression. PLOS ONE 7: e38761. doi:10.1371/journal.pone.0038761. PubMed: 22701706. PubMed: 22701706.
- 28. Walther S, Hügli S, Höfle O, Federspiel A, Horn H et al. (2012) Frontal white matter integrity is related to psychomotor retardation in major depression. Neurobiol Dis 47: 13-19. doi:10.1016/j.nbd.2012.03.019. PubMed: 22426387.
- 29. Volkers AC, Tulen JH, van den Broek WW, Bruijn JA, Passchier J et al. (2003) Motor activity and autonomic cardiac functioning in major depressive disorder. J Affect Disord 76: 23-30. doi:10.1016/S0165-0327(02)00066-6. PubMed: 12943930.
- 30. Berle JO, Hauge ER, Oedegaard KJ, Holsten F, Fasmer OB (2010) Actigraphic registration of motor activity reveals a more structured behavioural pattern in schizophrenia than in major depression. BMC Res Notes 3: 149. doi:10.1186/1756-0500-3-149. PubMed: 20507606.
- 31. Nakamura T, Kiyono K, Yoshiuchi K, Nakahara R, Struzik ZR et al. (2007) Universal scaling law in human behavioral organization. Phys Rev Lett 99: 138103. doi:10.1103/PhysRevLett.99.138103. PubMed: 17930642.
- 32. Nakamura T, Takumi T, Takano A, Aoyagi N, Yoshiuchi K et al. (2008) Of mice and men--universality and breakdown of behavioral organization. PLOS ONE 3: e2050. doi:10.1371/journal.pone.0002050. PubMed: 18446212.
- 33. Kahneman D, Krueger AB, Schkade DA, Schwarz N, Stone AA (2004) A survey method for characterizing daily life experience: the day reconstruction method. Science 306: 1776-1780. doi:10.1126/science.1103572. PubMed: 15576620.
- 34. McColl E (2004) Best practice in symptom assessment: a review. Gut 53: 49-54. PubMed: 15082615.
- 35. Stone AA, Schwartz JE, Schkade D, Schwarz N, Krueger A et al. (2006) A population approach to the study of emotion: diurnal rhythms of a working day examined with the Day Reconstruction Method. Emotion 6: 139-149. doi:10.1037/1528-35126.96.36.199. PubMed: 16637757.
- 36. Stone AA, Broderick JE, Porter LS, Kaell AT (1997) The experience of rheumatoid arthritis pain and fatigue: examining momentary reports and correlates over one week. Arthritis Care Res 10: 185-193. doi:10.1002/art.1790100306. PubMed: 9335630.
- 37. Stone AA, Broderick JE, Shiffman SS, Schwartz JE (2004) Understanding recall of weekly pain from a momentary assessment perspective: absolute agreement, between- and within-person consistency, and judged change in weekly pain. Pain 107: 61-69. doi:10.1016/j.pain.2003.09.020. PubMed: 14715390.
- 38. Friedberg F, Sohl SJ (2008) Memory for fatigue in chronic fatigue syndrome: the relation between weekly recall and momentary ratings. Int J Behav Med 15: 29-33. doi:10.1007/BF03003071. PubMed: 18444018.
- 39. Stavrakakis N, de Jonge P, Ormel J, Oldehinkel AJ (2012) Bidirectional prospective associations between physical activity and depressive symptoms. The TRAILS Study. J Adolesc Health Care 50: 503-508. doi:10.1016/j.jadohealth.2011.09.004.
- 40. Lindwall M, Larsman P, Hagger MS (2011) The reciprocal relationship between physical activity and depression in older European adults: a prospective cross-lagged panel design using SHARE data. Health Psychol 30: 453-462. doi:10.1037/a0023268. PubMed: 21480713.
- 41. Harris AH, Cronkite R, Moos R (2006) Physical activity, exercise coping, and depression in a 10-year cohort study of depressed patients. J Affect Disord 93: 79-85. doi:10.1016/j.jad.2006.02.013. PubMed: 16545873.
- 42. Strawbridge WJ (2002) Physical activity reduces the risk of subsequent depression for older adults. Am J Epidemiol 156: 328-334. doi:10.1093/aje/kwf047. PubMed: 12181102.
- 43. Camacho TC, Roberts RE, Lazarus NB, Kaplan GA, Cohen RD (1991) Physical activity and depression: evidence from the Alameda County Study. Am J Epidemiol 134: 220-231. PubMed: 1862805.
- 44. Damasio AR (2004) Looking for Spinoza: joy, sorrow, and the feeling brain. London: Vintage Publishing House.
- 45. Kim J, Kikuchi H, Yamamoto Y (2013) Systematic comparison between ecological momentary assessment and day reconstruction method for fatigue and mood states in healthy adults. Br J Health Psychol 18: 155-167. doi:10.1111/bjhp.12000. PubMed: 23017062.
- 46. Fukui I (1997) The depression and anxiety mood scale (DAMS): scale development and validation. JPN J Behav Ther 23: 83-93.
- 47. Kikuchi H, Yoshiuchi K, Ohashi K, Yamamoto Y, Akabayashi A (2007) Tension-type headache and physical activity: an actigraphic study. Cephalalgia 27: 1236-1243. doi:10.1111/j.1468-2982.2007.01436.x. PubMed: 17850350.
- 48. Sano W, Nakamura T, Yoshiuchi K, Kitajima T, Tsuchiya A et al. (2012) Enhanced persistency of resting and active periods of locomotor activity in schizophrenia. PLOS ONE 7: e43539. doi:10.1371/journal.pone.0043539. PubMed: 22952701.
- 49. Schwartz JE, Stone AA (1998) Strategies for analyzing ecological momentary assessment data. Health Psychol 17: 6-16. doi:10.1037/0278-6188.8.131.52. PubMed: 9459065.
- 50. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29: 1165-1188. doi:10.1214/aos/1013699998.
- 51. Suls J, Bunde J (2005) Anger, anxiety, and depression as risk factors for cardiovascular disease: the problems and implications of overlapping affective dispositions. Psychol Bull 131: 260-300. doi:10.1037/0033-2909.131.2.260. PubMed: 15740422.
- 52. Bleil ME, Gianaros PJ, Jennings JR, Flory JD, Manuck SB (2008) Trait negative affect: toward an integrated model of understanding psychological risk for impairment in cardiac autonomic function. Psychosom Med 70: 328-337. doi:10.1097/PSY.0b013e31816baefa. PubMed: 18378862.
- 53. Thieme K, Turk DC, Flor H (2004) Comorbid depression and anxiety in Fibromyalgia syndrome: Relationship to somatic and psychosocial variables. Psychosom Med 66: 837-844. doi:10.1097/01.psy.0000146329.63158.40. PubMed: 15564347.
- 54. Demyttenaere K, De Fruyt J, Stahl SM (2005) The many faces of fatigue in major depressive disorder. Int J Neuropsychopharmacol 8: 93-105. doi:10.1017/S1461145704004729. PubMed: 15482632.
- 55. Kennedy SH (2008) Core symptoms of major depressive disorder: relevance to diagnosis and treatment. Dialogues Clin Neurosci 10: 271-277. PubMed: 18979940.
- 56. Tylee A, Gastpar M, Lepine JP, Mendlewicz J (1999) DEpress II (Depression Research in European Society II): a patient survey of the symptoms, disability and current management of depression in the community DEPRES Steering Committee. Int Clin Psychopharmacol 14: 139-151.
- 57. Baldwin DS, Papakostas GI (2006) Symptoms of fatigue and sleepiness in major depressive disorder. J Clin Psychiatry 67 Suppl 6: 9-15. doi:10.4088/JCP.0906e09. PubMed: 16848671.
- 58. Skapinakis P, Lewis G, Mavreas V (2003) Unexplained fatigue syndromes in a multinational primary care sample: specificity of definition and prevalence and distinctiveness from depression and generalized anxiety. Am J Psychiatry 160: 785-787. doi:10.1176/appi.ajp.160.4.785. PubMed: 12668371.
- 59. Tchekmedyian NS, Kallich J, McDermott A, Fayers P, Erder MH (2003) The relationship between psychologic distress and cancer-related fatigue. Cancer 98: 198-203. doi:10.1002/cncr.11463. PubMed: 12833472.
- 60. Stone AA, Shiffman S, Atienza AA, Nebeling L (2007) The science of real-time data capture: self-reports in health research. Oxford; New York: Oxford University Press.
- 61. Singer JD, Willett JB (2003) Applied longitudinal data analysis: modeling change and event occurrence. Oxford; New York: Oxford University Press.
- 62. Linhart H (1988) A test whether two AIC’s differ significantly. S Afr Stat J 22: 153-161.