Are we validly assessing major depression disorder risk and associated factors among mothers of young children? A cross-sectional study involving home visitation programs

Failure to account for misclassification error accruing from imperfect case-finding instruments can produce biased estimates of suspected major depression disorder (MDD) risk factor associations. The objective of this study was to estimate the impact of misclassification error on the magnitude of measures of association between suspected risk factors and MDD assessed using the Center of Epidemiological Studies on Depression—Short Form during the prenatal and postnatal periods. Baseline data were collected from 520 mothers participating in two home visitation studies in Oklahoma City between 2010 and 2014. A Bayesian binomial latent class model was used to compare the prevalence proportion ratio (PPR) between suspected risk factors and MDD with and without adjustment for misclassification error and confounding by period of MDD symptom on-set. Adjustment for misclassification error and confounding by period of MDD on-set (prenatal vs postnatal) showed that the association between suspected risk factors and MDD is underestimated (-) and overestimated (+) differentially in different source populations of low-income mothers. The median bias in the magnitude of PPR estimates ranged between -.47 (95% Bayesian Credible Intervals [BCI]: -10.67, 1.90) for intimate partner violence to +.06 (95%BCI: -0.37, 0.47) for race/ethnicity among native-born US residents. Among recent Hispanic immigrants, bias ranged from -.77 (95%BCI: -15.31, 0.96) for history of childhood maltreatment to +.10 (95%BCI: -0.17, 0.39) for adequacy of family resources. Overall, the extent of bias on measures of association between maternal MDD and suspected risk factors is considerable without adjustment for misclassification error and is even higher for confounding by period of MDD assessment. Consideration of these biases in MDD prevention research is warranted.


Introduction
Major depression disorder (MDD) is prevalent among women and is correlated with poor health and psychosocial outcomes for both mothers and the children under their care [1,2]. PLOS  proportion was under-or over-estimated) [26]. To our knowledge, no study has examined the individual or joint impact of misclassification error and confounding by period of MDD assessment on measures of association between suspected risk factors and MDD in home visitation settings or among women in general. Therefore, the objective of our study is to extend our previous work [26] by examining (1) the impact of CESD-SF case-finding instrument misclassification error, and (2) the joint impact of misclassification error and confounding by period of MDD assessment on measures of association between MDD and suspected risk factors among mothers of young children enrolled at two home visitation program study sites. We also compare and contrast the impact of these issues between the two study sites.

Methods and procedures
Oklahoma University Health Sciences Center IRB approved study related procedures and activities.

Data source
This study is a secondary analysis of the baseline component of two ongoing home visitation studies targeting mothers of young children in Oklahoma, which targeted two distinct populations. One study site, SITE I, targeted a diverse racial/ethnic mix of low-income mothers while the other, SITE II, targeted a population of low-income, first and second-generation Hispanic immigrant mothers. The objective of these studies was to examine the effectiveness of evidence-based home visitation programs on improving parent-infant/child outcomes such as prevention of child maltreatment, reducing parent substance and alcohol abuse, intimate partner violence and parenting stress. The two study populations had very different distributions of risk factors and frequency of MDD [26], which resulted in important differences in the measures of association between the sites. This led to the decision to report the results separately for the two study sites.

Study design
The baseline components of two conveniently chosen ongoing home visitation study sites were analyzed as cross-sectional designs. Baseline data were collected between December 2010 and December 2014 from consecutively enrolled mothers of children aged five years or younger.

Study setting
Participants in the ongoing home visitation studies were recruited from physician offices, medical centers, faith-based organizations, schools, mental health agencies, other HV programs, and welfare programs (e.g. Women, Infants and Children program) in Oklahoma City. At baseline, prior to enrollment in home visitation services, mothers completed a battery of questionnaires including questions on basic socio-demographic information, family and social supports, history of intimate partner violence, childhood maltreatment and MDD symptoms (using the CESD-SF). These questionnaires were completed using an audio computer assisted self-interview (ACASI) application with assistance from a trained data collector if needed.

Study participants
The three inclusion criteria for the current analysis were: 1) being a mother, 2) having at least one child who was five years old or younger at baseline (pregnant women with no other children were ineligible), and 3) having a completed CESD-SF at baseline. For our study analyses, because we focused on baseline data, miscarriages, stillbirth or death of children after baseline assessments did not influence our study findings.
Informed verbal and written consent was obtained from all eligible study participants. For minors, involved in the study, we obtained their verbal and written consent as well as that of their parents or guardians. Differences in the distribution of study characteristics at the two study sites (SITE I and II) were examined and are summarized elsewhere [26].

MDD outcome ascertainment
The operational definition of MDD status was determined using the CESD-SF. Here, the results of the CESD-SF are not used as a diagnosis of MDD but rather are indicative of the presence of MDD-related symptoms. Only a qualified health professional can diagnose MDD through a more thorough diagnostic interview. Mothers at baseline completed the CESD-SF in English or Spanish prior to initiation of home visitation services. The CESD-SF is a 12-item multi-choice instrument designed to assess if an individual has exhibited some level of MDDrelated symptoms during the past week with scores ranging from 0 to 36 [29]. The CESD-SF is derived from the 20-item scale originally developed as a measure of MDD-related symptoms in community-based epidemiological studies [29,30]. This instrument has been used to screen for MDD during the prenatal, postnatal and combined periods [31][32][33]. Having moderate MDD symptoms was defined operationally as a CESD-SF score of 10 or above [29]. This threshold is comparable to the cut-off point threshold of 16 (�16) on the CESD20 [29]. The CESD-SF showed good internal consistency (Cronbach's alpha) for the assessment of depression-related symptoms at both SITE I (0.92) and II (0.93).
We operationally define periods of MDD assessment among study participants as 'prenatal' or 'postnatal' depending on whether the CESD-SF baseline assessment were administered during pregnancy or after childbirth, respectively. Eligible pregnant women or mothers had at least one child who was five years old or younger. In this study, the term the 'misclassification error' refers to the incorrect classification of an individual as either a false positive or false negative by the CESD-SF. 'Confounding by period of MDD assessment' refers to the distortion of the relationship (measured by the prevalence proportion ratio) between an individual's true MDD status and suspected risk factor as a function of the differential proportion distribution of assessments (prenatal versus postnatal) among individuals with and without a risk factor of interest.

Measurement and definition of suspected risk factors
Information on suspected MDD risk factors (e.g. socio-demographic characteristics, adequacy of family resources and social support, history of childhood maltreatment and intimate partner violence) is derived from self-report questionnaires administered at baseline. Factors that have been previously identified as risk factors for MDD among mothers were examined [2]. Self-reported race and ethnicity was categorized as White, Black, and Other (i.e. Hispanic, Native American and Asians). The adequacy of resources in households with children and the degree to which a respondent's social relationships provide various dimensions of social support were measured using the 30-items Family Resources Scale-Revised [34] and the 12-items Social Provisions Scale-Short Form [35], respectively. A history of childhood maltreatment was determined based on a self-report of any child abuse or neglect event that occurred during a mother's childhood. Presence of intimate partner violence (among mothers who reported being in a relationship at baseline) was determined based on a self-report of physical assault by a partner (i.e. physical assault victimization sub-scale) in the past six months on the Conflict Tactics Scale Second version [36].

Statistical analysis
Measures of association (estimated by prevalence proportion ratios) were generated for each study site separately since they targeted very different populations of mothers and the impact of the different risk factors were expected to vary between sites. Prevalence proportion ratios (PPRs) were obtained by comparing prevalence proportions between sub-groups defined by levels or categories of a characteristic/risk factor to a reference sub-group (i.e. PPR = prevalence proportion in sub-group 1/ prevalence proportion in sub-group 2 [reference group]).
The Bayesian latent class models used for the estimation of prevalence proportions are described in our previous study [26]. Briefly, the probability of a test result and probability of MDD are related via conditional probability concepts as follows: P (T + ) = P (T + |D + ) P (D + ) + P (T + |D -) P (D -) where P(T + ) corresponds to the proportion of participants testing positive for MDD (observed prevalence), P (T + |D + ) to the sensitivity of the test, P (T + |D -) corresponds to one minus the specificity of the test, and P (D + ) to the true prevalence of MDD.
Therefore, P (D+) = (P (T + |D + ) + P (T -|D + ))/ (P (T + |D + ) + P (T + |D -) + P (T -|D -) + P (T -| D + )). In the absence of a gold standard, this equation cannot be solved directly since there are three unknown parameters (sensitivity, specificity and true prevalence) but only one degree of freedom from the observed data (number of positive tests out of the total sample size fixes the number testing negative). A Gibbs sampler is used to randomly sample the joint posterior density of the parameters of interest (sensitivity, specificity and true prevalence) which is the product of the likelihood of the data (latent and observed) and prior distributions of all parameters. The samples from the joint posterior distribution are then used to generate the marginal posterior densities of prevalence proportion, sensitivity and specificity parameters.
For PPR estimation, all investigated risk factors were dichotomized. The FRS and SPS scales were dichotomized based on the median value of the participants' scores to represent high versus low family resources and social support respectively. Marital status was dichotomized into married or living with a partner versus single (i.e. never married, divorced, separated or widowed). Being born in the US, having healthcare insurance, history of childhood maltreatment and history of intimate partner violence were dichotomized as yes/no variables. Educational level was dichotomized as attainment of high school or General Education Development (GED) equivalent or higher versus less than a high school diploma/GED. Employment status was dichotomized as full or part-time versus unemployed and maternal age was dichotomized (based on the median age of 25 years old).
For each sub-group defined by the presence or absence of a characteristic, the MDD prevalence proportions were obtained using a Bayesian latent class analysis model in the absence of a 'gold standard' with and without adjustment for CESD-SF misclassification error and confounding by period of MDD assessment. The median estimate of the posterior density distribution of each PPR for each investigated characteristic including its 95% Bayesian Credible Interval (95%BCI) were obtained separately. The 95% BCIs give us the 95% probability coverage range of median PPR values. S1 File shows the WinBUGS program code used to generate these estimates.
Convergence was assessed by examining the plots of chain histories, the results from the Geweke test [37], Heidelberger-Welch stationary test [38], and Gelman-Rubin test [39]. Win-BUGS software (version 1.4.3, MRC Biostatistics Unit, Cambridge, UK) was used to implement the Bayesian latent class models [40].

Handling of missing data
WinBUGS automatically simulates values for missing observations according to the specified likelihood distribution (in our case-binomial distribution), conditional on the current values of all relevant unknown parameters (i.e. prevalence proportions and PPRs) described above.

Prior distributions
Informative prior estimates of CESD-SF sensitivity and specificity among mothers of young children were based on a previous systematic review where the posterior estimates of sensitivity and specificity were obtained for CESD20 with consideration for misclassification error in the reference standards [41]. Only one study, Tandon et al [33], provided prenatal and postnatal period specific diagnostic performance estimates. The period-specific posterior estimates derived from a Bayesian Hierarchical Summary Receiver-Operating Characteristic Curve (HSROC) meta-analysis model [41] are considered as prior information (Table 1). Informative priors are a key part of Bayesian inference that represent information about an uncertain parameter (in our case the sensitivity and specificity estimates of the CESD-SF). These priors are combined with the probability distribution of new data (True Positives, True Negatives, False Positives, and False Negatives) to yield a posterior distribution of case-finding instrument sensitivity and specificity estimates from which summary estimates of parameters (i.e. median prevalence proportion, PPRs and 95% Bayesian Credible Intervals [BCI]) are estimated.
The first misclassification error-adjusted PPR estimate (without adjustment for confounding by period of MDD assessment) was generated by (a) combining data across prenatal and postnatal periods, and (b) using prior values based on combined periods to compare MDD prevalence proportions in risk factor sub-groups (i.e. prevalence proportion 1 / prevalence proportion 2 ) The second misclassification error-adjusted PPR estimate (with adjustment for confounding by period of MDD assessment) was generated by (a) separating data into prenatal versus postnatal period MDD assessments and (b) using period-specific prior values to compare MDD prevalence proportions in risk factor sub-groups (i.e. prevalence proportion 1(prenatal + postnatal) / prevalence proportion 2(prenatal + postnatal) ).

Study population
As of December 31 2014 (date analytical sample was obtained), 429 and 687 referrals had been received by SITE I and II, respectively; 318 (74%) and 467 (68%) were eligible. Among eligible participants, 179 (56%) and 341 (73%) mothers were successfully recruited into the study and completed baseline assessments at SITE I (low income mothers who were US residents) and SITE II (recent Hispanic immigrant mothers), respectively. Most socio-demographic characteristics differed between the two study sites [26]. Specifically, there were older (>25 years old), more married (or living with a partner), less high school graduates, more foreign-born mothers at SITE II than SITE I (p<0.05).

Observed measures of association (PPR) between maternal MDD and suspected risk factors
The measures of association (i.e. PPR) of investigated risk factors were quite different between the study sites (Tables 2 and 3: crude estimates). Being single, having a history of intimate partner violence, low levels of social supports and family resource were associated with a higher prevalence of MDD at SITE I (Table 2). At SITE II, in addition to risk factors identified at SITE I, having a history of childhood maltreatment and unemployment were also associated with MDD (Table 3). Overall, the prevalence of MDD was 50% higher at SITE I than II (PPR: 1.50; 95% BCI: 1.23; 1.83).

Magnitude of bias in measures of association (PPR) ignoring confounding by period of MDD assessment
When all data were combined (i.e. prenatal and postnatal MDD assessments) and analyzed using diagnostic performance priors for the combined periods as a whole, the association between MDD and suspected risk factors seemed to be biased towards the null at both study sites (Tables 2 and 3 For SITE II, the magnitude of bias estimates were generally higher ranging from 0.08 (95% BCI: <0.01, 0.15) for social support to -0.01 (95%BCI: -0.81; 0.68) for period of MDD assessment. Although, adjustment for misclassification error increased uncertainty (wider 95%   High scores of family resources (FRS) and social support (SPS) were associated with a lower prevalence of MDD (Table 3).

Fig 1. Crude and adjusted measures of association between suspected risk factors and MDD assessed by the CESD-SF at SITE I.
Red diamond symbol-crude PPR estimates are based on non-informative priors and ignore both misclassification error (i.e. assume 100% CESD-SF sensitivity and specificity) and confounding by period of MDD assessment. Dark blue square symbol-misclassification error adjusted PPR estimates ignoring confounding by period of MDD assessment. Light green triangle-misclassification error adjusted PPR estimates adjusted for confounding by period of MDD assessment. HS-High school (or GED equivalent). MEmisclassification error. https://doi.org/10.1371/journal.pone.0209735.g001 Major depression disorder risk factors among mothers of young children

Magnitude of bias in measures of association (PPR) adjusted for CESD-SF misclassification error and confounding by period of MDD assessment
Across the risk factors examined at both study sites, adjusting for the differential diagnostic performance of the CESD-SF during the prenatal and postnatal periods and the different proportions of mothers who had assessments in these periods showed that a higher magnitude of bias (Tables 2 and 3) was present than was observed when the analysis was conducted on the combined periods. The prevalence of MDD was higher at SITE I than SITE II after adjusting for both misclassification error and the confounding by period of MDD assessment  Table 2). The high magnitude of bias for intimate partner violence resulted in an increase in the median PPR of nearly 23%, but also greater uncertainty (i.e. wider 95%BCI). Similar to the misclassification error-adjusted PPRs, PPRs adjusted for both confounding and misclassification error suggested a higher MDD prevalence among single mothers, those who reported intimate partner violence and White mothers (compared to Black mothers) with the exception of having healthcare insurance ( Table 2). Before adjustment for confounding, there was an association between MDD and healthcare insurance (misclassification error-adjusted PPR: 1.49; 95%BCI: 1.03; 2.50; Bias: -0.03; 95%BCI: -1.12; 0.86); however, joint adjustment for confounding and misclassification error resulted in a 95% PPR estimate probability coverage that overlaps one (adjusted PPR: 1.30; 95%BCI: 0.82; 3.24; Bias: -0.07; 95%BCI: -2.00; 0.83).

Discussion
This study is the first to demonstrate that misclassification error and confounding by period of MDD assessment (prenatal versus postnatal) can introduce bias and considerable uncertainty in estimating the association between potential risk factors and MDD among two distinct groups of mothers involved in home visitation programs. This analysis extends our previous work (i.e. impact of misclassification error on MDD prevalence proportion estimates) [26] to encompass measures of association (PPR) comparing prevalence proportions.
Our previous work [26] showed that when using prenatal and postnatal period specific sensitivity and specificity of the CESD-SF, associated misclassification error led to nearly no bias in prevalence estimates. In contrast, ignoring differences in CESD-SF sensitivity and specificity between these periods showed considerable MDD prevalence bias. Our current analyses extend this previous work to show that misclassification error and confounding by period of MDD assessment can also lead to the underestimation in some (e.g. intimate partner violence at SITE I and II) and overestimation in other (e.g. period of MDD assessment at SITE I and II) measures of risk-factor associations (i.e. Prevalence Proportion Ratios) with varying magnitudes depending on the source population of interest.
Our current findings show that failure to account for the differential diagnostic performance of the CESD-SF during prenatal and postnatal periods and associated confounding effects can erroneously create an impression of negligible bias and uncertainty in estimation of measures of association than actually exists. Indeed, as evidenced by the generally larger median bias estimates for suspected MDD risk factor associations based on the joint-adjustment for misclassification error and confounding versus adjustment for misclassification error only, the strength of an association could be underestimated or overestimated. Our results also suggest that adjustment for misclassification error and confounding come with the cost of diminished precision of PPR estimates; however, this could be due to the small sample sizes in our stratified sub-group analysis as illustrated by the study site comparisons examining the same risk factors (where, SITE I has a smaller sample size [hence wider 95%BCI] than SITE II).
Differential diagnostic performance of the CESD-SF between prenatal and postnatal periods was reflected in the prior values, which were based on a study by Tandon et al [33]. The sensitivity and specificity values for the CESD20 were similar in the prenatal and postnatal period, but differed when these periods were combined. When data from these periods are combined, sensitivity is underestimated while the specificity is overestimated compared to the period-specific estimates. When period-specific adjustments are made for misclassification error and then combined, a lower proportion of false negatives is presumed. This results in a lower or negligible impact of misclassification error adjustments on the magnitude of MDD prevalence estimates in the risk factor sub-groups (i.e. risk present versus absent) across assessment periods.
In light of the misclassification error that could result from the use of case-finding instrument cut-point thresholds to identify, compare and contrast mothers with and without MDDrelated symptoms, the use of continuous scores from these instruments should be explored as an alternative approach to investigate risk factor associations in future research. While measurement error (i.e. misclassification involving a continuous scale test result) cannot be ruled out even when using case-finding instrument scores, this approach can provide a useful doseresponse gradient at different levels or categories of suspected risk factors. This is central to determining "safe", "hazardous" or beneficial levels of modifiable factors with intervention potential.
The impact of confounding by period of MDD assessment on the extent of bias observed could be explained by the: 1) differential proportion distribution of mothers with a particular risk factor (characteristic) who had assessments in the prenatal versus postnatal periods and 2) differential diagnostic performance of the CESD-SF during these periods.
In our study, the impact of the differential proportion distribution of prenatal and postnatal mothers in risk factor categories or strata (i.e. presence versus absence of a characteristic) is illustrated by a 95% probability coverage indicative of a protective effect for high social supports scores on MDD (crude PPR) at SITE II. After adjustment for misclassification error, the median PPR was lower with a narrower 95%BCI indicating an attenuation effect. However, joint-adjustment for misclassification error and confounding resulted in a 95% probability coverage of the PPR that suggested a null association could not be ruled out. In this example, the proportion of mothers who had low levels social supports (i.e. 1 st and 2 nd quartiles) was 26% higher among mothers who had postnatal versus prenatal MDD assessments. Generally, although the median bias tended to increase after joint adjustment for misclassification error and confounding at both study sites (I and II), it did not do so all the time (e.g. social support at SITE II, where median bias decreased). This makes it challenging to predict the direction of bias on measures of association without valid data analysis (i.e. addressing outcome [MDD] misclassification error within categories of a confounder [prenatal versus postnatal]. Observed variations in the extent of bias on measures of association across different risk factors in our study data have been demonstrated before in a general population study. Findings from a study by Savoca [42] examining the impact of misclassification error on sociodemographic correlates of MDD using National Comorbidity Survey (NCS) data found that without adjustment for misclassified MDD diagnoses, the magnitude of the measure of association between gender and MDD (prevalence odds ratios) and marital status and MDD were underestimated by 42% and 60%, respectively. Unlike the validation approach (criterion standard based on psychiatrist diagnoses) used by Savoca, the use of the Bayesian approach in our study is a more cost efficient approach for the adjustment of misclassification error when examining epidemiological associations. That is, the Bayesian approach does not require readministration of reference standards or use of experts to verify MDD diagnoses but rather uses existing information (as priors) to correct for and estimate impact of misclassification error on investigated associations.
Our study findings also reinforce the fact that some risk factors of MDD may be different in different source populations after adjustment for misclassification error and confounding by period of MDD assessment. Therefore, preventive interventions that are designed for specific populations may not necessary be ideal in other populations with potentially different indicators of MDD risk. As demonstrated by our results, characteristics such as adequacy of family resources may have a stronger protective association among low-income recent Hispanic immigrants (SITE II) than among US resident mothers (SITE I).

Study limitations
One limitation of this study was that the eligibility criteria of the parent studies were such that only mothers at high risk of child maltreatment (an outcome correlated with MDD) [2] were recruited for home visitation services [43]. Therefore, because the study population may have been more homogenous than the general population of mothers, the observed magnitude of measures of association between suspected risk factors and MDD may be lower than those observed in general population studies. However, this probably had little impact on our assessment of misclassification error impact on measures of association (PPR).
Another limitation of the study is the fact that the CESD-SF uses a seven-day reference period for MDD symptoms (at baseline assessment). This probably underestimated the 'true' prevalence of MDD if symptoms present during 8 to 14 days prior to the interview disappeared the following week. However, because MDD symptoms (i.e. an episode of MDD) usually last for two to six months without treatment [44,45], the impact of the short symptom reference period on risk factor associations investigated is likely small. Moreover, comparisons of sensitivity and specificity between one week and two week symptom reference period instruments have shown substantial overlap in previous studies [46]. Furthermore, the prior estimates used for the CESD-SF in our analysis are based on the assumption that the sensitivity and specificity values of the 20-and 12-item CESD instrument are comparable. Although this assumption is supported by validity equipoise between even shorter versions (2-and 5-item CESD instruments) of the 20-item CESD in previous literature [47], it is plausible that the use of CESD12 prior values may have resulted in different (but comparable) findings than those reported in this study. Ideally, while the use of prior values from a similar version of the 12-item CESD-SF is recommended, the impact of using CESD20 (versus CESD12) prior values on our findings is likely small but nonetheless cannot be ruled out.
Due to the small study sample sizes, simultaneous control for multiple potential confounding factors was not possible. This means reported misclassification adjusted PPR may be biased by potential confounders not controlled for other than period of MDD assessment. Future research should simultaneously examine the joint influence of misclassification error and confounding by other population characteristics on epidemiological associations.
Additionally, given the fact that only one test (CESD-SF) was examined in this study, the prior information suggested for sensitivity and specificity was not updated by the observed data. Moreover, the small sub-group sample sizes, especially after stratification by period of MDD assessment resulted in very uncertain posterior distribution summary estimates (i.e. wide 95% BCI for PPR estimates). Given the relatively high levels of uncertainty associated with our adjusted measures of associations (especially at SITE I), future studies or simulation studies with larger sample sizes are needed to further support recommendations for joint adjustment for misclassification error and confounding by period of MDD assessment. The use of multiple (or more than one) case-finding instrument to determine the operational definition of MDD could also be explored to reduce the level of uncertainty in observed measures of association.
It can also be argued that our study exposures or risk factors of interest (e.g. intimate partner violence) may have been misclassified. Such misclassification could be either differential (i.e. sensitivity and specificity of the violence assessment tool is different depending on a mother's MDD status) or non-differential (i.e. sensitivity and specificity of the violence assessment tool is the same irrespective of a mother's MDD status). Assuming non-differential misclassification of risk factors, our observed level of bias may be underestimated [48]. In contrast, the directionality and magnitude of bias under differential misclassification scenarios is not predictable. Future research (with larger sample sizes) should examine the composite effects of confounding, MDD and exposure (risk factor) misclassification on measures of association.
Despite these limitations, our study findings underscore the substantial impact differential misclassification error and confounding by period of MDD assessment can have on inferences regarding suspected MDD risk factors. Biased measures of association can result in poorly planned preventive interventions that ignore or miss etiologic risk factors related to MDD illness. Anderson.