Aim 1.

To characterize states’ telemental health policies and identify state-level determinants of expansion. We hypothesize that state telemental health expansion (i.e., policy adoption) will be predicted by high COVID-19 infections and deaths, rurality, mental health professional shortage, and broadband internet coverage.

Aim 2.

To examine effects of state telemental health policy expansion during the COVID-19 pandemic on a) access to, use of, and expenditures related to mental health care and b) mental health outcomes, functioning, and employment. We hypothesize that state telemental health expansion (i.e., policy adoption) will a) increase (or mitigate pandemic-related reductions in) access to, use of, and expenditures on mental health care, and b) improve (or mitigate pandemic-related worsening of) mental health symptoms and functional outcomes, e.g., quality of life and employment.

Aim 3.

To understand key partner perspectives on how state telemental health policy implementation and expansion influenced mental health services delivery during the pandemic, focusing on the impact in safety-net settings. Through interviews with state leaders, administrators, clinicians, and clinic staff, we will collect feedback on policy-to-practice telemental health implementation and impacts on mental health care quality and equity to explain Aim 1 and 2 findings about state telemental health policy approaches during the COVID-19 pandemic. We will also elicit input regarding the risks and benefits of relaxing telemental health regulations and reimbursement, especially for underserved patients.

Aim 1 legal mapping (“50 state survey”).

For Aim 1, our study team will assemble a database using legal mapping to characterize telemental health policies across all 50 U.S. states and identify state-level determinants of telemental health expansion. We will identify relevant telehealth policies (including policy provisions) through full-text searches of large legal research databases (e.g., Thomson Reuters Westlaw, Lexis Nexis). This primary data collection will involve extensive review of various state-specific legal documents including executive orders, Medicaid policy statements, session laws, and other state regulatory documents compiled on state-specific websites. Effective dates will be collected and coded as the date that the policy took effect. Policies focus on mental health care access, use, cost, and outcomes, including Medicaid reimbursement policies and private payer policies, and policies dictating requirements for providers and other health care professionals (Table 1). To ensure validity of our policy mapping data, two team members will independently find and code documents for a sample of states. We will compare the findings, identify differences in interpretation, and ensure consistency in approach before coding the remainder of the policies. Once our policy dataset is complete, we will compare our findings to any existing databases, e.g., Center for Connected Health Policy (CCHP). Should inconsistencies occur between our results and secondary data sources, we will consult the text of the relevant law and seek clarification from a legal expert on our study team. We will resolve any discrepancies through discussion with the larger research team. The constructed telemental health policy database will include variables representing telemental health policy expansion, including whether policies were adopted and the number of policies adopted, for each of the 50 states.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Telehealth policy adoption expansion over time with substantial variation across states.

https://doi.org/10.1371/journal.pone.0312665.t001

Aim 2 MEPS. Aim 2 will use the MEPS panel data. The MEPS Household Component (Panels 23–26, 2018–2022) follows and collects data from a nationally representative sample of households over time, using an overlapping panel design [21]. The Agency for Healthcare Research and Quality (AHRQ) selects a new panel of households each year, and collects data covering two years in five interview rounds (mean, 4.6 completed interviews) [22]. MEPS elicits data on sociodemographics, employment, self-rated physical and mental health, chronic and mental health conditions, insurance coverage, access to care, and health care use and expenditures. Data are arranged into longitudinal files that identify responses to each interview round, as well as the date and reason for each healthcare utilization event. Average point-in-time response rates for recent panels and the combined overall response rates for recent years are respectively 46% and 41%. Beginning 2021, the MEPS event files included indicators for whether the event was in-person or telehealth, as well as telehealth modality (phone, video, or other).

Aim 3 qualitative interview and focus group data.

In Aim 3, interviews and focus groups will be conducted using semi-structured interview and discussion guides tailored for each group, with open-ended questions and probes as needed. Question development will be guided by Fortney et al.’s framework [23] and will include constructs that cannot be easily obtained using MEPS. Questions will focus on how three domains impact telemental health access and quality: 1) telehealth regulation at the community level, 2) telehealth implementation at the organizational level, and 3) telehealth quality at the individual level. The goal of the interviews with state leaders and focus groups with safety-net clinicians is to gather insights about what differentiates between high and low telehealth expansion states and identify context-specific strategies, as well as the impact of state telehealth policies on mental health services. State leaders will be asked about their perceptions on how state telemental health policies were implemented and the impact of these policies on the delivery of mental health services in their state. Safety-net clinic leaders, as well as clinicians/staff, will be asked about their experience providing a mix of in-person, video, telephone, and other telehealth modalities, in addition to barriers and facilitators to high-quality, equitable telemental health implementation.

The interview and focus group discussion guides will be refined based on feedback from an Advisory Board comprised of research experts and representatives from the National Council for Mental Wellbeing and National Association of Community Health Centers. The Advisory Board will provide input regarding interview and focus group design and content, as well as pilot test with cognitive interviews. The qualitative study team will conduct all interviews and focus groups virtually (e.g., phone, Zoom). Interviewees will be asked for verbal consent after review of study goals, risks, and benefits. Interviews will last approximately 30–60 minutes and focus groups will last 60–90 minutes. Participants will receive $50 and $100 incentives, respectively.

Aim 1 survival analysis.

To estimate the number of months until adoption of each telemental health policy across all 50 states between 2018–2022, we will conduct discrete-time hazard models [26] (survival analysis) by calculating logit-hazards monthly with 95% confidence intervals. Time to adoption will be calculated from 1/1/2018 until telemental health policy adoption or until the end of the period of interest (i.e., censored data), whichever comes first. We will incorporate a priori state-level covariates (Table 4; e.g., state demographics and payor mix) into all models to examine the influence of these variables on state telemental health expansion. To understand the impact of pandemic onset, we will include a fixed effect for time using a post-March 2020 indicator. We will also include the time-varying predictor of COVID-19 cases/deaths to capture state pandemic severity. Sensitivity analyses will be performed to investigate potential residual confounding by modeling time in varying segments, as well as lagging COVID-19 cases/deaths by varying number of months. Additionally, states that expanded Medicaid programs may rely on telehealth to meet the needs of newly insured disadvantaged populations, so our sensitivity analyses will examine for associations between telemental health policy adoption and state Medicaid expansion using Kaiser Family Foundation’s database [27]. Finally, we will examine for associations between state summary policy index and contextual variables using count data regression models with state random effects. Based on the 50 state sample, we should have 80% power to detect hazard ratios for covariates in the Cox models of at least 2.33 assuming a 0.05 two-sided level of significance. This minimally detectable hazard ratio is based on a general, standardized covariate.

Aim 2 difference-in-differences study.

To compare trends in outcomes among those living in states before and after expansion of state telemental health policies, a staggered difference-in-differnces approach will be used [28]. We will analyze each of three cohorts separately. For individual policy outcomes, analyses will be conducted for the affected population (e.g., Medicaid for audio-only reimbursement).

MEPS data enable us to compare changes over time in outcomes for specific individuals and, therefore, to control for time-invariant unobserved characteristics that may be correlated with pandemic effects. The regression model for our MEPS analyses is:

Y_itcs is an outcome for subject i residing in commuting zone c and state s interviewed at time t (refers to calendar year for utilization/cost outcomes, and to interview round for mental health/functioning outcomes);

α_i are individual fixed effects, which account for time-invariant individual characteristics;

TH_ts is an indicator variable equal to “1” if state s had a specified telemental health policy in effect during time t; alternatively, it is a count variable equal to the number of telemental health policies in effect at state s during time t;

is an indicator variable with the value “1” in the post-pandemic period and “0” otherwise. is interacted with T_ts to estimate a separate coefficient, φ, for states that adopted telemental health policy during the pandemic;

Trend_u(i) is a linear trend in the time, u(i), since subject i’s first MEPS interview; and an interaction term with Trend_u(i) to allow for subject i’s underlying outcome trajectories to differ depending on baseline self-rated mental health, MH_i0. (Baseline values are from the first interview.) Accounting for heterogeneity in trajectories will minimize the bias in the estimates of telemental health expansion effects that could occur if mental health at baseline is unequally distributed between states before and after expansion of telemental health.

X_it, the vector of individual characteristics, is limited because individual fixed effects absorb the effects of time-invariant individual characteristics. We still include characteristics that change over time and are not expected to be secondary to pandemic effects (e.g., marital status).

P_ts is a vector of pandemic-related policies in effect in state s at time t, V_ts is the vaccination rate in state s at time t, and Z_c is a vector of additional commuting zone-level covariates including broadband internet availability, social distancing, etc. ε_itcs is the error term. The Greek symbols are coefficients to be estimated.

When the treatment (telemental health policy) is implemented in staggered fashion, Callaway and Sant’Anna [28] have shown that the simple difference-in-differences estimator may be biased. In this case, the simple estimator is the weighted average of the treatment effects at each of the implementation times. Callaway and Sant’Anna developed an approach to test and deal with this problem. In our study, however, we address it through model specification using a post-March 2020 indicator variable, that is, we allow for different policy effects before and after pandemic onset. We will also develop a scheme for putting policies into the regression model one, two, three at a time and so forth to figure out which have big effects on mental health outcomes and examine for policy correlation using the variance inflation factor.

We will use linear models, including linear probability models for binary outcomes, to facilitate direct interpretation of the coefficients and avoid the implicit interactions present in nonlinear models [27, 29]. Since fixed effects exist at the level of the unit of analysis, we cannot estimate nonlinear models for MEPS in particular. Nonetheless, linear probability models yield reliable estimates of mean effects [30] and are widely used [21, 22, 31]. Since the error terms in these models are heteroskedastic of known form, we will use heteroskedasticity-consistent variance estimators. We will examine the intra-class correlation (ICC) coefficient to assess magnitude of dependence at the state levels and ensure appropriate accounting for clustering. To account for the sampling design of MEPS, including weighting and clustering, we will perform all analyses using Stata survey commands.

Furthermore, we will conduct subgroup analyses and compare to main analyses. We will stratify by age group (including seniors and adolescents aged 10–17 years) for the general population, for men and women, and for underserved subgroups where the effects of telemental health expansion are likely concentrated, including (1) persons with high school or no degree (28% of the U.S. population) [32]; (2) African Americans and Hispanic Americans (~31% of total) [33]; (3) those living in rural areas (~14% of the U.S. population); and (4) those with probable mental illness and a new episode of mental health care [13, 34, 35]. For subgroups 1–3, we will estimate the effects of telemental health expansion by estimating separate models. For subgroup 4, we will examine process measures of care quality—that is, treatment initiation, at least minimally adequate follow-up, and need for acute psychiatric care as done in previous research [13, 34, 35]. Statistical power to detect significant interactions in these subgroup will be limited, however. Our sensitivity analyses will also estimate probit/logit models and compare the marginal effects by averaging the predicted margins over the whole sample. In doing so, we will need to exclude fixed effects for MEPS panel data.

We estimated numbers for the smallest cohort of working-age Medicaid adults, assuming that we will have greater power to detect differences for larger cohorts (Table 3). With MEPS data, we will be able to detect differences of ~3% (α = 0.05) or ~4% (α = 0.01) for outcomes (e.g., baseline 17% with mental health expenditures [36]), but this should be a conservative estimate given that we will have repeated observations for subjects observed in these data. The Chi-Square test is a conservative simplification of the regression modeling approach that will be used for the planned analysis. In our preliminary analysis, we identified effects of a similar magnitude, suggesting that it is reasonable to expect that we will have a sufficiently large sample to detect policy effects on outcomes.

Aim 3 qualitative analysis.

Two qualitative analytical approaches will be used. The first approach will include rapid qualitative analysis methods [37, 38], which has been shown to save time and reduce costs [39] while yielding valid results like those from traditional coding [37]. For the rapid analysis approach, a templated summary table of key domains based on the interview or focus group guide will be developed. All transcripts will be divided and independently summarized by study team members. This initial rapid analysis will also identify additional/emerged domains. Then, senior researchers on our study team will randomly select interviews and focus groups to conduct secondary reviews of summaries and discuss discrepancies with the team to ensure consistency. All summaries will then be consolidated into high-level summary documents to identify key points and commonly occurring themes across all focus groups. We will also examine interview and focus group summaries grouped by type (e.g., high telemental health/urban, high telemental health/rural, low telemental health/urban, low telemental health/rural). The second approach will include a thematic analysis using both inductive and deductive methods to allow coding and theme/sub-theme development [40]. This process will expand on what we learn from the initial rapid analysis. Our research team will jointly develop the list of codes, using both an inductive “grounded theory” development of codes and a deductive coding based on the study theoretical framework (a priori codes). An audit trail will be used to document all qualitative methodological decisions. To ensure quantitative findings inform the qualitative approach, we will share relevant Aim 1 and Aim 2 state findings and ask for interviewee feedback. For example, we may describe which telemental health policies were in effect based on Aim 1 and share the percent of individuals reporting telemental health use over past month derived from Aim 2. We will use Atlas.ti (v.9+) to organize data analysis.

In accordance with the Belmont Report and under the authority of the Common Rule, the UCLA Office of the Human Research Protection Program (OHRPP) determined that our quantitative project does not meet the definition of human subjects research (IRB#22–001956) and our qualitative study meets the criteria for an exemption from IRB review (IRB#22–001993).