Introduction

Over the last five years, with the shift to value-based care, community resource referral technologies have been adopted by many U.S. health systems.[1–3] These platforms, which aim to connect patients to community-based resources for health-related socioeconomic needs (e.g. food and housing support, transportation) serve to advance health systems’ population health care delivery models and their efforts to mitigate inequities due to social and structural determinants of health and disease.[4,5] Mainstream data sources about healthful community resources are commonly outdated, idiosyncratic, analog, lack eligibility information and must be accessed outside the usual clinical informatics workflow.[3,6] As a result, clinician and patient efforts to make use of these resources to promote health or manage with illness are often frustrating and inefficient.[3] Sustainability of these interventions in healthcare practice depends not only on their seamless integration with usual clinical workflows, but also on their demonstrated impact on individual and population health-related outcomes and costs.

In contrast to medical drug and device interventions, information interventions delivered by a clinician to a patient can quickly spread to other community members through social networks. Evidence suggests that social networks can potentiate the impact or credibility of health-related information [7,8] and that patients generally view information obtained from a clinician as highly trustworthy.[9] Therefore, information spread through social networks from an intervention delivered in the clinical setting has the intriguing potential to meaningfully impact people in the community beyond those who directly received the intervention. Alternatively, negative information or misinformation spread could be harmful to individual and public health efforts. However, traditional clinical trial designs, which typically assess the impact of an intervention on the individual patient, do not capture the dynamic, multi-level impact of information-based health interventions.

CommunityRx is a scalable, evidence-based information technology intervention designed together with a broad diversity of stakeholders, including residents, on Chicago’s South Side to promote population health by connecting patients to health-promoting community resources (“resources”).[10,11] CommunityRx was created, in part, due to systematic deficiencies in mainstream data sources which under-represent health-promoting businesses and organizations in higher poverty communities.[6] These deficiencies impede the efforts of healthcare professionals to meaningfully and equitably execute on clinical best practice guidelines that indicate a wide range of community resources for various health conditions.

The primary mode by which CommunityRx disseminates resource information is via the “HealtheRx,” a printed list of local resources personalized to the patient’s age, gender, home address, health conditions and preferred language (Fig 1). A HealtheRx is auto-generated at the point of medical care using software algorithms integrated with the electronic medical record (EMR) to match individuals to an indicated set of community resources (Fig 2). With the press of a digital button in the EMR by a clinician (e.g. physician, nurse or clerk), extant data in the patient’s chart (e.g., age, gender, home address, preferred language, ICD-9/10 codes) are consumed into the CommunityRx algorithm to generate a HealtheRx.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Sample HealtheRx generated for a patient with cancer.

https://doi.org/10.1371/journal.pcbi.1009471.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Visualization of how a HealtheRx is Generated Using Data from a Patient Electronic Medical Record and the CommunityRx Software Algorithm.

https://doi.org/10.1371/journal.pcbi.1009471.g002

Indicated resources for each condition and status are informed by best clinical practice guidelines, expert opinion and community member input about self-care activities for more than 30 common health conditions (e.g., diabetes, hypertension, obesity), social conditions (e.g. food insecurity, housing instability, domestic violence) and statuses (e.g., newborn, adolescent, pregnant). Each HealtheRx includes resources to address an individual’s basic needs (e.g. food, housing), physical and mental wellness (e.g. fitness, nutrition, yoga), disease self-management (e.g. smoking cessation, weight loss) and caregiving (e.g. cancer support, respite care, home hospice) needs. Clinicians deliver the HealtheRx to patients at discharge from a medical visit, including emergency and hospital care. An independent quasi-experimental study of CRx comparing people who received one or more HealtheRx to matched controls found a significant decrease in hospitalization among Medicare beneficiaries and a decrease in emergency department utilization among Medicaid beneficiaries.[10,12,13] Cost savings over the short term were modest to neutral. Delivery of the intervention by a clerk is less costly than delivery by a nurse or physician. Experimentation is needed to inform the lowest cost delivery method while maintaining information spread to the broader population.

Inspired by the Kilbridge et al. prescription management process for drugs, community resource prescribing requires several steps.[11,14] The community resource prescribing “decision-making” process is initiated during the clinical encounter, supported by the software algorithms described above. The personalized prescription is auto-generated from extant data in the patient’s EMR (e.g. age, gender, home address, preferred language, problem list) plus any new data (e.g. a new diagnosis or social risk) entered during the visit. Once the HealtheRx is generated and printed, it is “fulfilled.” Fulfillment begins with “evaluation” when a clinician or clinical staff member reviews the HealtheRx with the patient and is “dispensed” when the patient contacts or visits the community resource provider. The community resource is “administered” when the patient uses, receives or consumes the recommended resources. Kilbridge’s drug e-prescribing model does not contemplate the possibility of sharing or diffiusion (although prescription medications are sometimes shared).[15] However, in the CRx iteration of the e-prescribing process model, we did anticipate that prescribed information could be shared. We therefore designed our primary research to assess for sharing.

With initial funding from a Center for Medicare and Medicaid Innovation award (3/2013-5/2015, “CRx-1”), CRx generated 253,479 HealtheRxs for more than 113,000 patients over a 30 month period.[11] The HealtheRxs were typically delivered by a physician, nurse or clinical clerk, depending on local site preferences. All patients living in the target geography were eligible, including tens of thousands of people with cancer, cardiovascular disease, pulmonary disease, obesity and other common chronic conditions. In a telephone survey of 458 HealtheRx recipients ages 18–93 years enrolled in CRx-1, 71% reported that they learned about a new resource from the HealtheRx and 83% reported the HealtheRx was “very useful.” In a subsequent real-world clinical trial (CRx-2), 411 patients ages 45–74 years were followed for three months after receiving a HealtheRx. Key outcomes, including self-efficacy for finding and knowledge about healthful community resources improved.[10,16] In addition, the CRx-1 and 2 studies yielded evidence of both intervention and information spread. Soon after the intervention was deployed at the first community health center, a physician “hacked” the CRx system in order to be able to use the intervention at a clinic site where he worked outside the target geography, thereby causing unexpected, but desirable spread of the intervention to other communities.[11] In CRx-1, 49% of participants reported sharing resource information from their HealtheRx with at least one person;[11] this finding was replicated in CRx-2.[10] Additionally, about half of nurses and physicians who participated in CRx-2 also shared HealtheRx information with others.[17] The repeated finding that information from the clinical intervention spread beyond the HealtheRx recipients to a more general population motivated a systems science approach to evaluating the impact of CRx.

Systems science methods are increasingly being used to complement traditional epidemiologic and ecological studies,[18] although few clinical trials have incorporated a systems science approach. Observational and experimental studies can occur in tandem with, and inform, the building of computational models.[19–21] Agent-based modeling (ABM) is a systems science simulation methodology that captures emergent behavior among individuals (“agents”) as a function of interactions between agents and their environment. ABM is particularly useful for studying phenomena affected by second-order effects resulting from stochastic and network interactions (e.g. the population-level effect of an individual-level intervention where individuals interact with and affect each other’s behavior).[22] Additionally, ABM enables linkage of individual choice behavior to emergent population-level outcomes (like total clinic visits or other health maintenance activities). In contrast to other methods (e.g. differential equations modeling) these micro-to-macro level dynamics make ABMs particularly well suited to study an informational intervention. ABM can also serve as an efficient adjunct for experimentation that is outside the limited scope of a clinical trial or too time-consuming or otherwise costly to conduct in vivo.

ABM has been used to examine the spread of infectious disease[23–25], effects of environmental exposures [26,27] and health-related interventions and policies.[28–36] Barbrook-Johnson et al. used ABM to examine the effects of the TELL ME intervention, an information-based intervention that deployed public health communication strategies to reduce the spread of influenza.[36] The TELL ME ABM was designed to predict the effect of communication strategies on influenza spread, but did not run simulations specifically to model information flow and dynamics. A description of the validated TELL ME ABM was presented as a teaching tool, to demonstrate how such models can be used to inform policy, even in the absence of complete data. Best practice dictates that the first step in any systems modeling development is to create the proper model design and then subject the design to evaluation including peer review.[37] We have previously described in the systems science literature the technical design details of the ABM used in this study as well as the associated model validation.[38,39]

Informed by and extending this work, the goals of this study are to: (1) describe the interdisciplinary method of model building and data sources used to develop an ABM to study an information-based health intervention, and (2) describe an in silico experiment to illustrate how an ABM can generate insights to complement a clinical trial of an information-based intervention. First, using the development of the CRx ABM as a case study, we describe interdisciplinary model building methods that could generalize to other information-based healthcare interventions. Next, to illustrate how ABM experimentation can complement findings generated using traditional clinical trial methods, we describe an in silico experiment to deepen our understanding of the information sharing findings from related clinical trials. To our knowledge, this is the first study to use ABM to examine the population-level information diffusion dynamics resulting from a clinical information intervention.

Methods

2.0 Interdisciplinary model-building process

The interdisciplinary process for building the CRx ABM involved several steps, informed by extant literature[21] and illustrated in Fig 3. Investigators, including physician scientists from the University of Chicago’s Biological Sciences Division and Consortium for Advances Science and Engineering, and Argonne National Laboratory’s Decision and Infrastructure Sciences Divsion, met regularly over the course of several years to: (1) come to a shared understanding about and vocabulary to enable interdisciplinary study of the CRx intervention, (2) identify and integrate public use, clinical trial and other data sources with the ABM drawing on complementary expertise (e.g. the systems scientists were experienced with time use data and the others were experienced with public health and trial data), (3) fill data gaps by designing and implementing new data collection strategies drawing on qualitative research expertise of both groups and survey research skills of the biomedical scientists, (4) describe and validate the model,[39] and (5) design and implement in silico experimentation (a focus of this paper).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Interdisciplinary process and timeline of clinical and computational trial activities with data inputs and outputs.

https://doi.org/10.1371/journal.pcbi.1009471.g003

In addition to observational data from publicly available demographic, economic and epidemiologic datasets[10,11,40,41] and primary data collection from clinical trials, the CRx ABM also used data sourced from expert informants. Expert informants—referred to as subject matter experts in the systems science literature—are commonly used in agent-based modeling to fill gaps in extant data.[42] Among the expert informants involved in developing the CRx ABM were people who lived in, shared demographic characteristics with and had decades of experience serving and providing medical care for people in the target geography. Primary data collection, including two clinical trials (CRx-1 and CRx-2), occurred in parallel with the CRx ABM building, with each approach informing the other (Fig 3).

3.5 The most important model processes at every time step

3.5.1 Agent activity behavior.

In addition to sociodemographic characteristics, agents were imbued with activity schedules for each simulated day. Agent activity schedules were developed from the publicly available 2016 American Time Use Survey (ATUS) dataset (https://www.bls.gov/tus/data.htm).[40] This dataset included 10,493 activity schedules covering weekdays and weekends, each associated with the demographic characteristics of survey respondents. By matching the SPEW and ATUS datasets on age, gender, race and ethnicity, each agent in the synthetic population was assigned a daily activity schedule, in 1-hour increments, from a distinct set of available ATUS schedules for weekdays and for weekends.

This matching resulted in a set of available schedules for each agent ranging between 2 to 1496. For each simulated day, an agent was randomly assigned a unique activity schedule from the set of available matched schedules. These daily schedule assignments resulted in agents being co-located in unique places. Information sharing occurred when co-located individuals interacted. The large range of available schedules for the daily random assignment was a modeling choice to introduce randomness of available activity schedules for each demographic subgroup in our agent population (alternatively, agents would follow the same schedules every simulated day). While a larger sample of available schedules does indeed introduce stochasticity, it was deemed better than the case of having groups of agents with very few schedules, which could result in unrealistically repetitive activity patterns. Future work will focus on the implication of this randomization on the resulting co-location networks. We also do not account for the seasonality of schedules due to the lack of available granular data. The implications of these modeling choices are a limitation of our model and the focus of future work.

Using the ATUS data, we classified all activities into one of two subsets: (1) activities of daily living (e.g., sleeping, washing and grooming, talking on the phone) or (2) physical and mental health maintenance or promotion (“health maintenance”) activities at both the individual (e.g., health-related self-care, doing yoga) and household level (e.g., obtaining medical care for a child or an adult in the household). Applying an iterative data collection and discussion process with expert informants, we then compared the subset of health maintenance ATUS activities to the complete list of service types listed on a HealtheRx (“service types” refers to kinds of services [e.g., fitness, weight management, smoking cessation] that a place could offer in contrast to a “resource” which refers to a specific place providing a specific service). Each ATUS activity from this subset was mapped to one or more relevant HealtheRx places where an agent might reasonably carry out that ATUS activity. For example, the ATUS activity “providing medical care to household children” was mapped to the HealtheRx services “fill prescriptions,” “medical supplies,” and “home care.” Expert informants were presented with both the subset of ATUS activities and HealtheRx service types and discussed the likelihood of an agent using a service while performing an activity for each mapping until consensus was achieved. Using the CRx resource inventory, each service type was associated with all places in the physical environment that provided that service.

We computed the average number of minutes per week spent doing each activity for each agent across all activity schedules that were demographically matched to that agent. Next, the time per week during which a given HealtheRx service could be used was determined by summing each agent’s average time doing every ATUS activity mapped to that HealtheRx service and dividing by the total number of agents for each matched schedule. This average was computed within each of four age groups (<31, 31–45, 46–65, and 65+ years), weighted by the number of people in the age group (Fig 4). These groups were defined using demographic, life course and healthcare policy considerations and are consistent with commonly applied strata for age in the clinical and population health literature.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Average minutes per week, stratified by age agents spent doing activities during which they could use a self-care service listed on the HealtheRx; Chicago, Illinois 2016–2018.

https://doi.org/10.1371/journal.pcbi.1009471.g004

3.5.2 Agent knowledge evolution.

Agents maintained dynamic β scores for resources they knew about. At initialization, agents were seeded with knowledge about resources using a distance-based random assignment algorithm. Agents were seeded with knowledge of 10 to 100 resources located within a low distance radius (parameterized as 1 mile), 1 to 5 resources within a medium distance (1–3 miles) and 1 to 5 resources for long distances (3+ miles). The first two weeks were considered a burn-in period for the entire simulation. We generally observed stabilized behavior by week 3 of the simulation. Results reported in this paper use output from week 3 of the simulation.

The β scores were boosted by information dosing, a factor based on exposure to resource information and the source of the information. Agents could be exposed to resource information by doctors, nurses, clinical clerks, social contacts and by first-hand use of the resource. The clinician sources provided information to agents by delivering a HealtheRx during a healthcare visit at a clinic (“clinical dose”). Social dosing occurred when agents co-located at places (e.g., a barber shop) and exchanged resource information (e.g. about the gym). Agents used a resource by going to its location (subsequently referred to as just “use”). The functional form of β score evolution and its parameterization were created in consultation with expert informants and through sensitivity analyses as previously described.[39] The β score for agent i about resource j at time t evolved according to the following functional form and is described below: (1)

The decay parameter λ (0.991 for the simulation run producing Fig 5, with an allowable range of values between 0.991 and 0.9994) accounted for receding knowledge of a resource. The source of resource information was associated with a value, ε_x, where x ∈ (Doctor, Nurse, Clinical Staff, Use, Peer, None}. “Use” indicated an agent had been to or consumed a given resource following the decision function in Eq 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Exemplar instance of the evolution of three agents’ knowledge (Beta, β) of eight resources over time (λ of 0.991 used in this instance based on sensitivity analysis and model calibration previously reported in [39]); Chicago, IL 2016–2018.

Note: Each column (n = 3) represents a unique agent. Each row represents a unique resource (n = 8). Each black dot indicates the β scores (left y-axis) at in point in time in hours (bottom x-axis). Information dosing events (receipt of information about a given resource) that occurred during a given hour are indicated by vertical lines as: receipt of a HealtheRx (blue), receipt of information about resources from a social contact (green) and use of a resource (red).

https://doi.org/10.1371/journal.pcbi.1009471.g005

For the simulation run used to produce Fig 5, the following values were used ε_x = {0.05, 0.15, 0.25, 0.2, 0.9, 1}. Sources expected to have a greater influence on the agent had higher dosing values (corresponding to smaller ε_x values). As modeled, ε_x only accounts for the source from which the information was received and not the type of resource information received. This modeling choice was based on one of our specific questions: how does the information source influence information diffusion dynamics? This question is salient to real-world deliberations about how best to deliver community resource referrals in the clinical context. The ABM is designed so it could easily be iterated to include a resource-specific information dynamic. Furthermore, information dynamics may vary by factors like the stigma associated with information about a given resource type (e.g. HIV/AIDS support group, food pantry or substance use counseling). Future model development could be iterated to account for this kind of dynamic. As referenced previously, the knowledge evolution process does not account for a negative effect (where negative or incorrect information could lead to an agent avoiding a specific resource). Although related studies have surfaced no evidence for negative information spread, it is certainly possible. This factor is a limitation of the model left for future model development.

Agents maintained knowledge about a maximum of 200 resources; resources with lower β scores were replaced by resources with higher β scores over time. The assumption of a memory load of 200 resources was made to represent the bounded knowledge and ability of the agents to maintain information for their decision-making (described below). The CRx ABM allows for the size and heterogeneity of agents’ cognitive capacity to be parameterized. Through the dynamics of agents’ β scores, the CRx ABM models internal agent states with respect to resource knowledge.

3.5.3 Agent information sharing behavior.

Information sharing was also modeled to depend on the nature of individuals’ activities (e.g., there is no information exchange when an agent was sleeping, whereas information can be exchanged when co-located agents were exercising in a gym). The propensity for an agent to receive information (“p-score”) was dependent on the activity in which the agent was engaged. Each ATUS activity in our subsets was assigned a propensity for information sharing. Information sharing was assumed to occur between two or more co-located people by way of face-to-face conversation. (Therefore, information shared digitally, for example, was not captured in the model.)

Because we found no extant data to inform assumptions about propensity for information sharing during various activities, we surveyed expert informants in iterative rounds to generate p-scores for each relevant activity. The survey elicited the likelihood (p-score = none, low, medium, high) that the respondent would receive resource information from another person while doing a given activity. For example, “sleeping” uniformly generated a p-score of “none.” Activities like “helping household adults” and “grocery shopping” produced a p-score of “medium” and “socializing, relaxing, and leisure as part of job” and “obtaining medical and care services for household adult” produced scores of “high.” Agents at a location share information with other co-located agents based on a threshold defined by the propensity for resource information sharing during that activity and individual random draws against that threshold. The characteristics of agents receiving information were not a factor in the information sharing dynamic. The design of this mechanism was chosen to reflect the fundamental information sharing dynamic commensurate with the expected propensity of resource information sharing determined with expert opinion, and can be considered as a limitation imposed by the data on the model. However, this information sharing dynamic does allow us to compare social dosing effect to the direct clinical dosing effect. The parameter values for these effects were critical to model calibration and was previously described in detail.[39]

3.5.4 Agent decision-making.

We then developed processes for agents’ decisions about using a resource (recall, defined as a specific service at a specific place). These decisions were dependent on agents’ information-sharing behavior and their knowledge evolution. Each of the parameters involved in the decision processes was informed by empirical data and expert opinion (Table A in S1 Text). Sensitivity analyses performed on selected parameters have been previously described.[39]

The agent’s decision to use a resource was modeled as a binary choice A/B Decision Model. Each activity was characterized by a decision type–A/B decision or not. Activities classified as health maintenance or promotion activities (the activities that the CRx intervention targets) faced an A/B decision choice. Other activities not related to health-maintenance behaviors did not require a decision choice–agents would proceed with such activity at the designated location. Agents presented with an ATUS activity in their daily schedule that was subject to an A/B decision faced a choice to use a resource or not. For example, when presented with the ATUS activity “fitness,” the agent chose whether to use a fitness facility (Decision A) or not (Decision B). If the agent decided to use the resource, they proceeded with the activity in their schedule mapped to that service. If they chose not to use the resource, they continued doing the previous activity in their schedule.

The following agent characteristics dictated the decision process for each agent i∈P¸ where P is the set of all agents:

1. natural activation level (α_i), derived from Skolasky et al. [50]
2. a function of their knowledge (β_i,j) about a specific resource j∈ R, where R is set of all resources (e.g., a community center), where they could use a specific service (e.g. a gym),
3. the distance to the resource for the agent (δ_i,j), and
4. the inherent inertia that they needed to overcome for using that resource (γ_j).

An agent’s decision to use the resource, Decision A, was made when the agent’s activation level (threshold) was exceeded by the agent’s combination of knowledge and effort required (distance and inertia). The relationship at the time (t) is given by: (2)

To provide an illustrative example–consider an agent following their daily hourly schedule until they come to an activity requiring an A/B decision: go to a gym or not. The agent uses the decision calculus described above to choose between the gym (A) or their current (B) activity. If the agent knows of no gym, the β will be low. If the gym is far away, the δ will be high. The agent will continue their previous activity (a B decision in Eq 2 above). The dynamic described in Eqs 1 & 2 allow us to isolate and measure the effect of information dosing on knowledge about and use of selected resources for health maintenance or promotion activities that were the focus of the CRx intervention. The model does not include other health promotion and maintenance activities occurring at other places, for example going for a walk outside or an informal support group at a home.

4.1 Generating endogenous information diffusion networks

We simulated the information diffusion processes within the CRx ABM and generated endogenous information diffusion networks based on hourly snapshots of co-located agents exchanging information based on ε_x values (described in Section 3.5.2) and the propensity to share information. In these networks, individual agents were network “nodes,” and information exchange events between agents were the network “edges.” The links in the networks signified the pathways through which information about resources was exchanged. We created an exemplar age-stratified visualization of these networks using Gephi (https://gephi.org) (Fig 6).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Visualization of network pathways through which agents, stratified by age (16–25 years = orange, 65+ years = purple) exchange information about resources; Chicago, Illinois 2016–2018.

https://doi.org/10.1371/journal.pcbi.1009471.g006

4.2 Experiments on propensity for information sharing

We ran computational experiments, varying input parameter values that governed the CRx ABM’s information diffusion processes. This process helped us analyze how propensity to share information (p-score) affects population level information networks. We parameterized the low, medium and high p-scores into adjustable rates of information exchanges to modify a fundamental generator of the information diffusion network. We show difference in network degree distributions across different levels of p-scores (Fig 7).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Differences in network degree distributions (distribution of total incoming and outgoing information pathways) as the rates of information exchanged are adjusted higher, black to red to blue; Chicago, IL 2016–2018.

https://doi.org/10.1371/journal.pcbi.1009471.g007

4.3 Experiment to determine optimal mode of CommunityRx intervention delivery

To estimate the magnitude of the spread of HealtheRx information from social dosing alone, we calculated the ratio of agents who retained HealtheRx information from social dosing exclusively to those who retained HealtheRx information from clinical dosing (using logged simulation data). Using a calibrated parameter set (technical description of model calibration and validation are summarized in Section 3.7 and described in detail in [39]), we used the ABM to run 4 week long simulations, with 15 runs for each of 3 HealtheRx delivery modes (delivered by a physician, nurse, or clinical clerk). We generally observed stable behavior by week 3 of the simulation. We used week 3 outputs to report results (the first two weeks were used as the burn-in period for the simulation). Parameter values for the 45 experimental runs are shown in Table C and D in S1 Text. For qualitative comparison of geographic spread of information across the 16 ZIP codes in the CRx ABM, we plotted the home location of agents who retained information from direct clinical versus social dosing during week 3 of the simulation (Fig 8 and Appendix B in S1 Text).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8.

Simulation of the geographic spread of community resource information via (A) clinical dosing and (B) social dosing using the CRx agent-based model. Agents who received clinical dosing could also receive social dosing. The base layer of this map was obtained from Stamen Maps available at https://stamen.com/open-source/.

https://doi.org/10.1371/journal.pcbi.1009471.g008

Results

Discussion

CRx is an information-based intervention that connects patients to healthful community resources. In prior studies,[10,11] we discovered that nearly half of patients and clinicians who received the intervention shared information about resources with others, providing evidence of a potentially potent but previously unexplored pathway through which a clinic-based intervention can impact population health. Based on these observations, this study describes how we built a computational laboratory to facilitate the quantification of the population-level impact of CRx. Prior studies of similar information-based interventions have relied on conventional observational and experimental designs.[5] These designs capture the interventions’ individual-level impact, but not the population-level effects resulting from the ease with which health-promoting information from the intervention can spread. Information technology is enabling the rapid growth of information-based interventions in public health and health care.[55] To our knowledge, ABM has not yet been used to study the population-level impact of a health information intervention delivered during a clinical encounter.

In contrast to a clinical trial, a computational laboratory is designed to efficiently explore “what-if” questions at large-scale. In the case of CRx, a clinical trial asked “What is the effect of a community resource information intervention on middle age and older adult patients’ self-efficacy for self-care?” We conducted an experiment to address the question: “What is the effect on information diffusion if the intervention is delivered by a physician versus a nurse versus a clerk?” Overcoming experimental capacity of a single clinical trial, the ABM will also allow us to ask other important questions like: “What if the patients who get the intervention are young adults?” Or, “What if the intervention is delivered based on screening versus using a universal approach? What will be the impact on health outcomes and how will the population effects of the intervention be different?”

This study shows how an interdisciplinary team of biomedical and systems scientists collaborated to build a computational laboratory. We have demonstrated that it is realistic enough for its purpose, by observing patterns that replicated intuitive or previously-known results. As expected, we demonstrated that synthetic agents exhibited diverse behavioral characteristics, reflective of the known geographic population, and that the synthetic intervention could be delivered to these agents in silico. We also showed that agents’ resource knowledge evolved as they were exposed to resource information and evolved differently depending on the information source. We illustrated the emergence of information diffusion networks and how the degree distributions of these networks changed as the values of key parameters like the propensity of information sharing were varied. Lastly, we demonstrated the computational feasibility of using highly granular activity data, an important feature of a study seeking ultimately to uncover when, where and how information spreads between people in a population. Combined with prior validation work, these results help to establish the model’s readiness for hypothesis-testing experimentation.[38,39]

To illustrate the potential of the ABM as a complement to clinical methods, we also conducted an in silico experiment to examine how specific resource information disseminates from an individual exposed to the CRx intervention in a clinical setting (“clinical dose”) to other individuals in the population through social dynamics (“social dose”). Through 45 experimental runs simulating mixing behaviors in a geographic population, this study finds that personalized community resource information delivered to a patient during a healthcare visit spreads to others in the community. Complementing the three-month trial’s observation that the total number of people reached by HealtheRx information was at least double the number of middle-aged and older patients in the intervention group [17], the ABM in silico experiment estimates this ratio to be approximately 4-fold, when including all people ages 16 and older in the geographic population. We also find that the mode of clinical delivery (physician, nurse, clerk) likely has little relative impact on the magnitude or dynamics of spread, suggesting that clinical implementation of community resource referral solutions might focus less on who delivers the intervention and more on optimizing patient and clinician access to it. These empirical and simulated observations give life to the concept of patient social dosing, a previously overlooked mechanism, and potential force multiplier through which community resource referral and other health-promoting information interventions may deliver value beyond the patient who receives direct intervention during a healthcare visit. To our knowledge, this is the first study to examine the population-level diffusion dynamics of a clinical information intervention.

Most ABMs are built using data from extant literature, secondary datasets and expert informants.[42] In 2010, Morell and colleagues proposed an integrated approach where data and insights generated through observational evaluations are used to drive building and parameterization of ABMs.[21] Public health researchers subsequently voiced support for this complementary approach.[19,20] Building on and extending these ideas, the CRx ABM was constructed in tandem with both observational and experimental studies and was parameterized using data from these studies. Building the ABM to simulate agent-level information sharing, a dynamic observed in CRx trials, revealed a need to assign a propensity for information sharing during agent activities. Because we had not anticipated this data need at the outset of the trial, we generated this parameter set by surveying expert informants. The addition of primary data collection to estimate the propensity for information-sharing during daily activities would increase the value of population-based time use surveys, like ATUS, to in silico experimentation with information-based interventions.

In parallel with ABM building, iteration and experimentation, we continue to operate clinical observational and experimental trials to deepen our understanding of how information about resources spreads and the impact of the resource information on a theory-driven set of health and healthcare outcomes.[56,57] These empirical data from clinical trials will be used to further parameterize and improve the ABM over time, which, in turn, will enable more advanced in silico experimentation. For instance, the CRx ABM can be used to compare community resource use in relation to varying rates of information spread and network size and other characteristics to inform assessments and decision-making regarding resource allocation. These, and other experiments, are a focus of future work.

Although ABM-based experimentation enables rapid iteration of community-wide trials under varying conditions that would be infeasible to conduct in the real world, there are limitations. The CRx ABM is driven by a decision-making model that is a simplified representation of factors influencing how people decide to use health-promoting community resources. For example, we only account for face-to-face information sharing, not other modalities like social media. The validity of ABM-based experimentation is dependent on the salient factors captured by the underlying behavioral models. Some agent characteristics that are likely dynamic in the real-world are static in this first iteration of the model (e.g., agent activation). Modeling behavior change is an active area of systems science research;[58] we expect that new discoveries will inform iterations of our model over time. Also, the CommunityRx ABM only accounts for knowledge about and use of local health-related resources that could be listed on the HealtheRx. Health maintenance activities could occur outside of these resources (e.g., walking in the neighborhood, an informal support group at someone’s home). We also have a limited understanding of which and how many resources a person needs to maintain health, how this information is retained in memory and how these factors vary among individuals. Lastly, the propensity for information sharing during specific activities is an important variable in our ABM, but our estimates may be limited by use of expert informant data. This limitation reveals a data need that will be important to future computational studies of information-based interventions and presents an exciting opportunity for new empirical research. Although these are important limitations, the in silico laboratory is flexible; we can iterate the decision model in tandem with emergent empirical data and advances in the field more generally. Thus, while the focus of the CRx ABM is on simulating the CRx intervention, the process of model building and computational experimentation presented is generalizable to other large-scale ABMs, for example those modeling information diffusion processes. However, generalizability may be limited because the model was validated only against the data that were used to inform the model building.

We have demonstrated the process and feasibility of integrating clinical trials and systems science methods to build a flexible laboratory for studying the population health impact of an information-based intervention delivered to individuals at the point of medical care. By using open source tools and sharing our methods,[38,39] we aim to build trust in our approach, prompt feedback from peers, and enable others to use, iterate and learn from our model. In addition to advancing knowledge specific to understanding and valuing the impact of CRx, this work serves to advance knowledge and testing of a fast-growing range of information-based health interventions being developed for delivery to promote patient and population health.

Supporting information

Acknowledgments

The authors (N.C., C.K., C.M.M., J.O.) would like to acknowledge facilities and material resources support provided by the U.S. Department of Energy Office of Science to Argonne National Laboratory under Contract No. DE-AC02-06CH11357.