Introduction

Given the growing interest in the development of collaborative and continuing care strategies for people with opioid use disorder (PWOUD) [1–4], there is considerable clinical and scientific importance associated with validating prescriber networks and subsequently understanding the benefit of physician networks on treatment retention. As defined by the Agency for Healthcare Research and Quality, care coordination is the “deliberate organization of patient care activities between two or more participants involved in a patient’s care to facilitate appropriate delivery of health care services” [5]. Opioid agonist treatment (OAT) is the standard treatment for opioid use disorder and improving treatment retention has been a key component in public health strategies to reduce the risk of overdose and subsequent mortality among PWOUD [6–10]. While structural and individual-level characteristics have an impact on retention in OAT [11], clinic-level characteristics, such as receiving care at a specialized treatment center or hospital [12], are also important determinants of OAT retention [11,13–17] which are directly amenable to systemic intervention.

Within Canada, 80% of all Canadians receiving OAT reside in Ontario or British Columbia (BC) [18]. The largest treatment provider organization in Ontario is the Canadian Addiction Treatment Centre (CATC), which operates outpatient and inpatient treatment, with specialized pharmacies [19]. In BC, OAT is provided through specialized treatment centres, such as rapid access to addiction care clinics, community health centers and physicians’ private offices. Like Ontario, BC boasts comprehensive linked health administrative data, however clinic-level identifiers are not available within these databases [20], thus limiting the opportunity for clinic-level comparisons in practice patterns and outcomes. Federal and Provincial governments have proposed increasing connections to care and collaborative care practices as interventions to improve outcomes for individuals with substance use disorders [21–23]. The Canadian Centre on Substance Abuse recommends single assessment processes, shared medical records and centralized access points as mechanisms to work towards collaborative care [24].

Evaluating the performance of collaborative care and speciality clinics can be labour-intensive and thus cost-prohibitive when conducted through observations of practices, qualitative interviews and surveys. The use of administrative data may be feasible for routine, ongoing evaluation but many datasets, lack or suppress facility-level identifiers, including only de-identified provider information. Two previous studies conducted in the United States demonstrated that patient-sharing, identified using administrative data, is an informative measure for determining relationships between physicians [25,26]. More recently, a study used Medicare data to identify shared care between primary care providers [27]. Given the case mix complexity and potential need for multidisciplinary care for people presenting with OUD, these methods require validation within this population and care context to ensure accuracy. However, no studies exist to validate the use of social network analysis to measure relationships among OAT physicians, and further identify clinics when administrative data lacks the identifying information on facilities or survey data is unavailable. The inclusion of de-identified clinic identifiers in Ontario’s CATC data presents an opportunity to fill this gap.

Social network analysis can be further applied to identify the impact of these networks on patient outcomes and health care system performance through the evaluation of care coordination [5,28,29]. While care coordination is challenging to measure through administrative claims data, “care density” is a surrogate measure which suggests that indirectly, through having visits with the same patients, physicians develop relationships that result in opportunities for direct communication and information sharing that may lower barriers to care coordination and ultimately lower costs [5]. Other applications of social network analysis include improving the implementation of new policies or clinical guidelines through network interventions that utilize the theory of diffusion of innovation by identifying and targeting individuals or groups for behaviour change [30,31]. Validating the identification of clinics via administrative data provides the groundwork to identify and target groups of clinicians on a clinic-by-clinic basis for a segmentation style network intervention [30] while utilizing the lower evaluation cost associated with administrative data.

We aimed to determine whether prescriber networks, constructed via de-identified physician IDs in daily drug dispensation records, can successfully identify distinct clinical practices. We employ electronic medical record data from the largest network of physicians providing OAT in Canada (a group of 105 OAT clinics in Ontario, Canada) to assess the concordance between constructed prescriber networks and actual relationships between physicians.

Methods

Results

Characteristics of physicians, clinics and identified facilities

We included 144 physicians treating 32,842 OAT clients at 105 clinics over the 7-year study period to assess 350 different versions of the created networks (S1 Table). Clinics were identified on an annual basis. Between January 1^st 2020 and December 31^st 2020 there were 59 CATC clinics that treated OAT clients, and we identified 36 communities where physician-clinic connection was defined as at least 15% of the physician’s client load (Table 1, S2 Table). Among these communities, recall (the number of physicians identified in the communities that were also in the administrative identified clinic) had a median of 50%, 1-purity (the number of physicians identified in communities that were not in the administrative identified clinic) had a median of 0.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of characteristics for network-identified clinics and administrative identified clinics from 2020.

https://doi.org/10.1371/journal.pone.0322064.t001

Among the 36 communities, we presented three clinics and their matching network-identified practice as an example visualization (Fig 1). Clinic #25 was identified in the networks with no additional or missing physicians, however as the two overlapping physicians’ sub-communities had no clients shared, they were identified as two separate distinct clusters. Clinic #154 in the network-identified clinics had missed the inclusion of three physicians, however, all missing physicians (identified as H, G, and F) had small client loads (one to four clients). Finally, Clinic #149 missed the inclusion of two physicians, with “H” having a small client load (4 clients, all treated at different clinics), and “I” having a large client load, but more of their client load at a different clinic.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Clinic structures based on administrative data with clinic-level identifiers (right side) and the comparable network-identified practice based on shared clients (left side).

Shared patient network (left) is created through prescription records and connections are defined as physicians prescribed to at least one shared patient. Clinic network (right) is identified through physician-clinic connection in the Canadian Addiction Treatment Centre (CATC) administrative records. Physicians are labelled to the unique practice. Edge thickness is the log of the shared clients plus 1. Vertex size is the log of the physicians’ client load plus 1. *Practice 25 is over two different network-identified clinics as they cannot be grouped without shared clients.

https://doi.org/10.1371/journal.pone.0322064.g001

Concordance statistics for network-identified clinics

When varying the detection algorithms and physician-clinic connection definitions, with ties defined under the relative threshold at 80%, sensitivity estimates ranged from 0.02 to 0.93 and PPV estimates ranged from 0.04 to 0.98 (Fig 2). Specificity estimates ranged from 0.49 to 1.00 and NPV estimates ranged from 0.44 to 1.00. Results from the Walktrap, Louvain, and Label Propagation algorithms had lower specificity and PPV compared to the modularity maximization algorithm. The Louvain algorithm had slightly higher estimates among all categories than Walktrap, except when physician-clinic connection was defined as any OAT client treated through the clinic. Label Propagation had the highest sensitivity, with the drawbacks of the lowest PPV and specificity. Among the modularity maximization algorithms, the fourth iterations’ 0.03 increase in PPV resulted in a 0.12 reduction in sensitivity. Altering the definition of a true physician-clinic connection from the percentage of a physician’s client load to any clients treated at the clinic reduced sensitivity to near zero, and NPV to near 0.50 (Fig 2, S3 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Concordance statistics for different community detection algorithms and physician-clinic connection definition to identify the gold-standard administrative clinics.

Abbreviations: MM – Modularity maximization.

https://doi.org/10.1371/journal.pone.0322064.g002

We altered the tie definition to assess concordance among all tie classifications with the community detection algorithm set to modularity maximization with two iterations and physician-clinic connection defined at the 15% level (Table 2). Overall agreement and negative agreement were unchanged at 0.96 and 0.98, respectively, across all definitions. Three different tie definitions featured the highest positive agreement of 0.49. When trying to achieve the best balance among the four validity measures, the solution with high specificity (0.98 (0.98, 0.98)), high NPV (0.98 (0.97, 0.98)), medium PPV (0.54 (0.52, 0.57)) and medium sensitivity (0.45 (0.43, 0.47)) had two different tie definitions with the same result: Relative percentage defined as at the top 80% of both physicians shared clients or an absolute threshold of at least one client. Gwet’s AC₁ estimates for both definitions were 0.96 (0.95, 0.96). Further, these were the highest performing set of all 350 versions of the detection algorithms, physician-clinic connection and tie definitions (S1 and S2 Fig and S3 Table). Other algorithms with higher PPV had low or near zero estimates for sensitivity or only medium NPV.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Concordance between physician pairs’ assigned network identified community and the gold-standard administrative identified clinic.

https://doi.org/10.1371/journal.pone.0322064.t002

Discussion

Using a previously validated methodology [26,27,36], we compared 350 different algorithms to validate the identification of OAT clinics from administrative data by creating prescriber networks from shared clients. Using a relative percentage to define network ties with two or three recursive iterations of the modularity maximization algorithm were identified as the best performing strategies for constructing network-defined clinics due to high specificity and NPV and moderate PPV and sensitivity. The misclassification from the algorithm occurred when physicians had low client loads or comparably high client loads across multiple clinics.

Generally, use of administrative databases for evaluation and quality improvement has limitations, particularly with missing information and the lack of identifying indicators for clinical sites and referral networks. These limitations are exceptionally relevant for clinical OUD management in the fentanyl-era, when rapidly accessible specialty addictions clinics [41] and collaborative care [1,4] have been introduced as one component to combat the declines in treatment retention and persistently high rates of overdose death among people who use drugs [8,42–44]. By validating the identification of clinics, we provide an opportunity to distinguish and evaluate clinical caseloads and clinic-level performance among databases that lack clinic-level identifiers, thus improving the use of administrative health records for ongoing evaluation and quality improvement particularly for opioid use disorder. Additionally in other clinical areas, care coordination, measured through indicators such as care density and centrality, has shown to have an impact on rates of hospitalization, health care costs, as well as on the referral and prescribing practices of physicians [5,28,29,45]. A previous study in the United States analyzed insurance claims to demonstrate that patients receiving care from doctors with higher levels of shared patients (i.e., higher care density) had significantly lower total costs and rates of hospitalization [28]. Otherwise, another study in the United States applied network analysis to demonstrate that higher care density is associated with a reduction in repeated co-prescription of interacting drugs by multiple providers [29]. Centrality, which describes the importance or influence of a node within a network, is another indicator of coordination [46]. For example, Barnett et al. compared the relative ratio of primary care physician (PCP) centrality to other physicians’ centrality to quantify the average centrality of PCPs compared to other physicians in a network and found that higher “centrality” of primary care providers within constructed hospital networks was associated with fewer medical specialist visits, as well as lower spending on imaging and tests [45]. Validation of network-constructed clinics may therefore have broader implications and applicability in programmatic and health system evaluation.

Identifying facilities with administrative data otherwise provides an opportunity for enhanced control of confounding, which may be otherwise unmeasured in comparative effectiveness studies, when there is a lack of clinic-level identifiers. A previous study that identified predictors of selection into OAT had estimated individual and prescriber-level impacts with residual variance between prescribers [47]. This unexplained variance could be related to clinic-level characteristics that were unable to be determined previously. Furthermore, measures of prescribing preferences are commonly used in comparative effectiveness studies using instrumental variable analysis to address unmeasured confounding by indication [48–52]. A 2021 systematic review of preference-based instrumental variable designs found that of 185 articles, 40% had implemented a facility-level measure of prescribing preference [50], highlighting another important use to identifying clinics with administrative data that lacks facility-level identifiers. Additionally, as physicians’ peers’ prescribing habits has been shown to impact treatment uptake and selection of medications [53–55], these outcomes could be improved by identifying the impact of clinic level prescribing practices.

The results from our chosen ‘best-performing’ settings had an estimated PPV of 0.54 and sensitivity of 0.45 which is comparable to the study by Kuo et al on US Medicare clients with similar sensitivity, however slightly lower PPVs [27]. Specifically, when Kuo et al defined a tie as at least 14 shared patient they had nearly the same sensitivity. However, we had much higher estimates for specificity and NPV. This may be explained by the administrative data having a higher proportion of physicians not practicing at the same clinics because they are easily identified if they do not share patients. Another previous study had similar results with a shared patient threshold of 3, resulting in a PPV of 0.60, sensitivity of 0.57, and specificity of 0.77, however their NPV was lower at 0.76 and their goal was to identify referral relationships, not team-based clinics [25]. Overall, our results align with other studies and confirm administrative databases can be used to identify physician relationships and clinics.

Prior studies building physician networks used an absolute shared patient threshold utilizing Medicare from the United States [25,27,36]. However, we found that when network ties were defined based on relative percentage thresholds the results were concordant among all of the percentage-based definitions. As such, and consistent with the findings and recommendations of Landon et al [26], our results suggest using relative thresholds over absolute shared patient thresholds for network construction to accommodate diverse clinical contexts with varying patient loads and case complexities. Another study by Landon et al suggested similar, as absolute thresholds will vary for different specialties [26]. While Landon et al did not report their results and comparisons, they concluded that defining a connection based on a relative threshold of the top 20% of ties was the best option [26]. Other studies evaluating prescriber networks defined connections based on episodes of care [56]. We found that episodes of care also had similar results to relative thresholds, suggesting that both definitions create valid prescriber networks.

Overall, our study found that Walktrap and Louvain algorithms underperformed compared to the modularity maximization algorithm. While there is no precedent in the prescriber networks literature that evaluated the use of different community detection algorithms, other complex network studies have evaluated differences in detection algorithms [57–61]. There is not a single algorithm that prevails as the optimal algorithm for community detection among different types of networks (i.e., small and large networks) and performance measures. For example, the Louvain algorithm provided the best fit in two studies [57,61], while the modularity maximization algorithm was superior in Berardo de Sousa et al [58]. Our modularity maximization method was different, such that we followed Stein et al’s approach of recursive application to find smaller communities. After the third iteration, we found the minimal increase in PPV resulted in larger decrease of sensitivity, suggesting that higher iterations are unnecessary. This aligns with Stein et al, as they found the best split for minimum community sizes with minimum number of total communities was 3.3, aligning with our results that second or third iteration was best [34].

Our study was not without limitations. First, as physicians treated clients at multiple OAT clinics in Ontario, the network-identified communities will necessarily result in a degree of misclassification as a physician can only be assigned to a single community. To address this concern, we used multiple definitions for physician-clinic connection. In other settings and provinces physicians often work in multiple clinics, and we found the algorithm captures true groupings from clinics with at least 15% of the physicians’ client-load. The algorithm may not be reliable at detecting clinics which exclusively or primarily employ prescribers with a small number of clients relative to the physicians’ overall client-load. Future analyses should employ other sensitivity analyses, such as different detection algorithms and connection definitions to ensure clinic identification is consistent. While physicians often work in multiple clinics, our methodology required prescribers to be assigned to only one clinic. Algorithms to detect overlapping communities within weighted networks exist, such as the Speaker-listener Label Propagation Algorithm [62], however they have not been used in prior prescriber network literature and are not widely implemented in software. Second, for the express purpose of defining OAT clinics, we focus our only on OAT prescription records. It is possible that some OAT physicians could also be working at non-OAT clinics and sharing non-OAT patients which we would not capture. The methodology can be extended in other applications to capture any prescriptions, or any records of care provision, between shared clients. We note however the unique benefits of using OAT prescription records, given requirements for frequent (weekly or biweekly) prescription renewals. We note that the method was validated to identify primary care offices in Texas using outpatient care records from US Medicare data [27], and communities within 51 different hospital referral regions using inpatient and outpatient care records from US Medicare data [26]. Elsewhere it has be implemented to identify hospital referral regions in England using hospital admission records to measure infection transfers between hospitals [63].

This analysis examined OAT prescriber network characteristics with the aim of validating the use of administrative data for identifying OAT prescriber networks. Identifying OAT prescriber networks from health administrative data can help determine the impact of physician networks on OAT outcomes and provide recommendations to improve collaborative care.

Supporting Information