https://orcid.org/0000-0002-4257-1982
Figures
Abstract
Introduction
Reasons for drug shortages are multi-factorial, and patients are greatly injured. So we needed to reduce the frequency and risk of drug shortages in hospitals. At present, the risk of drug shortages in medical institutions rarely used prediction models. To this end, we attempted to proactively predict the risk of drug shortages in hospital drug procurement to make further decisions or implement interventions.
Methods
We collated data obtained using the centralized procurement platform of Hebei Province and defined independent and dependent variables to be included in the model. The data were divided into a training set and a validation set according to 7:3. Univariate and multivariate logistic regression were used to determine independent risk factors, and discrimination (using the receiver operating characteristic curve), calibration (Hosmer-Lemeshow test), and decision curve analysis were validated.
Results
As a result, volume-based procurement, therapeutic class, dosage form, distribution firm, take orders, order date, and unit price were regarded as independent risk factors for drug shortages. In the training (AUC = 0.707) and validation (AUC = 0.688) sets, the nomogram exhibited a sufficient level of discrimination.
Citation: Dong J, Gao Y, Liu Y, Yang X (2023) Development and validation of the nomogram to predict the risk of hospital drug shortages: A prediction model. PLoS ONE 18(4): e0284528. https://doi.org/10.1371/journal.pone.0284528
Editor: Rosella Levaggi, University of Brescia: Universita degli Studi di Brescia, ITALY
Received: October 10, 2022; Accepted: April 3, 2023; Published: April 14, 2023
Copyright: © 2023 Dong et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because of Regulations of the Second Hospital of Hebei Medical University. Data are available from the Procurement Department of the Second Hospital of Hebei Medical University for researchers who meet the criteria for access to confidential data. The contact person of the Procurement Department, Xie Weiwei, has access to the data for this study. E-mail address: 731970096@qq.com.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
As for drug shortage, one definition given by the FDA is "a shortage occurring when demands exceed supply at any point in the supply chain, ultimately creating a ’stock-out’ at the point of appropriate service delivery to the patient if the cause of shortage cannot be resolved in a timely manner relative to the clinical needs of the patient" [1]. Drug shortage is a global problem [2]. There are many reasons for drug shortage, and supply and demand are the most common ones. For example, there is a shortage of drugs in various medical institutions [3, 4]. The specific situation is that after we submit the order, the distribution company cannot deliver some drugs to the hospital within the agreed time. There are many reasons for this, such as traffic problems [5, 6], etc. In this way, the potential risk of stock-outs is formed. Once the stock of drugs is exhausted, it will affect the patient. However, regardless of its reasons, drug shortage has a significant impact on patients as long as it occurs, including inadequate nursing, poor drug safety and treatment results, poor substitution effect or increased side effects, increased adverse events, prolonged hospitalization, increased patient pain, and even death [7–13]. The additional cost caused by the shortage of drugs is also huge as the drug procurement and shortage management impose hundreds of millions of dollars and nearly ten million hours of additional management time every year [14–17].
At the macro level, each country must take specific measures to manage drug shortages [18–20], for example, China’s volume-based procurement (VBP). It is a national policy that the state selects varieties from generic drugs that have passed the quality and efficacy consistency evaluation, "packs" the scattered procurement volume of medical institutions nationwide to form a large-scale group purchase effect, and conducts price negotiations with drug manufacturers at the national level. At the micro level, Although we have taken some measures, such as reporting the shortage, we still need an active strategy to optimize drug management, from passively accepting the shortage report to actively predicting drug shortages, including establishing a prediction model [21].
At present, the prediction related to drug stock-outs mainly uses time series analysis to predict the use of a drug in the future [22, 23]. However, this kind of prediction is generally used to predict the market potential of drugs. It focuses on the time factor; that is, the time series analysis method reflects the trend of a variable over time. Therefore, the model based on logistic regression is a more suitable method for predicting drug shortages. After the model is visualized, it can be more intuitively displayed using the nomogram. Although some studies [24] have reported that logistic regression-based models are used to predict the drug shortage risk in the pharmacy sector, some of these variables are difficult to obtain. In addition, these studies were not comprehensive enough to evaluate the model visualize it, or show its clinical utility. This paper aimed to establish and validate a prediction model based on multiple logistic regression. This model is used to predict the risk of hospital drug shortages. We used the nomogram to intuitively display the shortage probability of drugs and, finally, decide whether to adopt intervention measures.
Materials and methods
Drug shortages information and study design
This study was performed at The Second Hospital of Hebei Medical University and retrospectively obtained drug procurement data from 1 to 31 March 2022. The data of this study were obtained from the centralized procurement platform of Hebei Province, the weekly epidemic situation broadcast on hospital news, and on-site collection.
Data meeting the following criteria were included in the study: (a) drugs existing in the procurement platform; (b) complete data; and (c) drugs in the hospital drug catalog. The exclusion criteria were: (a) active pharmaceutical ingredients and (b) narcotic and psychotropic drugs.
This study aimed to establish a model to predict the probability of drug shortages. The drug information was divided into a training set and a validation set according to 7:3. The training set (n = 4836) was used to establish the model, and the validation set (n = 2077) was used to test the diagnostic performance of the model. Approval of the ethics committee of the hospital was not required for this study.
Data collection and operational definition
Information on drug procurement and shortage factors was obtained from the Hebei Province Central Procurement Platform, weekly epidemic broadcasts in the hospital news, and on-site survey at the receiving department. A total of 11 variables were evaluated. These variables included volume-based procurement (yes/no), therapeutic class, drug dosage form, distribution firm, order date, take orders (in time/not in time), unit price, COVID-19 (whether the manufacturer is in the COVID-19 epidemic area), manufacturer (foreign-funded/Chinese funded), controlled drug list (whether the drugs are in the list of hospital controlled drugs), and number of continuous stock-out times. Drug therapeutic classes were divided into antibacterial drugs, drugs of the respiratory system, hormone-electrolyte-vitamin (HEV) drugs, antineoplastic, immunomodulatory drugs, endocrine drugs, drugs of the nervous system, drugs of the digestive system, drugs of the blood system, drugs of the circulatory system, and others. Drug dosage forms were divided into injection, oral, and others. The distribution firms were divided into three echelons according to business volume. The first echelon consisted of Sinopharm holding Hebei Pharmaceutical and Le Ren Tang; the second echelon consisted of China Resources Hebei Medical University, China Resources YiSheng, Hebei LongHai, and Hebei General; and the third echelon was the other distribution companies. The purchasing and ordering time was from Monday to Friday. Taking orders in time was defined as the time of taking orders and the time of submitting orders on the same day. Drug prices entered the regression analyses in the form of quintiles. The information on whether the manufacturer was in the middle- or high-risk area according to the weekly report of the epidemic was collected. The number of continuous stock-outs of drugs was collected at the site where the drugs were stored. According to the data distribution, it was defined as zero time and > = one time. The outcome was defined as follows: it was shortages if the quantity of delivered drugs was lower than the order quantity and not if they were equal.
Statistical analysis
Categorical variables were expressed as percentages, and continuous variables were presented as medians. Univariate logistic regression analyses were performed in the training set to identify independent risk factors. The variables with P < 0.05 in the univariate analyses were analyzed in multivariate logistic regression analyses. The final model selection was performed using a backward step-down selection process with the Akaike information criterion (AIC). After that, a nomogram was constructed using the independent risk factors obtained from the multivariable logistic regression analyses. The receiver operating characteristic (ROC) curve was used to discriminate in the training and validation sets. The calibration curve was used to determine whether the predicted shortage probability of the nomogram was consistent with the actual stock-out probability and to analyze it with the Hosmer-Lemeshow test. The decision curve analysis (DCA) was used to estimate the net benefit and clinical utility of the nomogram. All statistical methods were performed using R version 4.1.2 (http://www.rproject.org) with pROC, calibrate, MASS, rms, foreign, and nricens data packages. P < 0.05 was considered statistically significant in all analyses.
Results
Drug shortages characteristics
A total of 6913 pieces of drug information were included in the research, of which 70% (n = 4836) were randomly assigned to the training set, and the remaining 30% (n = 2077) were assigned to the validation set. The selection of drug purchasing information is presented in Fig 1. The drug information characteristics of the training and validation sets are presented in Table 1.
Univariate and multivariate analysis
Univariate analysis showed that volume-based procurement (P = 0.000), therapeutic class (P = 0.000), dosage form (P = 0.000), controlled drug list (P = 0.000), manufacturer (P = 0.009), distribution firm (P = 0.000), take orders (P = 0.000), order date (P = 0.000), and unit price (P = 0.000) were significant risk factors for drug shortages. Related risk factors with P < 0.05 in univariate analyses were adjusted for the multivariate analysis. After the multivariate analysis, only volume-based procurement (P = 0.000), therapeutic class (P = 0.000), dosage form (P = 0.000), distribution firm (P = 0.000), take orders (P = 0.000), order date (P = 0.000), and unit price (P = 0.000) were still independent risk factors for drug shortages (Table 2).
Construction and validation of the nomogram
According to the results of multivariate analysis, the nomogram for drug stock-outs was established (Fig 2). We identified the score of each independent factor and the total score according to the point scale at the top of the nomogram. Then, we predicted the corresponding stock-out probability according to the total score and the point scale at the bottom of the nomogram. Our analyses showed that the model had a sufficient level of discrimination, with an area under the curve (AUC) of 0.707 (95% CI = 0.686–0.728) in the training set (Fig 3A). AUC in the validation set was 0.688 (95% CI = 0.655–0.722) (Fig 3B). In addition, the calibration curve of the nomogram demonstrated good agreement between the actual and predicted drug stock-out probabilities (Fig 4), and the Hosmer-Lemeshow text showed P > 0.05. We used DCA to evaluate the clinical utility and net benefit of the drug stock-out nomogram to further evaluate its potential application worth. These results indicate that more clinical net benefits could be obtained for both training and validation sets (Fig 5). The plot showed that, for predicted probability thresholds between 4 and 29%, model-based decisions had a more net benefit than either the non-interventions or the interventions. The risk factor scores in the nomogram are shown in Table 3. The risk of drug stock-outs was < 10% if the total score was < 200and > 60% if the total score was > 372.
The ROC analyses for the predictive model in training set(A) and validation set(B).
The calibration curve indicates good consistency between training set(A) and validation set(B).
Decision curve analyses for the drug stock-outs nomogram in training set(A) and validation set(B).
Discussion
We established and validated a nomogram in this study for predicting the risk of shortages. Our findings showed that the nomogram had favorable discrimination, calibration, and a certain clinical net benefit. Seven predictors were included in the nomogram: volume-based procurement, therapeutic class, drug dosage form, distribution firm, take orders, order date, and unit price. Among them, drug dosage forms and therapeutic classes are consistent with previous reports [24].
Among these predictors, drugs without volume-based procurement were more likely to be shortages than those with volume-based procurement (OR = 0.43). This shows the advantages of volume-based procurement, which can effectively reduce the risk of drug stock-outs [25]. In the drug therapeutic class, antibiotics (reference), antineoplastic (OR = 0.98, P = 0.919), immunomodulatory drugs (OR = 0.60, P = 0.108), and endocrine drugs (OR = 0.91, P = 0.733) were more likely to be shortages than other types of drugs. Among them, antibiotics and antineoplastics were often in short supply worldwide [26–28]. Other studies had shown that endocrine drugs were also among the drugs that were lacking locally [29]. Therefore, we suggest that special attention should be paid to the above four categories of medicines, assessing them before hospital introduction and making contingency plans to deal with stock-outs after hospital introduction. The risk of shortage of other dosage forms (atomized and external drug dosage forms) had a higher risk of shortage (OR = 1.71) than injection and oral agents, which might vary from place to place [30]. Compared with the distribution firms in the first echelon, the distribution firms in the second echelon (OR = 1.66) and others (OR = 1.44) were more prone to the risk of shortage. It might be related to the size of the company and its management strategy. Once it was found in the procurement platform that the distribution firm does not receive orders in time, the risk of shortage will almost double (OR = 1.95). The distribution companies in the first echelon had the lowest risk of shortage, which was consistent with other studies [31]. From this perspective, we should select some large distribution companies (the first echelon) when purchasing drugs, and pay attention to the order information at any time, inquire about the situation, and temporary purchase the drug product from another distribution company if we found that the order was not received on time. In this way, we could minimize the shortage as much as possible. Compared with ordering on Monday, the risk of ordering on Tuesday and Thursday is lower, and the OR was 0.75 and 0.43, respectively. The risk of shortages increased when ordering on Wednesday (OR = 1.28). However, there was no difference in the risk when ordering on Friday (P = 0.661). The risk of shortage was the lowest on Tuesday and Thursday, followed by Monday and Friday, and the risk of shortage was the highest on Wednesday. According to the above results, we should try our best to adjust the order time or set up the safety stock to optimize hospital drug management. The unit price of drugs was grouped by quintiles. Compared with the first quintile, the second, fourth, and fifth quintiles were less prone to the risk of shortage, and the OR was 0.68, 0.64,0.41, respectively. In comparison, there was no difference in the third quintile (P = 0.059). This means that the higher the price, the less likely it was to be stock-outs [32]. When the drug price is at a low level, it means that the manufacturer should reduce the cost by minimizing the investment [33]. The risk of the drug shortages greatly increases once the price is close to or lower than the cost. Therefore, when drug procurement pays excessive attention to price, the ensuing supply problems should be considered [34].
Some of the factors included in our study were not independent risk factors for drug shortages. The results showed that the COVID-19 factors were not an independent risk factor for drug shortages. The possible reason was that China’s epidemic prevention and control policy was very effective, and China’s strong economic resilience ensured the smooth distribution of drugs [35]. Also, the list of hospital-controlled drugs was not a risk factor for drug shortages. The reason might be that the purpose of formulating the list of hospital-controlled drugs was to prevent the abuse of auxiliary drugs; thus, there was no apparent correlation between drug shortages, even if some distribution firms considered the sales quota of auxiliary drugs and reduced the willingness to deliver. However, the analysis results showed no statistical significance in the risk of shortage of drugs in the controlled drug list compared with drugs outside the list. The study also found no statistical significance in the probability of shortage for both Chinese and foreign-funded drugs. This is inconsistent with some literature reports [24]. The reason may be that the national conditions of each country are different or the volume-based procurement makes it possible to obtain sufficient profit without operation to ensure supply.
In general, using this model may help identify high-risk drugs and the shortage risk per hospital purchase. This can prepare for the next step of planning or implementation of interventions. For example, when updating the hospital drug supply catalog, it can be used to identify high-risk drugs or plan for drug shortages when purchasing. It should also be attempted to reduce the risk of shortage before drugs enter the hospital or undergo the procurement process. In addition, we not only validated the discrimination of the model but also the calibration as well as the decision curve analyses. The model performed well in these validations. Finally, we obtained a nomogram, which could help us easily get a score for a drug as well as the risk of shortage and allows us to quickly evaluate two drugs with different information on the same name. Thus, our model still has limitations. The included independent variables are still not comprehensive. This is mainly because some of these unincorporated independent variables are difficult to collect or quantify, such as the occurrence of the domino effect [36]. Other independent variables cannot be obtained due to the business secrets of the manufacturer, such as raw materials, production line transformation, and whether to expand production. Changes in drug prices can also cause changes in the risk of drug shortage [32]. Therefore, in the future, the consideration of drug price fluctuations and other factors that have not been noticed should be added and included in the prediction model.
As we used retrospective data for modeling, we can use cohort data to continuously test and revise the model in the future. We can also use other medical institutions for external validation to modify the prediction model to achieve better discrimination. In the present study, we used data on hospital drug purchase to establish a prediction model with good prediction ability. This model includes a total of seven variables, all of which are related to hospital drug procurement. It contains some factors reported in the literature and factors not noticed before. After verification, the model performs well in discrimination, calibration, and clinical applicability. Using this model, we can estimate the out-of-stock risk of drugs and decide what intervention measures we need to take. This reflects the initiative of out-of-stock management of hospital drugs. Even in the future, we still need to validate, improve, and expand this model constantly, but the potential of using data modeling for drug shortage management is huge.
Conclusion
In summary, we have successfully developed and verified a nomogram for predicting the risk of drug shortage based on drug characteristics. The nomogram shows good diagnostic performance, which is very helpful for the hospital to select a drug that is not often out of stock.
Acknowledgments
We thank Weiwei Xie and Xikun Wu for their comments on this writing, thank the Department of Pharmacy of The Second Hospital of Hebei Medical University for the support.
References
- 1. Acosta A., Vanegas E. P., Rovira J., et al. Medicine Shortages: Gaps between Countries and Global Perspectives. Front. Pharmacol. 2019; 10: 763. pmid:31379565
- 2. Shukar S, Zahoor F, Hayat K, et al. Drug Shortage: Causes, Impact, and Mitigation Strategies. Front Pharmacol. 2021; 12: 693426. pmid:34305603
- 3. Donnelly Katie A, Zocchi Mark S, Katy Tamara A, et al. Prescription Drug Shortages: Pediatric Emergency and Critical Care Medications. Pediatr Emerg Care. 2021; 37(11):e726–e731. pmid:30829846
- 4. Darija Kuruc Poje Domagoj Kifer, Huys Isabelle, et al. Patients perspectives on drug shortages in six European hospital settings—a cross sectional study. BMC Health Serv Res. 2021; 21(1): 689. pmid:34253212
- 5. Dill Susie, Ahn Jin. Drug Shortages in Developed Countries-Reasons, Therapeutic Consequences, and Handling. Eur. J. Clin. Pharmacol. 2014; 70(12), 1405–1412. pmid:25228250
- 6. Mazer-Amirshahi M., Pourmand A., Singer S., Pines J. M., and Van Den Anker J. Critical Drug Shortages: Implications for Emergency Medicine. Acad. Emerg. Med. 2014, 21(6), 704–711. pmid:25039558
- 7. Mclaughlin M., Kotis D., Thomson K., et al. Effects on Patient Care Caused by Drug Shortages: a Survey. Jmcp 2013; 19(9): 783–788. pmid:24156647
- 8. Rider A. E., Templet D. J., Daley M. J., et al. Clinical Dilemmas and a Review of Strategies to Manage Drug Shortages. J. Pharm. Pract. 2013; 26(3): 183–191. pmid:23553544
- 9. Blaine K. P., Press C., Lau K., et al. Comparative Effectiveness of Epsilon-Aminocaproic Acid and Tranexamic Acid on Postoperative Bleeding Following Cardiac Surgery during a National Medication Shortage. J. Clin. Anesth. 2016; 35: 516–523. pmid:27871586
- 10. Hughes Kaitlin M, Elizabeth SGoswami, Morris Jennifer L. Impact of a Drug Shortage on Medication Errors and Clinical Outcomes in the Pediatric Intensive Care Unit. J Pediatr Pharmacol Ther. 2015; 20(6): 453–61. pmid:26766934
- 11. Bible Jaimee R, Evans David C, Brett Payne, et al. Impact of drug shortages on patients receiving parenteral nutrition after laparotomy. JPEN J Parenter Enteral Nutr. 2014; 38(2 Suppl): 65S–71S. pmid:25239113
- 12. Haider A., Qian Y., Lu Z., et al. Implications of the Parenteral Opioid Shortage for Prescription Patterns and Pain Control Among Hospitalized Patients with Cancer Referred to Palliative Care. JAMA Oncol. 2019; 5(6): 841–846. pmid:30896771
- 13. Phuong J. M., Penm J., Chaar B., et al. The Impacts of Medication Shortages on Patient Outcomes: a Scoping Review. PloS one 2019; 14(5), e0215837. pmid:31050671
- 14. Fox ER. ASHP guidelines on managing drug shortages. Am J Health-Syst Pharm. 2018;75:e593–e601.
- 15. Ventola CL. The drug shortage crisis in the United States: causes, impact, and management strategies. P T. 2011;36(11):740–757. pmid:22346307
- 16. Paavola A. Drug shortages cost hospitals $359M in labor costs, survey finds. Becker’s Hospital Review. Published June 26, 2019. Accessed June 1, 2020. https://www.beckershospitalreview.com/pharmacy/drug-shortages-cost-hospitals-359m-in-labor-costs-survey-finds.html
- 17. Kaakeh R, Sweet B, Reilly C, et al. Impact of drug shortages on U.S. health systems. Am J Health-Syst Pharm. 2011;68(19):1811–1819. pmid:21930639
- 18. Yang C., Wu L., Cai W., et al. Current Situation, Determinants, and Solutions to Drug Shortages in Shanxi Province, China: a Qualitative Study. PloS one 2016; 11(10), e0165183.
- 19. Walker J., Chaar B. B., Vera N., et al. Medicine Shortages in Fiji: A Qualitative Exploration of Stakeholders’ Views. PloS one 2017; 12(6), e0178429. pmid:28582409
- 20. Atif M., Malik I., Mushtaq I., et al. Medicines Shortages in Pakistan: a Qualitative Study to Explore Current Situation, Reasons and Possible Solutions to Overcome the Barriers. BMJ open 2019; 9(9): e027028. pmid:31488466
- 21. Pearson Thomas A, Califf Robert M, Rebecca Roper, et al. Precision Health Analytics With Predictive Analytics and Implementation Research: JACC State-of-the-Art Review. J Am Coll Cardiol. 2020; 76(3): 306–320. pmid:32674794
- 22. Fenna Jennifer, Chu Cherry, Hassan Rola, et al. Extent of a valsartan drug shortage and its effect on antihypertensive drug use in the Canadian population: a national cross-sectional study. CMAJ Open. 2021; 9(4): E1128–E1133. pmid:34876414
- 23. Brokenshire Samantha A, Lemon Stephen J, Benjamin Staley, et al. Impact of Opioid Restrictions During a Critical Drug Shortage Period: Interrupted Time Series for Institutional Opioid Utilization. Pain Med. 2021; 22(1): 203–211. pmid:32875327
- 24. Liu I, Colmenares E, Tak C, et al. Development and validation of a predictive model to predict and manage drug shortages. Am J Health Syst Pharm. 2021; 78(14): 1309–1316. pmid:33821926
- 25. Seidman Gabriel, Atun Rifat. Do changes to supply chains and procurement processes yield cost savings and improve availability of pharmaceuticals, vaccines or health products? A systematic review of evidence from low-income and middle-income countries. BMJ Glob Health. 2017; 2(2): e000243. pmid:28589028
- 26. M Videau D Lebel , Bussières J-F. Drug shortages in Canada: Data for 2016–2017 and perspectives on the problem. Ann Pharm Fr. 2019; 77(3): 205–211.
- 27. Omer Sumaira, Pan Mengyuan, Ali Salamat, et al. Perceptions of pharmacists towards drug shortages in the healthcare system of Pakistan and its impact on patient care: findings from a cross-sectional survey. BMJ Open. 2021; 11(12): e050196. pmid:34949612
- 28. Pauwels Kim, Simoens Steven, Casteels Minne, et al. Insights into European drug shortages: a survey of hospital pharmacists. PLoS One. 2015; 10(3): e0119322. pmid:25775406
- 29. Yazed Sulaiman AlRuthia Hadeel AlKofide, Refaa’h AlAjmi, et al. Drug shortages in large hospitals in Riyadh: a cross-sectional study. ANNALS OF SAUDI MEDICINE. 2017 Sep-Oct;37(5):375–385. pmid:28988252
- 30. Zhang Wei, Daphne P Guh Huiying Sun, et al. Factors associated with drug shortages in Canada: a retrospective cohort study. CMAJ Open. 2020; 8(3): E535–E544. pmid:32873582
- 31. Helen M Parsons Susanne Schmidt, Karnad Anand B, et al. Association Between the Number of Suppliers for Critical Antineoplastics and Drug Shortages: Implications for Future Drug Shortages and Treatment. Journal of Oncology Practice. 2016 Mar;12(3):e289–98, 249–50. pmid:26837565
- 32. Chintan V Dave Ajinkya Pawar, Fox Erin R, et al. Predictors of Drug Shortages and Association with Generic Drug Prices: A Retrospective Cohort Study. Value Health. 2018; 21(11): 1286–1290. pmid:30442275
- 33. J Woodcock M Wosinska. Economic and technological drivers of generic sterile injectable drug shortages. Clin Pharmacol Ther. 2013; 93(2): 170–6.
- 34. XIE Jin-ping , ZHNAG Xue-rui, ZHANG He, et al. Study on Drug Purchasing Mode of Typical Developed Countries and its Enlightenment. Health Economics Research. 2022,39(01): 64–68.
- 35. LIU Juan TANG Jiafu , LIU Jiang. An empirical study on the impact of COVID-19 pandemic prevention and control policies on the high-quality development of express delivery industry. Systems Engineering-Theory & Practice. 2022, Vol. 42, Issue (3): 651–663.
- 36. Fox ER, Sweet BV, Jensen V. Drug shortages: a complex health care crisis. Mayo Clin Proc. 2014; 89(3): 361–373. pmid:24582195