https://orcid.org/0000-0002-9652-8293
Figures
Abstract
Innovating in Medical Device (MD) industry is challenging. This study aims to develop and validate an evidence-based framework that helps innovators of small and large enterprises (SEs and LEs) assess their readiness for successful MD development and deployment. We conducted a key-informant process (stage 1) where 25 international experts identified a list of emergent Health Technology Assessment (HTA) themes they believed were essential to company success. A sample of 22 European and US selected companies (13 SEs and 9 LEs) then reached a consensus on a list of key themes through a robust Delphi process (stage 2). Finally, we constructed (stage 3) and validated (stage 4) the checklist for SEs and LEs. The checklist for SEs and LEs included 21 and 15 items (i.e., fundamental Yes/No questions) with nine overlapping criteria for both SEs and LEs. In both groups, MD success was driven by three major item categories: (i) R&D assessment strategy; (ii) device-outcome measures; (iii) company profiling. Alongside the retrospective validation study, we collected 40 case studies on MDs (23 successes and 17 failures) across the selected enterprises. The retrospective validation provided the proportion of successful and failed case studies that met the ’MeDKET’ criteria. We discovered that early HTA plays a pivotal role in MD industry success with different implications based on enterprise size. This study is the first of its kind to provide a holistic picture of the perceived role of early-stage HTA in MD industry success.
Citation: Manetti S, Lettieri E, Ni MZ (2023) Development and validation of Medical Device Key Evidence Tool (‘MeDKET’): An evidence-based framework to explain success in selected European and US companies. PLoS ONE 18(7): e0288126. https://doi.org/10.1371/journal.pone.0288126
Editor: Simone Borsci, Universiteit Twente, NETHERLANDS
Received: April 23, 2022; Accepted: June 20, 2023; Published: July 13, 2023
Copyright: © 2023 Manetti et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: The English National Institute for Health Research Medtech and In vitro Diagnostics Co-operative (MICs) program funded the research. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The Research and Development (R&D) process in Medical Device (MD) industry is complex, expensive, and has plenty of pitfalls. This process is more multifaceted than the innovation continuum in other industries or sectors [1]. The unique complexity results from several factors: (i) the pressure from different stakeholders; (ii) the interface of many disciplines; (iii) the need for coordination between product development and clinical use through user involvement; (iv) the strong ethical implications; (v) the regulatory constraints [2, 3]. In this way, each R&D phase results sensitive to external constraints (i.e., clinical, legislative, social, ethical etc.), generating an interdependency among the different development phases than in other industries, including the pharmaceutical sector [4]. These dynamics create an accentuated need for feedback, continuous evidence generation and analysis alongside R&D using evaluation and audits [5].
Fewer than 6% of MDs reach the market each year after many years of development efforts and high costs sustained by companies, governments, and R&D investors, which could fund cost-ineffective technologies that never achieve implementation [4]. The development process in the MD industry may, therefore, require large amounts of time and resources but result in limited benefits for companies, governments, and -ultimately—society. The negative consequences also affect the whole society, which loses opportunities for healthcare improvements and maximising the return from the biomedical innovation efforts [5–7].
The specificities of the MD industry generate a particular case in the market access of new MDs due to the highly regulated aspect of the industry and the interdependencies of each new product with existing frameworks, products, and systems in clinical use [5, 8]. Indeed, the strictly regulated feature of the MD industry is similar to the automotive or nuclear industries, which are dynamic areas with permanent changes and a short lifetime of products [9]. Manufacturers have many regulatory constraints to satisfy for market approval, and these obligations continue even after selling the device, namely post-market surveillance and adverse event reporting [9]. Moreover, the barriers to commercialisation are related to technological (e.g., short life cycle), human-factor (e.g., user acceptability and adoption), and organisational challenges (e.g., existing alternatives) [4, 10, 11].
Technology producers should understand that the MD industry’s traditional definition of product success is evolving. New criteria and requirements for achieving success push relevant stakeholders involved in the R&D process (i.e., company, academia, and government) to assess innovative products as earlier as possible. The previous factors, combined with increasing healthcare budget constraints and reimbursement schemes, threaten the MD industry’s future, which is globally in accelerated and compulsory growth [12, 13].
Early-stage HTA is an emerging ’species’ of HTA in which technologies in development are evaluated to support healthcare decision-making from the initial idea up to phase III-like trials, anticipating market access and reimbursement [14, 15]. The rationale behind early-stage HTA is to inform internal investment decisions, select potential products or prototypes to take forward and avoid investments in new technologies that are less likely to succeed. In a later stage, these analyses can add value and efficiency to the road to reimbursement and coverage decisions. Early-stage HTA should make effective and affordable technology readily available to patients to maximise the benefits of biomedical R&D efforts for the public and society. The academic literature provides few reference guidelines for early-stage HTA [6, 16–19]. A critical factor for non-progression decisions at NICE Medical Technologies Evaluation Programme (MTEP) during its first three years (2010–2013) appears to be the little or no attention to an early economic evaluation of the new device by the manufacturer [20].
This exploratory study aims to develop and field-test an evidence-based framework to understand whether and to what extent early-stage HTA might influence the product success of small and large enterprises (SEs and LEs) from a company perspective. Incorporating early-stage HTA into R&D and company decision-making might be beneficial. However, evidence remains sparse on whether (and how) the MD industry conducts early-stage HTA. The implications of conducting early-stage HTA on future (post-market) product success must be made clear and straightforward. Our implicit aim is, therefore, to identify the drivers of MD industry success.
Materials & methods
We applied a multi-method approach, which was successfully used in producing the ’POCKET’ toolkit [21] for developing an evidence-based framework for point-of-care testing devices.
As outlined in Fig 1, this research encompassed four stages. We conducted a key-informant process (stage 1) where 25 international experts identified a list of emergent HTA themes pointed out as critical to company success. A sample of 22 European and US companies then reached a consensus on a list of key themes through a robust Delphi process (stage 2). In stage 3, we constructed the ’MEDKET’ checklist for SEs and LEs by defining and prioritising key themes using comments and ratings from stages 1 and 2. Finally, in stage 4, we field-tested ’MEDKET’ by collecting 40 case studies across the selected enterprises.
Key-informant process
A three-step key-informant process was addressed to international experts and methodologists to identify a list of emergent themes that explained the MD company’s success.
Preliminary dialogues (Step 1).
Initial discussions at Karolinska Institutet involved senior academics in healthcare innovation management and informed the development of essential themes covered in the next step.
Semi-structured interview study (Step 2).
We conducted 25 semi-structured interviews by phone or in person with key informants (15 academics and ten industry professionals with experience-based or professional-based knowledge in the design, development, and deployment of MDs). The expert academics (2 from Italy and 13 from the UK) were identified by academic publishing output; some also worked as freelance consultants for the MD industry. The industry experts (6 from Italy and four from the UK) were well-known professionals working in spin-off companies based on our academic institutions. We performed a general thematic analysis of the interviews to extract emergent themes.
Questionnaire piloting (Step 3).
We translated the list of emergent themes, identified by the key informants, into a series of statements to form a survey questionnaire (Manetti et al., 2017). We piloted the questionnaire with the academic staff of the Health Economics Research Centre (HERC, University of Oxford) and re-piloted the revised version a month later. The aim was to make the easily understandable questions in case of no scientific background in biomedical technology innovation/early-stage HTA.
Delphi process
A robust Delphi process with skilled professionals working in the MD industry aimed to reach a consensus on theme relevance.
Company selection (step 1).
We selected the Delphi participants at both company and professional levels. Inclusion criteria for SEs and LEs were as follows: (i) intense R&D process; (ii) high specialisation; (iii) international partners. We made every effort to include relevant experience or expertise existing outside Italy. We set a minimum of eight participants for each group (i.e., SEs and LEs) as a pragmatic sample size, given time and resources (Huddy et al., 2021).
Inclusion criteria for practitioners were as follows: (i) (at least) four years of experience within the company; (ii) a comprehensive view of R&D and market access (based on previous or current professional experience); (iii) accurate representation of the company needs and aims. We reached the leaders or CEOs of each firm by invitation included in their trade association newsletter (i.e., ASSOBIOMEDICA for Italy) or through direct contacts. After the approval of the CEO of each firm, we interviewed the CEO or the persons indicated by the CEO. We excluded six SEs after preliminary informal talks with the management team because the companies were not interested in this research and three LEs after their CEOs and other senior professionals were unavailable.
Results drove the final number of Delphi participants, as more interviews were carried out until no new themes emerged. Tables 1 and 2 show the sample for SEs (13 enterprises and 13 professionals) and for LEs (9 firms and 13 practitioners). The interviewees in the SE group gained 11.9±7.8 years of professional experience, whereas the practitioners in the LE group accumulated 13.2±8.7 years.
Semi-structured interview study (step 2).
The semi-structured interview study aimed to identify: (i) the degree of relevance of each HTA dimension in each R&D phase; (ii) starting time, approaches and perspectives used in early economic evaluation; (iii) barriers to market access and adoption of MDs; (iv) the costs and benefits of conducting early-stage HTA during R&D; (v) the perceived role of early-stage HTA in MD industry success.
We conducted 13 semi-structured interviews in the SE and 13 in the LE groups. Indeed, out of 9, 3 selected companies indicated more than one proper person. Out of 26, 15 interviews (12 in Italy and 3 in Finland) were conducted face-to-face by one interviewer (SM), whereas the remaining were via phone or "Skype". The interview structure resulted from the key-informant process (stage 1). It was further refined in conjunction with an expert in mixed research (MN) to allow the collection of both quantitative and qualitative data while consenting to flexibility and original suggestions. Interviews lasted for approximately 60 min, were digitally recorded, and then transcribed for analysis. The 12 interviews conducted in Italy were in Italian and translated into English. Two authors (SM and MN) co-analysed the data independently to minimise biases. Initially, interview data were coded based on predetermined themes; additional themes emerged and were coded during the interview analyses. In four cases, participants were contacted to clarify the interpretation of data. Data were analysed with NVivo V.10.1.1 software (QSR International, Melbourne, Victoria, Australia).
Delphi survey (step 3).
The Delphi questionnaire has been extensively described elsewhere [22]. The questionnaire statements were built on a five-point Likert scale (from 1 (strongly disagree) to 5 (strongly agree)) concerning theme relevance. Responders could also add free text comments alongside each statement if they felt it was needed to justify responses or suggest further new statements. Any new statements presented in free-text answers were iteratively added to the Delphi questionnaire. This iterative method allowed participants to reconsider their responses regarding the group results. This process was repeated until a consensus was reached. The consensus was set at ≥80% across statements that received ≥4 (agree or strongly agree).
The survey was administered online using a web method (Survey Monkey www.surveymonkey.com). In a few cases, the questionnaire was administered by phone; the score was assigned using the qualitative answers of the experts. Survey data were analysed using Stata 15 software for Windows (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC), and descriptive analyses were carried out (frequencies, median scores, range of scores).
Framework development
The final stage aimed to construct the ’MEDKET’ checklist for SEs and LEs. The framework development was informed by the intermediate outputs from the previous stages, i.e., the list of emergent HTA themes (S1 Table) and the list of key themes agreed (S2 Table).
A separate analysis of small vs large firms (step 1).
We defined and prioritised key themes separately for small and large firms using comments and ratings from previous stages.
Conversion of themes into framework items (step 2).
First, we classified each prioritised theme into three major categories: (i) R&D processes; (ii) device outcome-measure; (iii) structure, which included structural peculiarities of the company and reference market segment. We organised the prioritised themes into two groups of relevance (i.e., primary and secondary) using comments and ratings from stages 1 and 2 combined with our interpretation. At last, we revised and edited the prioritised themes to ease the use of the checklist. We indifferently use the term’ item’ or ’driver’ to refer to the framework items.
Ethics statement
The involvement of the Ethics Committee of Sant’Anna School of Advanced Studies was unnecessary, as the study recruited expert academics in HTA and MD developers involved in a qualitative study. There was no patient contact or involvement, and no access to patient data and medical records was necessary. The study was conducted by the EU General Data Protection Regulation EU-2016/679 (GDPR). Academic and industry experts were invited to participate through an information letter describing the study’s aims and methodology, and participants signed a written consensus. Data were stored on a protected server and collected anonymously.
Results
Definition of medical device success
We found out that SEs and LEs perceived success differently. Indeed, SEs perceived success as business continuity, whereas LEs identified success as large-scale utilisation and value experimented by patients and target users. The different perceptions of MD success led us to elaborate on two separate definitions based on enterprises’ size (Table 3).
The definitions of success as business continuity for SEs and market share for LEs involve measurable indicators that are relatively easy to obtain. The second definition of success for LEs, i.e., success perceived as achieving incremental value for patients and end-users, can be more challenging to quantify. "We are very driven by patient value and whatever enables us to provide more value to the patients. That is a success. Patient outcomes are a success. No economic value, no simple technology development. If a surgeon finds value in using the device. That is a success." CEO, large company]
The ’MEDKET’ framework
Table 4 shows the ’MEDKET’ checklist combined for SEs and LEs, highlighting the overlaps and the differences between the two company groups.
’MEDKET’ for SEs and LEs included 21 (13 primary and eight secondary) and 15 (12 primary and three secondary) items, with nine overlapping themes for both SEs and LEs. In both groups, success was driven by three major item categories: (i) R&D processes (e.g., starting time of assessment activities and identification of funding needs); (ii) device outcome measures (e.g., economic sustainability); (iii) structural peculiarities of the company (e.g., business model) and the specific market (e.g., being an incumbent in the reference market segment). Most ’MEDKET’ drivers belonged to the first item category (10 and 7 drivers for SEs and LEs, respectively).
Field-testing of ’MEDKET’
The field-testing of the MEDKET checklist through 40 retrospective case studies collected from a selected sample of European and US small and large enterprises showed that primary and secondary framework items might drive MD success. Successful case studies had, on average, a more significant proportion of positive responses (i.e., Yes answers) on the ’MEDKET’ checklist compared with failed case studies. Similarities and differences between the successful and the failed case studies also emerged regarding the target MD (e.g., the purpose of use, risk class) and its innovation degree. Further information on the MDs assessed in the 40 case studies and successes/failures collected across case studies are detailed in S3 Table and S3 Appendix, respectively.
Table 5 presents how SEs performed on the ’MEDKET’ checklist by reporting the proportion of successful and failed case studies that met the related ’MEDKET’ criterion. The SE group gathered 12 MD successes and 8 MD failures. All the failures occurred before or very close to launching (i.e., TRL≥6). The starting time of early economic evaluations for the 20 case studies collected in the SE group is shown in S4 Appendix.
Table 6 details the field-testing results by reporting the Percentage of successful and failed case studies that met the related ’MEDKET’ criterion. The LE group gathered 11 MD successes and 9 MD failures. Among the failed cases, 5 were post-market failures (i.e., TRL = 9), and four occurred before launch or very close to launching (i.e., TRL≥6).
Discussions
In this exploratory study, we investigated the level of early-stage HTA incorporation into R&D and company decision-making using a four-stage mixed-method approach. The research culminated in a multiple case study that led us to obtain the external validation of the ’MEDKET’ framework. Moreover, we explored to what extent conducting early-stage HTA alongside biomedical R&D might drive future (post-market) product success by gathering retrospective case studies of successful and failed MDs from a selected sample of European and US small and large enterprises. The perceived role of early-stage HTA could have different implications on achieving success according to the enterprise size. Our study suggests original insights and reflections on the perceived role of early-stage HTA in achieving success for SEs and LEs.
Our study confirms evidence from the literature concerning the role of early-stage HTA in SEs. Small companies do not perform pre-market or early-stage analyses consistent with an early-stage HTA approach. This is not only due to the need for more resources and opportunities to conduct pre-market HTA but also to a specific knowledge-gap problem (i.e., skills, awareness, mentality). Many developers need to be made aware of health economics concepts, human-factor aspects, and market challenges, as they appear driven mainly by technological value and pioneer spirit. After several years of development efforts, SE developers realise they have to deal with the challenge of financial sustainability. However, last-minute adjustments are not possible at later stages when the device has already reached a high level of technology maturity. A "last-minute" approach to HTA may increase the risk of product failure and, for mono-product companies and start-ups, the risk of company failure. One company has failed during the conduction of the present study.
Our study sheds light on the perceived role of early-stage HTA in large companies to achieve post-market success. Our study provides evidence of the fact that LEs perform R&D analyses consistently with an early-stage HTA strategy, as conducting pre-market or early-stage HTA analyses can: (i) increase the overall R&D efficiency and efficacy; (ii) anticipate the interruption of devices in development that are likely to be unsuccessful in later R&D phases (failing fast, failing cheap); (iii) approach the R&D process more systematically while increasing the probability of future (post-market) product success. This evidence is well documented in the literature [7, 10, 14, 15, 23]. Third, investing the proper resources (i.e., funding, time, staff) to iteratively conduct early-stage HTA alongside R&D has been deemed necessary to reach post-market success. However, analysis quality, iterations’ number, and the choice of assessment dimensions represent a trade-off between investments and risks in large companies. During the Delphi study, experts acknowledged the existence of an ’optimum’ threshold of ’how’ to conduct early-stage HTA in each TRL, beyond which the allocation of additional resources can overcome the expected benefits for the company.
Moreover, LEs consider making early-stage HTA analyses more formalised and structured as unnecessary. The last point may explain the limited availability of case studies on early-stage HTA of innovative MDs in the published literature [24, 25]. Fourth, the perceived role of HTA in MD industry success changes between the pre-market and post-market phases. Early-stage HTA has been identified as a driver of future (post-market) product success. In contrast, the perceptions of mainstream or post-market HTA are controversial, also considering the lack of globally agreed rules on what and how constitutes evidence on new or novel MDs that will reach the market [3]. Mainstream HTA has been felt as a ’necessary evil’ whose negative results can affect market perceptions and MD success. However, the influence of positive impacts on MD reimbursement and adoption remains uncertain and unclear.
Our findings led us to draw a series of recommendations targeted to the crucial actors of the biomedical ecosystem (i.e., industry, academia, and governments).
Anticipating financial sustainability, health economics, and human factors alongside a biomedical technology’s lifecycle is essential for surviving the market. SEs should urgently adopt early-stage HTA to develop MDs more likely to meet the market’s expectations incorporating human-factor and financial requirements from the early stages of concept, design, and development. Furthermore, systematically integrating these requirements into the R&D process of innovative MDs might build more confidence in potential investors, ease searching for funding, and guarantee company survival and market continuum.
LEs should address the ’optimum’ threshold of ’how’ conducting early-stage HTA by adopting a ’prioritisation’ approach that maps the TRL reached by the product with the assessment dimension(s) that is(are) priority (e.g., pricing in TRL 6). Both SEs and LEs should increase and promote collaborations with academia, which can have a strategic role in conducting and disseminating early-stage HTA analyses. Finally, governments should provide comprehensive governance and leadership to define globally agreed rules on what and how constitutes reliable evidence for market approval and regulatory process of new or novel MDs to increase data sharing and public health transparency.
This research does present strengths and limitations. We highlight that the study was conducted using a company perspective. Thus, a mutual understanding of the implications associated with the perceived role of early-stage HTA as a driver of MD industry success needs to be further understood across other relevant stakeholders (i.e., policymakers, reimbursement bodies, research centres, and healthcare providers) with purposes of alignment. Another limitation concerns the adoption of a small sample size of participants that prevents us from drawing any conclusion regarding geographical variations. Whether (and how) the role of early-stage HTA in MD industry success varies considerably across countries is a critical consideration that needs further investigation. Finally, many study design limitations are typical to qualitative research, especially the Delphi process, which may lead to a compromise position rather than true consensus and a small sample size with unpredictable representation. Given the time and resources of this exploratory exercise, a certain degree of convenience sampling was required, as a random sampling approach was not feasible. Among the study strengths, we highlight (i) the use of a multi-method approach; (ii) a fruitful collaboration between academia and industry; (iii) the collection of multidimensional data on MD successes and failures by facing sparse and fragmented evidence characterising this emerging sector. In this sense, our study is the first to provide a holistic picture of the perceived role of early-stage HTAs in MD industry success.
Conclusions
In this exploratory study, we investigated the level of early-stage HTA incorporation into MD R&D and company decision-making. Despite some limitations common to qualitative research, this study is the first to provide a holistic picture of the perceived role of early-stage HTAs in MD industry success. Our findings led us to draw relevant recommendations addressed to industry, governments, and academia to increase the efficacy and efficiency of the overall biomedical technology lifecycle.
Supporting information
S3 Table. Medical Devices assessed in the case studies.
https://doi.org/10.1371/journal.pone.0288126.s003
(DOCX)
S1 Appendix. Verbatim quotation on the ‘MeDKET’ item #5.
https://doi.org/10.1371/journal.pone.0288126.s004
(DOCX)
S2 Appendix. Verbatim quotation on the ‘MeDKET’ item #6.
https://doi.org/10.1371/journal.pone.0288126.s005
(DOCX)
S3 Appendix. Successful and failed case studies.
https://doi.org/10.1371/journal.pone.0288126.s006
(DOCX)
S4 Appendix. Starting time of early economic evaluation in SEs.
https://doi.org/10.1371/journal.pone.0288126.s007
(DOCX)
References
- 1. Vainieri M, Ferrè F, Manetti S. An Integrated Framework to Measure the Performance of Inter-Organizational Programme on Health Technology Assessment. Sustainability, 2021, 13(7): 3873.
- 2. Goodman CS. Introduction to Health Technology Assessment. J Cardiovasc Echogr 2004;22(3):99–106.
- 3.
Barlow J. Managing Innovation in Healthcare. World Scientific Publishing Europe, editor. London; 2020.
- 4. Warty RR, Smith V, Salih M, Fox D, Mcarthur SL, Mol B. Barriers to the Diffusion of Medical Technologies Within Healthcare: A Systematic Review. IEEE Access. 2021;9:139043–58.
- 5. Markiewicz K, van Til JA, Steuten LMG, IJzerman MJ. Commercial viability of medical devices using Headroom and return on investment calculation. Technol Forecast Soc Change. 2016;112:338–46.
- 6. Markiewicz K, Van Til JA, Ijzerman MJ. Medical devices early assessment methods: Systematic literature review. Int J Technol Assess Health Care. 2014;30(2):137–46. pmid:24805836
- 7. Vallejo-Torres L, Steuten LMG, Buxton MJ, Girling AJ, Lilford RJ, Young T. Integrating health economics modelling in the product development cycle of medical devices: a Bayesian approach. Int J Technol Assess Health Care. 2008;24(4):459–64.
- 8. Kolominsky-Rabas PL, Djanatliev A, Wahlster P, Gantner-Bär M, Hofmann B, German R, et al. Technology foresight for medical device development through hybrid simulation: The ProHTA Project. Technol Forecast Soc Change 2015;97:105–14.
- 9. Fuchs S, Olberg B, Panteli D, Perleth M, Busse R. HTA of medical devices: Challenges and ideas for the future from a European perspective. Health Policy (New York). 2017;121(3):215–29. pmid:27751533
- 10. Ni M, Borsci S, Walne S, Mclister AP, Buckle P, Barlow JG. The Lean and Agile Multidimensional Process (LAMP)–a new framework for rapid and iterative evidence generation to support healthcare technology design and development. Expert Rev Med Devices. 2020;17. pmid:32167800
- 11. Borsci S, Buckle P, Huddy J, Alaestante Z, Ni Z, Hanna B. Usability study of pH strips for nasogastric tube placement. PLoS One. 2017;1–14. pmid:29190683
- 12.
Barker R. 2030-The future of medicine: Avoiding a medical meltdown. OUP Oxford. 2010.
- 13.
Stuckler D, Sanjay B. The body economic: why austerity kills. Basic Book. 2013.
- 14. Pietzsch JB, Paté-Cornell ME. Early technology assessment of new medical devices. Int J Technol Assess Health Care. 2008;24(1):36–44. pmid:18218167
- 15. Vallejo-Torres L, Steuten L, Parkinson B, Girling AJ, Buxton MJ. Integrating health economics into the product development cycle: A case study of absorbable pins for treating hallux valgus. Med Decis Mak. 2011;31(4):596–610. pmid:21127320
- 16. Hartz S, John J. Contribution of economic evaluation to decision making in early phases of product development: a methodological and empirical review. Int J Technol Assess Health Care. 2008;24(4):465–72. pmid:18828942
- 17. Ijzerman MJ, Koffijberg H, Fenwick E, Krahn M. Emerging Use of Early Health Technology Assessment in Medical Product Development: A Scoping Review of the Literature. Pharmacoeconomics. 2017;35(7):727–40. pmid:28432642
- 18. Pham B, Tu HAT, Han D, Pechlivanoglou P, Miller F, Rac V, et al. Early economic evaluation of emerging health technologies: protocol of a systematic review. Syst Rev. 2014;3(1):81. pmid:25055987
- 19. Vallejo-Torres L, Steuten L, Parkinson B, Girling AJ, Buxton MJ. Integrating Health Economics Into the Product Development Cycle. Med Decis Mak. 2011;31(4):596–610.
- 20. Keltie K, Bousfield DR, Cole H, Sims AJ. Medical Technologies Evaluation Programme: A review of NICE progression decisions, 2010–2013. Heal Policy Technol. 2016;5(3):243–50.
- 21. Huddy JR, Ni MZ, Barlow J, Hanna GB. Qualitative analysis of stakeholder interviews to identify the barriers and facilitators to the adoption of point-of-care diagnostic tests in the UK. BMJ Open. 2021;11(4).
- 22.
Manetti S, Burns RM, Turchetti G. Evidence-gathering across industry and academia on early Health Technology Assessment (HTA) of medical devices: survey design and piloting. In: Eskola H., Vaisanen O., Viik J. HJ, editor. IFMBE Proceedings. Springer Verlag; 2017. p. 631–634.
- 23. IJzerman MJ, Steuten LM. Early assessment of medical technologies to inform product development and market access: a review of methods and applications. Appl Heal Econ Heal Policy. 2011;9(5):331–47. pmid:21875163
- 24. Manetti S, Vainieri M, Guidotti E, Zuccarino S, Ferré F, Morelli MS, et al. Research protocol for the validation of a new portable technology for real-time continuous monitoring of Early Warning Score (EWS) in hospital practice and for an early-stage multistakeholder assessment. BMJ Open. 2020;1–7. pmid:33273048
- 25. Manetti S, Cumetti M, De Benedictis A, Lettieri E. Adoption of novel biomarker test parameters with machine learning-based algorithms for the early detection of sepsis in hospital practice. Journal of Nursing Management, 2022, 30.8: 3754–3764. pmid:36125938