https://orcid.org/0000-0003-0881-2145
Figures
Abstract
Background
While the effectiveness of patient-reported outcome measures (PROMs) as an intervention to impact patient pathways has been established for cancer care, it is unknown for other indications. We assessed the cost-effectiveness of a PROM-based monitoring and alert intervention for early detection of critical recovery paths following hip and knee replacement.
Methods and findings
The cost-effectiveness analysis (CEA) is based on a multicentre randomised controlled trial encompassing 3,697 patients with hip replacement and 3,110 patients with knee replacement enrolled from 2019 to 2020 in 9 German hospitals. The analysis was conducted with a subset of 546 hip and 492 knee replacement cases with longitudinal cost data from 24 statutory health insurances. Patients were randomised 1:1 to a PROM-based remote monitoring and alert intervention or to a standard care group. All patients were assessed at 12-months post-surgery via digitally collected PROMs. Patients within the intervention group were additionally assessed at 1-, 3-, and 6-months post-surgery to be contacted in case of critical recovery paths. For the effect evaluation, a PROM-based composite measure (PRO-CM) was developed, combining changes across various PROMs in a single index ranging from 0 to 100. The PRO-CM included 6 PROMs focused on quality of life and various aspects of physical and mental health. The primary outcome was the incremental cost-effectiveness ratio (ICER). The intervention group showed incremental outcomes of 2.54 units PRO-CM (95% confidence interval (CI) [0.93, 4.14]; p = 0.002) for patients with hip and 0.87 (95% CI [−0.94, 2.67]; p = 0.347) for patients with knee replacement. Within the 12-months post-surgery period the intervention group had less costs of 376.43€ (95% CI [−639.74, −113.12]; p = 0.005) in patients with hip, and 375.50€ (95% CI [−767.40, 16.39]; p = 0.060) in patients with knee replacement, revealing a dominant ICER for both procedures. However, it remains unclear which step of the multistage intervention contributes most to the positive effect.
Conclusions
The intervention significantly improved patient outcomes at lower costs in patients with hip replacements when compared with standard care. Further it showed a nonsignificant cost reduction in knee replacement patients. This reinforces the notion that PROMs can be utilised as a cost-effective instrument for remote monitoring in standard care settings.
Author summary
Why was this study done?
- Hip and knee replacements are high volume treatments in most Western countries and represent a substantial driver for national healthcare expenditures.
- Evidence on the cost-effectiveness of using patient-reported outcome measures (PROMs) for monitoring in orthopaedics is very limited.
- The increased use of PROMs after orthopaedic surgery, the lack of standardised recovery follow-ups post-surgery combined with positive evidence from other treatment areas motivated the design of the “PROMoting Quality” trial.
What did the researchers do and find?
- We performed a cost-effectiveness analysis (CEA) based on a multicentre randomised controlled trial encompassing 546 patients with hip and 492 patients with knee replacement enrolled from 2019 to 2020 in 9 German hospitals.
- Patients were randomised to a PROM-based remote monitoring and alert intervention for early detection of critical recovery paths, or to a standard care group without monitoring.
- The intervention improved patient outcomes in hip replacements when compared with standard care and showed a cost reduction in both patients with hip and knee replacement.
What do these findings mean?
- Undetected critical recovery after surgery result in both undesired health outcomes and increased health care expenditures.
- In times of resource constraints and staff shortages, evidence from the “PROMoting Quality” trial can inform medical professionals and policy makers that the use of a PROM monitoring and alert system post-surgery can help to improve health outcomes for patients with hip replacement and reduce healthcare expenditures for both procedures, and thus be implemented on system level.
- It can give guidance to the discussions about advancing the digitalisation of the health care sector and further initiates research of using PROMs as monitoring and alert system for other conditions such as chronic diseases. However, it remains unclear exactly which step of the multistage intervention causes the positive effect.
Citation: Schöner L, Kuklinski D, Wittich L, Steinbeck V, Langenberger B, Breitkreuz T, et al. (2024) Cost-effectiveness of a patient-reported outcome-based remote monitoring and alert intervention for early detection of critical recovery after joint replacement: A randomised controlled trial. PLoS Med 21(10): e1004459. https://doi.org/10.1371/journal.pmed.1004459
Academic Editor: David Beard, University of Oxford, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
Received: December 8, 2023; Accepted: August 14, 2024; Published: October 9, 2024
Copyright: © 2024 Schöner et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Since the data contains sensitive patient information it is legally not allowed to make the data publicly accessible to others due to the German data protection law and the data protection agreements within the trial (see S1 Appendix). In order to enable verifiability of the study results after completion of the project (09/2023), the data will be stored at the research institutions - TU Berlin and aQua Institute - for a period of 10 years after project completion. Data access can only be granted in exceptional cases. Requests for data access should be addressed to the Data Protection Officer at TU Berlin: Anette Hiller (info@datenschutz.tu-berlin.de).
Funding: The PROMoting Quality project was funded by the Innovation Fund of the of Joint Federal Committee Germany (https://innovationsfonds.g-ba.de/) under grant number 01NVF18016. The PROMoting Quality consortium lead was at The Department of Health Care Management at the Technische Universität Berlin, with RB as head of the department. Project funding was paid to the PROMoting Quality consortium institutions and covered the salaried employment positions of LS, VS, LW, BL and CP at the Technische Universität Berlin. The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.
Competing interests: The project was funded by the Innovation Fund of the of Joint Federal Committee Germany (01NVF18016). Project funding was paid to the consortium institutions and covered the employment positions of LS, VS, LW, BL and CP. CP reports a salaried position with MedTech company Stryker that is separate and independent from his university research employment and position. FC reports an employment relation with Heartbeat Medical Solutions GmBH. GM reports receiving royalties or contracts, grants, consulting fees, payments for lectures and leadership in other boards, all outside the submitted work. RB reports being member of the government commission on modern and needs-based hospital care, outside the submitted work. DS reports receiving payments for lectures from Zimmer Biomet outside the submitted work. All other authors declare no competing interests.
Abbreviations: ASA, American Society of Anesthesiologists; BMI, body mass index; CEA, cost-effectiveness analysis; CI, confidence interval; GP, general practitioner; HRQoL, health-related quality of life; ICER, incremental cost-effectiveness ratio; ICHOM, International Consortium for Health Outcomes Measurement; OECD, Organisation for Economic Co-operation and Development; PRO-CM, PRO composite measure; PROM, patient-reported outcome measure; RCT, randomised controlled trial; SD, standard deviation
Introduction
Hip and knee replacement surgery is considered an effective treatment option for patients suffering from hip or knee osteoarthritis [1]. In recent years, the complementary use of patient-reported outcome measures (PROMs) for evaluating treatment outcome has become more common [2] and influential [3,4]. While traditionally success of hip and knee replacement surgery was measured by outcome indicators such as complications or revision rates, PROMs refocus outcome assessment on the patient perspective by measuring, e.g., health-related quality of life (HRQoL), pain, and functional status [5].
These instruments have confirmed the effectiveness of hip and knee replacement surgery [6,7], but also showed that a significant number of patients rate their surgery as not beneficial. Kahlenberg and colleagues [8] report that around 15% to 20% of patients are dissatisfied with their results, with non-optimal postoperative functional outcome and lasting pain in the joint being the main determinants.
In Western countries, there is a high rate of hip and knee replacements. For example, in Germany, Pabinger and collegues [9,10] reported intervention rates of 280 (hip replacement) and 230 (knee replacement) per 100,000 population, compared to the Organisation for Economic Co-operation and Development (OECD) average of 160 hip and 150 knee replacements per 100,000 population [11]. Furthermore, various projections report further increases of primary and revision procedures in Western countries [3,4], driven by an ageing population, an increasing number of younger patients and general health system advancements [12,13]. This places a substantial financial strain on national healthcare budgets.
One potential lever to improve mid-to-long-term outcomes, and in turn decrease follow-up treatments, is to monitor patients via PROMs post-surgery to detect critical recovery pathways and, if necessary, adapt treatment protocols [14]. This concept is based on evidence gathered from studies in the field of oncology that showed that regular monitoring of cancer patients via PROMs improved HRQoL, decreased the number of hospital admissions and emergency room visits, and increased the length of overall survival [15,16]. In further oncological studies, it was shown that the intervention was cost-effective [17,18]. Whether a PROM-based monitoring and alert system works to improve treatment outcomes for patients after undergoing hip or knee replacements in a cost-effective way has not been tested yet. Particularly because of the high prevalence of these procedures and the associated healthcare expenditure, even small effects or cost savings can have a considerable impact on system level. Ressources can be used more efficiently, and clinical practice and patient-centeredness can be enhanced through standardised post-surgery follow-ups. Evidence on this field can further give guidance to the discussions about advancing the digitalisation of the health care sector and further initiate research of using PROMs as monitoring and alert system for other conditions such as chronic diseases.
Therefore, we devised and executed the large-scale, two-arm multicentre patient-level randomised controlled trial (RCT) PROMoting Quality [14]. The main objective of the study was to assess the cost-effectiveness of the PROM-based monitoring and alert intervention to proactively identify critical recovery following joint replacement surgery. The early detection aims to improve health outcomes for patients and at the same time reduce costs for the health system, e.g., by reducing the number of post-surgery in- and outpatient visits. In this paper, we are reporting the primary outcome of the PROMoting Quality trial, the intervention`s cost-effectiveness.
Methods
Ethics statement
The “PROMoting Quality” study was approved by the ethics committee of the Charité—Universitätsmedizin, Berlin (EA4/169/19) (see S1 Appendix for ethics committee protocol). All participants gave written informed consent for data collection and anaylsis at the time of recruitment and could withdraw from the study at any time without stating reasons until the final follow-up.
Study design and participants
As specified in the study protocol [14], we executed a prospective multicentre two-armed parallel RCT in 9 different hospitals across Germany. Hospitals recruited patients between October 1, 2019 and December 31, 2020. Inclusion criteria for study participation were adult patients (age 18 years or older) undergoing elective primary hip or knee replacements that matched specific predefined procedure codes (see S1 Table). Exclusion criteria were emergency and life-threatening cases, patients classified under the American Society of Anesthesiologists (ASA) categories 4–6 [19] (i.e., patients with a severe life-threatening disease, moribund patients who are not expected to survive without an operation within 24 h, brain-dead patients), and those lacking direct or indirect access to an e-mail account, or the ability to use digital PROMs. Detailed inclusion and exclusion criteria are listed in the published study protocol [14].
The “PROMoting Quality” study was registered with the German Register for Clinical Studies (DRKS) under DRKS00019916. It was conducted in accordance with the Consolidated Standards of Reporting Trials (CONSORT) guidelines [20], ensuring transparency and accuracy in reporting (see S1 CONSORT Checklist).
Randomisation and masking
Patients were randomised 1:1 to standard care plus PROM-based monitoring and alert intervention or to standard care only. The randomisation took place at hospital discharge and was carried out automatically via the PROM collection and alert software (Heartbeat ONE). For more information on the software solution, see S2 Appendix. To prevent any potential bias in intervention assignment, allocation sequence concealment was employed. Trial personnel and participants were kept unaware of the allocation sequence until the actual assignment. Beginning with the 1-month follow-up, study nurses could not be blinded due to the nature of the intervention, which required active participation. At no point of the study, participants were informed about their group allocation.
Procedures
The study employed a PROM-based post-surgery remote monitoring and alert system for patients in the intervention group in addition to the standard of care (for a schematic illustration of the study design, see S1 Fig). PROMs were collected at hospital admission (baseline) and discharge (randomisation), and at 1-, 3-, 6-, and 12-months post-surgery. The intervention occurred in 4 steps: Patients in the intervention group were monitored and evaluated at 1-, 3-, and 6-months post-surgery based on their digitally collected disease-specific and generic PROMs (step 1). An automated alert was triggered to inform a study nurse when PROM scores in the intervention group significantly deteriorated or surpassed a predefined threshold between 2 measurement points (step 2). Subsequently, patients were contacted by the study nurse to consult on their current health status (step 3). If deemed necessary, they were referred to their aftercare physician or respective specialists to discuss possible treatment and medication adjustments, for example, to receive (additional) physiotherapy or adjust pain medication (step 4).
Patients in the control group received the standard of care and, for evaluation purposes, PROM questionnaires at hospital admission, discharge, and 12-months post-surgery. Standard of care in this context includes the clinical patient pathway, i.e., hospital admission, surgery, usually 4 to 7 days of inpatient hospitalisation, and hospital discharge, followed by inpatient or outpatient rehabilitation. After rehabilitation, patients receive aftercare (e.g., check-ups) in a non-standardised manner, usually by their outpatient specialist or general practitioner (GP).
For a detailed description of the study design, see Kuklinski and colleagues [14], and for further information on threshold alerts, see S2 and S3 Tables.
Outcomes
For the selection of outcome measures, we followed the International Consortium for Health Outcomes Measurement (ICHOM) standard set for Hip- and Knee-Osteoarthritis [21], with minor modifications. We employed a range of PROMs, including the EuroQol 5 dimensions, 5 levels (EQ-5D-5L) and the EuroQol visual analogue scale (EQ-VAS) [22] to capture HRQoL, as well as Hip Disability and Osteoarthritis Outcome Score Physical Function Short-form (HOOS-PS) and Knee Injury and Osteoarthritis Outcome Score Physical Function Short-form (KOOS-PS) [23,24] to assess joint-associated problems and functionality. Analogous pain scales were utilised to evaluate pain in hip (left and right), knee (left and right), and lower back. In addition, Patient-Reported Outcomes Measurement Information System (PROMIS) Depression Shortform (PROMIS-D-SF) and Fatigue Shortform (PROMIS-F-SF) [25] were included to capture patients’ mental health status (see S4 Table). Additionally, data on adverse events, such as reoperations and readmissions within 30, 60, and 90 days, were collected.
We defined patient-reported outcomes (PRO) as PROM-score differences between hospital admission and 12-months follow-up. As per study protocol [14], for the effect evaluation of the cost-effectiveness analysis (CEA) we were using a PRO composite measure (PRO-CM) as the main effect parameter, which combines multiple PROs into a multidimensional quality index following Schöner and colleagues [26]. EQ-5D-5L and EQ-VAS formed the generic HRQoL dimension, HOOS-PS/KOOS-PS, and the pain scales formed the physical health dimension, while PROMIS Depression and Fatigue formed the mental health dimension. The individual PROs were standardised to z-scores and assigned weights following the preference-based approach from Schöner and colleagues [26] before being aggregated via linear additive aggregation. The resulting index was transformed onto a scale with theoretical values ranging from 0 to 100 (T-Score with predefined mean = 50 and standard deviation (SD) = 10), with 0 representing the worst and 100 representing the best possible outcome.
Cost data
For the CEA, we examined potential cost effects of the intervention on the utilisation and provision of outpatient medical care (including prescription of drugs, remedies such as physical therapy), as well as in- and outpatient hospital treatments, and compared them between study arms over a 1-year period post-surgery. The costs of the intervention (including personnel and software implementation) were also considered in the cost analysis.
We sourced treatment costs from health insurance claims data of 24 participating statutory health insurances, i.e., BARMER and 23 of the BKK Dachverband (i.e., the federal association of company health insurances), which collectively cover approximately 18 million persons in Germany (approx. 23% of the population). Patient-level cost data was available on high detail for the period from 1 year pre- to 1 year post-surgery.
Intervention costs were divided into personnel costs, software implementation, and user license fees. Personnel resources in terms of working minutes were established through structured interviews with the study nurses and priced at the average minutely wage of German care personnel (see S5 and S6 Tables). Costs for software implementation and license fees were derived in close collaboration with the software provider.
To determine incremental costs, we adopted the statutory health insurance (i.e., payer) perspective by combining all treatment costs with intervention costs included in the intervention group.
Economic and statistical analysis
The PROMoting Quality trial was designed with the assumption of a 0.15 SD change in the main effect parameter PRO-CM, with a 5% error probability and 80% power. This threshold was chosen based on benchmarks in the literature, where 0.2, 0.5, and 0.8 SDs are considered “small,” “medium,” and “large” effect sizes, respectively [27]. Due to the lack of comparable interventions in existing literature and the expectation of small effects, a conservative 0.15 SD change was assumed to avoid underpowering the study. For more details on the a priori power calculation and statistical analysis plan of the PROMoting Quality trial, see S1 Appendix.
Missing 12-month PROM-scores were imputed with random-forest-based multiple imputation, assuming data was missing at random (no evidence was found that data was missing depending on baseline characteristics or previous score developments).
We performed a comparative analysis using parametric two-sample t tests (for normal distributions) and nonparametric Wilcoxon rank-sum tests (for non-normal distributions) at a two-sided 5% significance level to compare baseline characteristics between the 2 study arms.
We employed a mixed effects modelling approach to examine the intervention effect on the PRO-CM, as well as on the individual secondary outcomes EQ-5D-5L, EQ-VAS, HOOS-PS, KOOS-PS, PROMIS-D-SF, PROMIS-F-SF, Pain, and the post-surgery treatment costs. We controlled for age, sex, mobilisation (rapid recovery <6 h after surgery or conventional care otherwise), and body mass index (BMI) as fixed effects, which were selected based on their theoretical relevance and previous literature [28]. Additionally, we incorporated the hospital as a random intercept in the model to account for potential cluster effects on hospital level. In a sensitifity analysis, we calculated regression models in which we also controlled for the baseline scores of the respective PROMs. Further, in a comparative analysis we employed t tests to assess the significance of group differences in the mean of each outcome variable. Statistical significance was judged at the two-sided 5% level and treatment effect estimates are presented with corresponding 95% confidence intervals (CIs).
For the economic evaluation, we applied a within-trial cost-effectiveness approach (i.e., we analyse the costs and outcomes observed during the trial duration without extrapolating data beyond the study period) over the 12-month post-surgery period following intention-to-treat principles. We conducted a generalised CEA based on the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 [29].
Due to the study duration of 12 months, no discounting of costs or effects was applied. The individual costs were initially analysed descriptively and then examined for differences between the study arms using parametric and nonparametric tests (t test, Wilcoxon rank-sum test). Since cost data were right-skewed and long-tailed, with extreme outliers above the 95% percentile, we performed a 95% winsorisation on the cost data (i.e., set all data above the 95% percentile to the 95% percentile) to avoid large distortions of the means and SDs. We found the 95% threshold to be the most appropriate to enable robust statistical analyses without excessively distorting the distribution or sacrificing valuable data points. To adjust for baseline cost differences between study arms, we utilised linear regression to adjust the post-surgery costs for differences in the pre-surgery costs [30].
Outcomes and costs were calculated as mean difference between control and intervention group. These incremental outcomes (nominator) and incremental costs (denominator) were used to calculate the study’s primary outcome, i.e., the incremental cost-effectiveness ratio (ICER). The ICER indicates how much additional cost is incurred by an additional unit of outcome.
To assess uncertainty associated with the economic evaluation, we performed a nonparametric bootstrapping (1,000 replications) in the incremental costs and effects. Results are reported as 95% CIs of incremental costs and incremental effects. Data points in the north-west and south-east quadrant of the cost-effectiveness plane represent a dominated or dominant strategy, respectively. Further, we report the probability of dominance, which we define as the proportion of point estimates with negative incremental costs and positive incremental effects (i.e., dominant point estimates in the south-east quadrant of the cost-efectiveness plane).
Sensitivity and scenario analyses
We focused our analysis and interpretation on 2 main models that only differed in the composition of costs (see Table 1: Base Case A and B). In a sensitivity analysis, we compared these results with 3 alternative scenarios to examine the robustness towards a different outcome composition (Table 1, Alternative 1 and 2) and checked for a potential imputation bias by applying the analysis only for the complete case, i.e., patients that answered all PROM surveys (Table 1, Alternative 3).
Since the application of the PRO-CM as main effect parameter was, to our knowledge, novel and could face challenges in its in interpretability we conducted 2 additional CEAs applying well-known and validated PROMs to test the robustness of the results. Hence, we performed a CEA with the generic EQ-5D-5L as effect parameter to assess the cost-effectiveness in terms of HRQoL and another CEA with the disease-specific HOOS-PS and KOOS-PS, respectively, to assess the cost-effectiveness in terms of physical health. The cost side in these additional robustness analyses were kept equal to the Base case A.
Results
Study population baseline characteristics
Between October 2019 and December 2020, 7,827 patients were recruited from nine hospitals across Germany. After exclusions, the remaining 6,807 patients were randomly assigned to the intervention or control group. Economic evaluation could be conducted on a group of 1,038 patients who belonged to one of the 24 participating health insurances and provided consent for the utilisation of their health insurance claims data. This yields 546 patients with hip replacement (Intervention: 284, Control: 262) and 492 patients with knee replacement (Intervention: 238, Control: 254) for whom we collected sufficient data to assess the primary outcome (see Fig 1). For the complete case analysis, i.e., without multiple imputation, a total of 433 patients with hip replacement (approximately 79%) and 388 patients with knee replacement (approximately 79%) answered all questionnaires.
PROMs, patient-reported outcome measures.
The baseline characteristics are summarised in Table 2. Patients with hip replacement had a mean age of 66.3 years on the day of surgery, patients with knee replacement 65.7 years. For both procedures, there was a higher proportion of female patients, with 65.4% for patients with hip replacement and 61.8% for patients with knee replacement. Approximately 32.8% of patients with hip replacement and 53.1% of patients with knee replacement were obese (i.e., BMI ≥ 30), and 38.4% of the patients with hip replacement, but only 29.0% of the patients with knee replacement had no comorbidities recorded. All baseline characteristics were also tested for differences between the groups, and we found no statistically significant differences.
Outcomes
Comparative analyses showed that, for patients with hip replacement, the PRO-CM [26] was significantly higher (2.54 incremental points; 95% CI [0.87, 4.21]; t test p = 0.003) in the intervention group, namely with 51.22 (SD 9.66), than for patients in the control group, namely with 48.68 (SD 10.21). The intervention group thus had a 5.22% higher PRO-CM than the control group (see Table 3).
These findings were supported in the regression results: Fig 2 visualises the mixed effect model point estimates of the intervention effect on PRO-CM, on secondary outcomes, and on post-surgery costs. It shows that the intervention had a highly significant impact on the effect measure and increased the PRO-CM on average by 0.24 SD (95% CI [0.07, 0.40]; p = 0.005).
Fig 2 displays the mixed effect model point estimates of the intervention effect on PRO-CM and post-surgery costs (solid lines) as well as on secondary outcomes (dashed lines). The lines represent their corresponding 95% CIs. The numbers in brackets indicate the corresponding p-values. Outcomes and costs were standardised to z-scores to enable comparability. Point estimates above zero indicate better health outcomes or higher costs in the intervention group (in SDs); point estimates below zero indicate worse health outcomes or lower costs in the intervention group compared to the control group. In patients with hip replacement HOOS-PS was used, in patients with knee replacement KOOS-PS was used. PRO-CM, patient-reported outcome composite measure; EQ-5D-5L, EuroQol 5 dimensions, 5 levels; EQ-VAS, EuroQol virtual analogue scale; HOOS-PS, Hip Disability and Osteoarthritis Outcome Score Physical Function Short-form; KOOS-PS, Knee Injury and Osteoarthritis Outcome Score Physical Function Short-form; PROMIS, Patient-Reported Outcomes Measurement Information System Depression Shortform (PROMIS-D-SF) and Fatigue Shortform (PROMIS-F-SF).
Breaking it down to the individual PROs, we found significant differences among patients with hip replacement between the intervention and the control group for all PROs except for fatigue symptoms and pain. The largest score improvements were observed in the physical health dimensions HOOS-PS and pain scores, with the intervention group showing a higher improvement in HOOS-PS with 37.38 (SD 17.23) points compared to the control group with 32.76 (SD 19.09) points (4.62 incremental points; 95% CI [1.55, 7.67]; t test p = 0.003). The point estimates in Fig 2 also show that the intervention effect size was largest for HOOS-PS improvement. The second highest improvement was observed in the HRQoL dimension measured by EQ-VAS. The intervention group had a significantly higher mean change of 23.46 (SD 20.92) compared to the control group with 18.64 (SD 23.10) (4.82 incremental points; 95% CI [1.05, 8.59]; t test p = 0.012). EQ-5D-5L improved on average by 0.336 (SD 0.263) for patients who received the PROM intervention compared to 0.277 (SD 0.278) for patients in control group (0.059 incremental points; 95% CI [0.014, 0.105]; t test p = 0.011). Lastly, patients with hip replacement in the intervention group had a mean improvement in depressive symptoms, captured by PROMIS-D-SF, of 3.41 (SD 7.44) compared to 1.87 (SD 8.46) in the control group (1.53 incremental points; 95% CI [0.20, 2.87]; t test p = 0.025).
For patients with knee replacement, the PRO-CM showed a higher score of 50.45 (SD 9.87) for the intervention group compared to the control group with 49.58 (SD 10.12). This PRO-CM difference of 1.8% between groups was not significant (Table 3). However, we found significant differences in individual score improvements of EQ-VAS and PROMIS-D-SF.
The improvement of the intervention group in EQ-VAS was 17.07 (SD 21.11) compared to 13.08 (SD 22.78) in the control group (3.99 incremental points; 95% CI [0.10, 7.89]; t test p = 0.045). PROMIS-D-SF improvement in the intervention was 2.75 (SD 8.13), which is significantly different from the control group with 1.19 (SD 8.12) (1.56 incremental points; 95% CI [0.11, 3.01]; t test p = 0.035). The displayed point estimates of the mixed effects model in Fig 2 show similar patterns.
Results of the sensitivity analyses that additionally controlled for baseline PROM scores show similar patterns in terms of significant mean differences while the effect size slightly decreased for patients with hip replacement. In patients with knee replacement controlling for the baseline scores increased the effect sizes albeit only slightly, while it showed a significant effect on PROMIS-D-SF. The results of the estimates can be found in S2 Fig for patients with hip replacement and S3 Fig for patients with knee replacement.
Within 1-year post-surgery adverse events ratios (readmission and reoperation) were generally low. Only 4 (1.4%) patients with hip replacement from the intervention group were readmitted to the hospital of which 2 (0.7%) were reoperated. From the control group, 6 patients (2.3%) were readmitted and 6 (2.3%) reoperated. The intervention group of patients with knee replacement had 8 (3.4%) readmissions and 8 (3.4%) reoperations compared to 9 (3.5%) readmissions and 11 (4.3%) reoperations in the control group. Despite the small effect in favour of the intervention group, these differences in adverse events were nonsignificant due to the small number of observations.
Costs
The adjusted 1-year post-surgery treatment costs were significantly lower in the intervention groups in both patients with hip and patients with knee replacement (Table 3). The intervention group of patients with hip replacement had mean post-surgery costs of 3,620.12€ compared to 3,996.55€ in the control group. This is 376.43€ less in the intervention group (95% CI [−645.52, −107.33]; Wilcoxon rank-sum test p = 0.004). The intervention group of patients with knee replacement had mean post-op costs of 4,907.62€ while the control group had mean costs of 5,283.13€. This is 375.50€ less in the intervention group (95% CI [−770.91, 19.91]; Wilcoxon rank-sum test p = 0.034). The mixed effect model, however, only showed a nonsignificant intervention effect on the costs in patients with knee replacement (Fig 2). See S7 and S8 Tables for more details on costs.
Primary cost-effectiveness analysis
The bootstrapped within-trial CEA for patients with hip replacement showed that during the 12-month follow-up period, the intervention group had larger outcome improvements (2.54 additional PRO-CM units, 95% CI [0.93, 4.14]; p = 0.002) at lower cost compared to the control group (−376.43€, 95% CI [−639.74, −113.12]; p = 0.005) (see Table 4).
Panel A of Fig 3 shows the results of the bootstrapped point estimates of incremental costs and effects for patients with hip replacement. The observed negative incremental costs and positive incremental effects revealed a dominant ICER. The intervention’s probability of dominance (i.e., the proportion of point estimates in the south-east quadrant) was 99.6%. The bootstrapped estimates on panel B of Fig 3 show that additionally considering the intervention costs of 123.78€ per patient, the intervention was still less costly (−252.73€, 95% CI [−516.05, 10.58]; p = 0.060) revealing a dominant ICER and a 97.5% probability of dominance.
Regarding patients with knee replacement, the PROM intervention showed to be effective, albeit non-significantly (0.87 additional PRO-CM units, 95% CI [−0.94, 2.67]; p = 0.347). At the same time, we observed lower costs in the intervention group (−375.50€, 95% CI [−767.40, 16.39]; p = 0.060) (see Table 4). Panel C of Fig 3 shows the results of the bootstrapped point estimates of incremental costs and incremental effects for patients with knee replacement. Compared to patients with hip replacement, we found visibly less point estimates in the south-east (i.e., dominant) quadrant in patients with knee replacement where we also observed some estimates in the south-west quadrant, in which effects are negative but costs are reduced. However, the probability of dominance was still 79.9%. Panel D of Fig 3 shows that adding the intervention costs to the treatment costs shifted the estimates cloud slightly up. It resulted in a cost difference between intervention and control group of −251.81€ (95% CI [−643.71, 140.09]; p = 0.208). The probability of dominance was 73.3%.
Since the intervention revealed dominant ICERs for both procedures, we do not report the exact ICER values as negative ratios are difficult to interpret and can be misleading.
Sensitivity analyses
The findings from our base case models were supported by the sensitivity analyses. While a composite measure with equally weighted PROs (Table 1, Alternative 1 and 2) led to very similar mean incremental outcomes of 2.43 (95% CI [0.82, 4.04]; p = 0.003) points in the intervention group of patients with hip replacement, the incremental outcomes in the intervention group of patients with knee replacement increased to 1.36 (95% CI [−0.44, 3.16]; p = 0.138) (see Table 4).
In the complete case analysis (Table 1, Alternative 3), we only considered patients of both groups that answered the complete set of PROM questionnaires to check for a potential imputation bias. In this scenario, we observed less costs in both intervention groups compared to the control groups, while the incremental outcomes with 3.06 (95% CI [1.05, 5.06]; p = 0.003) in patients with hip replacement and 1.48 (95% CI [−0.50, 3.47]; p = 0.144) in patients with knee replacement were higher than in all other scenarios. These scenario analyses indicated robust results in terms of larger outcome improvements and cost savings for both intervention groups.
Finally, the 2 CEAs using the EQ-5D-5L and the HOOS-PS and KOOS-PS, respectively, were in line with our base case analysis and further confirmed the robustnesss of the results. For the results of these additional analyses, see S4 and S5 Figs for patients with hip replacement and S6 and S7 Figs for patients with knee replacement. For patients with hip replacement, both the EQ-5D-5L CEA and the HOOS-PS CEA revealed a dominant intervention with 99.6% probability of dominance and 99.8%, respectively. Thereby patients with hip replacement in the intervention group on average improved by 0.22 SD in EQ-5D-5L (95% CI [0.05, 0.39]; p = 0.011) and by 0.26 SD in HOOS-PS (95% CI [0.10, 0.42]; p = 0.002) more compared to the control group. In patients with knee replacement, the intervention showed less cost-effective, in line with the base case results. The incremental effect in EQ-5D-5L was 0.07 SD (95% CI [−0.12, 0.26]; p = 0.455) for the intervention group and in KOOS-PS 0.00 SD (95% CI [−0.18, 0.17]; p = 0.991). The probability of dominance was 74.2% and 45.5%, respectively.
Discussion
We conducted the “PROMoting Quality” trial to explore the cost-effectiveness of a PROM-based monitoring and alert intervention for hip and knee replacement patients within the first 12-months after surgery. Overall, we found the intervention to be cost-effective for both procedures, but to a different degree in size and significance. For patients with hip replacement, the probability of dominance of the intervention was 99.6% without intervention cost (97.5% with intervention cost) with a positive outcome effect of 2.54 additional PRO-CM units and lower cost of −376.43€ (−252.73€) between intervention and control group. These estimations position the intervention in the dominant quadrant of the bootstrapped ICER results, indicating superior outcomes at lower costs. Sensitivity analyses confirmed the robustness of our results.
When examining the hip replacement results more closely, they showed that effects were driven by relatively large improvements of the intervention compared to the control group in nearly all PROM dimensions, except the pain scale and PROMIS-F-SF (see Fig 2), with the largest effects in HRQoL and functional status. Comparing the results to those of all 6,807 patients (i.e., including patients without cost data), Steinbeck and colleagues [31] reveal slight differences in the affected health dimensions: Main effects of the intervention for all patients were observed in the EQ-VAS, the PROMIS-F-SF, and PROMIS-D-SF, with nonsignificant, smaller effects for functional status measured by HOOS-PS. While both the original sample with all patients and our patient group showed positive effects of the intervention, the differences in the PRO dimensions must be due to slightly different sample characteristics (see S9 Table). In particular, we see that there are differences in the sex distribution: In our patient group, the proportion of female patients was higher in both hip patients (65%) and knee patients (62%) than in the overall sample with all patients (hip 56%, knee 54%), suggesting that female patients might benefit more from the intervention regarding their physical wellbeing which is supported by Langenberger and colleagues [32]. A recent study by Steinbeck and colleagues [33] suggests that future studies should focus more on the gender health gap to identify gender-specific differences and improve patient-centred care. However, apart from this we only see nonsignificant differences in the sample characteristics.
We initially expected to see the major effects for functional status improvements due to treatment protocol adjustments based on PROM-alerts, but also observed significant positive effects for HRQoL and mental health in our sample. This could be caused by the “being taken care of” aspect of remotely monitoring patients via PROMs [34]. Even though we cannot compare our results to similar studies for patients with hip or knee replacements, as to our knowledge they do not exist yet, the improvements in HRQoL of our intervention are in line with studies in other fields [35,36]. In particular for oncologic patients, Basch and colleagues [16], for example, reported a significant positive effect of symptom monitoring via PROMs on the EQ-5D-5L. Furthermore, a systematic review by Gibbons and colleagues [36] concluded from 116 randomised trials small but significant effects of PROMs as an intervention on HRQoL, but inconclusive impacts on mental health, functioning, and pain.
Moreover, the intervention also showed a positive effect on aftercare health expenditures for patients with hip replacement. The largest factor of the 376.43€ cost savings can be explained by fewer GP visits, fewer prescriptions, and less expenses for remedies such as physiotherapy (see S7 Table). This reduction in GP visits (and consequently receiving less prescriptions and remedies) may potentially be attributed to feeling well taken care of. Patients might have confidence that their recovery is on track when they are not being contacted [37].
Regarding patients with knee replacement, the effects of the intervention were more moderate and only partly significant. The probability of the intervention being dominant was 79.9% when not considering the intervention cost (73.3% with intervention cost). While there were positive outcome effects in terms of PRO-CM and favourable cost effects, they both were nonsignificant considering a 5% significance level (only the comparative Wilcoxon rank-sum test showed significant cost differences between groups). One reason for this difference in relation to patients with hip replacement could be differences in patient characteristics. Patients with knee replacement had a considerably higher BMI and more comorbidities than patients with hip replacement (Table 2), potentially leading to a smaller effect of the intervention, and also on post-surgery health expenditures. For example, a heterogeneity analysis of the PROMoting Quality trial by Langenberger and colleagues [32] showed that the intervention for patients with knee replacement has a larger positive effect for non-obese patients. Moreover, overweight and comorbid patients might visit their GP more often than their healthier counterparts due to their generally worse health status. It can also be assumed that patients with knee replacement need a significantly longer recovery time than patients with hip replacement due to the more complex nature of the surgery. It is therefore reasonable to assume that a longer follow-up period might be appropriate for patients with knee replacement in order to see significant effects. Additionally, patients with knee replacement usually receive closer postoperative care than patients with hip replacement. This can considerably decrease the impact of the “being taken care of” effect for health expenditures.
This is also confirmed by looking at the break down of the 12-month post-surgery health expenditures (see S8 Table). In patients with knee replacement, we did not observe significant differences in visits for any of the physician categories, and only small differences in cost for outpatient care, remedies, and aids. Nevertheless, our results showed a positive impact of the intervention on overall health expenditures (−375.50€). As for the outcome effect, we saw that the significant effects of the EQ-VAS and the PROMIS-F-SF drive the overall positive but nonsignificant effect of the PRO-CM in the intervention group. These results correspond with results reported for the whole study population with all patients [31,32].
The study has several strengths. Firstly, the large-scale trial PROMoting Quality provides data from 6,807 patients with hip and knee replacement collected in 9 top tier hospitals in Germany over a period of more than 2 years. Secondly, comprehensive billing data from 24 German statutory health insurances of 1,038 patients was added which allows the longitudinal analysis from the payer’s perspective in the first place. Therefore, our CEA draws from a large and carefully collected data set on a high level of detail. The existing data enabled us to include the actual costs incurred per patient into our analyses and to differentiate among different cost categories. Consequently, we investigate cost-effectiveness on actual cost data of included patients, and do not simulate costs as is quite common in other cost-effectiveness analyses [17,18,38]. Lastly, we combined several PROs into a single measure, which considers each instrument, and therefore gives a holistic picture of post-surgery patient-reported health outcome improvement. Additionally, we performed various scenario and sensitivity analyses that increase the interpretative power and robustness of the results.
However, this study also comes with limitations. Firstly, we could only include the 1,038 patients with cost data in the cost-effectiveness analyses. This is less than initially anticipated. With this sample size we could, nevertheless, show significant effects and, hence, rule out potential type 1 errors. As demonstrated for the full sample, in Steinbeck and colleagues [31] and Langenberger and colleagues [32], outcome effects are positive for the intervention group with different effects per health dimension and patient subgroup. Hence, cost-effectiveness might also differ slightly for different samples. As the availability of cost data was not dependent on choice to participate but on the affiliation to a statutory health insurance, we can exclude a potential selection bias. Secondly, the predefined PROM alerts were developed upon a Delphi method and for the assumed average patient with hip or knee replacement. As shown in Kuklinski and colleagues [7], PROM alert thresholds should be defined for individual patient characteristics, in particular their preoperative PROM scores. Therefore, the intervention was not completely tailored to patients’ recovery pathways. We hypothesise that with a more patient-tailored approach the effect of the intervention could be increased. Thirdly, we cannot disentangle the effects of the 4 different steps of the intervention, but can only interpret the combined effect of the intervention. Further studies should look at the effects of each step individually.
Evidence from this trial has multiple implications for policy and practice. Firstly, we showed that implementing a PROM monitoring and alert system improves health outcomes for patients with hip and knee replacement in most dimensions. It is highly likely (73.3% to 99.9%) that this health outcome improvement reveals a dominant ICER, even when considering intervention cost. Secondly, hospitals and physicians are able to (1) learn more about patients’ post-surgery recovery pathways; (2) monitor them for critical developments; and (3) manage the patient’s concern better. Thirdly and even more importantly, the observed decrease of healthcare expenditures means less resource consumption. Thus, in times of rising medical personnel shortages and health services demand, our intervention—and similar PROM-monitoring interventions—could potentially relieve the healthcare system. To realise such benefits on a large scale, the intervention would need to be implemented into standard care pathways. In Germany, one possibility could be, for example, an implementation as digital health application via the country’s fast-track process [39].
In conclusion, as PROMs play a central role in the development of patient-centred care models, this study demonstrated that they can contribute to lower cost and higher patient-reported quality of care. Therefore, health system stakeholders should no longer be reluctant to integrate these measures into standard care pathways.
Supporting information
S1 Appendix. Ethics approval, data protection plan, and statistical analysis plan.
https://doi.org/10.1371/journal.pmed.1004459.s001
(PDF)
S2 Appendix. Software description Heartbeat Medical.
https://doi.org/10.1371/journal.pmed.1004459.s002
(PDF)
S2 Fig. Mixed effect sensitivity regression analysis—hip patients.
https://doi.org/10.1371/journal.pmed.1004459.s005
(TIF)
S3 Fig. Mixed effect sensitivity regression analysis—knee patients.
https://doi.org/10.1371/journal.pmed.1004459.s006
(TIF)
S4 Fig. Cost-effectiveness plane EQ-5D-5L CEA—hip patients.
https://doi.org/10.1371/journal.pmed.1004459.s007
(TIF)
S5 Fig. Cost-effectiveness plane HOOS-PS CEA—hip patients.
https://doi.org/10.1371/journal.pmed.1004459.s008
(TIF)
S6 Fig. Cost-effectiveness plane EQ-5D-5L CEA—knee patients.
https://doi.org/10.1371/journal.pmed.1004459.s009
(TIF)
S7 Fig. Cost-effectiveness plane KOOS-PS CEA—knee patients.
https://doi.org/10.1371/journal.pmed.1004459.s010
(TIF)
S1 Table. Procedure codes (OPS) for inclusion of patients.
https://doi.org/10.1371/journal.pmed.1004459.s011
(DOCX)
S2 Table. Absolute and relative intervention alert thresholds for each PROM-score.
https://doi.org/10.1371/journal.pmed.1004459.s012
(DOCX)
S3 Table. Critical values and alert reactions.
https://doi.org/10.1371/journal.pmed.1004459.s013
(DOCX)
S5 Table. Calculation and pricing of personnel minutes.
https://doi.org/10.1371/journal.pmed.1004459.s015
(DOCX)
S6 Table. Calculation of the required staff minutes for the intervention.
https://doi.org/10.1371/journal.pmed.1004459.s016
(DOCX)
S7 Table. Post-surgery costs for hip replacement patients.
https://doi.org/10.1371/journal.pmed.1004459.s017
(DOCX)
S8 Table. Post-surgery costs for knee replacement patients.
https://doi.org/10.1371/journal.pmed.1004459.s018
(DOCX)
S9 Table. Descriptive statistics of the whole study sample.
https://doi.org/10.1371/journal.pmed.1004459.s019
(DOCX)
Acknowledgments
Special thanks to all participating consortium members, partner institutions and hospitals, study nurses, and patients.
References
- 1. Gademan MGJ, Hofstede SN, Vliet Vlieland TPM, Nelissen RGHH, Marang-van de Mheen PJ. Indication criteria for total hip or knee arthroplasty in osteoarthritis: a state-of-the-science overview. BMC Musculoskelet Disord. 2016;17:463. pmid:27829422.
- 2. Harris K, Dawson J, Gibbons E, Lim CR, Beard DJ, Fitzpatrick R, et al. Systematic review of measurement properties of patient-reported outcome measures used in patients undergoing hip and knee arthroplasty. Patient Relat Outcome Meas. 2016;7:101–8. Epub 2016/07/25. pmid:27524925.
- 3. Price AJ, Alvand A, Troelsen A, Katz JN, Hooper G, Gray A, et al. Knee replacement. Lancet. 2018;392:1672–82. pmid:30496082.
- 4. Ferguson RJ, Palmer A JR, Taylor A, Porter ML, Malchau H, Glyn-Jones S. Hip replacement. Lancet. 2018;392:1662–71. pmid:30496081.
- 5. Whitebird RR, Solberg LI, Ziegenfuss JY, Asche SE, Norton CK, Swiontkowski MF, et al. Personalized outcomes for hip and knee replacement: the patients point of view. J Patient Rep Outcomes. 2021;5:116. Epub 2021/11/04. pmid:34735662.
- 6. Skou ST, Roos EM, Laursen MB, Rathleff MS, Arendt-Nielsen L, Simonsen O, et al. A Randomized, Controlled Trial of Total Knee Replacement. N Engl J Med. 2015;373:1597–606. pmid:26488691.
- 7. Kuklinski D, Marques CJ, Bohlen K, Westphal KC, Lampe F, Geissler A. Thresholds for meaningful improvement in WOMAC scores need to be adjusted to patient characteristics after hip and knee replacement. J Orthop. 2022;29:50–9. pmid:35125779
- 8. Kahlenberg CA, Nwachukwu BU, McLawhorn AS, Cross MB, Cornell CN, Padgett DE. Patient Satisfaction After Total Knee Replacement: A Systematic Review. HSS J. 2018;14:192–201. Epub 2018/06/05. pmid:29983663.
- 9. Pabinger C, Lothaller H, Geissler A. Utilization rates of knee-arthroplasty in OECD countries. Osteoarthr Cartil. 2015;23:1664–73. pmid:26028142.
- 10. Pabinger C, Geissler A. Utilization rates of hip arthroplasty in OECD countries. Osteoarthr Cartil. 2014;22:734–41. pmid:24780823.
- 11.
Health at a Glance 2023. OECD; 2023.
- 12. Maradit Kremers H, Larson DR, Crowson CS, Kremers WK, Washington RE, Steiner CA, et al. Prevalence of Total Hip and Knee Replacement in the United States. J Bone Joint Surg Am. 2015;97:1386–97. pmid:26333733.
- 13. Bayliss LE, Culliford D, Monk AP, Glyn-Jones S, Prieto-Alhambra D, Judge A, et al. The effect of patient age at intervention on risk of implant revision after total replacement of the hip or knee: a population-based cohort study. Lancet. 2017;389:1424–30. Epub 2017/02/14. pmid:28209371.
- 14. Kuklinski D, Oschmann L, Pross C, Busse R, Geissler A. The use of digitally collected patient-reported outcome measures for newly operated patients with total knee and hip replacements to improve post-treatment recovery: study protocol for a randomized controlled trial. Trials. 2020;21:322. pmid:32272962.
- 15. Denis F, Lethrosne C, Pourel N, Molinier O, Pointreau Y, Domont J, et al. Randomized Trial Comparing a Web-Mediated Follow-up With Routine Surveillance in Lung Cancer Patients. J Natl Cancer Inst. 2017;109. pmid:28423407.
- 16. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2016;34:557–65. pmid:26644527.
- 17. Lizée T, Basch E, Trémolières P, Voog E, Domont J, Peyraga G, et al. Cost-Effectiveness of Web-Based Patient-Reported Outcome Surveillance in Patients With Lung Cancer. J Thorac Oncol. 2019;14:1012–20. Epub 2019/02/15. pmid:30776447.
- 18. Nixon NA, Spackman E, Clement F, Verma S, Manns B. Cost-effectiveness of symptom monitoring with patient-reported outcomes during routine cancer treatment. J Cancer Policy. 2018;15:32–6.
- 19.
Daniel John Doyle Joseph Maxwell Hendrix, Garmon Emily H. American Society of Anesthesiologists Classification. In: Doyle DJ, Hendrix JM, Garmon EH, editors. StatPearls [Internet]. StatPearls Publishing; 2023.
- 20. Turner L, Shamseer L, Altman DG, Weeks L, Peters J, Kober T, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030. Epub 2012/11/14. pmid:23152285.
- 21. Rolfson O, Wissig S, van Maasakkers L, Stowell C, Ackerman I, Ayers D, et al. Defining an International Standard Set of Outcome Measures for Patients With Hip or Knee Osteoarthritis: Consensus of the International Consortium for Health Outcomes Measurement Hip and Knee Osteoarthritis Working Group. Arthritis Care Res (Hoboken). 2016;68:1631–9. pmid:26881821.
- 22. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20:1727–36. Epub 2011/04/09. pmid:21479777.
- 23. Nilsdotter AK, Lohmander LS, Klässbo M, Roos EM. Hip disability and osteoarthritis outcome score (HOOS)—validity and responsiveness in total hip replacement. BMC Musculoskelet Disord. 2003;4:10. Epub 2003/05/30. pmid:12777182.
- 24. Roos EM, Lohmander LS. The Knee injury and Osteoarthritis Outcome Score (KOOS): from joint injury to osteoarthritis. Health Qual Life Outcomes. 2003;1:64. Epub 2003/11/03. pmid:14613558.
- 25. Wahl Inka, Bernd Löwe Matthias Rose. Das Patient-Reported Outcomes Measurement Information System (PROMIS): Übersetzung der Item-Banken für Depressivität und Angst ins Deutsche. Klin Diagnostik Eval. 2011;3:236–61. Available from: https://fis-uke.de/portal/de/publications/das-patientreported-outcomes-measurement-information-system-promis-ubersetzung-der-itembanken-fur-depressivitat-und-angst-ins-deutsche(c57ec959-c271-4055-b854-b90c87727ab0)/export.html.
- 26. Schöner L, Kuklinski D, Geissler A, Busse R, Pross C. A composite measure for patient-reported outcomes in orthopedic care: design principles and validity checks. Qual Life Res. 2023:1–11. Epub 2023/03/24. pmid:36964454.
- 27. Teare MD, Dimairo M, Shephard N, Hayman A, Whitehead A, Walters SJ. Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study. Trials. 2014;15:264. Epub 2014/07/03. pmid:24993581.
- 28. Twisk J, Bosman L, Hoekstra T, Rijnhart, Welten M, Heymans M. Different ways to estimate treatment effects in randomised controlled trials. Contemp Clin Trials Commun. 2018;10:80–5. Epub 2018/03/28. pmid:29696162.
- 29. Husereau D, Drummond M, Augustovski F, Bekker-Grob E de, Briggs AH, Carswell C, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Explanation and Elaboration: A Report of the ISPOR CHEERS II Good Practices Task Force. Value Health. 2022;25:10–31. pmid:35031088.
- 30. van Asselt ADI, van Mastrigt GAPG, Dirksen CD, Arntz A, Severens JL, Kessels AGH. How to deal with cost differences at baseline. Pharmacoeconomics. 2009;27:519–28. pmid:19640014.
- 31. Steinbeck V, Langenberger B, Schöner L, Wittich L, Klauser W, Mayer M, et al. Electronic Patient-Reported Outcome Monitoring to Improve Quality of Life After Joint Replacement: Secondary Analysis of a Randomized Clinical Trial. JAMA Netw Open. 2023;6:e2331301. Epub 2023/09/05. pmid:37656459.
- 32. Langenberger B, Steinbeck V, Schöner L, Busse R, Pross C, Kuklinski D. Exploring treatment effect heterogeneity of a PROMs alert intervention in knee and hip arthroplasty patients: A causal forest application. Comput Biol Med. 2023;163:107118. Epub 2023/06/22. pmid:37392619.
- 33. Steinbeck V, Bischof AY, Schöner L, Langenberger B, Kuklinski D, Geissler A, et al. Gender health gap pre- and post-joint arthroplasty: identifying affected patient-reported health domains. Int J Equity Health. 2024;23:1–11. pmid:38413981
- 34. Lapin BR, Honomichl RD, Thompson NR, Rose S, Sugano D, Udeh B, et al. Association Between Patient Experience With Patient-Reported Outcome Measurements and Overall Satisfaction With Care in Neurology. Value Health. 2019;22:555–63. pmid:31104734.
- 35. Ishaque S, Karnon J, Chen G, Nair R, Salter AB. A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual Life Res. 2019;28:567–92. Epub 2018/10/03. pmid:30284183.
- 36. Gibbons C, Porter I, Gonçalves-Bradley DC, Stoilov S, Ricci-Cabello I, Tsangaris E, et al. Routine provision of feedback from patient-reported outcome measurements to healthcare providers and patients in clinical practice. Cochrane Database Syst Rev. 2021;10:CD011589. Epub 2021/10/12. pmid:34637526.
- 37. Greenhalgh J, Gooding K, Gibbons E, Dalkin S, Wright J, Valderas J, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes. 2018;2:42. Epub 2018/09/15. pmid:30294712.
- 38. Beard DJ, Davies LJ, Cook JA, MacLennan G, Price A, Kent S, et al. The clinical and cost-effectiveness of total versus partial knee replacement in patients with medial compartment osteoarthritis (TOPKAT): 5-year outcomes of a randomised controlled trial. Lancet. 2019;394:746–56. Epub 2019/07/17. pmid:31326135.
- 39. Stern AD, Brönneke J, Debatin JF, Hagen J, Matthies H, Patel S, et al. Advancing digital health applications: priorities for innovation in real-world evidence generation. Lancet Digit Health. 2022;4:e200–e206. pmid:35216754.