2.1 Ethics statement

This study is based on an observational clinical trial registered at ClinicalTrials.gov (NCT05209555) and approved by the Medical Research Ethics Committee (MEC-U, Nieuwegein, The Netherlands; approval number A21.313/R21.054) and the Institutional Review Board of Catharina Hospital (Eindhoven, The Netherlands; approval granted). Written informed consent was obtained from all participants prior to inclusion. The study was conducted in accordance with the principles of the Declaration of Helsinki (Fortaleza, Brazil, October 2013) and with applicable Dutch law.

2.2. Participants and sample size

A cohort study was performed in the Catharina Hospital in Eindhoven, the Netherlands, a tertiary referral hospital, providing cardiac surgery. The sample size of 50 patients is determined by assuming a 50% incidence of POD in this patient population. This sample suffices to achieve a significant difference compared to an AUROC of 0.5 (for MoCA) and a minimum BAMCOG Area Under the Receiver Operation Curve (AUROC) of 0.72 with α = 0.05 and a power (1-β) of 0.8. To account for a 20% loss to follow-up, sixty patients scheduled for open aortic valve replacement (AVR) surgery were enrolled in this study between January and October 2022. Inclusion criteria comprised individuals aged 65 years or older, scheduled for AVR surgery, proficient in Dutch, and capable of engaging in tablet-based gaming. Exclusion criteria encompassed documented learning disorders (e.g., dyslexia, dyscalculia, non-verbal disorder and language development disorder), psychiatric disorders, dementia, use of lithium or clozapine, presence of a pacemaker, defibrillator or neurostimulator, and three or more days of postoperative sedation. Ten patients were excluded during the study period due to the transition to an alternative surgical procedure (6 cases), absence of cardiopulmonary bypass during surgery (1 case), and refusal to participate (3 cases), resulting in a final analysis cohort of fifty patients.

2.3. Materials

2.4. Procedure and analysis

Informed consent was obtained, and the initial baseline measurements for DeltaScan, MoCA (Dutch V8.1) and BAMCOG were conducted at the day before surgery. Postoperatively, MoCA and BAMCOG assessments were repeated on the first, third, and seventh day, once daily using alternate forms (MoCA Dutch V8.1, 7.2 and 7.3, respectively). Due to the variable course of delirium, DeltaScan and DOS (Delirium Observation Scale) were measured twice a day postoperatively, also on the first, third and seventh day. Fig 1 summarizes the measurements performed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Summary measurements during study.

https://doi.org/10.1371/journal.pdig.0001005.g001

This paper addresses two research questions: the main is whether postoperative delirium (POD) and postoperative encephalopathy (POE) can be predicted using preoperative MoCA and BAMCOG scores, where POD and POE are defined as the occurrence, at least once, of a Delirium Observation Screening (DOS) score greater than 3 and a DeltaScan score greater than 3, respectively. The second question examines differences between delirious and non-delirious patients based on factors such as age, co-morbidities, pre- and perioperative medication use, alcohol consumption, and duration of extracorporeal perfusion.

Descriptive data are presented as mean and standard deviation (SD), median and interquartile range (IQR) where appropriate. AUROC calculations with 95% confidence intervals of the ROC-curves were created using RStudio. A logistic regression analysis was conducted (using RStudio) to examine the association between BAMCOG scores and POD (measured by DOS). To classify the level of education, the Dutch Verhage scale was used. [24]

4.1. Results and interpretation

This study suggests that the BAMCOG poorly predicts postoperative POE, but demonstrates a good predictive performance for POD classified with DOS. In contrast, the MoCA poorly predicts both POE and POD. Logistic regression analysis indicated that higher preoperative BAMCOG scores are significantly associated with a decreased incidence of POD measured by DOS. Another interesting result of this study is the average reduction in hospital stay by 1 to 1.5 days for patients without POE and POD, highlighting the importance of identifying high-risk individuals to improve perioperative care.

This study provides, to our knowledge, the first evaluation of the predictive value of both the BAMCOG and the MoCA POD and POE. However, our findings regarding BAMCOG’s ability to predict POD add to previous studies, such as Dworkin et al. [26] and Segernäs et al. [27], which highlighted the validity of cognitive screeners (Mini-Cog and MMSE, respectively) in predicting POD after cardiac surgery. Also other predictive approaches such as a machine learning model of Nagata et al. have shown promise. [28] However, unlike these models and conventional tests, our study shows potential advantages of tablet-based assessments, such as reducing stress and making the process feel less like a traditional test, thereby minimizing performance anxiety and producing more valid cognitive responses. Consistent with this, another key advantage of our tablet-based games is that they are self-administered, allowing patients to play in a private setting without the researcher’s active involvement or questioning. In contrast, tests like the MoCA may induce stress due to real-time observation and supervision by the researcher or clinician. Lastly, these previous studies did not or explore the potential benefits of gamified systems, as we do.

4.2. Role of gamification

Gamification could explain also the difference in predictive power between BAMCOG and MoCA, as it may elicit more ecologically valid cognitive responses compared to traditional methods. While research on gamified preoperative cognitive screening and its predictive value for POD remains limited, extensive studies in other healthcare domains have shown promising outcomes for gamification. For instance, a meta-analysis by Vermeir et al. [29] revealed that gamified cognitive tasks significantly enhance motivation and engagement compared to conventional tasks, underscoring the potential benefits of gamification, which we revealed in this study. Additionally, research on gamification in mental health and health behaviour has demonstrated substantial positive effects. A systematic review [28] identified seven key benefits of gamification in health and well-being, with the primary advantage being the enhancement of intrinsic motivation. [30] Furthermore, another review [31] on gamification in mental health highlighted its potential to increase patient engagement and motivation, helping individuals achieve therapeutic goals, such as promoting mental wellness and alleviating psychological symptoms. Supporting these findings, our research group reported a good System Usability Scale (SUS) score of 79.7 for BAMCOG usability, potentially attributed due to the tool’s game-like and fun elements. (manuscript submitted)

4.3. Gold standard delirium diagnosis

Our results indicate varying predictive values depending on the diagnostic method used for delirium. While the gold standard for diagnosing delirium involves assessment by a delirium expert using Diagnostic and Statistical Manual of mental disorders (DSM-5-TR) criteria, in practice, this process is challenging to implement consistently in a busy hospital setting. Tools such as the DOS, with reported sensitivity and specificity of 90% and 91% [32,33], serve as practical alternatives when expert evaluation is not immediately available. However, previous research indicates significant interobserver variability in diagnosing delirium using these tools. [34,35] The inherent subjectivity of DOS scores highlights the need for more objective methods of delirium detection.

To address this, we incorporated the DeltaScan into our study, aiming to enhance delirium detection by integrating multiple diagnostic approaches. However, our results reveal that POE was observed in 38 patients (76%), whereas only 14 patients (28%) exhibited the phenotype of postoperative delirium as determined by DOS. An interpretation for these results is that individuals experiencing POE may be in a pre-stage of delirium that is not inherently linked to preoperative cognitive function. However, the transformation into delirium or its absence appears to be associated with the individual’s preoperative cognitive function.

4.4. Limitations

A key limitation of this study is the lack of a definitive gold standard for diagnosing delirium. [36] While the DSM-5-TR and ICD are reference standards, their use is limited to clinical settings with trained professionals, reducing practicality. [37] We used DeltaScan and DOS to address this, but evidence supporting DeltaScan’s efficacy, particularly for patients with pre-operative cognitive impairments, remains limited. Additionally, DeltaScan is not yet considered a gold standard for delirium detection. The same applies to the (DOS), which – although widely used in clinical practice – is a screening tool rather than a diagnostic standard, and may miss hypoactive or atypical forms of delirium. [38,39]

Another limitation is the lack of evidence regarding BAMCOG’s predictive accuracy for mild cognitive impairment (MCI). Although our prior research (manuscript submitted) showed moderate validity with MoCA for detecting MCI, BAMCOG was initially designed for online self-monitoring rather than as a tablet-based cognitive screener. [13] However, this study highlights the potential of BAMCOG to possibly identify patients at risk for POD, regardless of its accuracy in detecting MCI.

A further consideration concerns the potential influence of digital literacy when using a game-based, tablet-delivered cognitive screener such as BAMCOG. Older adults may vary in their familiarity and comfort with digital devices, which could theoretically affect performance. In our analysis, age was included as a proxy for digital proficiency, but it did not significantly affect the association between BAMCOG scores and POD risk. This suggests that the predictive value of BAMCOG is not merely driven by differences in digital skills, but rather reflects cognitive functioning. Importantly, previous research by Wu et al. [40] and Bayer et al. [41] has shown that tablet-based cognitive tests and digital cognitive assessment tools are valid, feasible, and usable among older adults, supporting the potential of such tools in clinical screening contexts. Nonetheless, future research should consider incorporating more direct measures of digital literacy to further examine its potential impact.

Lastly, the causes of POD after cardiac surgery, such as reperfusion ischemia, non-pulsatile flow due to cardiopulmonary bypass (CBP), systemic inflammation and cerebral hyperthermia, remain debated. [42,43] Logically, the effects of these perioperative mechanisms would manifest in the initial postoperative days. In our study, most cases of delirium and encephalopathy occurred within the first three postoperative days, likely linked to surgery-specific factors. Similar trends have been observed in non-cardiac surgery studies, such as that of Iamaroon et al. [44]

4.5. Recommendations

To establish BAMCOG as a reliable preoperative cognitive screening tool for POD prediction, further studies are needed to validate its efficacy. This involves setting appropriate cutoff scores for BAMCOG and assessing how factors like age and education influence results. A sample size of approximately 230 patients would allow for a reasonable estimation of both sensitivity and specificity of a diagnostic tool like DeltaScan, assuming a delirium prevalence of 20%, an expected sensitivity of 80%, and a confidence level of 95% with a margin of error of approximately 10%. Additionally, clinical validation should include confirming delirium diagnoses with experts like geriatricians or psychiatrists, despite the challenges of time and cost. Future research should also investigate BAMCOG’s role in detecting mild cognitive impairment (MCI), which was not a primary focus of this study. Finally, a study that conducts preoperative cognitive screening further in advance of surgery and evaluates subsequent prehabilitation programs will help validate our hypothesis that prehabilitation can reduce the incidence of POD.

4.6. Conclusions

Our findings indicate a limited predictive performance of the MoCA for anticipating postoperative Encephalopathy and Delirium. However, they indicate a good predictive performance of the BAMCOG for predicting postoperative delirium as detected by DOS. This suggests that preoperative cognitive screening with BAMCOG could hold promise in identifying patients who could benefit from preoperative prehabilitation and lifestyle interventions aimed at reducing the risk of postoperative delirium. While BAMCOG lacks clinical validation for diagnosing MCI, further research is necessary to establish its clinical utility as a preoperative cognitive screener. Future studies with larger sample sizes across diverse surgical patient populations and using optimal gold-standard delirium diagnoses are needed. Nonetheless, our study highlights the potential benefits of incorporating gamification in preoperative cognitive screening, opening new avenues for enhancing patient care and outcomes.