Figures
Abstract
Many individuals visit rural telemedicine centres to obtain safe and effective health remedies for their physical and emotional illnesses. This study investigates the antecedents of patients’ satisfaction relating to telemedicine adoption in rural public hospitals settings in Bangladesh through the adaptation of Expectation Disconfirmation Theory extended by Social Cognitive Theory. This research advances a theoretically sustained prediction model forecasting patients’ satisfaction with telemedicine to enable informed decision making. A research model explores four potential antecedents: expectations, performance, disconfirmation, and enjoyment; that significantly contribute to predicting patients’ satisfaction concerning telemedicine adoption in Bangladesh. This model is validated using two-staged structural equation modeling and artificial neural network approaches. The findings demonstrate the determinants of patients’ satisfaction with telemedicine. The presented model will assist medical practitioners, academics, and information systems practitioners to develop high-quality decisions in the future application of telemedicine. Pertinent implications, limitations and future research directions are endorsed securing long-term telemedicine sustainability.
Citation: Zobair KM, Sanzogni L, Houghton L, Islam MZ (2021) Forecasting care seekers satisfaction with telemedicine using machine learning and structural equation modeling. PLoS ONE 16(9): e0257300. https://doi.org/10.1371/journal.pone.0257300
Editor: Camelia Delcea, Bucharest University of Economic Studies, ROMANIA
Received: December 7, 2020; Accepted: August 29, 2021; Published: September 24, 2021
Copyright: © 2021 Zobair et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Telemedicine is a promising, growing beneficial approach to healing, providing remotely based support to the medically underprivileged and sparsely populated regions with inadequate access to health-based facilities. Despite the increasing demand for telemedicine, the adoption rate is below expectations [1]. Research on telemedicine initiatives demonstrates that it often fails to secure its deployment goals. Al-Samarraie, Ghazal, Alzahrani, and Moody revealed that 75% of projects are abandoned or incapable of last in Middle Eastern countries, which is as high as 90% in developing countries [2]. Within developed country’s context, Uscher-Pines et al. [3] reported that in the USA, the adoption rate is increasing < 20% of licenced treatment facilities offered telemedicine by 2019, but this rate remained lower than the adoption for telemedicine by other organisations; in 2016, 26% of all mental facilities, and 58% of all hospitals used telemedicine for various health conditions. Findings remain inconsistent, making informed judgement and conclusion about adoption issues difficult due to insufficient empirical evidence [4], particularly in sparsely populated rural communities [5].
Telemedicine growth has been concurrent with the growth of technological advancement in the 20th and 21st Centuries and sought to overcome distance in healthcare services delivery, particularly in the US and then replicated globally [6]. Telemedicine potentially improves accessibility, promotes service quality, reduces healthcare costs [7], and increases stakeholders’ choice and convenience. Many health professionals and care seekers welcome this innovative healthcare provision [7] and it’s increasingly being deployed in numerous clinical facilities [8] to sound effects on both developed and developing countries hospital settings. Telemedicine plays a pivotal role in reducing health disparity [9] between uneven health distributions in rural and urban settings, and is geared towards safeguarding health concerning steadiness.
The Bangladeshi Government has given higher priority to the integration of telemedicine into its public healthcare systems [10] to support rural and remote areas where around 70% of the population lives [11]. The formal public telemedicine healthcare systems were implemented in 2010 in rural Bangladesh [10] to make the partial fulfilment of the Digital Sonar Bangla Vision in 2021 a reality [12]. Despite the passion for public telemedicine schemes in Bangladesh, the functional adoption into clinical practices has persisted insufficient in rural settings. However, a recent study by Zobair et al. [10] highlighted the presence of 84 active telemedicine centres in 57 of Bangladesh’s 488 Upazila (i.e. subdistrict) public hospitals. The integration of 27 specialised, and district-level medical colleges and hospitals steadily providing specialist healthcare support to existing rural telemedicine centres is considered a momentous shift and one of the most significant developments in the public healthcare sector in Bangladesh [10].
Many satisfied care seekers are eventually expected to return to telemedicine for obtaining primary remedies concerning their mental and physical illness [13]. While care givers facilitate a variety of individually Information and Communication Technology (ICT) supported health services (i.e., telemedicine, telehealth, e-Health and m-health), individuals can actively participate in choosing what is more appropriate and conveniently satisfying to them [14]. For example, Anderson and Sullivan [15] reported that there is rising managerial interest in consumer satisfaction as a means of assessing service quality, which is believed to be the key indicator of organisational future reputations and growth. Hence, it is mandatory to understand consumers’ post-consumption evaluation of product/service quality, given pre-consumption expectations for forming their satisfaction judgement [15]. Literature indicates that patients’ satisfaction is a significant and influential indicator in healthcare intervention [16].
To date, past contributions concerning satisfaction with telemedicine have mostly focused on general measures of satisfaction. For example [6, 17–21], investigate antecedents to satisfaction with telemedicine without theorising or advocating novel techniques in the unique context of telemedicine. Although, a recent study by Kissi et al. [22] for instance, applied the technology acceptance model to predict physicians’ satisfaction with telemedicine. Interestingly, they point to research on the antecedents to physicians’ satisfaction with telemedicine, without validating satisfaction using Expectation Disconfirmation Theory (EDT), which remains a dominant method to study satisfaction proposed and developed by Oliver [23, 24]. Measured by the impact and amount of research conducted within satisfaction, it is fair to comment that EDT has been the dominant theory used to study satisfaction [25]. This implies that the existing satisfaction literature pertinent to telemedicine is not methodologically up to date in the way the investigations had been approached. This is supported by Whitten and Mair [26], who criticised that by digging beneath the surface of telemedicine research, the findings provide evidence that illustrates the futility of attempting to generalise satisfaction across all telemedicine frameworks. Nevertheless, telemedicine appears well accepted by rural health seekers in many developed countries [27], indicating their positive attitudes and evaluations for these services. Conversely, care seekers’ satisfaction judgments concerning telemedicine in developing countries could be based on different determinants, for example, service expectations, performance, disconfirmation, and enjoyment that actively contribute to the formation of the cognitive behaviours [28], stimulating their evaluative judgement of satisfaction decisions [29]. Following that positioning, a careful and thorough critique unique theoretical and methodological perspectives is warranted.
In contrast, Information Systems (IS) satisfaction literature [30–35] studied the antecedents to users’ satisfaction. Surprisingly, these authors have not applied EDT to explore how consumers’ expectations, performance and disconfirmation, contribute to forming their satisfaction judgements. Evidence suggests that the theoretical importance and widespread use of EDT in practice is highly important, as valid and reliable measures need to be ensured [36]. Another dominant antecedent to satisfaction is enjoyment (i.e., intrinsic motivation) from Social Cognitive theory (SCT) [37], but is consistently missing from many IS satisfaction articles. For example, a prior study by Venkatesh [38] revealed that intrinsic motivation role as a lever generating favourable user perceptions has not been adequately exploited. Subsequent empirical work by Vallerand [39] acknowledged that enjoyment and satisfaction are strongly allied.
From a methodological standpoint, some researchers theorised that satisfaction within telemedicine research had methodological and analytical limitations. For example, Kissi et al. [22] applied Technology Acceptance Model (TAM) to investigate health providers satisfaction with telemedicine, while Serrano et al. [40] applied EDT to identify patient satisfaction with telemedicine for diabetic retinopathy screenings. Surprisingly, they adopted a linear modelling approach. Venkatesh and Goyal [41] suggested that linear models are unable to reveal complexities that are expected in theories, and lead to oversimplifying the complexity of the joint effects of the components. In contrast, IS satisfaction research, for example [41, 42], applied a polynomial regression (a non-linear analysis) to better explained EDT theory. Following their work, this research applies a unique non-linear methodological approach using an Artificial Neural Network, recognizing both linear and non-linear relationships [43]. It is imperative to understand that both EDT and SCT are quite generalisable theories which have been applied to various contexts/settings. Thus, adding a new context of telemedicine may not truly add adequate contributions or values to the existing telemedicine literature, unless specific/alternative theorisations pertinent to the context of telemedicine are forwarded. Apart from the research conducted by Serrano et al. [40], scant research has been done on antecedents to patients’ satisfaction with telemedicine by employing EDT. This is surprising given this research focus on the context of customised theory building (integrated EDT and SCT), development, and testing pertinent to telemedicine. The authors anticipate that this alternative theorisation and unique methodological approach (i.e., Artificial Intelligence (AI)) will answer the crucial research questions by better grounding the phenomenon of satisfaction with telemedicine further unwrapping its contribution to uncover how antecedents to satisfaction contribute to predicting patients’ decisions towards future continuity of telemedicine.
Satisfaction incorporates individuals’ psychological and economic considerations related to product/service expectations and performance perceptions [44]. Albeit, measuring patients’ satisfaction related to telemedicine exposure is an onerous task due to interaction with numerous stakeholders (i.e., physicians, patients, nurses, and administrative and ICT staff) and technology stimuli [45]. These issues further compound the complexity of quantifying the comparability of satisfaction cognitions across all stakeholders (i.e., patients, and health providers) and the technologies (i.e., telemedicine) [44]. Hoque and Sorwar [46] for example, discovered that in Bangladesh ICT mediated healthcare services (i.e., m-Health), are investigated on the supply side (i.e., providing infrastructure), with little attention been devoted to the demand side (i.e., patients’ satisfaction). Another recent study by Anderberg et al. [47] reminded that there is a pressing need to study and predict care seekers’ behaviours towards the efficacy of telemedicine pursuits, which is becoming increasingly critical as the number of health technology solutions accelerates. Care seekers satisfaction in the present study refers to the extent of a patient’s evaluations resulting from a telemedicine service experience [48].
Evidence suggests that current healthcare services are becoming heavily reliant on technology [49]; thus, thoughtful research on predicting patients satisfaction with technology facilitated telemedicine appears critical for strengthening and sustaining this provision. Machine Learning (ML) and AI technologies have currently been gained from a wide range of special attractions for health informatics research. Shaikh et al. [50] for instance, indicated that AI particularly, Artificial Neural Networks have been used to predict future health risks. AI has been actively researched as helpful for humans [51] to transcend limitations. Vallée et al. [52] state that Artificial Neural Networks are valuable in enabling the interpretation of complex phenomenon, discovering new patterns and predicting outcomes. Artificial Neural Network (ANN) or Deep Neural Network (DNN) have seldom been used to predict patients’ satisfaction with telemedicine experience in emerging economies’ rural settings including Bangladesh.
Several classic theories have been proposed to better understand and explore the determination of consumers’ satisfaction judgements concerning products/services (i.e., Expectancy Disconfirmation Theory [24, 29, 53, 54], Assimilation Theory [29, 55–57], Equity Theory [29], Attribution Theory [29, 58], and Performance Theory [29]) in consumer satisfaction literature [29]. These satisfaction theories are rarely practised in ICT-mediated healthcare services, particularly in telemedicine research within developing countries context.
To our knowledge, no theoretically sustained prediction grounded research satisfactorily disaggregated the antecedents to care seekers’ (i.e., patients) satisfaction with telemedicine, particularly in emerging economies including Bangladesh. The existing Bangladeshi literature for instance, has not yet explored how antecedents of patients’ service continuity behaviour (i.e., expectations, performance, disconfirmation, and enjoyment) interact in forming, predicting and forecasting satisfaction with telemedicine experience. It is, therefore, imperative and essential to determine the elements (i.e., antecedents) that lead to form patients’ satisfaction, and to incorporate those antecedents are deemed crucial to predicting and forecasting future telemedicine adoptability and sustainability. To the authors’ knowledge, less emphasis has been devoted to this content in both developed and developing countries’ telemedicine research, while patients’ satisfaction experience significantly contribute to embedding their pre-consumption (i.e., expectations) and post-consumption (i.e., performance) cognitions [54] in the process leading towards their satisfaction judgement. Rarely do researchers in emerging economies reflect on this vital question. Moreover, there have been scantly reported investigations into integrated EDT and SCT and validated using a two-staged Partial Least Square Structural Equation Modeling (PLS-SEM) and DNN approach in the telemedicine domain, representing a considerable knowledge gap in the current health informatics literature. This research therefore investigates ways in which DNN can assist telemedicine to better enable predictions, increase patient recovery and satisfaction and support health professionals in better quality decision making.
Theoretical model and hypotheses
Consumers’ products and services satisfaction have captured considerable interest [59] in various consumption settings. A growing wave of interest in research on consumer satisfaction has provoked diverse reflective interpretations of the causes and effects of satisfaction cognitions [24]. A plethora of prior research on antecedents of satisfaction, for example [23–25, 29, 53, 59], confirmed that consumers’ products/services satisfaction is prominent in the field of marketing and plays a critical role in the formation of satisfaction [60] however, satisfaction decisions are complex [23] phenomenon as related to peoples’ consumption behaviours [57]. Within the context of product evaluation and consumer satisfaction studies, R. E. Anderson proposed four theories, namely, Cognitive Dissonance Theory, Contrast Theory, Generalised Negativity Theory, and Assimilation-Contrast Theory [55]. The study acknowledged that consumerism is an interrelated complex phenomenon that incorporates ecological, social, political, ethical, economic, and technological dimensions, which need to be studied separately. Another study by LaTour and Peat [59] reminded that extensive research is desirable to fully establish the antecedents of satisfaction associated with numerous kinds of products and services/systems.
Within the marketing context, Churchill Jr and Surprenant [61] determined that satisfaction serves a significant role culminating in purchases and consumptions and post-purchase phenomenon (e.g., attitude change, repeat purchase, and brand loyalty).
In the IS literature, Venkatesh and Goyal [41], for example, stated that expectations and disconfirmation (fulfilment of expectations) contribute to the formation of satisfaction judgement. Another IS study by Bhattacherjee [62] examined how individuals’ cognitive beliefs (i.e., satisfaction/dissatisfaction judgement) influence their continuance or discontinuance behaviours. For example, individuals’ service continuance behaviour is entirely dependent on their satisfaction with the service quality and the service usefulness [62]. To understand an individual’s satisfaction is crucial because it is a vital indicator of ICT success, a key predictor of continuance intention of ICT usage/adoption [63].
Conceptual definition of antecedents to satisfaction
Satisfaction is a complex concept that can be defined in a variety of consumption settings, including health information systems (i.e., telemedicine, telehealth, e-health and m-Health). Satisfaction is conceptualised as the state of enormously complex human thoughts embodied in any format of an evaluation of the targeted phenomenon. Oliver [64] defines satisfaction as ‘…the summary psychological state resulting when the emotion surrounding disconfirmed expectations is coupled with the consumer’s prior feelings about the consumption experience’. According to Tse and Wilton [65] satisfaction refers to ‘the consumer’s response to the evaluation of the perceived discrepancy between prior expectations (or some other norm of performance) and the actual performance of the product as perceived after its consumption’. A study by Yi [66] defined that satisfaction is ‘an evaluation rendered that the (consumption) experience was at least as good as it was supposed to be’. Within the IS context, satisfaction comprises consumers’ in-depth perception of pleasurable accomplishment of service, and the loyalty of commitment to the service provider [67]. For example, consumers compare their perception of product or service performance based on their expectations and some norms of performance [66] which contributes to the formation of satisfaction judgements.
Linking EDT, and SCT with care seeker’s satisfaction judgements will provide convincingly an informative way to explore how these antecedents interact and combine these perceptions with standards levels to form their satisfaction judgements [68]. Further, it may expand the generability and applicability of the EDT [69] and SCT frameworks [48]. We argue that EDT and SCT are justifiable because simplified EDT models are often unable to explain more complex phenomena, whereas extended framewoks can [63]. Further, this proposed research model (see Fig 1) identifies the associations between cognitive determinants (i.e., expectations, performance and disconfirmation) along with an affective (i.e., enjoyment) determinant contribute to satisfaction judgement [70]. Motivated by prior research on the EDT and SCT, and analysis of the proposed model postulating four antecednts to satisfaction, the following hypotheses are posited to investigate the research question.
Expectations
Expectations refer to ‘a set of pre-exposer beliefs about the product’ [41] or service. Expectations as ‘subjective notions of things to come or a type of hypothesis formulated by the consumer’ [57]. Within the IS context, Lankton and Wilson [48] defined that expectations are perceptions of forthcoming service performance that are commonly thought to reflect what an individual believes or anticipates. For example, within e-services, expectations are predictions related to information and service quality [48]. Another study by Zeithaml, Berry, and Parasuraman [71] confirmed that external communications to consumer and service quality information contribute to the formation of expectations. Oliver and Bearden [72] found that disconfirmation (i.e., the fulfilment of expectations) of expectations dominantly influence customer evaluation and judgement of product/service performance.
To measure assimilation effects on EDT research, Lankton et al. [69] noted that when assimilation effects occur; expectations and disconfirmation will have a positive impact on satisfaction. This indicates that an individual finds the differences between initial expectations and actual performance small enough to rely on their initial expectations when forming satisfaction judgements [69]. The links between disconfirmation and satisfaction represent a contrast effect, indicating that an individual finds initial expectations and performance significant enough that he/she relies on discrepancy during satisfaction evaluation [69]. Within telemedicine context, for instance, if a patient finds a positive discrepancy (i.e., performance is better than initial expectations), he/she feels happy and contented than if the discrepancy is negative (i.e., performance is worse than expected) [69]. Consistent with these findings, the following hypotheses are proposed:
- Hypothesis 1 (H1): Expectations positively influence patients’ satisfaction in telemedicine usage.
- Hypothesis 2 (H2): Expectations positively influences disconfirmation relating to telemedicine usage.
- Hypothesis 3 (H3): Expectations positively influences performance relating to telemedicine usage.
Performance
Performance in this study refers to an individual’s (i.e., patient) post-consumption perception (i.e., belief) about how the technology (i.e., telemedicine) performed on his/her pre-consumption expectation during the use period [63]. Performance refers to the extent of an individual’s expressive dimension of phycological preference [66]. For example, consumers compare (i.e., disconfirmation) their perception of product or service performance with a set of standards (i.e., expectations or some norm of performance) and perceived performance primarily functions for disconfirmation [66]. Within the service context, Halstead, Hartman, and Schmidt [73] revealed that performance and disconfirmation are conclusively related and are potential antecedents of consumers’ satisfaction. Within the e-Health adoption context, Lankton and Wilson [48] found positive relationships of performance and expectations with satisfaction.
Lankton et al. [69] confirmed that expectations help to provide a judgement about performance, while performance plays two roles. For example, performance positively influences disconfirmation holding expectations constant, indicating the higher performance perceived by an individual, the more likely performance will exceed his/her expectations, resulting in positive disconfirmation [69]. Also, performance plays a mediating role between expectations and satisfaction which can reflect the assimilation effect of performance on expectations [69]. Within telemedicine context for example, if a patient perceives the discrepancy between telemedicine service expectations and performance is small enough; performance should have a substantial mediating effect on his/her satisfaction judgement [69]. Consistent with prior studies, the following hypotheses are proposed:
- Hypothesis 4 (H4): Performance positively influences patients’ satisfaction in telemedicine usage.
- Hypothesis 5 (H5): Performance positively influences disconfirmation relating to telemedicine usage.
Disconfirmation
Disconfirmation occupies a central position in satisfaction research as a crucial intervening factor arises from discrepancies between prior expectations, and actual performance that generates satisfaction and dissatisfaction judgements [61]. Spreng, MacKenzie, and Olshavsky [74] revealed that disconfirmation judgements suggest that feelings of satisfaction arise when consumers (i.e., patients) compare their perceptions of the performance of services according to their expectations. Disconfirmation in the present study refers to the cognitive comparison between a patient’s pre-consumption service expectations and post-consumption telemedicine service performance. Halstead et al. [73] found that consumers comparison standards (i.e., expectations), disconfirmation beliefs, service performance, quality, affective responses, attribution, and equity judgements, are dominant sources and predictors of satisfaction judgement. Another study by Anderson and Sullivan [15] acknowledged that disconfirmation is a dominant antecedent of satisfaction. Consistent with prior studies, the following hypothesis is proposed:
- Hypothesis 6 (H6): Disconfirmation positively influences patients’ satisfaction in telemedicine usage.
Enjoyment
The integration of individuals perceptual, evaluative, and psychological cognitions generate their consumption satisfaction [66]. Evidence suggests that the enjoyment construct was extracted from Social Cognitive Theory (i.e., form of affect) [48]. According to SCT, enjoyment or attitude represents an individual’s affective reactions towards his/her behaviour and feelings of pleasantness/unpleasantness [48]. Consistent with Lankton and Wilson [48], enjoyment in this study refers to the perception that the use of telemedicine services will be enjoyable/pleasant in its own right. ‘Enjoyment is characterised by a sense of novelty or accomplishment’ [75]. Within technology adoption, Trevino and Webster [76] reminded that if an activity ‘feels good’ and is intrinsically motivating, individuals are more likely involved in that activity for its own sake, while Venkatesh [38] defined that ‘the pleasure and inherent satisfaction derived from a specific activity’. Within e-health, Lankton and Wilson [48] for instance, asserted that if individuals perceive the services enjoyable it will increase their expectations of service performance, while Yi [66] noted that both expectations and performance contribute to forming their satisfaction judgements. Consistent with these findings, the following hypotheses are proposed:
- Hypothesis 7 (H7): Enjoyment positively influences patients’ satisfaction in telemedicine usage.
- Hypothesis 8 (H8): Enjoyment positively influences expectations in telemedicine usage.
Research methodology
This research has drawn from a large project, which predominantly explores barriers, facilitators and expectations of telemedicine adoption in rural public hospitals settings in Bangladesh. This study was shaped in quantitative analysis and divided into two sections. The first section includes comprehensive literature review to develop study’s conceptual model. The second section aims to evaluate the questionnaire, empirically validate the proposed research model and hypotheses using a two-staged PLS-SEM and DNN approach. SmartPLS v.3.2.7 was used for data analysis. We believe that applying DNN will be useful in recognizing both linear and non-linear relationships [43] related to normality, linearity, and homoscedasticity [77].
Literature indicates that neural networks learns complex relationships among variables to provide solutions to difficult problems; and consistently reliable in terms of accuracy of findings [78]. Tan et al. [77], revealed that ANN is estimated to be a more robust method for providing with higher prediction accuracy and outperforming other conventional regression analyses. However, due to its ‘black box’ operational nature it is inappropriate for validating hypothesis of casual relationships [77, 79] thus; PLS-SEM approach fills the position.
Within the interoperability and explainability context, this study employed three different explanation methods to validate the estimation of the variables’ contributions in the “black box” network model [80]. The interpretability methods used include Shapley Additive exPlanations (SHAP) [81], Local Interpretable Model-Agnostic Explanations (LIME) [82] and Gradients [83]. Literature indicates that Explainability methods often vary in performance according to domains of application. For example, Lundberg and Lee [81] revealed that the Shapley method is persistently appropriate to better interpret the predictions of complex models in various tasks while, Atanasova et al. [84] showed that gradient-based explanations perform best for natural language processing tasks. LIME and SHAP have shown better performance as general-purpose model explanation methods [81, 82]. We believe that applying the three methods will be advantageous in obtaining a better estimation of variable importance and predictive accuracy in our model. These methods are essential to interpret a prediction model [81] correctly, and they compute an estimation of the contribution of each feature for the predictions [85]. This research utilises the Python programming language along with the Python Standard Library consisting of NumPy, Keras, Sklearn, and Pandas to develop the proposed Neural Network model. Further, this study uses the TensorFlow library, SHAP, LIME, and GradientTape to implement the interpretability and explainability methods.
Instrument development
To test the research model and hypotheses, a questionnaire was developed, and constructs of the conceptual model were assigned [10]. Primarily, a conceptual model with key constructs, corresponding indicators, and scales was synthesised from existing, established research (see S1 Appendix) [86], and extended/adapted to fit the proposed study [87, 88]. It was then further refined through data collected from participant observation, group discussions, and in-depth interviews before finalising the constructs, indicators, and scales, leading to the establishment of a survey instrument to collect data to validate the hypotheses and the model empirically [89]. A seven-point Likert scale was utilised (i.e., 1 = very strongly disagree to 7 = very strongly agree) [48], which persisted appropriate scale for healthcare surveys [90]. A group of management information systems professionals examined the survey questionnaire for logical consistency, contextual relevance, terminology and measurement content clarity [91]. A pre-test was conducted to examine the wording, sequence, length and format of questionnaire items [92]. A group of fifteen Bangladeshi PhD students were invited to pre-test the questionnaire due to their academic skills, research experience and knowledge regarding Bangladesh telemedicine services. Using their feedback, an iterative rectification process was performed until the questionnaire reached an acceptable range of criterion validity [10]. The questionnaire was initially developed in English, translated into the local language (Bangla), retranslated into English and adjusted so that both versions were comparable to ensure internal validity [93]. The survey questionnaires were distributed with written consent forms to respondents who agreed to participate in the survey. The consent forms reinforced that participation was voluntary and they could withdraw their involvement anytime. The respondents returned the completed questionnaires and the signed consent forms.
Sample and data collection
The survey process commenced after receiving written approval from the Directorate General of Health Services (DGHS), Ministry of Health and Family Welfare (MOHFW) of Bangladesh and ethical clearance from Research Ethics and Integrity authority at Griffith University, Australia. A cross-sectional survey of telemedicine users was conducted in 2017 in three Upazilas (sub-districts) telemedicine centres in Bangladesh. For this study, 500 rural patients who received telemedicine services at least once from any selected telemedicine centres in the past 12 months constitute the sampling frame. The study excluded non-users because patients in Bangladesh cannot access to telemedicine services without a physician’s referral. Non-telemedicine users including those unable to communicate effectively in any form were excluded from the study. Comprehensive data collection procedures have been described previously [10]. The sample was drawn from selected telemedicine centres using a multistage random sampling design. At first, three districts—Pabna, Khulna and Satkhira where telemedicine services are available were selected randomly. From these three districts, three Upazila telemedicine centres—Bera, Dacope and Devhata were randomly selected as survey implementation site. From the patient lists collected from the selected telemedicine centres, 500 users were randomly selected, consisting of proportionate samples from Bera (n = 53), Dacope (n = 242), and Devhata (n = 205) Upazila telemedicine centres. Each group was statistically representative of the telemedicine population with commonalities in telemedicine infrastructure and clinical methods provided by the government. It is imperative to note that telemedicine in public hospitals has not yet been adopted in every part of the country.
Patients’ addresses and phone numbers were collected from the selected telemedicine centres. Eligible individuals were contacted by phone and invited to participate in face-to-face interviews at a telemedicine centre during office hours. Those unable to travel to the centres were asked to participate according to their convenience. A closed-form interviewer-administered questionnaire was used because several questions were relatively technical and would be hard for respondents with limited literacy to interpret on their own. The study met the sampling target by obtaining 500 valid responses that were scrutinised for completeness. Eight samples were excluded due to incomplete responses; 492 samples were preserved for analysis. The survey’s demographics are presented in Table 1.
Data analysis and results
Measurement model
The measurement model was evaluated by testing internal consistency reliability, indicator reliability, convergent validity, and discriminant validity [94]. Except for DISC 4 (0.680), all standardised outer loadings for each indicator in the model (see Table 2) were higher than the threshold value of 0.70, confirming indicator reliability [94, 95]. The outer loadings of indicator DISC 4 (0.680) are close to the threshold value of 0.70 and are confirmed acceptable. The indicators’ outer loadings between 0.40 to 0.70 should only be considered for removal if this increases composite reliability and AVE above the threshold value [94]. Both Cronbach’s alpha and composite reliability (> 0.70) (see Table 2) confirmed the model’s statistical significance and established strong evidence of internal consistency reliability [94]. The Cronbach’s alpha for ENJ (0.67) is less than the threshold value of (> 0.70) but is considered acceptable in exploratory research ranging from 0.60 to 0.70 [94]. Convergent validity (≥ 0.50) was evaluated using the average variance extracted (AVE) values for each construct (see Table 2) [94]. AVE values were higher than > 0.50, demonstrating high levels of convergent validity [96]. The AVE values for each construct explain the variance of more than half of their corresponding indicators, validating convergent validity [95, 97].
Discriminant validity defines the extent to which a construct in a model is distinct from other constructs by empirical standards [97]. A construct accepts more variance from its assigned items than from any other constructs [97]. The findings (in bold) in the correlation matrix (see Table 3) confirm that the square root of the AVE for each construct is higher than the correlation with other constructs confirming acceptable discriminant validity [94].
Additionally, this study measured the distinctiveness of a latent construct using the heterotrait-monotrait ratio (HTMT) criterion test. Literature indicates that the HTMT criterion is essential to evaluate the constructs’ discriminant validity [98–100]. The cut-off scores should be smaller than 0.85 (more strict threshold) or 0.90 (more lenient threshold) to interpret the results [98, 99]. The results (see Table 4) demonstrate that HTMT is significantly less than 0.85 or 0.90, authenticating that all measured constructs illustrated their discriminant validity [98] except two pairs. Our results provided evidence that we established discriminant validity among all constructs. However, we cannot confirm discriminant validity between two pairs of constructs such as ENJ and DISC, and EXP and ENJ contain the HTMT value of 0.925 and 0.910, respectively, suggesting a lack of discriminant validity.
Further, the bootstrapping technique was applied for testing whether the HTMT value is significantly less than 1 [94]. Our results demonstrated that HTMT is considerably less than 1 [98]. We found that neither of the confidence intervals includes the value 1 [94] except ENJ and DISC, EXP and ENJ, similar to the previous test. This indicates that the lower and upper bounds of the confidence interval of HTMT for the relationships between ENJ and DISC are 0.925 and 1.007, EXP and ENJ are 0.910 and 1.007, thus below the threshold value of 0.90 or significantly smaller than 1 (95% percentile confidence interval), suggesting a lack of discriminant validity [94]. We could not establish discriminant validity between these constructs in HTMT test. These results are unexpected and need further investigation.
Structural model
The structural model confirmed that five hypotheses were validated by the relationships (p < 0.01) (see Fig 2). The four proposed latent constructs had significant effects (see Tables 2 and 5) on patients’ satisfaction (SAT) related to telemedicine usage and adoption. The findings demonstrate that the relationships between expectations (EXP) and satisfaction (SAT) (β = 0.350, t = 6.089, p < 0.01), performance (PERF) and satisfaction (SAT) (β = 0.155, t = 2.833, p < 0.01), disconfirmation (DISC) and satisfaction (SAT) (β = 0.184, t = 3.773, p < 0.01) and enjoyment (ENJ) and satisfaction (SAT) (β = 0.168, t = 3.030, p < 0.01) were statistically significant, confirming support for H1, H4, H6 and H7 (see Table 5). An R2 ≈ 0.53 indicates that about 53% of the variance (i.e., SAT) in the model was jointly explained by the EXP, PERF, DISC and ENJ constructs. The results were between moderate and substantial R2 values (i.e., 33%, 67%) [97], suggesting a high predictive capability of this satisfaction model. Further, R2 demonstrates each construct’s significance and its associative contribution to overall R2 [101].
The structural model further revealed that four other hypotheses were statistically significant and validated. The relationships between expectations (EXP) and disconfirmation (DISC) (β = 0.533, t = 11.756, p < 0.01), expectations (EXP) and performance (PERF) (β = 0.530, t = 14.086, p < 0.01), performance (PERF) and disconfirmation (DISC) (β = 0.278, t = 6.201, p < 0.01), and enjoyment (ENJ) and expectations (EXP) (β = 0.661, t = 21.016, p < 0.01), were statistically significant. This validates H2, H3, H5, and H8 (see Table 5) demonstrating that these constructs have a substantial influence on their endogenous latent constructs.
The practical relevance of significant effects should be investigated by testing the effect sizes of the latent constructs’ relationships [98]. Literature indicates that the f2 values ranging from 0.020 to 0.150, 0.150 to 0.350, or larger or equal to 0.350, demonstrating weak, medium or large effect size, respectively [98]. The f2 value (see Table 5) for all hypothesised relationships ranges from 0.027 to 0.777 (weak to large) in our sample. The effect size is a measure of the magnitude of an effect independent of sample size, and it is unusual and unlikely that most constructs will have a large effect size in the model [98].
Additionally, the present study assessed overall saturated model fit evaluation. Due to recent PLS-SEM developments, the overall model fit can be estimated using standardised root mean squared residual (SRMR), squared Euclidean distance, and the geodesic distance [99]. The measure of fit (SRMR) and the test of overall model fit squared Euclidian distance (dULS) and the geodesic distance (dG) is preferable in casual research [98, 99]. The discrepancy between the two matrices is measured by squared dULS, dG, and, the SRMR [98, 99] assuming, at a 5% significance level [98]. The recommended threshold value of SRMR should be below 0.080, and all discrepancy measures (dULS and dG) should be below the 95% quantile of their reference distribution (HI95), indicating that the estimated model was not rejected at a 5% significance level [98]. In our model (see Table 6), all values of discrepancy were below the 95% quantile of their corresponding reference distribution (HI95) except (dULS) authenticating acceptable overall model fit. Further, the SRMR is below the threshold value of 0.080, indicating a good model fit [98]. This test has recently been developed in PLS-SEM and IS researchers are encouraged to use this evaluation in casual research [98, 99].
Deep neural network analysis
Machine learning enables to develop any complex model to obtain new knowledge through an iterative process of learning with structured data pre-processed from big data [43]. Deep learning is a machine learning approach attracting much attention from academic and industrial communities [102]. It has been widely used in many theoretical and practical fields of investigations [103] including computer vision, natural language processing [104], image recognition, language translations [105] and health trends and predictions [43]. The DNN comprise an information processing structure consisting of multiple layers and interconnected neuron units with fully functional relationships [106].
The configuration of our proposed DNN model (see the network model in Fig 3 consists of one input layer X with four input variables namely, performance (X1), expectations (X2), disconfirmation (X3), and enjoyment (X4), seven hidden layers (h1 − h7) with seventeen (N = 17), eighteen (N = 18), seventeen (N = 17), twenty (N = 20), seventeen (N = 17), nineteen (N = 19), and seventeen (N = 17) neurons respectively. Neural network dynamics are set in motion when neurons in the first hidden layer receive inputs (as constant values in tensors or arrays) from the input layer. The input values are multiplied with corresponding weights and added to the bias to generate intermediate values. The intermediate values are then transformed using a non-linear activation function [107]. The outputs of the activations from a layer serve as the inputs for the next layer. The process of intermediate value and activation computations are performed at each layer until the output layer [78]. The central part of DNN functionality is the learning and training procedure in which the errors determined at the output layer are successively reduced by adjusting the weights and biases throughout the network [108]. Through iterative weights adjustment, the network learns the best set of weights to estimate the target variable, in our case satisfaction [43]. The principle of adjusting weights is to reduce the errors and optimise the classification outcomes of the network model [109].
Wij represents the parameter matrix consisting of weights from layer i to layer j. The dimension of each weight matrix is give in ().
One of the central challenges in DNN model development is the best model architecture identification and selections. Model selection is the process of selecting the hyperparameters of the best-performing model. The hyperparameters must be set manually after finding the optimal hyperparameter configuration as per the model selection process [110].
The present study determined the hyperparameters of the model using a random search strategy (using the Keras-tuner library). The random search strategy is used to seek out the optimal neural network model [111]. We designed the random search consisting of eight runs with randomly assigning 4 to 20 neurons to each layer. Each run included 250 trials in random order (see the search results in Tables 8 to 15). The detailed hyperparameter search space is shown in Table 7. In each run, we gradually incremented the number of hidden layers until we obsereve no decrement in loss (see Table 15). Literature indicates that the random search method has recently become a popular alternative to grid search [111]. Choosing a random search for this research is a more efficient method than a grid search. Typically, only a subset of a model’s tuneable hyperparameters is vital for optimising performance [111]. Thus, we chose a random search technique for the hyperparameters optimisation using three runs, 250 experiments each with 1000 epochs.
Top 5 Models out of 250 candidate models from Run 1.
Top 5 Models out of 250 candidate models from Run 2.
Top 5 Models out of 250 candidate models from Run 3.
Top 5 Models out of 250 candidate models from Run 4.
Top 5 Models out of 250 candidate models from Run 5.
Top 5 Models out of 250 candidate models from Run 6.
Top 5 Models out of 250 candidate models from Run 7.
Top 5 Models out of 250 candidate models from Run 8.
The RMSE scores obtained by the top five models, out of 250 trials in each Run, are shown in Tables 8 to 15. We observe that Model 1 in Run 7 is the best-performing model. It is worth noting that this study tested several activation functions, including ReLU, PReLU, LeakyReLU, ELU, and tanh. Literature indicates that these activation functions possess advantages and disadvantages [112]. However, we found that the tanh activation function outperforms others in our experiments. Literature indicates that tanh is a commonly used activation function and leads to faster convergence during training [113].
The Hyperparameter search results of Run 7 are exhibited in Table 14 where the RMSE scores obtained by the top 5 models are 0.07719, 0.08734, 0.08783, 0.09711, and 0.09879 respectively, showing that Model 1 is the best-performing model.
Our findings demonstrated minimal or no differences between train and test errors (see Table 16), confirming a non-overfitting and non-underfitting model [108]. The neural network’s efficiency and prediction accuracy is measured by the root mean square error (RMSE) [114]. The findings (see Table 16) confirmed that the values of RMSE with minimal training and testing errors (see Table 16), indicating an optimal fit DNN model.
This study employed SHAP which is a game-theoretic approach that uses shapely values [85]. Shapely values express the average contributions of each player in a team to the outcome of a game, in our case contributions of each feature to the prediction. LIME builds a linear regression model from perturbed samples similar to a test instance and assigns contribution scores to the features using a LASSO regularisation method [82]. Gradient (of a model with respect to the input variables) draws a measure of variation which attributes the degrees of changes of a prediction to the input variables [80]. In the above methods, a higher score indicates the corresponding variables is more important than the variables with lower scores.
Note that these are local interpretation methods, meaning that the explanations provided are for a single instance. In the following, represents the local importance of the jth variable for the ith instance and
represents the global importance of the jth variable. In SHAP Ij is computed by summing the absolute shapely values across the instances [85] which is given by:
where, n is the total number of instances. LIME provides global importance by a module named SP-LIME [82] where the importance of the jth variable is computed as:
In gradient based method, the global importance Ij is computed as [80, 83]:
Here,
is the gradient of a model f(X) with respect of the jth variable. We obtain the gradient
using GradientTape.
SHAP, LIME, and Gradient-based interpretability assessment
The eXplainable Artificial Intelligence (XAI) methods have gained popularity in both academia and industries to interpret internal logic and the outcome of “black-box models” [115]. A plethora of current literature indicates that various interoperability methods exist in machine learning; however, it is often unclear how these approaches are related and preferable over another [81]. These approaches differ in feature importance measures. From a functional point of view, LIME fits approximate model predictions locally [116]; SHAP measures the entire class of additive feature attributions both locally and globally [81]; Gradients based explainability methods link features importance variability by measuring the change in gradients of the outputs with respect to the inputs [116]. Due to the functional variability, the results of LIME, SHAP and Gradient-based methods can vary. Slack et al. [117] have shown that LIME is more unstable than SHAP due to the random input perturbations used by LIME to construct the training set for explanation extraction. Likewise, Lai et al. [116] observed variations in the outputs of SHAP and LIME methods.
Given the emphasis on the interpretability and explainability of the Machine Learning models, specifically DNNs, this study experiments with comprehensive interpretations using SHAP, LIME, and Gradient-based methods. The magnitudes of the outputs of the methods associated with each feature represent how significant the feature’s contribution is to the prediction of a model [115]. The outcomes of SHAP, LIME, and Gradient-based methods on our study are presented in Figs 4–6, respectively. The greater the magnitude, the more significant its contribution to the model’s prediction [115].
Note. X1 = performance; X2 = expectations; X3 = disconfirmation; X4 = enjoyment.
Note. X1 = performance; X2 = expectations; X3 = disconfirmation; X4 = enjoyment.
Note. X1 = performance; X2 = expectations; X3 = disconfirmation; X4 = enjoyment.
The findings from the SHAP, LIME and GradientTape analyses (see Figs 4–6) show that X2 (i.e., expectations) is the most dominant contributor, while X3 (i.e., disconfirmation), X4 (i.e., enjoyment) and X1 (i.e., performance) are second, third and fourth-order contributors respectively. SHAP produces slightly different orders for X4 and X3 (however, the magnitudes of X3 and X4 are very close). Except for the scarcely ordered variability in one instance, all three SHAP, LIME, and GradientTape methods are consistent on the features’ contributions to predicting care seekers’ satisfaction with telemedicine experience in rural Bangladesh settings. Nonetheless, our findings still imply that each construct (i.e., performance, expectations, disconfirmation and enjoyment) is a strong determinant regardless of high, medium, or low magnitudes, predicting health seekers’ satisfaction with telemedicine services in rural Bangladesh.
Discussion of the results
The empirical findings from DNN and PLS-SEM analyses, authenticated that four antecedents—expectations, performance, disconfirmation, and enjoyment—play vital roles in forming, predicting and forecasting care seekers’ satisfaction with telemedicine, laying the groundwork for further policy intervention and research.
Based on the DNN model the findings provide evidence that expectations had the most significant effect on satisfaction making it the strongest predictor of care seekers’ satisfaction with telemedicine. This model illustrates that higher levels of patients’ satisfaction are reported in efforts to rationalise their earlier high expectations associated with telemedicine services [53]. This indicates that expectations remain crucial for assessing and estimating patients’ satisfaction levels. In keeping with the findings from PLS-SEM we have confirmed that expectation is the dominant predictor of satisfaction with telemedicine. This verifies Bearden and Teel [53] observations, that expectations are presumed to accomplish the function of an adaptation level in the way they defined the standard (i.e., expectations) against which subsequent performance is being judged. Our results corroborated by Bhattacherjee [62] revealed that expectation is a dominant antecedent contributing to the process of satisfaction formation. Applied to telemedicine adoption success, our findings tie in with a recent study by Zobair et al. [10] and acknowledged that expectations could be compared with a baseline of patient satisfaction with telemedicine experience. Bhattacherjee [62] found satisfaction is a dominant predictor of their usage continuance. The research findings are consistent with prior studies [44, 62, 69, 118], validating that expectations is a leading antecedent contributed to forming and predicting satisfaction with telemedicine in rural hospital settings in Bangladesh.
The findings from DNN analysis provide evidence that disconfirmation is the second highest antecedent predicting satisfaction which is congruent with the PLS-SEM analysis. The fact that both approaches are statistically significant, provide evidence that disconfirmation has dominating effects on satisfaction with telemedicine. The research findings are congruent with Lankton et al. [42] observations, that individuals compare their initial expectations with perceived service performance fostering satisfaction judgement, while Spreng and Page Jr [36] for example, claimed that disconfirmation is a distinct cognitive state that is subjectively perceived by an individual which can be measured independently. Further, our findings validate Ho and Wu [119] remarks, that positive disconfirmation has a substantial effect on satisfaction in using the Internet, while Lankton and McKnight [63] recognised that positive disconfirmation (i.e., usefulness and ease of use) influences consumers’ satisfaction judgement. This indicates that pshchological dimension of satisfaction provides a means that wheather an individual’s expectations are confirmed or disconfirmed by their perceived performance [44]. Consistant with this view, this study argues that care seekers optimise the discrepancy between their perceived pre-consumption expectations and post-consumption performance to control their satisfaction judgement pertinent to telemedicine. These imply that patients’ satisfaction is distinct from their cognitive disconfirmation process [73] which in turn affect their satisfaction judgements related to telemedicine experience. From these research findings, we conclude that disconfirmation is a strong antecedent that played a key role to predict patients’ satisfaction with telemedicine.
From a hedonic point of view, the findings from DNN and PLS-SEM analyses provide strong evidence that enjoyment has a significant impact on satisfaction (third highest position in both PLS-SEM and DNN analyses), making it another dominant predictor of patients’ satisfaction with telemedicine. This implies that patients’ satisfaction judgement is formed by their positive perceptions and enjoyment concerning high service quality and reliability. This is particularly relevant to telemedicine service processes consisting of several attributes (i.e., video conferencing, health providers attitude, long waiting) which may be perceived enjoyable or unenjoyable turnings into patients’ satisfaction or dissatisfaction. The research findings are largely congruent with Venkatesh [120] observations, who found strong associations between perceived pleasure and satisfaction towards performing the behaviour in technology adoption. This study argues that enjoyment plays a vital role in the formation of satisfaction indicating that patients’ enjoyment raises, the levels of satisfaction also increase concurrently. Our findings appear rational and expected since we argue that patients’ emotions and feelings strongly affect the level of their cognitive influence on satisfaction judgement. A similar pattern of findings by Davis, Bagozzi, and Warshaw [121] voiced that enjoyment has a dominant effect on technology usage intentions. These findings support Carlsmith and Aronson [122] remarks, that dissonance (psychological discomfort) influences individuals’ hedonic levels related to their satisfaction judgement. Our findings show the evidence that enjoyment is a leading antecedent to predicting patients’ satisfaction with telemedicine in rural hospital settings in Bangladesh.
Findings from PLS-SEM and DNN analyses reveal that performance is the fourth contributor to forming patients’ satisfaction with telemedicine. The DNN model has comparative advantages to interpret the prediction of complex systems better and is congruent with Tan et al. [77] statements that DNN model is estimated to be a more robust method for providing higher prediction accuracy and outperforming other conventional regression analyses. Both the DNN and PLS-SEM approaches confirm that performance has a significant impact on satisfaction authenticating its statistical significance. Our findings support Serrano et al. [40] statement that service performance strongly influences patients’ satisfaction, while Lankton and Wilson [48] found that performance has a dominating effect on patients’ satisfaction judgement. Similar findings by Lankton and McKnight [63] echoed that service performance positively influences satisfaction and has the most significant impact on continuous intentions. We argue that if telemedicine consistently delivers high-quality services (i.e., performance) it is more likely to have satisfied care seekers resulting in the continuance of usage. A similar conclusion highlighted by Garcia and Adelakun [45] acknowledged that convenience (i.e., performance) and appropriate telemedicine service profoundly influence patients’ satisfaction. These findings are largely consistent with prior studies [24, 29, 36, 48, 66, 73, 123], confirming that performance is a dominant antecedent to patients’ satisfaction with telemedicine.
Our findings authenticated that four antecedents—expectations, performance, disconfirmation, and enjoyment—play vital roles in forming, predicting, and forecasting care seekers’ satisfaction with telemedicine. They differ from other satisfaction studies within the telemedicine literature. Existing literature [7, 17, 45, 124] investigated satisfaction in the context of telemedicine but did not (surprisingly) apply EDT theory to explore how consumers’ expectations, performance, and disconfirmation contribute to their satisfaction judgments. They point to research on the antecedents to satisfaction with telemedicine, without validating satisfaction using EDT, which remains a leading theory to study satisfaction proposed by Oliver [23, 24]. This implies that telemedicine’s existing satisfaction literature is not theoretically up to date in the way the investigations had been approached, particularly in emerging economies’ context. Against this backdrop, the study’s findings fill these significant gaps contributing to health informatics and behavioural literature by clarifying the complex interplay between patients’ satisfaction and determinants of continuity behaviour in telemedicine’s domain.
From a methodological point of view, some researchers theorised that satisfaction within telemedicine research had methodological and analytical limitations. For example [22], applied Technology Acceptance Model (TAM) to investigate health providers satisfaction with telemedicine, while Serrano et al. [40] applied EDT to identify patient satisfaction with telemedicine for diabetic retinopathy screenings. Interestingly, they adopted a linear modelling approach. Venkatesh and Goyal [41] suggested that linear models are unable to reveal complexities that are expected in theories and lead to oversimplifying the complexity of the combined effects of the components. Following their work, this research applies a unique non-linear methodological approach using Deep Neural Network (DNN), recognising both linear and non-linear relationships [125].
Additionally, this research used the PLS-SEM method, a full-fledged variance-based approach ideal for linear, non-linear, recursive, and non-recursive structural models [98]. The present research investigates how DNN, and PLS-SEM can assist telemedicine in enabling predictions better, increasing patient recovery and satisfaction, and supporting health professionals in better quality decision-making. Hence, our findings confirm that the four constructs (i.e., expectations, performance, disconfirmation, and enjoyment) significantly affect predicting patient satisfaction decisions and subsequent future continuity with telemedicine in rural settings. Failure to recognise these determinants and impacts on health seekers’ satisfaction with telemedicine will undervalue attempts at implementation in emerging economies.
Contributions and managerial implications
The theoretical and methodological contributions of this research illustrate the importance of customised theorisation of existing EDT and SCT along with the introduction of unique Machine Learning and Artificial Intelligence technology to answer the crucial research questions by better grounding the phenomenon of satisfaction with telemedicine. This raises satisfaction’s contributory benchmark to uncover how its antecedents contribute to predicting patients’ decisions towards telemedicine’s future continuity. Second, the apparent strength of this research is the large sample size and the interpretations of a range of key contributors that play a vital role in strengthening and widening telemedicine scope in developing countries’ rural settings. Third, this research explores the complex interplay between satisfaction and its antecedents illuminating novel insights into predictions of patients’ future telemedicine usage trends thereby assisting health professionals, academics, policymakers, and IS community to higher quality informed decisions for people-centred future models of care. Fourth, this research complements a body of new knowledge related to patients’ unique behavioural characteristics towards their satisfaction judgements and delivered additional insights into effective decisions strategies to pursuit successful telemedicine deployment in large scale. Fifth, this research employs a two-staged DNN and PLS-SEM approach to delineate the elements of satisfaction that persistently contribute to imparting newfound knowledge and best practices in both developed and developing countries contexts, validating a methodological contribution. Sixth, the inclusion of three interoperability and explainability methods (SHAP, LIME and Gradient) has made a significant contribution to promoting accurate interpretation and better estimation of the factors/variables’ contributions and predictions accuracy in the network model. Additionally, this research opens a new avenue for Machine Learning and Artificial Intelligence-based health informatics research and provides valuable directions to academic and IS practitioners to integrate approaches to strengthen telemedicine sustainability.
From a managerial standpoint, this research explores distinctive patients’ cognitive and affective nature within the healthcare environment that has theoretically been linked to satisfaction [126]. The findings from this research illustrate that care seekers’ expectations are presumed to accomplish the function of adaptation to new conditions/services in the way they defined the standard against which subsequent performance is being judged, while high service performance positively impacts on their continuance intentions. This suggests that health providers should strive to provide high-quality services leading to satisfied patients resulting in their retention. Further, health providers need to prioritise patients’ coordination, simplistic referral process to specialised physicians, prompt service delivery, and supportiveness that lead to patients positive impression (i.e., disconfirmation) towards future service continuity. This implies that the psychological dimension of satisfaction mediates whether an individual’s expectations are confirmed or disconfirmed by their perceived performance. This research proves that enjoyment plays a vital role in the formation of satisfaction, indicating that as patients’ enjoyment increases, the levels of satisfaction also increase. Given these findings, the health providers, and policymakers should design effective strategies, develop favourable policy guidelines and implement satisfaction management plans for achieving goals.
From a practical point of view, these findings have important implications for information systems, particularly in health informatics research. The discoveries associated with theory and practices complement each other well in concept generation and development of telemedicine healthcare systems in both developed and developing countries rural settings. Our proposed model emerged as a dominant benchmark model, played a strong role in forming, predicting and forecasting patients’ satisfaction with telemedicine and similar ICT facilitated healthcare systems such as e-Health, telehealth, and m-Health. Furthermore, several issues pertaining to the intricate patterns of antecedents to patients’ satisfaction with telemedicine were deliberated, to shed new lights on fostering the deployment of this provision in emerging economies. Additionally, the validity, robustness and functionality of this robust model has been confirmed, signifying implications across health industries to fostering its effective deployment in Bangladesh and globally. However, it is apparent that more in-depth investigations further required in this context.
Limitations and future research directions
Nevertheless, some limitations do exist in this study and should be considered when interpreting findings. First, this study reflected only on the prediction of care seekers’ satisfaction with telemedicine. Future research should examine the health providers’ satisfaction with telemedicine, this may provide additional evidence. Second, only public telemedicine systems were included in the current study. Private and NGO provided telemedicine systems could be involved in future research to observe the variances between public and private service interests. Third, this study focused on a few antecedents to care seekers’ satisfaction with telemedicine. Future research could be broadened by including age, gender, education, economic conditions, health literacy and situational effects that may significantly influence patients’ satisfaction related to telemedicine provision. Fifth, data for this study were collected in 2017, demonstrating a significant limitation. The time-series data can be collected for future research that may provide additional evidence related to telemedicine satisfaction. Finally, the present study was limited to Bangladesh. Combining this study with cross-sectional data from similar settings would provide extensive understandings of patients’ satisfaction with telemedicine in a global context.
Conclusion
This research successfully verified and determined the potential antecedents that play key roles in predicting and forecasting patients’ satisfaction with telemedicine experience in rural public hospitals in Bangladesh. The apparent strengths of this research are to effectively introduce artificial neural network along with PLS-SEM to explore the elements of intricate patterns of patients’ satisfaction judgements with telemedicine experience, in order to boosting the ability of effective organisational decision making to respond more rapidly to future healthcare problems. The integration of EDT and SCT has promoted a complete theoretical view of satisfaction formation process in telemedicine framework. Our study, which validates that expectation and performance are predictors of disconfirmation, while disconfirmation accurately captures an individual’s judgement of their perceived discrepancy between expectations and performance [36]. Congruent with the findings from Davis et al. [121], we observed that enjoyment dominantly contributes to forming patients’ satisfaction decision towards telemedicine usage. These findings underscore the importance of how cognitive factors (i.e., EDT) and emotional (i.e., affective) factor (i.e., SCT) remarkably play critical roles in the formation of satisfaction. This indicates that abandoning enjoyment (i.e., factor) to understand the antecedent to satisfaction with telemedicine would be irrational and undesirable, but complementing this notion would be extremely fruitful [127]. This indicates that enjoyment is a new component of EDT, demonstrating a contributory benchmark to explore the antecedents to satisfaction with telemedicine. However, due to theoretical and practical limitations, the present research did not test additional determinants might be involved in patients’ cognitive evaluations [54] in the arena of satisfaction with telemedicine. Finally, this study recommends that health providers, policymakers, and stakeholders may set realistic and achievable goals in building telemedicine into an institutionalised health infrastructure for providing better quality patient care with greater flexibility through modernised, specialised healthcare support for medically underprivileged rural communities in Bangladesh and similar settings. Rural and remote communities often have limited access to dedicated public medical facilities and most lack private after-hours medical practitioners. Hence, these sparsely populated regions should continue to be a priority.
Research highlights
- A theoretically sustained prediction model contributed to health informatics research.
- Deep Neural Network and PLS-SEM advance Expectation Disconfirmation Theory and Social Cognitive Theory adaptability.
- Four antecedents of satisfaction were discovered in telemedicine settings.
- Machine Learning and Artificial Intelligence technologies forecasting antecedents to satisfaction with telemedicine.
Supporting information
S1 Appendix. Summary of constructs with measurement items.
https://doi.org/10.1371/journal.pone.0257300.s001
(PDF)
S1 Table. Demographics characteristics of the sample and survey responses.
https://doi.org/10.1371/journal.pone.0257300.s002
(PDF)
Acknowledgments
The authors gratefully acknowledge the study participants and health bodies of the Directorate General of Health Services, Ministry of Health and Family Welfare (MOHFW) of Bangladesh, who helped conduct this research.
References
- 1. Bullock D, Vehe R, Zhang L, Correll C. Telemedicine and other care models in pediatric rheumatology: an exploratory study of parents’ perceptions of barriers to care and care preferences. Pediatric Rheumatology Online Journal. 2017;15(1). pmid:28693580
- 2. Al-Samarraie H, Ghazal S, Alzahrani AI, Moody L. Telemedicine in Middle Eastern countries: Progress, barriers, and policy recommendations. International Journal of Medical Informatics. 2020;141:104232. pmid:32707430
- 3. Uscher-Pines L, Cantor J, Huskamp H, Mehrotra A, Busch A, M B. Adoption of telemedicine services by substance abuse treatment facilities in the U.S. Journal of Substance Abuse Treatment. 2020;117:108060. pmid:32811631
- 4. Noel K, Yagudayev S, Messina C, Schoenfeld E, Hou W, Kelly G. Tele-transitions of care. A 12-month, parallel-group, superiority randomized controlled trial protocol, evaluating the use of telehealth versus standard transitions of care in the prevention of avoidable hospital readmissions. Contemporary clinical trials communications. 2018;12:9–16. pmid:30211339
- 5. Prieto-Egido I, Simó-Reigadas J, Martínez-Fernández A. Interdisciplinary Alliances to Deploy Telemedicine Services in Isolated Communities: The Napo Project Case. Sustainability. 2018;10(7).
- 6.
Henderson C. American Telemedicine and Project ECHO [Doctoral dissertation]. Harvard Medical School; 2016.
- 7. Jansen-Kosterink S, Dekker-van Weering M, van Velsen L. Patient acceptance of a telemedicine service for rehabilitation care: A focus group study. International journal of medical informatics. 2019;125:22–29. pmid:30914177
- 8. Greenberg AS, Steinway C, Wu K, Thomas B, Chuo J, DiGiovine M, et al. 80. Role of Telemedicine In The Transition From Pediatric To Adult Care. Journal of Adolescent Health. 2019;64(2):S43.
- 9. LeRouge C, Garfield MJ. Crossing the Telemedicine Chasm: Have the U.S. Barriers to Widespread Adoption of Telemedicine Been Significantly Reduced? International Journal of Environmental Research and Public Health. 2013;10(12):6472–6484. pmid:24287864
- 10. Zobair KM, Sanzogni L, Sandhu K. Expectations of telemedicine health service adoption in rural Bangladesh. Social Science & Medicine. 2019;238:112485. pmid:31476664
- 11. Darkwa EK, Newman MS, Kawkab M, Chowdhury ME. A qualitative study of factors influencing retention of doctors and nurses at rural healthcare facilities in Bangladesh. BMC Health Services Research. 2015;15(1):344. pmid:26307265
- 12.
Commission P. Outline Perspective Plan of Bangladesh 2010-2021, Making Vision 2021 A Reality. GE Division, Dhaka. 2010.
- 13. Glueckauf RL, Ketterson TU. Telehealth Interventions for Individuals With Chronic Illness: Research Review and Implications for Practice. Professional Psychology: Research and Practice. 2004;35(6):615.
- 14. Hur I, Cousins KC, Stahl BC. A critical perspective of engagement in online health communities. European Journal of Information Systems. 2019; p. 1–26.
- 15. Anderson EW, Sullivan MW. The antecedents and consequences of customer satisfaction for firms. Marketing science. 1993;12(2):125–143.
- 16. Hudak PL, Hogg-Johnson S, Bombardier C, McKeever PD, Wright JG. Testing a new theory of patient satisfaction with treatment outcome. Medical care. 2004; p. 726–739. pmid:15258474
- 17. Chae Y, Lee J, Ho S, Kim H, Jun K, Won J. Patient satisfaction with telemedicine in home health services for the elderly. International journal of medical informatics. 2001;61:167–173. pmid:11311671
- 18. Collins K, Nicolson P, Bowns I. Patient satisfaction in telemedicine. Health Informatics Journal. 2000;6(2):81–85.
- 19. Gustke SS, Balch DC, West VL, Rogers LO. Patient Satisfaction with Telemedicine. Telemedicine Journal. 2000;6(1):5–13.
- 20. Müller K, Alstadhaug K, Bekkelund S. Headache patients’ satisfaction with telemedicine: A 12-month follow-up randomized non-inferiority trial. European Journal of Neurology. 2017;24. pmid:28432757
- 21. Mustafa S, Yang L, Mortezavi M, Vadamalai K, Ramsey A. Patient Satisfaction with Telemedicine Encounters in an Allergy and Immunology Practice During the coronavirus disease 2019 pandemic. Annals of Allergy, Asthma & Immunology. 2020;125:478–479.
- 22. Kissi J, Dai B, Dogbe CS, Banahene J, Ernest O. Predictive factors of physicians’ satisfaction with telemedicine services acceptance. Health Informatics Journal. 2020;26(3):1866–1880. pmid:31854222
- 23. Oliver RL. Effect of expectation and disconfirmation on postexposure product evaluations: An alternative interpretation. Journal of applied psychology. 1977;62(4):480.
- 24. Oliver RL. A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of marketing research. 1980; p. 460–469.
- 25. Wallin Andreassen T. Antecedents to satisfaction with service recovery. European Journal of Marketing. 2000;34(1/2):156–175.
- 26. Whitten P, Mair F. Telemedicine and Patient Satisfaction: Current Status and Future Directions. Telemedicine journal and e-health: the official journal of the American Telemedicine Association. 2000;6:417–423. pmid:11242550
- 27. Bros J, Poulet C, Arnol N, Deschaux C, Gandit M, Charavel M. Acceptance of Telemonitoring Among Patients with Obstructive Sleep Apnea Syndrome: How is the Perceived Interest by and for Patients? Telemedicine journal and e-health: the official journal of the American Telemedicine Association. 2018;24:351–359.
- 28. Lin HC, Chang CM. What motivates health information exchange in social media? The roles of the social cognitive theory and perceived interactivity. Information & Management. 2018;55(6):771–780.
- 29. Oliver RL, DeSarbo WS. Response determinants in satisfaction judgments. Journal of consumer research. 1988;14(4):495–507.
- 30. Mendoza Aldana J, Piechulek H, Al-Sabir A. Client satisfaction and Quality of Health Care in rural Bangladesh. Bulletin of the World Health Organization. 2001;79:512–517. pmid:11436472
- 31. Andaleeb SS, Siddiqui N, Khandakar S. Patient satisfaction with health services in Bangladesh. Health policy and planning. 2007;22(4):263–273. pmid:17545252
- 32. Bliemel M, Hassanein K. Consumer Satisfaction with Online Health Information Retrieval: A Model and Empirical Study. e-Service Journal. 2007;5(2):53–84.
- 33. Ding X, Long X, Li L, Liang H, Wang Q, Cai S. Antecedents of satisfaction and engagement of low-carbon bicycle-sharing using in China. Environmental Science and Pollution Research. 2019;26(9):8533–8542. pmid:30706263
- 34. Wu L. The antecedents of customer satisfaction and its link to complaint intentions in online shopping: An integration of justice, technology, and trust. International Journal of Information Management. 2013;33(1):166–176.
- 35.
Zhang ZH, Xu ZQ, Xiang M. Antecedents of Satisfaction in Animation Brand. In: Journal of Physics: Conference Series. vol. 1168. IOP Publishing; 2019. p. 032028.
- 36. Spreng RA, Page TJ Jr. A test of alternative measures of disconfirmation. Decision Sciences. 2003;34(1):31–62.
- 37. Bandura A. Social Cognitive Theory: An Agentic Perspective. Annual Review of Psychology. 2001;52(1):1–26. pmid:11148297
- 38. Venkatesh V. Creation of favorable user perceptions: Exploring the role of intrinsic motivation. MIS quarterly. 1999; p. 239–260.
- 39.
Vallerand RJ. Toward A Hierarchical Model of Intrinsic and Extrinsic Motivation. vol. 29 of Advances in Experimental Social Psychology. Academic Press; 1997. p. 271–360.
- 40. Serrano C, Shah V, Abramoff M. Use of Expectation Disconfirmation Theory to Test Patient Satisfaction with Asynchronous Telemedicine for Diabetic Retinopathy Detection. International Journal of Telemedicine and Applications. 2018;2018:1–14. pmid:30405712
- 41. Venkatesh V, Goyal S. Expectation disconfirmation and technology adoption: polynomial modeling and response surface analysis. MIS quarterly. 2010; p. 281–303.
- 42. Lankton NK, McKnight DH, Wright RT, Thatcher JB. Research note—Using expectation disconfirmation theory and polynomial modeling to understand trust in technology. Information Systems Research. 2016;27(1):197–213.
- 43. Baek JW, Chung K. Context Deep Neural Network Model for Predicting Depression Risk Using Multiple Regression. IEEE Access. 2020;8:18171–18181.
- 44. Johnson MD, Fornell C. A framework for comparing customer satisfaction across individuals and product categories. Journal of economic psychology. 1991;12(2):267–286.
- 45. Garcia R, Adelakun O. A Conceptual Framework and Pilot Study for Examining Telemedicine Satisfaction Research. Journal of medical systems. 2019;43(3):51. pmid:30684065
- 46. Hoque Sorwar G. Understanding factors influencing the adoption of mHealth by the elderly: An extension of the UTAUT model. INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS. 2017;101:75–84. pmid:28347450
- 47. Anderberg P, Eivazzadeh S, Berglund JS. A Novel Instrument for Measuring Older People’s Attitudes Toward Technology (TechPH): Development and Validation. Journal of medical Internet research. 2019;21(5):e13951. pmid:31124467
- 48. Lankton NK, Wilson EV. Factors influencing expectations of e-health services within a direct-effects model of user satisfaction. E-Service Journal. 2007;5(2):85–112.
- 49. Jewer J. Patients’ intention to use online postings of ED wait times: A modified UTAUT model. International Journal of Medical Informatics. 2018;112:34–39. pmid:29500019
- 50. Shaikh AB, Sarim M, Raffat SK, Ahsan K, Nadeem A, Siddiq M. Artificial neural network: a tool for diagnosing osteoporosis. Res J Recent Sci ISSN. 2014;2277:2502.
- 51. Hill T, Marquez L, O’Connor M, Remus W. Artificial neural network models for forecasting and decision making. International journal of forecasting. 1994;10(1):5–15.
- 52. Vallée A, Cinaud A, Blachier V, Lelong H, Safar ME, Blacher J. Coronary heart disease diagnosis by artificial neural networks including aortic pulse wave velocity index and clinical parameters. Journal of Hypertension. 2019;37(8):1682–1688. pmid:30870247
- 53. Bearden WO, Teel JE. Selected determinants of consumer satisfaction and complaint reports. Journal of marketing Research. 1983;20(1):21–28.
- 54. LaBarbera PA, Mazursky D. A longitudinal assessment of consumer satisfaction dissatisfaction: the dynamic aspect of the cognitive process. Journal of marketing research. 1983;20(4):393–404.
- 55. Anderson RE. Consumer dissatisfaction: The effect of disconfirmed expectancy on perceived product performance. Journal of marketing research. 1973;10(1):38–44.
- 56. Hovland CI, Harvey O, Sherif M. Assimilation and contrast effects in reactions to communication and attitude change. The Journal of Abnormal and Social Psychology. 1957;55(2):244. pmid:13474895
- 57. Olson JC, Dover PA. Disconfirmation of consumer expectations through product trial. Journal of Applied psychology. 1979;64(2):179.
- 58. Weiner B. An attributional theory of achievement motivation and emotion. Psychological review. 1985;92(4):548. pmid:3903815
- 59.
LaTour SA, Peat NC. The role of situationally-produced expectations, others’ experiences, and prior experience in determining consumer satisfaction. ACR North American Advances. 1980.
- 60. Oliver RL. A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of marketing research. 1980;17(4):460–469.
- 61. Churchill GA Jr, Surprenant C. An investigation into the determinants of customer satisfaction. Journal of marketing research. 1982;19(4):491–504.
- 62. Bhattacherjee A. Understanding information systems continuance: an expectation-confirmation model. MIS quarterly. 2001; p. 351–370.
- 63. Lankton NK, McKnight HD. Examining two expectation disconfirmation theory models: assimilation and asymmetry effects. Journal of the Association for Information Systems. 2012;13(2):1.
- 64. Oliver RL. Measurement and evaluation of satisfaction processes in retail settings. Journal of retailing. 1981.
- 65. Tse DK, Wilton PC. Models of consumer satisfaction formation: An extension. Journal of marketing research. 1988;25(2):204–212.
- 66. Yi Y. A critical review of consumer satisfaction. Review of marketing. 1990;4(1):68–123.
- 67. Shankar V, Smith AK, Rangaswamy A. Customer satisfaction and loyalty in online and offline environments. International journal of research in marketing. 2003;20(2):153–175.
- 68. Fournier S, Mick DG. Rediscovering satisfaction. Journal of marketing. 1999;63(4):5–23.
- 69. Lankton N, McKnight DH, Thatcher JB. Incorporating trust-in-technology into Expectation Disconfirmation Theory. The Journal of Strategic Information Systems. 2014;23(2):128–145.
- 70. Compeau D, Higgins CA, Huff S. Social cognitive theory and individual reactions to computing technology: A longitudinal study. MIS quarterly. 1999; p. 145–158.
- 71. Zeithaml VA, Berry LL, Parasuraman A. The Nature and Determinants of Customer Expectations of Service. Journal of the Academy of Marketing Science. 1993;21(1):1–12.
- 72. Oliver RL, Bearden WO. Disconfirmation processes and consumer evaluations in product usage. Journal of Business Research. 1985;13(3):235–246.
- 73. Halstead D, Hartman D, Schmidt SL. Multisource effects on the satisfaction formation process. Journal of the Academy of marketing science. 1994;22(2):114–129.
- 74. Spreng RA, MacKenzie SB, Olshavsky RW. A reexamination of the determinants of consumer satisfaction. Journal of marketing. 1996;60(3):15–32.
- 75.
Czikszentmihalyi M. Flow: The psychology of optimal experience. New York: Harper & Row; 1990.
- 76. Trevino LK, Webster J. Flow in computer-mediated communication: Electronic mail and voice mail evaluation and impacts. Communication research. 1992;19(5):539–573.
- 77. Tan GWH, Ooi KB, Leong LY, Lin B. Predicting the drivers of behavioral intention to use mobile learning: A hybrid SEM-Neural Networks approach. Computers in Human Behavior. 2014;36:198–213.
- 78. Ozonoh M, Oboirien B, Higginson A, Daramola M. Performance evaluation of gasification system efficiency using artificial neural network. Renewable Energy. 2020;145:2253–2270.
- 79. Tsay D, Patterson C. From machine learning to artificial intelligence applications in cardiac care: Real-world examples in improving imaging and patient access. Circulation. 2018;138(22):2569–2575. pmid:30571349
- 80. Montavon G, Samek W, Müller KR. Methods for interpreting and understanding deep neural networks. Journal of Digital Signal Processing. 2018;73(2):1–15.
- 81.
Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Advances in neural information processing systems; 2018. p. 4765–4774.
- 82.
Ribeiro MT, Singh S, Guestrin C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016:1135–1144.
- 83. Baehrens D, Schroeter T, Harmeling S, Kawanabe M, Hansen K, Müller KR. How to explain individual classification decisions. The Journal of Machine Learning Research. 2010;11:1803–1831.
- 84.
Atanasova P, Simonsen J, Lioma C, Augenstein I. A Diagnostic Study of Explainability Techniques for Text Classification. In proceedings of Empirical Methods in Natural Language Processing (EMNLP). 2020:3256–3274.
- 85. García MV, Aznarte JL. Shapley additive explanations for NO2 forecasting. Ecological Informatics. 2020;56:101039.
- 86. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS quarterly. 2003; p. 425–478.
- 87. Padovani M, Carvalho MM. Integrated PPM Process: Scale Development and Validation. International Journal of Project Management. 2016;34(4):627–642.
- 88. Webster J, Watson RT. Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly. 2002; p. xiii–xxiii.
- 89. Zobair KM, Sanzogni L, Sandhu K. Telemedicine Healthcare Service Adoption Barriers in Rural Bangladesh. Australasian Journal of Information Systems. 2020;24.
- 90. Andaleeb SS. Service quality perceptions and patient satisfaction: a study of hospitals in a developing country. Social science & medicine. 2001;52(9):1359–1370. pmid:11286361
- 91. Zhang M, Luo M, Nie R, Zhang Y. Technical attributes, health attribute, consumer attributes and their roles in adoption intention of healthcare wearable technology. International Journal of Medical Informatics. 2017. pmid:29132639
- 92. Akter MS, Upal M, Hani U. Service quality perception and satisfaction: a study over sub-urban public hospitals in Bangladesh. Journal of Services Research. 2008; p. 125.
- 93. Islam F, Rahman A, Halim A, Eriksson C, Rahman F, Dalal K. Perceptions of health care providers and patients on quality of care in maternal and neonatal health in fourteen Bangladesh government healthcare facilities: a mixed-method study. BMC health services research. 2015;15(1):237. pmid:26084893
- 94.
Hair JF Jr, Hult GTM, Ringle C, Sarstedt M. A primer on partial least squares structural equation modeling (PLS-SEM). Sage Publications; 2017.
- 95. Roldán JL, Sánchez-Franco MJ. Variance-based structural equation modeling: guidelines for using partial least squares. Research methodologies, innovations and philosophies in software systems engineering and information systems. 2012;193.
- 96.
Henseler J, Hubona G, Ray PA. In: Partial least squares path modeling: Updated guidelines. Springer; 2017. p. 19–39.
- 97.
Henseler J, Ringle CM, Sinkovics RR. In: The use of partial least squares path modeling in international marketing. Emerald Group Publishing Limited; 2009. p. 277–319.
- 98. Benitez J, Henseler J, Castillo A, Schuberth F. How to perform and report an impactful analysis using partial least squares: Guidelines for confirmatory and explanatory IS research. Information & Management. 2020;57(2):103168.
- 99. Hair JF, Howard MC, Nitzl C. Assessing measurement model quality in PLS-SEM using confirmatory composite analysis. Journal of Business Research. 2020;109:101–110.
- 100. Otter V, Beer L. Alley cropping systems as Ecological Focus Areas: A PLS-analysis of German farmers’ acceptance behaviour. Journal of Cleaner Production. 2021;280:123702.
- 101.
Wilson B. In: Using PLS to investigate interaction effects between higher order branding constructs. Springer; 2010. p. 621–652.
- 102.
Bengio Y. Deep Learning of Representations: Looking Forward. In: Dediu AH, Martín-Vide C, Mitkov R, Truthe B, editors. Statistical Language and Speech Processing. Berlin, Heidelberg: Springer Berlin Heidelberg; 2013. p. 1–37.
- 103.
Zhang Y, Chen D, Ye C. Deep Neural Networks: WASD Neuronet Models, Algorithms, and Applications. Chapman & Hall/CRC Artificial Intelligence and Robotics Series. CRC Press; 2019.
- 104.
Ameur H, Jamoussi S, Hamadou AB. A Deep Neural Network Model for Predicting User Behavior on Facebook. In: 2019 International Joint Conference on Neural Networks (IJCNN); 2019. p. 1–8.
- 105.
Marcus G. Deep learning: A critical appraisal. arXiv preprint arXiv:180100631. 2018.
- 106.
Hecht-Nielsen. Theory of the backpropagation neural network. In: International Joint Conference on Neural Networks; 1989. p. 593–605 vol.1.
- 107. He J, Li L, Xu J, Zheng C. ReLU deep neural networks and linear finite elements. Journal of Computational Mathematics. 2020;38(3):502–527.
- 108. Tamaddon-Jahromi HR, Chakshu NK, Sazonov I, Evans LM, Thomas H, Nithiarasu P. Data-driven inverse modelling through neural network (deep learning) and computational heat transfer. Computer Methods in Applied Mechanics and Engineering. 2020;369:113217.
- 109.
Ma F, Zhang J, Liang W, Xue J. Automated Classification of Atrial Fibrillation Using Artificial Neural Network for Wearable Devices. Mathematical Problems in Engineering. 2020.
- 110. Eitel F, Schulz MA, Seiler M, Walter H, Ritter K. Promises and pitfalls of deep neural networks in neuroimaging-based psychiatric research. Experimental Neurology. 2021;339:113608. pmid:33513353
- 111. Lujan-Moreno GA, Howard PR, Rojas OG, Montgomery DC. Design of experiments and response surface methodology to tune machine learning hyperparameters, with a random forest case-study. Expert Systems with Applications. 2018;109:195–205.
- 112.
Adriana M, Holban, S. The Most Used Activation Functions: Classic Versus Current. In: Proceedings of International Conference on Development and Application Systems (DAS). 2020:141–415.
- 113. Özkan C, Sunar A F The Comparison of Activation Functions for Multispectral Landsat TM Image Classification. Photogrammetric Engineering and Remote Sensing. 2003:1225–1234.
- 114.
Sharma SK, Govindaluri SM, Al Balushi SM. Predicting determinants of Internet banking adoption. Management Research Review. 2015.
- 115.
Bodria F, Giannotti F, Guidotti R, Naretto F, Pedreschi D, Rinzivillo S. Benchmarking and Survey of Explanation Methods for Black Box Models. CoRR. 2021.
- 116.
Lai V, Cai Z, Tan C. Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019:486–495.
- 117.
Slack D, Hilgard S, Jia E, Singh S, Lakkaraju H. Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. In: Proceedings of the 2020 AAAI/ACM Conference on AI, Ethics, and Society (AIES). 2020.
- 118. Bhattacherjee A, Premkumar G. Understanding changes in belief and attitude toward information technology usage: A theoretical model and longitudinal test. MIS quarterly. 2004; p. 229–254.
- 119.
Ho C, Wu W. Antecedents of customer satisfaction on the internet: an empirical study of online shopping. In: Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers. vol.
- 120. Venkatesh V. Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information systems research. 2000;11(4):342–365.
- 121. Davis FD, Bagozzi RP, Warshaw PR. Extrinsic and intrinsic motivation to use computers in the workplace 1. Journal of applied social psychology. 1992;22(14):1111–1132.
- 122. Carlsmith JM, Aronson E. Some hedonic consequences of the confirmation and disconfirmation of expectances. The Journal of Abnormal and Social Psychology. 1963;66(2):151. pmid:14018688
- 123. Cadotte ER, Woodruff RB, Jenkins RL. Expectations and norms in models of consumer satisfaction. Journal of marketing Research. 1987; p. 305–314.
- 124. Lu J, Feng J, Fan Z, Huang L, Zheng C, Li W, et al. Automated strabismus detection based on deep neural networks for telemedicine application. Knowledge-based systems. 2019;13.
- 125. Chung K, Yoo H, Choe DE. Ambient context-based modeling for health risk assessment using deep neural network. Journal of Ambient Intelligence and Humanized Computing. 2020;11(4):1387–1395.
- 126. Thompson A, Sunol R. Expectations as Determinants of Patient Satisfaction: Concepts, Theory and Evidence. International journal for quality in health care: journal of the International Society for Quality in Health Care / ISQua. 1995;7:127–41. pmid:7655809
- 127. Holbrook MB, Hirschman EC. The experiential aspects of consumption: Consumer fantasies, feelings, and fun. Journal of consumer research. 1982;9(2):132–140.