Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Awareness of and interaction with physician rating websites: A cross-sectional study in Austria

  • Bernhard Guetz,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Project administration, Writing – original draft

    Affiliation Department of Marketing and International Management, Alpen-Adria-Universitaet Klagenfurt, Klagenfurt am Wörthersee, Austria

    ⨯
  • Sonja Bidmon

    Roles Conceptualization, Formal analysis, Methodology, Supervision, Writing – review & editing

    sonja.bidmon@aau.at

    Affiliation Department of Marketing and International Management, Alpen-Adria-Universitaet Klagenfurt, Klagenfurt am Wörthersee, Austria

    ⨯

Abstract

To date, the digital assessment of service experiences represents a decisive process step of a feedback culture in numerous economic areas. In view of this digitalization of service assessments, the importance of Physician Rating Websites (PRWs) has also increased steadily in recent years. Even though these websites could be perceived as a powerful communication tool for the exchange of health specific information, the knowledge about whether and how different population segments use these portals has been limited so far. For this reason, our aim was to investigate the level of awareness regarding PRWs among the study population and to discover how users interact with this specific type of online portals. We performed an online survey including 558 participants. To ensure the attention and integrity of participants, attention checks were included in the questionnaire. Study participants who did not exceed the mentioned security levels were excluded from the study. Statistical analyses were carried out, using IBM SPSS Statistics 27. To illustrate the relationship between demographic variables and dependent variables, two tailed chi square tests were performed. Comparison of means and t-testing was used to investigate the relationship between psychographic variables and the dependent variables. In addition to that, the awareness levels regarding different rating portals were evaluated using descriptive methods. Our results suggest that the general awareness regarding PRWs is relatively high (75.6%, 423/558), especially among female (x21 = 9.880, P = .002), middle-aged (x29 = 26.810, P = .002), more highly educated (x24 = 19.038, P = .001), urban (x21 = 6.274, P = .012), digitally literate (t203 = 2.63, P = .009) individuals and particularly among respondents with a higher eHealth literacy (t203 = 2.37, P = .019). Even though more than three quarters of the respondents know that PRWs exist, compared to other rating platforms, they are only in the lower midfield. The upper ranks are taken by websites on which restaurant visits (98.9%, 552/558), hotel stays (97.7%, 545/558) or movies (95.5%, 533/558) can be rated. The most popular PRWs in Austria include Docfinder.at (31.3%, 175/558; 77.8%, 434/558) followed by the evaluation tools provided by Google.at (8.24%, 46/558; 70.3%, 392/558) and Herold.at (1.61%, 9/558; 44.8%, 250/558). In Austria, PRWs seem to be characterized by a high degree of interaction (89.2%, 498/558) with a wide variety of different types of interactions. While many respondents use PRWs to retrieve general information (83.2%, 464/558), there are significantly fewer who read physicians’ reviews (60.9%, 340/558) and use this portal to select a physician (60.6%, 338/558). Respondents who have already rated a doctor themselves belong to the smallest group accounting for just 14.7% (82/558). Significant effects regarding the interaction with PRWs exist between different genders, ages, education levels, marital statuses, occupations and areas of living. In addition to that, respondents with better feelings towards the internet, greater digital literacy as well as a higher eHealth literacy were also characterized with a higher interaction rate regarding PRWs. According to the high level of awareness of and interaction with PRWs within our study population, PRWs appear to be a successful medium for health-related communication. Especially for female, middle-aged, more highly educated, urban and more technology savvy population segments, PRWs seem to represent an effective tool to support the health-specific decision-making process.

Citation: Guetz B, Bidmon S (2022) Awareness of and interaction with physician rating websites: A cross-sectional study in Austria. PLoS ONE 17(12): e0278510. https://doi.org/10.1371/journal.pone.0278510

Editor: Mona Dür, Duervation, AUSTRIA

Received: December 28, 2021; Accepted: November 17, 2022; Published: December 30, 2022

Copyright: © 2022 Guetz, Bidmon. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data files are available from the figshare database (10.6084/m9.figshare.17694767).

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Abbreviations: Compulsory Ed., Compulsory Education; eHealth, electronic Health; MTurk, Mechanical Turk; PRW, Physician Rating Website; PRWs, Physician Rating Websites; Sal. employee, Salaried employee; Voc. Sec. Ed., Vocational Secondary Education

Introduction

Due to the increasingly competitive environment in the private and public health sector, the importance of customer orientation in health management has grown steadily in recent years [1]. In addition to that, the rise in patient orientation has also led to patients being more involved in the medical decision-making process [2]. These relative factors are a clear signal of the increasing importance of patients as opinion leaders [3–5]. Traditionally, the role as opinion leader has been particularly important within the family association and circle of friends [6–9]. Nevertheless, with the rise of the internet and network technology, this rather limited circle of stakeholders has expanded considerably [10–12]. The opportunities offered by these technologies enable people to network and exchange information detached from time and residence or location [13–15]. Moreover, this modern type of information exchange does not stop when it comes to information regarding the healthcare sector [16–18].

In this context, Physician Rating Websites (PRWs) could be a promising means for health-specific communication purposes [19]. These online portals allow users to discuss, rate and comment on physicians’ medical and service quality in a social media setting [20]. In this context, PRWs offer users the possibility to generally inform themselves about the expected medical service and allow them to choose between different alternatives according to their personal needs [21]. Additionally, the importance of PRWs has grown steadily in recent years [22], albeit at a much lower rate than was expected [23]. In analogy to the increasing societal attentiveness towards PRWs, the related scientific interest has also evolved steadily during the recent past [22–38]. Amongst others, this fact is represented in the growing number of different studies from regions all over the world like North America [20, 24, 39–52], Europe [1, 33, 36, 51, 53–65], Asia [66–74] or Australia [75–77]. Within such investigations, a wide range of research areas was examined, such as the frequency of ratings [36, 77], the effects of negative reviews [20, 48, 70], the content of reviews [1, 24, 39–47, 49, 53, 57, 63, 66–69, 71–76], factors that have an impact on the evaluation behavior [56, 61, 64], or the behavior of physicians [62]. Another investigated area in this context is patients’ awareness of and interaction with PRWs. In this context, different studies from England [54, 65], the United States [50, 52, 78], Germany [33, 51, 58] and Switzerland [28] could be identified. Through literature search, we were even able to identify studies by Austrian research institutions on awareness of and interaction with PRWs, but these studies were conducted with a German data set [1, 56, 58]. Although the same language is spoken in Austria and Germany, the health care systems differ substantially and it cannot be assumed that research results are comparable between Austria and Germany with regard to the awareness and use of PRWs. This conclusion is based on cultural differences with regard to various aspects like corporate governance [79–81], multiculturalism [82], but also with regard to health-specific perception [83] and the integration of online media and online tools in health care [84, 85].

Within the studies already performed, different factors that seemed to be related to health-specific technology use, like the usage of PRWs, could be identified. These include demographic variables such as gender, age, education, marital status and occupation [33, 51, 54, 58]. Furthermore, different psychographic criteria that seem to be related to the use of health-specific digital technologies were discovered. Among them are eHealth literacy, digital literacy as well as the feelings towards the internet [30, 58, 86]. In this context, eHealth literacy refers to a person’s capacity to look for, comprehend, and evaluate health information from online resources and to apply that knowledge to an issue regarding their personal health [87]. Digital literacy, by comparison, is the ability to use digital technologies to solve a variety of different challenges in daily life [88]. In contrast to digital literacy, eHealth literacy focuses exclusively on the use of digital technologies to search for and use information in a health-specific environment. Finally, feelings towards the internet describe the general attitude of people towards the internet as well as towards web-based applications [58].

Moreover, there seem to be at least two further factors that potentially affect the process of engaging with these portals. PRWs can be seen as a specific form of electronic Word-of-Mouth [89] and therefore represent a kind of substitute for recommendations, which are usually given within personal interactions. This substitution might be more important in regions characterized by a higher degree of resident anonymity like urban regions [90] as compared to rural ones, which might lead to a difference between rural and urban regions with regard to the prevalence of usage of PRWs. In addition to that, the level of skepticism towards online reviews (review skepticism) could also negatively influence the engagement with PRWs [29].

The above-mentioned facts show that there has been a research gap with regard to the awareness of and interaction with physician rating websites in Austria so far. For this reason, this paper addresses four distinct research questions with regard to awareness of and interaction with PRWs in Austria. They are elaborated in detail in the next two sections.

Awareness regarding physician rating websites (RQ1 and RQ2) and brand awareness of specific PRWs (RQ3)

Awareness can occur in different manifestations. In common parlance, awareness is defined as mental perception of the existence of a product, a service, a brand or a particular situation [91]. A similar conceptualization of awareness can also be found in a number of related publications that examine the general awareness of PRWs [33, 51, 54, 58]. For this reason, we investigate how the awareness regarding PRWs is distributed according to different sociodemographic and psychographic segments within the study population (RQ1).

In addition to that, the awareness regarding PRWs can further be compared to the awareness gained by rating portals in other realms of daily life like hotel rating portals or employer rating portals. One published study chose this procedure and compared the awareness regarding PRWs with the awareness regarding alternative online rating portals [50]. Accordingly, RQ2 investigates the level of awareness regarding PRWs contrasted to alternative online rating portals.

Another form of awareness, frequently examined in advertising studies, is brand awareness. For consumers, brand awareness plays a crucial role within their individual decision-making process [92]. In this context, a distinction can consistently be found between the two well-known types of brand awareness, i.e. brand recall (recall) and brand recognition (recognition) [93]. Recall refers to the retrieval of a brand name or symbol that has been stored in an individual’s memory in the past [94]. In contrast to that, recognition describes the ability to remember things previously experienced when they are displayed or mentioned [95]. In this context, PRWs can also be understood as brands, as they often feature their own brand name and brand logo. The focus is thus on the extent to which individuals can recall and recognize the names and respective brands of distinct Austrian PRWs. Therefore, the third purpose of our study is to investigate the brand awareness (recall and recognition) of distinct Austrian PRWs according to the study population (RQ3).

Interaction with physician rating websites (RQ4)

Interaction is understood as an occurrence in which individuals or objects convey or respond to each other [91]. If computers are involved, the term human computer interaction (HCI) is used to refer to the actions that a user performs on a computer [96]. A particular form of HCI is the human website interaction, which describes how individuals interact with certain websites [97]. In the area of PRWs, there are different possibilities of interaction, which are variously expressed depending on the functionality of the respective website. In general, PRWs commonly contain information regarding physicians’ practices like opening hours, telephone number or the address, which shows that those websites could serve as a source of general information [19]. When it comes to the evaluation aspect, however, users can generally choose between reading patient ratings of physicians and providing personal feedback on physicians [20]. From a decision supporting perspective, PRWs probably serve only two purposes from the patients’ perspectives, which are the search for physicians and the selection of future physicians [35, 41, 98]. In this context, our fourth research question is:

RQ4: How do patients interact with PRWs and what are the driving factors for distinct usage of PRWs?

Methods

The present study did not require approval of an ethics board in Austria, according to section 8c of the Federal Hospitals and Health Care Institutions Act as well as section 30 of the Universities Act. Corresponding to this legal framework, consultation of an ethics board is obligatory only for clinical trials of medicinal products and medical devices, as well as for the application of new medical methods and applied medical research on humans. Since the study scope is not health related, the study design and data collection did not require ethical approval. No sensitive issues were raised within the questionnaire and the evaluation method did not allow any conclusions to be drawn about the person who had responded to the survey. In addition, the questions were kept very general and there was at no time a risk of harm in any form by answering them. No sensitive data were requested and all panel members gave their consent to the data collection. The study was conducted by usage of the crowdsourcing platform Clickworker.com. All projects and orders are subjected to an auditing process before being accepted. In this procedure, trained employees check whether the survey is feasible, whether personal data is requested and whether discriminatory or unethical content occurs. Orders in which personal data are requested, or include discriminatory or unethical content are not accepted. Participants were informed and appropriately instructed about their voluntary involvement in an online survey, as well as the fact that their data would be treated with absolute confidentiality using suitable methods, procedures, and protocols. Individuals who agreed to participate voluntarily were informed about the survey in writing before and after completing the questionnaire. The data was processed in a strictly confidential and anonymous manner [99].

Questionnaire design and mode

The questionnaire used for the current study was designed by using existing and validated scales. The sociodemographic variables were taken from Bidmon et al. [15] and the respective items were used to measure gender, age, education level, and marital status. Occupation was measured using a 9 item scale adapted from Emmert et al. [51]. Within the area of living part, the survey participants were asked whether they lived in a more urban or a more rural area. In this case, the two item scale of Liff et al. [100] was used as a template. Feelings towards the internet and digital literacy were both measured on a seven-point scale taken from Terlutter et al. [58]. In addition to that, eHealth literacy was measured using the German version of the seven item eHealth literacy scale developed by Norman & Skinner [101], which has been translated into German and published by Soellner et al. [102]. Finally, the psychographic variable review skepticism was measured on a four item scale developed by Skarmeas & Leonidou [103]. In the second part of the questionnaire, general information regarding the participants’ knowledge about online rating and review websites was gathered. To investigate the awareness of PRWs in comparison with alternative rating portals, the multiple response scale adapted from Hanauer et al. [50] was used. In addition to that, within this questionnaire area, Keller’s Brand Recall scale [104] was adapted for the study focus of PRWs. The third questionnaire section commenced with the attention check. Participants had to select a specific answer, defined in the introductory text, in order to proceed with the questionnaire. The attention check was created following Oppenheimer et al. [105] and Kung et al. [106]. The attention check was followed by the question regarding brand recognition. This multiple response scale that asked for knowledge regarding different PRW brands was taken from Emmert et al. [51] and adapted to the Austrian market. Finally, the degree of interaction with PRWs was assessed using a multiple response scale developed by Burkle & Keegan [20]. The original scale was extended to include the categories search for physician(s), select physician(s), and search for additional information. The questionnaire and the justification of the items can be found in the supporting information section (S1 File).

All items used were translated/back translated by both, an English and a German native speaker who each had fluent language skills in the foreign language. In order to identify potential ambiguity or uncertainty, a pre-study with 20 participants was performed. After slight modifications as well as the implementation of an ‘other’ option within some items, the adapted version of the questionnaire was used to carry out the main study.

Data collection procedure

As mentioned above, the data collection was performed by using a specific online survey tool (Clickworker.com) which is a crowdsourcing platform for research similar to Amazon’s Mechanical Turk (MTurk), however even more widespread in Europe. Clickworkers voluntarily participate in the available surveys, whereby they only see those surveys for which they are approved via a profile filter. This filter is defined by the entries in the order (age, gender, country, native language,….). All clickworkers from Austria between the ages of eighteen and ninety-nine were able to take part in the survey. Participation was possible via the desktop application as well as via the Android and iOS app. Participants were recruited by making the survey accessible within the system for all persons who met the criteria mentioned above. We followed recommendations to mitigate validity threats when using crowdsourcing platforms (see e.g., [107]) by including 3 security levels in the online questionnaire. First, the attention and integrity of survey participants was ensured by logic tasks and attention checks. Before participants were able to start answering the questions, they had to solve a mathematical equation to prove that they are human as well as confirming their eligibility. Furthermore, an attention check was included. Within this attention check, participants had to select a certain answer option within a specific question to prove that they had read the introduction text of the third question group [105, 106]. Finally, a cookie was set for each participant to prevent repeated participation. In addition to these security levels, the order of possible answers varied for knowledge related questions in each questionnaire cycle. An example equation as well as the applied introduction text and the description of the attention check can be found in the supporting information section (S2 File).

Data preparation

The survey data was obtained by using the survey tool Lime Survey. Since the settings of the questionnaire instrument did not allow for non-answered questions in the area of awareness and interaction with PRWs, there was no missing data within the survey.

Data analysis

Statistical analyses were conducted by using IBM SPSS Statistics 27. Two-tailed chi-square tests were used to illustrate the correlation between demographic characteristics (e.g. age or gender) and dependent variables (e.g. general awareness or type of interaction). Moreover, we used comparisons of means and t-testing to show correlations between psychographic characteristics (e.g. feelings towards the internet or eHealth literacy) and the key variables (e.g. general awareness or type of interaction). Finally, brand recall and recognition were contrasted by descriptive methods.

Results

The initial survey sample consisted of 852 respondents. As a first data cleansing step, the exclusion of those participants who had not passed the attention check led to a reduction of the total sample to n = 613. Another thorough data check controlling for extremely short answer times, flatliners and inconsistent answer patterns led to a final survey sample of n = 558 respondents. The described data selection process is illustrated in the supporting information section (S3 File). A sample description can be found in Table 1.

thumbnail
Download:
Table 1. Sample description with regard to sociodemographic variables (n = 558).

https://doi.org/10.1371/journal.pone.0278510.t001

Results on awareness (RQ1–RQ3)

The following section summarizes our results on awareness regarding PRWs. In this context, a distinction is made between the three following forms of awareness: general awareness (RQ1), awareness compared to other rating portals (RQ2) and brand awareness (RQ3).

RQ1: In total 75.6% (422/558) of our study participants were aware that PRWs exist. Statistically significant effects regarding the general awareness of PRWs were found within the demographic characteristics of gender (x21 = 9.880, P = .002, V = .13), age (x29 = 26.810, P = .002, V = .22), education (x24 = 19.038, P = .001, V = .19), occupation (x24 = 10.937, P = .027, V = .14) and area of living (x21 = 6.274, P = .012, V = .11). Table 2 summarizes the characteristics of the individual relationships between the variables.

thumbnail
Download:
Table 2. Coherence of demographic variables and general awareness regarding PRWs.

https://doi.org/10.1371/journal.pone.0278510.t002

Significant effects were found for the investigated psychographic variables digital literacy and general awareness (t203 = 2.63, p = .009) as well as between eHealth literacy and general awareness (t203 = 2.37, p = .019). Table 3 shows the differences within the mean values of the psychographic variables and the results of significance testing (t-tests).

thumbnail
Download:
Table 3. Coherence of psychographic variables and general awareness regarding PRWs.

https://doi.org/10.1371/journal.pone.0278510.t003

RQ2: In order to show differences within popularity levels regarding different online rating websites, we related the general awareness regarding PRWs to the awareness pertaining to alternative rating portals. Within our sample, the most pronounced online rating option was the rating of restaurants. 552 out of 558 (98.9%) study respondents indicated that they are aware of the possibility to rate their experiences with specific restaurants online. This was subsequently followed by the possibility to evaluate experiences regarding hotels 97.7% (545/558), movies 95.5% (533/558), and books 91.0% (508/558). As shown above, 75.6% (422/558) of the study participants were aware that PRWs provide the possibility to evaluate physicians online and a further 248 out of 558 (44.6%) respondents were aware that hospital rating websites exist. Table 4 represents the individual items including the respective awareness scores.

thumbnail
Download:
Table 4. Awareness of rating websites.

https://doi.org/10.1371/journal.pone.0278510.t004

RQ3: Within the third awareness analysis, we try to describe the market penetration in terms of brand awareness for individual PRWs. Recall was the highest for Docfinder.at (31.36%) followed by the review pages of Google.at (8.24%) and Herold.at (1.61%). In addition to the valid responses regarding PRWs for the Austrian market, a number of non-valid websites were mentioned. These include portals that do not have a rating function for medical services (e.g., Tripadvisor.at) as well as websites on which it is not possible to rate or review Austrian physicians (e.g., Jameda.de). An overview of the recall rates for the individual responses can be found in Table 5.

thumbnail
Download:
Table 5. PRW brand recall.

https://doi.org/10.1371/journal.pone.0278510.t005

A similar pattern could be demonstrated for recognition. Most respondents recognized Docfinder.at (78.10%) as well as the review sections of Google.at (68.40%) and Herold.at (44.8%) as potential PRWs. Detailed information regarding the brand recognition survey part can be found in Table 6.

thumbnail
Download:
Table 6. PRW brand recognition.

https://doi.org/10.1371/journal.pone.0278510.t006

Results on interaction (RQ4)

RQ4: Most study participants 498/558 indicated that they had interacted with PRWs at least once in the past, i.e., 89.2% of our study respondents can be classified as users of PRWs. However, there were considerable differences in the types of interaction in which respondents engaged with regard to PRWs. In this context, 83.2% (464/558) of participants indicated that they had used PRWs to search for general information such as the address or the telephone number of a specific physician. In addition to that, 76.5% (427/558) of respondents used those websites to search for physicians followed by 60.9% (340/558) respondents who indicated that they use those websites to read physicians’ ratings. 60.6% (338/558) of respondents indicated that they use PRWs to select future physicians. Finally, 14.7% (82/558) of respondents reported that they had provided personal feedback by rating a specific physician on PRWs at least once in the past.

Significant effects regarding the search for general information exist between females and males (x21 = 4.902, P = .027) and different age categories (x29 = 17.483, P < .001). Regarding the utilization of PRWs to search for physicians, significant effects were demonstrated for gender (x21 = 6.922, P = .009), age (x21 = 33.638, P = < .001) and occupation (x24 = 10.723, P = .030). Moreover, regarding the use of PRWs to read physicians’ ratings, significant effects were found between different genders (x21 = 13.419, P < .001), ages (x29 = 34.631, P < .001), education levels (x24 = 26.656, P < .001) and areas of living (x21 = 10.557, P = .001). With respect to the utilization of PRWs to select future physicians, significant effects within the demographic variables gender (x21 = 15.451, P < .001), age (x29 = 18.441, P = .030), marital status (x23 = 8.207, P = .042) and area of living (x21 = 7.570, P = .006) could be found. Finally, no significant differences regarding the provision of personal feedback on PRWS could be demonstrated within the demographic variables. Table 7 compares the results in terms of demographic characteristics associated with the interaction on PRWs.

thumbnail
Download:
Table 7. Types of interaction with PRWs and association with demographic variables.

https://doi.org/10.1371/journal.pone.0278510.t007

With respect to psychographic characteristics and interaction with PRWs, significant mean differences were found for four of the five interaction types. Respondents who used PRWs to search for general information were characterized by better feelings towards the internet (t556 = 2.30, P = .022), a higher digital literacy (t556 = 2.64, P = .008) as well as a higher eHealth literacy (t116 = 4.56, P < .001). In addition to that, participants who used PRWs to search for physicians were characterized by a higher eHealth literacy (t187 = 3.24, P = .001). Respondents who used PRWs to read physicians’ ratings were also characterized by a higher eHealth literacy (t391 = 4.034, P < .001). Finally, participants who used PRWs to select future physicians were characterized by better feelings towards the internet (t556 = 2.549, P = .011) as well as a higher eHealth literacy (t410 = 3.650, P < .001). No significant differences were detected when focusing on psychographic characteristics and the utilization of PRWs to provide personal feedback. Additionally, review skepticism had no significant impact on the different types of interaction, even though non-users revealed a higher level of review skepticism than users by trend. An overview regarding psychographic characteristics related to the interaction with PRWs can be found in Table 8.

thumbnail
Download:
Table 8. Types of interaction with PRWs and association with psychographic variables.

https://doi.org/10.1371/journal.pone.0278510.t008

Discussion

Principal results

In general, three core contributions could be achieved by using the results of the current study. First of all, it is the first study ever to deal with PRWs in Austria. However, the results should also be of considerable interest for the observation and further development of international PRW as well as rating and review website research. In this context, we were able to investigate the influence of demographic and psychographic factors on awareness of, and interaction with PRWs. Furthermore, we were able to show that there is not one type of interaction on PRWs, but several different forms of interaction. In this context, these forms of interaction are not only characterized by different usage rates, but also by individual factors influencing the respective use behavior.

General awareness (RQ1)

Within our study sample, general awareness regarding PRWs was significantly above the awareness level of the studies by Patel et al. (2018) [54] (15.2%), Terlutter et al. (2014) [58] (29.3%), and Emmert et al. (2013) [51] (32.09%). Similar levels were measured in the studies of Hanauer et al. (2014) [50] (65.0%), McLennan et al. (2019) [33] (72.5%) and Hanauer et al. (2014) [78] (74.0%).

Even though the general awareness of PRWs seems to be relatively high within the study population (75.6%), there are significant differences regarding the awareness level within various demographic and psychographic characteristics. Women generally had a higher awareness level than men. A similar relationship was demonstrated within the studies by McLennan et al. (2017) [33] and Emmert et al. (2013) [51]. In addition to that, we were able to show that there is a significant difference regarding general awareness of PRWs between participants in different areas of living. In this context, respondents living in more urban areas had a higher awareness level than respondents living in more rural areas. One reason for this effect could be that urban areas are characterized by a higher level of anonymity [90] and therefore PRWs are more likely to be consulted as health specific advisors than family and friends. Additionally, respondents with a higher digital as well as a higher eHealth literacy were also characterized by a higher awareness level regarding PRWs. A comparable result was obtained by Emmert et al. (2013) [51]. This situation may be related to the fact that these groups of people are characterized by a more pronounced online behavior, which also leads to a stronger expression of PRWs general awareness.

Awareness compared to alternative rating portals (RQ2)

In RQ2 we tried to discover whether the level of awareness is high or low compared to alternative rating websites. Although PRWs are known by more than three quarters of the study population, they seem to remain in the lower midfield in terms of awareness compared to other rating portals. Similarly, in the McLennan et al. (2017) [33] study, awareness of PRWs was estimated to be relatively low compared to alternative review portals. The low awareness compared to alternative rating portals could be due to the fact that some of those alternative websites have been available for a very long time and thus the market penetration seems to be more pronounced (see e.g., [108–110]). Conversely, this circumstance could also be a reason for the lower awareness of rather novel online portals like real estate rating websites (61.1%), teacher rating websites (48.7%) or hospital rating websites (44.6%).

Brand awareness of specific PRWs (RQ3)

In terms of unaided brand awareness (recall), Docfinder.at received the highest level of brand awareness within the study population. The subsequent brand recall areas are occupied by the evaluation areas of Google.at and Herold.at. A similar picture emerges in the brand awareness study part. Even in this area, Docfinder.at, Google.at and Herold.at occupy ranks 1–3. In particular, the high recall value of 31.36% shows that Docfinder.com appears to be the best-known brand in the Austrian PRWs sector. This fact is reinforced by the relatively high gap of 23.12 percentage points to second place in the recall area and 7.5 percentage points to second place in the recognition area.

Interaction with physician rating websites (RQ4)

Interaction with PRWs seems to be uniformly high (89.2%). The interaction level is thus significantly higher than in the studies already published, which showed levels between 0.4% in England (2018) [54] and 39% in the United States of America (2017) [52]. We attribute the fact that the interaction level is higher than the awareness level (75.6%) to respondents having used PRWs in the past without being aware of it. This conclusion is also supported by the collected data. The highest interaction levels were found for using those websites to search for general information (83.2%) and to search for physicians (76.5%). These types of interaction are not PRW specific and can also be conducted via health-specific websites or search engines. After the search for general information and the search for physicians, a PRW-specific form of interaction occurs with the reading of physicians’ ratings. More than half of the study participants reported using PRWs for this type of interaction. Since this is a very specific form of interaction, it can be assumed that people who use PRWs to read physicians’ ratings are aware of the platform itself, while the search for general information and the search for physicians can also be carried out unconsciously on PRWs. While 60.9% of the study participants indicated that they use PRWs to read physicians’ ratings and 60.6% used these portals to select future physicians, the number of respondents who also provided personal feedback online was relatively low (14.7%). This result is in line with the body of literature that has shown that while people frequently read online reviews, they rarely post reviews (see e.g., [111–114]).

Besides the results regarding the general interaction with PRWs, differences in the interaction types with PRWs between several demographic groups could be demonstrated. In this context, women were more likely to use PRWs to search for general information, to search for physicians, to read physicians’ ratings and to select future physicians than men. Similar results were presented by Emmert et al. (2013) [51] and Terlutter et al. (2014) [58] who also found a higher general interaction with PRWs by female respondents. In addition to that, different age groups are also characterized by a varying use behavior. Thus, it was shown that in the youngest age group fewer people used PRWs to search for general information, to search for physicians, to read physicians’ ratings and to select future physicians than in the older age groups. This circumstance could be due to the fact that younger persons and young adults are often characterized by a rather risky health behavior and accordingly do not deal sufficiently with health-specific issues (see e.g., [115–117]). Furthermore, differences in the area of living were also found. Thus, it was shown that more people living in urban regions use PRWs to read physicians’ ratings and select future physicians than people living in rural regions. This result supports our assumption that PRWs are used more heavily in urban regions than in rural regions. One reason for this could be the higher level of anonymity in urban regions [90]. Since the relationship with family and friends is not as established in cities as in rural areas, alternative information channels such as PRWs are more likely to be used as health specific information source.

In addition to that, occupation had an impact on whether respondents used PRWs to search for physicians. In that respect, it was observed that more salaried employees, unemployed participants, retired respondents and self-employed study participants use PRWs to search for physicians than study participants who are currently in training, such as pupils or students. Since it can be assumed that a large proportion of the study participants who are currently in training belong to the younger age group, we again attribute the lower use to the rather risky health behavior of this age group (see e.g., [115–117]).

When focusing on the psychographic variables, it was found that eHealth literacy seems to have an influence on the likelihood of PRWs use. In contrast to that, a relation between digital literacy and the use of PRWs was only found for one category of use. Respondents with higher digital literacy were more likely to use PRWs to search for general information than respondents with lower digital literacy. A similar result was demonstrated in the study by Hanauer et al. (2014) [78]. Finally, it was shown that respondents with better feelings towards the internet had a higher probability to use PRWs to search for general information and select future physicians than respondents with worse feelings towards the internet.

Limitations

Some limitations could be identified within this study. As mentioned in the methods section, especially within online surveys it must always be expected that study participants are not very attentive, show acquiescence bias (propensity to agree or disagree with almost all questionnaire items) or extremity bias (propensity to select bend points) [118]. As described, we tried to limit these risks by using logic tasks as well as attention checks and by removing completed questionnaires including a conspicuous response behavior. In addition to that, participants of an online survey may have a more comprehensive understanding of online topics. This may have led to a more pronounced representation of awareness of and interaction with PRWs within the study population. Another limitation can be found in the scope of the survey instrument. Although we have expanded and adapted the scales used, they still do not cover all potential areas of investigation. For example, when comparing different rating portals, additional categories can also be included. As we used a quantitative research setting, we did not focus on identifying reasons for the use or non-use of PRWs. Future studies could therefore also focus on the barriers and facilitators of PRW usage. By using Clickworker.com as an online tool for conducting the study, it was possible to reach a large number of individuals. Nevertheless, it must be pointed out at this point that this excluded people without Clickworker.com accounts from participating. Finally, our study results may not represent the entire Austrian population, as our study participants were predominantly more highly educated, female and middle-aged.

Practical implications

Considering the high level of awareness of and interaction with PRWs within the study population, it can be assumed that these platforms have become the first port of call when searching for physicians in the digital environment. This circumstance shows that these portals represent the physician as well as the medical practice to the outside world and for this reason PRWs should be maintained and serviced accordingly by physicians. In contrast to the external effect, PRWs also have a more internal aspect. 14.7% of our study population stated that they had already rated a physician. This result indicates that certain aspects, which otherwise would have to be investigated by market research processes, can be represented on PRWs. These include, for example, the survey of patient satisfaction, the search for critical incidents or the search for indications for the improvement of process-related performance in the medical practice. This demonstrates that PRWs can be used as a cost-effective market research tool from a physician’s perspective. It has to be noted, however, that a sufficient number of reviews should be in place to ensure an objective representation of the medical service performance.

Results show that care should be taken when using ratings to build expectations regarding a future visit of a specific physician. Although many patients read ratings, they are only written by a relatively small proportion of the general population. For this reason, it is necessary to ensure that there are enough reviews to draw a representative digital picture of the physician’s practice.

The present study shows that PRWs are already known by the majority of the study population and that these portals are used by more than half of them. This can basically be interpreted as a good sign for maintainers of PRWs. However, the rating function should be used by significantly more patients to increase the use and trustworthiness of PRWs by both, physicians and patients. It is therefore advisable to develop strategies to increase the willingness of patients to rate physicians on PRWs.

Even for policy makers the results of the current study could support their future decision-making behavior. As we have shown, different levels of digital and eHealth literacy are present in the population. The promotion of further education in the field of digital and eHealth literacy could therefore contribute to people feeling more confident when using digital media and being able to distinguish true information from false information.

Conclusions

Our results have led to a number of relevant findings that grant deeper insights into whether and how different population segments have used PRWs so far. According to the sample used in the current study, the majority of the study population is aware that PRWs exist. Significant correlations regarding the general awareness of PRWs were found within the demographic variables gender, age, education, occupation and area of living as well as for respondents reporting a higher digital and eHealth literacy. Notwithstanding, compared to the awareness regarding alternative rating websites, PRWs are still in the lower midfield, as the upper ranks are occupied by restaurant rating websites, hotel rating websites or movie rating websites. In terms of recall and recognition, the Austrian PRW Docfinder.at received the highest level of brand awareness, followed by the rating websites of Google.at and Herold.at. Besides the results with respect to awareness, the usage of PRWs is also characterized by a high level of interaction. Significant correlations were found within most of the investigated demographic and psychographic variables. Even though our results suggest a high awareness of and interaction with PRWs across the entire population, it seems to be most pronounced among female, middle-aged, more highly educated and urban population segments. Moreover, feelings towards the internet, digital- as well as- eHealth literacy seem to play an important role in the PRWs use behavior. This shows that PRWs represent an effective tool for sustained interaction with female, middle-aged, higher educated, urban and digitally literate population segments. Given the high level of awareness and interaction regarding PRWs within our study population, however, PRWs seem to represent an effective tool for health related communication, at least in the long run.

Supporting information

S1 File. Questionnaire PRWs.

https://doi.org/10.1371/journal.pone.0278510.s001

(PDF)

S2 File. Example equation, introduction text and attention check.

https://doi.org/10.1371/journal.pone.0278510.s002

(PDF)

S3 File. Data selection process.

https://doi.org/10.1371/journal.pone.0278510.s003

(PDF)

References

  1. 1. Bidmon S, Elshiewy O, Terlutter R, Boztug Y. What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2020;22(2):e13830.
  2. 2. Sacristán JA, Aguarón A, Avendaño-Solá C, Garrido P, Carrión J, Gutiérrez A, et al. Patient involvement in clinical research: why, when, and how. Patient Prefer Adherence Dove Press; 2016;10:631–640.
  3. 3. Flodgren G, O’Brien MA, Parmelli E, Grimshaw JM. Local opinion leaders: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev John Wiley & Sons, Ltd; 2019;(6): CD000125. pmid:31232458
  4. 4. Willis E. Patients’ self-efficacy within online health communities: facilitating chronic disease self-management behaviors through peer education. Health Commun Taylor & Francis; 2016;31(3):299–307. pmid:26325224
  5. 5. Kim E, Scheufele DA, Han JY, Shah D. Opinion leaders in online cancer support groups: An investigation of their antecedents and consequences. Health Commun Taylor & Francis; 2017;32(2):142–151. pmid:27192376
  6. 6. Harris KM. How do patients choose physicians? Evidence from a national survey of enrollees in employment‐related health plans. Health Serv Res Wiley Online Library; 2003;38(2):711–732. pmid:12785569
  7. 7. Hoerger TJ, Howard LZ. Search behavior and choice of physician in the market for prenatal care. Med Care Lippincott Williams & Wilkins; 1995;33(4):332–349. pmid:7731276
  8. 8. Lupton D, Donaldson C, Lloyd P. Caveat emptor or blissful ignorance? Patients and the consumerist ethos. Soc Sci Med Elsevier; 1991;33(5):559–568. pmid:1962227
  9. 9. Hibbard JH, Weeks EC. Consumerism in health care: Prevalence and predictors. Med Care JSTOR; 1987;1019–1032.
  10. 10. Baldwin M, Spong A, Doward L, Gnanasakthy A. Patient-reported outcomes, patient-reported information. Patient Patient-Centered Outcomes Res Springer; 2011;4(1):11–17.
  11. 11. Smith KC, Brundage MD, Tolbert E, Little EA, Bantug ET, Snyder CF, Board PRODPSA. Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer Springer; 2016;24(10):4149–4157. pmid:27165054
  12. 12. Bravo P, Edwards A, Barr PJ, Scholl I, Elwyn G, McAllister M. Conceptualising patient empowerment: a mixed methods study. BMC Health Serv Res BioMed Central; 2015;15(1):252. pmid:26126998
  13. 13. Carras MC, Mojtabai R, Cullen B. Beyond social media: a cross-sectional survey of other Internet and mobile phone applications in a community psychiatry population. J Psychiatr Pract NIH Public Access; 2018;24(2):127–135.
  14. 14. Laugesen J, Hassanein K, Yuan Y. The impact of internet health information on patient compliance: a research model and an empirical study. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2015;17(6):e143.
  15. 15. Bidmon S, Terlutter R. Gender differences in searching for health information on the internet and the virtual patient-physician relationship in Germany: Exploratory results on how men and women differ and why. J Med Internet Res 2015;17(6):e156. pmid:26099325
  16. 16. Keränen NS, Kangas M, Immonen M, Similä H, Enwald H, Korpelainen R, et al. Use of information and communication technologies among older people with and without frailty: a population-based survey. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2017;19(2):e29. pmid:28196791
  17. 17. Zucco R, Lavano F, Anfosso R, Bianco A, Pileggi C, Pavia M. Internet and social media use for antibiotic-related information seeking: Findings from a survey among adult population in Italy. Int J Med Inform Elsevier; 2018;111:131–139. pmid:29425624
  18. 18. Fox S. The social life of health information, 2011. Pew Internet & American Life Project Washington, DC; 2011.
    • 19. Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: A systematic review. J Med Internet Res 2013;15(2):e24. pmid:23372115
    • 20. Burkle CM, Keegan MT. Popularity of internet physician rating sites and their apparent influence on patients’ choices of physicians. BMC Health Serv Res [Internet] BMC Health Services Research; 2015;15(1):1–7. pmid:26410383
    • 21. Sharma RD, Tripathi S, Sahu SK, Mittal S, Anand A. Predicting Online Doctor Ratings from User Reviews Using Convolutional Neural Networks. Int J Mach Learn Comput [Internet] 2016;6(2):149–154.
    • 22. Schulz PJ, Rothenfluh F. Influence of Health Literacy on Effects of Patient Rating Websites: Survey Study Using a Hypothetical Situation and Fictitious Doctors. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2020;22(4):e14134. pmid:32250275
    • 23. Han X, Li B, Zhang T, Qu J. Factors associated with the actual behavior and intention of rating physicians on physician rating websites: Cross-sectional study. J Med Internet Res 2020;22(6): e14417. pmid:32496198
    • 24. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: An analysis of physician-rating websites. J Gen Intern Med 2010;25(9):942–946. pmid:20464523
    • 25. Hong YA, Liang C, Radcliff TA, Wigfall LT, Street RL. What do patients say about doctors online? A systematic review of studies on patient online reviews. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2019;21(4):e12521.
    • 26. Mulgund P, Sharman R, Anand P, Shekhar S, Karadi P. Data quality issues with physician-rating websites: Systematic review. J Med Internet Res 2020;22(9): e15916. pmid:32986000
    • 27. Shah AM, Yan X, Qayyum A, Naqvi RA, Shah SJ. Mining topic and sentiment dynamics in physician rating websites during the early wave of the COVID-19 pandemic: Machine learning approach. Int J Med Inform Elsevier; 2021;149:104434. pmid:33667929
    • 28. Rothenfluh F, Germeni E, Schulz PJ. Consumer decision-making based on review websites: Are there differences between choosing a hotel and choosing a physician? J Med Internet Res 2016;18(6):e129. pmid:27311623
    • 29. Rothenfluh F, Schulz PJ. Physician rating websites: What Aspects are important to identify a good doctor, and are patients capable of assessing them? A mixed-methods approach including physicians’ and health care consumers’ perspectives. J Med Internet Res 2017;19(5):e127. pmid:28461285
    • 30. Rothenfluh F, Schulz PJ. Content, Quality, and Assessment Tools of Physician-Rating Websites in 12 Countries: Quantitative Analysis. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2018;20(6):e212. pmid:29903704
    • 31. Lagu T, Metayer K, Moran M, Ortiz L, Priya A, Goff SL, et al. Website characteristics and physician reviews on commercial physician-rating websites. JAMA—J Am Med Assoc 2017;317(7):766–768. pmid:28241346
    • 32. Lagu T, Norton CM, Russo LM, Priya A, Goff SL, Lindenauer PK. Reporting of patient experience data on health systems’ websites and commercial physician-rating websites: mixed-methods analysis. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2019;21(3):e12007. pmid:30916654
    • 33. McLennan S, Strech D, Meyer A, Kahrass H. Public awareness and use of German physician ratings websites: Cross-sectional survey of four North German cities. J Med Internet Res 2017;19(11),e387. pmid:29122739
    • 34. McLennan S. Quantitative ratings and narrative comments on Swiss physician rating websites: frequency analysis. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2019;21(7):e13816. pmid:31350838
    • 35. McLennan S, Strech D, Kahrass H. Why are so few patients rating their physicians on German physician rating websites? A qualitative study. BMC Health Serv Res BioMed Central; 2018;18(1):670. pmid:30157842
    • 36. McLennan S, Strech D, Reimann S. Developments in the frequency of ratings and evaluation tendencies: a review of German physician rating websites. J Med Internet Res JMIR Publications Inc.; 2017;19(8),e299. pmid:28842391
    • 37. Carbonell G, Meshi D, Brand M. The Use of Recommendations on Physician Rating Websites: The Number of Raters Makes the Difference When Adjusting Decisions. Health Commun Taylor & Francis; 2018;1–10. pmid:30222006
    • 38. Syed UA, Acevedo D, Narzikul AC, Coomer W, Beredjiklian PK, Abboud JA. Physician Rating Websites: an Analysis of Physician Evaluation and Physician Perception. Arch Bone Jt Surg Mashhad University of Medical Sciences; 2019;7(2):136–142. pmid:31211191
    • 39. Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res JMIR Publications Inc.; 2012;14(1),e2003. pmid:22366336
    • 40. Trehan SK, Nguyen JT, Marx R, Cross MB, Pan TJ, Daluiski A, et al. Online Patient Ratings Are Not Correlated with Total Knee Replacement Surgeon–Specific Outcomes. HSS J 2018;14(2):177–180. pmid:29983660
    • 41. Liu JJ, Justin Matelski J, Bell CM. Scope, breadth, and differences in online physician ratings related to geography, specialty, and year: Observational retrospective study. J Med Internet Res 2018;20(3):e7475. pmid:29514775
    • 42. Jack RA, Burn MB, McCulloch PC, Liberman SR, Varner KE, Harris JD. Does experience matter? A meta-analysis of physician rating websites of orthopaedic surgeons. Musculoskelet Surg Springer; 2017;1–9.
    • 43. Yan Q, Jensen KJ, Thomas R, Field AR, Jiang Z, Goei C, et al. Digital Footprint of Academic Vascular Surgeons in the Southern United States on Physician Rating Websites: Cross-sectional Evaluation Study. JMIR cardio JMIR Publications Inc., Toronto, Canada; 2021;5(1):e22975. pmid:33625359
    • 44. Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2011;13(4):e1960. pmid:22088924
    • 45. Zhao HH, Luu M, Spiegel B, Daskivich TJ. Correlation of online physician rating subscores and association with overall satisfaction: observational study of 212,933 providers. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2020;22(10):e11258. pmid:33107826
    • 46. Nwachukwu BU, Adjei J, Trehan SK, Chang B, Amoo-Achampong K, Nguyen JT, et al. Rating a sports medicine surgeon’s “quality” in the modern era: an analysis of popular physician online rating websites. HSS Journal® SAGE Publications Sage CA: Los Angeles, CA; 2016;12(3):272–277. pmid:27703422
    • 47. Frost C, Mesfin A. Online reviews of orthopedic surgeons: an emerging trend. Orthopedics SLACK Incorporated Thorofare, NJ; 2015;38(4):e257–e262. pmid:25901617
    • 48. Li S, Feng B, Chen M, Bell RA. Physician review websites: Effects of the proportion and position of negative reviews on readers willingness to choose the doctor. J Health Commun [Internet] 2015;20(4):453–461. pmid:25749406
    • 49. López A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med Springer; 2012;27(6):685–692. pmid:22215270
    • 50. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public Awareness, Perception, and Use of Online Physician Rating Sites. Jama [Internet] 2014;311(7):734–735. pmid:24549555
    • 51. Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: Cross-sectional study. J Med Internet Res 2013;15(8):e187. pmid:23985220
    • 52. Holliday AM, Kachalia A, Meyer GS, Sequist TD. Physician and Patient Views on Public Physician Rating Websites: A Cross-Sectional Study. J Gen Intern Med Journal of General Internal Medicine; 2017;32(6):626–631. pmid:28150098
    • 53. Powell J, Boylan A-M, Greaves F. Harnessing patient feedback data: A challenge for policy and service improvement. Digit Heal 2015;1:205520761561791. pmid:29942547
    • 54. Patel S, Cain R, Neailey K, Hooberman L. Public Awareness, Usage, and Predictors for the Use of Doctor Rating Websites: Cross-Sectional Study in England. J Med Internet Res [Internet] 2018;20(7):e243. pmid:30045831
    • 55. Emmert M, Meier F. An analysis of online evaluations on a physician rating website: Evidence from a german public reporting instrument. J Med Internet Res 2013;15(8):e2655. pmid:23919987
    • 56. Bidmon S, Terlutter R, Röttl J. What explains usage of mobile physician-rating apps results from a web-based questionnaire. J Med Internet Res 2014;16(6):e3122. pmid:24918859
    • 57. Bäumer FS, Kersting J, Kuršelis V, Geierhos M. Rate Your Physician: Findings from a Lithuanian Physician Rating Website. Int Conf Inf Softw Technol Springer; 2018,43–58.
    • 58. Terlutter R, Bidmon S, Röttl J. Who uses physician-rating websites? differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res 2014;16(3):e3145. pmid:24686918
    • 59. Roettl J, Bidmon S, Terlutter R. What predicts patients’ willingness to undergo online treatment and pay for online treatment? Results from a web-based survey to investigate the changing patient-physician relationship. J Med Internet Res 2016;18(2), e5244. pmid:26846162
    • 60. Guetz B, Bidmon S. Awareness of and Interaction with Physician Rating Websites: A Cross-Sectional Study in Austria. 19th International Conference on Research in Advertising (ICORIA 2021), Bordeaux; 2021.
      • 61. Guetz B, Bidmon S. The Impact of Social Influence on the Intention to Use Physician Rating Websites—A Randomized Experiment. Pap Present Online EMAC Conf Madrid; 2021 May 25–28, 2021.
        • 62. Emmert M, Sauter L, Jablonski L, Sander U, Taheri-Zadeh F. Do physicians respond to web-based patient ratings? An analysis of physicians’ responses to more than one million web-based ratings over a six-year period. J Med Internet Res 2017;19(7): e7538. pmid:28747292
        • 63. Bidmon S. Patient Satisfaction with the Primary Care Physician and Usage of Physician Rating Websites: How Do They Relate to Each Other? Adv Advert Res X Springer; 2019,15–28.
        • 64. Haug M, Gewald H. Why do I rate?-Shedding Light on the Factors Influencing the Participation on Physician Rating Websites. Proc 52nd Hawaii Int Conf Syst Sci 2019.
          • 65. Greaves F, Pape UJ, Lee H, Smith DM, Darzi A, Majeed A, et al. Patients’ ratings of family physician practices on the internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res JMIR Publications Inc.; 2012;14(5), e2280. pmid:23076301
          • 66. Hao H. The development of online doctor reviews in China: An analysis of the largest online doctor review website in China. J Med Internet Res 2015;17(6):e134. pmid:26032933
          • 67. Zhang W, Deng Z, Hong Z, Evans R, Ma J, Zhang H. Unhappy Patients Are Not Alike: Content Analysis of the Negative Comments from China’s Good Doctor Website. J Med Internet Res JMIR Publications Inc.; 2018;20(1), e8223. pmid:29371176
          • 68. Li J, Liu M, Li X, Liu X, Liu J. Developing embedded taxonomy and mining patients’ interests from web-based physician reviews: Mixed-methods approach. J Med Internet Res 2018;20(8),e8868. pmid:30115610
          • 69. Hao H, Zhang K. The voice of Chinese health consumers: a text mining approach to web-based physician reviews. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2016;18(5):e108. pmid:27165558
          • 70. Deng Z, Hong Z, Zhang W, Evans R, Chen Y. The effect of online effort and reputation of physicians on patients’ choice: 3-wave data analysis of china’s good doctor website. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2019;21(3):e10170. pmid:30848726
          • 71. Wu QL, Tang L. What Satisfies Parents of Pediatric Patients in China: A Grounded Theory Building Analysis of Online Physician Reviews. Health Commun Taylor & Francis; 2022;37(10): 1329–1336.
          • 72. Hao H, Zhang K, Wang W, Gao G. A tale of two countries: international comparison of online doctor reviews between China and the United States. Int J Med Inform Elsevier; 2017;99:37–44. pmid:28118920
          • 73. Wang J-N, Chiu Y-L, Yu H, Hsu Y-T. Understanding a nonlinear causal relationship between rewards and physicians’ contributions in Online Health Care Communities: Longitudinal Study. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2017;19(12):e9082. pmid:29269344
          • 74. Li J, Li F, Liu X, Ma L. Differentiation Strategy in Online Physician Competition: Does Specialization Matter? Telemed e-Health Mary Ann Liebert, Inc., publishers 140 Huguenot Street, 3rd Floor New …; 2020;26(5):629–638. pmid:31584342
          • 75. Atkinson S. Current status of online rating of Australian doctors. Aust J Prim Health CSIRO; 2014;20(3):222–223. pmid:24852125
          • 76. Kroon M, Park JS. Negative reviews online: an exploratory analysis of patient complaints about dental services in Western Australia. Aust Dent J Wiley Online Library; 2022;67(2):138–147. pmid:34877684
          • 77. Bird S. Patients’ use of social media: e-rating of doctors. Aust Fam Physician [Internet] 2014;43(12):885–886. pmid:25705741
          • 78. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Parental awareness and use of online physician rating sites. Pediatrics American Academy of Pediatrics Elk Grove Village, IL, USA; 2014;134(4):e966–e975. pmid:25246629
          • 79. Muller M, Lundblad N, Mayrhofer W, Söderström M. A comparison of human resource management practices in Austria, Germany and Sweden. Ger J Hum Resour Manag SAGE Publications Sage UK: London, England; 1999;13(1):67–81.
          • 80. Frankó K. Intercultural communication—The effect of cultural differences in everyday working environment of companies (from Germany, Switzerland, Austria) in Hungary. Knowl Learn Glob Empower Proc Manag Knowl Learn Int Conf 2012 International School for Social and Business Studies, Celje, Slovenia; 2012,767–773.
            • 81. Apfelthaler G, Muller HJ, Rehder RR. Corporate global culture as competitive advantage: learning from Germany and Japan in Alabama and Austria? J World Bus Elsevier; 2002;37(2):108–118.
            • 82. Koopmans R. Trade-offs between equality and difference: Immigrant integration, multiculturalism and the welfare state in cross-national perspective. J Ethn Migr Stud Taylor & Francis; 2010;36(1):1–26.
            • 83. Schaeffer D, Berens E-M, Vogt D. Health literacy in the German population: results of a representative survey. Dtsch Arztebl Int Deutscher Arzte-Verlag GmbH; 2017;114(4):53–60.
            • 84. Hüsers J, Hübner U, Esdar M, Ammenwerth E, Hackl WO, Naumann L, et al. Innovative power of health care organisations affects IT adoption: A bi-national health IT benchmark comparing Austria and Germany. J Med Syst Springer; 2017;41(2):33. pmid:28054195
            • 85. Gall W, Aly A-F, Sojer R, Spahni S, Ammenwerth E. The national e-medication approaches in Germany, Switzerland and Austria: A structured comparison. Int J Med Inform Elsevier; 2016;93:14–25.
            • 86. Wong DK-K, Cheung M-K. Online health information seeking and ehealth literacy among patients attending a primary care clinic in Hong Kong: A cross-sectional survey. J Med Internet Res JMIR Publications Inc., Toronto, Canada; 2019;21(3):e10831.
            • 87. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res JMIR Publications Inc.; 2012;14(1),e1619.
            • 88. Reddy P, Sharma B, Chaudhary K. Digital literacy: A review of literature. Int J Technoethics IGI Global; 2020;11(2):65–94.
            • 89. Haug M. Why do I rate?—Shedding Light on the Factors Influencing the Participation on Physician Rating Websites. 2019;6:4355–4364.
            • 90. Dunkelman MJ. Next-door strangers: The crisis of urban anonymity. Hedgehog Rev Institute for Advanced Studies in Culture; 2017;19(2):44–57.
            • 91. Walter E. Cambridge advanced learner’s dictionary. Cambridge university press; 2008. ISBN:3125179882
              • 92. Rossiter JR. ‘Branding’explained: defining and measuring brand awareness and brand attitude. J Brand Manag Springer; 2014;21(7–8):533–540.
              • 93. Percy L, Rossiter JR. A model of brand awareness and brand attitude advertising strategies. Psychol Mark Wiley Online Library; 1992;9(4):263–274.
              • 94. Robertson KR. Recall and recognition effects of brand name imagery. Psychol Mark Wiley Online Library; 1987;4(1):3–15.
              • 95. Du Plessis E. Recognition versus recall. J Advert Res World Advertising Research Center Ltd.; 1994;34(3):75–92.
              • 96. Sinha G, Shahi R, Shankar M. Human computer interaction. 2010 3rd Int Conf Emerg Trends Eng Technol IEEE; 2010,1–4.
                • 97. Sundar SS. Social psychology of interactivity in human-website interaction. Oxford Handb internet Psychol Oxford University Press; 2012.
                  • 98. Haug M, Gewald H. Are Friendly and Competent the Same?:-The Role of the Doctor-Patient Relationship in Physician Ratings. Proc 2018 ACM SIGMIS Conf Comput People Res ACM; 2018,157.
                    • 99. Roettl J, Terlutter R. The same video game in 2D, 3D or virtual reality–How does technology impact game evaluation and brand placements? PLoS One Public Library of Science San Francisco, CA USA; 2018;13(7):e0200724. pmid:30028839
                    • 100. Liff JM, Chow W, Greenberg RS. Rural–urban differences in stage at diagnosis. Possible relationship to cancer screening. Cancer Wiley Online Library; 1991;67(5):1454–1459.
                    • 101. Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res JMIR Publications Inc.; 2006;8(4),e507. pmid:17213046
                    • 102. Soellner R, Huber S, Reder M. The concept of ehealth literacy and its measurement: German translation of the eHEALS. J Media Psychol 2014;26(1):29–38.
                    • 103. Skarmeas D, Leonidou CN. When consumers doubt, Watch out! The role of CSR skepticism. J Bus Res Elsevier Inc.; 2013;66(10):1831–1838.
                    • 104. Keller KL. Building customer-based brand equity: A blueprint for creating strong brands. Marketing Science Institute Cambridge, MA; 2001.
                      • 105. Oppenheimer DM, Meyvis T, Davidenko N. Instructional manipulation checks: Detecting satisficing to increase statistical power. J Exp Soc Psychol Elsevier; 2009;45(4):867–872.
                      • 106. Kung FYH, Kwok N, Brown DJ. Are Attention Check Questions a Threat to Scale Validity? Appl Psychol 2018;67(2):264–283.
                      • 107. Aguinis H, Villamor I, Ramani RS. MTurk research: Review and recommendations. J Manage SAGE Publications Sage CA: Los Angeles, CA; 2021;47(4):823–837.
                      • 108. Aral S. The problem with online ratings. MIT Sloan Manag Rev Massachusetts Institute of Technology, Cambridge, MA; 2014;55(2):47–52.
                      • 109. Musante MD, Bojanic DC, Zhang J. An evaluation of hotel website attribute utilization and effectiveness by hotel class. J Vacat Mark SAGE Publications Sage UK: London, England; 2009;15(3):203–215.
                      • 110. Golbeck J. Filmtrust: movie recommendations from semantic web-based social networks. CCNC 2006 2006 3rd IEEE Consum Commun Netw Conf 2006 Citeseer; 2006,1314–1315.
                        • 111. Yoo KH, Gretzel U. What motivates consumers to write online travel reviews? Inf Technol Tour Cognizant Communication Corporation; 2008;10(4):283–295.
                        • 112. Dixit S, Badgaiyan AJ, Khare A. An integrated model for predicting consumer’s intention to write online reviews. J Retail Consum Serv Elsevier; 2019;46:112–120.
                        • 113. Michael L, Otterbacher J. Write like I write: Herding in the language of online reviews. Eighth Int AAAI Conf Weblogs Soc Media 2014.
                          • 114. Kaur K, Singh T. What Motivates Consumers to Write Online Reviews? Qualitative Research in the Indian Cultural Context. J Glob Mark Taylor & Francis; 2021;170–188.
                          • 115. Conner M, Norman P. Health behaviour: Current issues and challenges. Psychology & health; 2017;32(8):895–906. pmid:28612656
                          • 116. Inchley J, Currie D. Growing up unequal: gender and socioeconomic differences in young people’s health and well-being. Health Behaviour in School-aged Children (HBSC) study: international report from the 2013/2014 survey. World Health Organization; 2016. ISBN:9289051361
                            • 117. Wakefield MA, Loken B, Hornik RC. Use of mass media campaigns to change health behaviour. Lancet Elsevier; 2010;376(9748):1261–1271. pmid:20933263
                            • 118. Nuno A, John FAVS. How to ask sensitive questions in conservation: A review of specialized questioning techniques. Biol Conserv Elsevier; 2015;189:5–15.