Reliability of information about the use of antiepileptic drugs during pregnancy from three major web search engines in China

Objectives The objective of this study was to assess the reliability of online information, as provided by three major search engines in China, about the usage of antiepileptic drugs (AEDs) during pregnancy. Method Over eight weeks, six physicians conducted a literature search on six computers and six smartphones at a frequency of once per week. During each web search on each computer and smartphone, three major search engines in China were used, namely, Baidu, Sogou and 360. The search terms used were a combination of words, including one AED name (valproate/oxcarbazepine/levetiracetam/lamotrigine) and one Chinese word ("huaiyun" or "renshen", which means pregnancy in Chinese). The top ten websites retrieved from each search were recorded. After the content of each website was evaluated, the sites were categorized into 9 types. Meanwhile, commercial advertisements on each web page were also registered. Results A total of 16,411 search results were assessed. After excluding the redundant web pages, 4840 search results were included in the data analysis. Only 12.05% of the search results were reliable, 47.75% were partly reliable, and 40.21% were unreliable. A total of 4139 (85.52%) webpages contained commercial advertisements. The results from a multivariate analysis suggested that websites with no advertisements and professional websites have an independent positive impact on reliability. Conclusion Overall, little information on AED usage during pregnancy provided by major search engines in China was reliable. Practice implications Accurate and professional online information for female patients with epilepsy should be provided through major efforts by the government, search engine companies, professional websites and epilepsy physicians.


Introduction
Estimates indicate that at least 65 million people suffer from epilepsy globally [1], and approximately 30% of patients are women of childbearing age [2]. Although the majority of women with epilepsy (WWE) can experience safe pregnancies, both mothers and babies are at an increased risk of poor pregnancy outcomes [3]. Most WWE must continue treatment with antiepileptic drugs (AEDs) and are thus concerned about not only the danger of seizures to both the mother and fetus but also the possible adverse teratogenic events of AEDs, which may result in congenital malformations and neuropsychological impairments. Preconceptual counseling is recommended and necessary for WWE [4], but the situation remains unsatisfactory [5,6], especially in China, where the number of epilepsy specialists is rather inadequate.
WWE are eager to acquire information regarding issues such as the influence of AEDs on their offspring and family planning [6]. Most pregnant women (91.9%) have access to the Internet, and 88.7% of them utilize it as an informational resource to navigate pregnancyrelated decisions [7]. With the popularity of computers and smartphones, both devices have become commonly used to search for information. However, patients often have difficulty in judging the quality and accuracy of information and tend to believe and rely on the information they find [8]. Increased demand for reliable medical resources is widespread [9]. It remains unclear whether the most popular web search engine in China could be a source of reliable information about the use of AEDs during pregnancy.
The aim of this study was to investigate the reliability of information about AED usage during pregnancy retrieved by Baidu/Sogou/360, which are the most popular search engines in China after Google's exit from mainland China in 2010 [10]. Given the rapid increase in the number of Chinese mobile Internet users in recent years, we performed the searches on both computers and smartphones. To obtain medium-and long-term search results in a relatively steady manner, we performed the searches once a week for eight consecutive weeks.

Searches
Six physicians with expertise in epilepsy from West China Hospital conducted the searches over the course of 8 weeks from November 2016 to January 2017 (for the search process, see Fig 1). Every week, each physician searched the Internet via three different search engines (Baidu, Sogou and 360) on a computer and smartphone simultaneously. The key words for each search were a combination of words in Chinese including an AED name (valproate acid/ oxcarbazepine/levetiracetam/lamotrigine) and "huaiyun"/"renshen" (both terms mean pregnancy in Chinese and are commonly used). Therefore, a total of eight pairs of search terms were used. The four types of AEDs were the most widely used AEDs among WWE in China. Moreover, the standard names of the four drugs rather than the branded names were adopted for the search. The top ten websites retrieved in each search were recorded (for a total of 480 web page results, accumulated from each computer and smartphone and the above three search engines by one physician every week). In total, 2880 web pages were browsed every week by six physicians.
Furthermore, all of the engines applied personalized searches, meaning that the presented website ranking is customized and filtered by search history. To avoid this condition as much as possible, we used six different computers and smartphones without signing into search engine accounts and cleared their search histories before conducting each search task.

Classification of information type
The top ten websites retrieved by each search were opened and evaluated carefully by physicians for classification. Six physicians independently classified the websites according to the contents. The nine categories of websites were online pharmacy websites; hospital websites (homepages of hospitals, mainly private hospitals); web-based encyclopedias (e.g., Baidu encyclopedia, Wikipedia); blogs; instructions for specific drugs; professional websites (academic organizations, academic journals); question-and-answer websites with answers from neurologists (Q&A from neurologists); question-and-answer websites with answers from physicians but not neurologists (Q&A from physicians); and question-and-answer websites with answers from Internet users (Q&A from Internet users). If the web pages could not be opened or the information presented on the websites was irrelevant to use of AEDs among WWE during pregnancy, then the websites were classified as void web pages. Each website was also classified as having or not having advertisements by the physicians. Since a large number of redundant websites could be searched by different physicians, most websites were actually evaluated and classified by more than one physician. Disagreements were resolved by inviting all the six physicians to vote, and the majority vote was used.
provided by the website was totally in accord with the guidelines, the website was evaluated as "reliable". If not all of the information was correct, the website was rated as "partly reliable". "Unreliable" referred to websites that were totally incorrect and had apparently erroneous information. Disagreements were resolved by inviting all six physicians to vote, and the majority vote was used. In our study, 91% of the evaluations were consistent.

Data analyses
Void web pages were not included in the analysis. Variables were reduced to a set of dichotomies to create dummy variables. Thus, a "first ranked variable" was reduced to a variable with a rank of first denoted as 1 and nonfirst rank denoted as 0, a "second ranked variable" had a second rank denoted as 1 and a nonsecond rank denoted as 0, etc. Nominal data were also reduced to dummy variables, including search engines.
Qualitative data are presented as n (%). Univariate and multivariate analysis were used to determine the factors that impacted the reliability of the information provided by websites. Different types of analysis methods were used for different variables in the univariate analysis. Specifically, we used The Wilcoxon rank sum test for ordinal response variables, the Kruskal-Wallis rank sum test for multiple independent rank-order data, and Spearman's rank correlation analysis for rank-order variables. Furthermore, Bonferroni correction was used to adjust for significant values. Cumulative logistic regression was used to evaluate the independent effect on reliability in the multivariate analysis. Independent variables were screened by the bidirectional stepwise regression method, and the significance level for entering effects was set as 0.30. P values of less than 0.05 were considered to be significant. Hypothesis tests were twosided. Analyses were performed using SAS version 9.4 (SAS institute Inc.).

Results
A total of 23,040 web pages were retrieved. After excluding 6629 void web pages (defined as web pages that could not be opened or provided irrelevant information), the remaining 16,411 web pages were evaluated and classified by physicians. Among these 16,411 Internet addresses, we identified addresses that occurred more than once and reorganized the information by clustering the observations. A total of 4840 unique sites were identified and included in the data analysis. Among the 4840 sites, 12.05% (583/4840) were determined to be reliable, 47.75% (2311/4840) were partly reliable, and the remaining 40.21% (1946) were unreliable.
Regarding the type of the web pages, online pharmacy websites were the most common type, followed by Q&A from physicians and Q&A from Internet users. Drug instructions, web-based encyclopedias and professional websites were the three least common types (Table 1). Professional websites only accounted for 6.92% (335/4840) of the 4840 web pages. The two search terms "huaiyun" and "renshen" could be found almost equally in most types of web pages, except for professional websites and web-based encyclopedias, in which the term "renshen" was much more common (Table 1). On a total of 505 web pages, the two terms appeared concurrently, resulting in a total of 5345 occurrences of "huaiyun" and "renshen" rather than 4840. Univariate analysis was used to determine the reliability of different types of web pages, which showed significant differences in search devices, search engines, search terms, drugs, the presence of advertisements, types of sites, and the ranking of search results (Table 2). After Bonferroni correction and paired comparisons, the results suggested that web pages from Baidu, professional websites, those about Levetiracetam, those ranking in the last 5 among the top ten, those without advertisements, and those using "renshen" were more reliable. A significant difference between computer and smartphone searches was found, with computer searches being more reliable (P = 0.0456) in the univariable analysis, while the multivariate analysis showed that the device used had no significant impact on reliability. Paired comparison between sources of information showed no statistically significant differences between Q&A from physicians and Q&A from Internet users as well as Q&A from neurologists and Q&A from Internet users. The other paired comparisons were significantly different. The most reliable website type was professional websites, followed by blogs, webbased encyclopedias, drug instructions, hospital websites, Q&A from neurologists, Q&A from Internet users, Q&A from physicians and online pharmacy websites. Significant differences in reliability were found between Q&A from neurologists and Q&A from physicians (P = 0.003).
Most websites [85.52% (4139/4840)] in the top ten results had advertisements. Interestingly, as search result ranks decreased, the proportion of advertisements decreased, and the web pages seemed to be more reliable (Fig 2).
Multivariate analysis was used to determine the independent factors of reliability and suggested that professional websites had the largest positive impact on reliability (standardized coefficient = 0.95), while hospital websites most negatively impacted the reliability with a standardized coefficient of -0.8447 (

Discussion
Pregnant women increasingly use the Internet as an important source of health information [11]. However, many pregnant women are confused by inaccurate information on websites Reliability of information about the use of antiepileptic drugs during pregnancy on the internet and, as a consequence, might experience anxiety [12]. To accurately assess the reliability and type of information about on the use of AEDs during pregnancy among WWE via the most popular search engines in China, we gathered and assessed 4840 unique websites.
There are several main findings. First, in most cases, the top ten search results on the main search engines in China were unreliable. Second, professional websites were seldom found among the top ten search results. Third, as the ranks of search results decreased, commercial advertisements increased, and the information became less reliable.
The Internet, with its all-pervasive influence, has been recognized as a potential means for transforming health care. As a well-known way of retrieving health-related information, web search engines enhance information availability and facilitate doctor-patient interaction [13], particularly in some lesser-known subspecialties. However, although the reliability of information and its impact on individuals have remained unclear until recently, the amount of health information on the Internet has been growing on a larger scale, and most users check only the top websites listed on the first page of search results [14]. In such circumstances, the reliability of information is clearly more important than the availability of information. Our study found that much information on websites is useless and even misleading. We found that only 12.05% of the search results were reliable. A similar study found that 19.5% of epilepsy-related information found on the main search engines in China in 2014 was correct and complete [15]. The situation seems dismal in China.
Among the 9 categories, professional websites seem to be most reliable, yet they represent a very small proportion and their contents are often too technical for patients to understand. There are many professional websites, including the China Association Against Epilepsy (http://www.caae.org.cn) and Dingxiangyuan (http://www.dxy.cn), but these professional websites are seldom found in the top ten search results. A similar study found that the representation of the Epilepsy Foundation website was 10% in Google search results [16]. Its representation is clearly higher than that in our study. Q&A websites, which are online counseling services, are currently very popular and seem to have a bright future. However, the reliability of the information provided by Q&A websites is unsatisfactory (e.g., stating that acupuncture and traditional Chinese medicine are better treatments than AEDs). Surprisingly, Q&A websites with answers from physician/neurologists did not offer high-quality information. Physicians lacking knowledge of epilepsy often provided incorrect information, while neurologists who are not familiar with AED usage during pregnancy always provided incomplete  [17]. Unlike private hospitals in the West, Chinese private hospitals generally only seek profit and ignore patients' health (e.g., even suggesting that WWE should implant subcutaneous magnets to cure epilepsy). This finding could explain the unreliable information provided by hospital websites. A large number of the web pages retrieved contained commercial advertisements. Most of the advertisements came from private hospitals and turned out to be specific links to their own websites. These advertisements were full of exaggeration and false claims to lure epilepsy patients into treatment. Investors of private hospitals seek a high financial return and make their websites more accessible by competitively bidding to obtain higher search rankings. Therefore, it is easy to understand why the search results contained so much misleading and inadequate information. The higher the rank of the search results is, the more commercial advertisements they contain and the less reliable they are (Fig 2). In this case, the most typical example is the death of Wei Zexi. Wei Zexi, a university student who died in April 2016 from a rare form of cancer, had sought treatment from a private hospital that appeared at the top of the list on his Baidu web search [18]. However, the treatment he received with an abandoned technique only accelerated his death. The incident caused widespread concern all over the world.
Proper search strategies can increase the probability of obtaining accurate information. Our study showed that the word "huaiyun" was more useful than the word "renshen" for obtaining more credible information. This variability in the results obtained by using different search options highlights the importance of guiding patients to patient-friendly, high-quality websites [19]. Therefore, epilepsy doctors should help patients improve their search strategies to find accurate information.
More than 660 million people use the Baidu mobile search engine every month, meaning that smartphones are an important Internet search tool for WWE. In our study, there was no significant difference between computers and smartphones in the reliability of their content. Previous studies have evaluated differences in the search results of different Google accounts and found some variability in the top 3 results, but the overall differences were different than expected [16]. In this study, we did not use the sign-in accounts of the engines in order to avoid the influence of the sign-in web history. Although Baidu is the largest search engine company in China and is frequently enmeshed in scandals in the medical field [20], it was still more reliable than the other two search engines.

Limitations
Our study has a limitation in that we used three main search engines to examine the reliability of website information yet were unable to include Google, the search engine used throughout the world. Health care information from Google might result in a different conclusion, and AED-related information might be more reliable.

Conclusion
In conclusion, as an important part of e-health, web search engines are a very efficient and low-cost way of obtaining health care information [21]. However, most current information on AEDs used by pregnant women is not only unreliable but also limited by the major search engines in China. Some information, largely provided by private hospitals, may mislead patients to obtain inappropriate treatments due to the perceived economic benefits.