Improved patient satisfaction and diagnostic accuracy in skin diseases with a Visual Clinical Decision Support System—A feasibility study with general practitioners

Patient satisfaction is an important indicator of health care quality, and it remains an important goal for optimal treatment outcomes to reduce the level of misdiagnoses and inappropriate or absent therapeutic actions. Digital support tools for differential diagnosis to assist clinicians in reaching the correct diagnosis may be helpful, but how the use of these affect patients is not clear. The primary objective of this feasibility study was to investigate patient experience and satisfaction in a primary care setting where general practitioners (GPs) used a visual clinical decision support system (CDSS) compared with standard consultations. Secondary objectives were diagnostic accuracy and length of consultation. Thirty-one patients with a dermatologist-confirmed skin diagnosis were allocated to consult GPs that had been randomized to conduct either standard consultations (SDR, n = 21) or CDSS (n = 16) on two separate study days one week apart. All patients were diagnosed independently by multiple GPs (n = 3–8) in both the SDR and CDSS study arms. Using the CDSS, more patients felt involved in the decision making (P = 0.05). In addition, more patients were exposed to images during the consultations (P = 6.8e-27), and 83% of those that were shown images replied they felt better supported in the consultation. The use of CDSS significantly improved the diagnostic accuracy (34%, P = 0.007), and did not increase the duration of the consultation (median 10 minutes in both arms). This study shows for the first time that compared with standard GP consultations, CDSS assist the GP on skin related diagnoses and improve patient satisfaction and diagnostic accuracy without impacting the duration of the consultations. This is likely to increase correct treatment choices, patient adherence, and overall result in better healthcare outcomes.


Background and rationale for the study
Differential diagnosis generator (DDX) VisualDx is a computer-assisted differential diagnosis tool with over 40,000 images of medical conditions, most related to dermatology. It can be accessed using mobile technology such as smartphones and tablet computers. Unlike textbooks indexed by disease, it allows physicians to enter patient descriptions like age, sex, and symptoms. The tool rapidly generates images of potential diagnoses that mostly match the entered criteria, enabling physicians to "rule in or out" by comparing a patient's skin condition with the images. Furthermore, it also provides concise disease-specific information as well as information on management, therapy, and handouts for patients. The content is written, reviewed, and frequently updated by more than 100 experts in the fields of medicine ( In the U.S. the Institute of Medicine recommends the use of healthcare IT resources, such as VisualDx and to date it is has been licensed in over 1500 hospitals and large clinics worldwide. More than 85 medical schools in the U.S. are using this tool to teach students how to quickly build a differential diagnosis. VisualDx promotes lifelong learning based on information-assisted decision-making rather on memory-based education. Rationale for the study Diagnostic accuracy and user satisfaction are usually the main focus in studies on differential diagnoses generators (DDX), including VisualDx, whereas patients' perspective is often not investigated. Despite the widespread use of VisualDx there is no data showing how the tool affects patient satisfaction. As an important indicator for quality healthcare, patient satisfaction and/or patient engagement should be included in those kinds of studies.
VisualDx has the potential to improve the quality of consultations by (i) engaging patients in the decision-making process, e.g. sharing images and information reassures patients and builds confidence in the doctor's diagnosis, (ii) reducing the length of consultations, and (iii) increasing the diagnostic accuracy. We will conduct a feasibility study to assess patient satisfaction as well as lengths of consultation and diagnostic accuracy of VisualDx consultations. The Pa tient S atisfaction through Vi sual Clinical D ecision S upport System (PSAVIDS) study takes place in Germany. Patients with skin diseases that are first diagnosed by experienced dermatologists (gold July 6, 2016 standard) will subsequently be examined by a group of GPs using VisualDx. A week later the same cohort of patients will be examined by another group of GPs without the use of VisualDx or other DDX. GPs are randomly assigned to one of the groups. We will analyze and compare above mentioned outcome measures of both groups.
To our knowledge this is the first study investigating the impact of VisualDx on patient satisfaction. Moreover, this feasibility study will also reveal whether the research design and implementation is practical, thus it will set the foundation of future large scale RCTs.

Objectives
The overall purpose of this study is to investigate whether the use of VisualDx by GPs improves the quality of skin complaints consultations in terms of patient satisfaction, consultation length, and diagnostic accuracy compared with standard medical consultations.

Primary objective :
• To directly compare the patient-satisfaction scores of VisualDx consultations with standard medical consultations.

Secondary objectives :
• To directly compare the consultation lengths of VisualDx consultations with standard medical consultations. • To evaluate the diagnostic accuracy of VisualDx consultations against a gold standard and to compare it with the diagnostic accuracy of standard medical consultations.

Study design 4.1 Statement of design
In order to investigate the impact of VisualDx consultations on patient satisfaction, consultation length, and diagnostic accuracy, a feasibility study will be conducted. Patients with confirmed skin disorders (chronic and newly diagnosed) will be recruited by two dermatologists from their own pool of patients. In addition, GPs will be enrolled and randomized to the VisualDx arm and control arm, respectively. The so called 'test patients' will then be examined by physicians in the VisualDx arm as well as in the Control arm (standard medical consultations; Figure 1 ). Patient satisfaction in both arms will be assessed after each examination by questionnaires, while case report forms (CRF) will be used to evaluate consultation lengths and diagnostic accuracy.
PSAVIDS study protocol

1.) Test patients
Inclusion criteria (ALL criteria must be met) • Male and female German residents ≥ 18 years of age • Suffering from skin disease/s (chronic and/or newly diagnosed) • Skin diseases must be confirmed by two dermatologists -gold standard • Skin diseases should not change significantly over the study period (to ensure that data is comparable) To ensure a mix of complexity of the cases, a broad range of skin conditions will be included.

2.) Physicians
Inclusion criteria (ALL criteria must be met) Exclusion criteria (one of the following criteria) • Previous use of an electronic differential builder (e.g. UpToDate, DynaMed, MCConsult, MediBox) • Extra education / special interest in dermatology • Medical students

1.) Test patients:
40 patients with confirmed skin diseases will be recruited.

3.) Number of examinations: 480
Each test patient is examined by 6 GPs, i.e. 6 diagnoses per patient, resulting in a total of 240 diagnoses per study arm ( Figure 2 ). A total of 480 patient satisfaction questionnaires and diagnoses, respectively, will be analyzed. Power calculation; the sample size is determined to be minimum 150 diagnoses for each group, based on the two independent groups of GPs, where outcome measures was based on a diagnostic accuracy in SDR and CDSS of 45∓5%, and 65∓5%, with an alpha of 0.05 resulted in a power calculation of 100%.

Study duration
Total study duration: 7 months (see Figure

Ethics exemption
Since the PSAVIDS does not pose any health and psychological risks to test patients and GPs (see 6. Potential risks for participants ), it is anticipated that this study does not need ethics approval from an Institutional Review Board. Proof of waiver: informal email.

Recruitment of physicians
Recruitment of GPs will be carried out in cooperation with the Bezirks Landesärztekammer Stade (Regional medical association of Stade, Germany).

Recruitment of patients
40 test patients will be recruited by two experienced dermatologists (Drs Breitbart) from their own pool of patients. The test patients need to meet ALL inclusion criteria ( 4.2 Study population ). To include a broad range of skin diseases, patients with 'easy to diagnose' skin conditions (e.g. atopic dermatitis, psoriasis, pityriasis rosea) and more 'difficult to diagnose' skin conditions (e.g. basal cell carcinoma, erythema nodosum, nodular vasculitis) will be recruited. Care will be taken to balance the proportions of 'easy to diagnose' and 'difficult to diagnose' skin conditions.
At recruitment a gold standard CRF for each enrolled patient will be completed by the dermatologists to confirm eligibility and to collect baseline data. These CRFs will be used as references (see 8.1 Data capture ) for the evaluation of the diagnostic accuracy. Each CRF will include a photo of the skin disease taken at the time of recruitment.
Photos will be taken at recruitment, on the VisualDx execution day, and on the control arm execution day. This will help to document changes that might appear to skin lesions/conditions during the preparation phase and study execution phase (internal control).

Informed consent
Written consent will be obtained from each test patient prior to study entry. The dermatologists will fully inform test patients about the study objectives, the design, the VisualDx intervention and about any risks that are associated with it ( 6. Potential risks for participants ). Furthermore, it will be explained how confidentiality of data will be maintained, especially with respect to the information about the participant which would otherwise be known only to the dermatologist, but in this case will be made known to the study team. Test patients will be asked to give their consent for their data to be included in a research database (see 8.1 Data capture ). The dermatologist has to mention the participant's right to withdraw from the study at any time. The patient has the option NOT to participate and it will be explained that nonparticipation has no effect on the future doctor-patient relationship.

1.) Test patients
Recruited test patients will receive financial compensation for each study day (250€/day) and for their travel costs (up to 50€/person/day). The total amount will be wire transferred to patients only after completion of the 2 nd study execution day.

2.) Physicians
Enrolled physicians will receive financial compensation for the study day (500€) and for their travel costs (up to 50€/person).

Randomization
Upon enrollment, 60 GPs will be allocated at random to the VisualDx arm and the control arm. Names of the physicians will be thrown in a hat with the first 30 drawn out allocated to the VisualDx arm and the remainder to the control arm. This random allocation minimizes selection bias and the effect of possible confounders.

VisualDx intervention (webinar)
GPs (n=30) randomized to the VisualDx arm will receive an invitation for a training webinar that will be hosted by VisualDx representatives. Attendance is mandatory, therefore two dates will be offered two weeks in advance of the study ( Figure 3 ). The aim of this webinar is to train GPs how to build a differential diagnosis with VisualDx and to demonstrate how patients can be engaged. In addition, the study aims, the design and execution will be explained briefly. Each GP will receive a tablet computer and his individual VisualDx log in details by mail prior to the training webinars. The tablet computer will have the VisualDx application installed. Although VisualDx has been described as intuitive to use, GPs will be encouraged to use the tool in their practice in advance of the study to gain confidence in building differential diagnoses. They will have 1 to 2 weeks for training and practicing ( Figure 3) . Papier et al. (2001) showed that with minimal training (10 minutes) non-dermatologists were able to effectively use the tool, so 1 week minimum for practicing should be sufficient.

Control arm
On the Control arm execution day each GP (n=30) will receive a tablet computer with Internet access to complete the electronic CRFs. They are allowed to use the Internet and their own textbooks as support tools for their standard medical consultations. This will mimic 'real world' situations. However, access to the VisualDx application as well as to similar applications (e.g. UpToDate, DynaMed, MCConsult, MediBox) and websites will be blocked.

Study execution
The PSAVIDS study will take place over two consecutive Saturdays. First the VisualDx arm will be executed. 40 test patients will be randomized to 5 blocks of 8 patients ( Figure 2 ), while 30 GPs will be randomized to 5 blocks of 6 physicians. Each patient will then be examined by 6 GPs. Immediately after each consultation patients need to complete the patient satisfaction questionnaire.
This procedure will be repeated on the following Saturday for the Control arm. The same set of test patients but different physicians will participate. It is estimated that a 'full visit' takes approximately 20 minutes (10 minutes consultation, 5 minutes questionnaire, 5 minutes break). In addition, on each study execution day photos of the skin conditions will be taken before the consultations start (see 0).

9 Instructions for participants
The following instructions will be given to participants to ensure a smooth study execution:

1) Test patients
• A clinician or medical assistant takes a photo of the skin disease at the beginning of each study day. • Patients are requested not to disclose any information regarding their skin disease to the GPs during the consultations , i.e. the correct diagnosis, any treatment plans, and advice and information they received from their dermatologists. Furthermore, patients are encouraged to act as if they were new patients with an undiagnosed skin disease. • After each consultation patients need to complete the electronic 'patient satisfaction questionnaire' which takes maximum 5 minutes.

2) Physicians
• GPs have maximum 10 minutes for a consultation (completion of CRF included) • Each consultation will be documented on an electronic CRF. • On the VisualDx execution day, GPs will use the tablet computers they received for training. GPs will use VisualDx to go straight to a known diagnosis page to use images and information to educate the patient or will use VisualDx to build a differential diagnosis. It is not mandatory to build a differential diagnosis if the diagnosis is known to the GP. They will be encouraged to use VisualDx to educate the patient with images and information. The tablet computers will only grant access to VisualDx, to the electronic CRFs, and to an English-German dictionary; access to other websites will be denied. • Control arm: for the standard medical consultations, GPs are allowed to use supporting tools as described in 5.7 Control arm .

1.) Test patients
There are hardly any risks involved for test patients. Only patients with a confirmed skin disease will be enrolled in this study. Apart from a pain-free visual inspection of the skin, no subsequent invasive procedures and tests and no medications are given to the patients. These patients are already seeing a dermatologist and some will be receiving medication. Therefore, anxiety and distress patients often feel when getting a new diagnosis and/or a treatment plan is very unlikely to occur.
The only discomfort and embarrassment that patients might feel arises from the fact that some have to get undressed for the examination.

2.) Physicians
Risks to GPs taking part in the study have not been identified.

1.) Test patients
All information about participation in this study will be kept confidential. The dermatologists will keep the signed informed consent forms. Each test patient will be given unique patient study number. The CRFs will contain personal data such as DOB, sex, and the unique patient study number. The patient questionnaire carries only the unique patient study number. Clearly identifiable information such as name, address, and insurance number will not be submitted to the study team and will stay with the dermatologists. Data analyses will be carried out only with anonymized data.

2.) Physicians
All information about participation in this study will be kept confidential. Each GP will receive a unique physician study number which will be documented on the CRFs. Patient questionnaires will not contain any physician identifiable data. Data collected on physician's individual performance will never be exposed. Results on diagnostic accuracy and consultation lengths will be published only in aggregate form and no individual be it patient or physician can ever be identified from research findings.
However, if a physician requests information on his individual performance, it will be disclosed to him.

Data capture
Four data capture instruments will be used: 1) Patient satisfaction questionnaire, 2) CRFs, 3) internet history of tablet computers, and 4) VisualDX usage logs. At the time of patient recruitment, dermatologists will document following information also using Google Forms (gold standard CRF): • Date of recruitment • Patient's DOB and sex • Occupation and highest education level • Chief complaint • ICD-10 code of confirmed diagnosis • Tests that were used for confirmation (e.g. histopathology) • Medications/treatment plans that the patient is receiving 3.) The Internet history of tablet computers used in both study arms will be analyzed and documented in Excel. This will reveal the type of support tools that were used by physicians.
4.) VisualDx usage logs will show which findings or symptoms and diagnoses were entered and viewed by GPs. These information will be transferred to Excel and analyzed.

Data management and analysis
Data in Google Forms (CRF, Patient satisfaction questionnaire) will be saved in CVS format and exported to Excel. Data will be stored in unchanged form (raw data set). Raw data will be reviewed to identify any discrepancies (incomplete data, illogical data, and illegible data) and plausibility checks will be conducted. After the database has been cleaned, it will be stored and locked. For statistical analyses the 'cleaned' database will be exported into SPSS 21.
Outcomes measures The unique patient study numbers will be used to link datasets, e.g. the gold standard CRF with VisualDx CRF and with the control CRF.
1.) Patient satisfaction scores; direct comparison of scores of both study arms 2.) Length of consultation: direct comparison of the median and average consultation length (minutes) of both study arms 3.) Diagnostic accuracy • VisualDx diagnoses compared to gold standard diagnoses: number of correct diagnoses • Control arm diagnoses compared to gold standard diagnoses: number of correct diagnoses

Publication and Dissemination of results
All efforts will be made to publish the results of this study in peer-reviewed scientific journals with consent of all research partners and participating organizations. Journals of interest would be family medicine journals, dermatology journals, and public health journals.
Dissemination of results to participants will take place via email and mail. In addition, participants will be made aware of the results if requested.