Introduction

The main purpose of an Objective Structured Clinical Examination (OSCE) is to assess medical students’ skills and attitudes prior to clinical practice [1–3], and it can also serve as an educational tool for healthcare professionals [4, 5]. In the field of radiological technology, OSCEs have been used for educational purposes in recent years [6–8]. The main clinical work of radiology technologists involves tasks related to radiography, and the skills and attitudes that must be acquired prior to clinical training include the capacity to treat patients and operate equipment and radiographic positioning. Accordingly, to confirm that students preparing for clinical training have acquired a certain level of competency regarding these skills and attitudes, they are assessed through the OSCE.

Before taking the OSCE, students must study the OSCE items. For example, in the field of radiography, students learn how to handle radiography equipment, how to set radiography conditions, and how to provide patient care as radiography skills to study radiography work. Faculty members give these technical instructions to students. Students learn through classroom lectures and practical instruction from the faculty. During the practical skills instruction, students receive a verbal explanation of the content from their supervisors and are supposed to reflect on their learning for improvement based on what they remember.

Smart glasses with a video camera that can record video data have become increasingly popular recently. In fact, researchers have reported on the use of these devices in education in the fields of medicine, nursing, and emergency medical services [9, 10]. The assumption behind their use is that by recording students’ activities during the OSCE, students can then use these recorded data as information for pre-OSCE and reflective learning. Hence, using these devices may enable innovative learning content through videos and engagement in reflective learning using objective information. This is a form of education that was not available until recently. However, to the best of our knowledge, there is currently no study on educational content related to smart glasses use in OSCE radiology education.

This study aimed to evaluate smart glasses-based educational content from three different views in OSCE practical skills content among radiological technology students. We also attempted to examine the educational effects of reflective learning with this new content. Our examination included three views: student, patient, and overall.

Materials and methods

Mock OSCE

In the general radiological technology field, radiographic techniques are tested as one of the OSCEs (e.g., chest radiography, body X-ray CT examination, head MRI examination). In this study, we selected chest radiography as the OSCE test item. Fig 1 shows the timing of the administration of the mock OSCEs, educational intervention, and self-reported comprehension survey. The OSCE, educational intervention, and self-report comprehension survey were conducted in the radiography laboratory of the Juntendo University Faculty of Health Sciences, Department of Radiological Technology Practice Building. Students and assessors met during the scheduled OSCE teaching time. Students were briefed on the OSCE by the assessor and then conducted the first mock OSCE. The mock patients were cared for by students who had not previously conducted an OSCE. After the educational intervention (conventional or smart glass education), a self-report comprehension survey was administered. The first OSCE, educational intervention, and comprehension survey were all conducted on the same day. The second mock OSCE was conducted four days after the educational intervention. Students viewed recordings of the first mock OSCE and examined them from three views. The recorded data obtained from filming were stored on a dedicated server and kept in the laboratory. Students viewed the recordings in the laboratory under the supervision of the researcher.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. The timing of the mock OSCEs before and after the educational intervention and self-reported comprehension survey.

https://doi.org/10.1371/journal.pone.0296417.g001

The evaluators were five faculty members who were licensed radiology technologists with at least five years of clinical experience. Each mock OSCE was conducted by two faculty evaluators.

The mock OSCE consisted of 18 evaluation items. These items are listed in Supplement 1. The evaluation was graded on a 4-point scale as follows: all items were completed (3 points); mostly completed (2 points); somewhat incomplete (1 point); and insufficient (0 points).

Supplement 1 18 evaluations in the mock OSCE

1. I. Clean and neat clothing (white coat, pants, shoes, etc.).
2. II. Hygienic and clean appearance (nails, hair, beard, nose hair, etc.).
3. III. Setting up the radiographic conditions (1) (radiographic distance).
4. IV. Setting up the radiographic conditions (2) (X-ray center and detector center).
5. V. Setting up the radiographic conditions (3) (tube voltage, tube current, and imaging time settings).
6. VI. Confirmation of patient name and imaging site.
7. VII. Explanation of changing clothes (obstruction shadow removal, confirming whether the person is wearing any metallic accessory, etc.).
8. VIII. Positioning (1) (PA chest radiography).
9. IX. Positioning (2) (remove the shoulder blade from the lung field).
10. X. Positioning (3) (position of detector and patient).
11. XI. Positioning (4) (X-ray tube position).
12. XII. Patient treatment (1) (easy-to-understand voice volume).
13. XIII. Patient treatment (2) (explanation of examination).
14. XIV. Patient treatment (3) (breathing cues).
15. XV. Patient treatment (4) (sincere response).
16. XVI. Patient treatment (5) (safe and smooth response).
17. XVII. A word of kindness after the end of the radiographic procedure.
18. XVIII. Implementation of hand hygiene practices compliant with the WHO guidelines.

Smart glasses

The smart glasses we used were the SMART GLASSES M400 (VUZIX). They were attached to the head of the mock patient and the student, allowing for capturing images from the view of the front of the head of each person (Fig 2).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Scene of the mock OSCE with the participants using smart glasses.

[Reprinted from Remote NAKAMA under a CC BY license, with permission from AVR Japan Co., Ltd., original copyright 2022].

https://doi.org/10.1371/journal.pone.0296417.g002

The compact camera head AG-UCK 20G (Panasonic) and memory card portable recorder AG-UMR 20 (Panasonic) were used as the video camera system for acquiring the video data. By using the online features of the Remote NAKAMA, one can simultaneously observe, on a single screen and in real-time, an image from the mock patient’s view, the student’s view, and the room camera (Fig 3). This screen can be shown to the student group during a mock OSCE. The Remote NAKAMA software requires a general-purpose computer to be run, and we used the notebook PC ASUS GAMING A15 (Model FA506IC). The recording software was OBS Studio version 27.2.4. Remote NAKAMA (AVR Japan Co., Ltd.) was used as the remote working eye system for the wearable device.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Educational content videos (“overall view,” “student view,” and “patient view”).

[Reprinted from Remote NAKAMA under a CC BY license, with permission from AVR Japan Co., Ltd., original copyright 2022].

https://doi.org/10.1371/journal.pone.0296417.g003

Results

Participants’ basic information is shown in Table 1. Upon comparing the control and intervention groups, we found no significant differences regarding sex, place of origin, school, or grade point average. Table 2 shows the scores for the first and second mock OSCE evaluations. A significant increase can be observed in the scores in the control group (0.36±0.1) compared to those of the intervention group (0.06±0.1) in evaluation item IV: “setting up the radiographic conditions (2) (X-ray center and detector center; p = 0.042).” No significant differences were found in the other items.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Participants’ characteristics.

https://doi.org/10.1371/journal.pone.0296417.t001

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Difference of scores for the mock OSCE evaluations before and after the educational intervention.

For assessment item IV: “setting radiographic conditions (2) X-ray center and detector center”, there was a significant increase in scores in the control group (0.36 ± 0.1) compared to the intervention group (0.06 ± 0.1) (p = 0.042). No significant differences were found for the other items.

https://doi.org/10.1371/journal.pone.0296417.t002

S1 Table shows the results of the self-reported comprehension survey. No significant differences were found for all items.

S1 Fig showed how students rated their preference for the video-based content from different views for the review learning process. The results show that the combination of the three views had the highest scores, while the student view alone had the lowest scores.

S2 Fig shows how students rated their preference for using smart glasses for specific educational items related to the radiographic process. The results show that “the entire radiographic process” had the highest scores for effectiveness, followed by “patient treatment” and “how you are seen by the patient.”

Discussion

In this study, we devised new video-based educational content from three views (student, mock patient, and overall views) aimed at effectively promoting reflective learning of OSCE practical skills content among radiological technology students.

Before we conducted this study, we expected that the use of video-based educational content with multiple views would provide objective methods for students to engage in reflective learning after the educational intervention, which would then lead to higher scores on the second mock OSCE and on the comprehension questionnaire. However, some items were rated significantly lower in the intervention group than in the control group. Therefore, the findings contradicted our expectations.

The lower scores for the second mock OSCE in the intervention group may be related to the fact that the students found it uncomfortable to perform the radiographic procedures with the smart glasses on their heads. In addition, students were not accustomed to using the smart glasses, which may have affected their ability to concentrate. Previous studies on the use of ophthalmic wearable devices in medical education have shown their usefulness, such as good image quality and user-friendliness in the field of plastic surgery [13]. In the current study, students in both groups had to operate the X-ray imaging device while managing how they treated mock patients, and the intervention group had to do so while wearing the device. As described above, this may have caused difficulty in movement, which may have, in turn, affected how they treated patients and operated equipment. Meanwhile, the control group may not have faced such hindrances. If this is indeed the case, stakeholders may need to develop a device that does not cause discomfort in students wearing it while performing radiographic procedures.

Regarding the results of the self-reported comprehension survey, the use of the ophthalmic wearable device enabled the students to check the status of the OSCE through video images. However, there were rare occasions when the students were wearing the device during the OSCE but could not access the video because the network was interrupted, and they thus had to deal with the disconnection. The extended time spent resolving these issues may have reduced their focus on the training and understanding of the training content. Moreover, the time that students in the control group spent on the educational intervention time followed the period established by the teachers, whereas the intervention group spent significantly more time on the process because they had to replace the device, check the images from the three cameras, prepare for recording, and troubleshoot any issues. In previous studies regarding the use of ophthalmic wearable devices in medical education, problems such as battery life [14–17], missing video frames [17], and network errors [18] were identified. Addressing these issues may require the devices to be easy to replace and wear.

Regarding examples of the use of wearable technology in medicine, Iqbal et al. [19] discovered that head-mounted displays were utilized in surgery (n = 7), imaging (n = 3), simulation and education (n = 2), and navigation tools (n = 1) [19]. In medical education, a case study demonstrated the use of a visual-based finger-tracking system during transurethral prostatectomy, where surgeons could identify areas of interest that were then live-streamed to trainees and medical students [20]. Another study showed a significant positive correlation between simulator and steering wheel performance in the rehabilitation of spinal cord injury patients. However, the limitations of the head-mounted display resulted in longer stopping times, and some participants experienced simulator sickness during the simulation [21].

Additionally, a report analyzing medical students’ satisfaction and teaching effectiveness found that real-time online clinical skills education using a wearable action camera yielded high levels of educational satisfaction. However, there were some negative comments regarding video quality. Nonetheless, the study indicated that online video education with wearable cameras could be a viable alternative to in-person teaching when classes are unavailable [22]. In previous studies, the instructor wore the wearable device while the learner viewed the video from the instructor’s view. In this study, the educational content could be seen from three views (student, mock patient, and overall), with the learner and mock patient wearing the wearable cameras. This allowed for effective learning through multiple video reflections, which was not possible previously. However, the use of multiple devices also led to complications like device malfunctions, which had been previously noted with wearable devices and did not improve learning satisfaction in some respects. While the use of wearable technology like smart glasses offers the potential for high educational satisfaction, it is essential to address problems encountered by users, such as device handling, stable network environments, and image quality maintenance).

The combination of the three views (mock patient, student, and overall views) showed the highest scores for effectiveness regarding the reflective learning process, while the student perspective alone showed the lowest scores. The OSCE evaluates clinical competence as a healthcare professional in a simulated clinical setting [23, 24]. Therefore, we hypothesized that in a retrospective OSCE to assess students’ clinical competence; it would be effective to visualize a combination of three views: students’ sincere response to a simulated patient (simulated patient perspective), instrument handling (student perspective), and the examination room environment (overall perspective). The student’s perspective would effectively confirm the accuracy of instrument handling in a retrospective study. Wearable cameras have the advantage of recording and visualizing the behavior of the person wearing the camera, as shown in the pilot’s 24-hour usage diary report [25]. We considered the use of wearable cameras to visualize students’ views to be effective for reflective learning in OSCEs. However, the student’s point of view in this study may be captured from a different direction than the student’s actual point of view, depending on the orientation of the head-mounted camera. Therefore, it is not necessarily the same as the student’s view. Therefore, some of the student perspective videos may not be effective for reflective learning.

This study focused on new video-based educational content and examined the learning outcomes from these videos. An essential aspect of this study was the importance of reflective learning from watching videos recorded with smart glasses. Regarding the specific educational items for which the use of the video-based educational content from different views was effective, students deemed them effective for better understanding “the entire radiographic process,” “patient treatment,” and “how you are seen by the patient.” These videos are most effectively obtained through a combination of the “patient perspective,” “student perspective,” and “overall perspective” that students identified as effective videos for reflective learning. From this, it appears that students find videos that allow them to see their movements, the overall flow of the shooting, and how it looks from the patient’s point of view to be effective for reflective learning.

The video information provided in this content allows students to observe their behavior objectively during the OSCE, which was previously impossible. It is also possible to obtain information students cite as effective teaching material for looking back and learning from the OSCE. Based on the above, we considered that reflective learning with this content could provide effective information in OSCE education.

However, the scores for some of the evaluation items and comprehension questionnaire items did not improve because of some specific problems. Specifically, as aforementioned, it is somewhat complicated to handle the ophthalmic wearable device used in this study because of its multifunctional nature; the device is rather large and may be uncomfortable to wear for some students. Moreover, the process of taking out the device from one student and the other wearing it is not smooth.

Regarding study limitations, one is the difference between the evaluators from one mock OSCE evaluation to another. One of several problems in this study was the lack of human resources. The mock OSCEs were evaluated by two faculty members. The mock OSCE was conducted once a week over eight weeks. This required the cooperation of a total of 16 faculty members. Although it would be optimal for the same faculty member to evaluate all mock OSCEs, it was difficult for the same two faculty members to evaluate all eight weeks because they also had other academic duties. Next, it is desirable to standardize the evaluation criteria for mock OSCEs with more detailed content. Currently, the evaluation criteria are generalized within the university but may differ from those in other schools. Therefore, generalization of results may not be guaranteed. In addition, since the OSCE survey covered 117 students and the number of test sites and evaluators was limited, it was not possible to conduct the survey on the same day for all eligible students. This may have affected the evaluator’s instructions. For the same reason, OSCE instruction could not be conducted on the same day and was spread over eight weeks; as with the OSCE evaluation, it would be desirable to use a more detailed and uniform instructional methodology to ensure uniformity. A limitation of the study design was the difficulty in designing the study as a cluster randomized controlled study.

Finally, in the study, the intervention group should have received training on how to handle smart glasses. The reason for not providing training sessions was time constraints, as there was a need to conduct an OSCE for a total of 117 participants (45 in the intervention group and 72 in the control group) within a limited period.

Conclusions

In the mock OSCE evaluation after the preliminary instruction, unexpectedly, there was a significant increase in the number of points in the control group compared with the intervention group in the setting up of the radiographic conditions regarding the x-ray center and detector center. The intervention group’s lower score on the second simulated OSCE may have been due to the students’ discomfort with wearing the smart glasses to perform the radiography procedure and their lack of familiarity with the smart glasses, which may have affected their ability to concentrate. It was suggested that these problems should be solved by making the device easy to handle and use (e.g., no network interruption).

Supporting information

Acknowledgments

The authors are grateful to Tamaki Nara of the Juntendo University Medical Technology Innovation Center for her support in terms of industry-academia collaboration. We also thank the Juntendo University students who supported this study. Finally, we would like to thank Editage (www.editage.com) for English language editing.