Fig 1.
Pie charts displaying the proportions of competence-based and behavior-based items in the evaluation grids for OSCE stations #1, #2 and #3 (A, B and C, respectively). Detailed evaluation grids are provided as S1–S3 Data.
Fig 2.
Distribution of mean OSCE grades (red) and mean fourth-year multiple-choice question (MCQ)-based grades (black). (A) Density plot showing the wider dispersion of OSCE grades compared to MCQ grades. (B) Relationship between student rank among the 379-student class, and grades obtained at OSCE and MCQ-based examinations, showing a flatter slope for OSCE and a steeper slope for MCQs, confirming the wider dispersion of OSCE grades than MCQ grades among the fourth-year class.
Table 1.
Descriptive statistics for mean multiple-choice-based examination grades and OSCE grades of the fourth-year class of medical school.
Fig 3.
Scatterplots of the relationships between OSCE grades and mean fourth-year teaching unit grades.
(A-C) Mean OSCE grades versus mean fourth-year multiple-choice question (MCQ)-based examination grades, traineeship skill and traineeship behavior grades. (D-F) Mean OSCE #1 grades versus mean fourth-year MCQ-based examination, traineeship skill and traineeship behavior grades. (G-I) Mean OSCE #2 grades versus mean fourth-year MCQ-based examination, traineeship skill and traineeship behavior grades. (J-L) Mean OSCE #3 grades versus mean fourth-year MCQ-based examination, traineeship skill and traineeship behavior grades. To highlight trends, a smoothing regression line was added to each plot using the geom_smooth function (R Software, ggplot2 package). P values and Spearman r coefficient were highlighted in green for significant and in red for non-significant correlations, respectively.
Table 2.
Correlation between OSCE grades and mean fourth-year multiple-choice-question-based grades.
Fig 4.
Proportion of students ranked in the top quarter based on fourth-year teaching unit grades who were ranked within the top quarter of OSCE grades (average of OSCE #1–3).
(A) Multiple-choice question (MCQ)-based examination grades (fourth-year average). (B) Traineeship skill grades (fourth-year average). (C) Traineeship behavior grades. There was a significant difference between the three proportions (P<0.001, Chi-square test).
Table 3.
Correlation between averaged knowledge-oriented and behavior-oriented items composing OSCE grades and mean fourth-year grades.
Fig 5.
Dot plot of the relationship between multiple-choice question (MCQ)-based grades obtained for teaching units and the ratio of the OSCE grades and those MCQ-based grades.
This plot highlights graphically that MCQs and OSCE evaluates students differently, since a non-neglectable proportion of students obtained better grades at OSCE than at MCQs, and more so among students with middle- or low-range grades at MCQs. To facilitate the reading, the dotted red line indicates the OSCE/MCQ ratio equal to 1.0. Students with an OSCE/MCQ ratio lower than 1.0 have a lower grade on the OSCE than the MCQ-based exam.
Fig 6.
Variation in ranking based on the mean fourth-year multiple-choice question (MCQ)-based grades, with incremental percentages of OSCE grade integrated into the final grade.
The upper and lower solid black lines represent thresholds for +50 or -50 rank variation, respectively. Results are displayed for integration of OSCE grade with a 10%, 20% and 40% coefficient.
Table 4.
Previous studies from the literature investigating correlations between OSCE and other academic assessment methods.