Differential performance of large language models in advanced cardiac life support assessment: A comprehensive multi-dimensional analysis of accuracy, consistency, and visual recognition capabilities
Fig 2
Performance percentages of large language models by question type.
Bar charts displaying the accuracy rates of four large language models—ChatGPT-4o, Gemini 2.0, Claude 3.5, and DeepSeek R1—across three question categories: visual (n = 12), knowledge-based (n = 29), and case-based (n = 9). Each chart presents three predefined accuracy metrics: overall accuracy (all responses correct), strict accuracy (at least two correct responses), and ideal accuracy (at least one correct response). ChatGPT-4o achieved perfect performance (100%) across all question types and metrics. Claude 3.5 showed excellent accuracy in knowledge and visual items but relatively lower performance in case-based questions. Gemini 2.0 performed comparably in knowledge-based questions, but underperformed in visual and case-based categories. DeepSeek R1 achieved 100% accuracy in knowledge-based questions but failed to answer any visual question correctly and showed moderate performance in case-based items. These findings highlight model-specific variability in reasoning, domain knowledge, and visual recognition capabilities.