scHilda: Hierarchical Integration of LLM with KG database for single cell type annotation
Fig 2
scHilda Performance Evaluation and Ablation Studies.
(A) Comparison of annotation agreement scores between scHilda and various existing methods across eight benchmark datasets, showcasing its SOTA performance. (B) Ablation study of different relationship searching types in the knowledge graph, verifying the significant contributions of pathway (PARTICIPATES_IN) and co-expression (COEXPRESSED_IN) relationships to annotation accuracy and the negative impact of the Marker relationship on the model. (C) Performance on LLMs of different scales, indicating that the scHilda framework effectively ensures a performance baseline, allowing lightweight models to achieve results close to top-tier models. (D) Comparison of model performance with and without explainability output (reasoning and evidence), showing that disabling this feature can reduce costs with almost no impact on accuracy. (E) The major type distribution in Top-3 candidates, showing the necessity of the major type determination. (F) The impact of different prompt strategies (LLM-biased, KG-biased, neutral) on performance, demonstrating that both LLM-biased and KG-biased approaches show a significant drop in performance compared to the neutral one.