The problem

Referral letters and clinical documentation are central to directing patients into acute healthcare. Across health systems, patients are referred from primary care, emergency settings, community services, or other clinical teams into acute, elective, urgent, or tertiary-level services. These referrals rely on clinical documentation, including referral letters, clinical notes, clinical summaries, or electronic referral forms, which all vary considerably in quality, structure, and completeness [1–3]. The triage of these referrals determines urgency, waiting times, and access to care, and is a major contributor to service delays across acute and specialist pathways [4–6].

The triage and prioritisation of referrals is frequently slow, inconsistent, and resource intensive [7]. In many health systems, this process is performed manually by clinicians or experienced nursing staff, who must integrate heterogeneous and often incomplete information to assign clinical priority [8–10]. This can be challenging, time-consuming, and cause frustration to the individuals triaging the referrals, and subsequent delays to the patient for whom the referral was sent. Under and over-triage of referral letters has been reported, and the subjectivity of the triage disposition often depends on the quality of the information in the letter, and the experience of the individual triaging [10].

As populations increase, the demand on health systems increases concurrently, and the volume of referrals that must be triaged becomes increasingly difficult with these challenges [7]. Waiting lists in healthcare are a focus of intent for health governing bodies internationally, and current triage systems to reduce these lists have no adopted universal framework, nor are they built to manage a rapidly rising referral burden. The triage and prioritisation of referral documentation across acute and specialist care represents an analogous challenge: a process that is slow, inconsistent, and reliant on manual clinical review of heterogenous documentation.

AI as a potential solution

Artificial intelligence (AI), in particular machine learning (ML), deep learning (DL), and natural language processing (NLP) have significant and rapidly expanding potential to support clinical decision-making. NLP systems can analyse free-text clinical documentation to extract clinically relevant features and classify patient urgency, if the system has been trained to a high standard [11,12]. ML models have demonstrated the ability to predict triage acuity, disposition, and clinical deterioration from structured patient data and early clinical observations in emergency settings while large language models (LLMs) can generate clinical reasoning based on the acquisition of extensive clinical knowledge, which represents a further development for AI-assisted triage from unstructured referral text [12–14]. Collectively, these approaches could offer a scalable, consistent, efficient and potentially safer adjunct or alternative to purely manual referral triage, with the capacity to reduce delays and improve prioritisation accuracy at scale.

Evidence gap

Despite this promise, the performance, safety, and clinical implementation characteristics of AI models for the triage of referral documentation have not been systematically synthesised. A search of PROSPERO conducted in February 2026 identified no completed or ongoing systematic reviews specifically evaluating AI models for interpretation and triage of clinical referral letters across acute and specialist care pathways. Existing reviews have addressed adjacent topics including AI for triage using physiological data and structured EHR inputs, AI for diagnostic support in specific clinical domains, and AI for the interpretation of diagnostic imaging and clinical notes in specific specialties but none has evaluated the specific intersection of free-text or semi-structured referral documentation and AI-assisted triage across the range of acute and specialist care pathways [15,16]. This gap limits the ability of clinicians, health service planners, and researchers to make evidence-informed decisions about the adoption of AI for referral triage.

Objectives

Eligibility criteria

All studies that examine the use of an AI tool for triage of referral documentation will be considered against the inclusion and exclusion criteria outlined in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Inclusion criteria.

https://doi.org/10.1371/journal.pone.0355476.t001

Information sources and search strategy

An electronic search strategy will be performed using MEDLINE, EMBASE, Web of Science, Scopus, CINAHL, and the Cochrane Central Register of Controlled Trials. The search strategy will be broad, and focus on four conceptual blocks, using Artificial Intelligence, Clinical Referral or Documentation Triage/Prioritisation and Acute/Specialist Care as key MeSH terms, exploded where available. There will be no language restrictions, papers not in English will be included and translated where necessary with translation processes documented in the review. To enhance completeness, we will perform backward and forward citation chaining of all included studies and relevant reviews. To capture emerging evidence in this rapidly developing field, we will additionally search Europe PMC for relevant preprints; these will be screened against the same eligibility criteria, flagged as non-peer-reviewed, and reported separately, and will not be pooled with peer-reviewed studies in any quantitative synthesis. The review will include studies published from January 2016–5 May 2026 (the date of the search execution), reflecting the emergence of modern LLM and deep learning methods in clinical applications and is consistent with the ten-year limit specified in the PROSPERO registration. An example search strategy is shown in Supplementary Material, Table 1 (S1 File).

Study records

Data extraction and data items

Two authors will extract data independently using a standardised extraction form developed within Covidence or Microsoft Excel. The form will be piloted on a random subset of three to five studies before formal extraction commences and refined as necessary. Discrepancies will be resolved through discussion, with a third reviewer consulted where consensus cannot be reached. All extraction decisions will be documented.

Missing data will be documented and reported as ‘not reported’ where applicable. Authors will be contacted for clarification where key data is missing. Analyses will be conducted using available data, and the potential impact of missing data on the findings will be considered in the interpretation of results.

The data items to be extracted, where possible, are presented in Table 2.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Summary of components for the data extraction tool.

https://doi.org/10.1371/journal.pone.0355476.t002

Risk of bias in individual studies

The risk of bias will be assessed independently by two reviewers. The following validated tools will be applied according to study design.

• AI Prediction Model Development and/or Validation: PROBAST-AI where applicable [21,22]
• Randomised Controlled Trials: Cochrane Risk of Bias 2 (ROB 2) [23]
• Non-randomised comparative studies: ROBINS-I (Risk of Bias in Non-Randomised Studies of Interventions) [24]
• Diagnostic accuracy studies evaluating an AI model against a clinical reference standard without explicit model development: QUADAS-2 ([25]

Where a study combines elements of multiple designs (for example, model development with concurrent prospective validation), the most appropriate tool will be selected, and the rationale documented. Disagreements in risk of bias judgements will be resolved by discussion; third-reviewer adjudication will be sought where consensus cannot be reached.

Data synthesis

Given the anticipated clinical, methodological, and statistical heterogeneity of included studies, results will be synthesised primarily using a structured narrative approach as the primary method, supplemented by quantitative synthesis where feasible. Metrics such as accuracy, sensitivity, specificity, and AUC may not be directly comparable across studies even when reported in a similar form, because their meaning depends on the reference standard, and the clinical consequence of over- and under-triage in each pathway. Accordingly, terms such as “performance” and “agreement” will be interpreted in relation to each study’s own reference standard and pathway, not as generalisable transferable constructs.

The review therefore has two complimentary synthesis aims. The first is a structured evidence map of the field, describing the types of AI models utilised, the types of referral documentation processed, the clinical pathway, and the reference standards assessed. The second is a synthesis of prioritisation performance, agreement with reference standards, and safety and implementation outcomes, if meaningful comparison can be achieved.

For both aims, a narrative synthesis will be conducted using the SWiM (Synthesis Without Meta-Analysis) reporting guideline, with structured summaries of studies tabulated to enable comparison of the primary and secondary objectives [26]. The synthesis will be stratified by (i) AI model class (traditional machine-learning classifiers, NLP pipelines, deep learning models, and LLM-based systems), reflecting their differing validation requirements and failure modes, and (ii) clinical pathway (emergency/acute versus outpatient/elective), reflecting differences in the clinical meaning of under- and over-triage. Stratification by model class will extend beyond performance to how each class has been evaluated, including validation designs, reference standards, and reported failure modes. Meta-analysis will be treated as a conditional and exploratory output and will be performed only where a minimum of three studies report comparable outcomes using a similar AI model class, clinical setting, reference standard, outcome definition, and threshold structure. Quantitative analyses will be conducted using StataNow 19.5 SE [27]. Where diagnostic accuracy meta-analysis is appropriate, bivariate random-effects or hierarchical summary receiver operating characteristic (HSROC) models will be used, depending on the availability of sensitivity and specificity estimates and the degree of threshold variation across studies.

Where meta-analysis is not feasible, results will be described narratively with reference to the direction and magnitude of effects across studies.

Reporting bias

Reporting bias will be assessed by considering potential publication bias and selective outcome reporting. Where sufficient studies are available (typically ≥10 and sufficiently homogeneous), funnel plots may be generated to explore potential small-study effects. Statistical tests for funnel plot asymmetry (e.g. Egger’s test) will be considered where appropriate. In addition, where study protocols or trial registry entries are available, reported outcomes will be compared against prespecified outcomes to assess selective outcome reporting. However, it is anticipated that there will be too few studies to explore funnel plots, and availability of study protocols may be limited in AI-based studies.

Certainty of evidence

GRADE will not be applied as the review focuses on AI model development and performance rather than intervention effects on patient outcomes. Instead, methodological quality and risk of bias will be assessed as outlined above.

Study status and timeline

This review has been prospectively registered with PROSPERO (CRD420251244654; https://www.crd.york.ac.uk/PROSPERO/view/CRD420251244654). Database searches were completed on 5 May 2026, and title/abstract screening is ongoing at the time of submission. No full-text screening, data extraction, risk-of-bias assessment, or synthesis has yet been undertaken. We anticipate the following timeline: (A) title/abstract and full-text screening will be completed by August 2026; (B) data extraction and risk-of-bias assessment will be completed by November 2026; and (C) data synthesis and final results are expected by February 2027. Any substantive amendments to the protocol after registration will be recorded in the PROSPERO record and reported transparently in the final review.

Protocol amendments

Any amendments to this protocol after the date of PROSPERO registration and prior to the completion of the review will be documented in a dated protocol-amendment log, with each amendment recorded as follows: (1) date of amendment; (2) section of protocol affected; (3) nature of the amendment; (4) rationale for the amendment; and (5) anticipated impact of the amendment on the review findings.

Substantive amendments affecting the review question, eligibility criteria, information sources, search strategy, outcomes, risk-of-bias assessment, or synthesis approach will also be reflected in an updated PROSPERO registration, with the version history maintained transparently in the registry. Minor amendments, such as corrections to typographical errors or clarification of pre-specified procedures that do not alter the review’s methodology, will be recorded in the internal amendment log only.

All deviations from the protocol encountered during the conduct of the review will be reported in the final review manuscript, in a dedicated subsection of the Methods, in accordance with PRISMA 2020 [20].