Introduction

The data generated from the laboratories are becoming decision engines [1]. So medical laboratories play a critical role in patient diagnosis and treatment. The defining feature of the Industry 4. 0 era is the immediate flow of information and the reactions to the changes without delay [2]. The importance of timely accurate test results from medical laboratories will continue to grow in future healthcare. This new paradigm requires medical laboratories meet higher quality standards to keep pace with the times. The QMS in a medical laboratory is complex and extensive. It encompasses personnel, instruments, reagents, methodologies, and environmental conditions horizontally. It also spans the entire testing process (pre-analytical, analytical, and post-analytical phases) vertically [3–5]. ISO 15189 is a QMS specifically designed for medical laboratories, regularly updated according to reflect advances in medicine and management science [6–9]. ISO 15189 QMS promotes the guidelines for periodical, systematic monitoring of processes and identifying areas which might need improvement [10]. The ISO 15189 accreditation project has been widely adopted in European countries and sub-Saharan Africa [11,12]. The latest version is ISO 15189:2022 emphasizing risk assessment implementation [3]. In the era of evidence-based medical practice, risk management is an essential quality improvement activity that must be integrated in all testing processes [13].

The initial phase of implementing a QMS is often unstable, characterized by vulnerabilities and challenges. This period is also critical in determining whether the system will be successfully established. As Laila O. AbdelWareth et al. ‘s research indicated team dynamics and work culture are still evolving during this phase due to the new environment [14]. Accurately identifying high-risk points and implementing effective preventive measures can help laboratories navigate the transition smoothly. Internal and external assessments are two distinct assessment methods. Both are conducted based on ISO 15189 standards and national regulations, but they differ in personnel, cost, and frequency. Internal assessments are performed by laboratory staff [4], with fixed labor costs and are conducted annually. External assessments, however, are carried out by assessors trained by CNAS with higher costs due to assessment fees [15]. In the first three years, external assessments are conducted annually, transitioning to biennial assessments after the third year in China.

To understand the impact of different assessment methods on medical laboratories during the initial stages of QMS implementation in China, this study compared the characteristics of NCs, recurrence rates, and corrective actions between internal assessments and external assessments. By identifying high-risk points and weaknesses, the study provides insights to help medical laboratories achieve a smoother transition.

Methods

Results

Distribution of NCs, root causes, major corrective actions in internal and external assessments

S1Table showed the distribution of NCs according to ISO 15189 requirements in internal and external assessments. The top three NC categories were Examination processes(n = 8, 22. 86%), Personnel management (n = 7, 20. 00%), Document and record control (n = 6, 17. 14%) in internal assessments, and Examination processes (n = 19, 28. 36%), Document and record control (n = 10, 14. 93%), and Others (n = 8, 11. 94%), Personnel management (n = 6, 8. 96%)in external. S2 Table showed the top root causes were Training Deficiencies (n = 14, 40. 00%), Training deficiencies (n = 11, 31. 43%) in internal assessments, and Training Deficiencies(n = 27, 40. 30%), Document and Record Deficiencies(n = 15, 22. 39%), Personnel Negligence(n = 14, 20. 90%) in external assessments. S3 Table showed the top implemented corrective actions were Personnel training(internal [n = 35, 47. 29%]and external [n = 67, 41. 36%]), the second in internal assessments were Supplementary records(n = 28, 37. 84%) and Document modification(n = 23, 14. 20%) in external assessment Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. NCs Distribution Trend Chart in Internal and External Assessments (A, B); Root Causes for NCs Distribution Trend Chart in Internal and External Assessments (C, D); Corrective Actions for NCs Distribution Chart in Internal and External Assessments (E, F).

https://doi.org/10.1371/journal.pone.0342129.g001

Analysis of the NCs characteristics in internal and external assessments

Table 1 showed the comparison and statistical analysis of the NCs in internal and external assessments. The NO. of Managment NCs, Technical NCs, total NCs and Corrective Actions in external assessments were significantly higher than internal assessment(p < 0. 05). No significant differences were observed in New occurrences NCs, Repeats NCs, Recurrence rate per year and Corrective action effectiveness Rate per year in internal and external assessments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparison and Statistical analysis of the NCs characteristics in internal and external assessments.

https://doi.org/10.1371/journal.pone.0342129.t001

Analysis of major corrective actions for NCs in internal and external assessments

Table 2 showed the distribution and the statistical analysis of corrective actions in internal and external assessments. The effectiveness Rate in personnel training was higher in external assessments than internal assessments, while the effectiveness Rates in Document modification and Supervision/ inspection were lower in external assessments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Comparison and statistical analysis of major corrective actions for NCs in internal and external assessments.

https://doi.org/10.1371/journal.pone.0342129.t002

The flow relationships between NCs and root causes, as well as root causes and corrective actions in Sankey diagrams

In Fig 2 From left to right in the Sankey diagram, the columns represent: NCs contents, categories of root causes, and frequency of occurrence. In the internal assessment, the flow from “Personnel” NCs to “Personnel Negligence” appeared most frequently (5 times). Over half of the flows between NC types and different root causes occurred only once. In the external assessment, the flow from “Examination Processes” NCs to “Training Deficiencies” was the most frequent (9 times). Similarly, over half of the flows occurred only once. Compared with the diagram from the internal assessments, more types of NCs are involved and more diverse and complex flows to root causes are exhibited in the external assessments. Fig 3 From left to right, the columns represent categories of root causes, types of corrective actions, and frequency of occurrence. In the internal assessments, the flow from Personnel Negligence to the Personnel Training was the most frequent (14 times). In the external assessments, the flow from “Training Deficiencies” to “Personnel Training” occurred most frequently (27 times). Compared with the internal assessments diagram, the external assessments involved a greater variety of corrective actions, with identical root causes flowing to a wider range of different corrective actions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The Sankey diagram tracing from NCs Contents to Root Causes in Internal assessments (A); The Sankey diagram tracing from NCs to Root Causes in External assessments (B).

https://doi.org/10.1371/journal.pone.0342129.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. The Sankey diagram tracing from Root Causes to Corrective Actions in Internal assessments (A); The Sankey diagram tracing from Root Causes to Corrective Actions in External assessments (B).

https://doi.org/10.1371/journal.pone.0342129.g003

Analysis and the characteristics of recurrent NCs

Table 3 showed No. of Corrective Actions in the recurrent NCs from external assessments was significantly higher than that from the internal assessments. No significant differences were observed in Recurrence Rate and the Recurrence interval time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Statistical analysis and the characteristics of recurrent NCs.

https://doi.org/10.1371/journal.pone.0342129.t003

The ‘Double Helix’ model of quality monitoring

The results revealed that in the initial phase of QMS establishment, external assessments detected a higher number and a broader distribution of NCs compared to internal assessments, exerting a wider chain and stronger supervisory influence on the QMS. However, after three years of operation, internal assessments began to identify NC types that were not captured by external assessments. The “chain” of internal assessment widened, and its supervisory influence strengthened. Together, internal and external assessments jointly ensure that the quality architecture climbing in a continuously dual-helix pattern. Both assessments involve root causes and corrective actions, which determine whether NC will recur. The two chains are linked by root causes and corrective actions, then a stable double-helix could be presented (Fig 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. The ‘Double Helix’ model of quality monitoring in Internal and External assessments.

https://doi.org/10.1371/journal.pone.0342129.g004

Discussion

Each medical laboratory should strive to establish and maintain a QMS because laboratory results play a vital role in disease diagnosis and treatment protocols [17]. As a key intervention to strengthen laboratory services, QMS implementation requires proper planning to avoid various difficulties during the process [18]. Therefore, the timely and precise identification of systemic high-risk points is essential to mitigate implementation challenges.

The results showed that during the initial establishment of the QSM, the high-risk points were in the examination processes, document/record control and personnel management. In the research on ISO 15189-accredited laboratories in Hong Kong, the results were largely aligned with the only discrepancy observed in personnel management. This discrepancy may be related to factors such as the duration of quality system operation and the number of laboratory staff. Among the requirements for examination processes, the area with the highest number of nonconformities (NCs) was internal quality control (IQC). The aim of IQC is to monitor the examination process to prevent the generation of erroneous information [19]. Therefore, assessors place particular emphasis on IQC performance. Current challenges in IQC implementation in China include the following [20]:(I)IQC procedures based on different quality control materials; (II)Selecting appropriate quality control materials;(III)Determining method performance (scientifically selecting TEa, CV, and Bias); (IV) Establishing statistical quality control strategies based on method performance and patient risk; (V)Strategies for addressing out-of-control situations; (VI)IQC for qualitative testing items. The NCs identified in our laboratory correspond to all six of these challenge areas, so addressing these risk points is key to improving our IQC quality. For a laboratory in the initial phase of establishing its QMS, there are significant inherent challenges in developing a document and record control system from scratch. Furthermore, our testing processes must not only comply with ISO 15189 requirements but also align with Chinese national standards, regulations, and the practical operational needs of clinical work. Personnel management procedures contains training, competency assessment, and authorization. Personnel management procedures contain training, competency assessment, and authorization. It is a systematic and flexible evaluation system that does not rely solely on training. The supervisor should focus not only on satisfactory acquisition of the knowledge, skills, but also evaluating the capability, professionalism and potential of the personel [21]. Internal assessments generated fewer NCs in the areas of Evaluation, Complaints, Information system and Risk Management than external assessments. This may be due to the flexible nature of evaluations and complaints, which requires assessors to adeptly integrate quality requirements with actual laboratory operations. While risk management has been given greater emphasis in ISO 15189:2022, internal assessors are mostly engaged in technical management work and may not have fully understood the updated requirements in the short term. Notably, NCs related to examination processes showed a rebound, further indicating instability during the initial QMS implementation phase. To ensure the stable quality level, it would be beneficial to implement quarterly or semi-annual monitoring and review of the corrective action outcomes. The number of NCs identified in external assessments showed high variability without a sustained downward trend, whereas internal assessments exhibited a marked initial decline followed by a slight rebound. In external assessments, no increasing trend was observed in the second year of operation (with five categories even showing a decrease). However, in the third year, two categories displayed an increase, indicating instability in the QSM. A post-accreditation “risk period” appears to exist, during which vigilance in quality activities and internal oversight may diminish. Consequently, laboratories that have implemented ISO 15189 for 1–2 years require enhanced monitoring following consecutive successful assessments, particularly within examination processes. The detection efficiency of internal assessments was initially lower, as many NCs identified externally were missed internally. Nevertheless, in the third year, some NCs were detected internally but not externally, suggesting that internal assessments initially served a supplementary function, with their complementary value becoming more prominent after three years. Therefore, although influenced by factors such as personnel competence and objectivity, internal assessments remain critically important during the first 3–4 years of ISO 15189 implementation and may yield progressively greater positive impact as the QMS matures.

Training deficiencies were the most common root cause in both internal and external assessments. Aligning with these findings, training was the predominant corrective action. For the medical laboratory personnel, training encompasses situational awareness of management system standards [22]. The dissemination of QMS requirements, implementation of technical requirements, and formulation of corrective actions all depend on robust training. We propose a systematic training optimization framework. This framework contains four actionable pillars: content comprehensiveness, trainer competency, format effectiveness, and assessment validity. The training needs to be repeated, with increased frequency for high-risk areas. Documentation and Record deficiencies were the second top root cause in external assessments, but fifth in internal assessments. Document modification (23 times) ranked second among corrective actions and was higher in external assessments. This difference reflects the varying capabilities of the two assessment types in identifying documentation issues. Since internal assessors are the main authors and approvers of documents, these documents naturally become blind spots in internal assessment. To effectively mitigate Document and Record Deficiencies, a comprehensive operational plan comprising four core measures is proposed. First, strictly enforce an “author recusal” and “cross-review” system. Second, conduct periodic systematic document and record review campaigns. Third, regularly engage external experts to perform document and record evaluations. Finally, establish a fully traceable lifecycle management system covering document creation, approval, issuance, review, and obsolescence.

The NO. of Managment NCs, Technical NCs and total NCs identified in external assessments were significantly higher than in internal. The difference in competency between internal and external assessors may be the primary reason: external assessors have extensive experience in laboratory quality management and have undergone multiple CNAS training sessions and assessments, giving them comprehensive mastery of QSM; whereas our department’s internal assessors received training from various institutions, typically lasting only 2–4 days, with only one assessment that was less rigorous than that for external assessors. Medical laboratories need to ensure that the internal assessment checklists could cover relevant compliance requirements in all relevant clauses [21]. The NO. of Corrective Actions in external assessments were significantly higher than internal. The high volume of corrective actions indicates both the complexity of the NCs and depth of the external assessments findings. It is suggested that external assessment are more effective at uncovering NCs and improve laboratory quality by revealing areas for enhancement in the QMS operation. These findings support the recommendation that laboratories in the initial stages of establishing QSM should regularly conduct external assessments to monitor their quality systems. Ian G. Wilson’ reserch showed similar proportions of NCs between internal and external assessments in a public health laboratory in United Kingdom [23]. With the prolonged establishment of the quality system, the gap in influence between internal and external assessments on lab quality will be narrowed. The number of corrective actions was significantly higher in external assessments. The greater number of NCs raised from external assessments is the main reason. Additionally, laboratories tend to implement multiple corrective actions to ensure their effectiveness and maintain accreditation status. No significant difference was observed in the corrective action effectiveness rate between the two assessment types, though this result may have been influenced by the small sample size. The data indicated a higher personnel training effectiveness rate in external assessments. Training conducted by the external assessors serves as an effective approach. During external assessments, assessors clarify the requirements of relevant clauses and highlight high-risk implementation points based on the specific context of the laboratory. This process helps laboratory staff gain an in-depth understanding of these NCs. Subsequently, within a short period (approximately one week) after the external assessment, relevant personnel conduct training for laboratory staff based on the learnings from the external assessors.

The analysis identified the most significant cause of NC recurrence was insufficient scope of corrective action implementation. (I) Corrective measures were primarily applied only within the group where the NC was initially identified (e.g., Reagents and Consumables). Due to sampling limitations or variations in assessor focus, NCs were not detected in other groups. (II) Although some corrective actions included a review of similar issues in other groups, the verification methods were often superficial and review was biased. There may have been a tendency to select well-performing groups or assays to expedite NC closure (internal quality control failures in Examination Processes). (III) Inadequate verification of training effectiveness: Recurrence rates were highest in Personnel Negligence and Training Deficiencies. The current training model: the group leader designs and delivers the training, followed by an immediate written or oral test. Trainer competence, training methodology, and the timing/format of assessment all critically influence outcomes. We recommend that training be delivered face-to-face, repeatedly. The trainer has strong system knowledge and communication skills. The assessments should be conducted separately, not on the same day as training. (IV) Cross-departmental coordination gaps: some corrective actions were intersects with other hospital departments. They were implemented without considering the workflows of these departments(e. g. , equipment management). The followings are recommended during the implementation of corrective actions. Treat the quality element linked to any NC as a high-risk item for all professional groups. A complete (non-sampling) review of compliance with the relevant clause should be conducted across all groups. The reviewers are leadership or personnel independent of any group’s interests, who possess a thorough understanding of quality-system requirements. Checklist templates should be designed to correspond directly to ISO 15189 requirements, allowing for sequential verification of each clause. Ensure the laboratory procedures explicitly consider the impact of related departmental policies.

The Sankey diagram revealed that the NCs identified in external assessments involve more requirement clauses and more diverse corrective actions compared to internal assessments. This discrepancy may stem from the fact that the medical laboratory was in the initial phase, where both staff competency and overall quality capabilities were still developing. Consequently, the effectiveness of internal assessments was weaker than that with well-established quality systems.

At the initial stage, the core function of external assessments is to identify quality deviations to ensure the overall compliance of the system. It effectively guides the quality framework in a continuous upward spiral. In contrast, the function of internal assessments focuses on horizontally extending supervision to achieve comprehensive risk prevention and control. It facilitates the thorough correction of NCs. This dual-helix model provides more stable safeguarding of the quality system than any single monitoring mechanism. Root causes and corrective actions drive substantive improvements in the quality system. They determine whether the “dual-helix” model remains stable and resistant to collapse.

To align with the sustainable development of healthcare systems, future updates to ISO 15189 will likely incorporate sustainability requirements [24]. Medical laboratories must continuously improve quality standards to adapt to the sustainability and digital era. The studies in Kenya and India have found that the implementation of the ISO 15189 lead to improvements in the pre-analytical, analytical, and post-analytical phases of testing, particularly in terms of sample management and External Quality Assessment (EQA) performance [25–27]. For developing or less economically developed regions, the ISO 15189 QSM serves as an effective tool for enhancing laboratory quality. We recommend adopting a “dual-helix” model to ensure the stability and accelerated advancement of the quality system.

This study uses data from a single center, the generalizability is constrained by factors specific to our laboratory setting. Such factors include: (I) laboratory scale and resource allocation, (II) service population characteristics, (III) personnel competency, (IV) QMS maturity level, and (V) the level of automation and informatization in testing processes. Although this study employed the internationally recognized ISO 15189 standard as an evaluation framework, variations in the interpretation and implementation of this standard may affect the generalizability across different countries, regions, and institutions. Additionally, this study may not have fully captured all NCs within the quality system due to the sampling methods used or variations in auditor competency. The depth of auditors’ (particularly internal auditors’) understanding of ISO 15189 clauses and their tolerance thresholds for deviations in the quality system may influence the data. Certain subgroup analyses may have been underpowered due to low event counts. This could have prevented the detection of statistically significant differences. In summary, when other laboratories applying the indicator framework and analytical methods developed in this study, they should conduct a systematic evaluation and make necessary adaptations based on their specific conditions.

Future research could further explore the following directions: extending the observation period to capture long-term dynamics in quality system evolution; quantifying changes in fundamental laboratory conditions (e. g. , personnel structure, equipment investment, informatization level) and incorporating them as covariates in the analysis; and employing multivariate models to assess the impacts of these conditional changes on the quality system. Collect quality data from laboratories with different characteristics and from different geographic regions. Identify meaningful laboratory quality indicators. Leverage artificial intelligence to screen sensitive quality indicators, facilitating precise identification of deviations within the quality system. Establish an automated AI-powered medical laboratory quality assessment system which can perform impartial, objective, and comprehensive evaluation.

Supporting information