Introduction

Clinical record (CR) is the primary tool used by health professionals for recording patient clinical information [1] It supports delivery of care, clinical decision making and continuity of care [2] primary functions [3], and it enables healthcare organizations to operate in accordance with the international quality standards [3–6] and also with national legal requirements [7, 8] (secondary functions [3]. A complete and accurate CR can help achieve safer practices [2], such as better communication and medication reconciliation, which are common reasons for medical errors worldwide [9–11]. Several authors reported that poor quality of the information in CR may be a predictor of poor quality of care [3, 12–16], and may be associated with higher rates of adverse patient safety events [2, 15]. This is true particularly considering adverse drug events, that may be caused by prescribing error in prescription writing, that involved illegibility, ambiguous abbreviations or lack of important piece of information such as date of prescription, dose or route of administration [17]. These aspects must be analysed through CRs review for quality assurance [18]

To determine whether healthcare is safe, effective, efficient, accessible, patient-centred and equitable, as required by IOM [19] and WHO [20], organizations have to measure the quality of care they provide [9]. The rationale for measuring quality improvement and patient safety is the belief that good performance reflects good-quality practice [21]. CR is one of the most appropriate sources in order to measure and evaluate quality of care [2, 3, 7, 12–15] and the diligence with which information is recorded by professionals in it is an important quality indicator that needs to be evaluated in healthcare organizations [2, 14, 16, 22]. Indeed, better recording of clinical information leads to better communication between healthcare providers and could contribute to better patient outcomes and safer healthcare [2, 23]. Developing a skilled and responsive healthcare workforce is a fundamental step in creating a safety climate within any healthcare system [24, 25] and in improving quality of care, such as clinical record keeping [14, 26]. For this reason, the involvement of healthcare workers (HCWs)—intended as direct-Care Healthcare Workers—in the process of quality assessment and management, as well as their collaborative work is crucial [27, 28]. Peer-review [29], as well as feedbacks [30], are widely used strategies to promote improvements and induce desired changes [31].

The Academic Hospital of Udine is the hub hospital of Friuli-Venezia Giulia (Italy). It consists of 42 inpatient units, it has 40,000 inpatients/year, 1,000,000 outpatient/year and 3,700 HCWs are employed. The Hospital implemented a variety of strategies, as contribution to healthcare quality at meso-level [32], to improve hospital system, promoting a safety climate and safety practices through systematic measurement of processes, such as the completeness of CR. This systematic measurement through a peer-review tool for CR quality assurance should help to demonstrate whether improvement efforts lead to change in the desired direction.

This study aims to develop a peer-review tool for clinical records quality assurance, presenting the seven-year experience in the evolution of it, and to describe the CR completeness. The second purpose is to describe the professional diligence toward recording clinical information in the CR through these seven years.

Methods

Tool design

Qualified professionals from each unit, 57 doctors and 44 nurses, were selected as direct-Care Healthcare Workers on a voluntary-basis, taking to account their attitude, interest and professional competence in quality of care and patient safety. Thus, they were formally appointed by chief of their unit. They were then trained at regular intervals by the Quality Unit trough meetings and focus groups concerning the structure of the paper-based CR in use, the specific process of CR quality assessment and the tool actually in use. Some of the professionals changed over the years, mainly due to turnover. For this reason, the training was repeated whenever HCWs in charge of the assessment or the tool itself changed across the years.

An electronic peer-review tool was developed to assess the quality of the paper-based CR. HCWs were completely independent in the peer-review process: this was considered as unobtrusive. The peer-review tool included several items that measure the filling in state of specific sections of the CR. Every item was assigned a score of 1 if the specific section was filled in, 0 if it was not, NA (Not Applicable) where the specific item was not applicable.

All items identified during these years (2010–2016) reached a total of 92, and they assessed activities that belong to different HCWs profiles: doctors (59 items), nurses (21 items), both of them (10 items) and therapists (2 items). The items explored different themes and moments of the hospitalization, as well as multiple levels reported in the CR, as shown in Fig 1. The main motivation for categorizing items was to group them into homogeneous main subject of patient care (themes) and in different steps within hospitalization (moments), considering also the role responsible for the activities. Explored themes consisted of: pain management (9 items), patient education (3 items), nutrition (4 items), patient falls (4 items), bedsores (3 items), surgery (14 items) and anaesthesia (9 items). Moments were: initial assessment (14 items), management (39 items), drug therapy (6 items), handover (25 items) and informed consent (5 items). The peer-review electronic tool was designed to explore items also in multiple levels, which should be considered as progressive steps in the care process. Level 1 examined documentation of relevant clinical assessments in CR (52 items)–for example, initial evaluation of pain. Level 2 investigated documentation of an in-depth analysis regarding that specific topic (29 items)–for example, details on the pain location or response to drugs. Level 3 evaluated re-assessment of the specific issue after intervention (9 items)–for example, pain evaluation in a specific time span after drug therapy. Achievement the third-level item was considered a marker of a healthcare system quality, as it demonstrates both measurement and management of the whole specific care process (in our example: correct and effective pain management). HCWs profiles, themes, moments and levels frameworks intersections within CR completeness peer-review tool are shown in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Model of the peer-review tool.

The total number of items identified during the seven years experience reached a total of 92 items. The items belong to different healthcare workers profiles: doctors (n. 59 items), nurses (n. 21), both of them (n. 10) and therapist (n. 2 –not shown). The items, moreover, explore 7 themes (n. 46), 5 moments (n. 89) and 3 levels (n. 90).

https://doi.org/10.1371/journal.pone.0261018.g001

The tool changed over this period: the number of items varied annually from 65 at the beginning to 56 final ones, with some items being dismissed, other added, and still others upgraded to better explore more advanced phases of clinical care. Among the dismissed items: 25 were deemed obsolete, six not enough informative (due to the sporadic information assessed), nine too arbitrary and 12 ready for upgrade (≥95% compliant for three consecutive quarters, at hospital-aggregated level). Among the 43 items added: 35 were the update version of previous items, while eight were completely new (Fig 2). Examples of items stopped because they already had high levels of compliance were those investigating the surgical report and the discharge letter. Examples of too arbitrary items were those regarding readability of handwritten clinical notes. Items about participation in clinical trials and obtaining informed consent, which were too sporadic, were found to be insufficiently informative.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The evolution of items of the peer-review tool.

The tool started in 2010 with 65 items and arrived at 56 items in 2016. During these years the items identified changed over time from a minimum of 59 (2013–2014) to a maximum of 70 (2011–2012). Some items were added (n. 43 items), some others were dismissed because considered obsolete (n. 25), arbitrary (n. 9), not informative (n. 6) or ready to be upgraded (n. 12) to better explore other phases of clinical care. No changes in the peer-review tool occurred in 2011 and 2014. Internal agreement was performed once a year from 2012 to 2016, while external agreement was performed from 2011 to 2015, both with the exception of 2014.

https://doi.org/10.1371/journal.pone.0261018.g002

Results

In the 2010–2016 period a total of 9,408 CRs (336 CRs quarterly) were evaluated by the doctors and nurses, trained as previously described, using the specific peer-review electronic tool. From 2010 to 2016 the overall completeness of CR improved significantly (χ², p<0.001), notably in the early years when from 79.6% in 2010 it reached 88.1% in 2013, settling to 86.5% in 2016 with a not linear overall trend (χ²_d, p<0.001) (Fig 3). As shown in Fig 3, analysing data about doctors and nurses separately, doctors’ behaviour showed a trend that is similar to the aggregated one [significant overall trend (χ², p<0.001), although not linear (χ²_d, p<0.001)]: from 77.8% in 2010 to a maximum of 86.0% in 2013, while nurses improved more consistently, reaching higher level of completeness: from 83.1% in 2010 to a maximum of 92.4% in 2013 (χ², p<0.001; χ²_d, p<0.001). Further details can be obtained consulting Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Overall and professional role proportion of items completeness.

The overall completeness improved from 79.6% in 2010 reaching 88.1% in 2013. The completeness of doctors’ items reached a maximum of 86.3% in 2013, starting from 77.8% in 2010. Nurses’ items increase from 83.1% in 2010 to a maximum of 91.5% in 2013.

https://doi.org/10.1371/journal.pone.0261018.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Item completeness by professional role.

https://doi.org/10.1371/journal.pone.0261018.t001

All themes, except patient falls, registered a significant (χ², p<0.001) improvement in first years, and then flattened in the last years (χ²_d, p<0.001). Particularly, in the items investigating nutrition, doctors improved from 65.9% (2010) to 89.3% (2012) and nurses from 59.5% (2011) to 79.6% (2013). Regarding bedsores (χ², p = 0.0121) nurses improved from 82.6% (2010) to 95.0% (2012). As far as pain management is concerned, doctors improved from 59.9% (2010) to 94.3% (2013). Moreover, considerable completeness level was achieved in patient education, reaching a maximum of 91.4% for doctors and 94.4% for nurses in 2016, with a significant overall trend, although not linear (χ², p<0.001; χ²_d, p<0.001).

Examining moments for both HCWs profiles, completeness of CR achieved 92.6% in initial assessment in 2016 (doctors: 88.6%, nurses: 96.2%) and a maximum of 90.2% in management in 2012 (doctors: 87.1%, nurses: 94.8%) with a significant (χ², p<0.001) improvement, although not linear also in these cases (χ²_d, p<0.001). Conversely the less compliant item was drug therapy, which reached a maximum of 80.0% completeness level in 2014, settling to 74.1% in 2016. More detailed information can be found in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Trend tests by themes and by moments and professional role.

https://doi.org/10.1371/journal.pone.0261018.t002

All the three levels achieved a significant improvement (χ², p<0.001), although not linear (χ²_d, p<0.001). Particularly, Level 1-assessment started in 2010 with 80.0%, reaching 89.4% in 2016, remained at a higher completeness than Levels 2-management (82.9%), and Level 3-re-assessment (82.4%), as shown in Table 3. When analysing data according to levels framework, overall result is driven up by nurses’ items completeness: for Level 1, in fact, nurses achieved the completeness of 96.6% as a maximum in 2015, while doctors stopped at 88.4% in 2014. Results of both levels of data validation (internal and external agreements) were always above the proportion of 75%. This cut-off was selected by the hospital leadership as satisfactory for assuring the quality of data collected and so the reliability of the performance measurement (Fig 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4.

Proportion of internal (A) and external (B) agreement in the data validation process. Internal agreement was performed once a year from 2012 to 2016, with the exception of 2014. External agreement was performed from 2011 to 2015 after the major changes of items, major changes of CR and in case of new professionals involved. The two proportions of agreement were above the threshold (75%) decided by the Quality Unit as satisfactory. Table legends show, for each year, the proportion of agreement and the number of items validated by the professionals of the Quality Unit and the peer-reviewers from each units.

https://doi.org/10.1371/journal.pone.0261018.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Trend tests by levels.

https://doi.org/10.1371/journal.pone.0261018.t003

Discussion

Clinical record is the main communication tool used by HCWs to ensure safe continuity of care. As primary source of written evidence of the care provided to patients, CR is the best source in order to evaluate quality and appropriateness of care in clinical governance [6]. With this study we answered the need to evaluate the information recorded by professionals in CRs and ensure that accurate and complete data are recorded consistently [6, 14, 15, 26].

The completeness of CR, assessed using this peer-review tool, showed an overall positive and significant trend through these seven years of experience, especially in early years. Although doctors’ diligence showed a positive trend that was similar to the overall assessment, nurses improved more consistently.

Except for the first two years, our findings showed an overall completeness of CR always being above 85% and demonstrated an improvement through the seven years of this experience. Level of completeness in the 42 Units remained above the threshold, even though some items have been changed over years, with old items becoming obsolete and new items being added for a deeper analysis of the care process, as required by the QU. Our results, therefore, reflect a good quality practice [21], especially in some clinical areas and moments of care. Both themes and moments, in fact, registered significant improvements, indicating greater attention by HCWs in several clinical areas, such as patient education and bedsores, or in different periods of care, as for initial assessment and management. Concerning moments, however, drug therapy only registered a slight improvement. This is probably due to the complexity of the explored drug therapy items. For example, the item concerning prescription aggregated several fields included in CR (legibility, indication of active substance, dose and route of administration, date and hour of prescription, medical doctor’s signature). Therefore, it was more challenging for professionals to fill in all requested fields simultaneously. However, the reduced completeness in drug therapy items could be a predictor of poor quality of care [10, 11, 23]. These findings are consistent with prior research [12, 18] and suggested that further investments and attention should be put on doctors behaviour in order to prevent potentially harmful consequences for patient safety [11, 12, 23]. Despite the results obtained in themes and moments, there is the opportunity to improve performance as far as levels are concerned, achieving a better completeness in the Level 2-management and Level 3-re-assessment, pursuing a better quality of care. As the initial assessment reflects the first approach to the patient, the higher completeness of Level 1 items can be explained if we assume that professionals were already used to record it from the beginning. On the contrary, management (Level 2) and re-assessment (Level 3) represent progressive levels of care and registered slight improvements in completeness probably because of the greater difficulty to implement them. The multilevel evolution of items, indeed, reflects the growth of both quality of care provided and its documentation in clinical records by HCWs.

Furthermore, data obtained in this 7-year experience point out different diligence between professional profiles, as doctors practice showed a positive trend, even though nurses improved more consistently. These data may suggest a recalcitrant attitude among physicians, whereas nurses appear to be more diligent in following procedures and document actions, as we previously demonstrated for Incident Reporting [33] and for drug prescription [18]. Our findings confirm existing literature on evaluating peer-review and feedbacks processes, which suggest that physicians are relatively poor at self-assessment [34], probably because they tend to seek continuing improvement in their specific areas of interest, rather than areas of greatest and common need [35]. These findings should be taken into account for further development of both the feedback method and this tool, such as exploring more in detail doctors items, in particular drug therapy [18] and nutrition [29, 36].

This electronic peer-review tool was structured to provide consistent measurement and being unobtrusive while respecting HCWs independence at the same time, as suggested by Wang et al. [37]. Moreover, the combination of peer-review by skilled HCWs and the possibility to confirm data with external agreement allowed the QU and the hospital leadership to obtain a reproducible and reliable picture of CR completeness within the Hospital. The transparency of data and a clear information flow is also fundamental for the hospital leadership in order to receive the message regarding performance as intended [35]. The involvement of appointed HCWs from each unit, in addition, enabled the hospital QU to collect a large amount of data in short time, as well as to give short-term feedback to both units and the hospital leadership. In this way we think to have optimised the feedback process with a relatively inexpensive strategy, as wished by Ivers et al. [35, 38–41]. These short-term feedbacks, given to professionals in two-weeks-time, enabled every unit to discuss its own performance during internal periodic meetings, and to implement actions or interventions for continuous improvement purposes. The whole review and feedback process enable the healthcare organization to reach improvements in a three-months-period and to keep monitoring CR completeness. The effectiveness of this process is supported also by the long-term follow-on provided by the QU. Our two-way information flow (from HCWs to Quality Unit and back) confirm findings of literature about the efficacy of rapid analysis of data, intensive feedback, and data delivering by representative and respected colleagues [30].

Although the CRs adopted in the Academic Hospital of Udine were paper-based, we are aware that several authors exhorted the adoption of electronic health records [42, 43]. Since recording information in CR is burdensome and time-consuming, electronic health records may contribute to facilitate the recording, supporting completeness, accessibility and exchange of patient information [4, 44]. Although data of electronic clinical records are entered by humans and therefore prone to error, they might be more complete, given the possibility of integrating multiple data sources [45]. The tool and the process of feedback we implemented is compatible with these systems: it can be facilitated in collecting health and social data. In addition, previous literature demonstrated the effectiveness of audit and feedback to clinicians for quality improvement initiatives using real-time data generation from electronic health records [46–49], allowing a more detailed examination of quality of care and operational inefficiencies [42].

Although several articles address CR completeness, they are characterized by diversity in the population observed and in extracted elements examined from CR [7, 12–14, 18, 26, 31, 50–52]. Our results derive from all CR sections as a whole and from all hospital units. We deem this information characteristic a strength, because it allowed the QU to collect a large amount of data from both clinical and surgical departments, but also from highly-specialized units as neurosurgery, hearth surgery, pediatrics and neonatal intensive care unit, solid organ transplant center and adult intensive care unit, while monitoring the whole hospital, after aggregation of data.

This peer-review method faces some limitations. First of all, data consistency could be difficult to be guaranteed, due to many HCWs belonging to different hospital units involved in the review. The large number of HCWs, in fact, requested a considerable educational effort in order to maintain tool knowledge updated when items or HCWs in charge of evaluation changed. Even if internal agreement missed in 2011 and 2014, and external agreement missed in 2014, since no major changes in items or appointed HCWs occurred in these years, we believe that missing data did not affect the peer-review process reliability, as both internal and external agreement were always over 75%. Moreover, although we performed the data validation through internal and external agreement (see Method section), we didn’t calculate any test score for reliability coefficient, such as kappa-statistics, and the reliability of collected data were evaluated only by joint-probability of agreement. Secondly, a diminished completeness of CR could be difficult to be distinguished from actual deficiency in clinical performance. Nevertheless, as several other authors [3, 12–16], we consider completeness CR as a proxy indicator of the quality of care provided to patients. Lack of CR completeness could be seen as an alert indicating that specific parts of CR are aimless or repetitive. Therefore, involving HCWs in the peer-review and feedback process could also be useful in order to modify the structure and usability of the CR, improving professional diligence toward recording medical information in it. In our opinion, such a new quality and clinical integrated approach could also benefit from a more active physicians involvement, because of their specific attention to clinical outcomes [31, 53].

Despite some limitations, the implementation of this periodical peer-review tool have several advantages and strengths. It is an unobtrusive and economical assessment tool, more reliable than non-quantitative methods, such as focus groups or clinical audits. The tool is structured with items familiar to HCWs and this facilitates data collection in two weeks-time. The improvement of the HCWs’ diligence in the registration of patient information are desirable to enhance quality of care, and thus patient safety and patient outcome. Peer-review [29], as well as feedbacks [30], are widely used strategies to promote improvement and induce desired changes [31]. Our approach allows HCWs to be part of the quality program, improving quality and safety of care by strengthening a safety climate, as wished by several studies, commentaries or analysis [9, 25, 28, 29]. Therefore, in our opinion, healthcare leadership should motivate healthcare professionals to participate in peer-review processes as an opportunity to look at their work and to be offered helpful criticism [29], in order to achieve safer practices, reducing medical errors and ensure patient safety. Although this tool has the limit of having been developed on and used with paper CR, measuring the HCWs’ behaviours and spreading the culture of healthcare quality allowed the Academic Hospital to deepen the level of process analysis and to highlight different compliance habits among HCWs profiles.

Conclusions

This peer-review tool, structured to be unobtrusive and including representative items, enables the Quality Unit and leadership of the hospital to have a reliable picture of clinical records completeness as an indicator of quality of care. The adoption of this tool allows healthcare workers to be part of the quality process during evaluation, supporting them in self-improvement and in finding standards for clinical records keeping. This would benefit in improve completeness and accuracy of information, and thus might improve communications and patient outcome.

Acknowledgments

The authors and the hospital leadership wish to thank all the healthcare workers of the Academic Hospital (Azienda Sanitaria Universitaria Integrata di Udine) involved in this project who collected data for the continuous improvement purposes in these years.