Fig 1.
The key steps in the Bidirectional Knowledge-Based Assessment of Compliance (BIKBAC) method.
Fig 2.
The top-down analysis step of the BiKBAC method’s compliance analysis algorithm.
The dashed rectangles represent iterative steps performed on a specific collection. The top-down analysis is performed on each operative guideline plan in the library.
Fig 3.
The bottom-up analysis step of the BiKBAC method’s compliance analysis algorithm.
Fig 4.
The missing-actions analysis of the BiKBAC compliance-analysis methodology.
Fig 5.
The DiscovErr system’s architecture, implementing the BiKBAC compliance-assessment methodology.
Fig 6.
An example of a comment made by the DiscovErr system.
The comment (A), regarding missing insulin therapy for a newly diagnosed patient with very high levels of HbA1c (>10%) and Glucose (>290 mg/dL), and the relevant excerpt from the original guideline (B).
Fig 7.
A diagram of the experimental steps in our evaluation of the DiscovErr system.
Fig 8.
The interface used by the experts for evaluating the comments of the system.
This interface was designed as part of our study. The expert can view the list of comments on the top left side of the screen (A). When selecting a specific comment, additional explanation is displayed (B) together with graphs showing raw data (C). The explanation includes the specific path of the guideline and a textual description with details about the system scores. The graphs are of one or more parameters that lead the system to its comment. Zoom into the part of the UI for meta-critiquing (D), shows the manner in which the experts expressed their opinion of the DiscovErr system comments.
Table 1.
Description of the data set used in the experiment.
Table 2.
The distribution of the comments given by the experts with respect to the type of compliance issue.
Table 3.
Completeness of the comments given by the experts relative to the unique compliance issues, by level of support of the three experts.
Table 4.
Correctness of the system comments according to both diabetes experts.
Table 5.
Indirect correctness of the experts’ comments, partitioned by level of support of the comments by the other agents, including the system.
Table 6.
Indirect Completeness of the experts in the manual compliance evaluation.
Fig 9.
A profile of the completeness and correctness of the experts and the system.
Table 7.
Summary of completeness and correctness of the system and the experts.