Fig 1.
The flow chart of the human-computer collaborative sleep scoring system.
The system consists of (A) fully automatic scoring, (B) reliability analysis, and (C) human-computer collaboration.
Fig 2.
Histogram of 12 sleep features in Wake, N1, N2, N3, and REM stages of 30 PSG data.
The X-axis represents the normalized feature values and the Y-axis represents the number of epochs. Features are power: (A) 0–30 Hz EEG, (B) 0–30 Hz EMG; power ratio: (C) 0–4 Hz / 0–30 Hz EEG, (D) 8–13 Hz / 0–30 Hz EEG, (E) 22–30 Hz / 0–30 Hz EEG; power (F) 0–4 Hz EOG; spectral frequency: (G) 0–30 Hz mean frequency EEG, (H) 0–30 Hz mean frequency EMG; duration ratio: (I) alpha ratio EEG, (J) Spindle ratio EEG, (K) SWS ratio EEG; amplitude: (L) mean amplitude EMG.
Table 1.
Features for automatic sleep scoring.
Fig 3.
The hypnogram and SWR features of subject no. 1.
The hypnograms scored by gold standard (A) and the automatic staging system (B). The values of features 0–30 E (C), SWS E (D), and 0–4 O (E).
Fig 4.
The hypnogram and SCD value of PSG from subject no. 3.
The hypnograms scored by gold standard (A) and the automatic staging system (B). The value of feature SCD (C). The red lines indicate disagreement between the expert and the automatic scoring system.
Fig 5.
The hypnogram and SCF values of subject no. 5.
The hypnograms scored by gold standard (A) and the automatic staging system (B). The value of feature SCF (C). The red lines indicate disagreement between the expert and the automatic scoring system.
Fig 6.
Reliability analysis examples.
Disagreements can be detected by using the SWR (A), SCD (B) and SCF (C) features.
Fig 7.
The architecture of the voting process.
According to the value of the SWR, SCD, and SCF features, the voting process determines a scored epoch as a high reliability or low reliability.
Fig 8.
Evaluation of the HCSS system.
(A) The agreement in overall, high-reliability and low-reliability epochs, along with the kappa coefficient between the manual scorings and the HCSS system collaborated scorings. (B) The average of the scoring time for one subject spent in manual and HCSS groups. Percentage of reduced manual scoring time with the assistance of the HCSS system; OA: overall, HR: high-reliability, LR: low-reliability.
Table 2.
Agreement comparison with respect to data with good and poor sleep efficiency.
Table 3.
Comparison of agreement between the gold standard, scorer 1 and scorer 2 for manual scoring and the HCSS method.
Table 4.
Percentage of low-reliability epochs across sleep stages where sleep experts have to examine.
Table 5.
Comparison of agreement between different automatic scoring methods.
Fig 9.
Hypnograms of the subject no. 4.
The hypnograms scored by fully manual scoring (scorer 2) (A), fully automatic staging (B) and the HCSS system (C).
Table 6.
Agreement comparisons with respect to different threshold levels in the reliability analysis.