Fig 1.
Smartphone signal acquisition procedure.
(A) A subject’s fingertip is placed on a smartphone lens during the signal acquisition procedure, (B)the screenshot of our developed app displays acquired PPG signal, instantaneous heart rate, and remaining time on a smartphone screen in real-time during the measurement procedure, (C) PPG sensor of the Nexus is measuring the PPG signal from the subject’s other finger as a gold standard measurement. The hand with the NeXus PPG sensor is in a still position during the total duration of the measurement procedure.
Fig 2.
A flow chart of our proposed diversity method.
Fig 3.
Color intensity signal and fingertip movement signal obtained from a smartphone camera video recording consisting of 3,600 successive images.
(A) Four source images (201st, 211th, 267th, and 296th image), (B) green color intensity values, visualized as a mapped image in each source image, (C) ROI in each source image, and (D) color intensity (red solid line) and fingertip movement (blue dashed line) signals. In Fig 3D, the y-values of 201st(red-rectangle), 211th(purple-star), 267th(blue-circle), and 296th(black-triangle) for color intensity signal (or red solid line) are directly calculated from the four average intensity values of green color in Fig 3B of the source images. On the other hand, the y-values of the four points on the fingertip movement signal (or blue dashed line) in Fig 3D are directly calculated from the four ROI sizes, shown in Fig 3C of the source images, shown in Fig 3A.
Fig 4.
Procedure of obtaining a point value on a color intensity signal from a source image.
(A) source image, (B) red, green, and blue channel images of the source image, (C) bit representation of intensity values for each color channel image and (D) green channel image consisting of the green values at each pixel of the source image.
Fig 5.
Bit rearrangement substep applied to every pixel of a source image for obtaining fingertip movement signal.
(A) RGB888 format representing a source image, (B) conversion of RGB888 format into RGB 565 format, and (C) bit rearrangement.
Fig 6.
Preprocessing of smartphone video camera images before acquiring a fingertip movement signal.
(A) The original frame from smartphone recording before any processing, (B) bit rearrangement, (C) edge detection, (D) smoothing, (E) binarization, and (F) ROI size which is calculated as a sample point of the fingertip movement signal.
Fig 7.
Curve and ROI selection procedures for fingertip movement signal.
(A) First frame, and (B) second frame.
Fig 8.
An example of clean and corrupted signals.
(A) A sinusoidal signal consisting of clean (30sec—40sec) and corrupted (40sec—50sec) parts, and its corresponding SQIs’ values: (B) standard deviation of instantaneous heart rate (STD–HR), (C) root mean square of the successive differences of peak-to-peak time intervals (RMSSD–T), and (D) standard deviation of peak values (STD–PV).
Fig 9.
A flow chart of the proposed MNA detection and diversity method.
Fig 10.
The color intensity and fingertip movement signals acquired using an iPhone.
Fig 11.
The SQIs’ values acquired from heterogeneous color intensity and fingertip movement signals shown in Fig 10.
(A) Standard deviation of instantaneous heart rate (STD–HR), (B) root mean square of the successive differences of peak-to-peak time intervals (RMSSD–T), and (C) standard deviation of peak values (STD–PV).
Fig 12.
Box plots for mean of SQIs of clean and corrupted segments.
(A) color intensity and (B) fingertip movement PPG signals. Central mark in each of the plot denotes median value of the SQIs while edges of the box denote 25th and 75th percentile value. The bars above and below to the box represent the extreme data points excluded from being considered outliers, and points represented by ‘+’ are considered outliers.
Table 1.
The p-values between SQIs’ values of clean and corrupted segments.
Fig 13.
MNA detection decisions made by our proposed MNA detection method.
(A) filtered fingertip movement and color intensity signals, (B) annotation from NeXus signal, (C) estimation of our proposed diversity method, (D) first-half of the color intensity signal (0s-60s), (E)second-half of the color intensity signal (60s-120s), (F) first-half of the fingertip movement signal (0s-60s), and (G) second-half of the fingertip movement signal (60s-120s).
Table 2.
Accuracy of the MNA detection using the proposed method.
Table 3.
Usable period ratio (clean segments ratio) using the proposed method.
Table 4.
Comparison of MNA detection performance between the proposed method, the RMSSD-T only method in [48], and the STD-PV only method in [48, 49].
Fig 14.
The two-dimensional representation of the SVM decision boundary for fingertip movement and color intensity signals.
The green stars are corrupted samples and red crosses represent the clean samples. The samples selected as support vectors are marked with black circles. (A) The SVM boundary after applying the SMOTE technique on fingertip movement signal, (B) the SVM boundary of fingertip movement signal without the SMOTE technique, (C) the SVM boundary after applying the SMOTE technique on color intensity signal, and (D) the SVM boundary of color intensity signal without the SMOTE technique.