2.7.1. Proposed appraisal method explanation.

In this study, we implemented a detailed scoring system to critically appraise the included studies, ensuring a rigorous evaluation of their quality and relevance. This system is based on three main criteria: validity, results, and relevance, with each criterion contributing to an overall score that reflects the study’s robustness and applicability.

1. a). Validity was assessed according to the variety of models applied in the study. Studies that applied more than four models received the highest score of 3 points, those using 3 or 4 models received 2 points, and studies applying fewer than three models received 1 point. This measure reflects the comprehensiveness of the study’s approach to model evaluation and comparison, which is crucial for robust and reliable findings.
2. b). Results were evaluated by examining the reported accuracy of the study’s outcomes. This measure acknowledges the significance of precise and reliable results in validating a study’s conclusions. Studies reporting an accuracy greater than 90% were awarded 3 points, those between 80% and 90% received 2 points, and studies with accuracy between 70% and 80% received 1 point. Studies reporting accuracy below 70% or not reporting accuracy at all received 0 points.
3. c). Relevance was determined by the extent to which the study focused on dysgraphia, handwriting, and classification problems. Studies mentioning “dysgraphia” and “handwriting” received the highest score of 3 points, indicating a direct relevance to our research focus. Studies mentioning only “dysgraphia” received 2 points, while those mentioning only “handwriting” received 1 point. Additionally, studies that addressed a classification problem received 1 point. Studies that did not mention either term or address classification problems received 0 points.

The total score for each study, ranging from 0 to 9, was calculated by summing the points from all three criteria. This total score allowed us to classify the studies into categories of high quality (7–9 points), moderate quality (4–6 points), and low quality (0–3 points). This structured approach ensured a comprehensive and transparent appraisal of the evidence, allowing us to base our conclusions on the most reliable and relevant studies available.

3.3.1. Dataset characteristics.

The analyzed data in the included papers were sourced from various repositories such as databases, registers, and health information systems (Table 2). Information about datasets was extracted in both general and specific ways. Multiple tasks were applied to gather the dataset, including copying sentences, writing digits, writing individual words, and performing the BHK test in communication with humans and robots. Two papers utilized multiple datasets (n = 2, 9%) [20,33], while only three used raw handwritten images (n = 3, 14%) [32,37,57], including words, sentences, and digits. The other studies incorporated additional features, including kinematic and temporal characteristics, and documented supplementary data captured during tasks using specialized equipment such as Wacom Tablets. Open-access datasets are provided in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of dataset information.

https://doi.org/10.1371/journal.pone.0328722.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Open access datasets.

https://doi.org/10.1371/journal.pone.0328722.t003

3.3.2. Studies’ methodology characteristics.

Various models have been applied in these studies (Table 4), including SVM and Random Forest (RF), which were used as benchmarks to compare with the innovative model in a considerable number of studies (SVM = 11, 52% - RF = 9, 42%). One study deviated from the norm by employing a clustering method instead of a traditional classifier, utilizing it as a classifier (n = 1, 4%). The primary evaluation metric is accuracy used by 19 studies (n = 19, 90%), ranging between 79% and 99%. Additionally, three studies used ROC and AUC metrics (n = 3, 14%), and three employed the Matthews correlation coefficient (n = 3, 14%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Overview of methodological approaches.

https://doi.org/10.1371/journal.pone.0328722.t004

3.5.1. SVM.

SVM [59] is a classification model designed to identify the optimal hyperplane that separates data points in the feature space while maximizing the distance between the closest points on either side of the hyperplane (Fig 5). This margin maximization produces a robust model, as demonstrated in linearly separable data examples.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Examples of linearly separable data distributions [60].

https://doi.org/10.1371/journal.pone.0328722.g005

In ideal scenarios, data are linearly separable, allowing for straightforward training of a linear SVM. However, real-world data often requires a more flexible approach. By introducing slack variables, SVMs can maximize the soft margin, permitting some classification errors and thereby improving the handling of nearly linearly separable data and reducing overfitting. For linearly inseparable data, SVMs utilize kernel functions to map data to higher-dimensional spaces, making them linearly separable.

Eleven of the included papers (n = 11, 52%) [20,31,34,38–43,46,48] utilized SVM for processing handwritten images, integrating additional features such as kinematic, temporal, and spatial data. The most effective kernels employed were the one-class SVM [42] and the Radial Basis Function (RBF) kernel [43]. Additionally, combining SVM with fusion models has been shown to be particularly effective in improving the performance of these systems [20]. In the context of dysgraphia diagnosis, SVMs excel in handling small, high-dimensional handwriting datasets, such as those incorporating kinematic, spatial, and temporal features from Wacom Tablets [20,43]. Their robustness to overfitting makes them particularly suitable for language-specific datasets with limited samples, as demonstrated by the use of one-class SVM and RBF kernels [42,43]. However, their high computational cost may pose challenges for real-time applications in educational or clinical settings, and performance relies on careful kernel selection to accommodate the variability in dysgraphic handwriting patterns. A summary of the advantages and disadvantages of the SVM algorithm for dysgraphia diagnosis is provided in Table 6.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Advantages and disadvantages of the SVM algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t006

3.5.2. KNN.

The K-Nearest Neighbors (KNN) algorithm classifies data points by their closeness to other data points within the feature space [61]. For a given input, it identifies the K closest neighbors and assigns the most common label among them to the input. This method assumes data points with similar characteristics are located near each other, making it practical for various classification tasks, including image processing (Fig 6). KNN is non-parametric and straightforward to implement but can be computationally intensive with large datasets and sensitive to the choice of K and distance metrics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. KNN algorithm overview [62,63].

https://doi.org/10.1371/journal.pone.0328722.g006

In four of the included papers (n = 4, 19%) [38,39,31,41], KNN was utilized; in three of these, it served as a classifier, while in one, a variant called Weighted KNN_F5 [64] was used for feature extraction. Weighted KNN_F5 assigns different weights to neighbors based on their distance, giving closer neighbors more influence on the classification outcome. While the results of KNN as a classifier were not particularly strong, Weighted KNN F5 for feature extraction showed the potential to enhance the model’s performance. For dysgraphia diagnosis, KNN’s simplicity makes it a valuable baseline in early-stage studies, effectively classifying spatial and temporal handwriting features [38,39]. Variants like Weighted KNN F5 enhance feature extraction, improving performance with small datasets [64]. However, its sensitivity to noise, such as inconsistent stroke patterns in dysgraphic handwriting, limits its accuracy in complex cases [31]. Additionally, KNN’s computational intensity with large or high-dimensional datasets poses challenges for scalable dysgraphia screening in educational settings. A summary of the advantages and disadvantages of the KNN algorithm for dysgraphia diagnosis is provided in Table 7.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Advantages and disadvantages of the KNN algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t007

3.5.3. Random forest.

The Random Forest algorithm [65] is a versatile and robust ensemble learning technique for classification and regression tasks. It works by constructing multiple decision trees [66] during training and outputting the mode of the classes (for classification) or mean prediction (for regression) of the individual trees (Fig 7). This approach leverages the wisdom of crowds, resulting in more accurate and robust predictions compared to a single decision tree.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Random forest process [67].

https://doi.org/10.1371/journal.pone.0328722.g007

Nine of the included papers (n = 9, 42%) [34–36,38,39,32,41,42,47] applied the Random Forest (RF) algorithm for dysgraphia detection based on images. While RF showed promising results when combined with other models, its performance was not as strong. In dysgraphia detection based on images, Random Forest is particularly effective because it can manage high-dimensional data and capture complex patterns [36,45]. In dysgraphia diagnosis, Random Forest’s ability to handle mixed feature types, such as spatial, temporal, and pressure-based features, makes it highly effective for multimodal handwriting analysis, achieving robust performance in complex datasets, particularly valuable for diverse handwriting patterns in dysgraphia studies [36,45]. However, its computational intensity may hinder real-time screening in clinical or educational settings, and performance can vary with imbalanced datasets, necessitating careful feature selection [36]. A summary of the advantages and disadvantages of the Random Forest algorithm for dysgraphia diagnosis is provided in Table 8.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. Advantages and disadvantages of the random forest algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t008

3.5.4. K-means.

K-means is an unsupervised machine algorithm that groups data into clusters to identify distinct classes [68]. The algorithm involves four main steps: First, K initial clustering centers are selected. Second, each sample in the dataset is allocated to the nearest cluster center based on distance calculations. Third, the cluster centers are recalculated for each group. Finally, the second and third steps are repeated until a specified stopping criterion, such as reaching the maximum number of iterations or achieving minimal changes in cluster centers, is met (Fig 8).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. The example of the K-means algorithm [69].

https://doi.org/10.1371/journal.pone.0328722.g008

K-means can also be adapted for classification tasks by defining the number of clusters, K, to correspond to the number of classes [44]. After the clustering process, new data points can be classified by assigning them to the nearest cluster center. This approach leverages the simplicity and efficiency of K-means to create a basic classification model that groups data points based on their feature similarity to predefined cluster centers. In dysgraphia diagnosis, K-means is effective for clustering dysgraphic vs. non-dysgraphic handwriting patterns in exploratory studies, leveraging its simplicity for small datasets [44]. Its computational efficiency supports initial detection tasks. However, its performance is limited by small, imbalanced dysgraphia datasets, which may result in poor clustering of complex handwriting patterns [44]. Additionally, the need to predefine the number of clusters poses challenges for heterogeneous dysgraphia data with varied writing styles. A summary of the advantages and disadvantages of the K-means algorithm for dysgraphia diagnosis is provided in Table 9.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. Advantages and disadvantages of the K-means algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t009

3.5.5. AdaBoost.

AdaBoost, short for Adaptive Boosting, is a powerful ensemble learning algorithm for classification tasks, introduced by Yoav Freund and Robert Schapire in 1996 [70]. AdaBoost’s core principle is to improve weak classifiers’ accuracy by integrating them into a robust, strong classifier (Fig 9). It starts by assigning equal weights to all training samples and iteratively trains a sequence of weak classifiers, typically simple models like decision stumps. After each iteration, it evaluates the classifier’s error rate and adjusts the sample weights, increasing the weights of misclassified samples to ensure subsequent classifiers focus on harder-to-classify cases. The final model is constructed by aggregating all the weak classifiers, where each classifier’s influence is proportional to its accuracy. AdaBoost is known for its ability to significantly enhance model performance, its adaptability to various types of weak classifiers, and its robustness against overfitting, although it can be sensitive to noisy data and outliers. This adaptive focusing mechanism and iterative process make AdaBoost a powerful tool in machine learning for creating highly accurate classification models. Notably, seven of the included papers utilized AdaBoost (n = 7, 33%) [20,31,38–40,43,35], with three of them (n = 3, 14%) employing it as the primary model in their studies [38,39,47]. For dysgraphia diagnosis, AdaBoost enhances the accuracy of weak classifiers, making it effective for small handwriting datasets with diverse features like spatial and kinematic attributes [20,38,39]. Its robustness to overfitting supports its use in combining multiple handwriting features. However, its sensitivity to noisy data, such as irregular strokes in dysgraphic samples, may reduce performance in complex cases [20]. Additionally, the need for careful tuning of weak classifiers increases complexity for practical dysgraphia applications. A summary of the advantages and disadvantages of the AdaBoost algorithm for dysgraphia diagnosis is provided in Table 10.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. Advantages and disadvantages of the AdaBoost algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t010

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. The example of the AdaBoost process [71].

https://doi.org/10.1371/journal.pone.0328722.g009

3.5.6. CNN.

Convolutional Neural Networks [72] are a type of deep learning algorithms [73] particularly well-suited for image processing tasks. CNNs are designed to automatically and adaptively learn hierarchical spatial features from input images. The architecture of a CNN typically consists of several layers, including convolutional layers, pooling layers, and fully connected layers (Fig 10). The convolutional layers perform convolution operations to the input image, using a set of learnable filters to produce feature maps that capture different input characteristics, such as edges, textures, and patterns. These feature maps are then passed through activation functions to introduce non-linearity, enhancing the model’s ability to learn complex representations. Pooling layers follow, performing down-sampling operations that shrink the spatial dimensions of the feature maps, thereby reducing computational requirements and providing some degree of translation invariance. Lastly, fully connected layers integrate the extracted features to perform the final classification or regression task. This hierarchical structure allows CNNs to learn more abstract and high-level features progressively, making them very effective for tasks such as image classification, object detection, and segmentation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Sample of a typical CNN [74].

https://doi.org/10.1371/journal.pone.0328722.g010

Seven of the included papers (n = 7, 33%) utilized CNNs, with two (n = 2, 9%) specifically employing well-known CNN-based models: one of the papers [37] using MobileNet [75] and the other paper [33] collectively incorporating Inception [76], VGGNet [77], and ResNet [78]. These studies achieved a high level of accuracy, with an average accuracy of 88% across all papers that used CNNs. Although CNNs are limited in capturing long-distance dependencies [52], they performed exceptionally well in most cases presented in the included papers. In dysgraphia diagnosis, CNNs excel in learning complex patterns from handwriting images, achieving high accuracy with models like MobileNet and ResNet [33,37]. Their ability to analyze spatial and kinematic features makes them ideal for robust image-based diagnosis. However, their reliance on large, diverse datasets is a challenge given the small sample sizes common in dysgraphia research [33]. Additionally, high computational demands may limit their use in resource-constrained educational or clinical settings. A summary of the advantages and disadvantages of CNNs for dysgraphia diagnosis is provided in Table 11.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. Advantages and disadvantages of the CNN algorithm in dysgraphia diagnosis.

https://doi.org/10.1371/journal.pone.0328722.t011

Overall, the choice of an AI model for dysgraphia detection depends on dataset characteristics and application constraints. The trade-offs discussed highlight the importance of balancing model complexity, computational resources, and dataset limitations in achieving effective dysgraphia diagnosis.

3.6.1. Categorization of features.

AI models for dysgraphia leverage diverse features from handwriting samples, classified into four categories. Each category’s role and application differ between online (real-time data, e.g., tablet-based writing) and offline (static data, e.g., scanned images) handwriting, reflecting distinct diagnostic insights.

3.6.1.1. Temporal features: Temporal features capture timing aspects of handwriting, such as stroke duration, inter-stroke pauses, and total writing time. In online handwriting, these features are directly measured via stylus or tablet devices, providing precise data on rhythm and pacing irregularities—hallmarks of dysgraphia. For example, temporal features can detect hesitations in letter formation, which are critical for identifying dysgraphic children who struggle with writing fluency. In offline handwriting, temporal features are inferred indirectly from stroke patterns or writing density, but their accuracy is limited due to the absence of real-time data. These features are particularly valuable in online settings for early detection, as they reveal subtle motor control issues not visible in static samples [38].

3.6.1.2. Spatial features: Spatial features encompass geometric characteristics, including letter size, shape, alignment, spacing between characters/words, and line adherence. In offline handwriting, spatial features are primary, extracted from scanned images to identify visual irregularities like inconsistent letter sizes or crowded spacing, common in dysgraphia. These features are robust for static analysis, as they rely on visible patterns rather than dynamic input. In online handwriting, spatial features are less dominant but still relevant, capturing real-time stroke geometry. Spatial features play a central role in both modalities for diagnosing dysgraphia by highlighting structural handwriting deficits, though offline applications are more prevalent due to their reliance on image-based data [79].

3.6.1.3. Pressure-based features: Pressure-based features measure the force exerted on the writing surface, including mean pressure, pressure variability, and peak force points. In online handwriting, these are captured directly by pressure-sensitive devices, revealing fine motor skill deficits (e.g., erratic pressure patterns indicating grip issues in dysgraphia). Pressure features are critical for assessing motor control precision, a key diagnostic marker. In offline handwriting, pressure features are rarely used, as scanned images lack force data, though some studies infer pressure from stroke thickness. These features are thus more significant in online modalities, enhancing models’ ability to detect motor-related dysgraphia symptoms [31].

3.6.1.4. Kinematic features: Kinematic features quantify handwriting motion, including writing speed, acceleration, jerk (rate of acceleration change), and stroke smoothness. In online handwriting, kinematic features are richly captured via real-time tracking, enabling precise analysis of movement fluidity (e.g., jerky strokes in dysgraphic writers). These features are essential for detecting motor coordination issues. In offline handwriting, kinematic features are inferred from stroke trajectories or curvature, but their granularity is reduced due to static data limitations. Kinematic features are vital in both modalities for assessing handwriting dynamics, with online applications offering superior detail for early diagnosis [39,31,79].

3.6.2. Trends in feature selection.

The selection of features in AI models often depends on the study’s specific objectives and the nature of the data available. Trends observed across the reviewed studies include:

1. a). Online Handwriting: Studies prioritize temporal and pressure-based features to capture real-time dynamics, such as irregular pauses or inconsistent pressure. For example, one study used temporal features to detect prolonged stroke durations, while another leveraged pressure variability to identify motor deficits [31].
2. b). Offline Handwriting: Spatial features dominate, focusing on letter shape, size, and spacing to detect visual irregularities. Kinematic features are less common but are used when stroke patterns allow motion inference [79].
3. c). Hybrid Approaches: Some studies integrate modalities, combining spatial features for offline analysis with temporal and kinematic features for online data, enhancing diagnostic accuracy. Hybrid models are increasingly common in studies aiming for comprehensive dysgraphia detection [21,38].

Additionally, the effectiveness of hybrid models depends on the consistency and precision of data acquisition tools. Variations in tablet sampling rates, resolution, or pressure sensitivity can introduce inconsistencies in feature definitions, impacting the reliability and generalizability of AI models across studies.

3.6.3. Relationship between features and model performance.

Feature choice significantly impacts model performance, as demonstrated by reviewed studies:

1. a). Spatial and pressure-based features enhance accuracy in offline handwriting models. For instance, one study [79] achieved 91% accuracy using spatial features (letter size, spacing) in a CNN model, effectively capturing visual dysgraphia markers.
2. b). Temporal and kinematic features improve sensitivity in online handwriting models. One study [31] reported a 15% sensitivity increase in an SVM model by incorporating kinematic features (writing speed), aiding early motor difficulty detection.
3. c). Combining features across modalities boosts diagnostic efficacy. However, inconsistent feature definitions (e.g., varying pressure metrics across devices) hinder cross-study comparisons and standardization [21,38].

Furthermore, clinical interpretability of features remains a crucial aspect for real-world adoption. Features that align closely with known dysgraphia symptoms, such as prolonged stroke latency or erratic pressure spikes, can enhance clinician trust and provide actionable insights during assessment.

4.1.1. Review process limitations.

Several limitations were identified during the scoping review process. The primary limitation was the availability of studies, with only 9 of the 21 included studies being open access. This restriction may have limited the comprehensiveness of the review, but fortunately, we accessed the other 12 studies by emailing the authors for access or purchasing the articles. Additionally, the sample sizes in many studies were small and often specific to certain languages, which may limit the generalizability of the findings.

4.1.2. Extent of information uncovered.

The heterogeneity in evaluation metrics, such as accuracy, ROC, AUC, and MCC, across studies complicates direct comparisons of AI model performance in dysgraphia diagnosis. This variability underscores the urgent need for standardized metrics to facilitate consistent and comparable evaluations. Each metric serves distinct purposes in assessing model performance: Accuracy measures overall correctness but can be misleading in imbalanced datasets common in dysgraphia studies, where non-dysgraphic cases often outnumber dysgraphic ones. ROC curves illustrate a model’s diagnostic ability across varying thresholds, AUC quantifies separability between classes, and MCC provides a balanced measure robust to class imbalance, making it particularly valuable for dysgraphia datasets with skewed distributions.

Trends in metric usage reveal a preference for accuracy and AUC in studies with large datasets, aiming to balance sensitivity and specificity, while MCC is increasingly adopted in imbalanced scenarios to ensure reliable detection of dysgraphic individuals. In clinical and educational contexts, these metrics have critical implications: high recall (sensitivity) is essential for school-based screening to minimize false negatives, which could delay interventions like occupational therapy for dysgraphic children, whereas high precision is prioritized in clinical settings to reduce false positives, avoiding unnecessary referrals and parental stress. For instance, a CNN-based screening tool deployed in educational platforms (e.g., via web-based systems) might prioritize recall to flag at-risk students, while a diagnostic tool for clinicians would emphasize precision to confirm cases.

These choices reflect meaningful trade-offs: false negatives risk delaying support for children in need, while false positives could lead to stress, mislabeling, and misallocated resources. Therefore, selecting metrics aligned with real-world diagnostic objectives is not merely technical—it directly impacts educational outcomes and patient well-being.

The implications of metric heterogeneity extend beyond academic challenges; they hinder the synthesis of findings and slow the development of standardized diagnostic models. To address this, future research should adopt a core metric set tailored to specific use cases, supported by public benchmark datasets to ensure fair and consistent comparisons. Best practices such as k-fold cross-validation, threshold tuning, and reporting multiple complementary metrics can further enhance metric reliability. Moreover, the development of interdisciplinary guidelines involving clinicians, educators, and AI researchers can help align evaluation criteria with practical diagnostic needs, ultimately fostering trust and real-world adoption of AI tools in dysgraphia diagnosis.

Another challenge lies in the incomplete or unclear information reported in some papers. For instance, certain studies omitted dataset details, such as size, requiring us to infer this from other papers using the same dataset. Additionally, reliance on self-reported data and the subjective nature of some assessments may introduce bias. Furthermore, the search was limited to major databases like PubMed, Scopus, Web of Science, IEEE Xplore, and SpringerLink, potentially missing relevant studies from other sources or unpublished works. Future studies could address this gap through broader, systematic searches to ensure a more comprehensive literature review.

4.1.3. Methodological and research biases.

A more nuanced interpretation of the results in light of current clinical practices reveals that while AI models are highly effective under controlled study conditions, their application in typical clinical settings may not always align with traditional diagnostic methods. This misalignment necessitates a careful integration strategy to ensure that AI tools augment rather than replace the clinician’s expertise. A critical comparison of different AI approaches, such as deep learning models like CNNs versus more traditional machine learning models like SVMs, highlights that while CNNs provide higher accuracy in image-based analyses due to their complex pattern recognition capabilities, SVMs offer advantages in terms of computational efficiency and transparency in decision-making. Examining methodological limitations across studies highlights issues such as the small and non-diverse sample sizes, which limit the extrapolation of findings to a broader population. Additionally, many studies do not undergo external validation, raising concerns about the robustness and reproducibility of the findings.

Discussion of potential biases in current research points to a prevalence of publication bias, where studies with positive findings are more likely to be published, and selection bias, where the data used may not adequately represent the target population. An analysis of gaps between laboratory findings and real-world application indicates that AI models often perform exceptionally well in controlled environments but may fail to deliver similar results in real-world settings. This discrepancy can be attributed to the complex and variable nature of real-world environments compared to the controlled conditions of most studies.