Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

You Only Look Once (YOLO) based machine learning algorithm for real-time detection of loop-mediated isothermal amplification (LAMP) diagnostics

  • Biniyam Mezgebo,

    Roles Formal analysis, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

    Affiliation Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis

  • Ryan Chaffee,

    Roles Data curation, Methodology, Validation, Writing – original draft, Writing – review & editing

    Affiliation Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis

  • L. Ricardo Castellanos,

    Roles Data curation, Methodology, Validation, Writing – review & editing

    Affiliation Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis

  • S. Ashraf,

    Roles Data curation, Methodology

    Affiliation Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis

  • J. Burke-Gaffney,

    Roles Data curation, Methodology

    Affiliation Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis

  • Johann D. D. Pitout,

    Roles Data curation, Investigation, Supervision

    Affiliations Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis, Alberta Precision Laboratories, Calgary, Alberta, Canada

  • Bogdan I. Iorga,

    Roles Investigation, Resources, Supervision

    Affiliation Université Paris-Saclay, CNRS UPR 2301, Institut de Chimie des Substances Naturelles, Gif-sur-Yvette, France

  • M. Ethan MacDonald,

    Roles Investigation, Methodology, Resources, Supervision, Writing – review & editing

    Affiliations Department of Biomedical Engineering, University of Calgary, Calgary, Alberta, Canada, Department of Electrical and Software Engineering, University of Calgary, Calgary, Alberta, Canada, Department of Radiology, University of Calgary, Calgary, Alberta, Canada, Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada, Alberta Children’s Hospital Research Institute, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada

  • Dylan R. Pillai

    Roles Conceptualization, Formal analysis, Investigation, Resources, Supervision, Writing – review & editing

    drpillai@ucalgary.ca

    Affiliations Department of Biomedical Sciences, Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis, Alberta Precision Laboratories, Calgary, Alberta, Canada

Abstract

Loop-mediated isothermal amplification (LAMP) is a widely used rapid and affordable molecular DNA amplification method with minimal resource requirements. However, visual interpretation of results is subjective and prone to errors, leading to potential false-positive and negative results. To address this limitation, a machine-learning approach is proposed for automated LAMP classification based on digital images. The approach utilizes You Only Look Once (YOLOv8), a fast and robust object detection algorithm to locate and classify tubes within LAMP images, enabling automated categorization as positive or negative. The trained model achieved a high overall accuracy of 97.4% in classifying LAMP images into positive or negative on the test set. Additionally, the approach had a 95.3% precision and 96.8% recall for positive cases and 93.3% precision and 95.8% recall for negative cases, demonstrating its potential for real-time LAMP diagnosis and enhanced assay performance. This project demonstrated platform suitability for real-time testing, offering an easy operation and rapid results.

Introduction

Diagnosing infectious diseases like malaria and COVID-19 relies on various techniques, each with limitations. Traditional methods, including pathogen culture, microscopy, and PCR-based diagnostics, often face cost, resource, and turnaround time limitations. While Quantitative Polymerase Chain Reaction (qPCR) offers high accuracy and is considered the gold standard for specific tests, its limitations include the need for specialized equipment, skilled personnel, and complex protocols, hindering its use in point-of-care (POC) settings [1,2]. In this context, isothermal loop-mediated amplification (LAMP) is a compelling alternative for POC diagnostics. LAMP’s simplicity and ability to amplify minimally processed or raw samples make it particularly suitable for resource-limited settings [3,4]. These advantages have fueled LAMP’s exploration in various fields, especially POC diagnostics for pathogens like SARS-CoV-2 [5]. Several POC platforms have been developed, demonstrating LAMP’s potential for rapid and affordable detection [69]. However, LAMP typically relies on qualitative visual detection, which can introduce ambiguity and potentially lead to misdiagnosis.

Integrating deep learning-based detection and prediction methods with existing diagnostic tools is gaining momentum, aiming to produce more time-efficient results and automate interpretation, ultimately assisting healthcare professionals in making informed data-driven decisions [10]. Automated diagnostics can also minimize human-related errors, benefiting patients and healthcare systems. In recent years, researchers have proposed diverse techniques for developing machine learning-driven portable and low-cost LAMP devices for rapid pathogen detection. These approaches range from simple rule-based systems to traditional machine learning (ML) and deep learning (DL) network-based methodologies [10,11]. In the context of LAMP signal analysis, researchers have explored a wide spectrum of ML algorithms to develop efficient classifier models. These include convolutional neural networks (CNNs), linear discriminant analysis, random forests, gradient boosting classifiers (GBCs), Bayesian networks, fuzzy logic, and decision trees [10,11]. For example, Rohaim et al. demonstrated the efficacy of a CNN model in identifying different colours in LAMP images [10]. Similarly, Song et al. developed a machine learning-based smartphone application to analyze SARS-CoV-2 diagnostics results using a LAMP with clustered regularly interspaced short palindromic repeats (CRISPR), referred to as the DAMPR assay system [11]. In their work, ML techniques were employed to quantify the SARS-CoV-2 gene concentration by analyzing 300 sample images for each concentration. The study compared LDA, RF, and GBC classifiers, with RF achieving the best accuracy of 99.38%.

YOLO, introduced by Redmon et al. (2016) [12], is a real-time object detection algorithm that leverages convolutional neural networks [1214]. Unlike conventional two-stage detectors such as R-CNN and Faster R-CNN, which first generate region proposals (RPs) potentially containing objects and then classify them using CNNs, YOLO adopts a single-stage approach [1518]. It divides the input image into a grid and simultaneously predicts bounding boxes, object presence, and class probabilities for each cell, streamlining the detection process. This unified approach significantly reduces processing time compared to multi-stage methods [1926].

YOLO has different versions; among recent advancements, YOLOv5 and YOLOv7 are the most popular variants. YOLOv5 leverages deep learning methodologies and a Cross-Stage Partial (CSP) network structure to improve efficiency and overall performance [26]. However, it still struggles with small object detection and challenging scenarios. YOLOv7, on the other hand, introduced the Trainable Bag of Freebies (TBoF) strategy, enhancing accuracy and generalization through data augmentation and other techniques. While effective, YOLOv7 can be sensitive to training data and model parameters and often demands substantial computational resources [27]. The YOLOv8 model directly addresses these limitations by replacing the anchor-based detection with an anchor-free approach. These modifications eliminate the need for dataset-specific anchor box tuning and reduce dependencies on hyperparameters. Furthermore, it implements adaptive mosaic augmentation, which is deactivated in the final ten training epochs to prevent performance degradation. Additionally, it introduces C2f modules that concatenate all bottleneck outputs, as opposed to C3’s single output, thereby improving gradient flow while simultaneously reducing computational overhead and accelerating Non-Maximum Suppression through fewer bounding box predictions [28].This paper focuses on applying YOLOv8, the latest iteration of the YOLO model, for LAMP image analysis. The YOLOv8 builds upon the strengths of its predecessors while introducing several key improvements: YOLOv8 surpasses previous YOLO versions in terms of accuracy, particularly in challenging scenarios involving small objects or dense environments. This enhanced precision makes it a more reliable tool for LAMP detection, leading to potentially more accurate diagnoses. Also, YOLOv8 strongly emphasizes real-time object detection, making it highly suitable for practical applications requiring rapid results. This characteristic is crucial for efficient diagnosis of malaria, COVID/RSV, and antimicrobial resistance (AMR) among others, in field settings, where timely diagnosis can significantly impact patient outcomes.

In this study, we harness YOLOv8’s efficiency and accuracy to automate the interpretation of LAMP results. The objective of this work is to propose addressing the issue of ambiguity limitation by leveraging machine learning approaches with LAMP to automate result interpretation while maintaining LAMP’s POC advantages. By formulating the classification of LAMP images as a multi-object detection problem, we develop a model capable of precisely locating tubes within LAMP images and classifying them as “Positive” or “Negative” based on visual characteristics. This integration of machine learning with LAMP technology offers several advantages: it streamlines the diagnostic process, reduces human error, and enables real-time disease diagnosis. Moreover, the speed of YOLO aligns well with LAMP’s rapid amplification, potentially creating a synergistic effect that could significantly accelerate the overall testing process. This study demonstrates the potential of combining advanced object detection algorithms with isothermal amplification techniques to enhance the efficiency and reliability of molecular diagnostics.

Results and discussion

This section discusses the results of applying YOLOv8 to LAMP images. Fig 1 shows the prediction results of the proposed research. The positive samples are marked with red bounding boxes, while the negative ones are highlighted with green. Each detected sample in the images is assigned a confidence score/probability that reflects the model’s certainty that a given detection corresponds to a real object and its corresponding class. A confidence threshold above 0.5 was utilized in the algorithm to determine the validity of the detections and their classifications.

thumbnail
Fig 1. Example images of the prediction results of the trained model on unseen data.

(a) – (f) Example images from the “white pearl” demonstrate the capacity of the algorithm to detect and classify positive and negative results of individual wells in 8-tube LAMP strips. The model demonstrates robust detection performance across different illumination conditions and color variations. Green and red bounding boxes indicate negative and positive LAMP reactions, respectively, with confidence scores above each detection. The model maintains high detection accuracy (confidence scores ranging from 0.73 to 0.91) despite challenging variations in: (1) background illumination (purple to brown tones), (2) fluorescence intensity (bright to dim), (3) sample opacity, and (4) image capture angles.

https://doi.org/10.1371/journal.pone.0339042.g001

Fig 2 shows the loss values for the box loss, object loss, and class loss at each epoch during the training and validation process. The box loss represents the difference between the predicted and ground-truth bounding box coordinates, the object loss represents the confidence score for each object detected in an image, and the class loss represents the probability of each detected object belonging to a specific class. Training an object detection model aims to minimize the total loss, combining box, object, and class loss. The loss values exhibit a decreasing trend as the training progresses, indicating an improvement in the model’s ability to detect tube samples in the images.

thumbnail
Fig 2. Training and validation metrics for the YOLOv8 model on LAMP image detection.

The plots show the progression of various loss functions (box, class, and dfl losses) for both training and validation sets for 80 epochs, with all metrics displayed on the same figure. Performance metrics based on the validation set, including precision, recall, mAP@50, and mAP@50-95 are also displayed, demonstrating the model’s improving accuracy in detecting and classifying LAMP assay results. The smooth convergence of loss functions and the upward trends in accuracy metrics indicate successful training and good generalization of the model for LAMP image analysis.

https://doi.org/10.1371/journal.pone.0339042.g002

The trained YOLOv8 model achieved promising performance in classifying LAMP reaction images, as shown in Fig 1. Table 1 shows the model’s generalization capabilities, showing a minimal performance degradation between the validation and test sets. In the validation set with 198 positive and 287 negative instances, the YoLOv8 model achieved strong performance with precision (P) values of 0.974 and 0.972 for positive and negative classes, respectively, along with recall (R) values of 0.980 and 0.953. The model retained comparable performance on the holdout test set containing 252 positive and 331 negative instances, achieving precision values of 0.953 and 0.933, and recall values of 0.968 and 0.958 for the positive and negative classes, respectively. Furthermore, the mAP@50 values remained consistent between the validation (0.990, 0.982) and test sets (0.986, 0.965). This consistency across both evaluation sets indicates effective model training and demonstrates reliable detection capabilities for both LAMP reaction outcomes. Furthermore, the model demonstrated high overall precision (0.943) and recall (0.963) on the test set, reinforcing its effectiveness in LAMP image analysis.

thumbnail
Table 1. Performance comparison of the YOLOv8 model between the validation and test sets.

https://doi.org/10.1371/journal.pone.0339042.t001

In order to provide a comprehensive evaluation, the performance of the YOLOv8 model was compared against that of both YOLOv7 and Faster R-CNN within the test set, which comprised 583 instances across the two classes (Table 2). The comparative analysis shows that the YOLOv8 model exceeded both alternative algorithms across all key metrics. Specifically, YOLOv8 attained the highest mAP@50 of 0.975 and an F1-score of 0.958, compared to YOLOv7’s scores of 0.920 and 0.934, and Faster R-CNN’s scores of 0.917 and 0.913, respectively. Furthermore, YOLOv8 exhibited a superior equilibrium between precision (0.943) and recall (0.963).

thumbnail
Table 2. Comparative performance analysis of YOLOv8, YOLOv7, and Faster R-CNN models on the test set for LAMP image classification.

https://doi.org/10.1371/journal.pone.0339042.t002

Fig 3 shows the confusion matrix for our model on the test set. It reveals strong overall performance, with 319 true negatives and 243 true positives out of 583 total instances. The model achieved an accuracy of 97.4%, a precision of 96.4% for positive class detection, and a recall of 95.3%. The model exhibited excellent specificity, correctly identifying 97.3% of negative samples. The model demonstrated a relatively balanced error profile, with a false negative rate of 3.57% and a false positive rate of 3.63%.

thumbnail
Fig 3. Confusion matrix demonstrating high-accuracy classification of LAMP images.

https://doi.org/10.1371/journal.pone.0339042.g003

Fig 4 presents a comprehensive set of evaluation metrics that complement the results shown in Table 1, providing a deeper analysis of our model’s performance. The figure includes precision-confidence (a), recall-confidence (b), precision-recall (c), and F1-confidence (d) curves for both individual classes and overall performance. The precision-recall curve (c) demonstrates the model’s ability to maintain high precision across a wide range of recall values, with mAP@0.5 reaching 0.944 for all classes. This indicates robust performance in balancing false positives and false negatives. The F1-confidence curve (d) shows a peak F1 score of 0.92 at a confidence threshold of 0.436, suggesting an optimal balance between precision and recall at this point. Furthermore, the precision-confidence and recall-confidence curves (a, b) illustrate the model’s high confidence in its predictions, maintaining high precision and recall values across a range of confidence thresholds.

thumbnail
Fig 4. Performance Metrics (a) Precision-Confidence, (b) Recall-Confidence, (c) Precision-Recall, (d) F1-Confidence.

https://doi.org/10.1371/journal.pone.0339042.g004

The high precision and recall values and the confusion matrix results indicate that the model successfully minimized both false positives and false negatives, which is crucial for diagnostic applications. The strong performance of the model across different illumination conditions and color variations, as evidenced by the example images, suggests good generalization capabilities. This is particularly important for LAMP-based diagnostics, where lighting conditions and sample characteristics may vary between laboratories and point-of-care settings. The confidence scores shown in the detection results indicate that the model not only makes correct classifications but also has high certainty, which is essential for reliable automated analysis.

While the model achieved promising results, this study had several limitations and areas for future improvement. A significant limitation of this study is the lack of external validation. Our model was developed using data from our laboratory, utilizing specific smartphones, controlled lighting, and standardized LAMP protocols. We have not assessed the efficacy of our model with data obtained from other laboratories, varying lighting conditions, different reagent batches, or alternative LAMP protocols. Without external validation, we cannot determine whether the model will perform effectively in diverse clinical settings characterized by different equipment, lighting, or operator practices. The performance metrics reported may not accurately depict the model’s performance in real-world clinical environments. Future research should emphasize multi-site validation studies across various laboratories and settings to establish the clinical applicability and reliability of this automated LAMP classification approach.

As a diagnostic method that can be utilized in the diagnosis of severe disease, most notably malaria and tuberculosis, the implementation of AI in the diagnostic workflow for LAMP raises ethical, legal, and social implications that are important to consider [28,29]. One crucial consideration is the fact that many AI approaches, including those utilized in this work, operate as black boxes. Thus, it is unknown what exactly the model is weighing to interpret the image. This means that changes to protocol or differing contexts that a user would not expect to affect the accuracy of the model may actually have large impacts. As a model deployed on a smartphone application, there are further considerations regarding the potential of software issues, potentially arising as unintended consequences of an operating system update (such as how applications interface with the phone camera), causing incorrect diagnosis. There is also a concern that reliance on AI interpretation of the results will degrade the provider’s ability to interpret the results of the test in the absence of the application, and it can be challenging to know how to proceed if the AI and provider disagree on result interpretation. It is also important to consider that many patients may not feel comfortable having a ML algorithm assist in the diagnosis of their disease, and there may be challenges regarding implementation due to public trust and legal disputes over inaccurate results [30].

Materials and methods

YOLOv8 algorithm and architecture

This sub-section provides a brief description of YOLOv8, one of the fastest and most accurate object detection algorithms [31]. The YOLO model architecture is typically structured into three primary components: the backbone, neck, and head, as shown in Fig 5.

thumbnail
Fig 5. The architecture of YOLO.

The input image shows a LAMP test result with fluorescent reactions. The YOLO architecture consists of three main components: (a) the backbone for hierarchical feature extraction at different scales, capturing both fine and coarse details, (b) the neck (green circles with bidirectional dotted arrows) for multi-scale feature aggregation and enhancement. The bidirectional arrows indicate feature fusion and information flow between different scales allowing the model to better understand multi-scale features. (c) the head (blue diamonds) for final predictions: object classification and bounding box regression. Having three heads suggest that predictions are made at three different scales which is typical in YOLO architectures for detecting objects of various sizes. The three rows represent different feature map scales for detecting various object sizes: top(small), middle (medium), and bottom (large).

https://doi.org/10.1371/journal.pone.0339042.g005

The architectural design incorporates an initial input layer that receives an image and a series of convolutional layers that systematically extract relevant information from the image. Each component is crucial in feature extraction, processing, and final output generation.

  • Backbone: consists of convolutional layers that extract features from the input images at varying scales. This component, often a pre-trained CNN like VGG16 or ResNet50, extracts valuable features from the input image. Lower-level features and high-level features are extracted on the shallow and deeper layers.
    • Neck: The neck is an intermediate component connecting the backbone and head. It is responsible for combining feature maps from different backbone layers using modules like Path Aggregation Networks (PAN) or Feature Pyramid Network (FPN).
    • Head: This component is responsible for final predictions; the head generates bounding boxes, objectness scores, and class labels.

Fig 6 shows the model architecture of YOLOv8. Continuous improvements in the three components have significantly enhanced the overall accuracy and speed of the YOLO network. YOLOv8 further revolutionized object detection by adopting an anchor-free approach and refining the grid-based prediction system. It divides the input image into a M × M grid, where each grid cell directly predicts objects. For each detected object, YOLOv8 predicts N values: x, y, w, h, θ, c1…ck. Where, (x, y) represents the center coordinates of the object relative to the grid cell, (w, h) denote the width and height of the object relative to the entire image, θ is the objectness score indicating the model’s confidence in the presence of an object, and (c1, c2, …ck) are the class probabilities for K different classes.

thumbnail
Fig 6. The architecture of YOLOv8 consists of a backbone, neck, and head, adopted from [31].

https://doi.org/10.1371/journal.pone.0339042.g006

The output of YOLOv8 is a tensor of size H × W×(4 + 1 + K), where H × W is the dimensions of the feature maps at each prediction scale, 4 represents the bounding box coordinates, 1 is for the objectness score, and K is the number of classes. YOLOv8 uses the maximum class probability as an indicator of object presence. The model uses a sigmoid function in the output layer as the activation function for class probabilities, representing the likelihood of an object belonging to each possible class [31]. Further, the output of YOLOv8 undergoes post-processing steps to refine predictions: A confidence threshold is applied to filter out low-confidence predictions, and non-maximum suppression (NMS) is used to eliminate redundant detections. YOLOv8’s grid cells efficiently handle tasks related to object localization and classification. The model estimates the probability of an object’s center falling within each grid cell, as formulated by Equation 1:

(1)

where: P(O) is the detection confidence for object O, θ is the objectness score, and IoU(prediction, groundtruth) is the Intersection over Union between the predicted and actual bounding boxes and quantifies the overlap between predicted and ground truth bounding boxes. YOLOv8 uses IoU to determine acceptable detection areas and to make decisions about object localization.

Experimental setup and model training

As a proof of principle, LAMP tests designed to detect antimicrobial resistance genes in bacteria were utilized. Samples consisted of various gram-negative bacteria with extended-spectrum beta-lactamase and carbapenemase genes. These tests were performed as previously described [32]. The tests were performed using an in-house developed device dubbed the White Pearl (Fig 3). This device consisted of three primary modules – (i) Thermal Lysis module: A dedicated heating block maintains a constant temperature of 95°C for initial sample lysis, (ii) Isothermal Amplification module: A separate heating block ensures consistent isothermal amplification at 64°C and (iii) Fluorescence Detection module: A transilluminator equipped with two 2 × 16 Side-mounted blue light emitting diode (LED) arrays, and an absorptive filter of orange acrylic Fig 7.

This work captured 3612 LAMP tube sample images using various Android and iOS smartphones under varied lighting conditions to reflect real-world scenarios. Sample images were captured on the white pearl transilluminator and a commercially available transilluminator (examples are shown in Fig 8). The total sample set was a combined collection of images taken by three different users, with 40% of the samples acquired by one individual and 30% each obtained by the other two individuals. This approach is crucial to train a robust model that can handle diverse image conditions, including colour variations, varied capture angles and inconsistent lighting typically encountered when using smartphone cameras. Manual annotation was performed on each image using the Computer Vision Annotation Tool (CVAT), where rectangular bounding boxes were drawn to delineate and specify individual LAMP sample reactions on each image. The image labels were subsequently saved in.txt format. To guarantee objective labelling and eliminate inter-observer variability, positive and negative classifications were assigned exclusively based on the bacterial isolates’ whole genome sequencing (WGS) results, rather than labels independent of subjective visual assessments, ensuring consistency and reliability in annotation. Sample runs that directly contradicted WGS data (i.e., runs that resulted in no amplification when WGS positive and full amplification when WGS negative) were excluded, as these results would only occur due to procedural, equipment, or human error.

thumbnail
Fig 8. Training dataset examples for YOLOv8 algorithm to detect and classify LAMP assay results.

Images depict 8-tube LAMP strips with positive results fluorescing bright green and negative results as a dull orange. (a) and (c) are images captured on a commercial transilluminator. (b) and (d) are images captured using the white pearl transilluminator.

https://doi.org/10.1371/journal.pone.0339042.g008

Furthermore, before model training, the dataset underwent preprocessing steps, including Min-Max normalization to standardize pixel intensities. This preprocessing step is essential for optimizing model convergence, reducing sensitivity to lighting variations, and enhancing numerical stability in the YOLOv8 model. Following the preprocessing steps, the dataset was partitioned at the level of the LAMP strip to avert data leakage, thereby ensuring that each complete strip (experimental batch) was exclusively allocated to either the training set, the validation set or the test set. The dataset was randomly split into training (70%), validation (15%), and testing (15%) sets at the LAMP strip level using a fixed random seed (seed = 42) to ensure reproducibility. Stratified sampling was implemented to maintain consistent class distributions across all subsets. The training set contained 2544 tube samples with 1120 positive (44%) and 1424 negative (56%) samples. The validation set comprised 485 samples with 198 positive (40.8%) and 287 negative (59.2%) samples. Also, the test set included 583 samples with 252 positive (43.2%) and 331 negative (56.8%) samples.

The model was trained on a machine with an NVIDIA RTX 3060 12GB GPU, Intel i7-10700F CPU operating at 2.90 GHz, 64 GB of RAM, and a Windows 10 64-bit operating system. Python version 3.11 and the PyTorch deep learning framework were used for script execution. The hyperparameter configuration comprised an input image resized to 640 x 640 pixels, a batch size of 8, a learning rate 0.01 using the default cosine annealing schedule, 80 training epochs, and an AdamW optimizer with a weight decay of 0.0005 and a momentum of 0.937.

Evaluation metrics

To validate our model’s performance, we used several established evaluation metrics commonly used in object detection tasks. These metrics, including precision, recall, and F1 score, were used to assess the performance of the trained YOLOv8 in terms of bounding box quality and class label predictions [33]. We used the Precision-Recall curve, a graphical representation of the trade-offs between precision and recall at varied thresholds, to visualize the model’s performance. The Average Precision (AP), which computes the area under this curve, provides a single value to indicate the model’s precision and recall performance. For our multi-object detection scenario, we extended this concept to the Mean Average Precision (mAP), calculating the average AP values across multiple object classes to gain broader insight into the model’s effectiveness. We specifically focused on mAP@50, which calculates mAP at an intersection-over-union (IoU) threshold of 0.50. mAP50 gives more insight into the model’s accuracy for easier-to-detect objects. The IoU metric, measuring the overlap between the model’s predicted bounding box and the ground truth (correct) bounding box, served as a key indicator of localization accuracy. Furthermore, we used mAP@50–95, averaging mAP values across IoU thresholds from 0.50 to 0.95, to provide a more comprehensive view of the model’s performance across varying degrees of localization precision. This multi-faceted evaluation approach allowed us to thoroughly assess our YOLOv8 model’s capabilities in detecting and classifying LAMP assay results.

Conclusions

In conclusion, this study demonstrates the successful application of the YOLOv8 algorithm for automated classification of LAMP reaction results. By leveraging convolutional neural networks and single-pass image processing, our model achieved an impressive overall accuracy of 96.4% in distinguishing between positive and negative samples. The high precision (95.3% for positive, 93.3% for negative) and recall (96.8% for positive, 95.8% for negative) values, along with balanced F1-scores exceeding 95% for both classes, underscore the robustness of our approach.

Our methodology, which incorporated a diverse dataset of smartphone-captured images from both Android and iOS devices, proves the model’s adaptability to real-world conditions. This adaptability, combined with the model’s ability to process images in real-time on edge devices, positions our solution as a valuable tool for point-of-care diagnostics. By automating the interpretation of LAMP results, we have addressed the limitations of manual visual interpretation, potentially enhancing the scalability and reliability of LAMP-based molecular techniques. Furthermore, while the model performed well on the current dataset, future work should evaluate its robustness on a more diverse set of LAMP images through the use of different imaging devices and diagnostic environments.

The integration of deep learning with LAMP technology not only improves assay performance but also opens new avenues for rapid, accurate, and accessible disease diagnosis. As we continue to refine and expand this approach, it holds promise for transforming point-of-care testing practices, particularly in resource-limited settings where rapid and reliable diagnostics are crucial.

Acknowledgments

The authors would like to thank Nabil Royez, Dr. Paul Williamson, and Lisa Oberding from the University of Calgary for scientific and technical assistance.

References

  1. 1. Anklam K, Kulow M, Yamazaki W, Döpfer D. Development of real-time PCR and loop-mediated isothermal amplification (LAMP) assays for the differential detection of digital dermatitis associated treponemes. PLoS One. 2017;12(5):e0178349. pmid:28542573
  2. 2. Bustin SA, Benes V, Garson JA, Hellemans J, Huggett J, Kubista M, et al. The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments. Clin Chem. 2009;55(4):611–22. pmid:19246619
  3. 3. Aydin-Schmidt B, Xu W, González IJ, Polley SD, Bell D, Shakely D, et al. Loop mediated isothermal amplification (LAMP) accurately detects malaria DNA from filter paper blood samples of low density parasitaemias. PLoS One. 2014;9(8):e103905. pmid:25105591
  4. 4. Notomi T, Okayama H, Masubuchi H, Yonekawa T, Watanabe K, Amino N, et al. Loop-mediated isothermal amplification of DNA. Nucleic Acids Res. 2000;28(12):E63. pmid:10871386
  5. 5. Baek YH, Um J, Antigua KJC, Park JH, Kim Y, Oh S, et al. Development of a reverse transcription-loop-mediated isothermal amplification as a rapid early-detection method for novel SARS-CoV-2. Emerg Microbes Infect. 2020;9(1):998–1007.
  6. 6. Soares ARC, Faria VCSd, Avelar DMd. Development and accuracy evaluation of a new loop-mediated isothermal amplification assay targeting the HSP70 gene for the diagnosis of cutaneous leishmaniasis. PLoS One. 2024;19(8):e0306967. pmid:39172895
  7. 7. Yu G, Shen Y, Zhong F, Ye B, Yang J, Chen G. Diagnostic accuracy of the loop-mediated isothermal amplification assay for extrapulmonary tuberculosis: A meta-analysis. PLoS One. 2018;13(6):e0199290. pmid:29944682
  8. 8. Liew MF, Siow WT, MacLaren G, See KC. Preparing for COVID-19: early experience from an intensive care unit in Singapore. Crit Care. 2020;24(1):83.
  9. 9. Yoo WS, Han HS, Kim JG, Kang K, Jeon H-S, Moon J-Y, et al. Development of a tablet PC-based portable device for colorimetric determination of assays including COVID-19 and other pathogenic microorganisms. RSC Adv. 2020;10(54):32946–52. pmid:35516518
  10. 10. Rohaim MA, Clayton E, Sahin I, Vilela J, Khalifa ME, Al-Natour MQ, et al. Artificial Intelligence-Assisted Loop Mediated Isothermal Amplification (AI-LAMP) for Rapid Detection of SARS-CoV-2. Viruses. 2020;12(9).
  11. 11. Song J, Cha B, Moon J, Jang H, Kim S, Jang J, et al. Smartphone-Based SARS-CoV-2 and Variants Detection System using Colorimetric DNAzyme Reaction Triggered by Loop-Mediated Isothermal Amplification (LAMP) with Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR). ACS Nano. 2022;16(7):11300–14. pmid:35735410
  12. 12. Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2016. p. 779–88.
  13. 13. Albawi S, Mohammed TA, Al-Zawi S. Understanding of a convolutional neural network. In: 2017 International Conference on Engineering and Technology (ICET). IEEE; 2017. p. 1–6.
  14. 14. Ragab MG, Abdulkadir SJ, Aziz N. Random Search One Dimensional CNN for Human Activity Recognition. In: 2020 International Conference on Computational Intelligence (ICCI). IEEE; 2020. p. 86–91.
  15. 15. Girshick R, Donahue J, Darrell T, Malik J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2014. p. 580–7.
  16. 16. Girshick R. Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV). IEEE; 2015. p. 1440–8.
  17. 17. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. 2015 Jun 4.
  18. 18. He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. 2017 Mar 20.
  19. 19. Bochkovskiy A, Wang CY, Liao HYM. YOLOv4: Optimal Speed and Accuracy of Object Detection. 2020.
  20. 20. Redmon J, Farhadi A. YOLOv3: An Incremental Improvement. 2018.
  21. 21. Redmon J, Farhadi A. YOLO9000: Better, Faster, Stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2017. p. 6517–25.
  22. 22. Li C, Li L, Jiang H, Weng K, Geng Y, Li L, et al. YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications. 2022.
  23. 23. Wang CY, Bochkovskiy A, Liao HYM. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2022.
  24. 24. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY. SSD: Single Shot MultiBox Detector. 2015.
  25. 25. Liu H, Duan X, Chen H, Lou H, Deng L. DBF‐YOLO:UAVSmall Targets Detection Based on Shallow Feature Fusion. IEEJ Transactions Elec Engng. 2023;18(4):605–12.
  26. 26. Liu H, Sun F, Gu J, Deng L. SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode. Sensors. 2022;22(15):5817.
  27. 27. Reis D, Kupec J, Hong J, Daoudi A. Real-Time Flying Object Detection with YOLOv8. 2023.
  28. 28. Antinori S, Ridolfo AL, Grande R, Galimberti L, Casalini G, Giacomelli A, et al. Loop-mediated isothermal amplification (LAMP) assay for the diagnosis of imported malaria: a narrative review. Infez Med. 2021;29(3):355–65. pmid:35146340
  29. 29. Shete PB, Farr K, Strnad L, Gray CM, Cattamanchi A. Diagnostic accuracy of TB-LAMP for pulmonary tuberculosis: a systematic review and meta-analysis. BMC Infect Dis. 2019;19(1):268. pmid:30890135
  30. 30. Čartolovni A, Tomičić A, Lazić Mosler E. Ethical, legal, and social considerations of AI-based medical decision-support tools: A scoping review. Int J Med Inform. 2022;161:104738. pmid:35299098
  31. 31. Terven J, Córdova-Esparza DM, Romero-González JA. A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Mach Learn Knowl Extr. 2023;5(4):1680–716.
  32. 32. Castellanos L, Chaffee R, Kumar H, Mezgebo BK, Kassau P, Peirano G, et al. A novel machine-learning aided platform for rapid detection of urine ESBLs and carbapenemases (URECA)-LAMP. J Clin Microbiol. 2024.
  33. 33. Padilla R, Passos WL, Dias TLB, Netto SL, da Silva EAB. A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics. 2021;10(3):279.