Experimental mares and stables.

The data for this study were collected from a sizeable equine farm in Ordos, Inner Mongolia. This farm is a professional institution engaged in the breeding, reproducing, and cultivating of equines, boasting significant practical value and credibility. The data were gathered from six parturition areas of the farm, including three outdoor parturition areas, each measuring 12  ×  5.4 meters (L  ×  W), accommodating 1 to 4 mares to move freely and complete parturition, and three indoor stables, each spanning 4  ×  3.2 meters (L  ×  W), each capable of accommodating one mare to move freely and complete parturition. All parturition areas are equipped with iron fences 1.2 meters high and feature individual feed troughs and water troughs to satisfy the mares’ needs for foraging and drinking. Mares nearing their expected parturition dates are dynamically moved to the parturition areas and parturition stables.

Parturition video collection.

Visible light video images were captured using the spherical camera DS-2DC4223IW-D (Hikvision, Hangzhou, China). The camera’s field of view was 57.6° (wide-angle) to 2.7° (telephoto), with controllable pan-tilt rotation and zoom, a horizontal range of 360°, and a vertical range of -15° to 90°, supporting 23x optical zoom and 16x digital zoom. The captured visible light videos had a resolution of 1920×1080, a bitrate of 4096Kbps, and a frame rate of 25fps and were encoded in H.264. The camera automatically switched to night mode under low-light conditions, capturing video in black and white tones. The night mode automatically activates infrared fill light, with a fill light distance of up to 15 meters. Cameras in the outdoor parturition area were mounted at a corner of the prepartum area, 2.7 meters above the ground, and cameras in the parturition room were fixed to the side walls at a height of 2 meters. All cameras automatically transmitted video data to the mare farm’s NVR DS-7932N-R4 (Hikvision, Hangzhou, China). Within the local area network, laptops installed with iVMS-4200 (V3.5.0.7) were used to control the camera angles and download video data, as illustrated in Fig 1. During non-rainy and non-snowy weather, some mares are transferred to outdoor birthing areas during the day, while all mares are moved back to the indoor stables at night. Therefore, the collected parturition data includes footage from the outdoor birthing area during the day on non-rainy and non-snowy days, as well as footage from the indoor stables during both the day and night. Configuration was done for four experimental parturition area cameras and the control room NVR, adopting a 24-hour continuous video recording scheme. Data collection lasted for two years and three months, and with the assistance of the on-site production director and veterinarians, 152 mare parturition behaviors were collected. All video files were stored in mp4 format, with the filenames based on the recording time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Parturition data collection.

https://doi.org/10.1371/journal.pone.0318498.g001

Data preprocessing.

The mare parturition process has three stages: early uterine contractions, fetal delivery, and placental expulsion. This paper’s research is confined to the visual identification of the fetal delivery stage, starting from when the elevated white fetal sac becomes visible at the mare’s posterior to the point where the foal’s body is completely separated from the mare. The visible images of this process are defined as mare parturition (MP), while images where this process is not visible are defined as mare non-parturition (MNP). In this paper, one parturition process of the data sample is defined as the period from 15 minutes before the fetal sac becomes apparent to the completion of parturition.

Excluding unusable files for reasons like camera soiling, 146 video files were obtained. To avoid overfitting issues caused by high similarity between consecutive or adjacent frames in the same video and to acquire effective image data, the video files were processed as follows:

1. (1) From these, 66 longer video files were selected for extracting images to be used as training data, and the remaining 80 shorter videos were reserved for subsequent testing.
2. (2) Due to the high similarity between adjacent frames, high-quality images were extracted from the videos using FFMPEG (V3.3.9) (with the quality parameter set to 1 for the highest quality output), saving one frame out of every ten as an image file.
3. (3) All the image files output from the previous step were compared for similarity using the SSIM algorithm[33], retaining only one image from those with a similarity greater than 0.76 to minimize the similarity between adjacent images.
4. (4) A manual selection process was implemented to eliminate anomalous images caused by obstructions, overexposure, or excessive blurring due to camera focus issues.
5. (5) To minimize training costs while preserving parturition features, this study employed a Python script to resize all image files generated in the previous step from 1920×1080 to 384×216 pixels and saved them as new image files.

After processing, a total of 5680 usable image files were obtained. As data augmentation will be applied to the training set but not to the test set, the augmented training data will be expanded to twice the size of the original dataset. Consequently, the training and test sets will be randomly divided in a 2:1 ratio. The training set thus consists of 3787 image files, while the test set comprises 1893 image files.

Data labelling and enhancement.

The open-source image annotation tool Labelme (version v5.1.1, licensed under the MIT Free Software License) was used to annotate all image files, with the final annotation results converted into the COCO dataset format[34]. The specific labeling standards are as follows:

1. (1) The entire body of the mare is taken as the input for object detection;
2. (2) The bounding boxes are tightly fitted around the mares;
3. (3) If multiple mare targets appear within an area, each target must be annotated.

This study employed the Albumentations library to apply random data augmentation to all training set data, incorporating three augmentation methods: random cropping of images (maximum height of 14, minimum height of 8, maximum width of 14, minimum width of 8), brightness and contrast enhancement (brightness limit of 0, contrast limit of 0 to 1), and image blurring (lower limit of 0.1, upper limit of 0.3). After data augmentation, the number of images in the training set expanded to 7574, with 1893 images in the test set. These augmentation methods can simulate issues such as small object occlusion, excessive image exposure, and camera zooming encountered in reality, thereby increasing the richness of the training samples, enhancing the diversity of the images, and improving the model’s generalization ability.

Fig 2 displays a comparison of images before and after data augmentation. It is evident that, following data augmentation, the complexity and diversity of the images have been significantly enhanced, and the effects of occlusion, brightness, contrast, and blurriness within the images contribute to improving the model’s accuracy and robustness, allowing it to better adapt to real-world scenarios and application demands. After data preprocessing, annotation, and augmentation, the final dataset for this study was formed, with the composition of the dataset as shown in Table 1, consisting of 7574 images in the training set and 1893 images in the test set, including a total of 5198 annotations of parturition mares and 11813 annotations of non-parturition mares. As the data collection process yielded far more instances of mares in a non-parturition state than in parturition, despite manual screening during the data preprocessing phase, an imbalance in dataset categories still exists, necessitating the use of specific model training strategies or improvement methods to ensure the model’s capability to identify both categories accurately.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Display of image enhancement effect.

https://doi.org/10.1371/journal.pone.0318498.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Equine parturition dataset.

https://doi.org/10.1371/journal.pone.0313772.t001

Establishment of datasets.

After data preprocessing, annotation, and augmentation, the final dataset for this study was formed, as shown in Table 1. The training set consists of 7,574 images, and the test set consists of 1,893 images, with 5,198 annotations for mares during parturition and 11,813 annotations for mares not in parturition. To better reflect typical scenarios in the actual production environment, the dataset includes various environmental conditions such as indoor and outdoor parturition areas, natural light, and artificial lighting, ensuring the diversity of the data. Due to the predominance of non-parturition states over parturition states during data collection, despite manual screening in the data preprocessing stage, there still exists an imbalance in dataset categories that requires specific model training strategies or improvement methods to ensure the model’s ability to recognize both categories accurately.

The process of mare parturition primarily involves two postures: standing and lying down. To assess the model’s generalization ability across different postures, we classified the test set into standing parturition, lateral recumbent parturition, standing non-parturition, and lateral recumbent non-parturition, with the number of images for each type as shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Test set labelling details.

https://doi.org/10.1371/journal.pone.0313772.t002

The backbone network with integrated attention modules.

In the backbone network design, to enhance the effectiveness and robustness of feature extraction, this study integrates three attention mechanism modules into ResNet101: CBAM (Convolutional Block Attention Module), SE (Squeeze-and-Excitation), and GCA (Global Context Attention). These modules are used to enhance channel attention, spatial attention, and global context modeling, respectively, thereby improving the model’s ability to capture key features in complex scenes. CBAM applies a channel attention module to weigh the importance of features in the feature map, while the spatial attention module focuses on the spatial distribution of the target regions. Woo et al. demonstrated the exceptional performance of the CBAM module in object detection tasks, showing that its lightweight design significantly improves detection accuracy without substantially increasing computational complexity. The SE module, through Squeeze and Excitation operations, recalibrates global features to enhance the expression of salient features. Hu et al. pointed out that the application of the SE module in various convolutional neural networks significantly improves performance in classification and object detection, especially when handling high-resolution images [35]. The GCA module, on the other hand, constructs global features through context-aware mechanisms, enhancing the model’s feature representation in complex backgrounds. Cao et al. introduced the GCA module, which effectively suppresses background noise while improving the quality of the target region’s features, making it especially suitable for object detection tasks in dynamic scenes [36].

In this study, we specifically integrated the attention module into the advanced stages of ResNet101, specifically adding attention module at the end of each residual unit in the third and fourth stages. The purpose of this design is to leverage the feature extraction capability of the attention module for deep optimization and recalibration of high-level features rich in semantic information, thereby effectively enhancing the ability to parse complex semantic features and ultimately improving the model’s capability to capture key visual information related to mare parturition and overall object detection performance.

As shown in Fig 3, the CBAM module is divided into two parts: the channel and spatial attention modules. The channel attention module first processes the input feature map, then the output feature map is multiplied by the input feature map before being fed into the spatial attention module for processing. After processing by the spatial attention module, a similar operation is performed, and the final output feature map is obtained, with its mathematical expression as Eq (1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. CBAM module architecture.

The upper part of the CBAM architecture consists of two components: the channel attention module and the spatial attention module. These modules are applied to the input feature map, and the output is obtained by element-wise multiplication with the input feature map. The lower part shows the specific placement of the CBAM module in ResNet101, where it is added at the end of each residual unit in the third and fourth stages.

https://doi.org/10.1371/journal.pone.0318498.g003

(1)

Where F is the input feature map, is the feature map of the output of the channel attention module, is the input of the spatial attention module, is the feature map of the output of the spatial attention module, is the final feature map of the output of the CBAM module, and  ⊗  denotes element-by-element multiplication.

The SE attention module (Squeeze-and-Excitation Networks) aims to enhance the feature representation capability by adaptively adjusting the channel-wise feature weights. It also offers good computational efficiency, making it suitable for multi-scenario adaptability requirements. In this study, the SE attention module is introduced for ablation comparison. The design of the SE attention module consists of two main steps: the Squeeze operation and the Excitation operation, as shown in Eq (2) and Eq (3).

In the Squeeze operation, the input feature map , where C, H, and W represent the number of channels, height, and width, respectively, the feature map is first compressed using global average pooling. This operation aggregates spatial information across all locations in each channel, resulting in a channel-wise weight vector .

(2)

Where represents the global feature representation of the c-th channel.

In the Excitation opWeration, the compressed feature z undergoes a nonlinear transformation through two fully connected layers, generating the channel weights , and recalibrating the features of each channel.

(3)

Where , and are the weight matrices, r is the scaling factor, usually set to 16; δ ( ⋅ )  is the ReLU activation function, and σ ( ⋅ )  is the Sigmoid function.

Finally, the channel weight s is applied to each channel of the input feature map X, with its mathematical expression as Eq (4).

(4)

The SE module effectively enhances significant features and suppresses redundant information, demonstrating good performance in multiple tasks.

The GCA (Global Context Attention) module enhances the feature representation of the target region through global context modeling while suppressing background noise. Its core lies in the computation of contextual information and the adaptive adjustment of weights, as described in Eq (5) and Eq (6).

For context feature computation, given the input feature map , where C, H, and W represent the number of channels, height, and width respectively, the global context representation is calculated by performing a weighted sum over the spatial dimensions.

(5)

Where is the normalized weight that indicates the importance of the feature at position  ( i , j ) .

In context feature enhancement, the global context representation g is processed by two fully connected layers to generate the channel weights .

(6)

Where and are the weight matrices, with r being the scale ratio (typically set to 16); δ ( ⋅ )  is the ReLU activation function, and σ ( ⋅ )  is the Sigmoid function.

Finally, the context weight w is applied to each channel of the input feature map X. with its mathematical expression as Eq (7).

(7)

The GCA module’s approach to contextual information modeling is particularly effective for scenes with complex backgrounds and dynamically changing targets, demonstrating significant performance improvements in video object detection tasks.

CARAFE-BFP module.

The Feature Pyramid Network (FPN)[37], as an architecture to improve multi-scale object detection performance, achieves effective integration of features across various scales through a top-down pyramid structure, overcoming the limitations of using single-scale feature maps to adapt to targets of varying sizes. To address the potential blurring and information loss during the upsampling process in FPN, CARAFE FPN[38] utilizes a content-aware feature reassembly approach, dynamically adjusting the upsampling kernels to enhance the accuracy of upsampling and the quality of feature maps. The Balanced Feature Pyramid (BFP) aims to resolve the issue of quality imbalance among features of different scales in FPN. An optimized feature fusion mechanism achieves more balanced information transfer between layers, enhancing detection capabilities for targets of various sizes.

This paper introduces the CARAFE operator on top of BFP to construct the CARAFE-BFP module based on the Libra R-CNN algorithm, aimed at mitigating the effects of imbalance between positive and negative samples in mare parturition detection while enhancing detection accuracy. As shown in Fig 4, C2 to C5 represents feature maps of different levels obtained from ResNet101, with varying spatial resolutions and semantic richness. By processing these feature maps with the CARAFE module, fusion, and upsampling, we obtain P2’ to P6’. Subsequently, through feature integration (Integrate) and feature refinement (Refine), new feature layers P2P5 are obtained. Integrate merges features P2’ to P5’ from different levels through a specific strategy to integrate multi-scale information. Refine further optimizes and adjusts features based on feature integration, improving features’ representational capability and quality to ensure that the combined features more effectively support subsequent detection tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. CARAFE-BFP Module.

The input feature maps come from the outputs of the FPN, namely P2’, P3’, P4’, and P5’. These feature maps are processed through the Integrate and Refine operations to generate new feature maps P2, P3, P4, and P5, which are used for subsequent object detection tasks.

https://doi.org/10.1371/journal.pone.0318498.g004

GRoIE region of interest extraction module.

In object detection algorithms, extracting Regions of Interest (RoI) is crucial for locating and identifying targets within images. To enhance the representation of fine-grained features and improve detection performance in multi-scale scenarios, this study introduces a novel RoI extraction module, GRoIE (Generic RoI Extractor). The module aims to optimize the multi-scale feature fusion capability during the feature extraction process, particularly for recognizing mare parturition behavior across different scales in complex scenes.

As illustrated in Fig 5, the GRoIE module achieves RoI extraction and enhancement through four steps: RoI Align, Preprocessing phase, Aggregation function, and Post-processing phase. This is important for enhancing object detection tasks in complex scenarios such as mare parturition recognition. Firstly, the RoI Align step extracts features corresponding to the prediction boxes from the feature map through a precise spatial sampling strategy. This study adjusts the output of RoI Align[39] from 7×7 to 14×14, resulting in a higher spatial resolution to ensure the preservation of fine details of mare parturition, laying the foundation for subsequent feature analysis. The Preprocessing phase employs convolution operations to process these features further, enhancing the representation of local details and providing richer information for feature integration. The Aggregation function is responsible for merging features from different sources into a unified feature representation through weighted summation, optimizing information fusion at different scales, and enhancing the model’s adaptability to changes in target states. Lastly, the Post-processing phase introduces an attention mechanism, emphasizing features crucial for parturition state recognition while suppressing background noise, significantly improving the accuracy and robustness of mare parturition recognition.Overall, the GRoIE module achieves efficient multi-scale feature fusion through a staged design. By integrating attention mechanisms, it enhances the selective representation of key features and optimizes the fine-grained processing capability of object detection. This provides comprehensive support for specific target detection in complex scenes and offers a technical guarantee for mare parturition recognition in challenging environments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. GRoIE Module.

https://doi.org/10.1371/journal.pone.0318498.g005

L-MPD network architecture.

This study introduces an L-MPD network model based on an improved Libra R-CNN for target detection of mare parturition. Fig 6 displays the structure of the L-MPD network, comprising four parts: (1) Backbone network utilizing ResNet101, with CBAM attention modules added in Stages 3 and 4 of ResNet101 to extract image features. (2) Employing CARAFE-BFP for further multi-scale feature extraction. (3) Sending the output of CARAFE-BFP to the Region Proposal Network (RPN) to generate region proposals. (4) Based on an improved RoI Extractor (Generic RoI Extractor), integrating all feature maps input from the CARAFE-BFP module for RoI extraction, allowing the extractor to adapt to targets of varying scales and sizes, and handling RoI feature extraction for multiple regions, using RoIAlign to extract features from each proposal box, ultimately performing classification and bounding box regression for parturition and non-parturition of mares.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. L-MPD network architecture.

https://doi.org/10.1371/journal.pone.0318498.g006

The backbone network extracts features from the input images for feature map-based target detection. In this study, ResNet101 is employed as the backbone network to extract deep semantic information, with cross-layer connections added in each residual block to address the vanishing gradient and degradation problems during deep neural network training. Additionally, ResNet101 introduces some optimization strategies, such as the Bottleneck structure and Pre-activation module, to improve model performance and reduce computational cost. Its residual structure can be represented by Eq (8). The ReLU activation function is described by Eq (9), and the batch normalization function[40] by Eq (10).

(8)

In the Eq (8), and denote the inputs and outputs of the lth layer of the network, respectively; F denotes a residual block consisting of several convolutions, an activation function (ReLU), and a batch normalization operation, is a learnable parameter in the residual block, and denotes a weight in the cross-layer connection. In particular, when the input and output dimensions are different, the dimensionality can be adapted by adding a 1 × 1 convolutional layer. For the ReLU activation function, its formula can be expressed as Eq (9).

(9)

The batch normalization function can be expressed as Eq (10).

(10)

In the Eq (10), BN(x) denotes the result after batch normalization of the input x, μ and σ denote the mean and variance of the current batch, respectively, γ and β denote the learnable scaling factor and offset, respectively, and β is a minimal number that avoids the denominator to be zero.

Instead of the original FPN network, this paper uses CARAFE-BFP further to enhance the feature extraction capability of the model. CARAFE is a lightweight generalized upsampling operator that uses simple interpolation and reassembles the feature vectors according to their similarity to improve the quality of the feature maps when performing the upsampling operation. Its formula can be expressed as Eq (11).

(11)

Where x denotes the input feature map, Conv denotes the convolution operation, and α and ψ denote the position weights and feature reconstruction vectors, respectively. Specifically, denotes the contribution of position  ( m , n )  to position  ( i , j ) , which can be calculated by the normalized softmax function.

In the RoI feature extraction part of this paper, RoI is extracted from the feature maps outputted from each layer of the CARAFE FPN network, and RoIAlign is used instead of RoIPool to reduce the target location bias of the target detection results. Because the mare partitioning features are much smaller than the mare’s share in the picture, the RoIAlign result is increased from the original 7  ×  7 to 14  ×  14 to detect the mare partitioning behaviours better. Then, the RoI features in the same region are summed and fused to make the model better adapted to the multi-scale target detection task.