https://orcid.org/0009-0006-0858-0232
Figures
Abstract
Contrails are line-shaped clouds formed in the exhaust of aircraft engines that significantly contribute to global warming. This paper confidently proposes integrating advanced image segmentation techniques to identify and monitor aircraft contrails to address the challenges associated with climate change. We propose the SegX-Net architecture, a highly efficient and lightweight model that combines the DeepLabV3+, upgraded, and ResNet-101 architectures to achieve superior segmentation accuracy. We evaluated the performance of our model on a comprehensive dataset from Google research and rigorously measured its efficacy with metrics such as IoU, F1 score, Sensitivity and Dice Coefficient. Our results demonstrate that our enhancements have significantly improved the efficacy of the SegX-Net model, with an outstanding IoU score of 98.86% and an impressive F1 score of 99.47%. These results unequivocally demonstrate the potential of image segmentation methods to effectively address and mitigate the impact of air conflict on global warming. Using our proposed SegX-Net architecture, stakeholders in the aviation industry can confidently monitor and mitigate the impact of aircraft shrinkage on the environment, significantly contributing to the global fight against climate change.
Citation: Nobel SMN, Hossain MA, Kabir MM, Mridha MF, Alfarhood S, Safran M (2024) SegX-Net: A novel image segmentation approach for contrail detection using deep learning. PLoS ONE 19(3): e0298160. https://doi.org/10.1371/journal.pone.0298160
Editor: Abul Bashar, Prince Mohammad Bin Fahd University, SAUDI ARABIA
Received: October 6, 2023; Accepted: January 19, 2024; Published: March 5, 2024
Copyright: © 2024 Nobel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming.
Funding: This research is funded by the Researchers Supporting Project Number (RSPD2023R1027), King Saud University, Riyadh, Saudi Arabia. The funder had a major role in study design, data analysis, decision to publish, and preparation and writing/reviewing the manuscript.
Competing interests: NO authors have competing interests.
1 Introduction
The skies above are dynamic ecosystems that react to different natural and artificial influences rather than just being empty canvases for atmospheric occurrences. Aircraft engines are among the latter and have drawn interest because of how they affect the atmosphere. More than half of all aviation’s climate-related emissions come from contrails, which exacerbate the effects of global warming [1]. High-altitude aircraft engines produce exhaust gases that may condense into contrails, observable trails. In the setting of climate change and environmental study, these long, wispy structures have drawn much attention. Contrails are complicated objects with wide-ranging effects; they are not merely ephemeral traces in the sky. On the one hand, they support the Earth’s atmosphere’s radiative forcing, which has a cooling and warming impact. They may deflect sunlight and retain emitted longwave radiation due to their microphysical characteristics and the ice crystals they contain, which can change the planet’s energy balance. Contrails are crucial in affecting climate dynamics, much like their natural counterparts, normal clouds. Contrails also have conflicting impacts, so how they affect the climate must be clarified. In Fig 1, an illustration showcases key components of high-altitude ice cloud formation: engine-emitted water vapor and soot condensing on pre-existing aerosols, resulting in the presence of frozen droplets and contrail ice particles. Due to the existence of ice crystals, they may simultaneously strengthen the greenhouse effect, which traps heat while reflecting sunlight and having cooling effects. Underscoring the complex connection between contrails and climate, research on how these opposing factors balance out is still underway. Scientists are attempting to determine how and to what degree engine designs, various fuels, and atmospheric conditions contribute to climate change in light of the increased air traffic causing a rise in aircraft emissions of contrails during the last two decades [2]. Despite efforts to curb emissions, the stability of these figures over the last decade emphasizes the pressing need for immediate and effective action [3]. Our research is driven by the urgent need to address the environmental consequences of aircraft contrails and their association with emissions. We recognized the need for an innovative approach that transcends conventional methods and harnesses the power of artificial intelligence to achieve exceptional results. Enter SegX-Net, a segmentation architecture tailored for contrail analysis. Unlike traditional approaches, SegX-Net capitalizes on a unique fusion of deep learning techniques. The basis of our research is a crucial environmental concern: the influence of aircraft contrails on climate change. Due to a substantial increase in air traffic, contrails significantly contribute to emissions and global warming. Conventional approaches must be more comprehensive in dealing with the complexities of identifying contrails from satellite images. Therefore, our primary objective is to propose a groundbreaking solution. We selected to modify DeeplabV3+, incorporating ResNet-101 as the backbone. The choice is based on the exceptional capability of ResNet-101 to extract delicate characteristics, which are crucial for deciphering the complicated patterns of contrails in satellite photos. Along with the acclaimed DeeplabV3+, we thoroughly compare well-known models like U-Net, U-Net++, Attention U-Net, Trans U-Net, Res U-Net, and Uc Trans U-Net. The dataset used in this research paper focuses on aircraft contrails, which are clouds of ice crystals formed in aircraft engine exhaust and contribute to global warming by trapping heat in the atmosphere.
The main contributions of the paper include:
- Introduced the SegX-Net architecture, a modified version of DeepLabV3+ with ResNet-101 integration, by customizing the encoder part of the network and leveraging transfer learning, leading to highly accurate and detailed contrail segmentation.
- The research emphasizes the significance of accurate aircraft contrail detection. By providing an advanced image segmentation solution, SegX-Net contributes to the understanding and mitigating contrail-induced environmental impact.
- Through extensive experimentation and evaluation, the study demonstrated that SegX-Net outperforms traditional DeepLabV3+ models with VGG16, VGG19, VIT, Xception, MobileNetV2, ResNet-18, ResNet34, and ResNet101 backbones. Our proposed architecture got exceptional F1-scores and Intersection over Union (IoU) values, indicating its precision, recall, and segmentation quality.
Our research paper contributes to understanding aircraft contrails’ environmental impact and proposes an enhanced architecture for accurate contrail identification through image segmentation. By integrating SegX-Net, we tried to fill a crucial research gap in accurately identifying and analyzing aircraft contrails.
The remainder of the paper is organized as follows. In Section 2, we explore the body of literature in-depth, looking at several image segmentation techniques and their uses. Section 3 describes the dataset description, preprocessing procedures, SegX-Net design, and contrail formation. Analysis of efficiency, insights into parameters, and comparison experiments about assessment measures are discussed in Section 4. Section 5 gives an in-depth discussion of the interpreted findings, including the implications for contrail detection and image recognition of the climate. Finally, Section 6 summarizes our contributions, highlights the importance of SegX-Net, and suggests possibilities for further study.
2 Related works
The foundation of any scientific endeavor lies in building upon the existing body of knowledge. In contrail detection using image segmentation, a thorough exploration of related studies is essential to position our work within the broader landscape. This section presents a comprehensive overview of the relevant literature, from image segmentation techniques to the intricate relationship between contrails and climate impact. By delving into these studies, we gain valuable insights that contribute to the foundation of our novel approach, SegX-Net, designed to transform contrail detection through advanced deep learning techniques. Global warming arises from a mix of natural and human-emitted gases that trap heat, causing the Earth’s temperature to increase. These greenhouse gases, like carbon dioxide and methane, obstruct the average energy radiation balance. In the realm of air transport’s environmental effects, aside from emissions and noise, the impact of contrails on the Earth’s radiation balance is a concerning area that needs more comprehensive understanding and precise data [4]. Previous studies have explored various strategies, such as operational changes in air traffic control, to mitigate contrail-induced greenhouse effects. While some investigations focus on the potential benefits of altering cruise flight levels, others acknowledge the complexity and uncertainty surrounding contrail formation and climate impact, underscoring the ongoing need for comprehensive research in this domain [5]. Identifying contrails in aerial images is a difficult task since they closely resemble natural cirrus clouds and undergo form variations over time [6]. Paoli et al. [7] discovered that in an aircraft regime, at lower temperatures and greater humidity, contrails begin to develop near the engine’s edge. While successful in predicting contrail formation, challenges persist in predicting persistence due to humidity uncertainties. The study delves into predicting contrail formation, persistence, and radiative forcing through aviation weather forecasts. Notably, the paper suggests considering the ambient atmosphere’s dynamics to predict strong contrails. This work contributes vital insights into mitigating aviation’s climate impact and underscores the need for refining contrail prediction methods to combat climate change [8]. An instrumental tool in this endeavor is image segmentation applied to satellite and aerial imagery, enabling the identification of critical climate change contributors like deforestation, urban heat islands, and melting glaciers. Precise segmentation of these regions empowers targeted interventions to mitigate the effects of global warming [9]. U Schumann et al. [10] investigated strategies to mitigate aviation’s climate impact through optimized flight routes considering contrail formation and fuel consumption. It introduces climate-optimized routing using the Contrail Cirrus Simulation Prediction tool (CoCiP) and discusses its potential to reduce global warming effects. The study addresses the radiative forcing of the contrail-induced cirrus cover and highlights the need for further validation and refinement of the CoCiP model for accurate contrail prediction. This research aligns with the broader discourse on aviation’s environmental consequences and emphasizes the significance of route optimization for climate protection. In another case, the paper of [11] comprehensively reviews climate change mitigation strategies, encompassing conventional efforts, negative emissions technologies, and radiative forcing geoengineering. It underscores the insufficiency of conventional mitigation to achieve Paris Agreement targets and explores alternative routes. The study highlights the importance of practical solutions like biogenic-based sequestration techniques, which require policy support, carbon pricing mechanisms, and increased research funding for effective implementation. Also, the study of K Segl et al. [12] introduces a novel approach for detecting small objects in high-resolution satellite imagery by combining supervised shape classification with unsupervised image segmentation iteratively. It emphasizes the significance of shape contrast and object size for accurate detection and discusses potential enhancements through multispectral or hyperspectral imagery. This approach holds practical implications for applications like urban monitoring and vegetation analysis, addressing the challenge of object detection in high-resolution panchromatic satellite images. A pioneering study by JP Hoffman et al. [13] introduces an innovative application of Convolutional Neural Networks (CNNs) in contrail detection within satellite imagery. Repurposing the U-Net architecture, initially developed for detecting sea ice leads, this approach accurately identifies contrails through image segmentation. Furthermore, a cooperative strategy utilizing Fuzzy C-means and Self-Organizing Maps attains high accuracy in segmenting satellite images [14]. Researching global cloud and aerosol properties, radiative energy balance, 3D cloud morphology, and infectious disease risk due to climate fluctuations highlighted the importance of studying contrails’ effects on radiative balance and cloud formation in various contexts. The proposed multistep protocol for contrail detection and segmentation in AVHRR images shows promise in identifying contrail properties yet acknowledges challenges in detecting certain types and improving algorithm precision [15]. The study also revealed that converting the RGB color space to HSV enhanced segmenting satellite images, indicating the practical utility of color space transformations in this context [16]. The paper by Andre L. Barbieri et al. [17]presents an entropy-based image segmentation method for color images from Google Earth, enabling automated monitoring of ecological and geographical changes. It underscores the significance of color information for precise segmentation and highlights applications in disaster mapping, climate change monitoring, and ecological studies. The approach’s potential for improvement via window size adjustments and complementary statistical measures is also emphasized. An automatic algorithm combining watershed segmentation and region merging demonstrates promising performance on Google Earth images [18]. Studies revealed that mathematical morphology and watershed transformation algorithms offer segmentation benefits, yet they grapple with challenges such as over-segmentation and computational complexity [19]. B Dezso et al. [20] presents a comprehensive review of graph-based image segmentation methods applied to satellite image classification. It evaluates four algorithms and discusses their theoretical foundations, implementation details, and potential improvements. The study highlights the significance of image segmentation in remote sensing for land cover identification and suggests avenues for future research to enhance algorithm performance and practical application. Leveraging geostationary satellite imagery, weather data and air traffic information, these studies offer insights into contrail evolution and climate impact. Deep learning techniques like instance segmentation are explored for efficient detection. Integration of multiple observation methods and identification of contrail-producing aircraft contribute to advancing contrail research for climate validation and modeling improvement [21]. Recently, there has been a significant increase in research efforts to improve image segmentation methods by using deep learning techniques. This is due to the impressive performance of deep learning models in visual tasks [22, 23].
Advancements in image processing have led to the emergence of image segmentation techniques, prominently in medical and satellite imagery domains. This paper introduces and evaluates three methods for satellite image segmentation: K-means Clustering, Thresholding Technique, and Active contour. By assessing their performance using parameters like Segmentation Accuracy and Correlation Ratio, the study aids in identifying practical options for satellite image analysis. The proposed Active Contours technique exhibits promising results, highlighting its potential for real-world implementation [24]. In the context of advancing semantic segmentation techniques, this study of [25] introduces an innovative algorithm addressing accuracy and object boundary segmentation challenges. The algorithm demonstrates improved performance in segmenting high-resolution images by leveraging multi-level cascading residual structures and multiple loss function constraints. While the experimental results on Cityscapes and CamVid datasets are promising, comprehensive analysis of its applicability, comparative assessments, and real-world implications remain avenues for future exploration. Recently, McCloskey et al. [26] conducted research and provided a restricted set of human-labeled Landsat photographs for the scientific community. In another recent study by Ng et al. [27], a comprehensive effort was made to create an open dataset for contrails observed over the United States. This was achieved by utilizing satellite footage from the GOES-16 satellite.
The subsequent unveiling of SegX-Net’s architecture, training process, and evaluation metrics in this paper is a significant step towards comprehending and mitigating contrail impacts on global warming, revolutionizing climate research. This paper critically reviews image segmentation methods to identify the most suitable techniques for contrail identification. Thus, we propose using an innovative AI-driven technique called SegX-Net for accurate contrail identification. Through this approach, we aim to revolutionize climate research by providing a powerful tool for monitoring and mitigating the effects of aircraft contrails on our environment.
3 Methodology
The research methodology is organized into interconnected phases, illustrated in Fig 2, to enhance contrail detection efficiency through the innovative SegX-Net architecture. It begins with meticulous data preprocessing and splitting into training and validation subsets. The dataset is split into training and valid subsets, enabling unbiased SegX-Net assessment. Crucial backbone network models, including VGG16 [28], VGG19 [29], VIT [30], Xception [31], ResNet18 [32], ResNet34 [33], ResNet101 [34] and MobileNetv2 [35], establish the foundation for SegX-Net.
Simultaneously, the architecture and backbones analyze the training subset, refining image segmentation precision. The subsequent phase involves comprehensive performance evaluation, leading to a succinct comparison table showcasing the contrail detection prowess of each backbone under SegX-Net. This methodical progression harnesses SegX-Net’s power to redefine contrail detection precision, contributing to advancements in this vital field. UNet and DeepLab are top-rated models in the field of aerial photography due to their exceptional performance in image segmentation tasks and their ability to effectively include both global and local information [6]. We used the DeepLabV3+ architecture as our segmentation network during this study. An improvement on the Deeplabv3 architecture, Deeplabv3+ has a more streamlined and effective decoding module for improving semantic segmentation performance and refining feature information [36]. When it comes to segmentation, Deeplabv3+ outperforms the previous Deeplab-series networks [37]. DeepLabV3+ is a state-of-the-art model for image segmentation tasks that has demonstrated excellent performance and accuracy in segmenting images. This architecture incorporates a powerful encoder-decoder structure, atrous convolutions, and skip connections to capture multi-scale contextual information and achieve precise segmentation results. By leveraging DeepLabV3+, we aimed to enhance the accuracy and effectiveness of our image segmentation tasks and achieve high-quality segmentation outputs, thus using a different architecture named SegX-Net. The network details are illustrated in Fig 5. The input images used in the present study are high-resolution satellite photographs that depict real-world circumstances, including prominent aircraft contrails. Segmentation aims to accurately detect and analyze these contrails in the photos, enabling a thorough investigation of their distribution and contributing to a full comprehension of their environmental influence, particularly in climate change.
3.1 Dataset description
In this study, we used a dataset obtained from Kaggle [38]. From there, we took 18000 images, which were divided into two subsets: a training set and a validation set. These images were sourced from the GOES-16 Advanced Baseline Imager (ABI) [39], and access to the original data was facilitated through Google Cloud Storage. The technical specifications of the ABI sensor, including resolution and spectral bands, are explicitly detailed to offer readers comprehensive insights into the data source. To adapt the full-disk images, bilinear resampling was applied, resulting in localized scene images. The training set comprised 14,400 images, accounting for 80 percent of the entire dataset, while the validation set contained 3,600 images, making up the remaining 20 percent. This division was crucial to ensure the effectiveness and generalizability of our model’s performance. The training set played a vital role in training our model, enabling it to learn and extract meaningful features from a diverse range of images. We also tested our model on 1000 new images that were not used during training and validation. This helps ensure the reliability and generalization capability of our model. In the dataset section, we present a diverse set of input images that play a pivotal role in our research on contrail detection, is shown in Fig 3. These images encompass various elements, including false color representations, ground truth contrail masks, and overlays of contrail masks on false color images. The false color images provide unique visual insights, showcasing essential spectral information for our analysis. Meanwhile, the ground truth contrail masks offer precise outlines of contrail regions, serving as valuable reference data for model evaluation and training. We comprehensively understand contrail distribution and spatial relationships within the original scenes by superimposing contrail masks on false color images. By utilizing such a substantial portion of the dataset for training, we optimized the model’s parameters and refined its segmentation predictions, ultimately enhancing its accuracy and performance. On the other hand, the valid set served as an independent and unseen dataset, used explicitly to evaluate our trained model’s generalization capabilities.
By assessing the model’s performance on this valid set, we obtained an unbiased estimation of its effectiveness in segmenting new, previously unseen images. The distribution of images in our training dataset shown in Fig 4 holds significant insights into the prevalence of contrail and non-contrail images. The first pie chart reveals a nuanced balance, with approximately 45.8% of images portraying contrails and 54.2% depicting scenes without contrails. This equilibrium underscores the diverse nature of our dataset, ensuring a comprehensive representation of real-world scenarios. Moreover, the second pie chart further illuminates this distribution, highlighting 29.4% percent contrail images and 70.6% percent non-contrail images. These proportions emphasize the prominence of non-contrail instances and reinforce the complexity of our task, as the model must discern and accurately segment contrails within a predominantly non-contrail context.
3.2 Data pre-processing
The image data underwent several crucial preprocessing steps in the initial training and validation preparation stages. Firstly, the images were converted into an array format with dimensions of 256x256, taking into account the presence of 3 color channels. The dataset was partitioned to ensure an unbiased evaluation, with 80% of the data being used for training and the remaining 20% for validation. One of the most essential steps in the preprocessing pipeline was image normalization, which aimed to establish consistency in pixel values across the dataset. The standard methods used included scaling the pixel values to a specific range, such as dividing them by the maximum value or applying a zero-mean normalization technique by subtracting the mean and dividing by the standard deviation. Implementing these preprocessing steps significantly improved the model’s resilience and adaptability, allowing it to handle various inputs and perform effectively on unseen data. It was crucial to consistently apply the same preprocessing techniques to both the training and validation sets to ensure fairness in evaluation and accurate performance assessment.
3.3 Proposed architecture
In this work, we proposed an enhanced architecture by integrating ResNet-101 as the backbone network into the DeepLabV3+ model [31]. The decision to incorporate ResNet-101 into our model is crucial for enhancing segmentation performance in our particular environment. The deep architecture and skip connections of ResNet-101 are crucial for effectively analyzing the delicate features in satellite photos and understanding the complicated patterns of contrails. Our model’s properties enable it to accurately record and transmit detailed information, which is particularly important for identifying and describing the unique and often subtle aspects of aircraft contrails in the images. The architecture’s profound and sophisticated feature extraction processes make it a prudent choice specifically designed to meet the requirements of our contrail detection assignment, thereby improving the model’s ability to segment objects accurately. Our proposed architecture is shown in Fig 5. Our research’s encoder component of DeepLabV3+ utilizes ResNet-101 as the backbone network. By incorporating ResNet-101, our model can leverage its capabilities to extract high-level abstract information from the input image [40]. The input image is passed through convolutional blocks in ResNet-101, which perform successive convolutions and pooling operations to extract hierarchical features at different levels of abstraction. The convolutional layers capture local patterns and spatial information, while the pooling layers downsample the feature maps while retaining important features. The decoding part of DeepLabV3+ focuses on refining and enhancing the features obtained from the encoder. The shallow features from the encoder are merged with the upsampled deep features, and convolutional operations are applied to refine the feature details. This refinement process aims to improve the segmentation predictions by enhancing the feature representation. Finally, the refined features’ resolution is restored through bilinear upsampling, resulting in the final segmentation map. Fig 5 visually represents the intricate structure of the network model described in this article. The figure highlights distinct elements, with the blue and yellow sections signifying 1x1 convolutions and 3x3 convolutions, respectively. Additionally, the figure demonstrates the utilization of maximum pooling and up-sampling techniques. During the encoding phase, the network performs through a series of operations, including 1x1 convolutions, which has channels of 64, three sets of 3x3 convolutions, which have 128 channels rate 6, 256 channels rate 12, 256 channels rate 18 respectively, and also maximum pooling. These operations collectively enable the network to extract and capture meaningful features from the input data. In the decoding phase, a single set of 1x1 convolutions is employed to restore the original size of the feature map. Following this, a combination of up-sampling and 3x3 convolutions is employed to generate the final prediction image, accomplishing the image segmentation task. The figure includes annotations that specify the layer names, output feature map sizes, and the corresponding operations involved, such as Conv for convolution, image pooling for maximum pooling, and upsample for up-sampling.
3.4 Residual module block
The Deep Residual Network, proposed by [41], introduced the concept of residual learning. The residual refers to the discrepancy between the observed and estimated values. Assuming the input to the network is denoted as x and the expected mapping as M(x), the network mapping can be reformulated as the residual, represented by F(x), as shown in Eq (1).
(1)
Here, x represents the characteristic mapping of the upper layer network, F(x) denotes the residual of the current layer and M(x) represents the observed value at that layer, forming the relationship depicted in Eq (2).
(2)
(3)
Although both M(x) and F(x) + x yield the same effect, optimizing F(x) is simpler compared to optimizing M(x). By considering the relationships between different layers, expressed in Eqs (3) and (4), the residual F(xN) is added to the previous layer’s output xN to obtain the output of the current layer, xN + 1.
(4)
To ensure optimal network performance, it is crucial to strike a balance in terms of network depth. While a certain depth may lead to the best model performance and lowest loss, further increasing the network depth could potentially result in network degradation. To address this, the concept of the residual network is introduced, enabling the residual F(x) to approach zero and maintaining the network in an optimal state.
3.5 ResNet-101 encoder
In the SegX-Net architecture, ResNet-101 serves as an essential encoder, contributing to the network’s ability to capture intricate features from input images effectively shown in Fig 6. ResNet-101, introduced by [42], is a deep convolutional neural network that introduces a novel approach to addressing the vanishing gradient problem in deep neural networks. ResNet-101, a variant comprising 101 layers, employs residual blocks that facilitate extracting meaningful features. Notably, in the SegX-Net framework, ResNet-101 is not a feature extractor but an integral component for enhancing the network’s segmentation capability. ResNet-101’s architecture is characterized by its depth and structure, involving stacked residual blocks. Each residual block incorporates convolutional layers and shortcut connections bypassing certain layers. This design allows the network to adjust input features directly and learn more abstract features through subsequent layers. The architecture also integrates bottleneck structures to optimize computational efficiency by reducing convolutional layer complexities. These bottlenecks employ 1x1, 3x3, and 1x1 convolutional layers to selectively decrease input and output channels. Additionally, ResNet-101 includes global average pooling and fully connected layers at the end, culminating in a final classification or segmentation output. Although not used for feature extraction in SegX-Net, ResNet-101’s depth and shortcut connections contribute to its proficiency as an encoder in the network’s image segmentation process.
3.6 Architectural framework and comparative backbone models
During this study, we harnessed the power of transfer learning to enhance the performance of our image segmentation model. After a thorough evaluation of VGG16, VGG19, VIT, Xception, MobileNet_V2, ResNet18, ResNet34, and ResNet101, we found that ResNet101 exhibited superior performance and accuracy, prompting its seamless integration as an encoder into our segmentation architecture. Fine-tuning the model on our dataset allowed it to adapt to our requirements while retaining essential learned features. Adopting transfer learning addressed data limitations and expedited training, leading to improved results and heightened accuracy in our image segmentation. This integration empowered our model to capture meaningful patterns effectively, making it a pivotal element in elevating our image segmentation research’s overall performance and effectiveness. This adaptation aimed to balance capturing detailed features and maintaining computational efficiency. By integrating ResNet-101 into SegX-Net, we harnessed the enhanced capabilities of ResNet-101 while preserving the efficient encoding-decoding architecture of DeepLabV3+. This integration successfully improved segmentation accuracy, as evidenced by our experimental results. In summary, the integration of ResNet-101 into SegX-Net represents a strategic enhancement to the original architecture of DeepLabV3+. This integration allowed us to leverage cutting-edge techniques, resulting in superior segmentation performance.
In this research, we benchmarked the performance of SegX-Net against several top image segmentation architectures to conduct a thorough comparison analysis. This comprised well-known models, including DeeplabV3+, Attention U-Net, Trans U-Net, Res U-Net, U-Net++, and U-Net. We sought to identify SegX-Net’s advantages relative to these well-known architectures by comparing these models across many essential performance measures, including IoU and F1 scores. By comparing SegX-Net with cutting-edge segmentation frameworks, this comparative method shows the exceptional contributions of SegX-Net and offers insightful information about how well it detects contrails.
3.7 Enhanced SegX-Net model
In this work, we proposed an enhanced version of the DeepLabV3+ model by modifying its encoder part with ResNet-101 shown in Fig 7. Specifically, we focused on improving the initial process by changing the first two blocks out of the original five blocks. In the first block, we replaced the 1x1 convolution followed by 256-neuron ReLU activation with a 1x1 convolution followed by 64-neuron ReLU activation—this modification aimed to reduce the dimensionality of the feature maps while preserving the relevant information. Similarly, in the second block, we replaced the 3x3 convolution followed by 256-neuron ReLU activation with a 3x3 convolution followed by 128-neuron ReLU activation, maintaining the same atrous rate of 6. This adjustment allowed for a more fine-grained feature representation at the atrous rate. We observed that these modifications in the encoder part yielded improved results compared to the original DeepLabV3+ model. The enhanced model exhibited enhanced segmentation accuracy and detail preservation, improving overall performance. Fig 7 compares the original DeepLabV3+ encoder part and our modified version. The figure highlights the changes made in the first two blocks, illustrating the altered architecture and visually representing the improved process. We conducted comprehensive evaluation experiments to quantify the performance improvement using various evaluation metrics, including IoU and F1 scores. The results demonstrate that our enhanced model outperforms the original DeepLabV3+ model, achieving higher accuracy and better segmentation results. These findings highlight the efficacy of the proposed modifications in the encoder part of DeepLabV3+ with ResNet-101 and their positive impact on the model’s overall performance. The following section presents a detailed analysis and comparison of the performance between the original DeepLabV3+ model and our enhanced version, further substantiating the superiority of the modified architecture.
4 Experimental results
In this research, we presented the results of our experiments evaluating the performance of the SegX-Net model for image segmentation in addressing contrail detection challenges. The dataset focused on contrails, obtained from NOAA GOES-16, was preprocessed by transforming images into 256x256 arrays with 3 color channels. Evaluation metrics included Intersection over Union (IoU), F1 score and Dice Loss as the loss function during training.
4.1 Evaluation metrics
To evaluate the performance of each model and effectively assess its learning capabilities, this experiment employed multiple control parameter variables for evaluation. The primary evaluation metrics included the F1 score and Intersection over Union (IoU).
4.1.1 IoU.
When it comes to measuring the accuracy of image segmentation, the Intersection over Union (IoU) is widely regarded as a representative evaluation metric.
(5)
IoU quantifies the overlap between the predicted values generated by the model and the true values represented by the sample labels. It provides a measure of the alignment and agreement between the predicted and ground truth segmentation masks, reflecting the accuracy and quality of the segmentation results.
4.1.2 F1 score.
Additionally, the F1 score provides a balanced measure of precision and recall, considering both the true positives and false positives in the segmentation predictions. Together, the F1 score and IoU offer comprehensive insights into the model’s performance and effectiveness in image segmentation tasks.
(6)
4.1.3 Dice loss.
The Dice Loss was used in during the research as a loss function during the training of our image segmentation model. Its utilization served two primary purposes: to guide the model’s optimization process and to align the predicted segmentation masks with the ground truth masks.
(7)
The Dice Loss was an integral part of our research, serving as a key component in optimizing the model’s performance and enhancing the accuracy and quality of our image segmentation results. Its usage helped align our model’s predictions with the ground truth, ultimately leading to improved segmentation accuracy and precise delineation of objects in the resulting segmentation masks. In the equation. Xi indicates the predicted target category and yi indicates the actual target category.
4.1.4 Dice coefficient.
Within the realm of image segmentation, where computers manipulate pixels and reality wields the brush, the Dice coefficient arises as a reliable measure of precision. It functions as a measurement tool to assess the degree of overlap between a predicted area, such as a tumor or a contrail, and the actual region. The correlation between the Dice score and the fit is directly proportional, indicating a stronger alignment between prediction and reality as the Dice score increases.
(8)
4.1.5 Sensitivity analysis.
Sensitivity in image segmentation, measures the algorithm’s ability to correctly identify positive instances, crucial in applications like medical imaging. A sensitivity value close to 1 indicates effective detection of relevant regions, while lower values suggest potential misses. Sensitivity analysis involves varying parameters to understand how the algorithm responds to changes, aiding optimization for specific applications.
(9)
4.2 Efficiency analysis and comparison
An essential aspect of evaluating the SegX-Net architecture’s effectiveness is assessing its computational efficiency in contrast to existing models. This analysis sheds light on the architectural optimizations that SegX-Net introduces. Notably, a comparative examination of parameter sizes between SegX-Net and the default DeepLabV3+ model with ResNet-101 showcases SegX-Net’s streamlined design, boasting a parameter count of 33,815,745 as opposed to the default model’s 35,278,721. This reduction in parameter count signifies SegX-Net’s potential for optimized memory usage, as evident by its memory size of 129 MB compared to DeepLabV3+ with ResNet101’s 135 MB. Furthermore, when considering time complexity, SegX-Net demonstrates its efficiency by achieving an average processing time of 694ms per iteration, iterated ten times on average, while the default DeepLabV3+ with ResNet-101 requires 901ms. This rigorous analysis underscores SegX-Net’s computational superiority, making it a promising solution for enhancing contrail detection efficiency and outperforming existing models in terms of both memory utilization and processing speed.
4.3 Parameter analysis
For evaluating the model, we employ key metrics such as Intersection over Union (IoU), F1-score and Dice Loss, Dice Coefficient and sensitivity analysis. The results are presented through insightful line plots, illustrating the Dice Loss, IoU and F1-score, Dice Coefficient and sensitivity analysis for both the training and valid sets over epochs. We have showcased the quantitative results obtained from the training and valid sets.
Fig 8 presents the F1 score plot, which balances precision and recall, reflecting the model’s segmentation performance.
Fig 9, displays the IoU plot, highlighting the model’s accuracy in capturing object boundaries and overall segmentation quality.
Fig 10, illustrates the Dice Loss plot, providing insights into the model’s optimization process and alignment with ground truth masks.
The effectiveness of SegX-Net is shown in Fig 11 the use of dice coefficient performance assessment.
The results of the sensitivity study shown in Fig 12 revealed that SegX-Net had an extraordinary true positive detection rate and a low number of false negatives, which confirmed its effectiveness in picture segmentation work.
4.4 Comparative experiment
In the results section, our experiments with the SegX-Net model unveil remarkable outcomes, showcasing an exceptional F1 score of 99.47% (Table 1) and an impressive IoU score of 98.86% and (Table 2). These scores, complemented by the Dice Loss plot (Fig 10) offering insights into the model’s optimization, the captivating IoU plot (Fig 8) demonstrating its unprecedented accuracy and the striking F1 score plot (Fig 9), collectively attest to the revolutionary capabilities of SegX-Net. Additionally, we have introduced Dice Coefficient (Fig 11) and Sensitivity Analysis (Fig 12), further solidifying the model’s exceptional segmentation performance. These extraordinary achievements establish SegX-Net as a pioneering solution, advancing contrail detection through cutting-edge artificial intelligence. The remarkable sensitivity, notably during training with 98.82%, shown in (Table 2), that the model can recognize most contrails in images. As we balance sensitivity and accuracy, we must consider environmental monitoring’s contrail detection goals. Although our SegX-Net achieves a slightly lower Dice Coefficient score on the training set compared to VGG16 (71.79% vs. 72.61%), it is crucial to take into account the larger context. Our model exhibits improved performance compared to VGG16, as shown by a Dice Coefficient score of 73.55% on the validation set. The validation set serves as a crucial indicator of generalization. This review highlights the strong and flexible nature of SegX-Net in accurately capturing the complex patterns of contrails. Although training performance is important, the validation results emphasize the model’s effectiveness in real-world situations, confirming its status as an advanced solution for contrail identification.
Here, we have shown a comprehensive analysis of the performance metrics for each model on both Intersection over Union (IoU), Dice-Coefficient in Table 1 and F1-score, Sensitivity Score in Table 2.
Table 1 provides an in-depth comparison of the IoU scores obtained by the different models on both the training and valid sets. The results clearly illustrate the exceptional segmentation accuracy achieved by SegX-Net, with an outstanding IoU score of 98.86% on the training set and an impressive 99.54% on the valid set. This demonstrates the model’s ability to accurately capture object boundaries and produce high-quality segmentations, outperforming all other models, including DeepLabV3+ with VGG16, VGG19, VIT, Xception, MobileNet_V2, ResNet18, ResNet34 and ResNet101.
In Table 2, we delve into the F1-scores of each model on the training and valid sets, shedding light on their segmentation performance in finer detail. The results reveal the exceptional capabilities of SegX-Net, achieving a remarkable F1-score of 99.47% on the training set and an extraordinary 99.77% on the valid set. These scores surpass the performance of DeepLabV3+ with VGG16, VGG19, VIT, Xception, MobileNet_V2, ResNet18, ResNet34 and ResNet101 by a substantial margin. Such exceptional F1-scores indicate the model’s precision and recall capabilities, affirming its ability to achieve superior segmentation results.
Our SegX-Net model significantly outperforms these models in terms of Intersection over Union (IoU) scores when compared to a group of cutting-edge competitors shown in Table 3, including U-Net, U-Net++, Attention U-Net, Trans U-Net, Res U-Net, Uc Trans U-Net, and DeeplabV3+. The results demonstrate our proposed SegX-Net’s better accuracy and efficiency in separating contrail clouds. In climate research, aviation studies, and environmental monitoring, where the precision of contrail cloud identification is crucial, SegX-Net demonstrates a greater capacity to properly identify contrail areas with a higher IoU score. As a result, it is a useful tool in these fields.
In parallel evaluations of the same models, we focus on F1 scores, a statistic that balances accuracy and recall in contrail cloud segmentation. SegX-Net is the clear winner once again as shown in Table 4, outperforming U-Net, U-Net++, Attention U-Net, Trans U-Net, Res U-Net, Uc Trans U-Net, and DeeplabV3+. SegX-Net’s superior F1 scores highlight its remarkable accuracy and recall skills in recognizing contrail clouds. These findings highlight the resilience of our proposed model, establishing it as a top option for accurate contrail cloud segmentation tasks, while its strong F1 scores indicate its use in aviation, and environmental impact assessments.
These comparative results are a testament to the revolutionary advancements achieved by SegX-Net in image segmentation for contrail detection. Our model has achieved outstanding performance with a remarkable IoU score of 97.90%, an F1 score of 99.51%, a Dice coefficient of 71.79%, and an impressive sensitivity of 98.41% from the testing set. These results are a testament to the reliability and accuracy of our model. The model’s extraordinary performance and accuracy set new standards in artificial intelligence and contribute significantly to addressing the challenges of climate change. The combination of state-of-the-art techniques and innovative methodologies within SegX-Net positions it as a leading solution for climate researchers and environmentalists seeking accurate and reliable image segmentation results. We present compelling prediction images from our test set, as illustrated in Fig 13. These images include Ash color, Ground truth, Prediction, and Contrail mask on Ash color. The false-color representations vividly capture essential spectral information, enhancing our analysis. The ground truth contrail masks offer precise outlines for rigorous model evaluation, serving as crucial reference data. These prediction images vividly demonstrate our contrail detection methods’ exceptional performance and accuracy, emphasizing our dedication to addressing challenges through innovative and unconventional AI approaches.
The false colour images offer vivid visual representations, capturing essential spectral information relevant to our analysis. The ground truth contrail masks serve as crucial reference data, providing precise outlines of contrail regions for rigorous model evaluation and validation. Furthermore, by superimposing the contrail masks on false colour images, we gain valuable insights into the spatial distribution and correlations of contrails within their original scenes. These prediction images showcase the remarkable performance and accuracy of our contrail detection methods, underscoring our commitment to addressing challenges through innovative and non-typical AI approaches.
5 Discussion
Results of our experiment emphasize the influence of enhanced segmentation methods in identifying contrails. The SegX-Net model, carefully developed and evaluated, exhibited exceptional performance metrics, attaining an IoU score of 98.86% and 99.54% and an F1-Score of 99.47% and 99.77% on the training and validation sets, respectively. The Dice Coefficient and Sensitivity scores serve to highlight the model’s reliability, in addition to the aforementioned measures. Our model has exceptional segmentation accuracy and sensitivity to complex contrail patterns, as seen by its Dice Coefficient scores of 71.79% on training and 73.55% on validation, along with a Sensitivity score of 98.82% on training.
When comparing SegX-Net to well-known models like U-Net, U-Net++, Attention U-Net, Trans U-Net, Res U-Net, Uc Trans U-Net, and DeeplabV3+, SegX-Net demonstrates superiority. Our algorithm frequently surpassed these standards, showcasing superior accuracy in contrail segmentation. This comparison confirms the effectiveness of SegX-Net and offers valuable insights into the model’s competitive advantage. SegX-Net stands out among contrail identification technologies because of its exceptional ability to identify subtle characteristics in contrails and its outstanding accuracy metrics.
In the future, it would be beneficial to investigate the scalability and generalizability of SegX-Net. Expanding the dataset to include a broader range of geographical regions and weather conditions could strengthen the model’s resilience. Moreover, the incorporation of sophisticated metrics and the investigation of adversarial training methods, such as Generative Adversarial Networks (GANs), show potential for enhancing the precision of the model. These considerations broaden the range of situations where SegX-Net can be used and contribute to the ongoing discussion on advanced methods for detecting contrails, which has significant consequences for environmental preservation.
6 Conclusions
In conclusion, this research paper has explored the integration of image segmentation techniques, specifically the SegX-Net architecture, to address climate change challenges related to aircraft contrails. By leveraging the DeepLabV3+ model as the baseline and integrating ResNet101 as the backbone network, we have demonstrated the effectiveness of the enhanced model in accurately identifying and segmenting contrails. The proposed modifications in the encoder part, along with transfer learning and data preprocessing techniques, have resulted in improved segmentation accuracy and detail preservation. The evaluation metrics, including IoU and F1 score, validate the superior performance of the SegX-Net architecture compared to the original DeepLabV3+ model. Overall, this research highlights the potential of image segmentation techniques in understanding and mitigating the environmental impact of aircraft contrails, contributing to the broader goal of combating climate change. Future work can focus on further refining the SegX-Net architecture and exploring additional applications of image segmentation in addressing climate change challenges.
References
- 1.
Beth Timmins. Contrails: How tweaking flight plans can help the climate [Internet]. BBC NEWS Business: 22 October 2021. Available from: https://www.bbc.com/news/business-58769351
- 2. Paoli R, Shariff K. Contrail modeling and simulation. Annual Review of Fluid Mechanics. 2016 Jan 8;48.
- 3.
Arabella Ruiz. 45 Carbon Dioxide, Greenhouse Gas, Climate Change Statistics [Internet]. Statistics: October 16, 2023. Available from: https://theroundup.org/co2-greenhouse-gas-emission-statistics/
- 4.
Sharma K, Jain S, Wu E, Fattah ZM, Sarin C, Maeshiro D, et al. Utilizing Computer Vision Algorithms to Detect Contrails. InAIAA AVIATION 2023 Forum 2023 (p. 3769).
- 5. Avila D, Sherry L, Thompson T. Reducing global warming by airline contrail avoidance: A case study of annual benefits for the contiguous United States. Transportation Research Interdisciplinary Perspectives. 2019 Sep 1; 2:100033.
- 6.
Wang Z. Combining UPerNet and ConvNeXt for Contrails Identification to reduce Global Warming. arXiv preprint arXiv:2310.04808. 2023 Oct 7.
- 7. Paoli R, Helie J, Poinsot T. Contrail formation in aircraft wakes. Journal of Fluid Mechanics. 2004 Mar;502:361–73.
- 8. Gierens K, Matthes S, Rohs S. How well can persistent contrails be predicted? Aerospace. 2020 Dec 2;7(12):169.
- 9. Hoegh-Guldberg O, Jacob D, Taylor M, Guillén Bolaños T, Bindi M, Brown S, et al. The human imperative of stabilizing global climate change at 1.5 C. Science. 2019 Sep 20;365(6459): eaaw6974. pmid:31604209
- 10.
Schumann U, Graf K, Mannstein H. Potential to reduce the climate impact of aviation by flight level changes. In3rd AIAA atmospheric space environments conference 2011 Jun 14 (p. 3376).
- 11. >Fawzy S, Osman AI, Doran J, Rooney DW. Strategies for mitigation of climate change: a review. Environmental Chemistry Letters. 2020 Nov; 18:2069–94.
- 12. Segl K, Kaufmann H. Detection of small objects from high-resolution panchromatic satellite imagery based on supervised image segmentation. IEEE Transactions on geoscience and remote sensing. 2001 Sep;39(9):2080–3.
- 13. Hoffman JP, Rahmes TF, Wimmers AJ, Feltz WF. The Application of a Convolutional Neural Network for the Detection of Contrails in Satellite Imagery. Remote Sensing. 2023 May 31;15(11):2854.
- 14. Awad M, Chehdi K, Nasri A. Multi-component image segmentation using a hybrid dynamic genetic algorithm and fuzzy C-means. IET image processing. 2009 Apr 1;3(2):52–62.
- 15. Weiss JM, Christopher SA, Welch RM. Automatic contrail detection and segmentation. IEEE transactions on geoscience and remote sensing. 1998 Sep;36(5):1609–19.
- 16.
Ganesan P, Rajini V. Assessment of satellite image segmentation in RGB and HSV color space using image quality measures. In2014 International conference on advances in electrical engineering (ICAEE) 2014 Jan 9 (pp. 1–5). IEEE.
- 17. Barbieri AL, De Arruda GF, Rodrigues FA, Bruno OM, da Fontoura Costa L. An entropy-based approach to automatic image segmentation of satellite images. Physica A: Statistical Mechanics and its Applications. 2011 Feb 1;390(3):512–8.
- 18.
Sahin K, Ulusoy I. Automatic multi-scale segmentation of high spatial resolution satellite images using watersheds. In2013 IEEE International Geoscience and Remote Sensing Symposium-IGARSS 2013 Jul 21 (pp. 2505–2508). IEEE.
- 19. Belaid LJ, Mourou W. Image segmentation: a watershed transformation algorithm. Image Analysis & Stereology. 2009;28(2):93–102.
- 20. Dezso B, Giachetta R, László I, Fekete I. Experimental study on graph-based image segmentation methods in the classification of satellite images. EARSeL eProceedings. 2012;11(1):12–24.
- 21. Chevallier R, Shapiro M, Engberg Z, Soler M, Delahaye D. Linear contrails detection, tracking and matching with aircraft using geostationary satellite and air traffic data. Aerospace. 2023 Jun 21;10(7):578.
- 22.
Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z, et al. Resnest: Split-attention networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition 2022 (pp. 2736–2746).
- 23.
Tan M, Le Q. Efficientnet: Rethinking model scaling for convolutional neural networks. InInternational conference on machine learning 2019 May 24 (pp. 6105–6114). PMLR.
- 24.
Deepika NP, Vishnu K. Different techniques for satellite image segmentation. In2015 Online international conference on green engineering and technologies (IC-GET) 2015 Nov 27 (pp. 1–6). IEEE.
- 25. Wang Y, Wang C, Wu H, Chen P. An improved Deeplabv3+ semantic segmentation algorithm with multiple loss constraints. Plos one. 2022 Jan 19;17(1): e0261582. pmid:35045083
- 26.
McCloskey KJ, Geraedts SD, Jackman BH, Meijer VR, Brand EW, Fork DK, et al. A human-labeled Landsat contrails dataset.
- 27.
Ng JY, McCloskey K, Cui J, Brand E, Sarna A, Goyal N, et al. OpenContrails: Benchmarking Contrail Detection on GOES-16 ABI. arXiv preprint arXiv:2304.02122. 2023 Apr 4.
- 28.
Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. 2014 Sep 4.
- 29.
Yu W, Yang K, Bai Y, Xiao T, Yao H, Rui Y. Visualizing and comparing AlexNet and VGG using deconvolutional layers. InProceedings of the 33 rd International Conference on Machine Learning 2016 Jun 21.
- 30.
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020 Oct 22.
- 31.
Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. InProceedings of the European conference on computer vision (ECCV) 2018 (pp. 801–818).
- 32.
Wang C, Ni P, Cao M. Research on crop disease recognition based on Multi-Branch ResNet-18. InJournal of Physics: Conference Series 2021 Jul 1 (Vol. 1961, No. 1, p. 012009). IOP Publishing.
- 33. Gao M, Qi D, Mu H, Chen J. A transfer residual neural network based on ResNet-34 for detection of wood knot defects. Forests. 2021 Feb 11;12(2):212.
- 34. Yu B, Yang L, Chen F. Semantic segmentation for high spatial resolution remote sensing images based on convolution neural network and pyramid pooling module. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2018 Aug 8;11(9):3252–61.
- 35.
Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC. Mobilenetv2: Inverted residuals and linear bottlenecks. InProceedings of the IEEE conference on computer vision and pattern recognition 2018 (pp. 4510–4520).
- 36. Wang J, Zhang X, Yan T, Tan A. DPNet: Dual-Pyramid Semantic Segmentation Network Based on Improved Deeplabv3 Plus. Electronics. 2023 Jul 21;12(14):3161.
- 37. Li Q, Kong Y. An Improved SAR Image Semantic Segmentation Deeplabv3+ Network Based on the Feature Post-Processing Module. Remote Sensing. 2023 Apr 19;15(8):2153.
- 38.
Joe Ng, Carl Elkin, Aaron Sarna, Walter Reade, Maggie Demkin. (2023). Google Research—Identify Contrails to Reduce Global Warming. Kaggle. https://kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming
- 39.
GEOSTATIONARY OPERATIONAL ENVIRONMENTAL SATELLITES—R SERIES-A collaborative NOAA & NASA program. INSTRUMENTS: ADVANCED BASELINE IMAGER (ABI)[Internet]. NASA: April 2021. Available from: https://www.goes-r.gov/spacesegment/abi.html
- 40.
Ma C, Ji Z, Gao M. Neural style transfer improves 3D cardiovascular MR image segmentation on inconsistent data. InMedical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22 2019 (pp. 128–136). Springer International Publishing.
- 41. Fan Z, Liu Y, Xia M, Hou J, Yan F, Zang Q. ResAt-UNet: A U-Shaped Network Using ResNet and Attention Module for Image Segmentation of Urban Buildings. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2023 Jan 23; 16:2094–111.
- 42.
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition 2016 (pp. 770–778).
- 43.
Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 2015 (pp. 234–241). Springer International Publishing.
- 44.
Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. Unet++: A nested u-net architecture for medical image segmentation. InDeep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4 2018 (pp. 3–11). Springer International Publishing.
- 45.
Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999. 2018 Apr 11.
- 46.
Xiao X, Lian S, Luo Z, Li S. Weighted res-unet for high-quality retina vessel segmentation. In2018 9th international conference on information technology in medicine and education (ITME) 2018 Oct 19 (pp. 327–331). IEEE.
- 47.
Wang H, Cao P, Wang J, Zaiane OR. Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer. InProceedings of the AAAI conference on artificial intelligence 2022 Jun 28 (Vol. 36, No. 3, pp. 2441–2449).
- 48. Baheti B, Innani S, Gajre S, Talbar S. Semantic scene segmentation in unstructured environment with modified DeepLabV3+. Pattern Recognition Letters. 2020 Oct 1; 138:223–9.