General monocular depth estimation techniques

Monocular depth estimation has emerged as a practical alternative to stereo vision and LiDAR, offering reduced complexity and lower cost. With the integration of deep learning, the accuracy and efficiency of these methods have improved considerably, enabling their use in applications such as pothole detection that demand scalable and affordable sensing technologies. Earlier studies primarily relied on handcrafted descriptors and conventional vision techniques, whereas modern approaches increasingly utilise data-driven models and innovative training strategies.

Deep learning has been a major driver of recent progress in monocular depth estimation. Godard et al. (2019) presented Monodepth2, a self-supervised approach that exploits stereo image pairs together with photometric consistency loss, thereby eliminating the need for ground-truth depth data. Their framework achieved competitive results on the KITTI benchmark and became a cornerstone for research in unsupervised depth estimation.

Supervised techniques have also advanced the field. Davydov et al. (2022) introduced a ConvNeXt-based model tailored for object-aware depth prediction, incorporating specialized loss formulations and resolution-sensitive mechanisms to enhance accuracy in focused tasks. In parallel, Khan et al. (2022) conducted a comprehensive review of depth completion approaches, which integrate RGB images with sparse depth maps. Their survey underscored the role of fusion methods and highlighted performance gains on datasets such as KITTI and NYUv2, measured through metrics including RMSE and REL. The use of multi-task learning has further enriched depth prediction pipelines. Qi et al. (2018) proposed GeoNet, which simultaneously estimates depth and surface normals while enforcing geometric constraints through a joint loss. This design fosters cross-task refinement and improves predictive accuracy. Likewise, Zhu et al. (2019) investigated monocular estimation of object-to-camera distances, employing CNNs and Bird’s Eye View projections to enhance the localization of far and off-center objects — a capability particularly valuable in autonomous driving contexts.

To improve robustness and generalisation, Ranftl et al. (2021) presented MiDaS, a model trained on diverse datasets that produces scale-invariant relative depth maps. This model excels in cross-domain settings, making it highly versatile for a wide range of applications. Building upon these advances, Dai et al. (2023) introduced transformer-based architectures that capture global spatial context more effectively, outperforming traditional CNNs on challenging outdoor depth datasets.

For refinement of depth predictions, Cheng et al. (2018) introduced the DYSPN spatial propagation network. This method adaptively learns local spatial affinities to refine initial depth estimates, boosting the recovery of fine surface details. Additionally, generative models such as Pix2Pix and CycleGAN have been applied to depth synthesis tasks, where Pix2Pix handles paired data and CycleGAN enables domain translation with unpaired data. These approaches have expanded the possibilities for depth generation from RGB images in scenarios where direct depth measurements are unavailable.

Pothole-specific depth estimation and detection approaches

Research in pothole detection and depth estimation addresses several unique challenges, including the need for real-time processing, robustness to variable lighting and weather conditions, and the requirement for accurate depth measurements within localised areas. Multiple key studies have contributed to advancing this field.

Yurdakul and Tasdemir [23] extended the YOLOv8 framework for real-time pothole detection by utilizing an RGB-D dataset captured with Intel RealSense sensors. Their design incorporates lightweight DSConv modules, resulting in improvements in both recall and precision. In contrast to vision-only strategies, Nawale et al. [24] proposed PotholeGuard. This LiDAR-driven segmentation method employs adaptive K-Nearest Neighbours refinement to mitigate the effects of uneven point cloud densities, thereby addressing the shortcomings commonly observed in purely 2D image-based techniques.

Li et al. [20] proposed a hybrid deep network that jointly processes RGB images and 3D scanner depth maps for simultaneous pothole segmentation and depth prediction. Despite demonstrating superior accuracy, their method depends on expensive and complex hardware setups, which limit practical scalability. Similarly, Ahmed and Raza [21] combined MiDaS depth estimation with YOLOv8 in a two-stage pipeline, achieving accurate detection and estimation results but introducing additional system complexity and latency.

Several other studies have explored pothole detection and depth estimation from different perspectives. Javed [25] introduced a unified deep learning framework for pavement assessment that achieved effective segmentation, although depth accuracy was not comprehensively evaluated. Kumar and Singh [26] emphasized the significance of model robustness under diverse lighting and weather variations; however, the absence of detailed depth metric validation limited its practical relevance for prioritizing road repairs. Similarly, Faizan [27] presented a monocular depth estimation approach designed for pothole detection, contributing insights into training strategies but without addressing real-time deployment constraints.

Mehta and Sharma [28] combined pothole detection with depth-based visual cues to estimate repair costs; however, their reliance on handcrafted features limited the model’s adaptability to varied pavement conditions. Patel and Zhao [29] provided a survey of vision-driven pothole detection techniques using RGB imagery, where they highlighted dataset limitations and the difficulties of domain adaptation as key barriers. Zhao et al. [30] introduced a multi-task learning framework that simultaneously addressed pothole detection and depth estimation, showing that the joint optimization of both tasks yielded notable accuracy improvements. Chen et al. [31] proposed a lightweight CNN architecture for mobile and real-time road inspection, which aligns closely with smartphone-oriented depth estimation approaches such as our own. Taken together, these works demonstrate the variety of strategies employed in pothole detection and depth estimation, while also underscoring the persistent challenges in balancing precision, scalability, and real-time applicability for real-world deployment.

As summarized in Table 1, current methods for pothole detection and depth estimation face critical challenges spanning hardware, algorithmic design, and deployment feasibility. Many approaches rely on costly or highly specialised sensing systems, deliver only relative rather than absolute depth, particularly within pothole regions, or lack robustness across diverse lighting, weather, and surface conditions. Furthermore, several employ computationally demanding, multi-stage processing pipelines that are unsuitable for real-time execution on mobile or edge devices. Collectively, these drawbacks highlight the need for a scalable, cost-effective, and robust solution that can produce accurate, metric-scale pothole depth measurements using widely available mobile hardware. In response, the following section presents our proposed framework, which directly addresses these issues through an integrated monocular RGB–LiDAR–deep learning archi_

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Limitations of related work in depth estimation and pothole detection.

https://doi.org/10.1371/journal.pone.0344604.t001

Dataset preparation

To support pothole segmentation and depth estimation from monocular RGB images, we developed a custom dataset tailored for real-world road conditions. Data collection was performed using an iPhone 14 Pro, leveraging its high-resolution camera coupled with integrated LiDAR sensor capabilities. This setup enabled the simultaneous acquisition of detailed road surface imagery and accurate depth information, ensuring alignment between visual and spatial data.

The dataset was gathered across multiple locations within Lahore, including Walton Road, Airport Road, and Manawan. These diverse sites were selected to capture variations in road surface types, lighting conditions, and pothole characteristics, thereby enhancing the dataset’s representativeness and robustness.

Dataset collection process

The dataset comprises nearly 25,000 images collected across diverse environmental settings, including variations in lighting, weather conditions, and traffic situations. Unlike datasets that rely on fixed-distance and fixed-angle captures [32], images were recorded at varying distances and perspectives to reflect better the variability encountered during real-world driving. For each RGB image, the iPhone’s LiDAR sensor simultaneously recorded a corresponding depth map, providing accurate ground truth depth data essential for model training and evaluation. This pairing of RGB and depth information facilitates precise supervision in both pothole segmentation and depth estimation tasks (Fig 1).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. The iOS application displays RGB imagery alongside depth maps generated using LiDAR sensors.

https://doi.org/10.1371/journal.pone.0344604.g001

LiDAR equipment specifications

The LiDAR sensor embedded in the iPhone 14 Pro offers convenient and relatively precise depth measurements, making it well-suited for road surface analysis. However, it is essential to acknowledge certain hardware limitations that may influence depth accuracy. The sensor achieves a horizontal accuracy of about 3–5 meters at a distance of 15 feet, with precision gradually decreasing for objects positioned farther away. Its effective operating range is best suited for short- to mid-range measurements, generally below 5 meters. Performance may vary depending on surface characteristics; highly reflective, glossy, or dark materials—such as wet road surfaces or shaded regions—can lead to reduced measurement reliability. Furthermore, environmental factors like intense sunlight, fog, or rain can degrade LiDAR readings. Nonetheless, our study found that minor lighting variations did not significantly affect the sensor’s performance [33].

Despite these constraints, the LiDAR depth measurements proved sufficiently accurate and stable for generating ground truth data in short-range road surface inspections, such as pothole analysis. The dataset compiled for this research encompasses a broad spectrum of environmental and contextual conditions to ensure robustness and validity. Regarding environmental factors, 60% of the images were captured during daylight hours, 30% during twilight (dawn or dusk), and 10% at night. Dry road surfaces accounted for 75% of the data, while wet conditions following rainfall comprised the remaining 25%.

To emulate a variety of vehicle configurations, cameras were mounted at heights between 0.5 and 4 meters, corresponding to different vehicle profiles. The pitch angle was adjusted from 15° to 75°, allowing coverage of both close-range surfaces and distant road features. In terms of geographic diversity, 60% of the images originated from urban streets such as Walton Road and Manawan, 25% were recorded on highways including Airport Road, and 15% came from rural road segments around the outskirts of Manawan and adjacent areas. This broad range of conditions contributes to the dataset’s representativeness and reliability for training and evaluating depth estimation and pothole detection models.

Dataset overview

The dataset comprises nearly 25,000 manually gathered RGB images, providing extensive coverage of diverse real-world road environments. Example images are shown in Fig 2. It is organized into two main categories:

• Pothole-containing road samples: Images capturing road surfaces with potholes of different depths, shapes, and severity levels, collected under diverse surface textures, illumination conditions, and camera perspectives.
• Plain (undamaged) road images: Images of smooth road surfaces free from visible defects, serving to balance the dataset and enhance the model’s ability to differentiate between damaged and undamaged areas.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Example images from the custom pothole dataset collected across different road segments in Lahore.

https://doi.org/10.1371/journal.pone.0344604.g002

The dataset is evenly balanced, with a 50:50 split between pothole-affected and undamaged road images, which supports accurate classification and segmentation performance in both cases. Each data sample in the curated dataset consists of a high-resolution RGB image together with a corresponding depth map generated by the iPhone 14 Pro LiDAR sensor. Selected samples further include segmentation masks that precisely delineate pothole regions, coupled with metadata that records capture details such as location, timestamp, and prevailing environmental conditions. This comprehensive dataset supports a wide range of deep learning tasks, enabling experiments in monocular depth estimation, semantic segmentation [34], and binary classification to distinguish between deteriorated and well-maintained road surfaces.

Ground truth validation

To confirm the accuracy and robustness of the depth information in our dataset, we designed a dual-stage validation protocol. This method combined direct physical depth measurements with stratification in different environmental settings, allowing us to validate the consistency of annotations and assess the capacity of models trained in this dataset to generalize effectively.

The first stage involved direct physical measurements of pothole depths using a laser rangefinder with a precision of ±1 mm. We selected 500 representative potholes and took measurements at five key points on each pothole surface: the center, top-left, top-right, bottom-left, and bottom-right. This multi-point approach captured the spatial variability of pothole depths, providing a detailed and accurate ground truth reference. Fig 3 illustrates the geometric principle applied for physical validation. Each measurement was modelled as a right-angled triangle, where the camera-to-road vertical distance represented the vertical distance (AB), the horizontal offset to the pothole served as the base (BC), and the hypotenuse (AC) was measured directly. This configuration enabled accurate cross-validation of calculated and annotated values, thereby ensuring the reliability of depth estimation for each pothole instance.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Triangular representation used for ground truth validation.

The hypotenuse (D) was physically measured to verify pothole depth estimates, with base (B) and depth (H) forming a right-angled triangle.

https://doi.org/10.1371/journal.pone.0344604.g003

In the second stage, we stratified the measurements across varying lighting conditions to evaluate model performance in realistic scenarios. Specifically, data acquisition was distributed as 200 samples during daylight, 200 samples at twilight (dusk or dawn), and 100 samples at nighttime. This environmental stratification facilitates thorough testing of depth prediction robustness across challenging illumination conditions frequently encountered in real-world road monitoring. Through this comprehensive validation scheme, the dataset and resulting models are better equipped to generalise across a broad range of road environments and lighting variations, thereby enhancing the practical applicability of automated pothole depth estimation.

Proposed architecture: MiDaS (DPT) for pothole depth estimation

We adopt the MiDaS framework with a dense prediction transformer (DPT) backbone for monocular depth estimation of potholes. Owing to its robust generalization across heterogeneous datasets and real-world environments, MiDaS is well-suited for estimating pothole depths from single RGB images.

Architecture overview.

The MiDaS framework consists of two main components (Fig 4). The encoder stage, implemented using either a vision transformer (ViT) or a ResNet backbone, extracts hierarchical features from the input image. This enables the model to capture both global context and local spatial detail across multiple resolutions.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Overview of the MiDaS architecture featuring the transformer-based depth estimation pipeline (a) and the Reassemble and Fusion component (b), illustrating the reconstruction of spatial feature maps and multi-scale feature fusion for dense depth prediction.

https://doi.org/10.1371/journal.pone.0344604.g004

The second stage includes the neck and decoder modules. Multi-scale features are reassembled through the Reassemble block, which converts tokens from the encoder into spatial feature maps. Convolutional and atrous convolutions refine these. The Fusion module then combines features through residual convolutional units, feature fusion blocks, and progressive upsampling with residual pooling, yielding a dense depth map aligned with the input image.

This hybrid design integrates transformer-based global modelling with convolutional detail preservation. Skip connections maintain fine spatial information, and pre- and post-processing strategies (resizing, normalization, test-time augmentation, and scale alignment) further improve inference stability.

Loss function.

The MiDaS model employs a composite loss that integrates multiple terms to optimize depth estimation performance:

1. Scale- and Shift-Invariant Logarithmic Loss (SILog) This term addresses the inherent ambiguity in monocular depth by mitigating scale and offset biases. It is defined as:

(1)

where:

(2)

Here, y_i denotes the ground truth depth at pixel i, and is the predicted depth. The first term penalizes pixel-wise relative depth errors, while the second term removes global shift in depth values.

1. 2. Gradient Matching Loss (L_grad) This term encourages alignment of depth gradients between prediction and ground truth to preserve fine edges such as pothole rims:

(3)

where and represent gradients in horizontal and vertical directions.

1. 3. Smoothness Loss (L_smooth) This term promotes spatial smoothness in depth, where the RGB image has low gradients, avoiding unrealistic depth noise in uniform surfaces:

(4)

where I_i is the input RGB intensity value.

The final training loss is given by:

(5)

where and are weights controlling the contribution of each term.

This combination allows MiDaS to maintain relative depth accuracy while preserving edge details and ensuring smoothness in uniform areas.

Evaluation metrics

Model accuracy and robustness were measured using the following established metrics:

• Mean absolute error (MAE) – Average absolute pixel-wise difference between predictions and ground truth depth.
• Root mean squared error (RMSE) – Square root of the mean of squared differences, emphasizing larger errors.
• Inverse RMSE (iRMSE) – RMSE computed on inverse depth values, sensitive to errors in nearby regions.
• Delta accuracy metrics () – Percentage of predicted depths within multiplicative thresholds of 1.25, 1.25², and 1.25³ of the ground truth.

Results

Table 2 and Fig 5 present the depth estimation performance of the proposed MiDaS model. The evaluation metrics demonstrate that the model achieves low error rates (MAE and RMSE) while maintaining high accuracy across multiple thresholds (, , and ). These results confirm the effectiveness of the DPT-based MiDaS framework for reliable pothole depth estimation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Performance of the proposed MiDaS (DPT) model on pothole depth estimation.

https://doi.org/10.1371/journal.pone.0344604.t002

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Performance of the proposed MiDaS (DPT) model in estimating pothole depth, showing results for MAE, RMSE, and accuracy metrics.

The model achieves high accuracy and consistently low error rates across all evaluation criteria.

https://doi.org/10.1371/journal.pone.0344604.g005

The findings indicate that the MiDaS-DPT framework maintains strong performance under varying road surfaces and lighting environments, providing accurate relative depth estimations (Table 3).

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Performance comparison of the proposed MiDaS-DPT model against state-of-the-art pothole-specific methods for depth estimation. Evaluation metrics include MAE, RMSE, and accuracy.

https://doi.org/10.1371/journal.pone.0344604.t003

Comparison with state-of-the-art methods

To rigorously evaluate the effectiveness of the proposed model, we conducted a comprehensive comparison against recent state-of-the-art (SOTA) approaches for pothole depth estimation. All models were assessed on our custom dataset of RGB images paired with LiDAR-derived ground-truth depth maps, ensuring a consistent and objective benchmark.

Pothole-specific depth estimation

We first compared our method with advanced pothole-specific models.

• Li et al. [23] introduced a hybrid network that combines RGB images with 3D scanner-generated depth maps for joint pothole segmentation and depth estimation. Although this model achieves high accuracy, its dependence on expensive 3D scanning hardware limits practical scalability and deployment.
• Ahmed and Raza [24] proposed a two-stage pipeline that combines MiDaS-based monocular depth estimation with YOLOv8 object detection. Although precise in both segmentation and depth estimation, the sequential design introduces additional computational complexity and latency.
• Javed [25] developed a unified deep learning model specializing in pavement analysis, which produced robust segmentation results. However, the model lacks a quantitative evaluation of the accuracy of the depth estimate, making its suitability for a specific depth assessment of potholes unclear.

The results (Fig 1, Table 1) clearly illustrate that our proposed method surpasses current pothole-specific depth estimation techniques. The results emphasize its superior accuracy in critical evaluation metrics and its robustness in adapting to varied road and environmental conditions.

General monocular depth estimation techniques

For a broader context, we further benchmarked well-known monocular depth estimation architectures, chosen for their varied learning strategies and depth representation capabilities. Each was trained and validated on our pothole dataset using only RGB input:

• Monodepth2 employs self-supervision, enforcing multi-view photometric consistency to predict depth without ground truth labels [26].
• Pix2Pix is a conditional GAN for paired image-to-image translation, adapted here to predict depth maps from RGB input [27].
• CycleGAN extends GANs to unpaired datasets, enabling flexible RGB-to-depth mappings even with limited ground truth [28].
• Dynamic spatial propagation network (DYSPN) refines sparse depth via dynamic pixel affinity learning and spatial propagation guided by RGB context [29].

As shown in Fig 2 and Table 2, our model outperforms general monocular depth estimation approaches, delivering enhanced accuracy and scalability without relying on costly sensors or complex multi-stage pipelines.

Ablation study of MiDaS loss function

To assess the contribution of different loss components in monocular depth estimation for pothole detection, we conducted an ablation study focusing on the scale- and shift-invariant logarithmic loss (SILog) and its auxiliary gradient loss. The SILog loss ensures robustness to global scale variations, while the gradient term improves boundary preservation, which is critical for detecting irregular pothole edges.

We further tested a hybrid formulation that combined SILog with Monodepth2’s photometric and smoothness losses. Although this hybrid improved local photometric consistency, it reduced global scale stability and slightly degraded depth accuracy compared to the full SILog plus gradient configuration. Additional variants—including SILog alone, Log MSE without shift invariance, standard MSE, and SILog without the gradient term—were also evaluated to isolate the role of each component.

The results in Table 4 highlight that the combination of SILog and gradient loss achieves the optimal trade-off between global scale invariance and precise edge delineation. Excluding the gradient term yields smoother predictions but weaker boundary localisation, which negatively impacts pothole detection. Similarly, removing shift invariance or reverting to standard MSE produces noticeable drops in depth accuracy due to poor handling of scale variations. Overall, the MiDaS loss with both SILog and gradient components provides the most reliable and robust performance for pothole depth estimation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Ablation study results of different loss functions for the MiDaS model in pothole depth estimation. The Full SILog loss with gradient matching achieves the best accuracy, while removing gradient or shift invariance, or replacing SILog with standard MSE, reduces performance.

https://doi.org/10.1371/journal.pone.0344604.t004

Limitations

Although the MiDaS model with the DPT architecture shows strong potential for monocular pothole depth estimation, several limitations should be noted. First, while the dataset covers diverse lighting conditions, road textures, and locations within Lahore, it is geographically constrained. The absence of samples from rural highways, mountainous terrains, and international urban environments may limit the model’s generalizability.

Second, all data were collected using an iPhone 14 Pro with a LiDAR sensor, which provides high-quality RGB and depth information. This setup does not reflect the performance of more widely available devices, such as standard dashcams or mid-range smartphones. As a result, the robustness of the model under lower-resolution inputs or noisier sensor data remains untested.

Third, because the approach relies solely on single RGB frames, it lacks geometric constraints available in stereo or multi-view systems. Although the SILog loss function mitigates scale ambiguity, accurate absolute depth estimation still requires known camera parameters or external scale references.

Fourth, the transformer-based MiDaS model, while producing high-quality predictions, is computationally demanding. This requirement may restrict deployment on edge devices or in real-time applications without further optimizations such as pruning, model compression, or hardware acceleration.

Finally, depth validation was performed using laser rangefinders with only five measurements per pothole (center and corners). This sparse sampling may introduce interpolation errors, particularly for potholes with irregular geometries or steep edges.