https://orcid.org/0000-0003-3446-2775
Figures
Abstract
The refraction error compensation model for ground object detection is established. Firstly, based on a 100-meter interval, the area between the aircraft and the ground objects is stratified, and the Hopfield model of the atmospheric refractive index is combined with the e-index model to achieve an accurate simulation of the atmospheric refractive index profile. Then, based on the information such as the line-of-sight direction, flight altitude, and observation wavelength of the ground object observation target by the aircraft, combined with Snell’s law, the distance from the projection of the aircraft on the Earth’s surface to the true position of the ground object is given, and the true elevation angle is derived. Finally, by calculating the difference between the apparent elevation angle and the true elevation angle of the aircraft, the elevation angle error model of the aircraft is established to correct the elevation angle error of the aircraft caused by atmospheric refraction and is applied to the precise identification and positioning of the observed target.
Citation: Yu D, Chen Z (2026) Modeling method for refraction error compensation in ground object detection. PLoS One 21(1): e0339265. https://doi.org/10.1371/journal.pone.0339265
Editor: Vinod Ayyappan, King Mongkut’s University of Technology North Bangkok, THAILAND
Received: May 25, 2025; Accepted: December 2, 2025; Published: January 27, 2026
Copyright: © 2026 Yu, Chen. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper.
Funding: This study was supported by the Shandong Provincial Natural Science Foundation in the form of a grant awarded to D.C.Y (GRANT NUMBER. ZR2024QA229) and the Shandong Provincial Natural Science Foundation in the form of a salary for D.C.Y. This study was supported by the Jining University 2023 ‘100 Outstanding Talents’ support program cultivation project in the form of a grant awarded to D.C.Y (GRANT NUMBER. 2023ZYRC34) and the Jining University 2023 ‘100 Outstanding Talents’ support program cultivation project in the form of a salary for D.C.Y. The specific roles of this author are articulated in the ‘author contributions’ section. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1 Introduction
The density of the Earth’s atmosphere is a continuous curve that varies with altitude. When observing a feature target from an aircraft, because the light has to traverse the atmosphere with different concentrations, the light is bent by refraction, which ultimately leads to a difference between the observed feature position and the actual situation, thus affecting the detection and localization accuracy of the surface target [1,2]. The elevation angle error of atmospheric refraction eventually leads to the distance error between the observed and actual positions of the feature target, and it has been shown that when the observed zenith angle is and
, the distance error between the true position and the apparent position of the feature due to atmospheric refraction reaches 2.22 m and 17.85 m, respectively [3]. Therefore, reducing the elevation angle error is of great value to improve the positioning accuracy of the detected target.
Astronomical atmospheric refraction has been widely studied in the last 20 years, such as the widely used Pulkovo atmospheric refraction table, which corrects for bending angle errors in astronomical observations, thus improving the accuracy of astronomical observations [4,5]. Starlight refraction navigation, as a spacecraft autonomous navigation method, the high-precision modeling of the starlight refraction angle when the apparent zenith distance is is an important factor affecting the navigation accuracy, which is also an application of the theory of astronomical atmospheric refraction [6]. In the field of Earth observation and remote sensing detection, the elevation angle error compensation technique is less studied, and the technique is less mature compared to the astronomical atmospheric refraction correction technique [7]. Synthetic aperture radar (SAR) has important applications in natural disaster monitoring, environmental quality monitoring, ocean monitoring, resource exploration, crop yield estimation, and surveying and mapping, and in order to improve the accuracy of the earth observation, the elevation error correction is particularly important [8–10]. In most of the current researches to improve the SAR positioning accuracy, it is mainly realized by designing the corresponding mechanical structure rather than compensating the elevation error [11]. For the problem of SAR remote sensing image distortion, most of the researches are mainly realized by developing corresponding image processing algorithms instead of compensating the elevation error to correct the distorted image [12]. Based on the current research status, a refractive error compensation modeling method is proposed for Earth surface detection targets. Firstly, the atmosphere is precisely stratified, and the refraction modeling is completed by combining the atmospheric temperature, pressure and other parameters, then the Hopfield model of atmospheric refractive index and the e-index model are combined to realize the simulation of the atmospheric refractive index contour in the process of construction, and finally the real elevation angle is calculated to get the elevation angle error.
The structure of this article is as follows. The geometric description and modeling process of elevation angle error are described in Sect 2. Sect 3 presents the calculation results of the elevation angle error and the influence of atmospheric parameters on the elevation angle error. Sect 4 discusses the model’s limitations, potential applications, and future improvements. Sect 5 presents the conclusion.
2 Modeling of elevation angle error
Light is refracted as it travels through the atmosphere, resulting in a positional deviation between the real position of the feature and the apparent position of the feature, and the angle between the line connecting the aircraft to the real position of the feature target and the line connecting the aircraft to the apparent position of the feature target is defined as the elevation angle error, as shown by γ in Fig 1. The apparent and real positions of the observed objects are marked in the figure. Elevation angle error γ, real elevation angle Z0, and apparent elevation angle Z0 at the observation position are also shown in Fig 1. The apparent and real positions of the observed objects are marked in the figure. The elevation angle error γ, as the value to be solved, can be expressed by the following equation,
Fig 1. Observational geometry of aircraft’s observation of the Earth.
where the apparent elevation angle Z0 (a known quantity) is the angle between the aircraft-apparent object line and the aircraft-Earth center line, and the true elevation angle θ (an unknown quantity) is the corresponding angle to the true ground object position. The unknown angle θ can be determined by solving Eq 2,
where h is the distance from the aircraft to the Earth’s surface, which is a known quantity. l is the distance from the projection of the aircraft on the Earth’s surface to the true position of the ground object, which is an unknown quantity and needs to be solved to obtain it. The specific solution process is as follows.
Fig 1. Observational geometry of aircraft’s observation of the Earth. The apparent and real positions of the observed objects are marked in the figure. Elevation angle error γ, real elevation angle Z0, and apparent elevation angle Z0 at the observation position are also shown in the figure.
Firstly, the Earth’s atmosphere is precisely stratified. To ensure the accuracy of the calculation results and the computing speed of the program, the distance between two adjacent atmospheric layers is generally fixed at 100 meters. After verification, when the height resolution is 100 meters, the fluctuation of elevation angle error is relatively small [7]. Suppose the atmospheric density is the same within the same layer, but different between different layers. Then, the Snell’s law is satisfied between adjacent layers. The specific form of Snell’s law is as follows,
where n0 is the refractive index at the observed position, Z0 is the apparent elevation angle from the observed position, ni is the refractive index at the ith layer, Zi is the elevation angle of the light at the ith atmospheric layer. Suppose the Earth’s atmosphere are divided into N layers, then i=1,2,3. Thus, the refraction angle of each layer can be calculated. The specific equation is as follows,
The horizontal distance between two adjacent atmospheres can be expressed as the following relationship,
where step represents the vertical distance between two adjacent atmospheres, which is 100 meters. Based on the horizontal distance between every two adjacent layers, the distance l between the projection of the aircraft on the ground surface and the true position of the ground object can be obtained,
The true elevation angle of the aircraft can be obtained through Eq (2), and then the elevation angle error γ caused by atmospheric refraction can be obtained through Eq (1).
The solution of the atmospheric refractive index in Eq (4) is a key point, which is related to the calculation of the refractive angle of each layer of the atmosphere. n0 represents the atmospheric refractive index of the apparent position of the ground object and can be expressed by the following equation,
where α=1/273.16, t, P, e, and λ respectively refer to the atmospheric temperature, pressure, vapor pressure, and wavelength at the location of the ground object, with units of ∘C, Pa, Pa, and μm respectively. The refractive index of the atmosphere along the path of light propagation can be expressed by the following equation,
where hd represents the elevation, which can be calculated by 40136+148.72×t, with the unit being m. Below the elevation, the refractive index of the atmosphere at each altitude position is given by the Hopfield model, and above the elevation, the refractive index of the atmosphere at each altitude position is given by the e-index model, where nH represents the refractive index of the atmosphere at the elevation. β is the piecewise fitting index. The Hopfield model is employed for altitudes below the elevation (hd) due to its accuracy in modeling the troposphere and stratosphere, where temperature and pressure gradients are significant. The e-index model is used above hd to account for the exponential decay of atmospheric density in the upper layers [13]. This hybrid approach ensures accuracy across all atmospheric regimes.
In summary, the modeling process detailed in this section effectively constructs the specific form of the unified master equation,
The atmospheric stratification and the application of Snell’s law translate the inputs—t, P, e, and λ —into a precise refractive index profile, which is then used to compute the final elevation angle error γ. This framework ensures that all critical factors are integrated into a comprehensive error compensation model.
3 Analysis of influencing factors of elevation angle error and calculation results
The influence of vapor pressure on elevation angle error
Although the water vapor content in the atmosphere is not high, it still has an impact on the refractive index, thereby affecting the elevation error. The water vapor pressure has obvious latitudinal distribution characteristics, with a maximum of approximately 30 hPa at the equator and gradually decreasing towards the poles. The variation range of water vapor pressure on the land surface is between 0 and 30 hPa, and it is relatively large near the ground. Then it decreases rapidly with the increase in height. Assuming that the vertical distribution of water vapor pressure is as shown in the following equation,
where e0 represents the water vapor pressure at the Earth’s surface, and indicates the distance above the Earth’s surface, with the unit of km.
In order to analyze the influence of vapor pressure on the elevation angle error, the elevation angle error was calculated respectively under the conditions of considering vapor pressure and not considering vapor pressure, and finally the gap between the two was calculated. The environmental parameters are: the longitude and latitude of Beijing area (,116.
), time 12:00:00 on September 1, 2022 (UTC), and wavelength 0.58 μm. Calculate the elevation angle errors of the aircraft at flight altitudes of 40 km, 70 km, and 100 km respectively under the conditions of e0= 30 hPa and e0= 0 (unit: arcseconds), and calculate the difference
between the two.
is defined as the difference between the elevation angle error under e0= 0 and the elevation angle error under e0= 30 hPa. The calculation results are shown in Table 1.
Based on the above calculation results, it is found that the influence of water vapor pressure on the elevation angle error is within 10−4 to 10−3 arcseconds, and its impact on the elevation angle error accuracy is approximately 0.005%. Therefore, under normal weather conditions, the influence of water vapor pressure on the refraction elevation angle error of ground objects can be ignored.
The influence of temperature on elevation angle error
The temperature of the Earth’s atmosphere does not change uniformly. In the troposphere and mesosphere, the atmospheric temperature decreases with the increase of altitude. Specifically, in the troposphere, for every 1 km increase in altitude, the temperature drops by approximately 6 ∘C. In the stratosphere, an inversion layer occurs, where the temperature rises with the increase of altitude. In the thermosphere, the temperature also increases with the increase of altitude. In order to analyze the influence of temperature on the elevation angle error, in practical operation, assuming the pressure remains constant and the vapor pressure e=0, the variation of the elevation angle error under the condition of temperature change is calculated respectively.
The environmental parameters are as follows: the longitude and latitude of Beijing area (,116.
), the time is 12:00 UTC on September 1, 2022, the solar activity index F107 is 60 sfu, the geomagnetic activity index Ap is 10 nT, and the wavelength is 0.58 μm. Based on this environmental parameter, the initial values of temperature and pressure at the location of the ground object were calculated using the NRLMSIS 2.0 model. These values can also be given through other models or measured data.
The elevation angle errors (in arcseconds) of the aircraft at flight altitudes of 40 km, 70 km, and 100 km under the conditions of temperature change (±1 K, ±5 K, ±10 K) were calculated respectively, as shown in Table 2.
Based on the ground object refraction elevation angle error values in Table 2, calculate the difference error between the elevation angle error under the conditions of change ±1 K, ±5 K and ±10 K and the elevation angle error under the condition of
being 0 respectively. Here, it is defined as: The difference between the elevation angle error with a
change of ±1 K and the elevation angle error under the condition of
being 0 is error1; the difference between the elevation angle error with a
change of ±5 K and the elevation angle error under the condition of
being 0 is error5; the difference between the elevation angle error with a
change of ±10 K and the elevation angle error under the condition of
being 0 is error10. The absolute values of the error1, error5, and error10 ranges within the 5-
elevation angle range of the aircraft at flight altitudes of 40 km, 70 km, and 100 km respectively are shown in Table 3.
It can be obtained from Table 3 that as the altitude of the aircraft increases, the change in the elevation angle error caused by temperature decreases. As the apparent direction of the aircraft and the deflection angle relative to the vertical line of the earth increase, the change in the elevation angle error caused by temperature increases. For every 1 K change in temperature, the elevation Aagle error changes approximately by the order of 10−3–10−2 arcseconds. For every 10 K change in temperature, the elevation angle error can vary by the order of 10−1 arcsecond. Through calculation, it is found that even if the temperature variation is 10 K, the influence on the accuracy of the elevation angle error is still less than 5%.
The influence of latitude on elevation angle error
Select one representative area from each of the low-latitude, mid-latitude and high-latitude land areas in the Northern Hemisphere of the Earth to calculate the elevation angle error. The specific information is as follows: The low-latitude region is selected as East Kalaman Province of Indonesia (N, 1
E), the mid-latitude region is selected as Beijing (40°N, 116°E), and the high-latitude region is selected as Yakute Region of Russia (70°N, 116°E). Under the conditions of calculating the elevation angle of 5–30° respectively, The elevation angle errors of the aircraft at flight altitudes of 40 km, 70 km, and 100 km are shown in Table 4. In addition to the geographical longitude and latitude, other environmental parameters are as follows: time 12:00:00 on September 1, 2022 (UTC), solar activity index F107 is 60 sfu, geomagnetic activity index Ap is 10 nT, and wavelength 0.58 μm. It can be obtained from Table 2.17 that it is found that the elevation angle error decreases with the increase of latitude. This is because in regions with high latitudes, the air pressure is high, the temperature is low, and the air is thin. Compared with the middle and low latitudes, the refraction effect is not particularly significant.
The influence of seasons on elevation angle error
In order to analyze the influence of seasons on elevation angle errors, the elevation angle errors in the Beijing area on April 15, 2021 (spring), July 15, 2021 (summer), October 15, 2021 (autumn), and January 15, 2022 (winter) were discussed. It is found that the elevation angle error is the largest in winter, the smallest in summer, and moderate in spring and autumn. However, as the altitude increased, the elevation angle error gradually increased in summer and gradually decreased in winter, as shown in Table 5. This is because the temperature is low in winter and the atmosphere contracts. At lower altitudes, the atmospheric density in winter is greater than that in summer, and the refraction effect is more obvious. However, as the altitude increases, the atmospheric density in winter decreases rapidly compared to summer, resulting in a gradual decrease in the altitude error in winter compared to summer.
Calculation result of elevation angle error
By calculating the variation law of the elevation angle error with height within the altitude range of 1–80 km under the conditions of 25°, 30°, 35° and 45° respectively at the apparent zenith distance, it is found that the elevation angle error first increases and then decreases with height, as shown in Fig 2. The elevation angle error is the largest at an altitude of approximately 15 km.
4 Discussion, limitations, and future applications
The developed hybrid model offers a reliable and computationally practical solution for compensating atmospheric refraction errors, effectively improving geo-location accuracy in airborne and satellite remote sensing under typical atmospheric conditions. A principal strength of this approach is its combination of the Hopfield and e-index models, which strikes an effective balance between physical fidelity and operational ease, all without necessitating dedicated hardware adjustments. Nevertheless, the model’s effectiveness is bounded by several constraints. Its reliability depends heavily on the accuracy of input atmospheric profiles, such as those derived from models like NRLMSIS 2.0. Performance may degrade in situations involving highly irregular or rapidly changing conditions—including extreme weather phenomena, pronounced atmospheric ducting, or over complex terrain where the assumption of spherical stratification and horizontal homogeneity breaks down. In these demanding scenarios, more sophisticated methods incorporating data-assimilative numerical weather prediction with comprehensive three-dimensional ray-tracing would be better suited, though requiring substantially greater computational resources. Overcoming these constraints through the incorporation of real-time meteorological inputs and refined stratification methodologies constitutes an important objective for subsequent research.
Our model is designed for integration into both real-time and post-processing systems. For live ground object detection, the compensation algorithm can be embedded directly into the onboard processing units of platforms like Synthetic Aperture Radar (SAR) satellites or high-altitude reconnaissance drones, where it would utilize real-time telemetry (altitude, look-angle) and pre-loaded or forecast atmospheric profiles to apply instantaneous corrections to the geolocation data. Alternatively, for applications where the highest possible accuracy is paramount, it functions effectively as an offline correction system in ground stations, processing raw data with precisely measured atmospheric conditions. This flexibility allows for its assimilation into a wide range of Earth observation systems, including environmental monitoring satellites, disaster early warning networks, and precision mapping aircraft, significantly enhancing their target positioning accuracy without requiring hardware modifications.
5 Conclusion
We propose an atmospheric refraction error compensation model for ground object detection. By stratifying the atmosphere at intervals of 100 meters and combining the Hopfield refractive index model and the e-index model, the atmospheric refractive index profile is accurately simulated. Based on Snell’s law and the observation parameters of the aircraft, the error model of the true elevation angle and the apparent elevation angle is derived. The research finds that the influence of vapor pressure on the elevation angle error can be ignored (<0.005%). Every 10 K change in temperature will lead to a maximum error of 0.28 arcseconds, and the error decreases with the increase of latitude. The elevation angle error reaches its peak at an altitude of 15 kilometers, showing a vertical distribution pattern that first increases and then decreases. This model provides an error correction scheme without mechanical structural adjustment for synthetic aperture radar and remote sensing detection systems, significantly improving the positioning accuracy of surface targets, and is applicable to fields such as environmental monitoring and disaster early warning.
References
- 1. Wu S, Wang H, Wang B. Construction of a backpropagation starlight atmospheric refraction model based on ray tracing. Appl Opt. 2023;62(14):3778–87. pmid:37706996
- 2. Dettwiller L. Study of the lateral shift due to atmospheric refraction: alternative analytical methods, and new results. J Opt Soc Am A Opt Image Sci Vis. 2024;41(7):1310–8. pmid:39889117
- 3. Peng X, Huang W, Li X, Yang L, Chen F. A spatiotemporal atmospheric refraction correction method for improving the geolocation accuracy of high-resolution remote sensing images. Remote Sensing. 2022;14(21):5344.
- 4. Wang Z, Jiang J. Refraction surface-based stellar atmospheric refraction correction and error estimation for terrestrial star tracker. IEEE Sensors J. 2022;22(10):9685–96.
- 5. Ma B, Jiang J, Yang J, Ma Y. A multiparameter alternative optimization atmospheric refraction correction method for star sensors with large-zenith-distance starlight. IEEE Sensors J. 2024;24(17):27541–52.
- 6. Ning X, Wang L, Bai X, Fang J. Autonomous satellite navigation using starlight refraction angle measurements. Advances in Space Research. 2013;51(9):1761–72.
- 7. Ye J, He H, Zhang L, Lin X, Qiang Y. An accurate calculation of the atmospheric refraction error of optical remote sensing images based on the fine-layered light vector method. IEEE J Sel Top Appl Earth Observations Remote Sensing. 2022;15:1514–25.
- 8. Tsokas A, Rysz M, Pardalos PM, Dipple K. SAR data applications in earth observation: an overview. Expert Systems with Applications. 2022;205:117342.
- 9. Amitrano D, Di Martino G, Di Simone A, Imperatore P. Flood detection with SAR: a review of techniques and datasets. Remote Sensing. 2024;16(4):656.
- 10. Zhang X, Feng S, Zhao C, Sun Z, Zhang S, Ji K. MGSFA-Net: multiscale global scattering feature association network for SAR ship target recognition. IEEE J Sel Top Appl Earth Observations Remote Sensing. 2024;17:4611–25.
- 11. Huang B, Li ZH, Tian XZ, Yang L, Zhang PJ, Chen B. Modeling and correction of pointing error of space-borne optical imager. Optik. 2021;247:167998.
- 12. Ye Y, Zhang J, Zhou L, Li J, Ren X, Fan J. Optical and SAR image fusion based on complementary feature decomposition and visual saliency features. IEEE Trans Geosci Remote Sensing. 2024;62:1–15.
- 13.
Yu D, Li H, Li B. A comparison of models for correcting astronomical atmospheric refraction. In: Seventh Symposium on Novel Photoelectronic Detection Technology and Applications. 2021. https://doi.org/10.1117/12.2585676