https://orcid.org/0000-0002-8918-7857
Figures
Abstract
This study evaluated the reliability of five types of seasonal frozen soil automatic observers (DTD1, DTD2, DTD3, DTD4, and DTD5) using parallel observation data collected from 1,172 meteorological observation stations across 24 provinces (cities, districts) in China from 2020 to 2024. The assessment was based on five key indicators: data integrity rate (with a qualified threshold of 98.00%), data standard deviation (threshold of 2.00 cm), comparable data agreement rate (threshold of 80.00%), comparable data misjudgment mean (threshold of 6.00 cm), and maximum frozen soil depth data correlation (threshold of 0.80). These indicators were used to comprehensively assess the reliability of the equipment. Over the five-year period, each set of equipment progressed through three stages: in the first stage, manual observation is predominant, in the second stage, automatic observation takes precedence, and in the third stage, the system operates independently. Equipment types achieving an independent operation rate exceeding 75% were deemed to meet the reliability standards. By the end of 2024, the independent operation rates for DTD1, DTD2, DTD3, DTD4, and DTD5 were 20.00%, 88.73%, 88.24%, 35.03%, and 23.56%, respectively. Notably, the DTD2 and DTD3 models met the reliability evaluation criteria. These findings provide a robust basis for the selection and deployment of seasonal frozen soil automatic observers in future meteorological observation networks.
Citation: Liu Z, Shao C, Liu C, Liu Y, Yang J (2025) Reliability assessment of seasonal frozen soil automatic observer. PLOS Clim 4(8): e0000688. https://doi.org/10.1371/journal.pclm.0000688
Editor: Sher Muhammad,, ICIMOD: International Centre for Integrated Mountain Development, NEPAL
Received: January 23, 2025; Accepted: July 14, 2025; Published: August 26, 2025
Copyright: © 2025 Liu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data used in this study are available in aggregated form in the tables and figures within the manuscript. Note that access to the raw data may require authorization from the China Meteorological Administration (CMA)
Funding: This work was supported by Hebei Key Laboratory of Meteorology and Ecological Environment and Qinhuangdao Municipal Science and Technology Plan (202302B025 to LZ and YJ; 202401A168 to YJ and LZ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Frozen soil can be classified into two primary types: permafrost and seasonal frozen soil [1]. Seasonal frozen soil is characterized by the freezing of the soil surface during winter and its subsequent thawing in summer [2]. The presence of seasonal frozen ground exerts profound impacts on various sectors, including engineering, transportation, hydrology, agriculture, ecology, and climate [3–4]. Specifically, when soil freezes in winter, its volume expands, causing surface heave, which can lead to problems in engineering projects and traffic infrastructure. Conversely, during summer thawing, soil contraction results in ground subsidence and reduced bearing capacity. This cyclical freezing and thawing can cause damage to building foundations, induce structural cracks, surface sinking, roadbed deformation, and railway track distortion. These issues can severely compromise the stability and lifespan of engineering structures and transportation hubs, and may even lead to roadbed collapse, such as the damage of Alaska Highway and Russian Siberian Railway, posing a significant threat to traffic safety. Even triggered “Norilsk Nickel Mine Accident”, “Yamal peninsula methane explosion pit” and “Siberian Batagaika giant pit” and other events. On slopes, the freezing and thawing of seasonal frozen soil can destabilize the soil, triggering landslides, collapses, and mudflows, which can damage construction projects and result in loss of life and property [5]. Additionally, the freezing and thawing processes can cause soil erosion and alter river courses, thereby affecting surface ecology and regional climate [6–7]. The presence of seasonally frozen layers in the soil can also impede the flow of soil moisture and nutrients, affecting fertilizer supply and leading to reduced crop yields [8]. Moreover, the efficiency of agricultural mechanization operations can be negatively impacted by frozen layers, simultaneously increasing the cost of agricultural production [9].
The Arctic active layer monitoring network (CALM), the United States permafrost research network (USAPRN), the Arctic land research station (INTERACT), the research station of Fairbanks University, the frozen soil Institute of the Russian Academy of Sciences, the Qinghai-Xizang Plateau Institute of the Chinese Academy of Sciences and other institutions have carried out frozen soil observation. Real-time monitoring of frozen soil conditions provides early warning of potential risks, reduces sudden project damage, improves climate model parameters in time, and predicts future warming scenarios. Accurate frozen soil observations can guide the design of projects in cold regions by selecting insulating materials, increasing the height of roadbeds, designing the depth of building foundations, and planning the life cycle of projects. By sharing data on frozen soil degradation, neighboring Arctic countries promote international cooperation to reduce black carbon emissions. Frozen soil observation data provide the basis for compliance assessment of international climate agreements, such as the Paris Agreement, and support the formulation of climate policy.
Seasonal frozen soil observation is an annual project uniformly arranged by the China Meteorological Administration (CMA). It focuses on monitoring the depth of soil freezing and the number of frozen layers [10]. In May 2020, five types of seasonal frozen soil automatic observers—DTD1, DTD2, DTD3, DTD4, and DTD5—from five instrument manufacturers obtained equipment licenses from the CMA. These observers were gradually deployed in meteorological observatories across 24 provinces (cities, districts) with seasonal frozen soil observation tasks. At this juncture, automatic observation of more than 20 surface meteorological elements, including seasonal frozen soil, has been essentially realized [11]. Prior to automation, manual measurements were conducted using the TB1–1 frozen soil instrument (also known as the Danilin frozen soil instrument; hereinafter referred to as “TB1-1”), with a measuring range typically between 50 and 450 cm [12]. Observers would bring the TB1–1 to the surface at 08:00 every morning to assess the degree of freezing by touching the water in the hose, thereby determining the number and depth of frozen soil layers [13]. When the hose water was in an ice-water mixture state, it was considered frozen soil. This method, which has been in use since the 1950s, is not only cumbersome and labor-intensive but also risks breaking the hose during extraction and examination, causing equipment damage and disrupting observation operations. Moreover, due to the subjective perception of the observer, the recorded data may contain human error [14]. In contrast, automated observation of frozen soil, which provides readings at a frequency of once per minute, can offer a detailed description of changes throughout the entire frozen period. It also eliminates the need for daily insertion and removal of observation tubes, thereby maintaining the temperature balance inside the tubes, reducing instrumental errors, and enhancing the accuracy of observation data.
As the nationwide promotion and use of five models of seasonal automatic observation equipment approaches its fifth year, it is imperative to assess the reliability of this equipment. This study evaluates the reliability of five types of seasonal frozen soil automatic observers (hereinafter referred to as “observers”) by comparing the parallel observation data of the DTD1, DTD2, DTD3, DTD4, and DTD5 models installed across 24 provinces (cities, districts) from 2020 to 2024 [15]. Devices with reliability exceeding 75% are selected, providing a data evaluation reference for the second phase of equipment licensing for seasonal frozen soil automatic observers by the CMA in 2025.
2. Equipment
2.1 Meteorological seasonal frozen soil observer
By the end of 2024, the China Meteorological Administration (CMA) had established 2,450 manned meteorological observation stations across mainland China (excluding Taiwan Province, Hong Kong, and Macao). All ground-based meteorological observation tasks were uniformly installed within the observation fields [16]. Among these stations, 1,172 were equipped with seasonal frozen soil observation tasks, which included both manual and automatic observation methods [17].
The artificial seasonal frozen soil TB1–1 instrument was installed at the southeast side of the observation site, 50 cm west of the deep ground temperature observation point. Automatic frozen soil sensors were also installed at the southeast side of the observation field, 50 cm south of the deep ground temperature observation point. The outer pipes of the frozen soil sensors were laid parallel to the deep ground temperature outer pipes [18], as shown in Fig 1.
The installation of the frozen soil sensor straight pipe was determined based on the maximum depth of frozen soil observed at each weather station (hereinafter referred to as “D”, with units in centimeters). The sensor could be installed in one of three configurations:
- 1). When D < 150 cm, an outer jacket pipe was installed 50 cm south of the corresponding 80 cm deep ground temperature observation point (the depth could be either 50 cm, 100 cm, or 150 cm).
- 2). When 300 cm > D ≥ 150 cm, the outer jacket pipe was installed in two sections: 150 cm and 300 cm in length, respectively, at 50 cm south of the corresponding 80 cm and 160 cm deep ground temperature observation points.
- 3). When 450 cm > D ≥ 300 cm, the sensor was installed in three sections: 150 cm, 300 cm, and 450 cm in length, respectively, at 50 cm south of the corresponding 80 cm, 160 cm, and 320 cm deep ground temperature observation points.
The outer casing pipes were installed using the drilling method [19]. The manual observation of the TB1–1 apparatus hose depth was standardized from 50 to 450 cm and used as a reference value for parallel observation and comparison.
2.2 Observer sensor principle
The five types of seasonal frozen soil automatic observers (DTD1, DTD2, DTD3, DTD4, and DTD5) utilized different measurement principles and algorithms. DTD1, DTD2, DTD3, and DTD5 are resistance-type sensors. These sensors operate based on the principle that the volume, resistance, and other physical properties of water change when its phase state changes. The measurement of frozen soil is achieved through a combination of this principle and a specific algorithm. Although the basic principle is the same for these four resistance-type models, the core measurement algorithms for frozen soil differ among them, similar to the TB1–1 instrument.
In contrast, DTD4 is a temperature-measurement type sensor. It determines the freezing point based on the temperature change characteristics associated with water freezing into ice or ice melting into water, thereby measuring the frozen soil.
3. Data and methods
3.1 Data source
This study compares the data from five types of automatic observers and the TB1–1 manual observation data from meteorological observation stations with seasonal frozen soil observation tasks across 24 provinces (cities, districts) in China. The 440 new sets of observers installed nationwide during the winter of 2020–2021 were not evaluated after the frozen soil period because the construction spanned a long period, and the equipment had less than three months to stabilize and adapt to the meteorological observatory environment following installation and commissioning.
From 2021 to 2022, an additional 709 sets of observers were installed nationwide, bringing the total to 1,149 stations that initiated the first phase of parallel frozen soil observation. Specifically, 262 sets of DTD1 equipment were installed in Shanxi, Jilin, Heilongjiang, and Shaanxi; 203 sets of DTD2 equipment were installed in Tianjin, Jiangsu, Anhui, Fujian, Shandong, Sichuan, Yunnan, and Xizang; 34 sets of DTD3 equipment were installed in Liaoning, Shanghai, and Shandong; 318 sets of DTD4 equipment were installed in Beijing, Inner Mongolia, Liaoning, Jilin, Jiangsu, Hunan, Gansu, and Ningxia; and 332 sets of DTD5 equipment were installed in Hebei, Zhejiang, Henan, Xizang, Xinjiang, and Qinghai. As meteorological inspection and evaluation activities progressed, some weather stations installed or replaced frozen soil equipment and resumed parallel observation and comparison activities. An additional 22 sets were added during 2022–2023, and one more set was added during 2023–2024, as detailed in Table 3. By the winter of 2024, a total of 1,172 sets of the five types of observers were in operation nationwide, with the distribution shown in Fig 2.
During the first stage, manual observations were primarily conducted, with data collected once daily at 08:00 and transmitted to the National Information Center via BUFF data files from the ground integrated observation service software. In the second stage, automatic observations were conducted, with data collected every minute and transmitted to the National Information Center via BUFF data files every minute, generating a fixed-format text file (frozen soil day document) daily. During the parallel observation stage, all TB1–1 data for the month were input into the frozen soil parallel observation and comparison software (with automatic import of observer data) before the end of each month. On the first day of the following month, the “Frozen Soil Observation Data Monthly Compilation File” was automatically generated, and the frozen soil daily file was compressed into a “Frozen Soil Automatic Observation Minute Data Zip Format Compressed File.” These compiled and compressed files were synchronized and uploaded to the National Information Center. During the independent operation stage, automatic observation prevailed, the TB1–1 was discontinued, and no further inspections or evaluations were conducted.
3.2 Methods
The evaluation of parallel observation equipment is guided by the observational equipment testing methods [20], seasonal frozen soil parallel observation business technical regulations [21], and automatic observation standards implemented by the CMA [22]. This process requires a minimum of two years and is typically divided into three stages. In the first year, manual observation serves as the primary method, with automatic observation providing supplementary data. Following the first year’s evaluation, the equipment progresses to the second stage, where automatic observation becomes the primary method and manual observation serves as a supplement. Upon passing the second stage evaluation, the equipment enters the third stage of independent operation. If the evaluation is not passed, parallel observation continues at the current stage. Based on the testing experience from 2019 to 2020, a comprehensive annual inspection of each observer is conducted after the frozen soil period ends, focusing on data completeness, accuracy, and comparability [23–25]. If all indicators meet the criteria, the equipment passes the inspection. If fewer than 30% of the observation equipment of the same model pass the inspection of all indicators in a given year, all equipment of that model is deemed non-compliant and continues to operate in the original observation phase. Within a five-year period, if more than 75% of the equipment installed nationwide for a given observer type passes the inspection and achieves independent operation, the reliability of that observer type is considered to meet the standard.
3.2.1 Data integrity.
Data integrity is assessed by evaluating the instrument’s ability to collect data [26]. The instrument is checked per minute in the standard format, which includes basic parameters (see Table 1) and minute data content (see Table 2). All minute data are tested individually within the year, excluding data missing due to external interference [27]. The data integrity rate of the instrument ( ) is calculated using the following Equation (1):
Where is the number of correct minutes and
is the total number of observations. If
≥98.00%, the observer passes the integrity test; Otherwise, it is deemed unqualified and fails the evaluation for the current year.
3.2.2 Data accuracy.
Data accuracy is a critical metric that reflects the quality of the data obtained by the observer [28]. To assess this, the manual observation data from the TB1–1 instrument at 08:00 serves as the reference standard for comparison with the data collected by the automatic observer. Initially, the comparison data samples from the equipment are checked to ensure that there are at least 10 samples available. If this requirement is met, the testing proceeds; otherwise, the evaluation of the equipment is halted due to an insufficient number of marked samples. Seasonal frozen soil is one of the most complex elements in meteorology, characterized by unstable layers, each with distinct upper and lower limits. The optimal matching of these upper and lower limits is illustrated in Fig 3.
Fig 3A presents a contrast between automatic (left) and manual (right) observations. The automatic observation system records data for up to eight layers, whereas historical manual observations have typically covered six layers nationwide [29]. Additionally, the automatic system may misjudge the frozen soil depth due to missed measurements, as discussed in Section 3.2.3. Therefore, it is essential to match the minimum error values of the upper and lower limits of the two layers for comparison. Fig 3B shows that the number of layers observed automatically is fewer than those observed manually, and the upper and lower limit values are matched into two layers for comparison. The non-frozen soil records in the manual layer result from multiple measurements by the observer misjudging the depth of frozen soil, as detailed in Section 3.2.3. Fig 3C demonstrates that the automatic and manual observation layers are identical, with their corresponding upper and lower limits matched into two layers. Based on this principle, a comparison dataset of matched upper and lower limits is formed.
The systematic error and standard deviation are calculated using Equations (2) and (3), respectively. Any abnormal field value exceeding three times the standard deviation is removed to ensure data quality [30–31].
- a). Systematic error
(cm) algorithm:
is the measurement value of the observer at time
,
is the observation value at time
of the TB1–1 and
is comparing the number of observation samples.
- b). The standard deviation
(cm) algorithm:
is the difference of the
-th measurement between the observer and TB1–1,
is the systematic error, and
is the number of observed samples.
If ≤2 cm, it passes the accuracy test, otherwise the observer is unqualified and fails the evaluation of the current year.
3.2.3 Data comparability.
Data comparability quantifies the degree of agreement between the automatic observer and the TB1–1 instrument in obtaining frozen soil data [32]. During the evaluation, the standard freezing thickness is denoted as H (unit: cm). The frozen soil thickness value that matches the freezing standard of TB1–1 is used as the correct identification thickness value for the observer, represented by H1 (unit: cm). The frozen soil thickness values not correctly identified by the observer within the upper and lower limits observed by TB1–1 are denoted by H2 (unit: cm), as illustrated in Fig 4.
Three sub-indicators are used to conduct data comparability: 1) Consistency Rate of Comparable Data: This measures the proportion of data points where the observer’s measurements agree with the TB1–1 standard [33]. 2) Misjudgment Mean of Comparable Data: This assesses the average deviation of the observer’s measurements from the TB1–1 standard [34]. 3) Correlation of Maximum Frozen Soil Depth Data: This evaluates the correlation between the maximum frozen soil depth data from the observer and TB1–1 [35–36].
These sub-indicators provide a comprehensive assessment of the observer’s performance relative to the TB1–1 instrument, defined as follow:
- a). Comparable data agreement rate
(%) algorithm:
During the inspection and assessment period, observer correctly identified the frozen soil thickness value (H1), Percent of the freezing depth value (H) of the reference standard.
- b). Misjudge mean
(cm) algorithm for comparable data:
During the inspection and evaluation, observer did not correctly identify the frozen soil thickness (H2) of the mean value (Fig 4).
c) Maximum frozen soil depth data correlation (dimensionless) algorithm:
is the maximum frozen soil value on day
of the observer,
is the maximum frozen soil value on day
of the TB1–1, and
is the number of comparative observation samples.
is the average of
samples from the observer, and
is the average of
samples from the TB1–1.
During the inspection and evaluation period, a comparison curve diagram (as shown in Fig 5) is generated based on the daily lower limit observation values of the first layer of frozen soil (i.e., the maximum daily frozen soil depth) obtained from the observer. This diagram is utilized to analyze the freeze-thaw change trends and assess the data correlation of the observer.
Using Equations (4), (5), and (6), the test results were required to meet the following criteria: ≥80.00%,
≤6 cm and
≥0.80. The results were then automatically smoothed and compared with the trend map of the manually observed maximum frozen soil depth. If the smoothed curve showed no obvious fractures or jumps and passed the comparability assessment, the observer was deemed qualified. Conversely, if these conditions were not met, the observer was considered unqualified and failed the evaluation for the current year.
4. Results
Following the conclusion of the seasonally frozen ground period, in accordance with the business requirements of the China Meteorological Administration (CMA), all parallel observation data collected from the observers were utilized to conduct the evaluation work. Specifically, the data integrity was assessed using the “frozen soil automatic observation minute data in zip format compressed file,” as detailed in Section 3.2.1. Subsequently, the accuracy and comparability of the equipment data were evaluated with the aid of the “frozen soil observation data monthly rectification file.” Data sets with a sample size of less than 10, as well as those lacking frozen soil observations due to elevated winter temperatures, were excluded from the analysis. Instruments associated with these excluded data sets remained in their original operational stages. By the end of 2024, the operational observers were categorized into three stages: the first stage included 349 sets, the second stage comprised 364 sets, and the independent operational stage encompassed 459 sets. The inspection and evaluation results are presented in Table 3.
4.1 Regional assessment
As indicated in Table 3 and Fig 2, the distribution and operational stages of the different types of observers across various regions are assessed. In Shanxi, 36.11% of the 108 DTD1-type observers have progressed to the second stage, with 21.30% achieving independent operation. In Jilin, 22% of the observers remain in the first stage, and none have reached independent operation. In Heilongjiang, 31 sets (12.90% of the total) have passed the evaluation, with one set operating independently. In Shaanxi, half of the 96 sets are in the second stage, and nearly 30% have achieved independent operation.
All DTD2-type observers in Tianjin and Sichuan are operating independently. In Anhui, only one set out of 78 is in the first stage, while 87.18% have achieved independent operation. All 112 sets in Shandong have passed the evaluation, with 98.21% operating independently. In Xizang, one-third of the observers are in the first stage. Fujian and Yunnan have been affected by the lack of samples in winter; however, more than half of the equipment passed the evaluation in 2023 and 2024.
Despite having the smallest number of installations, 30 out of 34 DTD3-type observers have entered independent operation, achieving an independence rate of 88.24%. All six sets in Shandong are operating independently. In Shanghai, one set remains in the second stage, primarily due to the limited sample size during the 2021–2023 period, which precluded a comprehensive assessment. In Liaoning, 24 out of 27 sets have transitioned to independent operation. Additionally, 15 out of 17 sets that were scheduled to be upgraded from DTD4 to DTD3 in 2023 have successfully passed the evaluation and advanced to the second stage.
The evaluation of DTD4 observers indicates that 80% of the 10 sets in Beijing have passed the assessment, with three sets currently operating in a single-track mode. In Inner Mongolia, out of 119 observers, 40% have reached the second stage, another 40% are in the first stage, and 26 sets are functioning independently. In Liaoning, 70% of the 17 observers are operating independently. In Jilin, an additional DTD4 set was installed during the winter of 2023. Among the 29 sets, 14 are in the first stage, and 6 are operating independently. In Jiangsu, two sets of DTD2 failed to pass the assessment following the installation of DTD4 in 2022. In 2023, a new set was under construction, with 25 out of 52% of the instruments operating independently and 9 sets having entered the second stage of operation. Due to a limited number of frozen soil samples reported over the past three consecutive years, no evaluations were conducted in Hunan. In Gansu and Ningxia, nearly 50% of the instruments are operating independently, while only approximately 10% are in the first stage.
In Hebei, 40.77% of the DTD5 observers have reached the independent operation stage, and an additional 24 sets are in the second stage. Zhejiang has fewer frozen soil samples. In Henan, half of the 34 sets are in the second stage, with 7 sets having passed the evaluation. In the Xizang dataset, 5 out of 21 sets have reached the independent stage, while 14 sets remain in the first stage. In the Xinjiang dataset, only 6 out of 103 sets have reached the independent stage, with nearly 70% of the sets still undergoing parallel observation in the first stage. For the Qinghai dataset, more than 60% of the 42 sets are in the first stage, and only 16.67% have reached the independent stage.
When evaluating the number of observers across all provinces (autonomous regions and municipalities), the impact assessment was excluded for Zhejiang, Hunan, Fujian, and Yunnan due to the low number of high-temperature samples in winter. No obvious regional characteristics were observed. In 11 regions, including Tianjin, Sichuan, Shandong, Yunnan, Shanghai, Anhui, Liaoning, Gansu, Ningxia, Hebei, and Beijing, more than 80% of the observers have completed the first stage assessment. In 7 regions, more than 50% of the instruments are in independent operation, with Tianjin and Sichuan achieving 100%, Shandong 88.21%, Liaoning 88.89%, Anhui 87.18%, Jiangsu 52.00%, and Yunnan 50.00%. Most of these regions have installed DTD2 and DTD3 observers. In contrast, Shanghai, Zhejiang, Fujian, and Hunan have no instruments in independent operation. Heilongjiang, Xinjiang, Jilin, Qinghai, Henan, Shanxi, Inner Mongolia, Xizang, and Shaanxi have less than 30% of their observers in independent operation, while other regions fall within the range of 30%–50%.
4.2 Evaluation of the five indicators
The five indicators were evaluated based on the different measurement principles and algorithms of the five observer models. The performance evaluation results of the five indicators are shown in Fig 6.
The data collection capability of each observer type was optimal among all indicators. In Fig 6A, the data integrity rate of the DTD3 and DTD2 observers were slightly better than that of other observer types, with fewer outliers. The DTD1 observer exhibited the most outliers and the lowest average value. In Fig 6B, the accuracy of data collected by the DTD2 and DTD3 observers were significantly better than that of other observers, with results more concentrated, fewer outliers, and a smaller standard deviation. In Fig 6C, the mean data consistency rate of the DTD4, DTD3, and DTD2 observers exceeded 90%, higher than that of other models. The DTD3 observer had the fewest outliers, while the mean value of DTD4 was the highest. The DTD1, DTD2, and DTD5 observers exhibited a large 1.5 IQR range, indicating high data dispersion. In Fig 6D, the mean values of the DTD2 and DTD3 observers were significantly lower than those of other models, with the maximum negative deviation being approximately 11.0 cm. Data dispersion was low, with the median line and mean values being small and essentially overlapping, and fewer outliers, indicating minimal data deviation. In contrast, the maximum negative deviation of other observers exceeded 58.0 cm. In Fig 6E, the data correlation of the maximum frozen soil depth measured by the DTD3 and DTD4 observers was better than that of other equipment, with a mean value exceeding 0.92. The data dispersion of DTD5 was the largest, with the minimum positive deviation reaching 0.73. Based on observation experience, different measurement principles of observers will not exacerbate the dispersion of measurement results. With the installation and maintenance of equipment in accordance with business regulations at the meteorological observation station, ensuring minimal clearance between the measured soil and the outer casing will also reduce equipment errors and improve data consistency. Only different measurement algorithms have the greatest impact on the deviation of inversion results. Overall, the five indicators of the DTD3 and DTD2 observers are better than other models of equipment, and the sensor inversion algorithms of the DTD3 and DTD2 observers are also better than other observers.
4.3 Performance of the five observer models
As shown in Table 3 and Figs 7 and 8, 203 sets of DTD2 observers were installed nationwide in 2021, with an additional 10 sets and 3 sets installed in 2022 and 2023, respectively. By the end of 2024, only 5 sets remained in the first stage of parallel observation, while 189 sets had transitioned to independent operation, achieving the highest independent operation rate of 88.73% among the five observer types. This high rate indicates that the reliability of the DTD2 equipment meets the evaluation requirements. Although DTD3 is the least installed equipment in China, it achieved a high independent operation rate of 88.24%, ranking second among the five observer types. All three-year inspections passed the first-stage evaluation, confirming that the reliability of the DTD3 equipment meets the evaluation requirements. By the winter of 2024, 314 sets of DTD4 observers were operational nationwide, with 83 sets in the first stage and 110 sets having passed the independent operation evaluation. The independent operation rate of 35.03% placed DTD4 in the middle among the five observer types. However, there is still a gap between the equipment reliability and the evaluation requirements.
DTD5 is the most widely installed observer type in China. By the winter of 2024, 44.71% of the observers were in the first stage, with only 78 sets having passed the independent operation evaluation, resulting in an independent operation rate of 23.56%. This rate is slightly higher than that of DTD1 but still indicates a significant gap between the equipment reliability and the evaluation requirements. By the winter of 2024, 260 sets of DTD1 observers had been installed nationwide, with 42.69% in the first stage. Only 52 sets had passed the independent operation evaluation, yielding the lowest independent operation rate of 20.00% among the five observer types. The reliability of the DTD1 equipment is clearly insufficient.
5. Conclusion and discussion
This study comprehensively assessed the reliability of five types of seasonal frozen soil automatic observers (DTD1, DTD2, DTD3, DTD4, and DTD5) using parallel observation data collected from 1,172 meteorological observation stations across 24 provinces (cities, districts) in China from 2020 to 2024. The evaluation was based on five key indicators: data integrity rate, data standard deviation, comparable data agreement rate, comparable data misjudgment mean, and maximum frozen soil depth data correlation. The results provide valuable insights into the performance and reliability of these observers.
The assessment results indicate that, aside from the influence of geographic temperature bias factors, there are no significant regional characteristics in the performance of the five types of observers. The independent operation rates of DTD2 and DTD3 observers exceed 88%, with both achieving rates higher than 75%. This high rate confirms that the reliability of these two types of equipment meets the assessment requirements. In contrast, the independent operation rates of the other observer types are less than 40%, indicating a notable gap between their reliability and the assessment criteria. DTD3 and DTD2 observers significantly outperformed the other observers across all five key indicators. Specifically, the data integrity rate, accuracy, and measurement algorithms of DTD2 and DTD3 were superior to those of other models, with fewer outliers and lower standard deviations. The mean data consistency rate of DTD4, DTD3, and DTD2 observers exceeded 90%, higher than that of other models. However, DTD4 did not achieve an independent operation rate higher than 75%. DTD1 and DTD5 observers exhibited poor performance on several indicators, particularly in terms of data integrity and misjudgment mean.
Considering the assessed performance of the five observer types in terms of independent operation rates and the five key indicators, the reliability of DTD2 and DTD3 is deemed ideal and capable of meeting the evaluation requirements. Other models still fall short of these requirements. Therefore, it is recommended that future operations gradually phase out the less reliable observer types and replace them with DTD2 and DTD3 observers. This transition is expected to enhance the quality of automatic observation data for national seasonal frozen soil.
References
- 1. Yuxing C, Liming J, Linlin L, et al. Time-series InSAR monitoring of permafrost deformation in the upper Heihe River based on Sentinel-1 SAR data. J Geophys. 2019;62(07):2441–54.
- 2.
GB/T 35234—2017, the ground meteorological observation standard of frozen soil [S].
- 3. Zhang T, Li HP, Hu CC, et al. Review and prospect of the effects of freeze-thaw on soil geotechnical properties. Sci Cold Arid Reg. 2021;5:349–56.
- 4. Ma D, Luo SQ, Guo DL, et al. Simulated effect of soil freeze-thaw process on surface hydrologic and thermal fluxes in frozen ground region of the Northern Hemisphere [J]. Sci Cold Arid Reg.2021;(1):18–29.
- 5. Bingxin H, Chao Y, Wei Y, et al. Numerical simulation of frozen soil slope stability based on an improved hydrothermal coupling model. J Glaciol Geocryol. 2024;1–13.
- 6. Yan L, Wenbing Y, Tianqi Z, et al. Progress of soil polycyclic aromatic hydrocarbon contamination in permafrost zone. Glacial Permafrost. 2022;44(05):1640–52.
- 7. Linfeng F, Fang J, Xingxing K, et al. Impacts of perennial permafrost degradation on river runoff in the northern Himalaya. Sci China: Earth Sciences. 2024;54(06):2020–30.
- 8. Hongwei H, Qilong Q, Chunshan S,et al. Freezing and thawing process and heat transfer law of seasonal frozen soil in the northern Songnen Plain. Sci Technol Eng. 2023;23(35):14947–54.
- 9. Shiyin T, Meizhu G, Jingan H, et al. Distribution characteristics of seasonal frozen soil and application in lightning protection grounding. Meteorol Environ Sci. 2022;5:95–104.
- 10.
CMA. Ground Meteorological Observation Code [M]. Beijing: Meteorological Press; 2003.
- 11.
CMA. The National Ground Meteorological Observation Automation Reform Plan [Z]. Beijing: CMA; 2018.
- 12.
QX/T 12—2023, frozen soil device [S].
- 13.
Shuli S, Dongdong C, Zhigang L, et al. Seasonally frozen ground observation [M]. Beijing. China Meteorological Press; 2023.
- 14. Zhifeng S, Xin L, Ziyong S, et al. Simulation of hydrothermal process of seasonally frozen ground in the upper reaches of Heihe River based on SHAW model. J Glaciol Geocryol. 2024;46(05):1678–91.
- 15.
Comprehensive Observation Department of CMA. Functional specification requirements of frozen soil automatic observer. Beijing: CMA; 2018.
- 16.
Notice of the Comprehensive Observation Department of the CMA on Issuing the List of National Meteorological Observation Stations and the List of Major Meteorological Observation Equipment Stations in 2024. Beijing: CMA; 2024.
- 17.
CMA. Standard for Automatic Observation of Ground Meteorological Administration (first edition) [M]. Beijing: Meteorological Press; 2020.
- 18.
Comprehensive Observation Department CMA. Technical regulations on parallel observation service of frozen soil. Beijing: CMA; 2020.
- 19.
GB/T 33705—2017, frequency-domain reflection method for soil water observation [S].
- 20.
Comprehensive Observation Department CMA. Letter on printing and distributing general provisions (revised) of test methods of special technical equipment for meteorological observation. Beijing: CMA; 2017.
- 21.
Comprehensive Observation Department CMA. Technical scheme for construction of automatic observation system. Beijing: CMA; 2020.
- 22.
Comprehensive Observation Department CMA. Specification for automatic observation of frozen soil. Beijing: CMA; 2019.
- 23. Shuang T, Lin T, Xiaoling L, et al. An efficient method for checking the integrity of data in the Cloud. China Commun. 2014;11(9):68–81.
- 24. Ning M, Zhihua R, Yan W, et al. Parallel observation and comparative analysis of the national precipitation weather phenomenon. Arid Zone Study. 2022;39(1):54–63.
- 25. Xiangyu L, Jingqiang C. Comparability assessment and comparative citation generation method. Computer Applications. 2024;:1–9.
- 26. Shaohui W, Zhengyu Z, Huaqun W, et al. Analysis and improvement of a remote data integrity verification scheme. Computer Sci. 2023;50(7):302–7.
- 27. Zhigang L, Dongli W, Changliang S, et al. Time consistency detection of automatic observation and minute data in Liaoning and Inner Mongolia in 2019. J Meteorol Environ. 2022;38(2):97–104.
- 28. Shuai Y, Wei Z, Wenjie Y, et al. GRACE data was combined with a new scale factor correction method to improve the accuracy of land water reserves. J Geophys. 2021;64(9):3068–82.
- 29. Qiangjun L, Wanli L, Yujun Z, et al. Climate characteristics and correlation analysis of frozen soil at the southern foot of Taihang Mountain. J Gansu Agric Univ. 2022;57(2):172–81.
- 30. Qiaona Q, Wei W. Study on the stability characteristics of multiple mode precipitation forecast. Weather. 2024;50(4):420–33.
- 31. Qingxia C, Xiaohui L, Chenglong T. Spatial Variation and Influencing Factors of Soil pH in Anshun City. Huan Jing Ke Xue. 2022;43(4):2124–32. pmid:35393836
- 32. Birong Z, Jia L, Mingguo W, et al. DMSP/ OLS and NPP/ VIIRS Chinese mainland study. Remote Sens Inform. 2021;36(3):99–107.
- 33. Zhenyan Y, Lihong W, Dawei G, et al. Quality control of automated meteorological observation data. Meteorol Sci. 2016;36(5):703–8.
- 34. Ling Y, Kang YY. Baarda crude difference miscalculation analysis in the data detection method. J Tongji Univ (Natural Science Edition). 2018;46(10):1440–7.
- 35. Xin T, Sachula , Fanhao M, et al. Analysis of the spatial and temporal evolution characteristics of the freezing-thaw state of the near-surface soil in Inner Mongolia in recent 40 years. Res Soil Water Conservation. 2025;32(1):148–59.
- 36. Huiran G, Chong X, Wanchang Z. Spatial parametric characterization of surface soil in plateau cold regions. J Glaciol Geocryol. 2023;45(6):1859–74.