Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Classification of land lot shapes in real estate sector using a convolutional neural network

  • Subin Ham,

    Roles Data curation, Formal analysis, Investigation, Validation, Visualization, Writing – original draft

    Affiliation Department of Real Estate, Kangwon National University, Chuncheon, Gangwon-do, Republic of Korea

  • Changro Lee

    Roles Conceptualization, Methodology, Validation, Writing – original draft, Writing – review & editing

    spatialstat@naver.com

    Affiliation Department of Real Estate, Kangwon National University, Chuncheon, Gangwon-do, Republic of Korea

Abstract

In the agriculture and real estate industries, land lot shapes have mostly been classified by visual inspection or hard-crafted rules. These conventional methods are time-consuming, resource-intensive, and subject to human bias. This study aims to fill this gap and alleviate problems inherent in traditional lot classification approaches. This study attempts to classify lot shapes automatically, using a convolutional neural network. A study area was chosen, image data of the lots in the area were collected and preprocessed, and an Xception neural network was specified to classify land lots according to their shapes. The test applied to a different area adjacent to the study area achieved an accuracy of 90.1% and area under the curve (AUC) of 0.96. Additionally, this study demonstrated that shape regularity can be quantified using the output scores from the neural network analysis. This is the first attempt to employ a deep learning algorithm for land management on a micro-spatial scale. The classification approach proposed in this study is expected to encourage the rapid and accurate classification of various lot shapes.

1. Introduction

Lot shapes have been analyzed across diverse industries such as farming, urban planning, and property valuation. In many countries, agricultural policymakers aim to consolidate fragmented land lots to enhance productivity; thus, understanding lot shapes is essential for implementing land consolidation programs. In urban planning, the shapes of land lots play a crucial role in designing street layouts and optimizing zone regulations. In property valuation, the shape of a lot can significantly impact its market value because regular shapes, such as squares, are often easier to develop and therefore command higher prices. Studies on land lot shapes have aided in improving agricultural productivity, increasing social benefits generated from urban development projects, and enhancing property appraisal accuracies.

Despite the importance of lot shape classification, many shapes are still classified manually. That is, land lots have been classified through visual inspection by human observers or by using hard-crafted rules. Common rules observed in previous studies include the ratio of depth to frontage in a lot, simplicity index, and roundness index [13]. However, visual inspection is time-consuming, resource-intensive, and subject to human bias. With hard-crafted rules, the proposed indices are often arbitrary and suitable only for the selected study areas they were designed for [4].

This study aims to automate land lot classification without human involvement and simultaneously increase classification accuracy. Specifically, we attempt to classify land lots according to their shapes using a deep learning algorithm. A study area consisting of approximately 20,000 lots was chosen, and the lots were automatically classified using a convolutional neural network. Convolutional neural networks, which are specialized image recognition tools, are widely used across various domains, including the analysis of medical images for diagnoses such as breast cancer [5] and facial recognition for authentication [6]. We also attempt to quantify shape regularity by utilizing the output scores obtained from the neural network adopted in this study. We demonstrate that a deep learning-supported approach to land-shape classification can replace existing methods effectively and efficiently.

In this study, a relevant neural network was trained by feeding imagery data to the network to classify lot shapes. This approach has the advantage of eliminating human involvement, enabling lot shapes to be classified accurately and promptly. Although neural networks specialized in computer vision have been extensively used in related fields, such as landform classification [7, 8] and plant classification [9], this technique has not been applied to the pattern analysis of land lots. To the best of our knowledge, this study is the first to employ a deep learning algorithm for land management on a micro-spatial scale.

The remainder of this paper is organized as follows. Section 2 reviews prior studies on lot shapes in rural and urban areas, and the emerging trend of image data and neural networks usage. Section 3 explains the dataset, study area, and neural network architecture. The classification results and discussion are presented in section 4. Finally, the findings are summarized and a path for future research is suggested in section 5.

2. Literature review

2.1 Classification of lot shapes in rural and urban areas

Studies on effective lot shapes started in the agricultural industry. In his seminal paper, Losch [10] proposed that a hexagonal lot is the most efficient for farming operations. Lee and Sallee [11] argued that a rectangular shape is superior to a hexagonal one when considering the costs of farming operations. Since then, there have been a few discussions about which shape is more productive for farming [12, 13]; the rectangle-shaped lot is accepted as a standard lot shape in farming practice today. These studies focused on large lots in rural regions and primarily aimed to minimize tillage time per hectare in land consolidation [14]. The lot shape was manually classified into a few groups after visually examining sample lots in the study area, such as rectangular, triangular, trapezoidal, and irregular shapes [15].

Later, lot shapes in cities were actively investigated following early studies on farmlands. In urban planning, the lot and its shape are the basic units for block subdivision, and different lot patterns are usually suggested for different block types [16]. Lot shape is also considered a critical determinant for urban redevelopment, and Gao and Asami [1] examined the impact of lot shape on the costs and benefits of redevelopment projects. They concluded that irregularly shaped lots provide landowners with fewer benefits, that is, higher reconstruction costs and lower price escalations between the original and redeveloped lots. In addition, lot shape plays an important role in property valuation: rectangular lots are typically appraised at higher prices than irregularly shaped ones because improvements can be made more efficiently on the former [17]. These studies focused on small-sized lots in urban areas and attempted to maximize the social benefits generated from urban planning and redevelopment, or enhance the accuracy of property valuation. Similar to studies in the agricultural industry, lot shape was categorized in a qualitative manner such as regular, intermediate, and irregular [18].

Recently, geographical information system (GIS)-driven classification methods have been proposed [2, 4, 19]. These studies used several GIS algorithms to calculate shape-perception indicators, such as simplicity index and roundness index, and compared different urban configurations, including planned, organic, tree-like, and grid-like layouts.

Conventional methods for classifying lot shapes, such as computing shape-perception indicators, have two main advantages. First, indicators, such as the roundness index, are straightforward and intuitive. Secondly, the calculation of these indices typically involves simple formulas and algorithms, thereby making them computationally efficient.

In short, previous studies employed two main approaches: visual inspection by human observers and the use of hard-crafted rules. The visual inspection approach is significantly hindered by the inspector’s subjectivity. For example, a trapezoidal land lot may be classified as irregular or rectangular depending on the inspector’s perspective. The vague definition of the lot shape also contributes to this subjectivity. The latter approach, which relies on hard-crafted rules, is generally applicable to relatively regular-shaped lots. Previous studies using this approach primarily analyzed rectangular lots and their variants for simplicity. Their main index was to calculate the depth-to-frontage ratios or similar metrics of typical rectangular lots and compare them with the measurements of other lots. Although more sophisticated indices, such as the compactness metric, were developed later [20, 21], this approach is still limited because it can be practically applied only to rectangular or quasi-rectangular lots.

We attempt to train a deep learning model by feeding lot-shape data into the model, to enable lot-shape recognition and classification without human judgement. This approach offers several advantages and fills a research gap. First, this study addresses the entire range of lot shapes found in practice, from highly irregular to perfectly square lots, offering a more inclusive and holistic approach for lot shape classification. Second, this study introduces the use of deep learning models for lot shape recognition and classification. By training these models with large amounts of shape data, this study moves away from manually crafted rules and human inspections. Finally, by leveraging advanced deep learning techniques, this study enhances the accuracy of lot shape classification.

2.2 Utilization of image data and neural networks

After lot shapes in cities were rigorously analyzed, studies on lot patterns were phased out from the literature for a while. Around 2015, research on shape was revived following the rising popularity of deep learning. Deep learning technique has spread quickly in all industries within a short time, and the neural network has emerged as the de facto standard model to implement this technique. Neural networks have excellent capabilities that cannot be realized through conventional models like regression analysis. These capabilities include capturing non-linear relationships in data, transferring knowledge learned in one task to another, and handling unstructured data [22]. In particular, the ability to process unstructured data, such as images and sounds, enables neural networks to achieve excellent performance in computer vision and speech recognition [2326].

In tandem with these trends, land shapes were analyzed through different approaches compared to prior studies: using neural networks and imagery data. Sunaga et al. [7] collected satellite images, fed them into convolutional neural networks, and classified similarly shaped terrains, such as plains, ridges, and volcanoes. The classification of land forms by exploiting neural networks and satellite images has become a popular research topic in the literature [8, 27, 28]. Neural networks and visual data have also been used in the building architecture industry to classify floor plans. Ahn [29] collected floor plan images and analyzed the shapes of apartment units using convolutional neural networks. Like studies on land form classification, the investigation of geometric elements in property floor plans within the framework of deep learning is a recent and popular research theme in building design [3032].

Although neural networks and images have been actively exploited to classify land forms and floor plans, this approach has not yet been applied to the analysis of lot shapes. To the best of our knowledge, this study is the first attempt to classify lot shapes on a microscale using a deep learning algorithm: lot shapes are not manually classified by visual inspection, but through a learning process based on image data. By adopting this approach, we demonstrate that lots can be classified more efficiently, and without human bias.

3. Dataset and architecture of a convolutional neural network

3.1 Dataset and study area

Peongsan-gu was chosen as the study area. It is a district in Changwon city, South Korea. Seongsan-gu is characterized by a mixed landscape of downtown urban and surrounding rural areas. Thus, it contains various lot shapes, from square-shaped lots in the downtown areas to extremely irregular-shaped lots in the suburbs.

Approximately 20,000 lots comprise Seongsan-gu, and image files (JPG format) for 18,976 lots were collected. Of these lots, 8,309 (44%) were labeled as regular shapes, and 10,667 (56%) were labeled as irregular shapes. Tax assessors determine shapes by visually examining lots on the cadastral map. Thus, these data can be considered balanced in terms of the distribution of labels. The 18,976 lots were further randomly split into 80% training and 20% validation data. Finally, the test dataset was obtained from a different area to reliably ascertain model validity. A part of lots in Masanhoewon-gu (3,845 lots) were used for testing. Masanhoewon-gu is another district in Changwon city which is bordered by the study area (Seongsan-gu) to the northwest. Table 1 presents the dataset configuration and Fig 1 shows the study and test areas.

thumbnail
Fig 1. Study and test areas (Figure created by the authors).

https://doi.org/10.1371/journal.pone.0308788.g001

Fig 2 shows a part of the study area at lot-level resolution. The northern area is characterized by rectangular lots, which are typical landscapes of downtown areas. In contrast, the southern area consists mainly of irregularly shaped lots, which are frequently observed in rural areas. This study attempts to classify lots with a wide spectrum of shape regularities, as shown in Fig 2.

thumbnail
Fig 2. Study area at a lot-level resolution (Figure created by the authors).

https://doi.org/10.1371/journal.pone.0308788.g002

3.2 Specifying a neural network

This study adopted the architecture of an Xception neural network. The Xception neural network is one of the convolutional neural networks, and has the advantage of using network parameters more efficiently compared to other convolutional neural networks [33].

Lot images were prepared in a 180×180 pixel-size and standardized to have values between zero and one. Data augmentation was applied to the training data. That is, horizontal flipping and random rotation techniques were used. This augmentation procedure helps the neural network learn different aspects of the images. The neural network architecture consists of one entry block and four repeated main blocks. The convolutional and pooling layers primarily comprise these blocks. At the end of the architecture, a final classification layer (an ordinary dense layer) was added to classify the land lots as regular or irregular. Fig 3 shows the network architecture used in this study. The architecture presented in the figure was determined by evaluating its performance on validation data during training. This architecture is well-suited for the dataset in this study for several reasons. First, it performs spatial convolutions on each channel of the input separately before mixing outputs via pointwise convolutions. This reduces the required number of parameters compared with traditional convolutional layers, thereby enhancing the network efficiency without sacrificing performance. Second, it is computationally efficient and tends to outperform networks of similar sizes, making it suitable for implementation in environments with limited computational resources, such as personal computers [33].

thumbnail
Fig 3. Architecture of the neural network.

Notes: Conv. layer, Batch. layer, Maxpool. layer, and Avgpool. layer represent the convolutional, batch normalization, maximum pooling, and average pooling layers, respectively.

https://doi.org/10.1371/journal.pone.0308788.g003

The neural network processes input images as follows. First, it starts with an image of a predefined size (180×180 pixels). The entry block extracts high-level features from the image. The main blocks further process the features extracted by the entry block. These blocks are composed exclusively of depth-wise separable convolutions. The exit block prepares the feature maps for classification and passes the output to a dense layer. The final dense layer uses a sigmoid activation function for classification. During training, the neural network uses backpropagation to compute the gradient of a binary cross-entry loss function. An adaptive moment estimation (Adam) optimizer [34] with a learning rate of 0.001 was employed. A rectified linear unit (ReLU) activation function was utilized for all layers except for the last dense layer, where a sigmoid activation function was employed. The network was trained for 15 epochs with a batch size of 32.

4. Results

4.1 Classification accuracy

Land lots (3,845 samples) in the test area of Masanhoewon-gu were used to evaluate the performance of the neural network, and a threshold of 0.5 was adopted. Table 2 shows the result of the classification, in which approximately 90.1% of the test lots were classified correctly ((2,154+1,309)/3,845). A receiver operating characteristic (ROC) curve is another popular metric. The area under the ROC curve (AUC) was computed as 0.96. ROC and AUC metrics are especially relevant for datasets where target variables are imbalanced [35]. An AUC value of 0.5 indicates that the neural network has no discrimination capacity, while an AUC value of 1.0 indicates an ideal neural network that correctly classifies all classes.

A classification accuracy of 90.1% and an AUC of 0.96 can indicate good or poor performance depending on the task characteristics at hand. In some applications, achieving more than 50% accuracy may be sufficient, whereas even 99% prediction accuracy can be trivial for an extremely imbalanced classification problem. In the real estate sector, Chen et al. [36] achieved accuracies of 77–90% and AUC values of 0.75–0.93, respectively, for house image classification. In the healthcare industry, where convolutional neural networks are extensively applied, Li et al. [37] reported accuracies of 85–94% and AUC values of 0.84–0.94 for skin cancer and retinopathy image classifications.

In this study, we concluded that an accuracy of 90.1% and AUC of 0.96 are practically sufficient, especially given the absence of human involvement in the classification process. When comparing this accuracy and AUC value with those of previous studies, the results of this study are both comparable and effective. In addition, an AUC value close to 1.0 does not always imply a perfect neural network; it may indicate overfitting to the training data. Therefore, we believe that the neural network proposed in this study is suitable for industrial deployment with minimal modifications. Based on the accuracy metric, the remaining 9.9% of the misclassified land lots can be addressed separately by experts, such as assessors.

4.2 Quantifying shape regularity

In data preprocessing, regular-shape lots were coded as one, and irregular-shape lots were coded as zero. Thus, the output score from the binary classification neural network indicates the probability for a lot to be classified as regular. Fig 4 shows example lots with varying shapes and the corresponding probabilities of being regular-shaped lots.

thumbnail
Fig 4. Land lots and probabilities of being regular-shaped lots.

https://doi.org/10.1371/journal.pone.0308788.g004

The first lot in the figure received a probability of 0.0008, and is indeed extremely irregular-shaped; it can be easily inferred that this lot has a significant disadvantage during building, because its irregular shape may considerably reduce the effective area. Progressing from left to right, the lot shape approaches a square pattern. The far-right lot in the figure receives a probability of 0.9992, and it can be easily understood that its effective area for construction is almost equal to the original lot size. It is well-known that a land lot with a square pattern is preferred in the market and generally commands the highest price when other site characteristics are equal.

Lot shape is used as a price determinant in tax assessments for property in South Korea. However, the classification is simplified into five categories: flag-shaped, irregular, trapezoidal, rectangular, and square-shaped. Accordingly, the corresponding price discounts in the assessment are also limited to five values: 11%, 6%, 3%, 1%, and 0% [38]. These discount values are only valid in the study area: they differ depending on administration jurisdictions. Thus, the approach proposed in this study provides several advantages over the current assessment system. First, it can significantly reduce the subjectivity in human judgement on lot shape: it is well-known that the same lot can be classified differently by different people. Second, unlike the time-consuming visual inspection of lot shapes by humans, it can speed up the classification task without compromising classification quality. Finally, it can expand the current number of shape categories (five categories) and incorporate more diverse shapes: the probability of being a regular-shaped lot can be used as a criterion for such an expanded classification.

5. Conclusion

For long, lot shapes have been investigated in fields ranging from agriculture to urban planning and property valuation. However, these studies tended to focus on rectangular or quasi-rectangular lots while classifying them manually. Thus, this study attempted to classify land lots according to the various shapes observed in practice, and a classification was performed using image data and a neural network.

Seongsan-gu was chosen as the study area, 18,976 lots in the area were fed into a convolutional neural network, and the network was trained to classify lot shapes. Its performance was evaluated based on a test dataset from an area close to the study area, and the network achieved an accuracy of 90.1%. Finally, we demonstrated that the shape regularity of the lots can be quantified effectively using the output probability from the network. The approach proposed in this study is expected to be applicable to relevant industries, including property valuation.

This study employed a single-input and single-output neural network: an image file was fed into the network and a binary classification result was obtained. However, one superior capability of a neural network is its flexible design. For example, a multi-input and single-output neural network can be envisioned: an image file and metadata (lot area, zoning, frontage width) can be fed into a neural network simultaneously, and the price of the subject lot can be estimated as an output. In future research, a neural network should be designed to have a more diverse architecture so that it can be utilized in more extensive areas.

References

  1. 1. Gao X., & Asami Y. Influence of lot size and shape on redevelopment projects. Land use policy, 2007, 24(1): 212–222.
  2. 2. Araldi A., & Fusco G. Decomposing and recomposing urban fabric: The city from the pedestrian point of view. Computational Science and Its Applications–ICCSA 2017: 17th International Conference, Trieste, Italy, July 3–6, 2017, Proceedings, Part IV 17. Springer International Publishing, 2017.
  3. 3. Araldi A., Wayens B., & Fusco G. Retail spatial organisation and the physical city: bridging retail geography and urban morphology. Belgeo. Revue belge de géographie, 2024, 3, https://doi.org/10.4000/11nj0.
  4. 4. Pont M.B., Stavroulaki G., Gil J.L., Marcus L., Serra M., Hausleitner B., et al. Quantitative comparison of cities: Distribution of street and building types based on density and centrality measures. InXI SSS: 11th International Space Syntax Symposium, 2017.
  5. 5. Yu H., Yang L. T., Zhang Q., Armstrong D., & Deen M. J. Convolutional neural networks for medical image analysis: state-of-the-art, comparisons, improvement and perspectives. Neurocomputing 444, 2021, 92–110.
  6. 6. Sahan J. M., Abbas E. I., & Abood Z. M. A facial recognition using a combination of a novel one dimension deep CNN and LDA. Materials Today: Proceedings, 2023, 80:3594–3599.
  7. 7. Sunaga Y., Natsuaki R., & Hirose A. Land form classification and similar land-shape discovery by using complex-valued convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10): 7907–7917.
  8. 8. Jin B., Ye P., Zhang X., Song W., & Li S. Object-oriented method combined with deep convolutional neural networks for land-use-type classification of remote sensing images. Journal of the Indian Society of Remote Sensing, 2019, 47(6): 951–965.
  9. 9. Yağ İ, & Altan A. Artificial intelligence-based robust hybrid algorithm design and implementation for real-time detection of plant diseases in agricultural environments. Biology, 2022, 11(12):1732. pmid:36552243
  10. 10. Losch A. Economics of location, 1954, New Haven: Yale University Press.
  11. 11. Lee D. R., & Sallee G. T. Theoretical patterns of farm shape and central place location. Journal of Regional Science, 1974, 14(3): 423–430.
  12. 12. Burns E. K. Nested hexagons: Central place theory (pp. 163–181), 1997, New Brunswick: Rutgers University Press.
  13. 13. Morgan F., & Bolton R. Hexagonal economic regions solve the location problem. The American mathematical monthly, 2002, 109(2): 165–172.
  14. 14. Gonzalez X. P., Alvarez C. J., & Crecente R. Evaluation of land distributions with joint regard to plot size and shape. Agricultural systems, 2004, 82(1): 31–43.
  15. 15. Gonzalez X. P., Marey M. F., & Alvarez C. J. Evaluation of productive rural land patterns with joint regard to the size, shape and dispersion of plots. Agricultural Systems, 2007, 92(1–3): 52–62.
  16. 16. Wiseman N., & Patterson Z. Testing block subdivision algorithms on block designs. Journal of Geographical Systems, 2016, 18(1): 17–43.
  17. 17. MLIT. Guidelines for real estate appraisal, Sejong: Ministry of Land, Infrastructure and Transport of South Korea, 2016.
  18. 18. Asami Y., & Niwa Y. Typical lots for detached houses in residential blocks and lot shape analysis. Regional Science and Urban Economics, 2008, 38(5): 424–437.
  19. 19. Barthelemy M. From paths to blocks: New measures for street patterns. Environment and Planning B: Urban Analytics and City Science, 2017, 44(2): 256–271.
  20. 20. Vialard A. A typology of block-faces, Doctoral dissertation, Georgia Institute of Technology, 2013.
  21. 21. Bobkova E., Marcus L., Berghauser Pont M., Stavroulaki I., & Bolin D. Structure of plot systems and economic activity in cities: Linking plot types to retail and food services in London, Amsterdam and Stockholm. Urban Science, 2019, 3(3): 66.
  22. 22. Géron A. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems, 2019, Sebastopol: O’Reilly Media.
  23. 23. Bell P., Fainberg J., Klejch O., Li J., Renals S., & Swietojanski P. Adaptation algorithms for neural network-based speech recognition: An overview. IEEE Open Journal of Signal Processing, 2020, 2: 33–66.
  24. 24. Buhrmester V., Münch D., & Arens M. Analysis of explainers of black box deep neural networks for computer vision: A survey. Machine Learning and Knowledge Extraction, 2021, 3(4): 966–989.
  25. 25. Javanmardi S., Ashtiani S. H. M., Verbeek F. J., & Martynenko A. Computer-vision classification of corn seed varieties using deep convolutional neural network. Journal of Stored Products Research, 2021, 92: 101800.
  26. 26. Diep Q. B., Phan H. Y., & Truong T. C. Crossmixed convolutional neural network for digital speech recognition. Plos One, 2024, 19(4): e0302394. pmid:38669233
  27. 27. Minetto R., Segundo M. P., & Sarkar S. Hydra: An ensemble of convolutional neural networks for geospatial land classification. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6530–6541.
  28. 28. Zhou F., Li R., Trajcevski G., & Zhang K. Land Deformation Prediction via Slope-Aware Graph Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 17, pp. 15033–15040), 2021.
  29. 29. Ahn E. An Analysis on the Evolution of Korean Apartment Unit Plans using Deep Learning. Journal of the Architectural Institute of Korea, 2021, 37(10): 13–22.
  30. 30. Sharma D., Gupta N., Chattopadhyay C., & Mehta S. A novel feature transform framework using deep neural network for multimodal floor plan retrieval. International Journal on Document Analysis and Recognition (IJDAR), 2019, 22(4): 417–429.
  31. 31. Lu Z., Wang T., Guo J., Meng W., Xiao J., Zhang W., et al. Data-driven floor plan understanding in rural residential buildings via deep recognition. Information Sciences, 2021, 567: 58–74.
  32. 32. Pizarro P. N., Massone L. M., Rojas F. R., & Ruiz R. O. Use of convolutional networks in the conceptual structural design of shear wall buildings layout. Engineering Structures, 2021, 239: 112311.
  33. 33. Chollet F. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251–1258), 2017.
  34. 34. Kingma D. P., & Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  35. 35. Grigorev A. Machine Learning Bookcamp, 2020, Shelter Island: Manning Publications.
  36. 36. Chen R., Li X., & Li J. Object-based features for house detection from RGB high-resolution images. Remote Sensing, 2018, 10(3): 451.
  37. 37. Li S., Kou P., Ma M., Yang H., Huang S., & Yang Z. Application of semi-supervised learning in image classification: Research on fusion of labeled and unlabeled data. IEEE Access, 2024, 12: 27331–27343.
  38. 38. Changwon City. Big data portal of Changwon, https://bigdata.changwon.go.kr/portal/EgovPageLink.do?menuNo=&link=stat, accessed on July 4, 2023.