PONE-D-24-26472Beyond the Greater Angkor Region: Automatic large-scale mapping of Khmer Empire reservoirs in satellite imagery using Deep LearningPLOS ONE

Dear Dr. Klassen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The two peer reviewers thought that this paper had merit, as do I, but improvements are recommended. The overall content and presentation requires some evaluation to streamline and present the overall aims, objectives and findings. With respect to the content itself, improvements in method descriptions, data availability and reference citations is in need of consideration. 

Please submit your revised manuscript by Oct 03 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Michael D. Petraglia, Ph.D.

Academic Editor

PLOS ONE

Journal requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating the following financial disclosure: 

 [European Union’s Horizon 2020 Marie Sklodowska-Curie grant agreement No. 896092  (sk)].  

Please state what role the funders took in the study.  If the funders had no role, please state: ""The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."" 

If this statement is not correct you must amend it as needed. 

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

[The authors thank the Cambodian government, including the Ministry of Culture and Fine Arts and the APSARA Authority. This project has received funding from the European Union’s Horizon 2020 Marie Sklodowska-Curie grant agreement No. 896092 and the University of Toronto. Administrative support for this project was provided by Leiden University, École française d’Extrême-Orient, and the University of Toronto. ]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

 [European Union’s Horizon 2020 Marie Sklodowska-Curie grant agreement No. 896092  (sk)].  

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

5. We note that you have indicated that there are restrictions to data sharing for this study. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. 

Before we proceed with your manuscript, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of recommended repositories, please see

https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible.

We will update your Data Availability statement on your behalf to reflect the information you provide.

6.  We note that 4, 7, 8 ,9 10 and 11 in your submission contain [map/satellite] images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a You may seek permission from the original copyright holder of 4, 7, 8 ,9 10 and 11 to publish the content specifically under the CC BY 4.0 license.  

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an ""Other"" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The use of artificial intelligence and remote imagery in archaeological research is a burgeoning field, and Landauer et al. provide an exciting example of the ways these relatively new methodological tools can be fruitfully applied. The authors construct a deep learning model using Deeplab V3+ to automate the identification of Angkor period water reservoirs, and report partial success. While the deep learning model is obviously not perfect, the authors contend that it is accurate and precise enough to aid archaeologists in later terrestrial survey and identification of archaeological sites. I ultimately agree with this assessment, and believe that this article would be a welcome addition to the literature with the minor changes below. However, I have a number of concerns which I would like to see the authors address prior to publication.

1. Foremost among these, the GitHub link provided directs only to a singular empty readme file, rather than the full body of the code. While I appreciate that the exact location of the archaeological reservoirs cannot be reported, the full code used in this analysis should be made easily available for replicability – especially considering the methodological nature of this paper.

2. Although the introduction is largely solid, lines 100 to 105 seem out of place. Rather than framing the research problem in question, they detail a failed attempt at a different project. Instead of including this section in the introduction, I might recommend shifting these lines to the discussion or conclusion as an area for future research. The background is similarly well written, with a note that the claim on lines 165-166, “This pattern of more and distinctly less densely occupied agricultural areas seems broadly consistent at the other urban centers surveyed in Cambodia in 2015,” needs a proper citation to the aforementioned survey.

3. The methods section includes material that may be better suited for either the discussion or the introduction. In particular, lines 184 to 205 do not outline the specific methods of this analysis but instead set up important background context. These lines should be moved to the introduction or background. Furthermore, the background section of this paper would benefit from an elaboration of the broader context this work takes place in. How, specifically, do the methods outlined in this paper differ from the works cited between lines 181 to 205? (Zimmer-Dauphinee, VanValkenburgh, and Wernke 2024) conducted similar research in another region; how does their approach differ?

4. Line 330 – The authors mention ‘Precision’ and ‘Recall’ as their quality metrics, but do not describe the exact equation for calculating these metrics.

5. While not necessarily for the acceptance of this paper, I would encourage the authors to consider Snyder and Haas 2024 which examines the efficacy of manual satellite survey. A review and comparison to the model vs. expert portion this study, as presented from lines 368 to 396.

6. At the moment, the discussion and conclusion narrowly situate the work in its relevance to future survey projects towards the Angkor region of Cambodia. While this is important in and of itself, the paper would be significantly improved through broader articulation with the satellite survey and artificial intelligence in the archaeological literature. For example, the authors could discuss similarities and differences between this work and many of the articles cited between lines 187 and 195.

7. Line 225 through 259 detail challenges with Bing satellite imagery. Why was other imagery not used instead? Google Earth allows users to examine imagery from a variety of dates, and has successfully been used in the past to monitor archaeological sites (Contreras and Brodie 2010; Contreras 2010).

Overall, this is a strong article that I look forward to seeing published. However, prior to that, I believe the authors need to situate their methodological approach, its relevance, and implications for future research in a broader context.

References Cited:

Contreras, Daniel A.

2010 Huaqueros and Remote Sensing Imagery: Assessing Looting Damage in the Virú Valley, Peru. Antiquity 84(324). Cambridge University Press: 544–555.

Contreras, Daniel A., and Neil Brodie

2010 The Utility of Publicly-Available Satellite Imagery for Investigating Looting of Archaeological Sites in Jordan. Journal of Field Archaeology 35(1). Taylor & Francis: 101–114.

Snyder, Thomas J., and Randall Haas

2024 Unstructured Satellite Survey Detects up to 20% of Archaeological Sites in Coastal Valleys of Southern Peru. PLOS ONE 19(2). Public Library of Science: e0292272.

Zimmer-Dauphinee, James, Parker VanValkenburgh, and Steven A. Wernke

2024 Eyes of the Machine: AI-Assisted Satellite Archaeological Survey in the Andes. Antiquity 98(397): 245–259.

Reviewer #2: This manuscript reports on the use of a deep learning model to identify ancient reservoir features in the Greater Angkor Region. The article is interesting and well written. I do have a few comments and concerns with the current draft that need to be addressed before proceeding further.

General Comments:

The main issue I find with the current manuscript is that the definition of an "acceptable" result or error rate is not well justified. In some fields, F1 scores less than 0.9 might be considered unacceptable, while others are perfectly fine with F1 scores of 0.5. The results presented in this manuscript are, by most standards, quite low. While this does not mean that the results you present are unusable, the burden is on you to: 1) define what your working definition of an "acceptable" performance metric is and 2) justify the utility by citing its comparison with other modes of identification and/or speed of evaluation. You do get to this latter part later in the manuscript, but don't really define what you deem as "acceptable".

As currently written, someone skeptical of AI who picks up this article will likely see this as another example of why these approaches "aren't worth it" or "don't work well". I think your strongest argument is to further emphasize the fact that with X amount of time to train a model (needs to be quantified), your method performs roughly as well as a student that needed 6+ months of training. As such, the tool is useful for initial exploratory analysis that experts can fine tune and verify much more quickly (similarly to having students assist in data analysis tasks). But with this, a more nuanced discussion of how you are determining acceptability of results is needed.

Other Specific Comments

Data availability: A link to a Github page was provided with the manuscript but there is no code currently uploaded in that location. This should be updated before acceptance. Additionally, there is no mention of this Github link in the manuscript, itself, and should be listed either in the acknowledgements or somewhere in the methods section so readers can find it.

Figures:

Figure 1 and Figure 2 need legends. It isn't clear what the different colors represent.

For Figure 3, either add legend for what the orange boundaries represent or include a description in the caption.

Line by Line Comments:

Line 112: I think a figure depicting the broader region and its different historic boundaries would be helpful here.

Lines 228-231: Do you know when these Bing images were taken? Are they all from the same time period or do they span multiple years or seasons?

Lines 243-244: "Bing images was taken in a different season..." Specify. Which season?

Lines 277-278: Please provide some additional details and justifications for why this architecture and backbone model were selected. Details on how they operate are needed for those unfamiliar with these kinds of methods.

Line 307: Can you indicate this on a map (either in an existing figure or a new one) to show the geographic coverage of the test areas?

Lines 346-347: "...different stochastic distribution of the color values found in the data." This is a particularly significant problem that this work (or future work) should address. Atmospheric corrections to normalize color values might significantly improve results in this case.

Lines 405-407: "These results suggest that a substantial speed improvement could be achieved in the overall mapping process if AI and human expert workflows are combined" This same argument has been made by several other studies as well. For example:

Davis, D. S. (2020). Defining what we study: The contribution of machine automation in archaeological research. Digital Applications in Archaeology and Cultural Heritage, 18, e00152. https://doi.org/10.1016/j.daach.2020.e00152

Zimmer-Dauphinee, J., VanValkenburgh, P., & Wernke, S. A. (2024). Eyes of the machine: AI-assisted satellite archaeological survey in the Andes. Antiquity, 98(397), 245-259.

Line 430: "...providing a reliable tool..." See my general comments, above, but is it an overstatement to call this tool "reliable", given that overall it only achieved an F1 of ~30-40%? This is why a better discussion of what is viewed as acceptable or reliable is needed.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Beyond the Greater Angkor Region: Automatic large-scale mapping of Khmer Empire reservoirs in satellite imagery using Deep Learning

PONE-D-24-26472R1

Dear Dr. Klassen,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Michael D. Petraglia, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: