PONE-D-21-34079ORION software tool for the geometrical calibration of all-sky camerasPLOS ONE

Dear Dr. Antuña-Sánchez,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The material in the paper looks interesting and sound, although the manuscript should be revised carefully to meet PLOS ONE publication criteria. 

The authors should put an effort to improve the paper by taking into account the comments of all the referees, in particular those by Reviewer 1 who raised important remarks particularly related to the methodology used for the computational experiments, and to the presentation and the use of English language in the paper. Some sentences are not easy to read, and some typos are still present at places. It would help if you had a proof check of the English by a professional. Overall the authors should carefully check the English of the paper more deeply. 

Furthermore, the scientific principles described in the paper look sound, but more statistical analysis should be performed as suggested by the reviewers to enhance the manuscript.

Please take carefully into account the comments of all the referees for improving the manuscript to meet PLOS ONE standards before resubmitting it to the journal.

Please submit your revised manuscript by Jan 13 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Sergio Consoli

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"This research was funded by the Ministerio de Ciencia, Innovaci´on y Universidades (grant no. RTI2018-097864-B-I00) and by Junta de Castilla y Le´on (grant no. VA227P20). "

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"No, The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: COMMENTS

This paper is on developing an open-source software for geometric calibration of all-sky cameras. This is an interesting application for publication in PLOS ONE. Authors argue that knowledge about the sky coordinates represented in each pixel of an all-sky camera is important for many applications.

In the introduction section, the authors give useful background about the all-sky cameras and geometric calibration which is the topic of the current paper. They introduce ORION to fill the gaps about geometric calibration of the sky images. A relevant question is, is there any other software (open access/paid) which can be used for calibration, over which this software is a major advancement?

The authors describe the instrumentation, the workflow and the theoretical principles behind ORION adequately in Section 2. In Section 3, the authors present the use of ORION.

Specific comments:

Numerous places where writing style cannot be easily understood has been corrected for in the manuscript. (See the notes in the PDF). Additional comments/suggestions are suggested below.

Rewrite L33-36. Rewrite as per the suggestion in the PDF.

Rewrite L37-40 something like “Extrinsic camera calibrations consists of determining camera orientation and requires dentification of the positions of the Sun [32, 35, 36] or any star [28, 37, 38] in several images and the correlation of these positions with the Sun or star coordinates.”

Rewrite L64 as “It is formed by a SONY IMX178 RGB CMOS sensor with a fisheye lens, both encapsulated in a weatherproof case with a BK7 glass dome on top.”

Rewrite L66-67 as “The camera can capture image with size of 3096 X 2080 pixels, a pixel scale of 5.4 arcmin/pixel and 14-bit resolution”.

L79-81- It will be useful to show the mathematical equation to calculate azimuth and zenith value of star from the information about the observer.

Rewrite L87-90 “As several images obtained at different times is used in the geometrical calibration to cover a wider range of pixel positions and angles, the ORION user must put this image set in a folder and introduce the folder path in ORION.”

L124-L125. Can authors present some statistical analysis (maybe in a Supplementary section) of how the accuracy varies with the size of the square box. The idea is to provide some convincing evidence of assuming this value in the software. I assume the users cannot change this value when theyb are turning the software.

More description about the image 1 is needed. Example, what does “Find the start position mean”. What does “Find Max Bright point. Is it right?” mean. I believe that an image is discarded if we cannot find the bright point.

L235-236 – The authors mention that they chose the image not having moon. I believe their analysis and software would not be useful if the images have moon?4

Overall impression

The scientific principles described in this paper are sound, however more statistical analysis has been suggested in a few places which can enhance the manuscript. Further, the language and writing style makes the MS difficult to read and make sense of the meaning in many places. Some of these mistakes have been corrected in the main text, however it is recommended that the author gets the manuscript checked and proofread by a native English speaker or by a professional editing service who can put the sentences in order to enhance the understanding. A major revision is suggested for the authors before the manuscript can be accepted. The authors should be more aware of scientific writing versus the free style writing which has been used in the MS.

Reviewer #2: This paper developed a new open-source software, ORION, to conduct geometrical calibration to all-sky cameras under restricted conditions (e.g., cloudless). From the writing perspective, it is good to provide the principles underlying the software and an example to show how it works. However, I think there are too many operation-level description (e.g., what button to click in this software), which makes some part of the paper more like a user guide. It would be improved if the authors deliver it from a perspective of an academic study instead of from a user guide scope (see detail in section-wise comments). A few issues about formats (e.g., thousand separator) and grammar need to be double checked (see detail in line-wise comments).

Section-wise comments:

2.4 Too detail in terms of the techniques (e.g., different forms of output). It makes it more like a user guide not academic study

3 Some operations may not be needed because that is more like a user-guide. For example, you mentioned what will happen after you click a specific button (e.g., “Start” and “Check” buttons).

Figure 1 You may want to include a legend for different shapes in your flow chart.

Figures 3-9 I do not suggest using screenshots of the software to deliver the functions because that makes it like a user guide. You may simply separate different functions or different stages of processing to different sections in your paper and simply visualize the results of each stage (or functions) you get, like Figure 2.

Line-wise comments

47 python3->Python 3

47 please give a reference link to Qt5. I am not sure what it is.

59 This is the second time you mention GOA-UVa but your first time mentions the full name. You may want to mention it in line 51

67 3096 X 2080 -> 3,096 X 2,080. You may need thousand separators.

77 a space before “To”?

157 You may want no indent before “where”

183 10,201thousand separators

184 10,201thousand separators

190 10,201thousand separators

338 difference -> different?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Tianyang Chen

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Attachments

Attachment

Submitted filename: PONE-D-21-34079_annotated.pdf

ORION software tool for the geometrical calibration of all-sky cameras

PONE-D-21-34079R1

Dear Dr. Antuña-Sánchez,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Sergio Consoli

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript is acceptable for publication now. The author has done a novel work in their field and PLOS ONE is a suitable jpurnal

Reviewer #3: This paper describes ORION, an open source software package created to calibrate all-sky cameras. ORION uses a set of photographs of the nighttime sky along with the known positions of stars to determine a mapping between image pixels and sky coordinates. ORION is meant to be low-cost, accurate, and easy to use. After a discussion of the theory behind geometric calibration, the paper presents an example of using ORION to generate the calibration matrices from a specific set of photographs.

L15: A definition/description of "hot pixels" would be useful to the reader.

L17: "kind of instrument", not "kind of instruments"

L31: "same size as the camera images", not "same size than the camera images"

L99-100: "on an image-by-image basis," not "on an image-by-image,"

L108: Add a space between the degrees symbol and the word "in". Missing spaces after a degree symbol also occurs on L135 (twice), L137, and L280 (twice).

L309-315: How do ORION's results compare to current geometrical calibration systems (mentioned on L38)?

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No