PONE-D-22-12822A novel machine learning approach to predict the export price of seafood products based on competitive information: evidence from the export of Vietnamese shrimp to the US market PLOS ONE

Dear Dr. Nguyen,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 04 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

José F. Vicent Francés, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating the following in the Acknowledgments Section of your manuscript: 

"This work was supported by the Hokkaido University DX Doctoral Fellowship [grant number JPMJSP2119]."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. 

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: 

"The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

"Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

Additional Editor Comments :

Due to reviewer feedback, my decision is Major Revision

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this paper, the authors use a super learner, with the aim of to give accurate and stable price predictions for Vietnamese shrimp products exported to the US market. In the paper, 10 algorithms are combined to predict the exported price of Vietnamese shrimp based on information from competitive exporters as China, Thailand, Indonesia, or India. In addition, a SHAP method is used to determine how each variable (predictor) influenced in the price.

In my opinion, the paper is interesting and novel but very simple. There are some technical weaknesses that must be solved and it will help clarify the visibility of the paper.

The use of acronyms is abused, which makes reading difficult. Please use them in necessary cases but not continuously.

First of all, the paper would have to be reorganized:

The Materials and Method section should be divided into Preliminaries, where all the methods and dataset used in the paper are included and Methodology, where they put

I am not sure if the variables the authors use (in table 1) are correlations or simple differences since the name is CorrelationPrice but the description talks about differences. It is mandatory to clarify that.

In line 209 the authors say that “After evaluating the suitability of the combination of algorithms…” but: How do the authors assess the adequacy of these algorithms? Why choose 10 algorithms if it is known in advance that some of them will not give precise results?

The inclusion of an explanatory flow chart is mandatory.

I don't understand the title of the Section “Results Machine Learning”. It must be changed.

Figures 2, 3, 4 and 5 are practically the same. That means that the predictions at 3, 6, 9 and 12 months based on super learner are practically identical. Is this temporary specification necessary?

Regarding the data, nothing is said about the amount of data used, nothing is said about data cleaning, nothing is said about whether this data is balanced or not.

An important weakness of the paper is the technical section. Then, nothing is said about the different types of algorithms used (they only describe a simple definition as Wikipedia) . They must specify much more the technical part of the data, the technical part of the algorithms used as well as the technical part of the super-learne. They are treated very weakly. For example, authors do not provide any table or description with the training parameters and the hyper-parameters of each model. This must be explicitly indicated.

Last, since the problem is a timeseries forecasting task, why authors have not used well-known algorithms in this field such as ARIMA, VAR or VARMAX models?

Reviewer #2: The authors use a super learner (regression model) as a predictor of Vietnamese shrimp prices as well as the SHAP method to estimate the most important features/factors that influence the predicted variable. The authors report strong results in the accuracy of the predictions and the feature importance analysis allows the highlight of useful information from the trained model. Their approach is interesting since the ability to propose (i) an accurate prediction of shrimp prices and (ii) the reasons behind the model prediction (or feature importance) can both be useful for producers or policymakers to develop more adapted strategies in the future.

Comment on data availability:

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction. The authors did not make all data used in their study fully available which thus prevents reviewers or readers from independently verifying the validity of their results. The authors write that the "Data cannot be shared publicly because it is sensitive and relates to Vietnam and other countries economics sector." but then that they have "we have some papers which relate this data, but have not been published yet". For me, these two statements may appear contradictory. Moreover, the data used in their study (which includes price, required farming certificates, and disease outbreaks from May-1995 to May-2019) seems to be published by international organizations such as the US Department of Agriculture (USDA), the Food and Agriculture Organization of the United Nations (FAO), and the World Trade Organization (WTO), and publicly available to consult or download. For example, the FAO statistics that are part of the data used by the authors seem to be publicly available (e.g. Global aquaculture production can be downloaded at https://www.fao.org/fishery/en/collection/aquaculture?lang=en). In my opinion, in order to show the reproducibility of their results, the authors should provide clear references to download the data as well as the source code of the scripts they used for data cleaning, features extraction, and classification. If, for reasons that should be approved by the journal, the authors are not authorized to reproduce and/or make the raw data used in their study available, they should at least provide an archive containing the extracted features used to train their models.

Comment on the reproducibility of results:

The authors did not detail enough the methodology used in their work in order to make it easily replicable. For each of their base models, the authors must also provide all information and/or parameters in order for the readers to be able to reproduce their results. A few examples include (but are not limited to) the depth of the decision tree, number of trees in the random forest (and other models), number of nodes per layer, error function, learning rate, optimization function.. for the neural network, regularization parameter C in the SVR, etc... Moreover, the authors should also specify if they were default parameters or, if not, how those parameters were selected. The source code of the scripts they used for data cleaning, features extraction and classification would greatly help in this regard.

In this state, the paper does not offer the required information (data and methodology) to replicate the proposed results and that essential information constitutes one of the most important prerequisites that should be met before this work can be accepted for publication.

Other comments about the paper:

1. The title says "evidence", but it is not clear what kind of evidence it's referring to. Moreover, the super learner approach is hardly novel in machine learning and should not be referred to as such in a general sense. If the authors referred to the first time a super learner is used to specifically predict shrimp prices, they should be more precise so that the reader is not misled to think that the super learner proposed by the authors is a novel approach in machine learning.

2. Page 8, "Finally, we obtained 13 independent variables for use in the super learner process to predict the Vietnamese export price": the analysis resulting in the choice of variables is interesting and is clearly described, however, how did the authors determine that these 13 variables were independent? The authors should specify the statistical tests that were performed, describe the methods used to select these tests, and publish the precise results of their statistical analysis that lead to the conclusion that these 13 variables were indeed independent.

3. Page 8: moreover, in addition to the analysis proposed to determine the choice of selected variables, the authors should also support their choice by performing a statistical analysis or a forward feature selection (or any other traditional feature selection technique, e.g. see https://doi.org/10.1016/B978-0-444-81892-8.50040-7). These kinds of analyses are helpful to quantify the importance of all variables and validate the choice proposed by the authors or discuss why the choice proposed by the authors varies from these more traditional methods. For example, in the discussion, the authors state that the price of Indian shrimps significantly affects the price of Vietnamese shrimps. It would interesting to perform a correlation analysis between the price of India and Vietnam and report the result. Additionally, a forward feature selection analysis (and the evolution of prediction accuracy when incrementally adding more features) will also show if the model would already perform well with a smaller number of features (or even with just the price of Indian shrimps) or if adding more features was essential to reach the excellent performances reported by the authors. This type of analysis becomes increasingly important in machine learning and, in my opinion, should be systematically performed.

4. Pages 10-14: if the target audience is machine learning experts, the description of the base models written by the authors is too general, often simplistic, and offers little useful information (no detailed explanation on the choice of parameters, model architecture, why the model was specifically chosen, etc..). If the target audience is not supposed to have prior knowledge of machine learning, then the descriptions should be easy and clear to understand. But at the risk of repeating myself, in addition to the model's description, authors are required to provide all the necessary information about their models to ensure the reproducibility of their results.

5. Page 11: Formula (1): please specify what p and n stand for. Formula (2): I would suggest writing beta under the argmin function to make it explicit that the argmin function finds the beta that minimizes the sum. Please also specify the N parameter so it doesn't get confused with the n of the first formula (you may also consider changing the n of Formula (1)).

6. As the authors correctly stated, a super learner model is at least as good, and (hopefully) better, than any individual models that compose it. Thus, comparing the super learner results to the candidate models does not give any information to the reader concerning the performance of the authors' approach compared to state-of-the-art models for price predictions. I think that it would have been much more interesting and informative, in order to evaluate the performance of their approach, if the authors would also have compared the super-learning results with state-of-the-art models for price predictions such as LSTM/GRU, ARIMA, etc.. (as done in https://doi.org/10.1007/s00521-020-05172-3).

7. Page 21: the sentence "These performances proved that the super learner is a novel approach that produced the predictions in this study." should be rephrased. Did the authors want to emphasize that the super learner produced the best predictions? Plus, although the authors demonstrated the superior results of the super learner compared to any of its composing predictors, as stated earlier, the super learner approach is hardly novel in machine learning and should not be referred to as such in a general sense by the authors.

8. Line 460: "contributing 70% of the value of India’s exports": Did the authors want to say "India's seafood exports"? Because the sentence tends to state that it's 70% of India's total exports.

9. Figure 2-5: price should have an uppercase 'p'

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A novel machine learning approach to predict the export price of seafood products based on competitive information: The case of the export of Vietnamese shrimp to the US market

PONE-D-22-12822R1

Dear Dr. Nguyen,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

José F. Vicent Francés, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: I believe the authors have adequately addressed the comments raised in the previous round of review. I particularly appreciate that the authors made their dataset fully available and provided all technical information to ensure the reproducibility of their results. For me, this manuscript is now acceptable for publication. Thank you to the authors for their interesting work.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Cédric Simar

**********