PCSY-D-24-00011

A Dynamical Systems Approach to Optimal Foraging

PLOS Complex Systems

Dear Dr. Chaturvedi,

Thank you for submitting your manuscript to PLOS Complex Systems. After careful consideration, we feel that it has merit but does not fully meet PLOS Complex Systems's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript within 60 days Jun 11 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at complexsystems@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcsy/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Frank Emmert-Streib

Section Editor

PLOS Complex Systems

Journal Requirements:

1. Please amend your detailed Financial Disclosure statement. This is published with the article. It must therefore be completed in full sentences and contain the exact wording you wish to be published.

a. State the initials, alongside each funding source, of each author to receive each grant.

b. State what role the funders took in the study. If the funders had no role in your study, please state: “The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

c. If any authors received a salary from any of your funders, please state which authors and which funders.

If you did not receive any funding for this study, please simply state: “The authors received no specific funding for this work.

2. We ask that a manuscript source file is provided at Revision. Please upload your manuscript file as a .doc, .docx, .rtf or .tex.

3. Please provide separate figure files in .tif or .eps format.

For more information about figure files please see our guidelines:

https://journals.plos.org/complexsystems/s/figures

https://journals.plos.org/complexsystems/s/figures#loc-file-requirements

Additional Editor Comments (if provided):

Dear Siddharth Chaturvedi,

Manuscript ID PCSY-D-24-00011 entitled "A Dynamical Systems Approach to Optimal Foraging" which you submitted to PLOS Complex Systems, has been reviewed. The comments of the reviewer(s) are included at the bottom of this letter.

The reviewer(s) have recommended publication, but also suggest some revisions to your manuscript. Therefore, I invite you to respond to the reviewer(s)' comments and revise your manuscript.

------------------------

Reviewer: 1

Recommendation: Major Revision

Comments:

Overall, the paper is well-written. My further comments are as follows:

1. Introduction should be extended by incorporating more details regarding the present research.

2. Include a pseudo code describing the various steps of the proposed method.

3. Comment upon the computational complexity of the proposed method.

4. Formulate the problem statement section mathematically.

5. Many intelligent modelling schemes exist in the literature such as Two Feedback PID Controllers Tuned with Teaching–Learning-Based Optimization Algorithm for Ball and Beam System, A recurrent neural network-based identification of complex nonlinear dynamical systems: a novel structure, stability analysis and a comparative study, Design of a novel robust recurrent neural network for the identification of complex nonlinear dynamical systems, Soft Computing Technique Based Online Identification and Control of Dynamical Systems. Improve your introduction by discussing these resources.

6. More in-depth discussion is required regarding the obtained results shown in Fig 2.

7. You should better highlight the performance specifications. Define the performance indicators for model evaluation properly.

Reviewer: 2

Recommendation: Accept

Comments:

The paper is interesting and very well written.

A few comments I have are as follows:

Page 3, lines 38-39: The authors could expand the thought in this sentence: "Such similarities can help us design artificially intelligent systems that are more explainable and predictable." This is interesting concept, but the reviewer is unsure about the link between existence of described phenomena and explainability of the agent.

Page 3, first sentence of an introduction: I have found this statement surprising. It is hard to believe that a complex phenomena like natural intelligence has such a concrete definition. Isn't it just one of definitions, just useful for this study?

Page 7: Equation (4) does not look linear while the comment before it suggests so. Please clarify this point.

Page 8: Abbreviation CTRNN was not introduced.

Reviewer: 3

Recommendation: Major Revision

Comments:

The manuscript reports results of a study into the emergence of adaptive foraging behaviour in artificial agents, which considered the agent and the environment as a coupled dynamical system.

The introduction includes a sufficient background material with respect to dynamical systems models of the considered problem. However, the motivation behind the choice of this general framework is somewhat lacking. I would suggest to explain why dynamical systems offer an adequate framework to investigate adaptive foraging, as opposed to other general approaches, including distributed Artificial Intelligence. Another conceptual gap is a comparison with swarm behavior where foraging is carried out not by a single biological organism, but by a colony, e.g., ant colony. In other words, placing the approach which is based on patch foraging with respect to say, the central place foraging theory or the ideal free distribution approach, would help the less informed general readership.

The paper is well-structured and well-written, with sufficient details provided in describing the model and implementation. While the mathematical formulation appears to be technically adequate, it would have been easier to understand the relationships among the key components of the model (position, controller, resources), if the meaning of coupling variables and other key dependencies were explained more intuitively.

The main approach taken in this study follows patch foraging methodology, within Optimal Foraging Theory. The Authors state that “Our focus is on modelling how adaptive foraging emerges in relation to environmental dynamics”. However, it is unclear what open challenges are being solved in this study, relative to the state-of-the-art. It would benefit the reader, if specific open questions are posed from the outset, and specific contributions of the study are listed with respect to these questions.

In general, it is nice to have a concise model which captures the salient factors in a minimal way. However, it is difficult to see how the presented results can be generalized to more complex setups, with competing agents. A policy which may be optimal for a single agent, may be completely inappropriate when there are multiple agents competing for the same resources. Similarly, even for a single agent, there may be non-trivial policies when the resources themselves are moving in complicated ways (e.g., predator-prey models). Again, it would have been easier to delineate these issues if the central place foraging or the ideal free distribution theories were mentioned earlier on.

I am not sure whether the Authors claim that their simulation which demonstrates the emergence of optimal foraging produces novel results in some specific aspects. In conclusion, they state “we aim to provide researchers with a new approach to studying how complexity emerges as natural agents adapt to their environment.” However, there are canonical studies, including experiments by Kacelnik et al., McNamara and Houston, and others, which date back to 1980s and 1990s. If the explicit model of the neurobehavioural component is suggested as a novelty, then this needs to be emphasized and explained more clearly.

The reported observation that “the learned control model uses a mechanism very similar to evidence accumulation” is an interesting one, and I suggest to expand on this potentially important connection more. Some of text in caption of Fig. 3 would fit better in the main text.

I would also suggest to explore the impact of noisy or imperfect information in the model, as currently there is no uncertainty in agent perception, decision-making or actions. The completely deterministic dynamics of the system, as the Authors note, allows for simple solutions, but does not clarify how these solutions may be generalized in more complex setups.

Thus, I would disagree with the conclusion that “The advantage of our dynamical systems approach to optimal foraging is that complex emergent behaviour can be studied using a minimal setup which only requires the numerical simulation of a system of differential equations and the optimisation of free parameters using automatic differentiation.” To study complex emergent behaviors one would need to develop a model accounting for imperfect sensing, processing and actions, as well as competition with other agents. Then, reducing, in a more general model, the noise to zero and the number of agents to one, would be expected to produce the results reported in this study.

As it stands, the manuscript describes a concise model demonstrating some emergent behaviour, in line with known results. I would suggest a major revision, significantly expanding the scope of analysis and results.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Complex Systems’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

--------------------

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

Reviewer #3: I don't know

--------------------

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

--------------------

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Complex Systems does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

--------------------

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: M.s: A Dynamical Systems Approach to Optimal Foraging

In this work, the authors have presented a novel approach to modelling this phenomenon in silico. They have achieved this by using a continuous coupled dynamical system for modelling the system. The dynamical system is composed of three differential equations, representing the position of the agent, the agent’s control policy, and the environmental resource dynamics. Crucially, the control policy is implemented as a neural differential equation which allows the control policy to adapt in order to solve the foraging task.

My comments are as follows:

Overall, the paper is well-written. My further comments are as follows:

1. Introduction should be extended by incorporating more details regarding the present research.

2. Include a pseudo code describing the various steps of the proposed method.

3. Comment upon the computational complexity of the proposed method.

4. Formulate the problem statement section mathematically.

5. Many intelligent modelling schemes exist in the literature such as Two Feedback PID Controllers Tuned with Teaching–Learning-Based Optimization Algorithm for Ball and Beam System, A recurrent neural network-based identification of complex nonlinear dynamical systems: a novel structure, stability analysis and a comparative study, Design of a novel robust recurrent neural network for the identification of complex nonlinear dynamical systems, Soft Computing Technique Based Online Identification and Control of Dynamical Systems. Improve your introduction by discussing these resources.

6. More in-depth discussion is required regarding the obtained results shown in Fig 2.

7. You should better highlight the performance specifications. Define the performance indicators for model evaluation properly.

Reviewer #2: The paper is interesting and very well written.

A few comments I have are as follows:

Page 3, lines 38-39: The authors could expand the thought in this sentence: "Such similarities can help us design artificially intelligent systems that are more explainable and predictable." This is interesting concept, but the reviewer is unsure about the link between existence of described phenomena and explainability of the agent.

Page 3, first sentence of an introduction: I have found this statement surprising. It is hard to believe that a complex phenomena like natural intelligence has such a concrete definition. Isn't it just one of definitions, just useful for this study?

Page 7: Equation (4) does not look linear while the comment before it suggests so. Please clarify this point.

Page 8: Abbreviation CTRNN was not introduced.

Reviewer #3: The manuscript reports results of a study into the emergence of adaptive foraging behaviour in artificial agents, which considered the agent and the environment as a coupled dynamical system.

The introduction includes a sufficient background material with respect to dynamical systems models of the considered problem. However, the motivation behind the choice of this general framework is somewhat lacking. I would suggest to explain why dynamical systems offer an adequate framework to investigate adaptive foraging, as opposed to other general approaches, including distributed Artificial Intelligence. Another conceptual gap is a comparison with swarm behavior where foraging is carried out not by a single biological organism, but by a colony, e.g., ant colony. In other words, placing the approach which is based on patch foraging with respect to say, the central place foraging theory or the ideal free distribution approach, would help the less informed general readership.

The paper is well-structured and well-written, with sufficient details provided in describing the model and implementation. While the mathematical formulation appears to be technically adequate, it would have been easier to understand the relationships among the key components of the model (position, controller, resources), if the meaning of coupling variables and other key dependencies were explained more intuitively.

The main approach taken in this study follows patch foraging methodology, within Optimal Foraging Theory. The Authors state that “Our focus is on modelling how adaptive foraging emerges in relation to environmental dynamics”. However, it is unclear what open challenges are being solved in this study, relative to the state-of-the-art. It would benefit the reader, if specific open questions are posed from the outset, and specific contributions of the study are listed with respect to these questions.

In general, it is nice to have a concise model which captures the salient factors in a minimal way. However, it is difficult to see how the presented results can be generalized to more complex setups, with competing agents. A policy which may be optimal for a single agent, may be completely inappropriate when there are multiple agents competing for the same resources. Similarly, even for a single agent, there may be non-trivial policies when the resources themselves are moving in complicated ways (e.g., predator-prey models). Again, it would have been easier to delineate these issues if the central place foraging or the ideal free distribution theories were mentioned earlier on.

I am not sure whether the Authors claim that their simulation which demonstrates the emergence of optimal foraging produces novel results in some specific aspects. In conclusion, they state “we aim to provide researchers with a new approach to studying how complexity emerges as natural agents adapt to their environment.” However, there are canonical studies, including experiments by Kacelnik et al., McNamara and Houston, and others, which date back to 1980s and 1990s. If the explicit model of the neurobehavioural component is suggested as a novelty, then this needs to be emphasized and explained more clearly.

The reported observation that “the learned control model uses a mechanism very similar to evidence accumulation” is an interesting one, and I suggest to expand on this potentially important connection more. Some of text in caption of Fig. 3 would fit better in the main text.

I would also suggest to explore the impact of noisy or imperfect information in the model, as currently there is no uncertaintyin agent perception, decision-making or actions. The completely deterministic dynamics of the system, as the Authors note, allows for simple solutions, but does not clarify how these solutions may be generalized in more complex setups.

Thus, I would disagree with the conclusion that “The advantage of our dynamical systems approach to optimal foraging is that complex emergent behaviour can be studied using a minimal setup which only requires the numerical simulation of a system of differential equations and the optimisation of free parameters using automatic differentiation.” To study complex emergent behaviors one would need to develop a model accounting for imperfect sensing, processing and actions, as well as competition with other agents. Then, reducing, in a more general model, the noise to zero and the number of agents to one, would be expected to produce the results reported in this study.

As it stands, the manuscript describes a concise model demonstrating some emergent behaviour, in line with known results. I would suggest a major revision, significantly expanding the scope of analysis and results.

--------------------

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

--------------------

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

A Dynamical Systems Approach to Optimal Foraging

PCSY-D-24-00011R1

Dear Mr Chaturvedi,

We are pleased to inform you that your manuscript 'A Dynamical Systems Approach to Optimal Foraging' has been provisionally accepted for publication in PLOS Complex Systems.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow-up email from a member of our team. 

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact complexsystems@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Complex Systems.

Best regards,

Shah Jamal Alam, Ph.D.

Academic Editor

PLOS Complex Systems

***********************************************************

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Does this manuscript meet PLOS Complex Systems's publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Complex Systems does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have well-revised their manuscript & as such I recommend its acceptance.

Reviewer #3: Thank you for the revisions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

**********