PCLM-D-24-00204

Human-in-the-loop MGA to generate energy system design options matching stakeholder needs

PLOS Climate

Dear Dr. Lombardi,

Thank you for submitting your manuscript to PLOS Climate. After careful consideration, we feel that it has merit but does not fully meet PLOS Climate’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Your manuscript has now been seen by two referees, whose comments are attached below. You will find that they value your manuscript and generally evaluate it positively. While the referees' points are minor, I believe that addressing the points from referee 2 carefully will improve the manuscript.

Please submit your revised manuscript by Dec 19 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at climate@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pclm/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Florian Egli

Academic Editor

PLOS Climate

Journal Requirements:

1. We ask that a manuscript source file is provided at Revision. Please upload your manuscript file as a .doc, .docx, .rtf or .tex.

2. Please upload a copy of Figure 1 to 6 which you refer to in your text on pages 4, 7, 12, 13, 14 and 16 to file inventory. Or, if the figure is no longer to be included as part of the submission please remove all reference to it within the text.

3. Figure 6: please (a) provide a direct link to the base layer of the map (i.e., the country or region border shape) and ensure this is also included in the figure legend; and (b) provide a link to the terms of use / license information for the base layer image or shapefile. We cannot publish proprietary or copyrighted maps (e.g. Google Maps, Mapquest) and the terms of use for your map base layer must be compatible with our CC-BY 4.0 license. 

Note: if you created the map in a software program like R or ArcGIS, please locate and indicate the source of the base map shapefile onto which data has been plotted.

If your map was obtained from a copyrighted source please amend the figure so that the base map used is from an openly available source. Alternatively, please provide explicit written permission from the copyright holder granting you the right to publish the material under our CC-BY 4.0 license.

Please note that the following CC BY licenses are compatible with PLOS license: CC BY 4.0, CC BY 2.0 and CC BY 3.0, meanwhile such licenses as CC BY-ND 3.0 and others are not compatible due to additional restrictions. 

If you are unsure whether you can use a map or not, please do reach out and we will be able to help you. The following websites are good examples of where you can source open access or public domain maps: 

* U.S. Geological Survey (USGS) - All maps are in the public domain. (http://www.usgs.gov) 

* PlaniGlobe - All maps are published under a Creative Commons license so please cite “PlaniGlobe, http://www.planiglobe.com, CC BY 2.0” in the image credit after the caption. (http://www.planiglobe.com/?lang=enl) 

* Natural Earth - All maps are public domain. (http://www.naturalearthdata.com/about/terms-of-use/)

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Climate’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Climate does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In this manuscript, the authors present a unique approach to interactively incorporate human preferences in exploring cost-effective future energy system design options. The method is demonstrated through a case study of the Portuguese energy system. The quality of the text, method descriptions, graphics and overall research conducted is very high. It is rare that I review a paper without requesting revisions, but in this case, I believe the manuscript can be published as is.

The approach addresses a key shortcoming of previous studies exploring the near-optimal design space of energy system models. While earlier research often provided a wide variety of alternative solutions to achieve given emission reduction targets cost-effectively, it frequently remained unclear how this excess of choice could be translated into actionable consensus through stakeholder engagement. This manuscript proposes a method to make energy infrastructure planning more participatory and accessible to real-world use cases, while maintaining a high level of technical detail, addressing this limitation.

Reviewer #2: Dear Authors,

Thanks for the opportunity to review your manuscript "Human-in-the-loop MGA". The paper is timely and relevant, with several energy system modelling groups pursuing ways to leverage the potential of MGA to help decision-makers advance the energy transition. It is well-written and clearly explained, and the results largely support the conclusions and implications (see my comment 2).

Overall, I think the manuscript is excellent, presents novel and relevant research, and requires only minor revisions before it can be accepted for publication. I would like the authors to consider the following comments:

1. Previous suggestions for including stakeholders in MGA workflows. I am not an expert in MGA, so I may be wrong, but I find it surprising that this is the first paper to integrate stakeholder preferences into an MGA-based workflow since the 1980s (when MGA began being used). After doing some quick searches, I found at least one recent paper (1) that discusses the possibility of integrating stakeholders in an MGA-based workflow. There may be more papers, perhaps not only from the energy system design field. I would encourage the authors to expand on what prior literature may have already proposed and (if nothing is there) explain the surprising lack of previous attempts in the literature.

(1) Finke, J., Kachirayil, F., McKenna, R. and Bertsch, V., 2024. Modelling to generate near-Pareto-optimal alternatives (MGPA) for the municipal energy transition. Applied Energy, 376, p.124126. https://doi.org/10.1016/j.apenergy.2024.124126

2. The claim that the authors conduct "a controlled experiment" that "demonstrates that HITL-MGA facilitates consensus formation" seems exaggerated to me. When I read this, I expected the authors to have gathered one or more groups of stakeholders (or "fake" stakeholders, such as students primed to have a set of preferences) and demonstrated that groups using the HITL-MGA approach arrived at a consensual decision faster or more easily than groups that did not use it. This is far from what this paper does.

My main suggestion to the authors is to rephrase the parts of the paper that refer to the hypothetical facilitation of consensus-building because all they show is that the HITL-MGA approach generates alternatives that are more aligned with a set of scenarios selected from an initial MGA design space. I agree with the authors that it is plausible that this makes it easier for stakeholders to agree on a preferred solution, but the paper does not "demonstrate" that.

Minor comments:

3. As far as I know, the authors' claim that the HITL-MGA approach facilitates consensus formation is based on the larger number of alternatives that they call "near highest consensus". Is there any evidence to support that having more alternatives that perform well across several stakeholder preferences makes agreement easier (e.g., in decision-making literature)? I would find the argument much more convincing if the authors could show that scientific studies have shown that stakeholder decision-making is facilitated by having more options that closely map to their collective preferences.

As a potential counter-argument, there are suggestions in the literature that some actors purposefully avoid including some alternatives in their modelling (e.g., scenarios with large emissions reductions) (2).

One can imagine that *the lack of alternatives* that align with the preferences of some stakeholders (e.g., large emissions reductions) *makes consensus-building easier* because those stakeholders do not have alternatives that align with their preferences for which to advocate. In other words, if there were alternatives in the modelling that showed that large emissions reductions are feasible with similar costs, it would make it *harder* for stakeholders who prefer large emissions reductions to compromise.

What are the authors' reflections on this?

(2) Royston, S., Foulds, C., Pasqualino, R. and Jones, A., 2023. Masters of the machinery: The politics of economic modelling within European Union energy policy. Energy Policy, 173, p.113386. https://doi.org/10.1016/j.enpol.2022.113386

4. I would have liked to read a short discussion of the potential limits of the approach in integrating stakeholder preferences. Some questions that came to mind: What about mutually exclusive preferences? For example, how could the HITL-MGA manage a stakeholder that wants less renewable energy, no nuclear energy, and lower emissions? More generally, the five examples of high-level preferences make sense to me, but real-world stakeholders may hold ambiguous or even contradictory preferences. How could the HITL-MGA handle this? What if the stakeholder preferences (e.g., lots of new nuclear) would lead to scenarios that fall outside the cost relaxation (i.e. costlier alternatives than allowed in the HITL-MGA space)?

The authors do not have to address all these questions in the text. However, I would welcome a short reflection on the potential difficulties or limitations of integrating real-world stakeholder preferences.

5. I believe the authors kept the number of alternatives generated in the first sample and the HITL-MGA space similar (260 vs 270) to keep the computation effort similar. However, this may not be immediately clear to everyone. Please make the reason why the spaces have to be of similar size more explicit.

6. Why are all preferences weighted the same? I would find it interesting to "shape" the HITL-MGA after the strength of stakeholder preferences (e.g., to reflect strong opposition to onshore wind concentration but only a mild preference for a low reliance on hydrogen). This probably would change nothing in the method beyond the value of the weights, but it could be useful to mention for someone wanting to apply the approach to a real case with stakeholder involvement.

7. Minor typos and suggestions:

- remove unnecessary "more" in the first sentence of the abstract -> "hides from view many economically comparable."

- the first two sentences in the introduction make it sound like the insufficient progress is due to "such slow decision-making". Is it really because of slow decisions that progress is insufficient?

- in the first sentence of the 2nd paragraph of the intro: "optimal solutions" to what problem?

- what about time? Spatial and technology preferences are mentioned but not time. It would be good to mention that you only compare two snapshots in time (today and 2050 energy systems)

- what about spatial preferences? If I am reading equation 3 correctly, you did not include them; why?

- "slow transition rate" is misleading because you only observe the final snapshot (if I understood it correctly) without knowing if the changes in the model happened over 30 years or in one year, even though the changes are smaller than in other alternatives.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Fabian Neumann

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Human-in-the-loop MGA to generate energy system design options matching stakeholder needs

PCLM-D-24-00204R1

Dear Dr. Lombardi,

We are pleased to inform you that your manuscript 'Human-in-the-loop MGA to generate energy system design options matching stakeholder needs' has been provisionally accepted for publication in PLOS Climate. Congratulations on your work.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow-up email from a member of our team. 

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact climate@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Climate.

Best regards,

Florian Egli

Academic Editor

PLOS Climate

***********************************************************

Additional Editor Comments (if provided):

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Does this manuscript meet PLOS Climate’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Climate does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: Thank you for your efforts to address all my comments. I appreciate your detailed responses to my questions and suggestions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********