PWAT-D-23-00103

Improving drinking water consumer confidence reports: Applying user-centered design

PLOS Water

Dear Dr. Fox,

Thank you for submitting your manuscript to PLOS Water. After careful consideration, we feel that it has merit but does not fully meet PLOS Water's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by 03 November 2023. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at water@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pwat/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Ilunga Kamika, Ph.D.

Guest Editor

PLOS Water

Journal Requirements:

1. Some material included in your submission may be copyrighted. According to PLOS’s copyright policy, authors who use figures or other material (e.g., graphics, clipart, maps) from another author or copyright holder must demonstrate or obtain permission to publish this material under the Creative Commons Attribution 4.0 International (CC BY 4.0) License used by PLOS journals. Please closely review the details of PLOS’s copyright requirements here: PLOS Licenses and Copyright. If you need to request permissions from a copyright holder, you may use PLOS's Copyright Content Permission form.

Please respond directly to this email or email the journal office and provide any known details concerning your material's license terms and permissions required for reuse, even if you have not yet obtained copyright permissions or are unsure of your material's copyright compatibility. 

Potential Copyright Issues:

Figure 2: please (a) provide a direct link to the base layer of the map (i.e., the country or region border shape) and ensure this is also included in the figure legend; and (b) provide a link to the terms of use / license information for the base layer image or shapefile. We cannot publish proprietary or copyrighted maps (e.g. Google Maps, Mapquest) and the terms of use for your map base layer must be compatible with our CC-BY 4.0 license. 

Note: if you created the map in a software program like R or ArcGIS, please locate and indicate the source of the basemap shapefile onto which data has been plotted.

If your map was obtained from a copyrighted source please amend the figure so that the base map used is from an openly available source. Alternatively, please provide explicit written permission from the copyright holder granting you the right to publish the material under our CC-BY 4.0 license.

Please note that the following CC BY licenses are compatible with PLOS license: CC BY 4.0, CC BY 2.0 and CC BY 3.0, meanwhile such licenses as CC BY-ND 3.0 and others are not compatible due to additional restrictions. 

If you are unsure whether you can use a map or not, please do reach out and we will be able to help you. The following websites are good examples of where you can source open access or public domain maps: 

* U.S. Geological Survey (USGS) - All maps are in the public domain. (http://www.usgs.gov) 

* PlaniGlobe - All maps are published under a Creative Commons license so please cite “PlaniGlobe, http://www.planiglobe.com, CC BY 2.0” in the image credit after the caption. (http://www.planiglobe.com/?lang=enl) 

* Natural Earth - All maps are public domain. (http://www.naturalearthdata.com/about/terms-of-use/)

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does this manuscript meet PLOS Water’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Water does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This article presents the results of a 10-person focus group consultation on the design of public drinking water quality reports in Baltimore City, United States. Interviews and small group discussion were conducted, the conversations are summarized in the results section with an appended report providing more details about recommendations and design ideas for water quality reports.

There is no critical analysis and engagement with academic literature is very limited. This can be seen quickly in the reference list which includes only 6 peer-reviewed works, 3 of which are about the design method that informed the project. This material is well-suited to a project report, which has been prepared. It is not clear what added-value is provided by the article.

With the exception of one paragraph towards the end of the discussion section, this paper focuses only on the US context. Basic demographic information about the ten participants is not provided so the reader cannot know their gender, age, race, income level, education level, city zoning, etc. All these factors are important when considering how health risk information will be perceived - as is emphasized in the wider literature on environmental health risk reporting. Without understanding more about the positionality of the participants, representativeness of the Baltimore consultation results is unclear.

Detailed comments:

Participants were selected based on nomination by leaders of groups that have environmental health interests. How might this bias the results?

Were all interviews done and small-group discussions led by the same researcher? Were interviews recorded and transcribed? How was the summary of interviews prepared?

Lines 130-132 Choice of visual design ideas and recommendations to take forward was based on participant interest and time constraints of the study. How was participant interest gauged – was it

about the number of participants or the intensity of interest? How were different opinions balanced in this process? Were there potentially good ideas, or ideas of high-interest, that did not get taken forward to prototype due to time constraints?

Indicate the number of focus group participants in the abstract

Put citations before punctuation

Proofreading for grammar is recommended

Reviewer #2: The article presents a compelling case study on the application of user-centered design principles in the context of policy reporting. However, the research encompasses three distinct scopes, which can benefit from a more focused approach. These scopes include:

Utilizing User-Centered Design for Consumer Feedback:

The first scope centers on the utilization of a user-centered design process as a primary method to gather valuable insights from consumers. These insights play a pivotal role in enhancing the quality of reports generated by policy makers. To achieve a more cohesive and impactful analysis, this scope should be further developed, elaborating on the specific methodologies employed and their outcomes.

Addressing User Perception of Information Complexity:

The second scope delves into the complexities perceived by users when digesting information provided by authorities. This is a critical aspect of effective policy communication. To enhance the article's depth, it is advisable to dedicate a separate section or chapter to this scope. This section can explore user perceptions, cognitive load, and potential solutions to simplify information presentation.

Identifying Essential Information for Water Utilities:

The third scope revolves around identifying and presenting relevant information through the primary communication tool used by water utilities. This aspect is fundamental to ensuring effective communication between utilities and their stakeholders. To provide a more comprehensive analysis, it may be beneficial to separate this scope into its own section, allowing for an in-depth examination of the information needs of water utilities and the methods employed to meet these needs.

Methodology:

In addition to refining the focus of each scope, the methodology employed in the research should be elucidated with greater detail. This is crucial for understanding how the user-centered design process was leveraged as a tool to enhance information. To improve the methodology section:

Describe the specific user-centered design methods used, such as user interviews, surveys, usability testing, or persona development.

Explain the rationale behind the selection of these methods and how they were tailored to each scope.

Provide insights into the data collection process, including sample sizes, recruitment strategies, and data analysis techniques.

Highlight any challenges or limitations encountered during the research and how they were mitigated.

By offering a more comprehensive and detailed account of the methodology, readers will gain a clearer understanding of the research process and its potential for improving policy reporting.

In conclusion, the article presents a promising exploration of user-centered design in policy reporting. To enhance its impact, it is advisable to refine and separate the three scopes, dedicating more attention to each, and provide a more detailed account of the research methodology. This will contribute to a more robust and informative analysis of the subject matter.

Reviewer #3: I found this paper to be well written and clear, and the methodology and conclusions were interesting.

The only thing I found lacking (and the reason I have suggested a minor revision) is a description of the existing CCRs. I would have appreciated a better statement of the problem that the study set out to address - what exactly do the current CCRs look like, and what makes them so confusing to read? The comments from users do not provide much detail or insight. A paragraph explaining their content, detailing their failings and outlining why they do not meet users' needs would help.

I also found the explanation of why Baltimore was chosen a little unclear. The authors mention that Baltimore has experienced water quality challenges for years, but does not mention how many years it has issued CCRs or give any compelling explanation of why it was a good choice of location for the study.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes: Clarissa Brocklehurst

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

 

PWAT-D-23-00103R1

Improving drinking water consumer confidence reports: Applying user-centered design

PLOS Water

Dear Dr. Mary Fox,

Thank you for submitting your manuscript to PLOS Water. After careful consideration, we feel that it has merit but does not fully meet PLOS Water's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by 19 February 2024. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at water@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pwat/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

We look forward to receiving your revised manuscript.

Kind regards,

Ilunga Kamika, Ph.D.

Guest Editor

PLOS Water

Journal Requirements:

1. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article's retracted status in the References list and also include a citation and full reference for the retraction notice.

2. Please provide separate figure files in .tif or .eps format only and remove any figures embedded in your manuscript file. Please also ensure that all files are under our size limit of 10MB.

For more information about figure files please see our guidelines:

https://journals.plos.org/water/s/figures  

https://journals.plos.org/water/s/figures#loc-file-requirements

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Does this manuscript meet PLOS Water’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #3: I don't know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Water does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for your thorough responses to my comments and to those of the other reviewers. With the added details and discussion, the article has a lot more depth. This has helped to better contextualize the findings and make it easier for the reader to understand and potentially build upon the approach.

Reviewer #3: I commend the authors for the work they have done to address the reviewers' comments, and feel the paper is now significantly improved.

There are a few additional things I would like to point out, and suggest the authors include in a further minor revision.

-the improved explanation of the nature of both the survey participants and Baltimore as the locality that this study was undertaken in highlights how very context specific it was. This is not a bad thing, but really should be acknowledged. Neither the title nor the abstract mention Baltimore, and I feel strongly they should. for instance, the title could be reworked to "Applying a user-centred design to improve drinking water consumer confidence reports: a case study of Baltimore".

-in the end this is a relatively small sample size of respondents in one city, so important that the authors stress that this provides insight into the process that can be used to improve the reports, and does not necessarily provide a blueprint for reports in other cities. This is mentioned but only in passing and I feel should be stressed more.

-the authors have responded to my comment about why Baltimore was chosen with some passion (they speak of a "calling to help our community") but this has not found its way into the paper. Mentioning that you had an opportunity in Baltimore to develop this process because of a group of committed professionals would go a long way to explain to the reader why Baltimore is the locality studied. I like the fact that you mention that Baltimore residents have various vulnerabilities (racial, ethnic and language) but you have not made the connection with the failings of the current CCR. We now know that they require a college level reading ability but you have not explained why this is a problem given the demographics of Baltimore - are there any statistics on education levels in the city or among vulnerable groups there that could make it clear why the current CCRs are so inaccessible? (I'll also mention that you say that they are not accessible to visually impaired individuals but it's not clear why this is as significant a problem as being incomprehensible to most residents).

-I'm very glad that you have provided details of the makeup of the study participants but I take exception to your statement that "we believe the group if generally representative of Baltimore City residents" when 90% of the participants were Black and the city is 62% Black, and 70% were female and the city is 53% female. I'm not a statistician but I don't think you can argue that those numbers are "generally representative". However, I think you can point out that you have (perhaps purposefully) over sampled the people you think are most likely to be at risk of finding the CCRs difficult to understand. This is where some statistics on college education or reading levels among various subgroups would be helpful. This would be consistent with the opening introductory paragraph in which you identify communities of colour as a key vulnerable group when it comes to access to information.

-one final small point - you mention in the conclusion that the focus group participants "expressed concern and sometimes alarm about the information in the Baltimore CCR" but you could make this clearer that you mean alarm over the implications of the information (not how it was presented, which they also found concerning). (I assume this is what you mean).

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: Yes: Clarissa Brocklehurst

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Applying user-centered design to improve drinking water consumer confidence reports: a Baltimore case study

PWAT-D-23-00103R2

Dear Dr Fox,

We are pleased to inform you that your manuscript 'Applying user-centered design to improve drinking water consumer confidence reports: a Baltimore case study' has been provisionally accepted for publication in PLOS Water.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow-up email from a member of our team. 

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they'll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact water@plos.org.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Water.

Best regards,

Ilunga Kamika, Ph.D.

Guest Editor

PLOS Water

***********************************************************

Reviewer Comments (if any, and for reference):

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Does this manuscript meet PLOS Water’s publication criteria? Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe methodologically and ethically rigorous research with conclusions that are appropriately drawn based on the data presented.

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: I don't know

**********

4. Have the authors made all data underlying the findings in their manuscript fully available (please refer to the Data Availability Statement at the start of the manuscript PDF file)?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS Water does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: The authors have addressed all my comments, and I commend them for their patient and careful revision of the paper.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

Do you want your identity to be public for this peer review? If you choose “no”, your identity will remain anonymous but your review may still be made public.

For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Clarissa Brocklehurst

**********