PONE-D-21-40688How to Improve Representativeness and Cost-effectiveness in Samples Recruited Through Facebook: A Comparison of Facebook ToolsPLOS ONE

Dear Dr. Neundorf,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Mar 31 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yu-Ru Lin

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. ()

(A preprint of the preliminary results was published here: https://osf.io/3g74n/)

Please clarify whether this publication was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

5. We note that Figure 1 in your submission contain map images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license.  

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

6. We note that Figure A.1 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure A.1 to publish the content specifically under the CC BY 4.0 license. 

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission. 

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

7. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The manuscript entitled “How to Improve Representativeness and Cost-effectiveness in Samples Recruited Through Facebook: A Comparison of Facebook Tools'' shows the benefits of using various Facebook tools for survey recruitment. The use of Facebook advertisement campaigns has increased dramatically over the past decade, but more research is indeed needed to explore the best practices and benefits of using social media platforms to recruit survey respondents. This work contributes to the literature on online survey methodology by providing concrete and comparative examples of Facebook ads campaigns conducted in four countries utilizing different combinations of features and tools available on the Facebook Ads Manager. Differently from the existing literature and past survey experiments, authors propose to use another feature available on Facebook to reach users who are more likely to complete the questionnaire and therefore compensate for the survey effort and improve the cost-efficiency. Overall, the manuscript is well written, the analysis is rigorous and the literature review is comprehensive and adequate. There are a few minor points I’d recommend clarifying before publication.

- The procedure to link the “Conversation” option to the questionnaire completion (described in the Supplementary Materials) seems much less straightforward than using the “Traffic” option. It seems to require several steps, thus I would recommend the authors to briefly mention it in the main text, focusing especially on the risks of using this option, for example in the context of rapid surveys (e.g. during the first phase of the COVID-19 pandemic) when the data collection needs to be quick and timely. Is there any step in this procedure that may prevent researchers from collecting timely data?

- Structure of the paper: As I said, the manuscript is well written, but it’s quite long and wordy and the structure confusing and sometimes difficult to follow (for example, same headings repeated multiple times). Authors can follow the Plos One guidelines in terms of structure of the manuscript (e.g., Introduction, Material and Methods, Results, etc) to facilitate reading. Given that the results presented here are several, it’s good to have results and discussion together as it already is. However, in the Materials and Methods, I found it often difficult to separate among the literature review, the description of Facebook tools, authors’ speculation and the actual implementation of the study, which is then fundamental to understand the rest of the study. My suggestion is to have two separate sections, one describing the use of Facebook tools to reach survey respondents, including its use in previous studies, and the second one describing clearly and concisely the study design and the methodology tested in the analysis (e.g. combinations of features, targeting, etc).

- Please report the time period of the data collection also in the Methods section when describing the data collection.

- Please report the languages used in the survey experiments and how this may have affected survey responses.

- Page 15: please clarify the variables used in the post-stratification weighting approach

- Please describe the limitations of this study.

- Revise the text for typos, capital letters, and tenses.

Figures and Tables:

- Figure 1: I think it’s ok to show the heterogeneity of Facebook usage across countries and its potential reach. As a clarification, I would suggest separating explicitly those countries with no Facebook users (e.g. Cina) using a different color code in the map (perhaps gray). Not sure whether I missed it in the text, but I would also highlight the proportion of Facebook users in the four countries under study.

- Figure 2 is not very informative, especially since the various options are not described in the text. It can be moved into the Supplementary Material. On the other hand, in my opinion figure A.1 in the Supplementary Material is important when describing the study design. We know from previous studies how different ads materials (e.g. caption, image, etc) may affect survey participation and response, so this represents a rather important element when describing the data collection scheme.

- Table 1 is also not informative, especially because Modes 1 to 8 are not mentioned in the main text. I had to double check again in the text whether I missed their descriptions. It would be much more useful if the table reported the number of participants recruited instead of the labels “Modes”

- Tables 2-6: please double check as symbols % or £ are sometimes missing

Reviewer #2: This article provides guidance to researchers interested in collecting online survey samples via Facebook. The authors evaluate different implementation strategies using surveys collected in four settings -- the UK, Turkey, Spain, and the Czech Republic – with an eye to maximizing representativeness. The key conclusions of the article are that researchers should allow the Facebook algorithms to maximize for the campaign objective of conversion, and that to ensure cost-effective recruitment researchers should target single demographic characteristics of interest (rather than selecting respondents according to multiple demographic features).

In general, I found this to be a thorough and carefully crafted manuscript that provides concrete guidance to a growing body of researchers who are collecting data on the Facebook platform. While the article is highly applied and methods-focused with an eye to a very specific platform, I recommend it for publication because of its potential usefulness to researchers in this field. I suggest minor revisions below among three main lines: (1) discussion and framing; (2) improvements to figures; (3) minor corrections.

Discussion and framing:

• At the superficial level, one could argue that the results of this study are somewhat obvious – if you optimize for conversion, more people will convert, and if you engage in demographic targeting, your sample will be more diverse. Having worked with Facebook advertising myself, I understand why the answers are not obvious and the nuanced analysis in the paper is merited, but I am wondering if you could emphasize a bit the more surprising aspects of your findings (which come out in the main body of the paper, but perhaps could be highlighted more in the abstract, introduction, and conclusion). In my mind, these are:

o Reach is a completely ineffective targeting strategy. On the other hand, conversion targeting does not lead to bias when compared with traffic targeting. In particular, I found the following point really interesting: “it is important to note that traffic samples in these countries are eight to eighteen times smaller than conversion samples, making the absolute number of less educated and female respondents still smaller than conversion samples”.

o That targeting by a single demographic is (cost-wise) competitive with not targeting at all, whereas targeting by multiple demographics is very expensive.

� Do you have any sense of why this could be? I am wondering if Facebook might have incomplete demographic data on platform participants, so that by requiring multiple demographic characteristics there might be few people with complete enough records to be considered.

� I am wondering if there is a cost-effective strategy here that you could recommend. For example, one could run the survey for a few days with no targeting; then run single targeting for underrepresented categories; then run multiple targeting for categories that are still underrepresented. Would the data you have make it possible to calculate the cost for such a strategy, relative to just doing single or multiple or no targeting?

• At some point in the discussion section I think it needs to be stated that a limitation of the paper is the fact that the Facebook platform is (1) non-transparent and (2) constantly changing. In other words, it is very hard to know how Facebook targets respondents or determines their demographic features, and Facebook’s approaches can change overnight without notice. Therefore there is a possibility that the findings set forth in this paper may not hold true in the future.

• The paper ends pretty abruptly, saying that researchers should experiment with ad images and text. This may be personal preference, but I would like to see a final paragraph that provides more high-level commentary about the themes in the paper and ends on a more inspiring/impactful/meaningful note.

Tables:

• In Table 4, would it be possible to put the number of respondents in the final row? I know this is elsewhere in the paper, but I think having a sense of sample size would help put the differences in perspective.

• For Tables 3 and 6, I found these to be quite dense. As one simple strategy for improving readability, would it be possible to align all of the % symbols (right now the stars push them to the left)? Alternatively, I think it might be preferable to skip a table altogether and illustrate these graphically…

Minor:

• Typos: awarness (p4), poppulation (p11), costed (p14), “and and” (p21)

• Grammatically, I would say “no targeting” instead of “none targeting”

• On p18, when you say “12 versus 7 words”, I would clarify “on average”

• On p21, do you mean “convenience sample” instead of “convenient sample”?

• On p20 (and possibly elsewhere), I would say that “Facebook ADVERTISING is a highly valuable research tool”, rather than “Facebook is”, because I’d think of Facebook as a platform rather than as a tool

• On p20, I would spell out MTurk as “Mechanical Turk” since this is the first time it is mentioned

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-21-40688R1How to Improve Representativeness and Cost-effectiveness in Samples Recruited Through Meta: A Comparison of Advertisement ToolsPLOS ONE

Dear Dr. Öztürk,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Oct 29 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Yu-Ru Lin

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

In this revision, the author made substantial changes and I see the manuscript has significant improvement. One reviewer noticed a few presentation issues and suggested the authors carefully check the manuscript.

One of the previous reviewers was not able to re-review the manuscript, so I check the manuscript myself. I found the authors address most of the primary concerns, except that it is unclear whether the question about the language [1] has been properly attended to in this revision.

[1] The review comment: "Please report the languages used in the survey experiments and how this may have affected survey responses."

Based on the reviewer's and my own assessment, I would recommend the authors submit a revised manuscript for consideration.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The authors have satisfactorily addressed my major concerns and I feel that the paper is clearly written and straightforward. I don't see substantive conceptual barriers to recommending it for publication. However, there are still outstanding issues that should be resolved:

(1) The paper needs a thorough round of copy editing. There are still numerous mistakes and irregularities; I have flagged examples below but there were often more cases in the paper, so I think a careful proofread is necessary:

- "Lead some scholars to questionING" p2.28

- Subject-verb disagreement: "data collection through Meta platforms HAS" p2.42, "reach campaigns emergE" p3.63, "Table 4 demonstrateS," p10.371

- Unnecessary hyphens: e.g. "data-collection methods", "survey-completers", "link-clickers", "set-up"

- The word after a colon should not be capitalized

- Missing articles: e.g. "outside of THE Meta environment" p3.55, "THE Meta algorithm" p3.91, "on THE Qualtrics survey platform" p4.98, "all of THE targeted demographic categories" p4.118, "finally, THE Czech Republic" p5.147, "THE purposes of our research p6.199, "shown to THE most Meta users....provide THE most link clicks" p8.277, "seen by THE most Meta users" p8.281, "that THE reach campaign objective" p8.287

- I believe p4.105 should read "with reference to convention"

- I believe p4.115 and p4.141 should read "demographic characteristics", not "properties"

- I believe p5.175 should read "old", not "old-aged"

- I believe p6.214 "the divide of..." should be rephrased

- I believe p10.329 should read "AMOUNT of traffic"

(2) On p3.66, the authors state that "the former is usually not even more expensive than the latter" - I would specify "in our experience", because it is hard to conclude this more generally.

(3) Meta is misquoted on p3.83; this does not match the text in the source.

(4) On p4.110, "you can" sounds a bit too casual to me.

(5) The text refers to Turkey as a "non-democratic country"; while it is not an open society, I believe that factually (technically) it is considered a democracy. Similarly, the paper implies that Spain is between the UK and Turkey in terms of political openness, but is this true? In my understanding it would be as open/democratic as the UK.

(6) On p5.162 you say that "campaigns from the same country stayed open for the same amount of time", but Figure 2 suggests otherwise.

(7) I think Table 2 would be clearer if the left column (A/B/C) were dropped and the italicized entries were called "click-through rate" and "completion rate". Then, the note at the bottom of the table could explain how these were calculated.

(8) On page 18, the description of the weights is missing some punctuation/explanatory words.

(9) On p9.378, the table says that participants recruited through conversion spent more time on average, but this is only true in 2/4 contexts.

Based on the number of errors identified, I would suggest that the paper be carefully checked again before it is approved for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

How to Improve Representativeness and Cost-effectiveness in Samples Recruited through Meta: A Comparison of Advertisement Tools

PONE-D-21-40688R2

Dear Dr. Öztürk,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Yu-Ru Lin

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

I have checked the authors' response and revision myself, to avoid further delay on the reviewers' end. I think the authors have satisfactorily addressed the review comments from the previous round, and I'll recommend it for publication. 

I do have a minor suggestion for the authors. In response #6, the current text reads: "each of the campaigns running in a country stayed open for the same amount of time." I think the current text still seems to be confusing. To bring sufficient clarity, I'd suggest something like: "all campaigns running in the same countries stayed open for the same amount of time, but the open durations of campaigns running in different countries may vary -- e.g.,  all campaigns in the UK stayed open for 5 days, and all campaigns in Turkey stayed open for 4 days."