PONE-D-24-37729Fact or folklore? A Quasi-Experimental Investigation into Taylor Swift's Influence on American football performancePLOS ONE

Dear Dr. Smoliga,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Feb 17 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Mohsin Shahzad

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for uploading your study's underlying data set. Unfortunately, the repository you have noted in your Data Availability statement does not qualify as an acceptable data repository according to PLOS's standards. At this time, please upload the minimal data set necessary to replicate your study's findings to a stable, public repository (such as figshare or Dryad) and provide us with the relevant URLs, DOIs, or accession numbers that may be used to access these data. For a list of recommended repositories and additional information on PLOS standards for data deposition, please see https://journals.plos.org/plosone/s/recommended-repositories.  (JNC (Straive) 26 Nov 2024: Potentially unacceptable repository)

3. Please include a copy of Table 1 which you refer to in your text on page 12.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: GENERAL COMMENTS

Thanks for the opportunity to read and review this paper. I quite enjoy the notion of using pop culture references alongside a little bit of a satirical take to highlight the importance of appropriate scientific communication and analysis of data, while also working in sports science myself – so I’d say this paper sits right in a niche ally of interest for me. Given the nature of this paper, I haven’t necessarily been overly critical in my review of the science, but in saying that this is a well written and conducted scientific study, albeit of a topic not overly important at face level. I think the premise of this paper is highlighted quite well in the authors approach, but there are opportunities to be more explicit (see specific comments on title and potential need for case study examples from medicine and health at different points). I’ve also left some detailed comments on how certain choices in the paper and communication of methodology is perhaps a little on the light side, and hence falls into a different but also important trap in scientific writing and research (i.e. isolated focus on individual outcomes, detail in replicating methods). Nonetheless, this is an enjoyable and easy read that uses a unique approach in highlighting a relevant problem to scientific communication to the media.

Specific comments on each section are provided below.

I recently received a review on one of my own papers where the reviewer provided an ‘about me’ section, which I found quite valuable in placing additional context around their comments, so it is something that I am trying to add to my own reviews:

I have a research background primarily in sports injury and performance biomechanics, but have a secondary research focus on using data and analytics approaches to characterise and improve sports performance. My work in this area has predominantly focused on netball, with a little in Australian football.

I am also open to being identified on papers I reviewed – so am signing off my review here.

Aaron Fox

Centre for Sport Research

Institute for Physical Activity and Nutrition

Deakin University

SPECIFIC COMMENTS

Title

I think the title should incorporate some element that highlights the broad notion that this paper is trying to highlight (i.e. parallels to simplification or fallacies in clinical and medical research). In it’s current form, I feel that a number of people who have zero interest in Taylor Swift and/or American football might look at this and simply skim over it, without understanding it’s broader purpose as a commentary of sorts for medical research (e.g. ‘drawing parallels to medical research and clinical practice’).

Abstract

I feel the statement “Kelce achieved an extra mean 7.1 (95% confidence interval: -12.7, 26.9) yards per game when Swift was in attendance (n=13 games in the Swift era), compared to matched games from the pre-Swift era however this was not statistically significant (p=0.476).” in the abstract has the potential to have the reverse effect of what you are trying to achieve with this paper – in that a number of people might only take in the details of the first half of this statement and simply fall victim to the “Swift effect” (i.e. ignore the aspect that it is not a statistically significant effect). You could apply a similar notion to a follow-up statement of when Swift was not in attendance. My thoughts would be that this ‘meatier’ discussion could be incorporated in the body of the paper, but the abstract should be incredibly clear that you are saying no effect exists – given that a large proportion of your audience might just read the abstract alone.

Introduction

The point on the paper being serious and inclusion of Taylor Swift references in the paper is perhaps valuable for the review, but I would hope that it gets removed if the paper is accepted and the in-text notations (i.e. red text) be removed. I think it’s far more fun to have this in a subtle manner.

I think the introduction could benefit from highlighting some actual examples of media miscommunicating or jumping to conclusions around healthcare that isn’t supported by science. You’ve cited some references (i.e. 11-13) here but I think it would be worth highlighting these specifically to demonstrate the importance.

Methods

I think early on in the methods a more detailed (e.g. 3-4 sentence) general description of the propensity score matching technique would be helpful.

In data collection section, it would be useful to see here the exact performance metrics and game outcomes (e.g. win/loss vs. margin) that were extracted – and also to provide a justification for their use. There are a huge amount of data points for receiving players, so justifying which ones you selected would be a good addition. Reading ahead this is done later in ‘step 3’ of the next section – but it may be preferable to highlight and define the performance outcome up here. On a similar point, the selection of one variable (while justified) might be too simplistic. For example, Kelce may have had a bad receiving game, but had a good blocking game (another role of the tight end) – and hence looking at other metrics like the number of pressures allowed on the quarterback would add to this. Reading on towards the end of the discussion I see that this is a potentially selective choice – but I still wonder whether the isolated focus is appropriate.

The online news sources could use a reference or be explicitly mentioned so that readers could check these data.

It would be nice to understand why only the pre-game Elo was used to control for matches. While it is noted as the strongest confounder, presumably other factors were also relevant and matching across multiple variables would be possible? To a larger point that the paper focuses on, being selective in your matching of games via a simplistic process might also skew your conclusions. I see later on that a stepwise regression was used to come to this conclusion, but only the Chief’s pre-game Elo data is reported in the table. It would help to have the other information in the results to support this decision. For example, does inclusion of the other factors help explain even a little more of the variance above 6%? In addition, I’d include this regression table in the methods to support/justify the decision.

Results

“All games from the Swift era were successfully matched” – is there a quantitative way in which this is acceptable, or is it more qualitative in viewing the overlap of data? Some more detail here would be useful.

Any time where differences are reported (e.g. “Kelce achieved 7.1 yards per game more in the Swift-era than he did in matched games”) I think it would be worthwhile to report the variance around the mean difference (e.g. 95% CI’s). This would probably help highlight the point better, as the lack of statistical significance would suggest some wide confidence intervals in these data.

Discussion

Similar to a point in the introduction, I think the discussion would benefit from some examples or case studies from the medical and health research field that convey the point you are trying to make in the relevant sections. You have citations to support the ideas/points, but presenting some authentic and eye-catching examples would help the cause.

In line with the above comment, finishing a paragraph with the statement “Stakes in Super Bowl betting may not be critically important for society, but the phenomenon of making decisions, including financial investment, based on unsupported claims is not confined to the realm of sports” feels a little incomplete. It likely leads onto the next section, but at least providing the examples outside of the realm of sports here would be nice to see.

“or mistresses from illicit affairs” – perhaps clarify this as ‘potential’ or ‘hypothetical’, you don’t want to assume that players are having affairs.

Reviewer #2: I write to submit my report on the protocol titled “Fact or folklore? A Quasi-Experimental Investigation into Taylor Swift's Influence on American football performance”

The study determine whether the media narrative that Swift’s presence at NFL games had any impact on player or team performance – and draw parallels to scientific journalism and clinical research.

This is an interesting paper and I have few comments as follows

General comments.

The authors did not assess the quality of the resulting matched samples. There was no mention of what statistical method was used to assess the quality of matched samples. No results (numerical or graphical) were provided on whether the matching improved the similarity index between pre-Swift and post-swift era

Methods

5: 1 matching could increase bias (Poor matches) BUT High Power due to increase in sample size as the authors indicated. Sensitivity analyses must be conducted by varying the number of matched samples to determine the extent to which the impact estimate varies. For instance, will 1:1, 2:1, 3:1, and 8:1 give us the same effect size estimate? This will also help determine whether if effect size estimate is not driven by ratio of the matched sample. What determine a narrow range? What is the reference value of what authors defined as a narrow range. Sensitivity analysis of varying the ratio may be better. The optimal matching approach is also based on the ratio of the matched sample and different ratio could lead to different optimal width.

Matching with replacement can often decrease bias because controls that look similar to many treated individuals can be used multiple times. The implication of matching with replacement and matching without replacement must be discussed. Again, would the results be different if the authors have adopted matching with replacement.

Struggling to understand why the approach is propensity score matching. Propensity is the probability of intervention (Swift appearance during Football games) given certain observed covariates. It is not clear how the authors generated the propensity scores. The Elo scores are exiting database used for the matching but I am not clear how the author generated the propensity scores. It could be, say, nearest neighbor matching, caliper matching, or optimal matching based on the Elo scores, but my confusion stems from the classical definition of propensity scores. A detailed explanation must be provided

Results

The figures in the paper are not visible at all

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Aaron Fox

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

The Folklore of the “Swift” Effect – Lessons for Medical Research and Clinical Practice

PONE-D-24-37729R1

Dear Dr. Smoliga,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Erfan Babaee Tirkolaee, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: The authors have addressed all comments to my satisfaction and also provided detailed clarification when the need arise. The manuscript has improved significantly

Reviewer #3: Please Acknowledge this limitation more directly in the Methods and not just in the Discussion.

Explain why receiving yards and win-loss were the only performance metrics.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Aaron Fox

Reviewer #2: No

Reviewer #3: Yes: Dr Aman Suresh T

**********