PONE-D-24-06886The Relationship between Line Manager Training in Mental Health and Organisational OutcomesPLOS ONE

Dear Dr. Blake,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Reviewer 1The paper "The Relationship between Line Manager Training in Mental Health and Organisational Outcomes" examines the effects of managerial training on organizational outcomes. While the introduction adequately establishes the research, incorporating a discussion on existing mental health training practices could improve its context. The literature review could be strengthened by including a variety of viewpoints and conflicting research according to the variables. The methodology section, centered around probit regression, could benefit from a discussion on its limitations and potential biases and it could be analyse using confirming statistical multivariate analysis methods. The results section is well-articulated and comprehensive, though adding visual data representations and concentrating data in tables can enhance understanding. The conclusions effectively recap the study’s findings; however, elaborating on their practical applications could enhance their utility in real-world settings; increasing references to previous studies are needed to confirm research assumptions. Finally, the references are comprehensive, yet updating them to include more of the latest studies would ensure the research's relevance and show the top industry standards.

Suggestions for Enhancement:

Introduction:

I'd like you to add insights on prevailing mental health training practices in the industry to anchor the research's significance better. Include a clear definition of your research questions, objectives, and hypothesis, perhaps in a table.

Literature Review:

Broaden the scope to include diverse research and a more comprehensive array of theoretical perspectives to enrich the narrative. Explain each of your research variables and define a table of authors where you include the top publications for each variable and dimension of your study.

Methodology:

Include more details on the selection of probit regression and address any inherent methodological weaknesses or biases. Explain why it was the best alternative for analyzing this data and if there's any other possible alternative method to validate your assumptions.

Results:

I'd like you to introduce graphical elements like charts or graphs to summarize and clarify key findings, compare your results with previous research, and determine if this study covers any gap in theory, methodology, or research methods.

Conclusions:

Detail the practical implications of the research findings, suggesting specific training or policy changes that could be derived from the study. Describe the stakeholder who are having the best use for your fingins. Describe how you achieve your results and goals.

References:

Update and verify that all references reflect the most current research, ensuring the study's alignment with the latest developments in the field.Reviewer 2This is an interesting study investigating the organizational level effects of line manager trainings for mental health. The authors found a way to assess these outcomes by analysing data from an existing dataset (panel survey data from firms in England) thereby adding important information about the effects of these trainings that go beyond an assessment by the participants.

The manuscript is well written and clearly structured and adds important information to this area of research.

However, I have some comments that I would like the authors to respond to before publication:

The authors are right in pointing out (page 5 line 90) that only few studies have examined the impact and influence of LM training on organisational-level outcomes (e.g., changes to productivity, turnover rates, and absenteeism). However, also the impact of LM training on employee outcomes is often not assessed. This also applies to the “Employee-level benefits” that the authors describe have been found (page 5 line 85). It does not look like employees have been asked about these benefits. Instead, the results describe managers’ own assessment of their gained competencies, their intentions to promote mental health at work and their increased confidence to support employees with mental health problems. I understand that this often is what can be measured in studies about manager training as asking employees would often result in a much larger study. However, since the authors address this topic I think it is important to mention that we actually know surprisingly little about the effects of LM trainings for employees, as most studies assess how the participants themselves (the managers) evaluate what they gained from having been trained. Results often show increased knowledge, confidence and good intentions, however, we most of the time do not know if this leads to actual implementing this in daily practice (for example more support for employees with mental health problems) and we know even less about if this leads to better outcomes for employees.

This lack of knowledge of manager training for employee outcome is discussed in for example the two papers below but can also be found in several reviews about manager training for employee mental health:

Rugulies, R., Aust, B., Greiner, B. A., Arensman, E., Kawakami, N., LaMontagne, A. D., & Madsen, I. E. (2023). Work-related causes of mental health conditions and interventions for their improvement in workplaces. The Lancet, 402(10410), 1368-1381.

Aust, B., Møller, J. L., Nordentoft, M., Frydendall, K. B., Bengtsen, E., Jensen, A. B., ... & Jaspers, S. Ø. (2023). How effective are organizational-level interventions in improving the psychosocial work environment, health, and retention of workers? A systematic overview of systematic reviews. Scandinavian Journal of Work, Environment & Health, 49(5), 315.

My next comment is about missing data. The authors report on page 11, line 200 that the sample consisted of 7139 participants after the four waves were merged. However, the numbers of answers for the different questions are much lower. In Table 3 the number of responses for the sickness absence related outcomes are between 1116 and 3566. In Table 4 the numbers for the responses for the organisational-level outcomes are between 3178 and 3189. Does that mean that only around half of the participants answered these questions? Please explain.

In addition, the number of answers for the questions about the proportion of sickness absence due to mental health is particularly low (n=1116). Do you have any explanation for why this question was not answered by most of the participants? An explanation could be that employers might not have that information. They might only have information about sickness absence in general, but not about the reason for sickness absence.

On page 10, line 186 the authors write: “As pooled panel data were used, we did not employ specific strategies to address missing data in the analysis as any missing data points were inherently handled through the nature of the dataset.” Could you please explain, what is meant by “were inherently handled through the nature of the dataset”?

Response rate: As reported in the result section (page 11, line 202), the response rate for the four waves was between 17% (2020) and 15% (2021-2023). This low response rate together with the high degree of missing data among those who answered should be addressed as a limitation of the study in the section strengths and limitations.

It is however positive that the sample included a lot of small to medium sized workplaces (SMEs). In fact, around 80% of the participating organisations had less than 50 employees. Most studies about mental health at work are conducted in larger organisations, so it is clearly a strength that this study includes many SMEs that often have fewer resources to be active in mental health promotion. This strength could also be mentioned in the discussion part. Another strength is the mix of sectors, as there often is an overrepresentation of studies about mental health at work conducted in the health care sector. In this study, it is therefore good to see that sectors such as production and construction are represented. (See for example the study by Greiner et al. that calls for more research about mental health interventions in the construction sector. (Greiner, B. A., Leduc, C., O’Brien, C., Cresswell-Smith, J., Rugulies, R., Wahlbeck, K., ... & Aust, B. (2022). The effectiveness of organisational-level workplace mental health interventions on mental health and wellbeing in construction workers: A systematic review and recommended research agenda. Plos one, 17(11), e0277114.).

However, it would also be good to know how many organisations actually offered Line Manager training in mental health, that is, how was the question “LM training in mental health” (page 8, line 158) answered by the organisations? Could you report this and also shown it by size of organisation? This would be very valuable additional information for this topic: How widespread are LM training in mental health in these organisations (especially in in this sample representing sectors often not studied)? Are LM trainings just as common among SME businesses than they are among larger organisations? Please add this information.

Another issue that should be addressed is that at least some of the survey waves were conducted during the COVID-19 pandemic (see page 7, from line 131). The first wave (January to March 2020) was probably not affected, especially since the questions are about the past 12 months, so before the pandemic came to Europe. However, wave 2 and 3 conducted in 2021 and 2022 and thereby asking about 2020 and 2021 were conducted while most workplaces were affected by the pandemic in one way or the other. The authors probably have no specific knowledge about how this affected these workplaces, but it could be mentioned that the need for more awareness for employee mental health might have been even higher during these years (because for example more social isolation) and therefore it is good to see that the effects of this study were found despite of this. The topic is mentioned in the introduction but could also be mentioned again in the discussion with regard to the results.

I also noticed that the mean of recurrent long-term cases of sickness-absence due to mental health was 17% in 2020, but 49% in 2021, 45% in 2022 and 40% in 2023 (page 10, line 177). Could that maybe be seen as an increase in mental health problems severity among those that were suffering from mental health problems due to the pandemic? The proportion of sickness absence due to mental health only increased slightly during the four waves.

Minor comment: On page 11, line 202 the authors write “Response rate was calculated as the percentage of people who completed and returned the survey out of the total number of people invited to take part.” Was there something to return? Or should it be “answered” since the survey was done by telephone interview

Please submit your revised manuscript by Jun 17 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Erum Shaikh

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. 

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

3. Thank you for stating in your Funding Statement: 

The data used here were originally collected as part of an Economic and Social Research Council funded project ‘Workplace mental-health and wellbeing practices, outcomes and productivity’ (Grant number: ES/W010216/1). This secondary analysis project ‘Mental health at work: a longitudinal exploration of line manager training provisions and impacts on productivity, individual and organisational outcomes’ was supported by the Economic and Social  Research Council [Productivity Institute: grant number: ES/V002740/1].

Please provide an amended statement that declares all the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement. 

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

4. When completing the data availability statement of the submission form, you indicated that you will make your data available on acceptance. We strongly recommend all authors decide on a data sharing plan before acceptance, as the process can be lengthy and hold up publication timelines. Please note that, though access restrictions are acceptable now, your entire data will need to be made freely accessible if your manuscript is accepted for publication. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If you are unable to adhere to our open data policy, please kindly revise your statement to explain your reasoning and we will seek the editor's input on an exemption. Please be assured that, once you have provided your new statement, the assessment of your exemption will not hold up the peer review process.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The paper "The Relationship between Line Manager Training in Mental Health and Organisational Outcomes" examines the effects of managerial training on organizational outcomes. While the introduction adequately establishes the research, incorporating a discussion on existing mental health training practices could improve its context. The literature review could be strengthened by including a variety of viewpoints and conflicting research according to the variables. The methodology section, centered around probit regression, could benefit from a discussion on its limitations and potential biases and it could be analyse using confirming statistical multivariate analysis methods. The results section is well-articulated and comprehensive, though adding visual data representations and concentrating data in tables can enhance understanding. The conclusions effectively recap the study’s findings; however, elaborating on their practical applications could enhance their utility in real-world settings; increasing references to previous studies are needed to confirm research assumptions. Finally, the references are comprehensive, yet updating them to include more of the latest studies would ensure the research's relevance and show the top industry standards.

Suggestions for Enhancement:

Introduction:

I'd like you to add insights on prevailing mental health training practices in the industry to anchor the research's significance better. Include a clear definition of your research questions, objectives, and hypothesis, perhaps in a table.

Literature Review:

Broaden the scope to include diverse research and a more comprehensive array of theoretical perspectives to enrich the narrative. Explain each of your research variables and define a table of authors where you include the top publications for each variable and dimension of your study.

Methodology:

Include more details on the selection of probit regression and address any inherent methodological weaknesses or biases. Explain why it was the best alternative for analyzing this data and if there's any other possible alternative method to validate your assumptions.

Results:

I'd like you to introduce graphical elements like charts or graphs to summarize and clarify key findings, compare your results with previous research, and determine if this study covers any gap in theory, methodology, or research methods.

Conclusions:

Detail the practical implications of the research findings, suggesting specific training or policy changes that could be derived from the study. Describe the stakeholder who are having the best use for your fingins. Describe how you achieve your results and goals.

References:

Update and verify that all references reflect the most current research, ensuring the study's alignment with the latest developments in the field.

Reviewer #2: This is an interesting study investigating the organizational level effects of line manager trainings for mental health. The authors found a way to assess these outcomes by analysing data from an existing dataset (panel survey data from firms in England) thereby adding important information about the effects of these trainings that go beyond an assessment by the participants.

The manuscript is well written and clearly structured and adds important information to this area of research.

However, I have some comments that I would like the authors to respond to before publication:

The authors are right in pointing out (page 5 line 90) that only few studies have examined the impact and influence of LM training on organisational-level outcomes (e.g., changes to productivity, turnover rates, and absenteeism). However, also the impact of LM training on employee outcomes is often not assessed. This also applies to the “Employee-level benefits” that the authors describe have been found (page 5 line 85). It does not look like employees have been asked about these benefits. Instead, the results describe managers’ own assessment of their gained competencies, their intentions to promote mental health at work and their increased confidence to support employees with mental health problems. I understand that this often is what can be measured in studies about manager training as asking employees would often result in a much larger study. However, since the authors address this topic I think it is important to mention that we actually know surprisingly little about the effects of LM trainings for employees, as most studies assess how the participants themselves (the managers) evaluate what they gained from having been trained. Results often show increased knowledge, confidence and good intentions, however, we most of the time do not know if this leads to actual implementing this in daily practice (for example more support for employees with mental health problems) and we know even less about if this leads to better outcomes for employees.

This lack of knowledge of manager training for employee outcome is discussed in for example the two papers below but can also be found in several reviews about manager training for employee mental health:

Rugulies, R., Aust, B., Greiner, B. A., Arensman, E., Kawakami, N., LaMontagne, A. D., & Madsen, I. E. (2023). Work-related causes of mental health conditions and interventions for their improvement in workplaces. The Lancet, 402(10410), 1368-1381.

Aust, B., Møller, J. L., Nordentoft, M., Frydendall, K. B., Bengtsen, E., Jensen, A. B., ... & Jaspers, S. Ø. (2023). How effective are organizational-level interventions in improving the psychosocial work environment, health, and retention of workers? A systematic overview of systematic reviews. Scandinavian Journal of Work, Environment & Health, 49(5), 315.

My next comment is about missing data. The authors report on page 11, line 200 that the sample consisted of 7139 participants after the four waves were merged. However, the numbers of answers for the different questions are much lower. In Table 3 the number of responses for the sickness absence related outcomes are between 1116 and 3566. In Table 4 the numbers for the responses for the organisational-level outcomes are between 3178 and 3189. Does that mean that only around half of the participants answered these questions? Please explain.

In addition, the number of answers for the questions about the proportion of sickness absence due to mental health is particularly low (n=1116). Do you have any explanation for why this question was not answered by most of the participants? An explanation could be that employers might not have that information. They might only have information about sickness absence in general, but not about the reason for sickness absence.

On page 10, line 186 the authors write: “As pooled panel data were used, we did not employ specific strategies to address missing data in the analysis as any missing data points were inherently handled through the nature of the dataset.” Could you please explain, what is meant by “were inherently handled through the nature of the dataset”?

Response rate: As reported in the result section (page 11, line 202), the response rate for the four waves was between 17% (2020) and 15% (2021-2023). This low response rate together with the high degree of missing data among those who answered should be addressed as a limitation of the study in the section strengths and limitations.

It is however positive that the sample included a lot of small to medium sized workplaces (SMEs). In fact, around 80% of the participating organisations had less than 50 employees. Most studies about mental health at work are conducted in larger organisations, so it is clearly a strength that this study includes many SMEs that often have fewer resources to be active in mental health promotion. This strength could also be mentioned in the discussion part. Another strength is the mix of sectors, as there often is an overrepresentation of studies about mental health at work conducted in the health care sector. In this study, it is therefore good to see that sectors such as production and construction are represented. (See for example the study by Greiner et al. that calls for more research about mental health interventions in the construction sector. (Greiner, B. A., Leduc, C., O’Brien, C., Cresswell-Smith, J., Rugulies, R., Wahlbeck, K., ... & Aust, B. (2022). The effectiveness of organisational-level workplace mental health interventions on mental health and wellbeing in construction workers: A systematic review and recommended research agenda. Plos one, 17(11), e0277114.).

However, it would also be good to know how many organisations actually offered Line Manager training in mental health, that is, how was the question “LM training in mental health” (page 8, line 158) answered by the organisations? Could you report this and also shown it by size of organisation? This would be very valuable additional information for this topic: How widespread are LM training in mental health in these organisations (especially in in this sample representing sectors often not studied)? Are LM trainings just as common among SME businesses than they are among larger organisations? Please add this information.

Another issue that should be addressed is that at least some of the survey waves were conducted during the COVID-19 pandemic (see page 7, from line 131). The first wave (January to March 2020) was probably not affected, especially since the questions are about the past 12 months, so before the pandemic came to Europe. However, wave 2 and 3 conducted in 2021 and 2022 and thereby asking about 2020 and 2021 were conducted while most workplaces were affected by the pandemic in one way or the other. The authors probably have no specific knowledge about how this affected these workplaces, but it could be mentioned that the need for more awareness for employee mental health might have been even higher during these years (because for example more social isolation) and therefore it is good to see that the effects of this study were found despite of this. The topic is mentioned in the introduction but could also be mentioned again in the discussion with regard to the results.

I also noticed that the mean of recurrent long-term cases of sickness-absence due to mental health was 17% in 2020, but 49% in 2021, 45% in 2022 and 40% in 2023 (page 10, line 177). Could that maybe be seen as an increase in mental health problems severity among those that were suffering from mental health problems due to the pandemic? The proportion of sickness absence due to mental health only increased slightly during the four waves.

Minor comment: On page 11, line 202 the authors write “Response rate was calculated as the percentage of people who completed and returned the survey out of the total number of people invited to take part.” Was there something to return? Or should it be “answered” since the survey was done by telephone interview

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

The Relationship between Line Manager Training in Mental Health and Organisational Outcomes

PONE-D-24-06886R1

Dear Dr. Blake,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Erum Shaikh

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: