Sport psychology and performance meta-analyses: A systematic review of the literature

Sport psychology as an academic pursuit is nearly two centuries old. An enduring goal since inception has been to understand how psychological techniques can improve athletic performance. Although much evidence exists in the form of meta-analytic reviews related to sport psychology and performance, a systematic review of these meta-analyses is absent from the literature. We aimed to synthesize the extant literature to gain insights into the overall impact of sport psychology on athletic performance. Guided by the PRISMA statement for systematic reviews, we reviewed relevant articles identified via the EBSCOhost interface. Thirty meta-analyses published between 1983 and 2021 met the inclusion criteria, covering 16 distinct sport psychology constructs. Overall, sport psychology interventions/variables hypothesized to enhance performance (e.g., cohesion, confidence, mindfulness) were shown to have a moderate beneficial effect (d = 0.51), whereas variables hypothesized to be detrimental to performance (e.g., cognitive anxiety, depression, ego climate) had a small negative effect (d = -0.21). The quality rating of meta-analyses did not significantly moderate the magnitude of observed effects, nor did the research design (i.e., intervention vs. correlation) of the primary studies included in the meta-analyses. Our review strengthens the evidence base for sport psychology techniques and may be of great practical value to practitioners. We provide recommendations for future research in the area.


Introduction
Sport performance matters. Verifying its global importance requires no more than opening a newspaper to the sports section, browsing the internet, looking at social media outlets, or scanning abundant sources of sport information. Sport psychology is an important avenue through which to better understand and improve sport performance. To date, a systematic review of published sport psychology and performance meta-analyses is absent from the literature. Given the undeniable importance of sport, the history of sport psychology in academics since 1830, and the global rise of sport psychology journals and organizations, a comprehensive several comprehensive edited textbooks have been produced detailing sport psychology developments across the world, such as Hanrahan and Andersen's (2010) Handbook of Applied Sport Psychology [9], Schinke, McGannon, and Smith's (2016) International Handbook of Sport Psychology [10], and Bertollo, Filho, and Terry's (2021) Advancements in Mental Skills Training [11] to name just a few. In short, sport psychology is global in both academic study and professional practice.

Meta-analysis in sport psychology
Several meta-analysis guides, computer programs, and sport psychology domain-specific primers have been popularized in the social sciences [12,13]. Sport psychology academics have conducted quantitative reviews on much studied constructs since the 1980s, with the first two appearing in 1983 in the form of Feltz and Landers' meta-analysis on mental practice [14], which included 98 articles dating from 1934, and Bond and Titus' cross-disciplinary metaanalysis on social facilitation [15], which summarized 241 studies including Triplett's (1898) often-cited study of social facilitation in cycling [16]. Although much meta-analytic evidence exists for various constructs in sport and exercise psychology [12] including several related to performance [17], the evidence is inconsistent. For example, two meta-analyses, both ostensibly summarizing evidence of the benefits to performance of task cohesion [18,19], produced very different mean effects (d = .24 vs d = 1.00) indicating that the true benefit lies somewhere in a wide range from small to large. Thus, the lack of a reliable evidence base for the use of sport psychology techniques represents a significant gap in the knowledge base for practitioners and researchers alike. A comprehensive systematic review of all published meta-analyses in the field of sport psychology has yet to be published.

Purpose and aim
We consider this review to be both necessary and long overdue for the following reasons: (a) the extensive history of sport psychology and performance research; (b) the prior publication of many meta-analyses summarizing various aspects of sport psychology research in a piecemeal fashion [12,17] but not its totality; and (c) the importance of better understanding and hopefully improving sport performance via the use of interventions based on solid evidence of their efficacy. Hence, we aimed to collate and evaluate this literature in a systematic way to gain improved understanding of the impact of sport psychology variables on sport performance by construct, research design, and meta-analysis quality, to enhance practical knowledge of sport psychology techniques and identify future lines of research inquiry. By systematically reviewing all identifiable meta-analytic reviews linking sport psychology techniques with sport performance, we aimed to evaluate the strength of the evidence base underpinning sport psychology interventions.

Materials and methods
This systematic review of meta-analyses followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [20]. We did not register our systematic review protocol in a database. However, we specified our search strategy, inclusion criteria, data extraction, and data analyses in advance of writing our manuscript. All details of our work are available from the lead author. Concerning ethics, this systematic review received a waiver from Texas Tech University Human Subject Review Board as it concerned archival data (i.e., published meta-analyses).

Eligibility criteria
Published meta-analyses were retained for extensive examination if they met the following inclusion criteria: (a) included meta-analytic data such as mean group, between or withingroup differences or correlates; (b) published prior to January 31, 2021; (c) published in a peer-reviewed journal; (d) investigated a recognized sport psychology construct; and (e) meta-analyzed data concerned with sport performance. There was no language of publication restriction. To align with our systematic review objectives, we gave much consideration to study participants and performance outcomes. Across multiple checks, all authors confirmed study eligibility. Three authors (ML, AL, and PT) completed the final inclusion assessments.

Information sources
Authors searched electronic databases, personal meta-analysis history, and checked with personal research contacts. Electronic database searches occurred in EBSCOhost with the following individual databases selected: APA PsycINFO, ERIC, Psychology and Behavioral Sciences Collection, and SPORTDiscus. An initial search concluded October 1, 2020. ML, AL, and PT rechecked the identified studies during the February-March, 2021 period, which resulted in the identification of two additional meta-analyses [21,22].

Search protocol
ML and ES initially conducted independent database searches. For the first search, ML used the following search terms: sport psychology with meta-analysis or quantitative review and sport and performance or sport � performance. For the second search, ES utilized a sport psychology textbook and used the chapter title terms (e.g., goal setting). In EBSCOhost, both searches used the advanced search option that provided three separate boxes for search terms such as box 1 (sport psychology), box 2 (meta-analysis), and box 3 (performance). Specific details of our search strategy were: Search by ML: • sport psychology, meta-analysis, sport and performance • sport psychology, meta-analysis or quantitative review, sport � performance • sport psychology, quantitative review, sport and performance • sport psychology, quantitative review, sport � performance Search by ES: • mental practice or mental imagery or mental rehearsal and sports performance and metaanalysis • goal setting and sports performance and meta-analysis • anxiety and stress and sports performance and meta-analysis • competition and sports performance and meta-analysis • diversity and sports performance and meta-analysis • cohesion and sports performance and meta-analysis • imagery and sports performance and meta-analysis • self-confidence and sports performance and meta-analysis • concentration and sports performance and meta-analysis • athletic injuries and sports performance and meta-analysis • overtraining and sports performance and meta-analysis • children and sports performance and meta-analysis The following specific search of the EBSCOhost with SPORTDiscus, APA PsycINFO, Psychology and Behavioral Sciences Collection, and ERIC databases, returned six results from 2002-2020, of which three were included [18,19,23] and three were excluded because they were not meta-analyses.

Study selection
As detailed in the PRISMA flow chart (Fig 1) and the specified inclusion criteria, a thorough study selection process was used. As mentioned in the search protocol, two authors (ML and ES) engaged independently with two separate searches and then worked together to verify the selected studies. Next, AL and PT examined the selected study list for accuracy. ML, AL, and PT, whilst rating the quality of included meta-analyses, also re-examined all selected studies to verify that each met the predetermined study inclusion criteria. Throughout the study selection process, disagreements were resolved through discussion until consensus was reached.

Data extraction process
Initially, ML, TH, and ES extracted data items 1, 2, 3 and 8 (see Data items). Subsequently, ML, AL, and PT extracted the remaining data (items 4-7, 9, 10). Checks occurred during the extraction process for potential discrepancies (e.g., checking the number of primary studies in a meta-analysis). It was unnecessary to contact any meta-analysis authors for missing information or clarification during the data extraction process because all studies reported the required information. Across the search for meta-analyses, all identified studies were reported in English. Thus, no translation software or searching out a native speaker occurred. All data extraction forms (e.g., data items and individual meta-analysis quality) are available from the first author.

Individual meta-analysis quality
ML, AL, and PT independently rated the quality of individual meta-analysis on the following 25 points found in the PRISMA checklist [20]: title; abstract structured summary; introduction rationale, objectives, and protocol and registration; methods eligibility criteria, information sources, search, study selection, data collection process, data items, risk of bias of individual studies, summary measures, synthesis of results, and risk of bias across studies; results study selection, study characteristics, risk of bias within studies, results of individual studies, synthesis of results, and risk of bias across studies; discussion summary of evidence, limitations, and conclusions; and funding. All meta-analyses were rated for quality by two coders to facilitate inter-coder reliability checks, and the mean quality ratings were used in subsequent analyses. One author (PT), having completed his own ratings, received the incoming ratings from ML and AL and ran the inter-coder analysis. Two rounds of ratings occurred due to discrepancies for seven meta-analyses, mainly between ML and AL. As no objective quality categorizations (i.e., a point system for grouping meta-analyses as poor, medium, good) currently exist, each https://doi.org/10.1371/journal.pone.0263408.g001 meta-analysis was allocated a quality score of up to a maximum of 25 points. All coding records are available upon request.

Planned methods of analysis
Several preplanned methods of analysis occurred. We first assessed the mean quality rating of each meta-analysis based on our 25-point PRISMA-based rating system. Next, we used a median split of quality ratings to determine whether standardized mean effects (SMDs) differed by the two formed categories, higher and lower quality meta-analyses. Meta-analysis authors reported either of two different effect size metrics (i.e., r and SMD); hence we converted all correlational effects to SMD (i.e., Cohen's d) values using an online effect size calculator (www.polyu.edu.hk/mm/effectsizefaqs/calculator/calculator.html). We interpreted the meaningfulness of effects based on Cohen's interpretation [24] with 0.20 as small, 0.50 as medium, 0.80 as large, and 1.30 as very large. As some psychological variables associate negatively with performance (e.g., confusion [25], cognitive anxiety [26]) whereas others associate positively (e.g., cohesion [23], mental practice [14]), we grouped meta-analyses according to whether the hypothesized effect with performance was positive or negative, and summarized the overall effects separately. By doing so, we avoided a scenario whereby the demonstrated positive and negative effects canceled one another out when combined. The effect of somatic anxiety on performance, which is hypothesized to follow an inverted-U relationship, was categorized as neutral [35]. Last, we grouped the included meta-analyses according to whether the primary studies were correlational in nature or involved an intervention and summarized these two groups of meta-analyses separately. Table 1 contains extracted data from 30 meta-analyses meeting the inclusion criteria, dating from 1983 [14] to 2021 [21]. The number of primary studies within the meta-analyses ranged from three [27] to 109 [28]. In terms of the description of participants included in the metaanalyses, 13 included participants described simply as athletes, whereas other meta-analyses identified a mix of elite athletes (e.g., professional, Olympic), recreational athletes, collegeaged volunteers (many from sport science departments), younger children to adolescents, and adult exercisers. Of the 30 included meta-analyses, the majority (n = 18) were published since 2010. The decadal breakdown of meta-analyses was 1980-1989 (n = 1 [14]), 1990-1999 (n = 6 [29][30][31][32][33][34]), 2000-2009 (n = 5 [23,25,26,35,36]), 2010-2019 (n = 12 [18,19,22,27,[37][38][39][40][41][42][43]48]), and 2020-2021 (n = 6 [21,28,[44][45][46][47]).
A wide variety of performance outcomes across many sports was evident, such as golf putting, dart throwing, maximal strength, and juggling; or categorical outcomes such as win/loss and Olympic team selection. Given the extensive list of performance outcomes and the incomplete descriptions provided in some meta-analyses, a clear categorization or count of performance types was not possible. Sufficient to conclude, researchers utilized many performance outcomes across a wide range of team and individual sports, motor skills, and strength and aerobic tasks.

Effect size data and bias correction
To best summarize the effects, we transformed all correlations to SMD values (i.e., Cohen's d).
Across all included meta-analyses shown in Table 2 and depicted in Fig 2, we identified 61 effects. Having corrected for bias, effect size values were assessed for meaningfulness [24],  It appears that meta-analysis standards have risen over the years since the PRISMA criteria were first introduced in 2009. All data for our analyses are shown in Table 2. Table 3 contains summary statistics with bias-corrected values used in the analyses. The overall mean effect for sport psychology constructs hypothesized to have a positive impact on performance was of moderate magnitude (d = 0.51, 95% CI = 0.42, 0.58, n = 36). The overall mean effect for sport psychology constructs hypothesized to have a negative impact on performance was small in magnitude (d = -0.21, 95% CI -0.31, -0.11, n = 24). In both instances, effects were larger, although not significantly so, among meta-analyses of higher quality compared to those of lower quality. Similarly, mean effects were larger but not significantly so, where reported effects in the original studies were based on interventional rather than correlational designs. This trend only applied to hypothesized positive effects because none of the original studies in the meta-analyses related to hypothesized negative effects used interventional designs.

Discussion
In this systematic review of meta-analyses, we synthesized the available evidence regarding effects of sport psychology interventions/constructs on sport performance. We aimed to consolidate the literature, evaluate the potential for meta-analysis quality to influence the results, and suggest recommendations for future research at both the single study and quantitative review stages. During the systematic review process, several meta-analysis characteristics came to light, such as the number of meta-analyses of sport psychology interventions (experimental designs) compared to those summarizing the effects of psychological constructs (correlation designs) on performance, the number of meta-analyses with exclusively athletes as participants, and constructs featuring in multiple meta-analyses, some of which (e.g., cohesion) produced very different effect size values. Thus, although our overall aim was to evaluate the strength of the evidence base for use of psychological interventions in sport, we also discuss the impact of these meta-analysis characteristics on the reliability of the evidence. When seen collectively, results of our review are supportive of using sport psychology techniques to help improve performance and confirm that variations in psychological constructs relate to variations in performance. For constructs hypothesized to have a positive effect on performance, the mean effect strength was moderate (d = 0.51) although there was substantial variation between constructs. For example, the beneficial effects on performance of task cohesion (d = 1.00) and self-efficacy (d = 0.82) are large, and the available evidence base for use of mindfulness interventions suggests a very large beneficial effect on performance (d = 1.35). Conversely, some hypothetically beneficial effects (2 of 36; 5.6%) were in the negligible-tosmall range (0.15-0.20) and most beneficial effects (19 of 36; 52.8%) were in the small-tomoderate range (0.22-0.49). It should be noted that in the world of sport, especially at the elite level, even a small beneficial effect on performance derived from a psychological intervention may prove the difference between success and failure and hence small effects may be of great practical value. To put the scale of the benefits into perspective, an authoritative and extensively cited review of healthy eating and physical activity interventions [49] produced an overall pooled effect size of 0.31 (compared to 0.51 for our study), suggesting sport psychology interventions designed to improve performance are generally more effective than interventions designed to promote healthy living. Among hypothetically negative effects (e.g., ego climate, cognitive anxiety, depression), the mean detrimental effect was small (d = -0.21) although again substantial variation among constructs was evident. Some hypothetically negative constructs (5 of 24; 20.8%) were found to actually provide benefits to performance, albeit in the negligible range (0.02-0.12) and only two constructs (8.3%), both from Lochbaum and colleagues' POMS meta-analysis [21], were shown to negatively affect performance above a moderate level (depression: d = -0.64; total mood disturbance, which incorporates the depression subscale: d = -0.84). Readers should note that the POMS and its derivatives assess six specific mood dimensions rather than the mood construct more broadly, and therefore results should not be extrapolated to other dimensions of mood [50].
Mean effects were larger among higher quality than lower quality meta-analyses for both hypothetically positive (d = 0.54 vs d = 0.45) and negative effects (d = -0.25 vs d = 0.17), but in neither case were the differences significant. It is reasonable to assume that the true effects were derived from the higher quality meta-analyses, although our conclusions remain the same regardless of study quality. Overall, our findings provide a more rigorous evidence base for the use of sport psychology techniques by practitioners than was previously available, representing a significant contribution to knowledge. Moreover, our systematic scrutiny of 30 meta-analyses published between 1983 and 2021 has facilitated a series of recommendations to improve the quality of future investigations in the sport psychology area.

Recommendations
The development of sport psychology as an academic discipline and area of professional practice relies on using evidence and theory to guide practice. Hence, a strong evidence base for the applied work of sport psychologists is of paramount importance. Although the beneficial effects of some sport psychology techniques are small, it is important to note the larger performance benefits for other techniques, which may be extremely meaningful for applied practice. Overall, however, especially given the heterogeneity of the observed effects, it would be wise for applied practitioners to avoid overpromising the benefits of sport psychology services to clients and perhaps underdelivering as a result [1].
The results of our systematic review can be used to generate recommendations for how the profession might conduct improved research to better inform applied practice. Much of the early research in sport psychology was exploratory and potential moderating variables were not always sufficiently controlled. Terry [51] outlined this in relation to the study of mood-performance relationships, identifying that physical and skills factors will very likely exert a greater influence on performance than psychological factors. Further, type of sport (e.g., individual vs. team), duration of activity (e.g., short vs. long duration), level of competition (e.g., elite vs. recreational), and performance measure (e.g., norm-referenced vs. selfreferenced) have all been implicated as potential moderators of the relationship between psychological variables and sport performance [51]. To detect the relatively subtle effects of psychological effects on performance, research designs need to be sufficiently sensitive to such potential confounds. Several specific methodological issues are worth discussing.
The first issue relates to measurement. Investigating the strength of a relationship requires the measured variables to be valid, accurate and reliable. Psychological variables in the metaanalyses we reviewed relied primarily on self-report outcome measures. The accuracy of selfreport data requires detailed inner knowledge of thoughts, emotions, and behavior. Research shows that the accuracy of self-report information is subject to substantial individual differences [52,53]. Therefore, self-report data, at best, are an estimate of the measure. Measurement issues are especially relevant to the assessment of performance, and considerable measurement variation was evident between meta-analyses. Some performance measures were more sensitive, especially those assessing physical performance relative to what is normal for the individual performer (i.e., self-referenced performance). Hence, having multiple baseline indicators of performance increases the probability of identifying genuine performance enhancement derived from a psychological intervention [54].
A second issue relates to clarifying the rationale for how and why specific psychological variables might influence performance. A comprehensive review of prerequisites and precursors of athletic talent [55] concluded that the superiority of Olympic champions over other elite athletes is determined in part by a range of psychological variables, including high intrinsic motivation, determination, dedication, persistence, and creativity, thereby identifying performance-related variables that might benefit from a psychological intervention. Identifying variables that influence the effectiveness of interventions is a challenging but essential issue for researchers seeking to control and assess factors that might influence results [49]. A key part of this process is to use theory to propose the mechanism(s) by which an intervention might affect performance and to hypothesize how large the effect might be.
A third issue relates to the characteristics of the research participants involved. Out of convenience, it is not uncommon for researchers to use undergraduate student participants for research projects, which may bias results and restrict the generalization of findings to the population of primary interest, often elite athletes. The level of training and physical conditioning of participants will clearly influence their performance. Highly trained athletes will typically make smaller gains in performance over time than novice athletes, due to a ceiling effect (i.e., they have less room for improvement). For example, consider runner A, who takes 20 minutes to run 5km one week but 19 minutes the next week, and Runner B who takes 30 minutes one week and 25 minutes the next. If we compare the two, Runner A runs faster than Runner B on both occasions, but Runner B improved more, so whose performance was better? If we also consider Runner C, a highly trained athlete with a personal best of 14 minutes, to run 1 minute quicker the following week would almost require a world record time, which is clearly unlikely. For this runner, an improvement of a few seconds would represent an excellent performance. Evidence shows that trained, highly motivated athletes may reach performance plateaus and as such are good candidates for psychological skills training. They are less likely to make performance gains due to increased training volume and therefore the impact of psychological skills interventions may emerge more clearly. Therefore, both test-retest and cross-sectional research designs should account for individual difference variables. Further, the range of individual difference factors will be context specific; for example, individual differences in strength will be more important in a study that uses weightlifting as the performance measure than one that uses darts as the performance measure, where individual differences in skill would be more important.
A fourth factor that has not been investigated extensively relates to the variables involved in learning sport psychology techniques. Techniques such as imagery, self-talk and goal setting all require cognitive processing and as such some people will learn them faster than others [56]. Further, some people are intuitive self-taught users of, for example, mood regulation strategies such as abdominal breathing or listening to music who, if recruited to participate in a study investigating the effects of learning such techniques on performance, would respond differently to novice users. Hence, a major challenge when testing the effects of a psychological intervention is to establish suitable controls. A traditional non-treatment group offers one option, but such an approach does not consider the influence of belief effects (i.e., placebo/ nocebo), which can either add or detract from the effectiveness of performance interventions [57]. If an individual believes that, an intervention will be effective, this provides a motivating effect for engagement and so performance may improve via increased effort rather than the effect of the intervention per se.
When there are positive beliefs that an intervention will work, it becomes important to distinguish belief effects from the proposed mechanism through which the intervention should be successful. Research has shown that field studies often report larger effects than laboratory studies, a finding attributed to higher motivation among participants in field studies [58]. If participants are motivated to improve, being part of an active training condition should be associated with improved performance regardless of any intervention. In a large online study of over 44,000 participants, active training in sport psychology interventions was associated with improved performance, but only marginally more than for an active control condition [59]. The study involved 4-time Olympic champion Michael Johnson narrating both the intervention and active control using motivational encouragement in both conditions. Researchers should establish not only the expected size of an effect but also to specify and assess why the intervention worked. Where researchers report performance improvement, it is fundamental to explain the proposed mechanism by which performance was enhanced and to test the extent to which the improvement can be explained by the proposed mechanism(s).

Limitations
Systematic reviews are inherently limited by the quality of the primary studies included. Our review was also limited by the quality of the meta-analyses that had summarized the primary studies. We identified the following specific limitations; (1) only 12 meta-analyses summarized primary studies that were exclusively intervention-based, (2) the lack of detail regarding control groups in the intervention meta-analyses, (3) cross-sectional and correlation-based metaanalyses by definition do not test causation, and therefore provide limited direct evidence of the efficacy of interventions, (4) the extensive array of performance measures even within a single meta-analysis, (5) the absence of mechanistic explanations for the observed effects, and (6) an absence of detail across intervention-based meta-analyses regarding number of sessions, participants' motivation to participate, level of expertise, and how the intervention was delivered. To ameliorate these concerns, we included a quality rating for all included meta-analyses. Having created higher and lower quality groups using a median split of quality ratings, we showed that effects were larger, although not significantly so, in the higher quality group of meta-analyses, all of which were published since 2015.

Conclusions
Journals are full of studies that investigate relationships between psychological variables and sport performance. Since 1983, researchers have utilized meta-analytic methods to summarize these single studies, and the pace is accelerating, with six relevant meta-analyses published since 2020. Unquestionably, sport psychology and performance research is fraught with limitations related to unsophisticated experimental designs. In our aggregation of the effect size values, most were small-to-moderate in meaningfulness with a handful of large values. Whether these moderate and large values could be replicated using more sophisticated research designs is unknown. We encourage use of improved research designs, at the minimum the use of control conditions. Likewise, we encourage researchers to adhere to meta-analytic guidelines such as PRISMA and for journals to insist on such adherence as a prerequisite for the acceptance of reviews. Although such guidelines can appear as a 'painting by numbers' approach, while reviewing the meta-analyses, we encountered difficulty in assessing and finding pertinent information for our study characteristics and quality ratings. In conclusion, much research exists in the form of quantitative reviews of studies published since 1934, almost 100 years after the very first publication about sport psychology and performance [2]. Sport psychology is now truly global in terms of academic pursuits and professional practice and the need for best practice information plus a strong evidence base for the efficacy of interventions is paramount. We should strive as a profession to research and provide best practices to athletes and the general community of those seeking performance improvements.