Development and factorial validity of the Psychological Skills Inventory for Sports, Youth Version – Short Form: Assessment of the psychometric properties

Researchers in sport often try to investigate relations between athletes’ psychological skills and their sports results to predict top athletic achievements or unexpectedly poor performances. The Psychology Skills Inventory for Sports (Youth version), PSIS-Y, was developed to measure psychological characteristics of young athletes–differentiating well more talented and less talented young athletes. Nevertheless, previous studies revealed its inadequate, factorial validity. Thus, the aim of this study was to develop and investigate the psychometric proprieties of a brief version of the PSIS-Y (PSIS-Y-SF) in a sample of young Croatian athletes. Participants (n = 304; 188 females and 116 male) were recruited in clubs/teams all over Croatia and all of them competed in the Croatian Championship in youth (n = 157) and junior category (n = 147). The PSIS-Y-SF was derived by ten expert psychologists with five of them who had past experiences of agonistic sport practice. Psychometric analysis included Confirmatory Factor Analysis (CFA), internal consistency analysis (Raykov’s Maximal Reliability), and correlation between subscales. Moreover, Multivariate Analyses of Variance (MANOVA) was run to test statistical differences between the players’ categories (male youth vs. male junior vs. female youth vs. female junior) in all of the subscales. Results of the CFA suggested the adequateness of the supposed six first-order factor solution for the PSIS-Y-SF. The Maximal Reliability statistics suggest a good internal consistency for all of the subscales and the MANOVA suggested differences between the player’s categories. The PSIS-Y-SF resulted to be a valid and reliable tool for the assessment of sports psychological skills. Findings from the psychometric evaluation of PSIS-Y-SF suggest that this is a useful tool, which may further assist in the measurement and conceptualization of sport psychological skills.


Introduction
According to the Achievement Goal Theory [1,2] and Self-Determination Theory [3], extensive research supported the key role of psychological attributes to promote athletes' performance [3][4][5]. Indeed, it has been widely demonstrated how sport success depends on the reciprocal enhancement of an athlete's physical and technical abilities as well as a functional pattern of psychological sport skills in relation to his/her own sport. This is exactly why researchers try to determine the relations between athletes' psychological skills and their sports results (most frequently their athletic performance and placement in a sports competition). Morris [6] stated it is not surprising that psychological characteristics often distinguish more successful elite athletes from less successful ones, because all elite athletes are constantly under a high level of pressure.
In the last decades, research has proven that specific psychological factors play a key role in predicting athletes' success [6][7][8], some crucial factors are: motivation, self-confidence, mental preparation, emotional management, and mental readiness [7][8][9][10][11][12][13][14] predominantly determined by the measurements at the same time point with athletes' performance. These factors are characteristic of a competitive and functional mindset that is necessary to reach high performance levels. Several studies [14][15][16] proved that the mindset really makes the difference, indeed, psychological skills showed to reliably discriminate between more and less successful athletes [6]. Thus, properly assessing the psychological skills involved in sport might be extremely useful both to select the best athletes, both to facilitate the young practitioners to develop mental skills [6] that are fundamental for a high-level sport career. By assessing and identifying the specific potentialities, or issues, of each athlete, is thus possible to strengthen the psychological preparation of athletes, to improve their well-being, and consequently, also their sport results.
A clear delineation of personal sport skills is interesting for several aims, for example to flexibly tailor psychological skills training interventions which can be adapted to different kind of athletes, also those with disabilities [17] in order to enhance their performance in competitions.
Reliable measures of psychological skills are needed also to test the efficacy of evidencebased techniques, such as psychological skills training and/or mindfulness-based interventions [18]. Psychological skills showed their importance in sport across the athletes' gender and in many different disciplines [10,11,19] a proper assessment represents the first step to conduct valuable studies.
The importance of psychological skills can also be easily noticed in volleyball [20][21][22][23][24][25], which is a very dynamic team sport characterized by very short rallies, lasting averagely only 7 sec [26], and tight set finals that additionally increase the psychological pressure. Also, constant alteration of very short defensive and attacking activities require every player to constantly use all psychological skills, especially fast attention focusing/refocusing and those skills used by players in coping with unexpected situations during the game, such as self-confidence, peaking under pressure, and control of competitive state anxiety [8].
Mahoney et al. [8] developed a 51-item questionnaire to assess motivation, self-confidence, anxiety control, mental preparation, team orientation, and concentration in 713 male and female athletes from 23 sports from elite to non-elite collegiate level. They constructed a results-driven sport-specific profile for assessing a wide range of psychological skills (Psychological Skills Inventory for Sport questionnaire form [PSIS]) related to top athletic performances. The questionnaire development was based on previous work by Mahoney et al. [8] with athletes at student but also at Olympic level. The original PSIS consisted of 51 items, on a true-or-false response. The revised version of the original PSIS (PSIS-5R) included 45 items assessed by participants on a five-point Likert scale [8]. This version was developed with the aim of distinguishing elite, pre-elite, and student athletes according to their psychological skills. The questionnaire measured six dimensions: mental preparation, motivation, concentration, self-confidence, team emphasis, and anxiety control.
The questionnaire differentiated male and female athletes, athletes of individual sports, and athletes of team sports, as well as groups of athletes at different levels of athletic skill. Statistically significant differences were found between elite athletes and non-elite athletes. Elite athletes were more motivated for achievement in their sport, experienced less problems with anxiety, referenced more internally, prepared themselves more kinesthetically and mentally, were more focused on their performance than their team members, and maintained more their concentration [8]. However, Murphy and Tammern [27], in their review on the psychometric properties of the PSIS-5R questionnaire, stated that the results of the included studies were inconsistent and ambiguous in terms of different aspects of reliability and validity.
In addition, other studies pointed to the inconsistency of the results obtained by comparing athletes of different situational efficacy [7,28,29]. Because of its limitations, Weinberg and Forlenza [30] pointed to the questionable applicability of the PSIS for research and applied purposes.
The Psychology Skills Inventory for Sports-Youth (PSIS-Y) version [31] was one of the first questionnaires, whose multidimensional focus was directed to psychological characteristics of young athletes. It was established that the questionnaire differentiated more talented and less talented young athletes, especially in females, whereas motivation and mental preparation were found to be useful indicators of differences between elite and sub-elite athletes, regardless of the sport type or participants' gender [32].
However, to our knowledge, only one study investigated the factorial validity of the PSIS-Y. Indeed, only Sindik and colleagues [33] performed a factor analysis (Principal Axis Factoring-PAF) with Promax rotation. Their results revealed an adequate six-factor solution-that explains almost the 55% of variance-with very high internal consistency values (Cronbach's alpha: from 0.829 to 0.916) and the correlations between the six factors did not exceed 0.42. Nevertheless, despite these interesting results, the PAF showed that (A) a very large number of items did not load on the hypothesized factor (e.g., Factor 1 and Factor 6) and (B) most of the items had high cross loadings values (from 0.404 to 0.552). These results revealed that both the PSIS-5R and the PSIS-Y have not a good, reliable, and stable factorial validity. Moreover, due to their length, both the PSIS-5R and the PSIS-Y are suitable neither for routine evaluation nor for quick and rapid administration [34].
Considering this background, the aim of this study was to develop and investigate the psychometric proprieties of a brief version of the PSIS-Y (PSIS-Y-SF) in a sample of young Croatian volleyball players.
All participants were examined in the same season period of the Croatian Championship (CC) tournament. Assessments were completed individually, immediately before the training session. Three hundred and four (n = 304) volleyball players (188 females, 61.8%; and 116 males 38.2%) playing in clubs/teams all over Croatia were recruited. Athletes were aged from 14 to 19 yrs (mean = 16.28 yrs, SD = 1.93 yrs) and all of them competed in CC in youth (157, 51.6%) and/or junior category (147, 48.4%). Participants started playing volley from 7 to 17 yrs of age (mean = 11.25 yrs, SD = 2.30 yrs) and they training from 2 to 28 hours per week (mean = 8.98 hrs, SD = 3.94 hrs). Finally, 72 (23.7%) athletes performed-at least-one match in the Croatian National Team.
An informed written consent for participating in the research was given by all participants and their parents. The study was approved by the local university ethics committee (University of Split, Human Research Ethics Committee) according to the Declaration of Helsinki.

Procedures
Translation and cross-cultural adaptation of the Psychological Skills Inventory for Sports Youth version-Short Form. According to guidelines [47], the validation procedure started with the translation of the PSIS-Y followed by a consultation with three experts in the field about each item. The Croatian-language PSIS-Y was then administered to a sample of young/junior volleyball players (n = 30) of both sexes (who did not enter in the research), with the specific purpose of testing the content validity of its items and each item that had been marked as less comprehensive or insufficiently applicable to volleyball was discussed again within the experts' board.
Development of the Psychological Skills Inventory for Sports-Youth version-Short Form. Once obtained the Croatian version of the PSIS-Y, a double-blind study procedure was used to select items of the PSIS-Y to create its short form. An external collaborator submitted PSIS-Y items to ten expert psychologists (five experts in the field-who had past experiences of agonistic sport practice). Both the collaborator and the psychologists were blind about the aim of this procedure and which questionnaire those items were taken from. Experts were asked to indicate-for each factor separately-the three items that most represented its construct: Motivation (MT), Self-Confidence (SC), Anxiety Control (AC), Mental Preparation (MP), Team Emphasis (TE), and Concentration (C). An agreement between experts higher than 80% was considered adequate to retain the items into the PSIS-Y-SF. The 18 items reported in Table 1 constituted the short version of the PSIS-Y: the PSIS-Y-SF.

Final version of the Psychological Skills Inventory for Sports-Youth version-Short Form.
Thus, the PSIS-Y-SF directly derived from the PSIS-Y [31] that in turn was an adaptation for youth of the PSIS-5R [8]. Whereas the latter consisted of 45 items that were assessed on a five-point Likert scale, the former consisted of 44 items. Both versions measure six dimensions of psychological skills: mental preparation, motivation, concentration, self-confidence, team emphasis, and anxiety control. The PSIS-Y was more adapted to athletes of younger age categories in terms of context as opposed to the original version of the PSIS or the PSIS-5R.
Consequently, the PSIS-Y-SF was an 18 items instrument measuring the same six factors of its previous versions retaining three items per factor. Items were scaled on five-point Likert scale ranging from 1 (= almost never) to 5 (= almost always). The total score of each scale was derived by computing the mean of items of the factor. Thus, it was possible to compute six different scores (one for each scale). According to previous versions of the questionnaire, no PSI-S-Y-SF total score could be computed.
Moreover, model comparisons were performed to exclude that the PSIS-Y-SF provided a configuration structure other than its long form PSIS-Y [55]: (A) a single factor model, (B) a second order model (hierarchical), and (C) a bi-factor model (hierarchical). According to scientific literature [35,53], the second order model was specified by posit an overarching general latent dimension-called "sport skills"-loaded by the six first-order latent dimensions (MT, SC, AC, MP, TE, and C). The bi-factor model is another form of hierarchical analysis, but unlike second-order model, the overarching dimension exerts direct effects only on the items and orthogonality between all of latent dimensions was imposed [35,53]. According to guidelines, comparisons were carried out by using the test differences in three fit indices, with the following criteria as cutoffs for model equivalence: ΔCFI (<0.010), and ΔRMSEA (<0.015 [35,[54][55][56][57]). Overrunning this cutoff in two out of these three indices is evidence of model worsening.
In addition, considering that the PSIS-Y-SF is a new instrument, items were tested regarding their ability to discriminate subjects with low or high sport skills [58,59]. According to Ebel [58] and Chiorri [59] for typical performance test items (e.g., Likert scale), the maximum total score and quartile rank for each subject were calculated. Subsequently, a series of independent sample t-test-and their effect size (Cohen's d) [60]-were computed to determinate item discriminating power by using as dependent variable the total score of the scale and the lowest and the highest quartile as grouping variable [58,59]. Item-total correlation (adjusted) was also computed.
Due to possible differences in the magnitude of factor loadings, Raykov's MR [61] was chosen as measure of internal consistency of each single factor reliability-instead of both Cronbach's alpha and Composite Reliability [61][62][63][64].
Multivariate Analysis of Variance (MANOVA) was performed to determine possible differences between players category (male youth vs. male junior vs. female youth vs. female juniorindependent variable) simultaneously on the PSIS-Y-SF subscales (dependent variables). According to guidelines [65][66][67], both univariate and multivariate normality were assessed as well as the equality of covariances matrices and the absence of multicollinearity. Finally, the Games-Howell test was chosen for performing post-hoc analysis [65,66,68,69].
Finally, ordinary least square (OLS) regression analyses were performed to evaluate possible associations and differences between psychological skill and sport training variables: number of hours per week, number of years involved in volleyball, and years in the sport.
Results from the CFA suggest an adequate six first-order factor solution for the PSIS-Y-SF. Despite the χ 2 was statistically significant [χ 2 (120) = 276.843; P<0.001], the others fit indices overcame the threshold for good model fit. Indeed, the RMSEA was equal to 0.066 [90%CI: 0.056-0.076; P(RMSEA<0.05) = 0.006], the CFI was equal to 0.974, and the WRMR was equal to 1.109. Moreover, each item loaded on the factor associated with itself: mean loadings = 0.737; SD loadings = 0.109 and ranging from 0.535 (item#15) to 0.953 (item#13). Consequently, items' explained variance (R 2 ) ranged between 0.286 (item#15) and 0.908 (item#13) with a mean R2 equal to 0.555 and an SD R2 equal to 0.161 (Table 2, Fig 1). Factor correlation matrix is displayed in S1 Table. The IDP analysis showed that each of the 18 items discriminated well between subjects with individuals with low and individuals with high sport psychological skills ( Table 2). The discrimination parameter t i ranged from -11.475 (item#13) to -28.919 (item#10), with an associated effect size (Cohen's d) that ranged respectively from 1.814 (item#13) to 4.866 (item#10)-mean t = -19.197; SD t = 3.879; mean d = 3.136; and SD d = 0.695. Thus, all the discrimination values revealed a strong relation between each item and the corresponding sport psychological skill.  Then, considering these results, the six first-order factor solution factor solution was compared with different competing models that could better explain the PSIS-Y-SF factorial structure [35,53,55]. As reported in Table 3, model comparisons revealed the superiority of the hypothesized solution, i.e., six first-order factor model-accounting for six different dimensions. Consequently, this factorial solution was chosen to perform successive analysis.
As reported in Table 4, correlation analyses-between the six factors of the PSIS-Y-SF-were also performed. As reported in Table 4, statistically non-significant to moderate correlations between factors were found. Correlations ranged from 0.084 (correlation between TE and MP)  Table 4.  Table 1, skewness ranged from -0.851 (TE) to 0.485 (C) and kurtosis ranged from -0.629 (MP) to 0.550 (TE). According to guidelines [65][66][67], Mahalanobis distance was performed for each subject and compared with the critical χ 2 value of 22.46 (df = 6, derived from the number of dependent variables; α = 0.001). This comparison revealed the presence of a single multivariate outlier and five multivariate influential cases (2.0% of the total sample) with Mardia's Kurtosis equal to 5.078; P<0.001. However, considering this negligible percentage these subjects were retained into the analyses.
Correlations between dependent variables were under the threshold for collinearity (0.8), as showed in Table 4. Finally, the Box's M test was equal to 86.594 (F = 1.321; P = 0.045) revealing that the covariance matrices were not homogeneous. However, according to guidelines, MAN-OVA is quite robust to small violations of assumptions [65][66][67]. Consequently, MANOVA revealed a statistically significant difference between players category (male youth vs. male significant univariate contrasts in the previous analysis (with the overall sample) were retained even though influential cases were removed. In order to test differences between player category groups within PSIS-Y-SF subscales, analyses of variance (ANOVA) with focused contrasts were conducted for each dependent variable-as shown in Table 5. More in detail, no overall statistical differences emerged between the specific player categories in the Anxiety Control subscale (F = 2.256, P = 0.082 ns; and η p 2 = 0.022) as well as in the Mental Preparation subscale (F = 2.072, P = 0.104 ns; and η p 2 = 0.020).
Associations between sport psychological skills and training variables. As reported in Table 6, OLS regressions revealed an association between years of training and the Team Emphasis subscale (B = 0.215; t = 3.817; P<0.001) as well as the Concentration subscale (reverse scored scale; B = -0.144; t = -2.535; P = 0.012). No statistically significant associations between the others sport psychological skills and years of training were found (Table 6).
In These results suggested that an increase in training hours was associated to sport higher sport psychological skills.
All relevant data are within S1 Dataset.

Discussion
In sports practice, top athletic achievements or unexpectedly poor achievements are often attributed to athletes' psychological skills. In line with the Achievement Goal Theory [1,2] and Self-Determination Theory [3], in last years, physical as well as psychological research highlighted the key role of psychological sport skills to endorse better athletes' performance [3][4][5]70]. This has led researchers to focus more closely on the relationship between the athlete's psychological abilities and their performance-showing how success depends on both physical and technical components and psychological factors. Within this context, the goal of this study was the development and initial validation of a short form of the PSIS-Y (PSIS-Y-SF): a new tool for measuring-in a more concise and precise way-psychological skills in athletes [34].
CFA successfully confirmed the six first-order factors of the original PSIS-Y-Motivation, Self-Confidence, Anxiety Control, Mental Preparation, Team Emphasis, and Concentrationeven in this short form. Also, the CFA suggested that the PSIS-Y-SF had good fit indices and a good structural validity, despite the WRMR was slightly over the recommended cut-off. Furthermore, items' factor loading revealed a strong relation between the items and the corresponding latent sport psychological skill [35,[52][53][54].
Moreover, this factorial structure was compared with several competing models-a single factor model, a second factor model, and a bi-factor one [35,[53][54][55][56][57]-and even in this case, results suggested the higher adequacy of the hypothesized six first-order structure.
In addition, the item discrimination power was assessed. Results of this analyses highlighted that each of the 18 items composing the PSIS-Y-SF well discriminated between subjects with low and individuals with high sport psychological skills-suggesting the goodness of the item to capture individuals with different levels of the skill as well as the ability of each single item to represent its underlying construct.
Reliability analysis was also performed providing satisfying results. Considering that the PSIS-Y-SF is a new tool assessing sport psychological skills, Cronbach's alpha was considered not adequate-due to the unknown possible (strong) differences in items' factor loading [64]. Thus, a more trustworthy weighted method to estimates reliability (MR [61][62][63]) was used: this index suggested a moderate to high internal consistency-perfectly in line with the previous (long) versions of the inventory [31]. Correlations between subscales were small to moderate, suggesting a different degree-and a different strength-of association between sport psychological skills.
In Table 5, mean scores and standard deviations are shown for each between-player category (male youth vs. male junior vs. female youth vs. female junior) in relation to each sport psychological skill. The analysis of multivariate variance and the Games-Howell post hoc contrasts analyses of variance revealed statistical differences between the four aforementioned player categories for all of the PSIS-Y-SF subscales-with the exception of the AC and MP subscales. These results are in line with the original PSIS version that showed good discriminant ability [8]. Moreover, these results are coherent with previous studies that suggest gender and age differences in motivation [71,72] as well as self-confidence, concentration, team emphasis, and mental preparation [7,8,72]. More in detail, considering the focused contrasts, in the Motivation subscale the "male junior" group showed a lower motivation than all of the other players' categories ("female junior", "female youth", and the "male youth" group). Regarding the Self-Confidence subscale, the "male youth" group showed statistically significant higher values than the "male junior" group-as described by previous studies [8]. At the same time, in the Team Emphasis subscale, the "female junior" group showed statistically significant higher values than all of the others groups. Finally, regarding the Concentration subscale (reversed score), "female junior" group showed a better ability to concentrate than all of the other players categories ("female youth", "male youth" and the "male junior" group).
Also, OLS regression analyses provided interesting findings. Indeed, on one hand, they revealed a statistically non-significant linear association between the majority of sport psychological skills and years of training. On the other hand, in line with literature, OLS regressions pointed out that a higher amount of training hours is associated with higher sport psychological skills (Table 6).
Despite these promising findings, several limitations have to be highlighted to this study. First of all, despite the sample size was adequate to correctly perform a CFA, it was too small to perform measurement invariance analyses. However, it should be highlight that measurement invariance analysis was not the main purpose of this study-indeed, this research aimed to develop a short form of a scale and to test its psychometric proprieties. Moreover, the "small sample size issue" could be solved by future researches that should increase the sample size. Moreover, the lack of a second measurement over the time does not allow performing longitudinal analyses as test-retest reliability, temporal stability, and longitudinal invariance. Future studies could address this issue by creating longitudinal research designs. Such designs are necessary anytime research aim is determining which psychological factors are important for successful sport performance over athlete's development from beginner to elite. As well as, longitudinal research designs should be preferred when determining which youth athlete's psychological factors could predict successful sport performance in elite athlete. In addition, it has to be underlined that this research was based on a sample composed by volleyball-players only and that no other questionnaires were administered. These two issues limit the generalizability (external validity) of the findings discussed above. Thus, future research studies should resolve this issue by administering a complete tests battery to a more heterogeneous sample. Moreover, a classical P-value approach was used in this study (P<0.050). However, according to the current debate on the usage of statistical significance [73], further indices (e.g., effect sizes) and/or statistics (e.g., uncertainty intervals) should be taken into account in order to carefully evaluate (non-)significant result as evidence for (no) effect/association. Finally, despite in the scale construction process items' semantic redundancy was strongly avoided, the use of only three items per subscale may have limited the measurement of the construct in questionwhich should therefore have been defined too narrowly. Moreover, another issue should be taken into consideration regarding brief scales. Despite-in this case-reliability indices were higher than acceptable values; brief scales could hide reliability problems. Indeed, a small number of items could lead to obtain reliability coefficients below necessary standards.
However, despite these limitations, the preliminary validation of the PSIS-Y-SF indicates that it could be a useful instrument in measuring an individual's sport psychological skills. Indeed, given the shortness of this instrument, it is likely to have high utility in both psychological and sport settings for the assessment of sport psychological skills as a core feature of performance as well as monitoring progress over the time. Moreover, the PSIS-Y-SF could be added in the planning of psychological intervention [55]. Indeed, on one hand, it could be used for the reduction of fatigue [74][75][76][77][78] and/or to improve the motivation to change [79][80][81][82] and/or to assess the need of social support to reduce psychological distress [83][84][85][86]. On the other hand, it could be used to increase athletes' self-esteem [87][88][89][90] and self-awareness [91][92][93] as well as emotions, flourishing, and well-being [94][95][96][97][98].
Moreover, this inventory provides a starting point for planning sport psychological intervention, because it provides an opportunity for sport psychologists-and other sport professional figures-to use the lower scored factors to convey to the athlete their understanding of these issues and a willingness to work with them on these difficulties.
Besides this utility, the PSIS-Y-SF provides a valid and reliable instrument to be used in research in providing six measures of performance predictors which can be tracked for changes over the time, especially following (psychological) treatment.

Conclusions
The present research has a possibility of practical application of its findings in the measurement of psychological skills of groups of players in volleyball clubs and national volleyball associations. By means of this measure, coaches in clubs and national associations could get support in making decisions about orientation of individual players for performing certain roles or player specialization at junior age.
It could be concluded that there is an initial strong indication supporting the use of the PSI-S-Y-SF, with additional item selection (i.e., item analysis), as a quality-measuring instrument for measuring psychological skills in sports (i.e., volleyball players). Thus, this preliminary validation of the PSIS-Y-SF indicated that this instrument could be considered as a useful tool to measure individual's psychological skill in sports.