Job stress and its measurement

Nowadays, job stress is by far the most significant source of stress for adults and it has been escalating over the past few decades [35–37]. Therefore, its identification at an early stage is an important aim in the workplace to avoid prolonged and negative consequences limiting work satisfaction, motivation, and well-being of employees, as well as potential resignation from work [38–40].

According to the literature and research, stress is a nonspecific response of a person to specific, individually assessed stressors [35, 41]. In the transactional model of stress proposed by Lazarus and Folkman [42] stress is defined as “a particular relationship between the person and the environment that is appraised by the person as taxing or exceeding his or her resources and endangering his or her well-being” [42]. Following this approach, subjective perceptions of an individual appraisal of the stressor as threatening, resulting with anxiety, discomfort, emotional tension, and difficulty in adjustment, are the crucial components of the stress [35, 41, 42]. Although there is no single definition of stress that is universally accepted [35, 43, 44], these two elements: subjectivity of stress experience and insufficient resources to cope are the most commonly shared elements in all the existing definitions [35, 42, 45, 46].

These two definitional elements are the basis of a commonly used scale assessing perceived stress, namely the Perceived Stress Scale [41], present in the literature and practice in several versions of different length, starting from the 14-item PSS-14 through the most popular 10-item PSS-10 [41, 47, 48] up to the shortest, 4-item version, i.e., PSS-4 [47, 49]. Based on stress-appraisal theory [42], all versions of PSS evaluate the degree to which an individual (here employee) has experienced distress and negative feelings stemming from perceiving life situations and the feeling that they cannot cope with a specific situation or stimulus [41, 43, 50]. The PSS is composed of items that measure how uncontrollable, unpredictable, and overloaded individuals evaluate their life situation [41] in different contexts, e.g. the workplace. Currently, the PSS-4 is commonly used and recommended in situations where a very short scale is required, like Internet surveys or telephone interviews [41, 47]. The PSS-4 is the scale with the combination of good psychometric properties with the ease of administrating in the applied settings.[47, 49] In addition to the above-mentioned definitional components of stress, there is also agreement between researchers that the identification of specific stressors is associated with many difficulties, because people often misattribute their feelings of stress to a particular source when that stress is actually due to another source [41, 44]. Therefore, more promising in assessing the level of stress is rather to evaluate a context area, which may represent a potential source of psychological harm to the individuals. One of such contexts is job stress; i.e., stress perceived by the employees in the context of workplace. With reference to this context, stress is defined as a particular individual’s subjective awareness or feeling of personal dysfunction as a result of perceived conditions or events occurring in the work settings [50]. Similarly to the definition of general stress [51, 52], subjective appraisal is identified as evoking an emotional reaction to stress and different coping strategies [42]. Consequently, because of the importance of subjective appraisal, the occurrence of the same situation may be perceived by one employee as stressful, and by another as not [42]. Therefore, self-descriptive measures of stress, asking participants about their subjective experience are very common in the literature and practice [14, 41, 43, 53]. Almost all existing measures of stress concentrate on these subjective feelings of employees, more commonly on emotional response to stimuli, such as psychological distress [54], which was confirmed in multiple studies as the first sign that environmental job demands exceed one’s perception of the ability to cope [35, 50, 53, 55], or a sign of perceiving the situation at work as threatening [54, 56–58]. One of the measures used to assess the level of stress in the workplace was the Perceived Stress Scale (PSS) [41], although it offers only a proxy for the perception of stress at the workplace. The PSS is aimed at measuring global perception of stress, without differentiating between life and work stress. These two types of stress are interrelated, and the PSS has often been, and continues to be, used as a measure of work-related stress due to the scale’s universality and the ease of relating the measurement to the context of, for example, the workplace. However, more specific job stress measures were also proposed [36, 59–61]. Further measures of stress at work are multidimensional scales, such as Job Content Questionnaire (JCQ) [60], Effort-Reward Imbalance Questionnaire (ERI-Q) [62], or Occupational Stress Indicator (OSI) [63]. All of the scales indicated to have good psychometric properties in different cultural contexts [64]. However, they are aimed at measuring specific antecedents of work related stress, such as e.g., decision latitude, psychological demands at work, social support in JCQ, lack of fairness at work in ERI-Q, work management, work schedules and work rules, among others, in OSI [14, 62, 63]. Although these factors were presented as significant predictors of stress at work [53, 65, 66], their high level does not automatically translate into a high level of stress of the employees, as research on organizational change indicated (18). As Mukerjee at al. suggested [16], employees may react to organisational changes with stress when they perceive job demands as hindrance instead of challenge. In their model of coping with stress, researchers also noted that concentrating solely on job demands cannot explain what factors enact one of these two perceptions of stress [40, 54, 67].

The role of job demands in predicting dual ways of coping with stress was defined in the Job Demands-Resources (JD-R) model [68, 69]. This approach assumed that employees’ stress in the workplace increases when the job demands, such as: high work pressure [68, 70], workload [14, 71], hassles [14], lack of control [14, 60] increase, but may be buffered by the employee’s resources utilized to balance these demands [68, 70]. The key components of individual adaptive capacity, identified in the context of job stress as personal resources are occupational self-efficacy [72], organizational-based self-esteem [73], and optimism [74].

While JD-R theory explores the role of demands-resources trade-off, the conservation of resources (COR) theory (55,56), being more general, shifts the emphasis from the balance between the job demands and employee’s resources into potential threat of losing valuable resources. In this approach, the psychological stress is defined as an individual’s reaction to an environment in which valuable resources are threatened. The COR theory underscores the critical role of gaining, retaining and protecting the valuable resources, namely personal, social and material resources in a stressful situation. Following this assumption, stress occurs when important resources are either threatened with loss, lost or not gained. Additionally, the basic principle of COR also states that resource loss is disproportionately more salient than resource gain [75], with gain cycle developing slower than loss cycle in individuals and organisations. At the organisational level, developing new resources such as new skills of employees to prepare for an upcoming change is a slower cycle than a potential loss of skills and competencies when this change has to be rapidly implemented [56]. Additionally, in occupational context, threat of losing skills and competencies and the following threat of losing a job position may lead to higher stress, even in highly competent employees.

One of the sources of job stress is organisational change [76], which has nowadays become a permanent element of occupational environment predicting the organisation sustainability in changing economic, social and ecological surroundings in many organisations [77]. In the context of organisational change, the dual model of coping with stress [16, 78], in line with the appraisal theories of stress [42], states that primary appraisal is important to identify the situation as stressful [41, 42]. Then, an individual evaluates their resources to cope with this difficult situation. CoR theory [45, 75] provides a significant theoretical background in defining these resources and their interplay in the context of organisational change [38, 75]. This theory proposes that among primary resources there are material resources, energetic resources (e.g. time), conditions (e.g. seniority) and personal resources (e.g. individual’s traits). The dual model of coping with organisational change underlies the unique role of two distinct types of resources important in this context, namely resilience of the employees, and perceived organisational support. They both determine the ways of coping with the stress evoked by organisational change (proactive or preventive), and predict commitment to change [26, 79, 80].

Digital transformation stress (DTS)

Based on previous research on job stress [14, 37, 69], we posit that the way of implementing ICT solutions in the organisation may be an important source of stress at workplace. This specific context—namely digital transformation, characterised by specific job demands and buffered by specific resources—should be distinguished from the general job stress, and technostress. According to research on organisational change [81, 82], we aimed to distinguish stress related to change as a consequence of ICT implementations and stress related to improper implementation process of this change. Moreover, we assume, following the Job Demands-Resources model of stress, that growing job demands and lack of important resources (e.g. technical and social support), evoked by digital transformation in the workplace, determine the employees’ negative emotional reactions to changes taking place in the organization during the implementation of IT solutions and new technologies [14, 56, 75, 83].

Following the conservation of resources (CoR) theory [54, 75], we also predict that some employees’ resources e.g., competencies and skills, previously pivotal for effective work, may become less adequate in a new situation, due to changes in ICT solutions. As a result, in the continuous process of digital transformation, individuals could experience negative emotions, distress, a decreasing sense of influence or control over technological changes taking place in the organization, and threat of losing their position or even their job [26, 75, 84, 85]. These theoretical assumptions provide substantial theoretical background to the construction of Digital Transformation Stress Scale.

In this paper, we distinguish digital transformation stress from technostress [27, 86, 87]. Technostress is defined as stress experienced by the individual due to their inability to adapt to new technologies because of the low level of competencies indispensable for taking advantage of modern Information Systems (IS) [5, 21]. Nowadays, many employees understand that ICT solutions and digitization are very important for organizations’ competitiveness and effectiveness [1, 9, 22] and their general attitudes and reaction to new technology are very positive [88, 89]. However, the sharp and improper implementation of new ICT solutions may increase job demands placed on employees as well as threat of losing their significant resources. In consequence, both of the above may increase job stress [89, 90], even among those employees who are highly competent in ICT [30, 89]. Therefore, we named this type of stress, stemming from digital transformation, as perceived digital transformation stress (DTS). In contrast to technostress, we assume that the DTS arises not because of negative attitudes to new technology per se, or lack of ICT skills or competencies, but because of the occurrence of a set of factors such as: 1) an improper way of implementing digital solutions and changes in workplaces [91], 2) unfit DT project management, 3) an increase in ICT demands, 4) incertitude of professional future evoked by the globality of the change and potential threat of losing valuable resources. Finally, 5) the DTS may also be related to the stress resulting from organizational changes due to digital transformation [3, 84, 92]. Therefore, DTS is a broader and more complex concept in comparison to technostress and stress related to organizational changes, due to the many factors that may cause it [55, 88, 89].

Although existing measures of technostress have contributed significantly to understanding the antecedents and consequences of stress related to information technology, they were generally not designed to evaluate stress caused exclusively by digital transformation [14, 27, 33, 86, 88]. Similarly, measures proposed in the context of organisational change are not aimed to differentiate digital transformation stress, experienced nowadays by employees in the context of COVID-19 pandemic from stress evoked by general or traditional organizational changes [84, 92].

Distinguishing digital transformation stress from other types of occupational stress is important to allow organizations to address and mitigate the consequences of the introduction of technological stressors in the workplace context [88]. Hence, there is a need to create dedicated psychometric tools to measure perceived stress due to DT process, which will enable data scientists and researchers to explore root causes of this type of stress [55, 88, 93, 94] and find ways to alleviate or address it for the benefit of employees who suffer from it, as well as the organizations undergoing digital transformation [55].

The development of the Digital Transformation Stress Scale

We developed the scale evaluated in this study to provide a short scale measuring perceived digital transformation stress–the Digital Transformation Stress Scale. The DTSS serves as a psychometric tool targeted to measure stress, caused by the style and manner of ICT tools’ implementation and management, related to time pressure, high workload, and expectations of high efficiency in the context of DT [14, 15, 19, 26]. Since our goal was to create a screening test for DTS, which, together with the sentiment analysis tool [89], set a complex instrument for perceived stress monitoring during DT projects, at intervals of short periods (4–6 weeks) [41, 47], we assumed a brief (few items), one-dimensional scale from the beginning.

We adopted here similar pragmatic approach as Marcatto et al. [79] and we proposed a small set of items, based on two scales assessing perceived stress, i.e., the Perceived Stress Scale [41] and the Perceived Stress at Work [95]. In both scales participants are asked to describe their feelings during the previous month to capture a relatively recent experience in their lives (in PSS) or at the workplace (in PSS-W). Both scales evaluate the degree to which an individual (or employee) has perceived life (of work) as unpredictable, uncontrollable, and overwhelming during the previous month.

Based on Parker, DeCotiis [50] we concentrated on measuring the DTS based on the concept of stress limited to an emotional response to stimuli that may have dysfunctional psychological or physiological consequences. Here, these stimuli were related to the ongoing process of DT in an organization. As people often misattribute sources of stress [41, 50], we decided to construct a scale containing direct inquiries about the perceived stress in the context of the digital transformation, e.g. the process of implementing ICT projects. In designing DTSS, we also assumed that the scale may be practically used as a tool to monitor the level of stress during DT process, in repeated measures design. Therefore, we decided to refer to PSS [41] in order to design our scale because it was proved that it can tap perceived stress fluctuation across different measurement points [41, 43, 55]. We asked our participants to describe their DTS in the previous recent months, because prior studies on the PSS showed that the predictive validity of the scale is expected to decrease rapidly after four to eight weeks [41]. Therefore, the perceived DTS measurement could be retested after that period and any changes could be observed. For the same reason, we wanted to obtain short feeding time (up to a few minutes) and ease to score.

Firstly, we prepared the initial list of 20 proposed items, describing symptoms of stress in the context of digital transformation [83, 96]. Accordingly, in our approach we used similar to PSS phrases referring to the sensations and the frequency of their occurrence, i.e., “how often have you felt” [41], similarly referring to previous month (or 4 weeks). However, the content of the items we contextualized to the situation of the digital transformation process, on two theories i.e., Stress appraisal [42] and CoR [75] Theories. According to Lazarus & Folkman’s (1984) stress-appraisal theory and Cohen (1983), we aim to capture whether the employee perceives the situation in the context of work, during the DT and/or ICT implementation, as stressful [41, 42]. Therefore, some items were designed to evaluate negative emotions, (i.e., upset in the first item, irritated in the second item and annoyed in the fourth), which could appear in DT process at the measurement moment. Whereas stress appraisal theory is limited to the assertion that what is perceived as stressful is what is stressful [41, 42, 75] therefore we designed some items we based on CoR theory [54, 67] Thus, two items refer to sense of control or influence: the third: “How often have you felt that you had no control over ICT changes connected with new procedures and your tasks?” and the fourth: “How often have you felt annoyed because of new work tasks/rules in connection with system/program changes whose implementation you had no influence on”. Another item (the sixth one) refers to lack of competencies and skills, i.e., specific personal resources [75] used to deal with ICT demands during ICT solution implementation: “How often have you felt that your competences and skills were insufficient to be proficient in new IT tools implemented at your workplace?”

Secondly, each proposed item was discussed with experts working in occupational psychology, financial and IT sector employees, and PhD candidates with the aim to optimize the initial list of items for further assessment [96]. Thirdly, the items were examined by the group of six competent judges (psychologists, experts in work and stress psychology), who, based on the DTS definition, on a separate questionnaire, independently evaluated the adequacy of each item. The item ratings were given on a five-point scale—from 1 to 5, where 1 meant that the item did not concern any aspect of DTS and 5 meant that the item was well aligned with the different symptoms of DTS. We used the intraclass correlation coefficient (ICC) to measure inter-rater agreement between judges [97–99].

For above mentioned (and practical reasons), from the beginning, our aim was to develop a brief and less time consuming scale [41, 47, 100, 101], possible to use as a screening test at the workplace, which meant it had to be quicker to complete [41, 47, 101]. Following evidence showing that a short form of the Perceived Stress Scale PSS-10 [41, 48, 102] obtained better psychometric properties than a longer version [47, 48, 103], we decided to select a very limited number of items, with the highest ratings of competent judges and with the high agreement of these ratings. Based on these criteria, we selected 6 items to the final DTSS, with ratings higher than 4.0 and with ICC = .82, p < .001.

We evaluated preliminary psychometric properties, namely theoretical validity, in our previous study aimed at developing and evaluating the tool for automatic detection of DT stress [89]. In that study, we used the Digital Transformation Stress Scale (DTSS) and we indicated significant associations between the level of self-reported digital transformation stress with the number of emotional markers evaluated based on sentiment analysis of issues (digital, technical, system problems) registered by employees and sent to a dedicated application defined as the help-desk [89].

Additionally, in our previous study aimed at developing internet intervention addressing DTS [55], we confirmed predictive validity of the DTS by presenting the relation between digital transformation stress, measured by DTS and job burnout, job self-efficacy and negative emotional attitudes to digital transformation [55, 104]. The level of DTS was negatively related to self-efficacy, negative emotional component of attitude towards DT, negative cognition component of attitude towards DT, and positively correlated to both burnout dimensions (with moderate association to exhaustion, and a bit weaker to disengagement). As predicted, digital transformation stress was weakly related to general positive attitudes toward digital change in the organisation [55]. Both studies confirmed that DTSS may serve as a good theoretical and predictive validity.

Aim of the studies

The central aim of our research is to validate psychometric properties of the Digital Transformation Stress Scale (DTSS), which assesses perceived digital transformation stress defined in terms of employees’ emotional response to the process of digital transformation in the workplace. To fulfil the general aim of this research, we ran two separate studies. In Study 1, we examined the initial psychometric properties of the DTS scale: factorial validity using the principal component analysis, and reliability of the scale evaluated by the Cronbach’s alpha.

In Study 2, we evaluated more advanced psychometric properties: internal structure, convergent and criterion validity, and similarities or differences between items in item functioning. We re-evaluated the assumed unifactorial structure of the scale in confirmatory factor analysis (CFA) in the structural equation modelling approach. We also assessed items functioning in Item Response Theory approach [105], to evaluate its similarities in difficulty and discrimination.

Following the results of Study 1, in Study 2, we continued to evaluate convergent and criterion validity [83, 106] presented also in our previous study [89], where we had indicated a significant association between the occurrence of negative emotion markers in sentiment analysis of helpdesk tickets in one specific organization, with digital stress level assessed using DTSS [89]. Corroborating the findings of the previous research on digital transformation stress, in Study 2 we examined the association between perceived digital transformation stress and general perceived stress at work, assuming that these two types of stress should be related, although not the same [55, 89]. Moreover, in the second study, based on the assumption that digital transformation stress does not stem from the lack of ICS skills, we also examined the association between digital transformation stress and self-assessed ICT skills. In addition, we predicted that employees working in organizations with an ongoing IT implementation project might experience a higher level of perceived digital transformation stress. Furthermore, assuming the COVID-19 pandemic caused acceleration of DT projects in organizations, we compared the level of perceived digital transformation stress in two groups of employees: those who declared that their professional life was affected by COVID-19 pandemic and those who declared they were not affected.

Participants and procedure

The participants constituted a sample of 229 adults (136 women, 75 men, 18 individuals not indicating their gender). All participants were professionally active, working in a range of diverse occupations (e.g., accountants, business analysts, financial analysts, teachers, IT specialists, and managers). A large majority of the participants (82%) had completed higher education and held a full-time job. Most of the participants were between 36 and 45 years old (36.7%). More than 50% of the participants declared their work experience to be over 10 years and nearly 36% of participants declared work seniority between 20 and 30 years. The majority of the participants used ICT in their daily work. The study was conducted in two slots, between the end of December 2019 and February 2020, and after the end of the national lockdown, between mid-June 2020 and the end of September 2020. A more detailed description of the sample is presented in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Sociodemographic information on the participants in Study 1 (n = 229).

https://doi.org/10.1371/journal.pone.0287223.t001

We mainly recruited the participants via social media, in particular LinkedIn and Facebook, as well as Messenger and WhatsApp. We also recruited participants using the snowball technique through contacts in various organizations from educational and business sectors. Because the study was aimed at employees who use ICT solutions at work, we asked a few Human Resources (HR) managers to send employees an email invitation to the study with a link to the survey. Participants accessed the survey by clicking on the link in the newsletter or email. All data was collected online via Google Forms and the organizational version of the Qualtrics platform, under the university license.

The study was conducted in compliance with ethical standards adopted by the American Psychological Association (APA 2010). The research protocol (with all text contents and compliance with the GDPR) was approved by the Ethical Committee of the University (number of decisions: 47/2020, 50/2020). Accordingly, prior to participation, all participants were informed about the general aim of the research and the anonymity of their data. After marking informed consent to the study, the questionnaire was activated. Participation was voluntary, and participants did not receive compensation for taking part in the study.

Measures

The Digital Transformation Stress Scale (DTSS) consisted of 6 items. The participants were asked to indicate on a five-points scale (1 = Never; 5 = Very often) the frequency of perceived stressful situations concerning the ICT implementation which they experienced during the last four weeks. An example of such an item is “How often have you felt annoyed because of new work tasks/rules involved with the system/program change which you had no influence on?” All items of the DTSS with their translation into English are presented in Table 2. The general indicator of digital transformation stress was prepared by averaging the answers of the participants.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Descriptive statistics and factor loadings of the Digital Transformation Stress Scale (DTSS) in Study 1 (N = 220).

https://doi.org/10.1371/journal.pone.0287223.t002

Socio-demographic information: participants were asked to indicate their age range, seniority level, gender, education level, occupation, and position in their current job.

Results

Discussion and conclusions

The main aim of the first study was to conduct an initial psychometric evaluation of Digital Transformation Stress Scale (DTSS). We examined the preliminary factorial validity and reliability of the tool. The results confirmed the good reliability of the DTSS and assumed unidimensional structure of the scale. The factor loadings of all items developed in the DTSS were moderate or high, therefore there was no need to eliminate any of the items from the Digital Transformation Stress Scale. In conclusion, the results of Study 1 provide support for the assumed structure and internal consistency of the DTSS in the sample of Polish employees. In the next study, we decided to examine the DTSS factorial structure in confirmatory factor analysis and to test theoretical validity of the DTSS by presenting its associations with other variables, e.g., general stress at work.

Participants and procedure

The participants of the second study constituted a sample of 558 adults, where 245 were female and 313 male. All participants, except one, were professionally active; most participants have experienced working remotely (n = 335, 60%), whereas 223 participants declared not to have worked remotely at all (40%). The structure of participants comprised of a range of diverse occupations like accountants, business analysts, financial analysts, teachers, IT specialists, and managers, but also engineers, receptionists etc. Majority of the participants had a master’s degree or above: 305 (54%) and 204 (37%) had a bachelor’s degree. Only 36 (6%) participants had the education equal to or lower than middle school. The average age in the sample group was 43.6. The youngest participants were 20 years old and the oldest and professionally active were 69 years old. Most participants were between 40 and 49 years old (183, i.e., 33%), and between 30 and 39 years old (167, i.e., 30%). We grouped the professional occupation declared by the participants into seven job position categories, similarly to Study 1. A more detailed description of the sample is presented in Table 3.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Sociodemographic information on the participants in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.t003

The participants were recruited to the study by a professional research agency. All data was collected in online mode only.

The research protocol (with all text contents and compliance with the GDPR) was approved by the Ethical Committee of the SWPS University of Social Sciences and Humanities, Warsaw, Poland, number of decisions: 47/2020, 50/2020, 3/2021, 8/2021. The study was conducted in compliance with ethical standards adopted by the American Psychological Association (APA, 2010). Accordingly, prior to participation, all participants were informed about the general aim of the research and the anonymity of their data. The questionnaire was activated only after the participants declared their informed consent to the study. Participation was voluntary, and the participants did not receive compensation for their participation in the study.

Measures

The Digital Transformation Stress Scale (DTSS) is a self-report scale consisting of six items. All items are presented in Table 4. The participants were asked to indicate on a five-points scale (1 = Never; 5 = Very often) the frequency of perceived stressful situations concerning the ICT implementation which they experienced during their last four weeks in the workplace. The general indicator was prepared by averaging the answers of the participants.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Descriptive statistics and factor loadings in the confirmatory factor analysis of the Digital Transformation Stress Scale (DTSS) in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.t004

The Perceived Stress Scale (PSS-4) [41, 112] comprised of four items and was based on the Polish version of PSS [113, 114], modified to relate to general stress at work [114]. Participants were asked to describe their feelings and thoughts related to their professional work during the last month using a five-point scale where 1 = Never and 5 = Almost always, e.g., “How often have you felt that you were unable to control the important things in your life at work?”.

Self-assessment ICT skills inventory—To assess specific ICT skills in different areas, we developed the 7 items of ICT skills self-assessment inventory based on The Digital Competence Framework for Citizens [115]. Firstly, participants were asked to estimate their ICT skills in the workplace in general, (“Please evaluate your computer skills in the workplace”), by indicating the answers on a 5-point scale where 1 meant Basic level—limited to elementary functionality and 5 meant Very advanced level—programming, graphic processing, computer operation of machines. There was also a possibility to mark the answer “I’m not using a computer at work”. Afterwards, respondents were asked to determine their skills in the listed area and their activity on the Internet as well. It was performed in a matrix of statements applied to different ICT skills, from using keyboard shortcuts and internet transfer to working in different programs commonly used in the workplace. Example statements are: “I can prepare a presentation in a dedicated program, I can choose the layout, background, template, charts, tables.”, “I can pay by an online bank transfer”. The responses evaluated their skills on a 5-point scale, where 1 means very low skill level and 5 means very high skill level. The reliability of the Self-assessment ICT inventory was high (Cronbach’s alpha = .88).

Digital transformation processes at the workplace–a one-item question: “Are there any implementation projects (IT) currently being carried out in the organization where you work or study, which affect your work or your activities?”. Respondents have been asked to indicate an answer among “Yes, there are”, “No, there are not”, “I do not know” and “Not applicable”.

Results

Items difficulty and discrimination.

Psychometric assessment of the items discrimination and difficulty was conducted using Item Factor Analysis in Item Response Model approach [120]. The current gold standard of estimation for IFA models is marginal maximum likelihood (MML) [121]. However, this estimation method in small samples does not provide information about global and absolute fit. Instead, we conducted assessment of model fit with weighted least squares estimator using mean and variance-corrected χ² (WLMSV). Similarly to CFA, we used CFI, TLI, RMSEA, SRMR and χ² test as measures of model fit with cut-off values described above to evaluate the goodness of the fit. Following the A two parameter model exhibited quite good fit measured by CFI = .963, TLI = .939 and SRMR = .036, but unacceptable fit by χ² test of absolute fit χ² (9) = 329.722, p < .001, and RMSEA = .253 [CI = .230, .276; p < .001]. A reduced model in which with one parameter fit significantly worse DIFFTEST(5) 24.211 p < .001. Thus, the original model was retained for further examination using MML estimation instead.

Model parameters obtained using MML and a logit link are shown in Table 5, which includes IFA item parameters (thresholds and loadings) as well as their Item Response Theory (IRT) analogous parameters of item discrimination and difficulty.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Item loadings, thresholds, discrimination, and difficulty parameters for all items of the Digital Transformation Stress Scale (DTSS) in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.t005

It can be seen that all the items have similar discrimination parameters and it can be assumed that they similarly discriminate participants with different levels of latent trait. Also, threshold parameters for all items have quite similar values with a similar spread across values of latent variable.

Fig 1 (left panel) displays the test information function. As can be seen in that plot, the scale lacks measurement precision with theta falling below -2 but has high precision in moderate and high values of the latent variable. Overall, the scale yields precise measurement for individuals with moderate to high DTS levels and relatively imprecise measurement for individuals with low DTS levels. Additionally, in Fig 1 (right panel) we present test information for ease of interpretation converted into a traditional measure of reliability that ranges from 0 to 1. It shows that test reliability is higher than .80 for latent variable values ranging from -1.8Z, which is almost 2 standard deviations below the mean. It means that for almost 95% of population the measurement reliability is higher than .80 and for almost 91% is .90 or higher. Importantly, reliability is high for highest levels of digital transformation stress.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Test information function and test reliability of the Digital Transformation Stress Scale (DTSS) in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.g001

We also present items information curves in Fig 2. As can be seen in Fig 2, one of the curves is a bit lower than others, namely the one for item 6. This indicates that overall degrees of measurement precision for this item are also relatively lower than for the rest of items. The highest plots are obtained for items 3 and 4, indicating higher measurement precision. Inspection of the item characteristics curves for all items (see S1 Fig) revealed that the pattern of curves is similar for all items, with a slightly higher curves for category 1 and category 5, and smaller for middle categories of responses. This may suggest that extreme categories of the response scale are more informative than the middle ones.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Items information functions of the Digital Transformation Stress Scale (DTSS) in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.g002

To test the reliability, we calculated Cronbach’s Alpha in IBM SPSS Statistics 27.0. The reliability was high (Cronbach’s Alpha α = .90) and it was similar to the results obtained in the first study.

Divergent validity.

Ongoing IT implementation. We expected that employees working in organizations with an ongoing digital transformation process should present higher digital transformation stress than those working in organizations with no digital transformation. Thus, we compared the level of digital transformation stress in these two groups using Student’s t test for independent samples. The results revealed that the level of DTS in employees working in organizations who implemented IT solutions was higher (M = 2.45, SD = 0.81) than for employees working in organizations without ongoing implementation (M = 2.29, SD = 0.79; t(426) = 2.14; p < .001, Cohen’s d = .21).

COVID-19 impact. The majority of participants (61.8%) stated that COVID-19 affected their professional lives, and we observed a significant difference in the level of DTS between the group of participants who did state that COVID-19 impacted their professional lives (M = 2.49, SD = 0.78) in comparison to participants who did not state that COVID-19 modified their working conditions (M = 2.24, SD = 0.79; t(556) = 3.72; p < .001, Cohen’s d = .32).

ICT skills. We verified if self-assessment of ICT skills is the factor of DTS. Thus, we examined the correlation between the DTSS and self-assessment of ICT skills inventory. For the entire sample (N = 558), we observed no correlation between two indicators: r = -.04, p = .31. Additionally, we calculated the correlation between ICT skills and digital transformation stress only for those participants who declared that there is an ongoing IT solutions implementation in their organization (see Table 6). There was significant negative correlation between self-assessment skills and digital transformation stress in the group of participants who reported that there was ongoing implementation IT process in their organisation. However, this correlation was very weak. Such a correlation was not present in the group of employees who declared that there is no such implementation in their organisation.

Download:

• PNG
  larger image
• TIFF
  original image

Table 6. Correlations among general stress, digital transformation stress and ICT skills in groups of employees working in organizations with and without ongoing digital solution implementation in Study 2 (N = 558).

https://doi.org/10.1371/journal.pone.0287223.t006

Discussions and conclusions

The first goal of Study 2 was to examine whether we could confirm the factorial structure of DTSS found in Study 1 in a new sample of professionally active participants. The second goal of the study was to evaluate the difficulty of the items and their discrimination, and the third goal was to evaluate the convergent and divergent validity of DTSS. Firstly, the results of Confirmatory Factor Analysis confirmed the unidimensional structure of Digital Transformation Stress Scale, observed in Study 1. Secondly, the level of difficulty and discrimination evaluated in IRT approach was similar for all items and the whole scale had a good reliability for moderate and high level of measure latent variable. Thirdly, confirming our predictions, the correlations coefficients indicated that the level of DTS was positively but rather moderately related with general, perceived stress in the workplace. As predicted, the correlation between the DTS and self-assessment ICT skills was also weak, confirming that this type of stress, in contrast to technostress, does not stem from lack of IT skills.

We also tested the role of COVID-19 impact, as perceived by the employees. Most of the participants stated that COVID-19 impacted their professional lives and as predicted, we observed significant differences in digital transformation stress level between two groups of participants: those who stated that COVID-19 impacted their professional life had an elevated level of DTS in comparison to those who felt unimpacted by COVID-19. The same pattern of results was observed in the presence of the factor of ongoing ICT solutions implementation processes. Employees working in organisations with ongoing implementations had a higher level of digital transformation stress in comparison to those employees who worked in the organisation without such implementations. Taken together, these results can be interpreted as prior evidence that acceleration of DT can be the source of DTS.

Limitations and future research directions

The present study has several limitations which need to be discussed. Firstly, respondents, especially in the first study, were invited mainly via social media, particularly through LinkedIn, and this may seriously limit the generalizability of our findings. The online way of conducting the study might have reduced the number of participants who were very strongly affected by DTS. Therefore, further research on the role of different modes of survey administration are necessary to assess the influence of this factor on the level of DT stress observed at the workplace

Secondly, as stated above, some limitation of our research includes the measure used to identify organizations with ongoing digital transformation process. As it was based only on self-reports provided by employees, it might not precisely capture the higher level of DTS demands related to implementations. It would be very interesting to select organizations just as they start the process of digital implementation and evaluate changes in the level of digital transformation stress of their employees longitudinally. Such research design would enable us to provide more reliable information about its causal relationship between employees’ demands, resources and stress.

Thirdly, as the preliminary set of items was rather short, quantitative research on DTS is required to identify such symptoms of DTS that may differentiate it from job stress and technostress. The following construction process may also include additional dimensions, related to secondary appraisals of stress evoked by change, identified in applied research on organisational change. As Olafsen et al. [124] proposed, an important factor linked to individuals’ reaction to change is change commitment, which can be described using three components: affective, normative, and calculative [125, 126]. These dimensions allowed to predict individual readiness to change, namely change self-efficacy, and are perceived as significant predictors of commitment to organisational change. These personal characteristics of the employees, perceived as a potential resource, may also be related to digital transformation stress.

Finally, the phenomenon of DTS is conceptually larger than only emotional dimension therefore further research on digital transformation stress should be expected. We assess that deeper analysis on digital transformation stress and digital transformation attitudes should be conducted, taking into account the multidimensional construct of stress i.e., cognitive, behavioural and emotional.

Further work should also examine how strongly DT stress impacts different work outcomes such as work commitment, job satisfaction, and burnout of employees. Additionally, more research is needed to examine the variables that allow to predict the level of DTS e.g., attitudes toward digital implementation, perceived work demands. This research should also be conducted after the COVID-19 pandemic, when DT processes will be simpler to plan and properly implemented in organizations.

Theoretical and practical implications

The general aim of our studies is to propose a self-descriptive and easy-to-administer measurement tool aimed to identify highly stressed employees who would benefit from specific psychological interventions reducing DT stress. The DTSS possesses satisfactory psychometric properties but is also attractive for reasons of cost and time effectiveness, especially in automated screening systems that might be constructed using this scale [55, 89]. Being a rather brief scale, DTSS is more user-friendly and may have a higher response rate in comparison to more elaborated measures.

In the future, DTS scale may become a substantial part of an automated system consisting of three hierarchical elements: a) preliminary screening, based on a qualitative analysis of help-desk tickets sent by employees [89], b) evaluation of the level of DT stress based on the DTSS [55] to identify highly stressed employees, c) invitation to take part in a psychological intervention, the effectiveness of which we presented in our previous study [55]. Designing and testing such systems may play a crucial role for preserving psychological well-being of the employees, especially when situation demands a rapid DT implementation at the workplace.

Moreover, in project management area [127, 128], the Digital Transformation Stress Scale may be used to compare different project management methodologies, typically used in implementing digital transformations, e.g., the waterfall technique with more modern agile techniques [128–131]. The waterfall, being more traditional, includes a set of techniques used for planning, estimating, and controlling activities [90, 96]. Using a simplification, it strictly separates implementation phases: the analysis phase, the implementation, and the testing phase. Often, final users–namely employees, are not involved in the initial stages but instead they start working with new solutions in the testing phase, very often under time pressure [131]. Agile technique is a more flexible [127, 129, 131, 132], iterative approach in which end-users are more actively taking part in the whole process and can gradually learn about and test new solutions [10, 127, 129]. Measurement of the perceived digital stress in key in these two project management methodologies. It allows to compare the level of digital transformation stress in different project phases and identify the phases in which the employees should be actively assisted. Therefore, using DTSS could be an efficient way to manage and improve the digital transformation process in organizations.

From the theoretical standpoint, DTSS can be used to further elaborate psychological antecedents (e.g., DT demands), and new sources of stress in various contexts e.g., at the workplace, in education, in the health-care systems—wherever the digital implementations are introduced in the hope of improving the effectiveness of the system. Research testing the associations between DTS and psychological well-being, burnout, disengagement is of great importance in understanding and evaluating the value of these costs.

Conclusions

The COVID-19 pandemic has highlighted the importance of digitalization in many areas of business, accelerating the digital transformation process. Although the digital maturity of organizations and employees is increasing overall, there are considerable consequences of DT. The negative effects of DT on employees appear due to the process of introducing changes under time pressure, without proper planning, and gradual implementation [33, 133]. In consequence, we observe a strong impact on employees’ comfort of work and their stress related to the acceleration of digital changes in organizations [38, 26, 27]. In our studies we present a psychometric evaluation of Digital Transformation Stress Scale designed to assess the level of digital transformation stress at the workplace. Our results confirmed good psychometric properties of the DTSS, and this may enable researchers to address the root causes of DTS with proper guidelines and interventions. Evaluation of the DTS level may also help alleviate this type of stress by helping employees deal with perceived digital transformation stress. As we presented in our previous studies, DTSS can be successfully applied as s screening tool to identify employees who suffer from this type of stress, and also to assess the effectiveness of the psychological intervention offered to reduce DTS [134]. This may also be beneficial at the organizational level by supporting businesses which would benefit from improved efficiency, satisfaction, and well-being of their employees.