https://orcid.org/0000-0002-7114-8884
Figures
Abstract
In some circumstances rule-governed behavior, a behavior that is governed by verbal rules instead of environmental consequences, may be beneficial for human beings. At the same time, rigid rule following is associated with psychopathology. Thus measurement of rule-governed behavior may be of special use in a clinical setting. The aim of this paper is to assess the psychometric properties of Polish adaptations of three questionnaires measuring generalized tendency to engage in various types of rule-governed behaviors: Generalized Pliance Questionnaire (GPQ), Generalized Self-Pliance Questionnaire (GSPQ), Generalized Tracking Questionnaire (GTQ). A forward-backward method was used for translation. Data was collected from two samples: general population (N = 669) and university students (N = 451). To measure the validity of the adapted scales the participants filled in a set of self-assessed questionnaires: Satisfaction with Life Scale (SWLS), Depression, Anxiety, and Stress Scale– 21 (DASS-21), General Self-Efficacy Scale (GSES), Acceptance and Action Questionnaire–II (AAQ-II), Cognitive Fusion Questionnaire (CFQ), Valuing Questionnaire (VQ) and Rumination—Reflection Questionnaire (RRQ). The exploratory and confirmatory analyses confirmed the unidimensional structure of each of the adapted scales. All of those scales presented good reliability (internal consistency measured with Cronbach Alpha) and item-total correlations. The Polish versions of questionnaires presented significant correlations in the expected directions with relevant psychological variables in line with the original studies. The measurement occurred invariant across both samples as well as gender. The results provide evidence that Polish versions of GPQ, GSPQ and GTQ present sufficient validity and reliability to be used in the Polish-speaking population.
Citation: Dudek J, Cyniak-Cieciura M, Ostaszewski P (2023) The Polish adaptation of the measurements of rule-governed behaviors: Generalized Pliance Questionnaire, Generalized Tracking Questionnaire and Generalized Self-Pliance Questionnaire. PLoS ONE 18(4): e0283795. https://doi.org/10.1371/journal.pone.0283795
Editor: Marco Innamorati, Universita degli Studi Europea di Roma, ITALY
Received: November 4, 2022; Accepted: March 19, 2023; Published: April 5, 2023
Copyright: © 2023 Dudek et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data and codes are available at: DOI:10.17605/OSF.IO/PS4RM.
Funding: The research was supported financially by the Faculty of Psychology in Warsaw, SWPS University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
To provide a behavior-analytic account of complex human behavior such as thinking and problem solving, in 1966 Skinner [1] introduced the term rule-governed behavior (RGB), defined as behavior that is under the control of rules and instructions, in contrast to contingency-shaped behaviors which are under control of direct contingencies in the environment. The functional behavioral analysis of the concept was first proposed almost 20 years later by Zettle and Hayes [2] and elaborated in detail within the framework of the Relational Frame Theory [3].
Beginning with Skinner, researchers emphasize that the ability to generate and follow verbal rules may be beneficial especially in contexts where learning through direct experience is dangerous (e.g. look both ways before crossing the street) or contingencies are delayed (e.g. attend classes and study to get the diploma). Thus, RGB helps people to achieve goals, learn from the experience of others, and cope with events before they occur [4]. However, under certain circumstances, RGB can also produce undesired consequences, such as insensitivity to real environmental contingencies, rigidly following verbal rules despite their effectiveness or even harmful outcome of rule following, or persistent avoidance [5–7].
Therefore, the concept of RGB has become particularly important in the domain of clinical behavior analysis, as it provides both explanation of the development of a number of psychopathological symptoms [5–8], and helps to develop psychotherapeutic interventions, such as Acceptance and Commitment Therapy (ACT) with its focus on the psychological flexibility model [9]. To support both basic and clinical research on the RGB, reliable and valid methods of assessing the behaviors are required. The aim of the present paper is to present the validation process of the three recently developed questionnaires measuring two functional types of RGB, generalized pliance (and self-pliance) and generalized tracking [10–12] in a Polish population.
Different functional types of RGB were first introduced by Zettle and Hayes [2], who distinguished pliance and tracking as two most fundamental functional classes of RGB, and a third type, augmenting, operating together with the two former classes by verbally changing the reinforcing or punishing strength of consequences included in the rules.
Pliance is defined as a functional class of rule-governed behavior under the control of history of multiple interactions in which the speaker provides the listener with the reinforcement contingent on the correspondence between the rule (e.g. do not touch hot pot) and the relevant behavior (refraining from touching the pot). An example of reinforcement in such circumstances may be praising the individual (e.g. great that you did not touch the pot [2, 3, 13]). Taking into account that the listener and the speaker may be the same person [14], under some circumstances those rules can also be generated by the individual and then called self-rules [10, 11].
Pliance, being the first type of rule-following developed [13], over-generalizes at some point in the child’s development. Yet, social interactions lead to contextualizing pliance (so that the child can distinguish when it is appropriate) and establishing tracking to help her to recognize natural consequences of her behavior [5, 7].
Lack of such learning history may lead to generalized pliance [11]. Generalized pliance can be problematic when it becomes the main source of impact on human behavior, as socially mediated consequences are less predictable and controllable than other types of consequences which may lead to lower contact with sources of positive reinforcement. Individuals displaying generalized pliance may be particularly insensitive to direct contingencies (e.g. a person believes that in order to obtain social approval she needs to start smoking, so she does that ignoring the negative consequences of smoking). As the child develops and gains fluency in relational framing, plys become more abstract (e.g. a person believes she needs to be a ‘good’ in order to be loved [15]) which may increase the likelihood social approval to be the main source of reinforcement.
As many of social rules treat aversive private events as events that need to be controlled or avoided, individuals displaying generalized pliance are more likely to engage in rigid patterns of behaviors and a tendency to engage in experiential avoidance—attempts to avoid, control or get rid of unwanted internal experiences even when doing so is harmful for the individual [5, 16].
Considering that generalized pliance may lead to losing the contact with the sources of positive reinforcement and behaviors being controlled by negative reinforcement (avoidance) it is considered a risk factor for developing psychopathology (e.g. [5, 6, 8] and psychological inflexibility–difficulties in engaging in meaningful actions due to the presence of unpleasant internal experiences that the person wants to avoid, get rid of or control [4].
In child development, pliance is seen as a condition to develop tracking [13]. In contrast to pliance, tracking is sensitive to the direct environmental contingencies, so that if they change the individual is changing the behavior accordingly [2] and it may be regarded as a flexible rule-governed behavior [12]. Tracking is a functional class of behaviors under the control of the history of multiple exemplars in which following the rule leads to natural consequences derived from the way the world is arranged (e.g. following the rule “when it is cold, wear a warm coat” leads to feeling warm even when it is cold outside [2, 3, 13]). Generalized tracking is a pattern of behaviors that may be developed when an individual has been exposed to multiple interactions in which she has been encouraged to observe and describe functional relationships among events, e.g. recognize natural consequences of her behavior. The individual engaged in tracking, behaving both as speaker and listener, learns to establish functional relationships among events and adjust her behavior accordingly [12]. Despite the interest in rule-governed behaviors especially in the area of contextual behavioral science, there are a number of limitations to experimental research investigating pliance and tracking [17]. One of the proposed explanations involves noting that both pliance and tracking are listener-oriented concepts [18] and therefore they cannot be produced by the speakers as the participants’ personal learning history may influence their performance more than the experimental rules [11, 12].
Thus, researchers proposed alternative strategy of measuring pliance and tracking by developing self-report measures, which explore the perspective of the listener and investigate the individual’s learning history and personal experience with formulating and following rules [11, 12]. A measure of generalized pliance, Generalized Pliance Questionnaire (GPQ, [11]), self-pliance, Generalized Self-Pliance Questionnaire (GSPQ, [10]), and a measure of generalized tracking, Generalized Tracking Questionnaire (GTQ, [12]), were created.
Empirical evidence with the GPQ shows that generalized pliance is connected to various aspects of psychological inflexibility and measures of distress [11]. Higher scores in the GPQ were predictive of lower levels of mindfulness and sensitivity to changing contingencies [19]. Generalized pliance was positively correlated with repetitive negative thinking, dysfunctional attitudes, difficulties in valued living, and negatively correlated with life satisfaction [11].
Generalized tracking measured with GTQ was negatively correlated with generalized pliance, experiential avoidance, tendency to ruminate and emotional symptoms and positively correlated with valued living, life satisfaction and general self-efficacy and a wide range of executive functions [12].
Measuring self-reported patterns of rule-governed behaviors, such as the GPQ and GTQ, may lead to development of research on complex human behavior by broadening the knowledge on rule-governed behaviors, its impact and development (e.g. differences across age, gender, and cultures). These instruments may also be used to explain variability of results in experimental analyses, predict development of psychopathology and behavioral rigidity, and eventually, to analyze mediators and moderators of psychological interventions (e.g. Acceptance and Commitment Therapy). Thus, it seems important to provide researchers and practitioners in the area of clinical behavior analysis and contextual behavioral science with appropriate questionnaires in countries and cultures in which people speak other languages than English or Spanish, such as Polish. The strategy of adopting already existing questionnaires with good psychometric qualities seems a good step, because it allows multilingual and multicultural comparisons of the same phenomena. We hope that the adaptation of GPQ. GSPQ, and GTQ into Polish language will aid in the development of contextual behavioral science in general, allow for multicultural analysis of the RGB, and most of all, provide a growing number of researchers and practitioners in Poland with valid questionnaires measuring RGB, widening the scope of both basic and applied research conducted.
Materials and methods
Scales’ adaptation
The original items of the Generalized Pliance Questionnaire (GPQ18–18 items, and its shorter version—GPQ9), Generalized Tracking Questionnaire (GTQ– 11 items), and Generalized Self-Pliance Questionnaire (GSPQ– 12 items) were translated to Polish by three independent translators, who were practitioners in third wave cognitive-behavioral approaches and/or experts in contextual behavioral science. Then, three independent versions of the Polish items were presented to three independent judges–scientists and practitioners in the area of contextual and behavioral science as well as third wave cognitive-behavioral therapy. Final versions of items’ translations were chosen based on the judges’ opinions, back-translated to English and accepted by the author of the original questionnaires. The original instructions and response scales were kept (see [10–12], thus each item was rated on a scale from 1 to 7 (with 7 = always true and 1 = never true). These versions of the scales were used in the study aiming at verifying their psychometric properties.
Participants
The psychometric properties of Polish versions of GPQ18, GPQ9, GTQ, and GSPQ were checked in two independent studies and different samples described below.
Sample A–students’ sample
The participants were recruited via university SONA research panel. Students received credits for taking part in the study in accordance with the university policy. The study was closed after the data from at least 400 participants was collected. The only exclusion criteria were being less than 18 years old. A total of 451 people completed the study: 371 women (82.3%) and 80 men (17.7%) in the age of 18–64 (M = 27.61, SD = 8.27). Most of them had a secondary educational level (N = 245, 54.3%), 206 (40.7%) declared higher education. Most of the participants were living in the city with more than 500 000 residents (N = 252, 55.9%) and the least—in the rural area (N = 48, 10.6%). A total of 87 people (19.3%) were living in a city with less than 100 000 residents and 64 people (14.2%) in a city with 100–500 000 residents.
Sample B–a general sample
The participants were recruited by the Pollster research panel (https://pollster.pl/), the only exclusion criteria were being less than 18 years old. The data collection was stopped when data from at least 600 participants was collected. A total of 669 people completed the study: 333 women (49.8%) and 336 men (50.2%) in the age of 18–65 (M = 40.96, SD = 13.49). Most of them had a secondary educational level (N = 360, 53.8%), 272 (40.7%) declared higher education and 37 (5.5%) finished only primary school. Most of the participants were living in the city with less than 100 000 residents (N = 254, 38%), 25.7% (N = 172)—in the city with 100–500 000 residents, 18.7% (N = 125) in the rural area, and 17.6% (N = 118) in a city with more than 500 000 residents.
Procedure
The study was conducted online between March—June 2021. The participants were asked to fill in a set of self-report questionnaires: a short demographic questionnaire, the Polish versions of GPQ18, GTQ and GSPQ and a few other measures aiming at verifying GPQ18, GPQ9, GTQ and GSPQ’s validity (their short description is presented below). All of the participants signed an informed consent. The study was conducted following the Declaration of Helsinki and received a positive opinion from the local Ethics Committee (Nr 5/2021).
Other measures
Satisfaction with Life Scale.
The Polish version of the Satisfaction with Life Scale (SWLS [20, 21]) was used to measure self-perceived well-being. It consists of five items. Participants rated each item on a 7-point scale, ranging from 7 = strongly agree to 1 = strongly disagree. Higher scores indicate a greater level of life satisfaction. Medium to large negative correlations were expected between SWLS and GPQ18, GPQ9, GSPQ, and positive—with GTQ.
Depression, Anxiety, and Stress Scale– 21.
The Polish version of Depression, Anxiety, and Stress Scale– 21 (DASS-21 [22, 23]) was used to measure the level of depression, anxiety and stress symptoms. It consists of 21 items and a 4-point Likert-type scale (3 = applied to me very much, or most of the time; 0 = did not apply to me at all). It contains three subscales: Depression, Anxiety, and Stress with higher scores indicating higher levels of symptoms. Medium to strong positive correlations were expected between DASS21 and GPQ18, GPQ9, and GSPQ, as well as negative with GTQ.
General Self-Efficacy Scale.
The Polish version of General Self-Efficacy Scale (GSES [24, 25]) was used to measure self-perceived self-efficacy. It comprises ten items assessed on a 4-point Likert scale (1 = not at all true, 4 = exactly true), which enable to calculate a general score (the higher the score, the higher the level of general self-efficacy). Medium negative correlations were expected between GSES and GPQ18, GPQ9, GSPQ, and positive with GTQ.
Acceptance and Action Questionnaire–II.
The Polish version of the Acceptance and Action Questionnaire-II (AAQ-II [26, 27]) was used to measure psychological inflexibility. It consists of seven statements. Participants rate each statement on a 7-point scale, ranging from 1 = never true to 7 = always true. Higher scores indicate higher psychological inflexibility. Medium positive correlations were expected between AAQ-II and GPQ18, GPQ9, GSPQ, and negative with GTQ.
Cognitive Fusion Questionnaire.
The Polish version of Cognitive Fusion Questionnaire (CFQ [28, 29]) was used to measure the level of cognitive fusion. The CFQ consists of seven items with a 7-point Likert-type scale (7 = always true; 1 = never true). Medium to strong positive correlations were expected between the CFQ and GPQ18, GPQ9, GSPQ, and negative–with GTQ.
Valuing Questionnaire.
The Polish version of Valuing Questionnaire (VQ [28, 30]) was used to measure the general valued living during the past week. VQ consists of ten items and a 6-point Likert scale (6 = completely true; 0 = not at all true). It contains two subscales: Progress (defined as enactment of values, including clear awareness of what is personally important, and perseverance), as well as Obstruction (defined as disruption of valued living due to avoidance of unwanted experience and distraction from values). It was expected that the VQ Progress scale will be negatively correlated with GPQ18, GPQ9, GSPQ, and positively with GTQ. Positive correlations were expected between VQ Obstruction scale and GPQ18, GPQ9, GSPQ, and negative with GTQ.
Rumination—Reflection Questionnaire.
The Polish version of Rumination—Reflection Questionnaire– 12 (RRQ12 [31, 32]) was used to measure the level of focus on one’s own experiences (rumination) motivated by fear, and the involvement in getting to know oneself (reflection) motivated by curiosity. RRQ12 consists of twelve items assessed with a 5-point Likert scale (1 –I strongly disagree, 5 –I strongly agree) and contains two subscales–Rumination and Reflection. It was expected that RRQ Rumination scale would be positively correlated with GPQ18, GPQ9, GSPQ, and negatively with GTQ. Negative correlations were expected between RRQ Reflection scale and GPQ18, GPQ9, GSPQ, and positive with GTQ.
The reliability coefficients (Cronbach Alphas) of the scales were satisfactory and are presented in Table 3.
Statistical analyses
There were no missing values within GPQ18, GTQ and GSPQ items in both samples. A cross-validation procedure was applied with the analyses done on a data from a students’ sample A and then replicated in a general sample B. Exploratory factor analysis (EFA) was conducted only on sample A and confirmatory factor analysis (CFA) only on sample B. The measurement invariance across samples (A and B) and gender was checked. Then, the corrected item-total correlations and Cronbach Alpha coefficients were calculated in two samples respectively. Finally, to provide information about the validity of the scales, r-Pearson correlations between GPQ18, GPQ9, GTQ, GSPQ and other measures were calculated (in two samples respectively). All the analyses were conducted with the use of SPSS v. 25, FACTOR v.11.05.01 [33] and R lavaan package [34].
Results
Exploratory factor analysis
The EFA was conducted with the use of FACTOR v. 11.05.01 based on the data from sample A. Data included in the analyses was categorical and according to Mardia’s test it did not meet the assumptions of the multivariate normal distribution, due to the exceeded kurtosis values (The results of Mardia’s test for the GPQ18 items: b = 31.68, Z(1140) = 23.81, p = 1.00 for skewness and b = 445.22, Z = 33.72, p < .001 for kurtosis. For the GPQ9 items: b = 5.10, Z(165) = 23.81, p = 1.00 for skewness and b = 118.02, Z = 14.35, p < .001 for kurtosis. For the GTQ items: b = 13.48, Z(286) = 1013.46, p = 1.00 for skewness and b = 202.88, Z = 37.59, p < .001 for kurtosis. For the GSPQ items: b = 10.90, Z(364) = 818.99.46, p = 1.00 for skewness and b = 208.94, Z = 23.71, p < .001 for kurtosis). Data was analyzed with the use of robust diagonally weighted least squares (RDWLS) extraction method with polychoric correlations and robust Promin rotation [33]. The number of dimensions was determined by means of the optimal implementation of parallel analysis (PA [35]). The Unidimensional Congruence (UniCo), Explained Common Variance (ECV), and Mean of Item Residual Absolute Loadings (MIREAL) indexes were used to assess the unidimensionality. Values larger than .95 and .85 in UniCo and ECV, respectively, as well as a value lower than .30 for the MIREAL suggest that data can be treated as essentially unidimensional [36]. In all the analyses the 95% bootstrap confidence intervals were estimated based on 500 samples.
GPQ18.
The Bartlett’s statistic was statistically significant (5102.1(153), p < .001), and the result of the Kaiser-Meyer-Olkin (KMO) test was satisfactory (.93, 95%CI [.90, .93]). The PA suggested extracting one factor accounting for 55.08% of variance (eigenvalue = 9.10). Table 1 shows that factor loadings were high for all the items: from .50 (item 2) to .86 (item 13). The UniCo, ECV and MIREAL values suggest that the data of the GPQ-18 can be treated as unidimensional (UniCo = .97 (95% CI [.96, .99], ECV = .90, (95%CI [.86, .90], MIREAL = .22, 95% CI [.19, .24]).
GPQ9.
The Bartlett’s statistic was statistically significant (2375.6(36), p < .001), and the result of the Kaiser-Meyer-Olkin (KMO) test was satisfactory (.89, 95%CI [.85, .90]). The PA suggested extracting one factor accounting for 59.96% of variance (eigenvalue = 4.99). Table 1 shows that factor loadings were high for all the items: from .49 (item 1) to .87 (item 4). The UniCo, ECV and MIREAL values suggest that the data of the GPQ-9 can be treated as unidimensional (UniCo = .95 (95% CI [.92, .98], ECV = .84, (95%CI [.81, .87], MIREAL = .28, 95% CI [.24, .32]).
GTQ.
The Bartlett’s statistic was statistically significant (2742.6(55), p < .001), and the result of the Kaiser-Meyer-Olkin (KMO) test was satisfactory (.92, 95%CI [.88, .92]). The PA suggested extracting one factor accounting for 60.71% of variance (eigenvalue = 6.06). Table 1 shows that factor loadings were high for all the items: from .65 (item 10) to .79 (item 7). The UniCo, ECV and MIREAL values suggest that the data of the GTQ can be treated as unidimensional (UniCo = .98 (95% CI [.97, .99], ECV = .88, (95%CI [.86, .91], MIREAL = .22, 95% CI [.17, .24]).
GSPQ.
The Bartlett’s statistic was statistically significant (2762.5(66), p < .001), and the result of the Kaiser-Meyer-Olkin (KMO) test was satisfactory (.91, 95%CI [.87, .92]). The PA suggested extracting one factor accounting for 59.69% of variance (eigenvalue = 6.07) 58.74% of variance (eigenvalue = 6.14). Table 1 shows that factor loadings were high for all the items: from .56 (item 7) to .82 (item 5). The UniCo, ECV and MIREAL values suggest that the data of the GSPQ can be treated as unidimensional (UniCo = .98 (95% CI [.98, .99], ECV = .89, (95%CI [.87, .91], MIREAL = .19, 95% CI [.14, .20]).
Summarizing, the EFA results suggest that all the scales measure unidimensional latent constructs and the one-factor solutions explain a significant portion of variance in each case.
Confirmatory factor analysis
The CFA was conducted with the use of the R lavaan package to analyze the fit of the one-factor model of the GPQ18, GPQ9, GTQ and GSPQ in a general sample B. A weighted least squares–mean (WLSM) estimation method with polychoric correlations was utilized. Goodness of fit was evaluated using the robust chi-square test, robust root-mean-square error of approximation (RMSEA), robust comparative fit index (CFI), the robust Tucker-Lewis index (TLI), and standardized root-mean-square residual (SRMR; the last one only in CFA). Hu and Bentler [37] proposed the following criteria of good model fit: RMSEA≤.10; SRMR≤.08; CFI≥.90; TLI≥.95, however recently the controversies to their application to categorical data have been raised [38]. Shi and Maydeu-Olivares [39] showed that SRMR is less sensitive to the choice of estimator, thus it is also reported. Standardized factor loading estimates are shown in Figs 1–4.
GPQ18.
The one-factor model exhibited a non-satisfactory fit due to the high value of robust RMSEA: robust χ²(135) = 2729.54, p < .001; robust RMSEA = .108 (95% CI: [.105, .112]), SRMR = .066, robust CFI = .983, robust TLI = .980. Therefore, we decided to analyse modification indices (MI) with a minimum value = 10. MI are univariate score tests that reflect the improvement of model fit after allowing some of the parameters to be free. After careful examination of MI we decided to modify the model allowing the error terms between items 16 and 17 (MI = 97.52), 10 and 11 (MI = 95.12), 4 and 5 (MI = 91.93), as well as 1 and 2 (MI = 71.56) to correlate. The decision was based on the high MI values as well as semantic equivalence of the paired items. The constraint was released one by one and resulted in the acceptable model fit: robust χ²(131) = 1937.53, p < .001; robust RMSEA = .090 (95% CI: [.087, .094]), SRMR = .056, robust CFI = .988, robust TLI = .986. The same procedure was followed in the case of GPQ9 and GSPQ, which is described below.
GPQ9.
The one-factor model exhibited a non-satisfactory fit due to the high value of robust RMSEA: robust χ²(27) = 810.56, p < .001; robust RMSEA = .129 (95% CI: [.122, .137]), SRMR = .064, robust CFI = .984, robust TLI = .979. Again, based on modification indices (MI), the error terms between items 10 and 11 (MI = 123.45), and 4 and 5 (MI = 75.76) were allowed to correlate. This improved the model fit: robust χ²(25) = 340.64, p < .001; robust RMSEA = .082 (95% CI: [.074, .90]), SRMR = .044, robust CFI = .994, robust TLI = .992.
GTQ.
The one-factor model exhibited a satisfactory fit: robust χ²(44) = 680.31, p < .001; robust RMSEA = .089 (95% CI: [.083, .095]), SRMR = .045, robust CFI = .992, robust TLI = .991.
GSPQ.
The one-factor model exhibited a non-satisfactory fit due to the high value of robust RMSEA: robust χ²(54) = 854.54, p < .001; robust RMSEA = .097 (95% CI: [.092, .103]), SRMR = .055, robust CFI = .985, robust TLI = .982. one more time, based on modification indices (MI), the error terms between items 1 and 2 (MI = 75.93), 11 and 12 (MI = 60.53) were allowed to correlate. This improved the model fit: robust χ²(52) = 553.44, p < .001; robust RMSEA = .078 (95% CI: [.072, .084]), SRMR = .048, robust CFI = .991, robust TLI = .989.
Summarizing, the results of CFA generally confirmed the one-factor solution obtained in original studies for each scale. In the case of three scales (GPQ18, GPQ9 and GSPQ) the original model presented a non-acceptable fit and some modifications were applied to obtain at least acceptable fit of the model. The final solutions present non-significant chi-square statistics (however, because of the big sample sizes this particular statistic is not a reliable source about the model fit, see [40]), satisfactory robust CFI, robust TLI and SRMR and at least acceptable values of robust RMSEA. Factor loadings were moderately or strongly related to their purported latent factor in the case of each scale.
Measurement invariance across samples (A and B) and gender
Metric, scalar and strict invariance across both samples (A and B) and gender were conducted. The relative fits of four increasingly restrictive models were compared: the multigroup baseline model (allowing factor loadings to vary across groups while the factor structure was identical across groups (i.e., configural invariance), the metric invariance model (placing equality constraints on factor loadings across groups), the scalar invariance model (placing equality constraints on factor loadings and item intercepts), and the strict invariance model (placing equality constraints on factor loadings, item intercepts and residuals). The models were compared taking into account the differences in robust RMSEA (ΔRMSEA), CFI (ΔCFI), and TLI (ΔTLI) indexes between nested models, with ΔCFI being regarded as least affected by the model complexity and sample size [41]. Although the chi-square statistics and their differences between the models are also presented, due to the big sample sizes they should not be treated as decisive; a ΔCFI, ΔTLI and ΔRMSEA less than .01 indicated invariance (see [40–43]).
The results of the analyses are presented in Table 2. Baseline models present well-fit in the case of each scale. When it comes to the invariance, it can be concluded that metric, scalar and strict invariance are supported across both samples A and B in the case of each scale except the GPQ9. At the same time, the measurement can be treated as invariant regardless of the gender in the case of each scale.
Validity
The bivariate r-Pearson correlations were calculated between GPQ18, GPQ9, GTQ and GSPQ themselves and between adapted scales and other tools measuring life satisfaction, the level of depression, anxiety and stress, the level of perceived general self-efficacy, psychological inflexibility, cognitive fusion, general valued living as well as rumination and reflection. The results are presented in Table 3. They generally support the convergent and divergent validity of adapted scales and are consistent across the samples (with differences not being tested directly).
The correlation between GPQ18 and GPQ9 were very high in both samples, suggesting that both tools measure the same construct (accordingly with the expectations). Pliance (measured by both GPQ18 and GPQ9) was strongly related to self-pliance, which was also consistent with a priori hypotheses. Tracking was weakly (and negatively) or non-significantly related to pliance and self-pliance, showing that these constructs reflect different and strongly independent rule-governed behaviors.
Pliance (as measured by GPQ18 and GPQ9) and self-pliance was moderately or strongly and positively related to the level of depressive, anxiety and stress symptoms, psychological inflexibility, cognitive fusion, obstruction of valued living and rumination. They showed weak positive or non-significant relationships with reflection, and progress in valued living. The relationship with general self-efficacy and life satisfaction was weak and negative or non-significant. These least results seem less expected, suggesting that possibly life satisfaction and self-efficacy are less affected by the level of presented pliance and self-pliance (the hypothesis worth further testing).
Tracking showed to be positively and moderately or strongly related to life satisfaction, general self-efficacy and progress in valued living and weakly with reflection. It was also negatively related (at least moderately) with depressive, anxiety and stress symptoms, psychological inflexibility, cognitive fusion, obstruction of valued living and rumination. The results are generally consistent with set hypotheses and those obtained in a study of original scale [12].
Discussion
The aim of the study was to evaluate the psychometric properties of Polish adaptations of the questionnaires measuring rule-governed behaviors such as pliance, self-pliance and tracking.
The EFA and CFA analyses corroborated unidimensional structure with high factor loadings found in the original validation studies for each of the measurements: GTQ [12], GSPQ [10] and both versions of GPQ: GPQ18 and GPQ9 [11]. Model fit to data was acceptable. All of the adapted scales presented good reliability (internal consistency) and item-total correlations.
Polish versions of questionnaires presented significant correlations in the expected directions with relevant psychological variables in line with the original studies [10–12]. Regarding ACT processes, pliance (measured by Polish versions of GPQ9 and GPQ18) and self-pliance (GSPQ) was positively related to, cognitive fusion and obstruction of valued living, and not related or negatively correlated with progress in valued living. This last score is slightly different from the results obtained by Ruiz and colleagues [11] and by Ruiz and Suárez-Falcón and colleagues [10], who showed consistent negative correlations between GPQ9, GPQ18 and GSPQ and progress in valued living. In our study, only in the student sample GPQ18 and GPQ9 were negatively correlated with progress in valued living, whereas in the general sample, as well as for GSPQ there were no significant correlations.
Pliance and self-pliance were negatively related to psychological flexibility as measured by AAQ-II [26, 27] similarly to the results obtained by Ruiz and Suárez-Falcón and colleagues [10] and Ruiz and colleagues [11] who also used AAQ-II in their studies. Despite AAQ-II have been questioned as a precise and adequate measure of psychological flexibility [44, 45], the result is in line with the results obtained by researchers that used different measurements with greater construct validity (e.g. CompACT [46]). Pliance and self-pliance were positively related with rumination and emotional symptoms in line with the original validation studies [10, 11] and showed weak and positive or non-significant relationship with reflection. Finally, they presented weak and negative or non-significant correlations with life satisfaction and self-efficacy. In contrast to the original studies, GPQ18 and GPQ9 in the general sample were not related to life satisfaction.
Finally pliance, self-pliance were positively correlated in both samples, yet pliance and self-pliance and tracking were negatively correlated only in the student sample, which is contrary to expectations and needs further replication.
Summarizing, pliance and self-pliance seem negatively related to psychological flexibility. Although the results of original studies suggest pliance may be a process leading to lower psychological flexibility and the lower level of life satisfaction, the latter was not found in the Polish sample.
Tracking, as measured by the Polish version of GTQ, was positively related to progress in valued living and negatively related with psychological inflexibility, cognitive fusion, obstruction to valued living. Moreover, it was positively correlated with life satisfaction, general self-efficacy and reflection and negatively with rumination and emotional symptoms in line with the original validation study [12]. The results support the idea that tracking, i.e. being in direct contact with contingencies and following the real consequences of behavior, is a process which may support living a valuable life, the feeling of greater self-efficacy, which may lead to greater satisfaction with life and, possibly, general health.
Despite indicating the validity of the Polish adaptations of GPQ, GSPQ and GTQ, the present study should be complemented by further research. Although in general correlations with other measures are in line with original validation studies, there may be questions regarding lack or very weak correlation of pliance with life satisfaction. At the same time, the relationship between tracking and life satisfaction as well as self-efficacy was consistent in the original and Polish study. The explanation may be the cultural differences between Polish and South American societies. The main difference between pliance and tracking is apparent source of reinforcement for rule-following: in pliance it is social or arbitrary, in tracking–non arbitrary. For some reason following the behavior considered as socially approved in Poland seems to have little impact on life satisfaction while it may still have detrimental effects for individuals’ general functioning (because of negative relationship with psychological flexibility). Differences between Irish as well as Columbian adolescents in pliance and psychological inflexibility has been recently reported by Stapleton et al. [47]. Further longitudinal research allowing for cause-effect conclusions should determine the relationship between pliance, tracking, life satisfaction and self-efficacy in cross-cultural studies.
The present study has some more limitations. First of all, the validity of the measurements should be tested in clinical samples due to its potential utility in research and practice in the area of psychopathology. Secondly, the validity of GTQ should also be confirmed in future studies including its relationship with executive functions. Further studies should include more precise and adequate measurements of psychological flexibility. Furthermore, the stability of the results needs to be established in additional study. The study has been conducted online and we have not compared the mode of administration of measurements (e.g. online vs pen-and-paper methods might be regarded by participants as providing different levels of anonymity, and this may lead to differences in how socially biased responding would be [48]). Finally, the samples were not representative of the Polish general and student’s population.
In conclusion, this study contributes with the GPQ, GSPQ and GTQ to the Polish language and they appear to be adequate to be used in Polish samples which may accelerate the development of research on pliance, self-pliance and tracking in Polish language and support clinicians working with clients.
Acknowledgments
We would like to thank our colleagues: Krystyna Pomorska, Jan Topczewski, Lidia Baran and Monika Suchowierska-Stephany, who participated in the preparation of Polish translations of the questionnaires.
References
- 1.
Skinner BF. An operant analysis of problem solving. In Kleinmuntz B, editor. Problem solving: research, method and theory. New York: Wiley; 1966. p.225–57.
- 2.
Zettle RD, Hayes SC. Rule-Governed Behavior: A Potential Theoretical Framework for Cognitive-Behavioral Therapy. In: Advances in cognitive–behavioral research and therapy Academic Press.1982, 73–118.
- 3.
Barnes-Holmes D, O’;Hora D, Roche B, Hayes SC, Bissett RT, Lyddy F. Understanding and Verbal Regulation. In: Hayes SC, Barnes-Holmes D, Roche B, editors. Relational Frame Theory: A Post-Skinnerian Account of Human Language and Cognition. Boston, MA: Springer US; 2001. p. 103–17. Available from: https://doi.org/10.1007/0-306-47638-X_6
- 4.
Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: An experiential approach to behavior change. New York, NY, US: Guilford Press; 1999.
- 5.
Luciano C, Valdivia-Salas S, Ruiz F. The self as the context for rule-governed behavior. In In McHugh, Stewart (Eds.), The self and perspective taking: Research and applications, Context Press, Oakland, 2012. p. 143–60.
- 6. McAuliffe D, Hughes S, Barnes-Holmes D. The Dark-Side of Rule Governed Behavior: An Experimental Analysis of Problematic Rule-Following in an Adolescent Population With Depressive Symptomatology. Behav Modif. 2014 Jul 1;38(4):587–613. pmid:24509717
- 7. Törneke N, Luciano C, Salas SV. Rule-governed behavior and psychological problems. Int J Psychol Psychol Ther. 2008;8:141–56.
- 8. Monestès JL, Villatte M, Stewart I, Loas G. Rule-Based Insensitivity and Delusion Maintenance in Schizophrenia. Psychol Rec. 2014 Jun 1;64(2):329–38.
- 9.
Hayes SC, Strosahl KD, Wilson KG. Acceptance and commitment therapy: The process and practice of mindful change, 2nd ed. New York NY, US: Guilford Press; 2012.
- 10. Ruiz FJ, Suárez-Falcón JC, Segura MA, García-Martín MB, Henao AM, Bedoya-Valderrama L. Development of the Generalized Self-Pliance Questionnaire. Paper presented at: The ACBS World Conference 18; 2020 Jul 14–16, online.
- 11. Ruiz FJ, Suárez-Falcón JC, Barbero-Rubio A, Flórez CL. Development and initial validation of the Generalized Pliance Questionnaire. J Context Behav Sci. 2019 Apr 1;12:189–98.
- 12. Ruiz FJ, García-Martín MB, Suárez-Falcón JC, Bedoya-Valderrama L, Segura-Vargas MA, Peña-Vargas A, et al. Development and initial validation of the Generalized Tracking Questionnaire. PLOS ONE. 2020 Jun 11;15(6):e0234393. pmid:32525904
- 13.
Hayes SC, Zettle RD, Rosenfarb I. Rule-Following. In: Hayes SC, editor. Rule-Governed Behavior: Cognition, Contingencies, and Instructional Control. Boston, MA: Springer US; 1989. p. 191–220. Available from: https://doi.org/10.1007/978-1-4757-0447-1_6
- 14.
Skinner BF. Verbal behavior. New York: Appleton-Century-Crofts.1957.
- 15. Monestès JL, Greville WJ, Hooper N. Derived insensitivity: Rule-based insensitivity to contingencies propagates through equivalence. Learn Motiv. 2017;59:55–63.
- 16. Hayes SC, Wilson KG, Gifford EV, Follette VM, Strosahl K. Experiential avoidance and behavioral disorders: A functional dimensional approach to diagnosis and treatment. J Consult Clin Psychol. 1996;64:1152–68.
- 17. Kissi A, Hughes S, Mertens G, Barnes-Holmes D, De Houwer J, Crombez G. A Systematic Review of Pliance, Tracking, and Augmenting. Behav Modif. 2017 Sep 1;41(5):683–707. pmid:28776429
- 18.
Hayes SC, Hayes LJ. The Verbal Action of the Listener as a Basis for Rule-Governance. In: Hayes SC, editor. Rule-Governed Behavior: Cognition, Contingencies, and Instructional Control Boston, MA: Springer US; 1989 p. 153–90. Available from: https://doi.org/10.1007/978-1-4757-0447-1_5
- 19. O’Connor M, Byrne P, Ruiz FJ, McHugh L. Generalized Pliance in Relation to Contingency Insensitivity and Mindfulness. Mindfulness. 2019 May 1;10(5):833–40.
- 20. Diener E, Emmons RA, Larsen RJ, Griffin S. The Satisfaction With Life Scale. J Pers Assess. 1985 Feb 1;49(1):71–5. pmid:16367493
- 21. Jankowski KS. Is the shift in chronotype associated with an alteration in well-being? Biol Rhythm Res. 2015 Mar 4;46(2):237–48.
- 22. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995 Mar;33(3):335–43. pmid:7726811
- 23. Makara-Studzińska M, Tyburski E, Załuski M, Adamczyk K, Mesterhazy J, Mesterhazy A. Confirmatory Factor Analysis of Three Versions of the Depression Anxiety Stress Scale (DASS-42, DASS-21, and DASS-12) in Polish Adults. Front Psychiatry 2022 Jan 4;12:770532. pmid:35058818
- 24.
Schwarzer R, Jerusalem M. (1995). Generalized Self-Efficacy scale. In Weinman J, Wright S, Johnston M, Measures in health psychology: A user’s portfolio. Causal and control beliefs, Windsor, UK: NFER-NELSON. 1995, (pp. 35–37).
- 25.
Juczyński Z. Narzȩdzia pomiaru w promocji i psychologii zdrowia. Pracownia Testów Psychologicznych Polskiego Towarzystwa Psychologicznego. Warszawa. 2009.
- 26. Bond FW, Hayes SC, Baer RA, Carpenter KM, Guenole N, Orcutt HK, et al. Preliminary Psychometric Properties of the Acceptance and Action Questionnaire–II: A Revised Measure of Psychological Inflexibility and Experiential Avoidance. Behav Ther. 2011 Dec 1;42(4):676–88. pmid:22035996
- 27. Kleszcz B, Dudek JE, Białaszek P, Bond FW. The Psychometric Properties of the Polish Version of the Acceptance and Action Questionnaire-II (AAQ-II). Stud Psychol. 2018 Jun 5;56(1):1–19.
- 28.
Baran L, Hyla M, Kleszcz B. Elastyczność psychologiczna. Polska adaptacja narzędzi pomiarowych dla praktyków i badaczy Katowice: Wydawnictwo Uniwersytetu Śląskiego; 2019. Available from: https://rebus.us.edu.pl/handle/20.500.12128/12945
- 29. Gillanders DT, Bolderston H, Bond FW, Dempster M, Flaxman PE, Campbell L, et al. The Development and Initial Validation of the Cognitive Fusion Questionnaire. Behav Ther. 2014 Jan 1;45(1):83–101. pmid:24411117
- 30. Smout M, Davies M, Burns N, Christie A. Development of the Valuing Questionnaire (VQ). J Context Behav Sci. 2014 Jul 1;3(3):164–72.
- 31.
Słowińska A, Zbieg A, Oleszkowicz A. Kwestionariusz Ruminacji-Refleksji (RRQ) Paula D. Trapnella i Jennifer D. Campbell—polska adaptacja metody. Paul D Trapnell and Jennifer D Campbell’s Rumination-Reflection Questionnaire (RRQ)—polish adaptation of the measure. 2014; Available from: https://repozytorium.ukw.edu.pl///handle/item/1371
- 32. Trapnell PD, Campbell JD. Private self-consciousness and the five-factor model of personality: Distinguishing rumination from reflection. J Pers Soc Psychol. 1999;76:284–304. pmid:10074710
- 33. Lorenzo-Seva U, Ferrando PJ. FACTOR: a computer program to fit the exploratory factor analysis model. Behav Res Methods. 2006 Feb;38(1):88–91. pmid:16817517
- 34. Rosseel Y. lavaan: An R Package for Structural Equation Modeling. J Stat Softw. 2012 May 24;48:1–36.
- 35. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. 2011 Jun;16(2):209–20. pmid:21500916
- 36. Ferrando PJ, Lorenzo-Seva U. Assessing the Quality and Appropriateness of Factor Solutions and Factor Score Estimates in Exploratory Item Factor Analysis. Educ Psychol Meas. 2018 Oct;78(5):762–80. pmid:32655169
- 37. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999 Jan 1;6(1):1–55.
- 38. Xia Y, Yang Y. RMSEA CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behav Res Methods. 2019 Feb 1;51(1):409–28.
- 39. Shi D, Maydeu-Olivares A. The Effect of Estimation Methods on SEM Fit Indices. Educ Psychol Meas. 2020 Jun;80(3):421–45. pmid:32425213
- 40. Cheung GW, Rensvold RB. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Struct Equ Model Multidiscip J. 2002 Apr 1;9(2):233–55.
- 41. Meade AW, Johnson EC, Braddy PW. Power and sensitivity of alternative fit indices in tests of measurement invariance. J Appl Psychol. 2008 May;93(3):568–92. pmid:18457487
- 42. Chen FF. Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Struct Equ Model Multidiscip J. 2007 Jul 31;14(3):464–504.
- 43.
Xu H, Tracey TJG. Use of multi-group confirmatory factor analysis in examining measurement invariance in counseling psychology research. Eur J Couns Psychol [Internet]. 2017 Feb 14 [cited 2022 Sep 18];6(1). Available from: https://www.psycharchives.org/en/item/ebd7c42d-bfdb-4399-b6f4-b54708edb8d0
- 44. Doorley JD, Goodman FR, Kelso KC, Kashdan TB. Psychological flexibility: What we know, what we do not know, and what we think we know. Soc Personal Psychol Compass. 2020;14(12):e12566.
- 45. Tyndall I, Waldeck D, Pancani L, Whelan R, Roche B, Dawson DL. The Acceptance and Action Questionnaire-II (AAQ-II) as a measure of experiential avoidance: Concerns over discriminant validity. J Context Behav Sci. 2019;12:278–84.
- 46. Waldeck D, Pancani L, Tyndall I. An examination of the construct validity of the Generalized Pliance Questionnaire. J Context Behav Sci. 2019;11:50–4.
- 47. Stapleton A, Ruiz FJ, McHugh L. Comparative investigation of adolescents’ generalized pliance and psychological inflexibility across cultural contexts. Psychol Rec. 2022;72:105–9.
- 48. Ward P, Clark T, Zabriskie R, Morris T. Paper/Pencil Versus Online Data Collection. J Leis Res. 2014 Mar 1;46(1):84–105.