https://orcid.org/0000-0002-6009-5561
Figures
Abstract
Research suggests that a jumping-to-conclusions (JTC) bias, excessive intuition, and reduced analysis in information processing may favor suboptimal decision-making, both in non-clinical and mentally disordered individuals. The temporal relationship between processing modes and JTC bias, however, remains unexplored. Therefore, using an experience sampling methodology (ESM) approach, this study examines the temporal associations between intuitive/analytical information processing, JTC bias, and delusions in non-clinical individuals and patients with schizophrenia. Specifically, we examine whether a high use of intuitive and/or a low use of analytical processing predicts subsequent JTC bias and paranoid conviction. In a smartphone-based ESM study, participants will be prompted four times per day over three consecutive days to answer questionnaires designed to measure JTC bias, paranoid conviction, and preceding everyday-life intuition/analysis. Our hierarchical data will be analyzed using multilevel modelling for hypothesis testing. Results will further elucidate the role of aberrant human reasoning, particularly intuition, in (non-)clinical delusions and delusion-like experiences, and also inform general information processing models.
Citation: Zander-Schellenberg T, Kuhn SAK, Möller J, Meyer AH, Huber C, Lieb R, et al. (2021) Is intuition allied with jumping to conclusions in decision-making? An intensive longitudinal study in patients with delusions and in non-clinical individuals. PLoS ONE 16(12): e0261296. https://doi.org/10.1371/journal.pone.0261296
Editor: Zezhi Li, National Institutes of Health, UNITED STATES
Received: November 23, 2020; Accepted: November 29, 2021; Published: December 20, 2021
Copyright: © 2021 Zander-Schellenberg et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Any pilot data reported in this submission are fully available and data collected during the study will be made fully available without restriction upon study completion; The data to be collected will be made fully open at OSF.
Funding: This research is supported by the Swiss National Science Foundation SNSF (Ambizione Fellowship no. 179897 awarded to Dr. Thea Zander-Schellenberg). The funders have no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Imagine Paula and Tom sitting together and planning their next holiday destination. Whereas Paula is comparing weather conditions, cultural offers, hotel prices and the like of different resorts, Tom simply goes with his gut feeling and chooses the destination that spontaneously feels best for him. After having chosen a specific destination, he will not be able to give rational reasons why he decided the way he did. In sum, Paula would make a list of pros and cons to extensively analyze the given alternatives, and Tom would decide rather intuitively.
In science, this differentiation of analytical versus intuitive information processing has been formalized in dual-process accounts of judgment and decision-making (see e.g., [1–5]). Intuitive Type 1 processes operate fast, affectively charged, holistically, and are based on associative memories, whereas analytical Type 2 processes are slower, effortful, intentional, and based on working memory and general cognitive capacity. In adaptive decision-making, both processes act jointly in order to guarantee conflict-free and flexible decision outcomes [3,6,7]. Thinking this through further, each mode of processing, taken by itself, has its distinct adaptive functions. Intuition, for instance, is associated with indicators of subjective wellbeing, such as experiencing meaning in life [8,9] and positive affect [10,11]. When in an intuitive mindset, people are able to spontaneously identify the essentials of the decision situation at hand by effectively integrating relevant information [12–14], feel connected to their true selves when making choices [15], adequately engage in parent-child interactions [16], and arrive at “smart” decisions under time pressure and with minimal cognitive effort [17–20]. In contrast, deliberate analysis is good for decision situations requiring extensive pondering and balancing pros and cons, and for accurate decision outcomes, for example, when solving laboratory tasks in accordance with the standard models of logic [21]. When engaging in analytical processing, people tend to perform better in lie detection [22], make everyday life decisions of self-rated and externally rated high quality [23], and are less prone to conjunction fallacies [24] and other deviations from logical reasoning such as framing effects [25].
People differ in their tendency to rely on intuition or analysis. Various validated trait questionnaires, such as the Rational Experiential Inventory (REI-40; [26]) or the Preference for Intuition/Deliberation scale (PID; [27]), have been created to capture these tendencies. Beside personal preferences [28–30] and certain characteristics of the decision situation that foster analysis or intuition (e.g., time pressure or complexity, especially in naturalistic decision-making; [31]), research suggests that also mental health conditions are intertwined with the applied processing mode [32,33]. In this context, disorders on the schizophrenia spectrum seem to be of prime interest. Research has shown that individuals suffering from schizophrenia exhibit certain reasoning abnormalities. For example, studies observed a lowered decision threshold in probabilistic reasoning tasks in this population (see [34] for a theoretical account). In this respect, many studies have robustly demonstrated that patients with schizophrenia arrive at a decision or judgment very early without gathering enough evidence to adequately process the information at hand–a phenomenon termed jumping-to-conclusions (JTC) bias (e.g., [35–38]). Moreover, reviews and meta-analyses have (a) demonstrated the existence of the JTC bias in patients with schizophrenia, especially in those who experience delusions, and in a less extreme form in healthy individuals, and (b) shown that the JTC bias distinguishes patients with schizophrenia from healthy and clinical controls [39–42]. As a result, various theoretical models on delusion formation and maintenance have incorporated the JTC bias as one potential (trait) vulnerability factor (e.g., [42,43]).
Recently, the JTC reasoning bias has been theoretically discussed in the context of dual-process accounts of judgment and decision-making [7,32,44,45]. In its classic form, these accounts propose a dynamic interaction between intuition and analysis as well as an enhanced error-proneness of intuitive Type 1 processes due to the human cognitive system, which tends to save effort whenever possible (e.g., [2,45]). In the field of delusion-associated reasoning biases, it can be surmised that, on the one hand, intuitive Type 1 processing is excessively active and/or that, on the other hand, analytical Type 2 processing is too reserved or even inactive and thus cannot override maladaptive Type 1 outcomes where necessary [32]. For example, in their Dual Stream Modulation Failure model, Speechley and Ngan [7] propose that delusions in schizophrenia arise as a result of a conflict modulation failure that prevents analytical Type 2 processes from effective functioning and correcting erroneous intuitive Type-1-based beliefs. In this respect, the JTC bias can be seen as one manifestation of an aberrant interaction between Type 1 and Type 2 processes, putatively resulting from an overreliance on maladaptive intuitive processes [32] that are unlikely to be corrected by Type 2 processes [7]. Assuming that the JTC bias forms one pillar in the creation of delusions, it can be argued that impaired analytical processes and an excessive and maladaptive use of intuitive processes engender a more pronounced JTC bias which, in turn, is linked to subsequent delusion-like or delusional beliefs. Importantly, both reasoning biases and delusions seem to be located on continuums which span slight aberrations in reasoning and non-pathological unusual experiences in the general population until severe reasoning biases and pathological delusions in individuals from the schizophrenia spectrum disorder. At the empirical level, a profound body of evidence has for instance demonstrated that both a JTC bias [39] and psychotic experiences [46,47] can be observed in non-clinical individuals, too, albeit to a lesser degree than in clinical individuals. Consequently, investigating the link between intuitive and analytical processes in relation to JTC bias is not only relevant for psychopathological models of delusions, but also for explanations of general judgment and decision-making models in healthy people. That is why, in our study, we start exactly at that point and aim to both contribute to a better understanding of delusion-related reasoning mechanisms in specific, and shed more light into the link between JTC bias and processing modes within hasty decision-making in general.
Supporting the notion of processing modes dovetailing with delusion-like beliefs, Freeman and colleagues [48], using a non-clinical sample, found small positive correlations between trait intuition preferences and persecutory ideation as well as small negative correlations between trait analysis preferences and persecutory ideation. Unfortunately, this pattern of high intuition preferences coexisting with lower levels of analysis preferences could not be replicated in a bigger follow-up study involving delusional patients with schizophrenia [49]. However, research in non-clinical samples observed a positive association of intuitive Type 1 processing with the occurrence of paranormal and superstitious beliefs and schizotypal traits in various student populations [50–54], and also with the occurrence of conspiracy theories [55,56], fake news [57], and pseudo-profound beliefs [58] in the general population. Moreover, experimental research found that intuitive processing (either laboratory-induced or naturally occurring as trait preference) can predict the belief in difficult-to-prove concepts like the belief in a God [59] and, in interaction with positive affect, also referential thinking [60] and even represent behavioral indicators of a leaning towards sympathetic magic [61]. Furthermore, a study [62] observed that intuitive decision makers rely on heuristics and thereby display several reasoning biases. At the same time, empirical data for the analytical Type 2 processes component show that, in healthy samples, impairments in analytical processing are observable both in clinical delusions [49,63] and as a correlate of paranoia [1,48] or false belief receptivity [57]. Only two studies reported a positive [54] or no association [64] of analytical processing with delusion occurrence. Regarding the JTC bias in particular, preliminary evidence suggests that premature decision-making is associated with less analytical processing [64] and linked to impairments in working memory, which is thought to be involved in analytical processing, thus aligning with the assumption that Type 1 and Type 2 processes interact in judgment and decision-making.
Importance of the research question
In a nutshell, there is initial empirical evidence [48,49] and a growing body of theoretical conceptions [7,32,44] indicating that an aberrant Type 1/Type 2 processing interaction is associated with clinical delusions as well as non-pathological delusion-like experiences and possibly also with the JTC bias (see above). However, recent contributions have focused predominantly on diminished Type 2 processing in schizophrenia (e.g., [44]) and associated concepts of belief flexibility (e.g., [63]) and metacognition [65] and have rather ignored intuitive processing and its association with the JTC bias. Thus, many questions on intuition in individuals with schizophrenia and individuals experiencing non-clinical delusion-like experiences are still open. To our knowledge, apart from one cross-sectional study [49], there are no studies directly addressing the relationship between intuition and the JTC bias. Furthermore, prior studies have used cross-sectional designs applying mostly self-report questionnaires that do not allow inferences about the temporal relationship between intuitive/analytical processing and the JTC bias.
Therefore, we aim to close this research gap by specifically focusing on everyday life processing modes in schizophrenia and individuals with non-clinical delusion-like experiences, approximated by inspecting everyday life intuitive and analytical decision-making. In doing so, we aim at both studying the link between intuition/analysis and the JTC bias in a clinical sample (where patients with schizophrenia serve as a population that typically shows distorted information processing mechanisms, see for instance [38–41]) and in a non-clinical population sample as well. Examining a non-clinical sample thus contributes to the understanding of underlying mechanisms in aberrant reasoning in general and provides further evidence on a putative psychosis continuum. To circumvent the drawbacks of cross-sectional and laboratory-based designs adopted in previous studies, we will exploit an intensive, longitudinal approach by conducting an experience sampling methodology (ESM) study. Fortunately, technological innovations nowadays allow assessing mental phenomena and behavior in intensive observational designs and as they occur in real life [66]. In schizophrenia research, ESM approaches using smartphones have demonstrated to be feasible and reliable assessment modalities [67] with high compliance rates among patients [68,69] and in high-frequency sampling schemes [70], thus minimizing recall bias and allowing examination of temporally directed associations in natural settings [71,72]. The main research question is whether a prior high use of intuitive processing in everyday life predicts the subsequent occurrence of the JTC bias in patients with schizophrenia and in non-clinical individuals. Embedding our research into the dual-process frameworks of judgment and decision-making, our secondary research question is whether a prior low use of analytical processing in daily life predicts the subsequent occurrence of the JTC bias in patients with schizophrenia and in non-clinical individuals. Applying a psychosis continuum approach, we will also tentatively examine whether delusion proneness moderates the link between aberrant intuitive and/or analytical processing and JTC bias in individuals without a diagnosis of schizophrenia.
Proposed hypotheses.
Addressing these research questions, we hypothesize that a prior high use of intuitive processing predicts subsequent JTC bias in patients with a schizophrenia disorder and in non-clinical individuals (Hypothesis 1). Also, aiming to replicate previous study results on delusion-related analytical reasoning [49] on a state level, we expect that a prior low use of analytical processing predicts subsequent JTC bias in patients with a schizophrenia disorder and in non-clinical individuals (Hypothesis 2). Given that intuitive processing emerges as a predictor of subsequent JTC bias in H1, we will also preliminarily test if prior intuition predicts subsequent paranoia in patients with a schizophrenia disorder (Hypothesis 3).
To investigate these associations on a psychosis continuum in individuals without a schizophrenia diagnosis, our fourth hypothesis is that the link between prior intuitive and/or analytical processing and subsequent JTC bias is moderated by delusion proneness (Hypothesis 4). A full description of these hypotheses along with sampling plans and intended analyses is provided in the Design Table (see Table 1 below).
To our knowledge, our hypotheses have not so far been tested in an intensive longitudinal design using a state operationalization of intuitive and analytical information processing. By addressing our research questions, we aim to further improve the understanding of putative delusion-underlying mechanisms of schizophrenia disorders. In the light of a putative psychosis continuum, we also expect to extend findings to a non-clinical sample and thus contribute to the identification of proximate factors underlying human aberrant reasoning. It is hoped that this study will not only advance the theoretical understanding within the framework of the Dual Stream Modulation Failure model [7,73], but, in the long term, may also inform clinical intervention. Showing that temporally preceding processing modes are linked to subsequent JTC bias and paranoid conviction could imply that interventions aiming to reduce clinically relevant delusions may also benefit from focusing more on processing modes rather than on the JTC bias alone (see [74] for an example of a pioneering clinical trial on this endeavor). Moreover, general information-processing models (e.g., [2]) will also be informed by our results.
Methods
Ethics information
This intensive longitudinal study forms part of a bigger project on “Intuition and jumping to conclusions in schizophrenia: Studying the temporal relationship in daily life and with neuroimaging” (SNSF grant number: 179897, see http://p3.snf.ch/project-179897). We registered the study at the Open Science Framework (see https://osf.io/8gkjx/). The study protocol and all data will be made publicly available there.
The study has been approved by the competent ethics committee and complies with all relevant ethical regulations. We will only assess participants who have provided informed consent. For patients, capacity to consent will be determined by the attending psychiatrist/psychologist. Each participant will be financially compensated with a money-equivalent local voucher, equating 130 Swiss Franc per person in total (including two fMRI sessions à 40 CHF each, maximum 12 ESM prompts à 2 CHF each prompt, a bonus of 5 CHF if at least ten of twelve ESM prompts are completed, and one clinical interview à 20 CHF).
Sampling plan
Participants.
Inclusion of 106 participants is planned. We target to recruit a total of 128 participants, allowing for a dropout rate of 20% (e.g., due to non-compliance), and will analyze data of all participants who provide valid data and fulfill all inclusion criteria. Half of the participants (N = 64) will be patients with a diagnosed ICD-10 schizophrenia disorder recruited from the local psychiatric university hospital. Diagnoses of patients will be based on diagnostic assessments conducted by trained psychologists or psychiatrists as part of the hospital admission process and will be derived from electronic records by the research staff (antipsychotic medication dose will also be derived from these records). Patients will be recruited both from psychiatric inpatient wards and from one outpatient ward of the local psychiatric university hospital. The other half of the participant sample (N = 64) will be individuals from the general population without a diagnosis of schizophrenia.
We did not conduct power analyses for the study at hand for the following reason: Given the multilevel study design, the nested data structure needs to be considered when calculating the optimal sample size a priori. Due to a current lack of intensive longitudinal studies on intuition in schizophrenia, we cannot make reliable assumptions regarding most of the important multilevel model parameters such as random predictor variances and covariances (as suggested by [70]), which, however, is indispensable when running a simulation study for nested data designs. In consultation with those of our co-authors working in clinical practice, our sample size decision is hence based upon the premise of clinical feasibility, and accounts for the rather low prevalence rates of schizophrenia disorders and the need to secure study feasibility (e.g., recruitment issues) within an adequate time span.
Inclusion and exclusion criteria.
Inclusion criteria for individuals with and without schizophrenia are: (a) fluent German, (b) IQ >70, (c) normal or corrected-to-normal vision, (d) age between 18 and 55 years, and (e) an ESM compliance rate for fixed prompts of at least 33% (following procedures in other ESM studies, e.g., [75]). An additional inclusion criterion for the patients is (f) the presence of a primary ICD-10 diagnosis of either schizophrenia, schizotypal, delusional, or another non-mood psychotic disorder (F20-F29) with clinically relevant delusional symptomatology. The inclusion criteria for the patients are relatively broad in scope in order to enhance generalizability of the results. Exclusion criteria for all participants are: (a) acute suicidality, (b) current neurological diseases, (c) current or past primary diagnosis of substance dependence, and (d) non-fluent German, and (e) if major technical issues occur (e.g., lost smartphones or smartphones not working reliably after start of ESM interval). Participants will also be excluded under this criterion if triggering problems with the ESM software should occur repeatedly, such as more than 67% of all prompts not being triggered and thus impairing participants technically from achieving a minimum compliance rate of 33% (which we do not expect, but hypothetically itemize). Additional exclusion criteria for the participants from the general population are: (f) current or past axis I mental and/or neurocognitive disorders, (g) current or past schizophrenia, schizotypal, and delusional disorders, and bipolar affective disorders in first-degree relatives (to exclude participants with a genetic psychosis risk).
Design
Procedure and materials.
Adult individuals without a diagnosis of schizophrenia will be recruited via public advertisements and will undergo a pre-screening telephone interview with one author in order to assess their eligibility for study participation. Patients considered eligible for the study will be approached by the same author and, if interested, invited for a laboratory assessment. During the laboratory assessment, both individuals without a diagnosis of schizophrenia and patients will be provided with detailed study information, informed consent procedures will be conducted, and first trait assessments will be undertaken (described below). These trait assessments will take place in order to rule out the presence of any current or lifetime schizophrenia or psychotic disorder and to capture potential delusion ideation in participants without a diagnosis of schizophrenia, to validate the life-time presence of a psychotic disorder and determine levels of psychopathology severity in patients, and to obtain measures of trait decision-making preferences and fluid intelligence (as a proxy for general intelligence) across all participants (the latter two not relevant for the study at hand). If considered fully eligible, participants will be invited for a second appointment within the next seven days where they will be trained to charge and use ESM devices that will then be handed out to participants, and a turnover appointment for the week after the last ESM prompt will be arranged.
Participants will be handed out smartphones in order to participate in the ESM sampling interval of three consecutive days. We will provide Android smartphones that have proved to be compatible with the chosen ESM web-based software movisensXS, Version 1.5 (movisens GmbH, Karlsruhe, Germany). All smartphones handed out to participants will be equipped with the ESM app movisensXS, which will automatically trigger prompts from sampling day 1 onwards until the end of sampling day 3. The devices will be pre-adapted by the research team instructor so that participants can only change minor study settings such as font size but will be unable to interfere with the study settings itself. If participants themselves own smartphones of the models specified above and wish to use these to participate in the ESM study, they will be supported in installing the movisensXS application on their personal devices.
Smartphone prompts will be triggered four times a day (10 a.m., 1 p.m., 5 p.m., 9 p.m.), resulting in a total of 12 prompts, and can be answered, postponed, or dismissed. Each assessment will take around 5–6 minutes. This sampling scheme is chosen to reduce and distribute study participation burden for schizophrenia inpatients who frequently experience severe illness-related difficulties in participating (e.g., concentration difficulties) and to enhance adherence to the ESM procedure. The time frames for prompts were chosen so as to ensure that the four prompts can be responded to during waking hours and that the four prompts appear at comparable intervals. The arrival of each prompt will be announced by a beep, and answering the prompt may be delayed by a maximum of 20 minutes only, as it has been suggested that the level of reliability of delayed reports diminishes fairly rapidly [76].
In order to enhance compliance, the instructor will train with each participant the completion of one representative ESM prompt and answer any related questions. Regarding remuneration procedure, all participants will be compensated proportionally to the amount of completed ESM prompts (2 Swiss Francs per prompt). To further increase compliance, the research team instructor will contact participants after the first day of ESM participation in person (patients), or else via the app-in-built chat, to inquire after potential problems or unclear points.
Baseline assessments.
Demographic assessments will comprise questions regarding age, gender, education level, and housing situation and will be designed by the research team. Patients with schizophrenia will also undergo a psychopathology rating (external assessment administered by a person from the research team).
In order to confirm the absence of any major Axis I DSM-IV mental disorder in non-clinical individuals, including a schizophrenia spectrum disorder, all individuals from the general population will undergo a structured clinical interview, the Mini International Neuropsychiatric Interview (MINI, [77]). Individuals who fulfill a prior and/or present diagnosis as assessed with the MINI are subsequently excluded from the study. For the group of individuals with a diagnosis of schizophrenia, the diagnosis given by the local university hospital ward will be confirmed with the sections for depression, mania and psychotic disorders of the MINI. All interviews will be conducted by trained members of the research team who regularly receive diagnostic supervision.
Delusion proneness in individuals without a diagnosis of schizophrenia will be assessed using the Peters Delusion Inventory (PDI; [78]) in its validated German translation (79). In 40 items, this inventory assesses present delusion-like beliefs of an everyday life nature instead of clinical symptoms. Each item (e.g., “Do you ever feel as if people are reading your mind?”) is answered in a dichotomous format (yes/no), and items answered with ‘yes’ yield 5-point Likert scale items regarding the levels of distress, preoccupation, and convictions linked to the belief (ranging from 1 = not at all distressing, hardly ever think about it, do not believe it’s true; to 5 = very much, think about it all the time, believe it’s absolutely true). This instrument has in its original version been used in various studies examining non-clinical populations (e.g., [80]) and demonstrates acceptable indices of reliability [78] and good validity [81]. The same holds true for the validated German translation [79]. Each participant will complete further assessments/testings that are, however, not of relevance for this study.
ESM assessments.
A detailed description of all ESM items used in this study is listed in S1 Table in S1 Appendix. Notably, while our hypotheses will be addressed inspecting predictors and outcomes always measured within the same prompt, the nature of our study is longitudinal since information processing modes are assessed with respect to an actual everyday-life decision of the participant “since the last prompt”and JTC bias is assessed “at the moment” (measures see below).
Intuitive and analytical processing (predictors) will be assessed on a state level with a short and state-adapted version of the REI-40 ([26], for the adapted version, see Supplementary material S1 Appendix) that was recently developed and validated with the aim of being used specifically in ESM designs. This state-adapted version will be used to approximate intuitive and analytical processing with regard to an actual everyday life decision of the respective participant. Like prominent self-report instruments in the field of dual process theories, we will target intuitive and analytical processing by focusing on self-reported decision-making as an observable proxy for Type 1 and Type 2 processes. Items were chosen based on differential factor loadings reported in REI-40 studies [26,82]; the full development strategy is reported elsewhere (see S1: [1]).
The JTC bias (outcome) will be assessed with an adapted version of the Fictitious Scenario Task designed by Lüdtke and colleagues [83]. This task is a JTC-adapted, close-to-everyday-life version of a measure originally created to assess the bias against disconfirmatory evidence. Moritz and colleagues [84] could show that this JTC-adapted version is as appropriate as classic JTC measures, such as the Beads Task to measure jumping to conclusions. At first glance, the scenarios in the Fictitious Scenario Task can be interpreted in different ways and are thus ambiguous (see S1 Fig in S1 Appendix for example item). Participants will be instructed to give a confidence rating on the validity of the four given explanations in the light of a first sentence (response options: 0%, 25%, 50%, 75%, or 100%). Subsequently, two further sentences will be revealed that may change the plausibility of presented explanations and for which new confidence ratings can be given (not of interest in this study). With each revealed sentence, participants can also make a final choice for any explanation given. As our primary JTC bias outcome, confidence ratings ≥ 75% after the first sentence are considered as hasty decision-making and indicative of the JTC bias. An alternative JTC bias outcome can be computed contingent on whether the primary outcome gives rise to a floor effect and does not display enough variability to set up statistical models or not: If not, a “final decision” statement made after sentence 1 (too early to make a valid judgment) is considered as hasty decision-making and indicative of the JTC bias present at the moment of the testing.
Paranoia, as a proxy for delusional symptoms in patients with schizophrenia, will be assessed with the 5-item brief state version of the Paranoia Checklist [85].
These ESM assessments as well as the assessment of state negative affect (see Table 1) will be completed by each participant at each prompt, while reactivity (see Table 1) will be assessed only at the last prompt of the day.
Analysis plan
Pre-processing.
Raw data from all participants will be obtained via the movisens webbased platform by one member of the research staff (SK) and will be cleaned and pre-processed before main analyses: First, raw laboratory assessment files and raw ESM data will be merged by a unique participant identifier. Second, all prompts that have not been filled out completely or have been dismissed will be treated as missing prompts. If unique ESM data points relevant to answering the main hypotheses are missing, the corresponding prompts will not be included in analyses relying on the respective data. If due to any (unexpected) technical distortions more prompts than planned, or theoretically impossible values (e.g., repeated prompts or inputs exceeding predefined input ranges), were stored, the data values will be checked and removed if necessary. Third, all individuals with a compliance rate for all fixed prompts below 33% will be excluded, as it will otherwise be difficult to maintain the assumption that data are missing completely at random [72].
Next, we will also check if main outcomes are normally distributed and, if this is not given, test if transformations can help to normalize data distributions. If transformation does not change the distribution statistically significantly as indicated by a Shapiro Wilkes test, we will keep the raw data for analyses. Lastly, we will also remove outliers from the corresponding variables, defined as values lower or higher than the mean ± 3 standard deviations; we will not remove outliers detected in major ESM variables, as removing outliers from these brief Likert scales might distort the data distribution substantially.
Due to the hierarchical structure of our data, where different observations (level 1) are nested within individuals (level 2), we will employ multilevel model analyses. As we aim to obtain regression estimates of level-1 associations that are unambiguously interpretable, we will person-mean center all time-varying predictors by subtracting the cluster’s (i.e., person’s) mean predictor value from the given predictor value. We thus follow the recommendations by [86] who have advised person-mean centering if a level-1 association is of substantial research interest, thereby obtaining a fairly accurate coefficient estimate unaffected by level-2 variable influences. As an example, a novel variable “intuitive processing centered” will be created with values that are centered around zero and that represent deviations in the predictor from an individual’s average predictor value: Positive values represent a more-than-usual intuitive processing for this individual, negative values represent a less-than-usual intuitive processing for this individual. For predictor and outcome variables, we will also compute person-mean variables for descriptive reporting.
ESM analyses.
As part of our preliminary analyses, we will compute ESM-related measures of compliance. To this end, we will derive each person’s mean number as well as corresponding standard deviations of completed prompts (individual compliance rates) and group-specific compliance rates. As done in other studies in the field of clinical psychology [67,69], participants will only be counted as “completers” and included in the study if they have completed at least 33% of all fixed prompts [72]. We will also test in an exploratory manner whether completers and non-completers differ significantly in demographic variables, psychopathological severity rating, state negative affect, or with regard to the ratio of individuals with and without a diagnosis of schizophrenia, using t-tests for independent samples or Mann-Whitney U tests (if data are not normally distributed), and chi square tests or Fisher’s exact tests (if any cell has < 5 observations). In order to assess to which degree reported information processing modes were “as usual”, we will also address to which degree the ESM procedure itself may have influenced reporting of processing modes by increasing self-awareness (denoting a reactivity phenomenon): We will tentatively examine the relevance of ESM reactivity for processing modes by regressing participants’ raw day-averaged information processing modes separately onto this day’s reactivity score (derived by averaging both reactivity item scores, see Supplementary material Table 1). All further descriptive and inference statistical tests, including primary analyses, will only be computed with completers.
All statistical pre-processing and analyses will be computed using R (Version 4.0.0 or later; [87] and will use a significance threshold of α = 5%.
Primary analyses.
As described above, our ESM data structure is of a hierarchical structure where it is assumed that observations within clusters (here: individuals) are not independent, thus requiring the use of multilevel models (MLM). Intra-class correlation (ICC) coefficients will be computed for major predictors and outcomes to obtain an estimate of between-person and within-person variances. We will use cumulative link mixed models [88], as our outcome variables are both assessed on an ordinal scale (JTC: three levels; self-reported paranoia: five levels). All MLMs will contain a random intercept and for each time-varying predictor an additional random slope if this improves model fit (as indicated by log-likelihood ratio statistics), thereby considering model complexity.
For H1, H2, and H3, we will refrain from including time-invariant covariates like age, sex, or antipsychotic medication dose as all these cannot have an influence on the estimated relationship between our time-varying predictor and outcome when the time-varying predictor is person-centered [89]. We will, however, include state negative affect as time-varying covariate in all models in order to control for potential confounding effects. For H4, which involves delusion proneness as time-invariant moderator, we will set up two different models, one with and one without antipsychotic medication dose as time-invariant covariate to control for. In the model with the covariate, we plan to enter the predictor antipsychotic medication dose as well as an interaction term of dose x delusion proneness (our original moderator). Again, the best fitting model for H4 will be determined based on log likelihood ratio test statistics. For each hypothesis, one final model will be used to derive relevant coefficients and other interpretable parameters.
For hypothesis H1, we will test whether prior intuitive processing (referring to an actual decision “since the last prompt”) predicts subsequent JTC bias (referring to a JTC bias assessed “at the moment of the prompt”). For hypothesis H2, we will test whether prior analytical processing (referring to an actual decision “since the last prompt”) predicts subsequent JTC bias (referring to a JTC bias assessed “at the moment of the prompt”). For hypothesis H3, we will test whether prior intuitive processing (referring to an actual decision “since the last prompt”) predicts subsequent paranoia (referring to state paranoia assessed “at the moment of the prompt”). For hypothesis H4, we will test whether prior intuitive processing and/or analytical processing (referring to an actual decision “since the last prompt”) predict subsequent JTC bias (assessed “at the moment of the prompt”) and whether these associations will vary with the level of trait delusion proneness (interaction between trait delusion proneness and intuitive processing and/or analytical processing). For each MLM, we will also run diagnostic checks to assess model assumptions, including the proportional odds assumption for ordinal outcomes. Full model specifications of all final models as well as assumptions on random effect variances will be provided in the supplemental material.
Contingent analyses.
Regarding Hypothesis 1–4, we plan to compute frequentist equivalent tests relying on two one-sided tests (TOST) if a predictor or interaction term in the null hypothesis significance tests does not turn statistically significant as theoretically reasoned. If we find that intuitive processing predicts subsequent paranoia as a proxy for delusional symptoms in patients with schizophrenia, we will also preliminarily investigate if intuitive processing predicts subsequent hallucinations.
Further post-hoc tests may become reasonable after main analyses, although not anticipated a priori; these post hoc analyses will then be transparently justified and reported in accordance with pre-registration recommendations [90].
Supporting information
S1 Appendix.
A. References for unpublished or submitted word. B. Figure S1: ESM Fictitious Scenario Task. C. Table S1. ESM items.
https://doi.org/10.1371/journal.pone.0261296.s001
(DOCX)
Acknowledgments
The authors thank Gregory Elbel, Elinor Sager, Lisa Rosenthaler, Jefimija Cubrilo, and Matthias Sivarajah for planned support in study initiation and data collection as well as Ursula Drew for proofreading.
References
- 1. Bronstein M V., Everaert J, Castro , Joormann J, Cannon TD. Pathways to paranoia: Analytic thinking and belief flexibility. Behav Res Ther. 2019; 113: 18–24. pmid:30580159
- 2. Evans JSBT, Stanovich KE. Dual-Process Theories of Higher Cognition. Perspect Psychol Sci. 2013; 8: 223–41. pmid:26172965
- 3. Kahneman D. A perspective on judgment and choice: Mapping bounded rationality. Am Psychol. 2003; 58: 697–720. pmid:14584987
- 4.
Kahneman D. Thinking, Fast and Slow. US, New York, NY.: Farrar, Straus and Giroux; 2011.
- 5. Stanovich KE, West RF. Individual differences in reasoning: Implications for the rationality debate? Behav Brain Sci. 2000; 23: 645–65. pmid:11301544
- 6. Epstein S. Integration of the cognitive and the psychodynamic unconscious. Am Psychol. 1994; 49: 709–24. pmid:8092614
- 7. Speechley WJ, Ngan ETC. Dual-stream modulation failure: A novel hypothesis for the formation and maintenance of delusions in schizophrenia. Med Hypotheses. 2008; 70: 1210–4. pmid:18328633
- 8. Heintzelman SJ, King LA. Meaning in life and intuition. J Pers Soc Psychol. 2016; 110: 477–92. pmid:26322984
- 9. Hicks JA, Cicero DC, Trent J, Burton CM, King LA. Positive Affect, Intuition, and Feelings of Meaning. J Pers Soc Psychol. 2010; 98: 967–79. pmid:20515252
- 10. Bolte A, Goschke T, Kuhl J. Emotion and Intuition. Psychol Sci. 2003; 14: 416–21. pmid:12930470
- 11. Zander-Schellenberg T, Remmers C, Zimmermann J, Thommen S, Lieb R. It was intuitive, and it felt good: a daily diary study on how people feel when making decisions. Cogn Emot. 2019; 33: 1505–13. pmid:30688133
- 12. Bowers KS, Regehr G, Balthazard C, Parker K. Intuition in the context of discovery. Cogn Psychol. 1990; 22: 72–110. https://doi.org/10.1016/0010-0285(90)90004-N.
- 13. Topolinski S, Strack F. The architecture of intuition: Fluency and affect determine intuitive judgments of semantic and visual coherence and judgments of grammaticality in artificial grammar learning. J Exp Psychol Gen. 2009; 138: 39–63. pmid:19203169
- 14. Zander T, Horr NK, Bolte A, Volz KG. Intuitive decision making as a gradual process: investigating semantic intuition-based and priming-based decisions with fMRI. Brain Behav. 2016; 6: n/a-n/a. pmid:27110441
- 15. Maglio SJ, Reich T. Feeling certain: Gut choice, the true self, and attitude certainty. Emotion. 2019; 19: 876–88. pmid:30198736
- 16. Parsons CE, Young KS, Stein A, Kringelbach ML. Intuitive parenting: understanding the neural mechanisms of parents’ adaptive responses to infants. Curr Opin Psychol. 2017; 15: 40–4. pmid:28813266
- 17. Ambady N. The Perils of Pondering: Intuition and Thin Slice Judgments. Psychol Inq. 2010; 21: 271–8. https://doi.org/10.1080/1047840X.2010.524882.
- 18. Bago B, De Neys W. The Smart System 1: evidence for the intuitive nature of correct responding on the bat-and-ball problem. Think Reason. 2019; 25: 257–99. https://doi.org/10.1080/13546783.2018.1507949.
- 19. Betsch T, Glöckner A. Intuition in Judgment and Decision Making: Extensive Thinking Without Effort. Psychol Inq. 2010; 21: 279–94. https://doi.org/10.1080/1047840X.2010.517737.
- 20. Gigerenzer G. Why Heuristics Work. Perspect Psychol Sci. 2008; 3: 20–9. pmid:26158666
- 21. Phillips WJ, Fletcher JM, Marks ADG, Hine DW. Thinking styles and decision making: A meta-analysis. Psychol Bull. 2016; 142: 260–90. pmid:26436538
- 22. Reinhard M-A. Need for Cognition and the process of lie detection. J Exp Soc Psychol. 2010; 46: 961–71. https://doi.org/10.1016/j.jesp.2010.06.002.
- 23. Wood NL, Highhouse S. Do self-reported decision styles relate with others’ impressions of decision quality? Pers Individ Dif. 2014; 70: 224–8. https://doi.org/10.1016/j.paid.2014.06.036.
- 24. Rogers P, Fisk JE, Lowrie E. Paranormal belief, thinking style preference and susceptibility to confirmatory conjunction errors. Conscious Cogn. 2018; 65: 182–96. pmid:30199770
- 25. Carnevale JJ, Inbar Y, Lerner JS. Individual differences in need for cognition and decision-making competence among leaders. Pers Individ Dif. 2011; 51: 274–8. https://doi.org/10.1016/j.paid.2010.07.002.
- 26. Pacini R, Epstein S. The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. J Pers Soc Psychol. 1999; 76: 972–87. pmid:10402681
- 27. Betsch C. Präferenz für Intuition und Deliberation (PID). Zeitschrift für Differ und Diagnostische Psychol. 2004; 25: 179–97. https://doi.org/10.1024/0170-1789.25.4.179.
- 28. Betsch C, Kunz JJ. Individual strategy preferences and decisional fit. J Behav Decis Mak. 2008; 21: 532–55. https://doi.org/10.1002/bdm.600.
- 29. Epstein S, Pacini R, Denes-Raj V, Heier H. Individual differences in intuitive–experiential and analytical–rational thinking styles. J Pers Soc Psychol. 1996; 71: 390–405. pmid:8765488
- 30. Sagiv L, Amit A, Ein-Gar D, Arieli S. Not All Great Minds Think Alike: Systematic and Intuitive Cognitive Styles. J Pers. 2014; 82: 402–17. pmid:24007535
- 31. Klein G. A naturalistic decision making perspective on studying intuitive decision making. J Appl Res Mem Cogn. 2015; 4: 164–8. https://doi.org/10.1016/j.jarmac.2015.07.001.
- 32. Ward T, Garety PA. Fast and slow thinking in distressing delusions: A review of the literature and implications for targeted therapy. Schizophr Res. 2019; 203: 80–7. pmid:28927863
- 33. Remmers C, Topolinski S, Buxton A, Dietrich DE, Michalak J. The beneficial and detrimental effects of major depression on intuitive decision-making. Cogn Emot. 2017; 31: 799–805. pmid:26963803
- 34. Moritz S, Pfuhl G, Lüdtke T, Menon M, Balzan RP, Andreou C. A two-stage cognitive theory of the positive symptoms of psychosis. Highlighting the role of lowered decision thresholds. J Behav Ther Exp Psychiatry. 2017; 56: 12–20. pmid:27501907
- 35. Andreou C, Schneider BC, Balzan R, Luedecke D, Roesch-Ely D, Moritz S. Neurocognitive deficits are relevant for the jumping-to-conclusions bias, but not for delusions: A longitudinal study. Schizophr Res Cogn. 2015; 2: 8–11. pmid:29379755
- 36. Moritz S, Woodward TS. Metacognitive training in schizophrenia: from basic research to knowledge translation and intervention. Curr Opin Psychiatry. 2007; 20: 619–25. pmid:17921766
- 37. Woodward TS, Munz M, LeClerc C, Lecomte T. Change in delusions is associated with change in “jumping to conclusions.” Psychiatry Res. 2009; 170: 124–7. pmid:19906443
- 38. Dudley R, Taylor P, Wickham S, Hutton P. Psychosis, delusions and the “Jumping to Conclusions” reasoning bias: A systematic review and meta-analysis. Schizophr Bull. 2016; 42: 652–65. pmid:26519952
- 39. McLean BF, Mattiske JK, Balzan RP. Association of the jumping to conclusions and evidence integration biases with delusions in psychosis: A detailed meta-analysis. Schizophr Bull. 2017; 43: 344–54. pmid:27169465
- 40. Ross RM, McKay R, Coltheart M, Langdon R. Jumping to conclusions about the beads task? A meta-analysis of delusional ideation and data-gathering. Schizophr Bull. 2015; 41: 1183–91. pmid:25616503
- 41. Van Dael F, Versmissen D, Janssen I, Myin-Germeys I, Van Os J, Krabbendam L. Data gathering: Biased in psychosis? Schizophr Bull. 2006; 32: 341–51. pmid:16254066
- 42. Broyd A, Balzan RP, Woodward TS, Allen P. Dopamine, cognitive biases and assessment of certainty: A neurocognitive model of delusions. Clin Psychol Rev. 2017; 54: 96–106. pmid:28448827
- 43. Garety PA, Freeman D. The past and future of delusions research: from the inexplicable to the treatable. Br J Psychiatry. 2013; 203: 327–33. pmid:24187067
- 44. Bronstein M V., Pennycook G, Joormann J, Corlett PR, Cannon TD. Dual-process theory, conflict processing, and delusional belief. Clin Psychol Rev. 2019; 72: 101748. pmid:31226640
- 45. Toplak ME, West RF, Stanovich KE. Assessing miserly information processing: An expansion of the Cognitive Reflection Test. Think Reason. 2014; 20: 147–68. https://doi.org/10.1080/13546783.2013.844729.
- 46. Freeman D, Garety PD, Bebbington PE, Smith B, Rollinson R, Fowler D, Kuipers E, Ray K, Dunn G. Psychological investigation of the structure of paranoia in a non-clinical population. Br J Psychiatry. 2005; 186: 427–35. pmid:15863749
- 47. van Os J, Hanssen M, Bijl RV, Ravelli A. Strauss (1969) revisited: a psychosis continuum in the general population?. Schizophr Res. 2000; 45: 11–20. pmid:10978868
- 48. Freeman D, Evans N, Lister R. Gut feelings, deliberative thought, and paranoid ideation: A study of experiential and rational reasoning. Psychiatry Res. 2012; 197: 119–22. pmid:22406393
- 49. Freeman D, Lister R, Evans N. The use of intuitive and analytic reasoning styles by patients with persecutory delusions. J Behav Ther Exp Psychiatry. 2014; 45: 454–8. pmid:25000504
- 50. Aarnio K, Lindeman M. Paranormal beliefs, education, and thinking styles. Pers Individ Dif. 2005; 39: 1227–36.
- 51. Genovese JEC. Paranormal beliefs, schizotypy, and thinking styles among teachers and future teachers. Pers Individ Dif. 2005; 39: 93–102.
- 52. Mækelæ MJ, Moritz S, Pfuhl G. Are Psychotic Experiences Related to Poorer Reflective Reasoning? Front Psychol. 2018; 9. pmid:29483886
- 53. Marks ADG, Hine DW, Blore RL, Phillips WJ. Assessing individual differences in adolescents’ preference for rational and experiential cognition. Pers Individ Dif. 2008; 44: 42–52.
- 54. Wolfradt U, Oubaid V, Straube ER, Bischoff N, Mischo J. Thinking styles, schizotypal traits and anomalous experiences. Pers Individ Dif. 1999; 27: 821–30.
- 55. Barron D, Furnham A, Weis L, Morgan KD, Towell T, Swami V. The relationship between schizotypal facets and conspiracist beliefs via cognitive processes. Psychiatry Res. 2018; 259: 15–20. pmid:29024855
- 56. Georgiou N, Delfabbro P, Balzan R. Conspiracy beliefs in the general population: The importance of psychopathology, cognitive style and educational attainment. Pers Individ Dif. 2019; 151: 109521. https://doi.org/10.1016/j.paid.2019.109521.
- 57. Bronstein M V., Pennycook G, Bear A, Rand DG, Cannon TD. Belief in Fake News is Associated with Delusionality, Dogmatism, Religious Fundamentalism, and Reduced Analytic Thinking. J Appl Res Mem Cogn. 2019; 8: 108–17. https://doi.org/10.1016/j.jarmac.2018.09.005.
- 58. Pennycook G, Cheyne G, Barr N, Koehler D, Fugelsang J. On the reception and detection of pseudo-profound bullshit. Judgm Decis Mak. 2015; 10: 549–63.
- 59. Shenhav A, Rand DG, Greene JD. Divine intuition: Cognitive style influences belief in God. J Exp Psychol Gen. 2012; 141: 423–8. pmid:21928924
- 60. King LA, Hicks JA. Positive affect, intuition and referential thinking. Pers Individ Dif. 2009; 46: 719–24. https://doi.org/10.1016/j.paid.2009.01.031.
- 61. King LA, Burton CM, Hicks JA, Drigotas SM. Ghosts, UFOs, and magic: Positive affect and the experiential system. J Pers Soc Psychol. 2007; 92: 905–19. pmid:17484612
- 62. Alós-Ferrer C, Hügelschäfer S. Faith in intuition and behavioral biases. J Econ Behav Organ. 2012; 84: 182–92. https://doi.org/10.1016/j.jebo.2012.08.004.
- 63. Ward T, Peters E, Jackson M, Day F, Garety PA. Data-Gathering, Belief Flexibility, and Reasoning Across the Psychosis Continuum. Schizophr Bull. 2018; 44: 126–36. pmid:28338872
- 64. Ross RM, Pennycook G, McKay R, Gervais WM, Langdon R, Coltheart M. Analytic cognitive style, not delusional ideation, predicts data gathering in a large beads task study. Cogn Neuropsychiatry. 2016; 21: 300–14. pmid:27341507
- 65. Lysaker PH, Keane JE, Culleton SP, Lundin NB. Schizophrenia, recovery and the self: An introduction to the special issue on metacognition. Schizophr Res Cogn. 2020; 19: 100167. pmid:31832347
- 66.
Csikszentmihalyi M, Larson R. Validity and reliability of the experience-sampling method. In: Flow and the foundations of positive psychology: Springer. Springer; 2014.
- 67. Granholm E, Holden JL, Mikhael T, Link PC, Swendsen J, Depp C, et al. What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia. Schizophr Bull. 2019; 1–10. pmid:30339224
- 68. Edwards CJ, Cella M, Tarrier N, Wykes T. The optimisation of experience sampling protocols in people with schizophrenia. Psychiatry Res. 2016; 244: 289–93. pmid:27512917
- 69. Hartley S, Varese F, Vasconcelos e Sa D, Udachina A, Barrowclough C, Bentall RP, et al. Compliance in experience sampling methodology: the role of demographic and clinical characteristics. Psychosis. 2014; 6: 70–3. https://doi.org/10.1080/17522439.2012.752520.
- 70. So SH wai Peters ER, Swendsen J Garety PA, Kapur S. Detecting improvements in acute psychotic symptoms using experience sampling methodology. Psychiatry Res. 2013; 210: 82–8. https://doi.org/10.1016/j.psychres.2013.05.010.
- 71. Rinner MTB, Meyer AH, Mikoteit T, Hoyer J, Imboden C, Hatzinger M, et al. General or specific? The memory–experience gap for individuals diagnosed with a major depressive disorder or a social phobia diagnosis, and individuals without such diagnoses. Memory. 2019; 27: 1194–203. pmid:31311430
- 72. Trull TJ, Ebner-Priemer UW. Ambulatory assessment in psychopathology research: A review of recommended reporting guidelines and current practices. J Abnorm Psychol. 2020; 129: 56–63. pmid:31868388
- 73. Speechley WJ, Whitman JC, Woodward TS. The contribution of hypersalience to the “jumping to conclusions” bias associated with delusions in schizophrenia. J Psychiatry Neurosci. 2010; 35: 7–17. pmid:20040242
- 74. Garety P, Ward T, Emsley R, Greenwood K, Freemaan D, Fowler D, Kuipers E, Bebbington P, Rus-Calafell M, McGourty A, Sacadura C, Collett N, James K, Hardy A. Effects of SlowMo, a Blended Digital Therapy Targeting Reasoning, on Paranoia Among People With Psychosis A Randomized Clinical Trial. JAMA Psychiatry. 2021; 78:714–25. pmid:33825827
- 75. Granholm E, Holden JL, Mikhael T, Link PC, Swendsen J, Depp C, et al. What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia. Schizophr Bull. 2019; https://doi.org/10.1093/schbul/sbz070.
- 76. Delespaul PAEG. Assessing schizophrenia in daily life: The experience sampling method. 1995.
- 77. Sheehan DV, Lecrubier Y, Sheehan H, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998; 59(suppl20):22–33. pmid:9881538
- 78. Peters E, Joseph S, Day S, Garety P. Measuring delusional ideation: The 21-item Peters et al. Delusions Inventory (PDI). Schizophr Bull. 2004; 30: 1005–22. pmid:15954204
- 79. Lincoln TM, Keller E, Rief W. Die Erfassung von Wahn und Halluzinationen in der Normalbevölkerung. Diagnostica. 2009; 55: 29–40. https://doi.org/10.1026/0012-1924.55.1.29.
- 80. Cafferkey K, Murphy J, Shevlin M. Jumping to conclusions: the association between delusional ideation and reasoning biases in a healthy student population. Psychosis. 2014; 6: 206–14. https://doi.org/10.1080/17522439.2013.850734.
- 81. Fonseca-Pedrero E, Paino M, Santarén-Rosell M, Lemos-Giráldez S, Muñiz J. Psychometric properties of the Peters et al Delusions Inventory 21 in college students. Compr Psychiatry. 2012; 53: 893–9. pmid:22440833
- 82. Björklund F, Bäckström M. Individual differences in processing styles: validity of the Rational-Experiential Inventory. Scand J Psychol. 2008; 49: 439–46. pmid:18452500
- 83. Lüdtke T, Kriston L, Schröder J, Lincoln TM, Moritz S. Negative affect and a fluctuating jumping to conclusions bias predict subsequent paranoia in daily life: An online experience sampling study. J Behav Ther Exp Psychiatry. 2017; 56: 106–12. pmid:27639287
- 84. Moritz S, Veckenstedt R, Hottenrott B, Woodward TS, Randjbar S, Lincoln TM. Different sides of the same coin? Intercorrelations of cognitive biases in schizophrenia. Cogn Neuropsychiatry. 2010; 15: 406–21. pmid:20146127
- 85. Schlier B, Moritz S, Lincoln TM. Measuring fluctuations in paranoia: Validity and psychometric properties of brief state versions of the Paranoia Checklist. Psychiatry Res. 2016; 241: 323–32. pmid:27227702
- 86. Enders CK, Tofighi D. Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychol Methods. 2007; 12: 121–38. pmid:17563168
- 87. R Core Team. R: A language and environment for statistical programming. 2013.
- 88. Christensen RHB. Cumulative Link Models for Ordinal Regression with the R Package ordinal. 2019; 1–40.
- 89.
Allison PD. Fixed effects regression methods for londitudinal data using SAS. SAS Press; 2005.
- 90. van ‘t Veer AE, Giner-Sorolla R. Pre-registration in social psychology—A discussion and suggested template. J Exp Soc Psychol. 2016; 67: 2–12. https://doi.org/10.1016/j.jesp.2016.03.004.