https://orcid.org/0000-0002-6993-2012
Figures
Abstract
Purpose
Visual complaints are a common consequence of acquired brain injury (ABI). Yet, they may be overlooked in clinical practice. The present study aims to develop a screening instrument for assessing visual complaints in individuals with ABI and evaluate its psychometrics properties in a community sample.
Methods
We constructed the Dutch Screening Visual Complaints questionnaire-acquired brain injury (SVCq-abi), a self-report 23-item questionnaire. A Dutch community sample of 1159 participants (Mean [SD] age, 60 [16] years) completed the SVCq-abi and other questionnaires on visual disturbances, executive functioning, mental health, and autistic traits. Confirmatory factor analyses were performed for 5 models (1-factor, 3-factor, 5-factor, second-order, and bifactor) on a random split of half of the sample, and cross-validated on the other half. In addition, we evaluated floor and ceiling effects, scale reliability, test-retest reliability, as well as convergent and divergent validity.
Results
A 5-factor structure of the SVCq-abi was adopted which showed an excellent model of fit, with all item loadings exceeding 0.5. The SVCq-abi subscales demonstrated various floor effects, acceptable scale reliability, moderate to good test-retest reliability (ICC = 0.65 to 0.73), along with sufficient convergent (r2 = 9% to 32%) and divergent validity (r2 = 2% to 13%).
Citation: Dol VL, Fuermaier ABM, Will EME, van Sorge AJ, Heutink J (2024) The Screening Visual Complaints questionnaire-acquired brain injury: Development and evaluation of psychometric properties in a community sample. PLoS ONE 19(12): e0314999. https://doi.org/10.1371/journal.pone.0314999
Editor: Yee Cheng Kueh, Universiti Sains Malaysia, MALAYSIA
Received: May 29, 2024; Accepted: November 19, 2024; Published: December 31, 2024
Copyright: © 2024 Dol et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data supporting the findings of this study are available via DataverseNL (https://doi.org/10.34894/58VDK7). Due to privacy concerns (i.e., the potential for re-identification of participants through the combination of variables), access to the data is restricted. Requests for re-use will be evaluated by the University of Groningen Digital Competence Centre, and approval from the Faculty of Behavioral and Social Sciences Ethics Board may be required (contact: dcc@rug.nl).
Funding: This work was supported by Visio Foundation, Amsterdam, The Netherlands (JH, EW, AvS, VD; https://visiofoundation.org/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Acquired brain injury (ABI) encompasses non-congenital brain injuries and can be the result of traumatic brain injury, stroke, tumor, infection and anoxia [1,2]. Problems with vision are a common consequence of ABI and have been documented in approximately 50 to 75% of individuals with ABI [3–9]. Often reported visual problems are reading disturbances, visual field defects, perceptual disorders, diplopia, depth impairment, and photophobia. These problems can significantly impact quality of life and independence, and may hamper rehabilitation processes [8,10–13]. Despite their impact, the visual consequences of ABI are often underestimated and overlooked in clinical practice [3].
A potential reason for this underestimation could be that many patients with ABI may not report visual problems spontaneously unless specifically questioned [3,14–16]. Moreover, screening for visual problems is not embedded in standard care in Dutch rehabilitation centers. Rehabilitation clinicians refer patients to an ophthalmologist or specialized visual rehabilitation center when they suspect visual dysfunctions are present, yet, visual problems may not always be visible to an observer [17]. These findings underline the need for vision screening to improve timely detection of visual problems in order to adapt further care accordingly, and refer to a specialist center when necessary.
While screening tools using tests of visual function and perception have been developed [18–20], these tools often require training and can take over 20 minutes to administer, making them less suitable for routine screening. Moreover, it is not clear if objective measures capture the full extent and range of vision-related problems individuals with ABI may experience [16,21,22]. Questionnaires, designed to capture the individual’s complaints and needs, may offer a practical alternative. However, current questionnaires may fall short in measuring some important visual domains (e.g., Cerebral Vision Screening questionnaire; CVSQ [23,24]) or focus on assessing the impact of visual impairments rather than assessing complaints at a functional level (e.g., Brain Injury associated Visual Impairment—Impact Questionnaire; BIVI-IQ-15 [25]).
Therefore, the aim of this study was to develop and validate a questionnaire to screen for visual complaints in individuals with ABI. For this purpose, we constructed the Screening Visual Complaints questionnaire-acquired brain injury (SVCq-abi), a self-report 23-item questionnaire. The SVCq-abi is an adaptation of the Screening Visual Complaints questionnaire (SVCq), a screening instrument that measures visual complaints in individuals with neurodegenerative diseases [26–28]. Psychometric properties were evaluated in a large Dutch community sample, including the factor structure, floor ceiling effects, scale reliability, test-retest reliability, as well as convergent and divergent validity.
Methods
Questionnaire development
Item bank generation.
We constructed an item bank with items describing complaints potentially relevant to individuals with ABI experiencing visual problems. We reviewed literature and relevant validated measures, including the SVCq [26–28], CVSQ [23,24], BIVI-IQ-15 [25], Vision Interview (VI; [3]), and the Brain Injury Vision Symptom questionnaires (BIVSS; [29]). We focused on items that measure complaints on a functional and activity level, ensuring accessibility for both inpatient and outpatient settings. In addition, a multidisciplinary focus group with professionals from visual rehabilitation and ophthalmology (neuropsychologist, occupational therapists, and ophthalmologist) formulated new items based on clinical experience. All items were categorized into constructs summarizing the focus of the items (e.g., both ‘SVCq item 8’ and ‘CVSQ item 8’, which assess color vision, were categorized under the same construct), resulting in a bank of 88 items categorized into 28 constructs. Each construct contained between 1 and 14 items.
Ranking exercise and version 1 development.
The focus group ranked all constructs from most important to least important for screening for visual problems in individuals with ABI. Subsequently, for each construct a key item was chosen. Given that the key items were predominantly derived from the SVCq, we opted for the SVCq as a base for the new instrument. Some SVCq items (n = 12) were modified to better fit the ABI population. Furthermore, based on the results of the ranking, some SVCq items were excluded (n = 2) and new items were added (n = 3). This process resulted in Version 1 of our questionnaire, comprising 21 items in total.
Version 1 pretest.
Three rehabilitation professionals reviewed Version 1 for clarity and completeness and provided feedback. In addition, a pretest of Version 1 was done to identify difficulties encountered when completing the questionnaires and to verify whether they interpreted the questions as they were conceptualized. Six participants, including 3 participants without ABI and 3 individuals with ABI that followed a rehabilitation trajectory at a visual rehabilitation center, completed Version 1 by means of a cognitive interview [30,31]. All participants were over 18 years of age and provided written informed consent. In sum, the cognitive interview implied that (step 1) participants were asked to think aloud while completing the questionnaire, their behavior and thoughts were observed (skipping questions, corrections of chosen response category, hesitation). Subsequently (step 2), participants were asked to provide background on behavior observed by the researcher, and asked for opinions and experiences with regard to the questionnaire. At this stage, participants were also specifically asked for their opinion on the differences between the original SVCq and the adapted version of the instrument for individuals with ABI. Finally, in case they felt certain complaints or aspects of visual problems were missing, the participants were given the opportunity to suggest new items.
Version 2 construction.
Based on the pretest and provided feedback, we made some modifications (including rephrasing questions, splitting a question into two, and changing the order of the questions) and determined Version 2 of the questionnaire, i.e. the SVCq-abi. The SVCq-abi includes 23 items, with 21 items identical or similar to those in the SVCq.
Study population
Participants comprised a convenience sample of the Dutch population, stratified by age and gender. We aimed to include a sample that roughly matched the age and gender distribution of an ABI population in rehabilitation settings, to enhance generalizability of results to the ABI population. All participants were older than 18 and had Dutch as their primary language.
Data collection
Participants were recruited between May 26, 2023 and June 29, 2023 via PanelInzicht. PanelInzicht is a Dutch online research panel focused on online quantitative data collection. We required a minimum sample size of 1,000 participants, as larger sample sizes enhance the accuracy of validity testing and facilitate the splitting of the sample for cross-validation of the factor models. Participants were invited to complete the SVCq-abi online, along with the CVSQ [23,24], Autism-spectrum Quotient-short (AQ-short) [32], the Depression Anxiety Stress scale-21 (DASS-21) [33], and the Behavioural Rating Inventory of Executive Function-Adult (BRIEF-A) [34]. Upon completion, participants received a small financial compensation. The estimated time for completing all questionnaires was around 15 minutes. Participant characteristics, including age, gender, education level, presence of severe ophthalmic conditions and presence of neurological conditions, were collected based on self-report. To evaluate test-retest reliability, a random subset of the sample was invited to complete the SVCq-abi and BRIEF-A again after a minimum of one week. Time to complete both questionnaires was estimated around 5 minutes.
The Ethical Committee Psychology of the University of Groningen (UG) approved the study protocol on June 22, 2022 (#PSY-2122-S-0162). All participants provided written informed consent.
Materials
SVCq-abi.
The SVCq-abi is a 23-item self-report questionnaire that aims to measure visual complaints in individuals with ABI (see S1 Appendix for the original Dutch questionnaire that was used in the present study; see S2 Appendix for an English translation). The SVCq-abi is an adaptation of the SVCq; a measure originally developed to measure visual complaints in individuals with neurodegenerative diseases [26–28]. A 3-factor structure was evidenced for the SVCq in a community sample, while further validation demonstrated a better fit of a 5-factor structure in individuals with multiple sclerosis and Parkinson’s disease. The 5-factor structure comprised the following subscales: function related, luminance related, task related, altered visual perception, and ocular discomfort.
The SVCq-abi starts with a semi-structured inventory question asking if visual complaints are present (item 1), employing a 3-point Likert scale (‘No/rarely (0)’, ‘sometimes (1)’, ‘often/always (2)’). If complaints are present (‘sometimes’ or ‘often/always’), respondents are asked to specify these complaints. The main body of the SVCq consists of 21 structured items (item 2 to item 22), each describing a visual complaint, and utilizes the same 3-point Likert scale. The final question, item 23, gauges the degree of limitations experienced in daily life resulting from the reported visual complaints, with response options ranging from 0 ‘no limitations’ to 10 ‘very severe limitations’. Scores for the main body of the SVCq-abi can be calculated by summing up the 21 items, with higher scores indicating higher frequency or severity of complaints.
CVSQ.
We used the translated Cerebral Vision Screening questionnaire, which was later modified by Dittrich [23,24,35]. Although the validity of the version of Dittrich is not known yet, the original CVSQ by Kerkhoff et al. showed good psychometric qualities. The CVSQ assesses visual complaints in individuals following stroke. The questionnaire comprises eight questions related to the presence of visual disabilities, with scoring ‘yes (1)’/‘no (0)’ / ’not applicable (NA)’ (Kerkhoff et al.-part), and 12 activity-related questions on a 5-point Likert scale with answer options ‘no problem (0)’ to ‘mostly a problem (4)’ (Dittrich-part). For each part, a score was calculated by summing up the items, with higher scores indicating higher visual disability. A composite score was formed by summing up all items.
AQ-short.
The AQ-short is an abridged version of the 50-item Autism spectrum Quotient, a self-report questionnaire that assesses autistic trains in individuals with normal intelligence [32]. The AQ-short consists of 28 items comprising two higher order factors assessing social behavioral difficulties (social behavior) and fascination for numbers and patterns (numbers/patterns). Participants are asked to respond to statements on a 4-point Likert scale, with response categories ‘definitely agree (1)’, ‘slightly agree (2)’, ‘slightly disagree (3)’ and ‘definitely disagree (4)’. Items are summed up to calculate subscale scores and a total composite score, with higher scores indicating more endorsement of autistic traits.
BRIEF-A.
The BRIEF-A measures respondents’ own perception of executive functioning in their daily life [34], measuring two domains: metacognition and behavioral regulation. The questionnaire is composed of 75 items on which respondents indicate how often a described problem has occurred over the past month, with response options ‘never (1)’, ‘sometimes (2)’, or ‘often (3)’. Subscale scores and a total composite score are calculated by summing the scores of the relevant items. A higher score represents poorer executive functions in everyday life. In addition, the BRIEF-A contains three scales to check the validity of answers: inconsistency, negativity, and infrequency. Respondents who scored above the cut-offs defined in the manual on any of the scales were excluded from the sample (negativity >3, inconsistency >7, infrequency >2).
DASS-21.
The DASS-21 is a questionnaire that measures emotional distress in the domains of depression, anxiety and stress [33]. Each of the three scales contains 7 items, scored on a 4-point Likert scale with response options ‘never (1)’, ‘sometimes (2)’, ‘often (3)’, or ‘very often (4)’. Subscale scores were formed by summing the scores of the relevant items. Higher scores indicate more severe symptoms of emotional distress.
Data analysis
All analyses were performed using R (version 3.0.2) [36]. Data was checked for normality and linearity; if the assumptions were violated, non-parametric tests were performed. A p-value of 0.05 or less was considered statistically significant.
Participant descriptives.
The participant characteristics were described with mean and standard deviation (SD). Subsamples were compared using a student’s t-test; proportions were compared using a chi-square test.
Confirmatory factor analysis.
The items of the SVCq-abi were categorized in alignment with the 5-factor structure of the original SVCq. This created the following structure of the SVCq-abi: function related (item 2, 3, 5, 10 and 20), luminance related (item 6, 11, 13 and 14), task related (item 8, 9, 15, 16 and 22), altered visual perception (item 4, 7, 12, 19, 21), and ocular discomfort (item 17 and 18).
For factor analysis, the sample was randomly split in two equally sized subsamples. A confirmatory factor analysis (CFA) was performed on subsample 1 to evaluate the fit of the proposed correlated 5-factor structure of the SVCq-abi (see Fig 1C for an illustration). A robust diagonally weighted least squares estimation with mean and variance adjusted test statistic and standard errors was employed (i.e. WLSMV) [37], as implemented in the R package lavaan [38]. WLSMV is the suggested estimation method for ordinal non-normal data [39–42]. To optimize model identification variances of the factors were fixed to 1. Additionally, the fit of competing factor models were explored, including a correlated 3-factor model (as proposed by Huizinga et al. [26]; see Fig 1B). Furthermore, we examined three models to test the possibility of a general ‘visual complaints’ factor, including a 1-factor model (see Fig 1A), a second-order model (5 first-order factors, and 1 general second-order factor; see Fig 1D), and a bifactor model (5 specific factors and 1 general factor; see Fig 1E).
Models as follows: (A) 1-factor model; (B) correlated 3-factor model; (C) correlated 5-factor model; (D) second-order model; (E) bifactor model; (F) bifactor model modified. SVCq-abi = Screening Visual Complaints questionnaire-acquired brain injury.
Goodness of fit indices were used to ascertain model fit: normed chi-square index (χ2 / df), Root-Mean-Square Error of Approximation (RMSEA), Standardized Root Mean-Square Residual (SRMR) and Comparative Fit Index (CFI). We used the normed χ2 as this parameter takes sample size into account [43]. A good model fit is indicated when normed χ2 values are ≤ 3.0, RMSEA ≤ 0.07 (with the upper limit of the confidence interval ≤ 0.08), SRMR ≤ .08, and CFI ≥ .95 (acceptable if CFI ≥ 0.90) [43–45]. Additionally, item loadings were evaluated, with values above 0.50 deemed adequate for capturing their respective constructs. For nested models, scaled chi-squared difference tests were performed to test for significant differences between the models (significant Δχ2) [46]. Non-nested models were compared by evaluating fit statistics. To ensure robustness of the results, CFA of the respective models were repeated in subsample 2 for cross-validation.
Item and scale evaluation.
We investigated the presence of floor and ceiling effects (< 70% of the respondents endorsing the lowest or highest response category was considered acceptable). Scale reliability was assessed by item-rest correlations (acceptable if > 0.30), inter-item correlations to identify possible item redundancy (acceptable if < 0.80), and internal consistency. Moreover, Average Variance Extracted (AVE) was calculated to measure the proportion of variance explained by each factor, with values above 0.50 indicating good convergence. To evaluate internal consistency, categorical omega (⍵) was calculated in case a first-order structure was selected, defining the amount of variance accounted for by the items that underlie the factor [47–50]. When a second-order or bifactor structure was established, omega hierarchical (⍵h) was used [51], which gauges the reliability of the general factor defined in the model adjusting for the specific factors and vice versa. Spearman-Brown’s coefficient was calculated when the factor comprised a 2-item scale [52]. Values of .70 and higher were considered as good internal consistency [53]. Test-retest reliability was evaluated by calculating intraclass correlation coefficients (ICC) between the two measurements, using a two-way mixed-effects model with absolute agreement. ICC values between 0.5 and 0.75 indicate moderate reliability, values higher than 0.75 indicate good reliability [54].
Convergent and divergent validity.
To establish convergent validity, the correlation between the SVCq-abi and the CVSQ was determined. As both questionnaires measure a similar construct, we expected moderate to large correlations. To evaluate divergent validity, we correlated SVCq-abi scores with the AQ-short, DASS-21 and BRIEF-A. Divergent validity was assessed under the assumption that these questionnaires measure a different construct than the SVCq-abi. We expected small correlations with the DASS-21 (visual complaints vs mental health) and the BRIEF-A (visual complaints vs executive dysfunction). For the AQ-short, which was assumed to measure an even more distinct construct (visual complaints vs autistic traits), we expected negligible correlations. All associations were established by Spearman’s correlation coefficients, and evaluated following Cohen’s criteria (r < 0.1, negligible; 0.1 ≤ r < 0.3, small; 0.3 ≤ r < 0.5, moderate; r ≥ 0.5, large; [55]). To adjust for multiple hypotheses testing, we used false discovery rate (FDR) adjusted p-values [56].
Results
Participant descriptives
Table 1 shows the characteristics of the study population. In total, 1159 participants were considered for inclusion. The two measurements of one participant were excluded due to double responses (n = 2). In addition, 142 participants were excluded based on the validity scales of the BRIEF-A, due to negative tendency (n = 30), inconsistency (n = 11), and infrequency (n = 100) of responses. After exclusion, 1021 were left for data analysis. For factor analyses, no significant differences were found between subsample 1 and subsample 2 with regard to age, gender, education level, presence of ophthalmic condition, and presence of neurological condition (Table 1).
Self-reported visual complaints
On the first item of SVCq-abi, 587 (57%) of the 1021 participants reported at least one visual complaint. In total, 644 visual complaints were reported, of which 522 (81%) were similar to the 21 complaints stated in the SVCq-abi. The most frequently reported visual complaints not covered by the SVCq-abi included: spots or floaters in the eye (n = 36), tearing of the eyes (n = 27), and difficulty watching a digital screen (n = 15).
Factor analyses
Table 2 presents the goodness of fit statistics of the CFA models of both subsamples. In subsample 1, the 1-factor, correlated 3-factor, correlated 5-factor and second-order model all demonstrated good fit statistics. The initial bifactor model did not produce an admissible solution and showed problematic negative standardized item loadings. Notably, item 11 showed an excessively high loading (λ = 228.4) in factor 2 and negative residual variance (σ2 = -52169.4). Fit statistics of the initial model are therefore not shown in Table 2. Modifications were made by constraining all item loadings above zero and excluding item 11 from factor 2 (see Fig 1F for an illustration), resulting in an admissible bifactor model with good model fit (Table 2). While some item loadings in the modified bifactor model did not meet significance level, this was likely due to underpowered analyses, as this issue was resolved when repeating CFA of the modified bifactor model in the complete sample. Cross-validation in subsample 2 confirmed good fit for all proposed models, including the modified bifactor model (Table 2).
Model comparison.
In subsample 1, the 5-factor model showed a good fit, and outperformed the 3-factor model (Δχ2 (7) = 67.5, p < .001). Covariances above 0.7 were found amongst the five factors, indicating potential evidence for a general (higher-order) factor that describes the correlations. Although the second-order model showed a good fit, a significant decrease in fit was found compared to the 5-factor structure (Δχ2 (5) = 19.8, p = 0.001). Moreover, fit of the 1-factor model was significantly worse than the 5-factor model (Δχ2 (10) = 111.1, p < .001). Similar results were found in subsample 2, with the correlated 5-factor model outperforming the 1-factor model (Δχ2 (10) = 151.4, p < .001), 3-factor model (Δχ2 (7) = 98.6, p < .001), and second-order model (Δχ2 (5) = 34.3, p < 0.001).
The 5-factor model was not nested within the modified bifactor model, which prohibited χ2 comparisons. The modified bifactor model exhibited superior fit across all indices in both subsamples.
Model selection.
Although the bifactor model showed superior fit, the fit indices of the correlated 5-factor model were comparable with only approximately 1% difference in CFI. This marginal difference questioned if the improved model fit outweighs the drawbacks associated with the bifactor model. Most importantly, to attain an admissible solution of the bifactor model we had to constrain the item loadings and remove item 11 from factor 2. However, given that item 11 theoretically aligns with factor 2, and showed no issues in other CFA models, this was not deemed preferable. Additionally, we favored a lenient item deletion stance since our sample represents a community sample instead of the target population. Lastly, a superior fit of the bifactor model could be expected considering it estimates more parameters [58,59]. Consequently, we favored a 5-factor structure over a bifactor structure for the SVCq-abi. Item loadings of the final 5-factor structure for the total sample are displayed in Table 3. Subscale scores of the 5 factors were calculated by summing up the relevant items.
Item and scale evaluation
Employing the 5-factor model, floor effects were found for item 20, item 14, all items in the task related subscale (item 6, 11, 13 and 14), the altered visual perception subscale (item 4, 7, 12, 19, and 21), and for item 17, as shown in Table 3. No ceiling effects were found. All item-rest correlations exceeded 0.3 and inter-item correlations did not surpass 0.80 (range 0.20–0.51) within the subscales. AVE values were above 0.5 for all subscales. Furthermore, all subscales showed good internal consistency, except for ocular discomfort, which showed insufficient internal consistency (Table 3).
With regard to test-retest reliability, of the 157 participants that completed the reassessment of the SVCq-abi, 14 were excluded based on BRIEF-A validity scales, due to a negativity tendency (n = 1), inconsistency (n = 1), or infrequency (n = 12) of responses. The retest was completed after 19 ± 11 days (range 12–28, median 20 days). As illustrated in Table 3, good test-retest reliability was found for the function related, task related and ocular discomfort subscales. Luminance related and altered visual perception showed moderate test-retest reliability.
Convergent and divergent validity
Table 4 presents the correlation coefficients between the SVCq-abi and the convergent and divergent measures. Regarding convergent validity, all correlations with the CVSQ were significant after FDR multiple hypothesis testing adjustment (padj < 0.001) and ranged from moderate to strong. For divergent validity, the SVCq-abi showed significant correlations with the BRIEF-A and DASS-21, with effect sizes ranging from weak to moderate. Between the SVCq-abi and the AQ-short roughly half of the correlation were insignificant and all effect sizes were considered negligible.
Discussion
The present study developed and evaluated the psychometric properties of the SVCq-abi, a screening tool for assessing visual complaints in individuals with ABI. The SVCq-abi is an adaptation of the SVCq; a questionnaire initially designed for individuals with neurodegenerative diseases. Following CFA, a correlated 5-factor model was favored over a 1-factor, correlated 3-factor, second-order, and bifactor model. Floor effects were found in various items of the SVCq-abi. Furthermore, through the evaluation of item-rest correlations, inter-item correlations, AVE and internal consistency, the subscales showed moderate to good scale reliability. With the exception of ocular discomfort, which showed insufficient internal consistency. We found moderate to good test-retest reliability of the subscales. Furthermore, moderate to large correlations were established with the convergent measures, and negligible to moderate correlations with the divergent measures.
We confirmed a 5-factor structure of the SVCq-abi in our community sample, aligning with previous findings on the structure of the SVCq [27,28]. In contrast, we employed WLSMV instead of DWLS as an estimation method, as WLSMV has been advised over DWLS when 3 response categories are used [39]. Furthermore, in our study, we evaluated competing models, including the bifactor model recommended by Chen and Zhang [60]. Although the bifactor model showed superior fit, the difference in fit indices with the 5-factor model was marginal, with approximately 1% difference in CFI. Therefore, we favored a 5-factor structure over a bifactor structure for the SVCq-abi due to several practical concerns associated with the bifactor model. Of particular significance was the problematic removal of item 11 from factor 2. This was not deemed preferable due to its theoretical alignment, and the fact it showed no issues in other CFA models. In addition, we took a cautious approach to item deletion considering it is a community sample.
No ceiling effects were found. However, approximately half of the items exhibited floor effects, indicating reduced sensitivity of these items in capturing variability among individuals with fewer visual complaints. Given our aim is to screen for individuals who experience more visual complaints, these floor effects could be less concerning in a community sample. Overall, the SVCq-abi demonstrated moderate to good scale reliability and test-retest reliability, similar to the original SVCq [26]. Small differences between the studies could be attributed to item adaptations and population variation. Notably, the ocular discomfort subscale of the SVCq-abi demonstrated insufficient internal consistency, potentially due to its two-item nature which may pose a threat for scale reliability [52,61]. Furthermore, test-retest reliability was moderate for luminance related and altered visual perception. Moderate reliability might pose issues for use in clinical practice, where repeatability of scores is essential.
In support of convergent validity, a higher level of visual complaints reported on the SVCq-abi corresponded with a higher level of visual complaints reported on the CVSQ, demonstrated by moderate to strong correlations. Notably, the CVSQ measures responses on a binary scale, which may influence the correlation strength with the SVCq-abi comprising 3 response options. Lowest correlations were observed between our questionnaire and the Dittrich-part of the CVSQ. This is potentially due to the fact that the Dittrich-part measures activity-related problems without specifically relating it to vision. Furthermore, the SVCq-abi subscale ocular discomfort showed relatively low correlations with the CVSQ. Such results were foreseeable as these items focus on eye comfort and health, and do not capture visual complaints on a functional level like the CVSQ. Huizinga et al. [26] evaluated convergent validity of ocular discomfort by its correlation with the ocular pain subscale of National Eye Institute Visual Functioning Questionnaire-25 [62] and found a strong association.
Divergent validity was assessed on the assumption that the SVCq-abi and the divergent questionnaires measure different psychological constructs. We observed negligible to moderate associations with the divergent measures. As hypothesized, particularly low associations were found with autistic traits reported on the AQ-short. Roughly half of the correlations of the SVCq-abi with the BRIEF-A and the DASS-21 were of moderate strength. In particular, the task related subscale showed mostly moderate correlations. The known association between executive dysfunction and emotional distress with visual functioning and performing tasks may have contributed to the strength of these correlations [63–70]. Additionally, an explanation may be the phenomenon of a general pathology factor that explains the endorsement of complaints across all measured domains [71]. The original SVCq showed similar correlations with the BRIEF-A and the DASS-21 [26]. Yet, associations in our sample appeared slightly higher, possibly due to the older age of our participants [72–75]. Nonetheless, we expected the association with the convergent measures to be higher than the divergent measures. In total, the SVCq-abi shared 9% to 32% of variance with the convergent measures, and 2% to 13% with the divergent measures. Taken together, evidence for convergent and divergent validity was considered minimal, but sufficient.
This study has some strengths and limitations. A strength is the exploration of competing factor models, adding to the robustness of the initial 5-factor structure. However, the factor structure, as well as the psychometric properties, were established in a community sample. It is unknown to which extent these properties can be generalized to individuals with ABI. Notably, the prevalent floor effects in half of the items may have distorted correlational structures, which potentially limits the applicability of the findings to a clinical sample. Therefore, further exploration of the psychometric properties in a sample with ABI is an important issue for future studies. Another limitation is that we employed a Dutch adapted version of the CVSQ as a convergent measure. Although the original CVSQ has been extensively validated, the Dittrich-part and the Dutch translation has not been validated yet, which may have influenced convergent validity results. Lastly, we used the scale-shifted fit indices of the WLSMV estimation method for CFA to enhance comparability with prior research, given it is the default method in most factor analysis software. However, although the RMSEA and CFI indices are widely applied with WLSMV, recent developments have raised concerns that the model fit is overestimated with these indices [76,77]. These findings raise questions about the robustness of our CFA results and it is essential to acknowledge that the field is evolving.
The SVCq-abi may be of benefit to individuals with ABI, and professionals within rehabilitative practice. It could offer a valuable and pragmatic method to screen for visual complaints, providing insights into various aspects of visual impairment. Unlike objective vision measurements and screening tests, the SVCq-abi requires minimal time and is easy to administer. Based on the current findings, we suggest making use of the 5-factor model for the SVCq-abi. In addition to the evaluation of the subscale scores, an evaluation of the individual items is highly recommended for additional insights into specific complaints and needs. Additionally, item 23 may offer insight into the extent of limitations experienced, and help guide further care. It is important to note that some validity and reliability issues of the SVCq-abi remain. These concerns require further exploration in an ABI sample to ensure the robustness and utility of the SVCq-abi in a clinical setting. In conclusion, the SVCq-abi shows fundamental psychometric properties and a robust 5-factor structure. It appears a valid tool to measure visual complaints in a community sample, which holds promise for use in individuals with ABI. Some validity and reliability issues emerged which could pose issues for use in clinical practice. Hence, further validation in individuals with ABI is essential.
Supporting information
S1 Appendix. The Screening Visual Complaints questionnaire-acquired brain injury [original Dutch questionnaire].
https://doi.org/10.1371/journal.pone.0314999.s001
(PDF)
S2 Appendix. The Screening Visual Complaints questionnaire-acquired brain injury [English translation].
https://doi.org/10.1371/journal.pone.0314999.s002
(PDF)
Acknowledgments
We thank the focus group and rehabilitation professionals for their contribution to the development of this instrument, including Maya Vriend and Femke van der Veer from Royal Dutch Visio, Huizen, and Wietske Rienstra, Tanja de Vries, and Rinze Benedictus from Revalidatie Friesland, Beetsterzwaag.
References
- 1. Bruns J Jr, Hauser WA. The epidemiology of traumatic brain injury: a review. Epilepsia. 2003;44(s10):2–10. pmid:14511388
- 2. Goldman L, Siddiqui EM, Khan A, Jahan S, Rehman MU, Mehan S, et al. Understanding Acquired Brain Injury: A Review. Biomedicines. 2022 Sep 2;10(9):2167. pmid:36140268
- 3. Berthold-Lindstedt M, Ygge J, Borg K. Visual dysfunction is underestimated in patients with acquired brain injury. J Rehabil Med. 2017 Apr 6;49(4):327–332. pmid:28350414
- 4. Hepworth LR, Rowe FJ, Walker MF, Rockliffe J, Noonan C, Howard C, et al. Post-stroke Visual Impairment: A Systematic Literature Review of Types and Recovery of Visual Conditions. Ophthalmol Res Int J. 2016 Jan 10;5(1):1–43.
- 5. Rowe FJ, Hepworth LR, Howard C, Hanna KL, Cheyne CP, Currie J. High incidence and prevalence of visual problems after acute stroke: An epidemiology study with implications for service delivery. PLoS One. 2019 Mar 6;14(3): e0213035. pmid:30840662
- 6. Brahm KD, Wilgenburg HM, Kirby J, Ingalla S, Chang CY, Goodrich GL. Visual impairment and dysfunction in combat-injured servicemembers with traumatic brain injury. Optom Vis Sci. 2009 Jul;86(7):817–25. pmid:19521270
- 7. Goodrich GL, Kirby J, Cockerham G, Ingalla SP, Lew HL. Visual function in patients of a polytrauma rehabilitation center: A descriptive study. J Rehabil Res Dev. 2007;44(7):929–36. pmid:18075950
- 8. Greenwald BD, Kapoor N, Singh AD. Visual impairments in the first year after traumatic brain injury. Brain Inj. 2012;26(11):1338–59. pmid:22897509
- 9. Merezhinskaya N, Mallia RK, Park D, Bryden DW, Mathur K, Barker FM 2nd. Visual Deficits and Dysfunctions Associated with Traumatic Brain Injury: A Systematic Review and Meta-analysis. Optom Vis Sci. 2019 Aug;96(8):542–555. pmid:31343512
- 10. Johnson R. Disorders of higher visual processing in patients with acquired brain injury. NeuroRehabilitation. 2022;50(3):331–341. pmid:35342058
- 11. de Haan GA, Heutink J, Melis-Dankers BJ, Brouwer WH, Tucha O. Difficulties in Daily Life Reported by Patients with Homonymous Visual Field Defects. J Neuroophthalmol. 2015 Sep;35(3):259–64. pmid:25815856
- 12. Han L, Law-Gibson D, Reding M. Key neurological impairments influence function-related group outcomes after stroke. Stroke. 2002 Jul;33(7):1920–4. pmid:12105376
- 13. Hepworth LR, Rowe FJ. Visual Impairment Following Stroke—The Impact on Quality of Life: A Systematic Review. Ophthalmol Res Int J. 2016 Jan 1;5(2):1–15.
- 14. Hepworth LR, Howard C, Hanna KL, Currie J, Rowe FJ. "Eye" Don’t See: An Analysis of Visual Symptom Reporting by Stroke Survivors from a Large Epidemiology Study. J Stroke Cerebrovasc Dis. 2021 Jun;30(6):105759. pmid:33813079
- 15. Rowe FJ, Wright D, Brand D, Jackson C, Harrison S, Maan T, et al. A prospective profile of visual field loss following stroke: prevalence, type, rehabilitation, and outcome. Biomed Res Int. 2013; 2013:719096. pmid:24089687
- 16. Berthold-Lindstedt M, Johansson J, Ygge J, Borg K. How to assess visual function in acquired brain injury-Asking is not enough. Brain Behav. 2021 Feb;11(2): e01958. pmid:33230981
- 17. Rowe FJ; VIS Group. Accuracy of referrals for visual assessment in a stroke population. Eye (Lond). 2011 Feb;25(2):161–7. pmid:21127506
- 18. Rowe FJ, Hepworth LR, Hanna KL, Howard C. Visual Impairment Screening Assessment (VISA) tool: pilot validation. BMJ Open. 2018 Mar 6;8(3): e020562. pmid:29511021
- 19. Quinn TJ, Livingstone I, Weir A, Shaw R, Breckenridge A, McAlpine C, et al. Accuracy and Feasibility of an Android-Based Digital Assessment Tool for Post Stroke Visual Disorders-The StrokeVision App. Front Neurol. 2018 Mar 28;9:146. pmid:29643830
- 20. Tarbert CM, Livingstone IA, Weir AJ. Assessment of visual impairment in stroke survivors. Annu Int Conf IEEE Eng Med Biol Soc. 2014; 2014:2185–8. pmid:25570419
- 21. French LM, Lange RT, Brickell T. Subjective cognitive complaints and neuropsychological test performance following military-related traumatic brain injury. J Rehabil Res Dev. 2014;51(6):933–50. pmid:25479042
- 22. Lannoo E, Colardyn F, Vandekerckhove T, De Deyne C, De Soete G, Jannes C. Subjective complaints versus neuropsychological test performance after moderate to severe head injury. Acta Neurochir (Wien). 1998;140(3):245–53. pmid:9638261
- 23. Kerkhoff G, Schaub J, Zihl J. Die Anamnese zerebral bedingter Sehstörungen [Anamnesis of brain-originated vision disorders]. Nervenarzt. 1990 Dec;61(12):711–8. German.
- 24. Neumann G, Schaadt AK, Reinhart S, Kerkhoff G. Clinical and Psychometric Evaluations of the Cerebral Vision Screening Questionnaire in 461 Nonaphasic Individuals Poststroke. Neurorehabil Neural Repair. 2016 Mar;30(3):187–98. pmid:25967758
- 25. Hepworth LR, Rowe FJ, Burnside G. Development of a patient reported outcome measures for measuring the impact of visual impairment following stroke. BMC Health Serv Res. 2019 May 31;19(1):348. pmid:31151449
- 26. Huizinga F, Heutink J, de Haan GA, van der Lijn I, van der Feen FE, Vrijling ACL, et al. The development of the Screening of Visual Complaints questionnaire for patients with neurodegenerative disorders: Evaluation of psychometric features in a community sample. PLoS One. 2020 Apr 29;15(4):e0232232. pmid:32348342
- 27. van der Lijn I, de Haan GA, van der Feen FE, Huizinga F, Fuermaier ABM, van Laar T, et al. The Screening Visual Complaints questionnaire (SVCq) in people with Parkinson’s disease-Confirmatory factor analysis and advice for its use in clinical practice. PLoS One. 2022 Sep 14;17(9):e0272559. pmid:36103511
- 28. van der Feen FE, de Haan GA, van der Lijn I, Fuermaier ABM, Heersema TJ, Meilof JF, et al. Confirmatory factor analysis of the Dutch Screening Visual Complaints questionnaire in people with multiple sclerosis. J Patient Rep Outcomes. 2022 Apr 9;6(1):36. pmid:35397039
- 29. Laukkanen H, Scheiman M, Hayes JR. Brain Injury Vision Symptom Survey (BIVSS) Questionnaire. Optom Vis Sci. 2017 Jan;94(1):43–50. pmid:28027193
- 30. Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs. 2003 Apr;42(1):57–63. pmid:12641812
- 31. Beatty PC, Willis GB. Research synthesis: the practice of cognitive interviewing. Public Opin Q. 2007;71(2):287–311.
- 32. Hoekstra RA, Vinkhuyzen AA, Wheelwright S, Bartels M, Boomsma DI, Baron-Cohen S, et al. The construction and validation of an abridged version of the autism-spectrum quotient (AQ-Short). J Autism Dev Disord. 2011 May;41(5):589–96. pmid:20697795
- 33. Lovibond SH, Lovibond PF. Manual for the depression anxiety stress scales. Psychology Foundation of Australia; 1996.
- 34. Roth RM, Gioia GA, Isquith PK. BRIEF-A: Behavior Rating Inventory of Executive Function—adult Version. Psychological Assessment Resources; 2005.
- 35.
Tant MLM. Visual performance in homonymous hemianopia: assessment, training and driving. s.n., 2002. 203 p.
- 36.
RStudio Team. RStudio: integrated Development for R [Internet]. Boston, MA: RStudio, PBC; 2022 [cited 2023 Mar 20]. http://www.rstudio.com/.
- 37. Muthén B. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika. 1984 Mar;49(1):115–32.
- 38. Rosseel Y. lavaan: An R Package for Structural Equation Modeling. J Stat Softw. 2012;48(2).
- 39. DiStefano C, Morgan GB. A Comparison of Diagonal Weighted Least Squares Robust Estimation Techniques for Ordinal Data. Struct Equ Modeling. 2014 Jul 3;21(3):425–38.
- 40. Beauducel A, Herzberg PY. On the Performance of Maximum Likelihood Versus Means and Variance Adjusted Weighted Least Squares Estimation in CFA. Struct Equ Modeling. 2006 Apr 28;13(2):186–203.
- 41. Flora DB, Curran PJ. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol Methods. 2004 Dec;9(4):466–91. pmid:15598100
- 42. Li CH. Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods. 2016 Sep;48(3):936–49. pmid:26174714
- 43. Hooper D, Coughlan J, Mullen M. Structural Equation Modelling: Guidelines for Determining Model Structural Equation Modelling: Guidelines for Determining Model Fit Fit. Electron J Bus Res Meth. 2008;6(1):53–60.
- 44. Steiger JH. Understanding the limitations of global fit assessment in structural equation modeling. Pers Individ Dif. 2007;42(5):893–8.
- 45. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.
- 46.
Satorra A. Scaled and Adjusted Restricted Tests in Multi-Sample Analysis of Moment Structures. In: Innovations in multivariate statistical analysis: A Festschrift for Heinz Neudecker. Boston: Springer US; 2000. p. 233–47. https://doi.org/10.1007/978-1-4615-4603-0
- 47. Brunner M, Nagy G, Wilhelm O. A tutorial on hierarchically structured constructs. J Pers. 2012 Aug;80(4):796–846. pmid:22091867
- 48. Green SB, Yang Y. Reliability of Summed Item Scores Using Structural Equation Modeling: An Alternative to Coefficient Alpha. Psychometrika. 2009 Mar 23;74(1):155–67.
- 49. Flora DB. Your Coefficient Alpha Is Probably Wrong, but Which Coefficient Omega Is Right? A Tutorial on Using R to Obtain Better Reliability Estimates. Adv Methods Pract Psychol Sci. 2020 Dec 6;3(4):484–501.
- 50. Zinbarg RE, Revelle W, Yovel I, Li W. Cronbach’s α, Revelle’s β, and Mcdonald’s ωH: their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005 Mar 2;70(1):123–33.
- 51.
McDonald RP. Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum; 1999.
- 52. Eisinga R, Grotenhuis Mt, Pelzer B. The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? Int J Public Health. 2013 Aug;58(4):637–42. pmid:23089674
- 53. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007 Jan;60(1):34–42. Epub 2006 Aug 24. pmid:17161752.
- 54. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016 Jun;15(2):155–63. pmid:27330520
- 55.
Cohen J. Statistical power analysis Jbr the behavioral. Sciences Hillsdale (NJ): Lawrence Erlbaum Associates. 1988;18–74. https://doi.org/10.4324/9780203771587 pmid:36153787
- 56. Benjamini Y, Hochberg Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Stat Soc B. 1995 Jan 1;57(1):289–300.
- 57.
UNESCO Institute for Statistics. International Standard Classification of Education (ISCED) 2011. International Standard Classification of Education (ISCED) 2011. Montreal: UNESCO Institute for Statistics. 2012.
- 58. Bornovalova MA, Choate AM, Fatimah H, Petersen KJ, Wiernik BM. Appropriate Use of Bifactor Analysis in Psychopathology Research: Appreciating Benefits and Limitations. Biol Psychiatry. 2020 Jul 1;88(1):18–27. pmid:32199605
- 59. Bonifay W, Cai L. On the Complexity of Item Response Theory Models. Multivariate Behav Res. 2017 Jul-Aug;52(4):465–484. pmid:28426237
- 60. Chen FF, Zhang Z. Bifactor Models in Psychometric Test Development. The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development. 2017 Jun 21;1–2:325–45.
- 61. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011 Jun 27; 2:53–55. pmid:28029643
- 62. Mangione CM, Lee PP, Pitts J, Gutierrez P, Berry S, Hays RD. Psychometric properties of the National Eye Institute Visual Function Questionnaire (NEI-VFQ). NEI-VFQ Field Test Investigators. Arch Ophthalmol. 1998 Nov;116(11):1496–504. pmid:9823352
- 63. Renaud J, Bédard E. Depression in the elderly with visual impairment and its association with quality of life. Clin Interv Aging. 2013;8:931–43. pmid:23888110
- 64. Kempen GI, Ballemans J, Ranchor AV, van Rens GH, Zijlstra GA. The impact of low vision on activities of daily living, symptoms of depression, feelings of anxiety and social support in community-living older adults seeking vision rehabilitation services. Qual Life Res. 2012 Oct;21(8):1405–11. pmid:22090173
- 65. Varadaraj V, Munoz B, Deal JA, An Y, Albert MS, Resnick SM, et al. Association of Vision Impairment With Cognitive Decline Across Multiple Domains in Older Adults. JAMA Netw Open. 2021 Jul 1;4(7): e2117416. pmid:34269806
- 66. Lin MY, Gutierrez PR, Stone KL, Yaffe K, Ensrud KE, Fink HA, et al. Vision impairment and combined vision and hearing impairment predict cognitive and functional decline in older women. J Am Geriatr Soc. 2004 Dec;52(12):1996–2002. pmid:15571533
- 67. Dearborn PJ, Elias MF, Sullivan KJ, Sullivan CE, Robbins MA. Poorer Visual Acuity Is Associated with Declines in Cognitive Performance Across Multiple Cognitive Domains: The Maine-Syracuse Longitudinal Study. J Int Neuropsychol Soc. 2018 Aug;24(7):746–754. pmid:29926789
- 68. Pereira FS, Yassuda MS, Oliveira AM, Forlenza OV. Executive dysfunction correlates with impaired functional status in older adults with varying degrees of cognitive impairment. Int Psychogeriatr. 2008 Dec;20(6):1104–15. pmid:18752698
- 69. Steffens DC, Bosworth HB, Provenzale JM, MacFall JR. Subcortical white matter lesions and functional impairment in geriatric depression. Depress Anxiety. 2002;15(1):23–8. pmid:11816049
- 70. Li W, Xiao WM, Chen YK, Qu JF, Liu YL, Fang XW, et al. Anxiety in Patients With Acute Ischemic Stroke: Risk Factors and Effects on Functional Status. Front Psychiatry. 2019 Apr 17; 10:257. pmid:31057444
- 71. Abramovitch A, Short T, Schweiger A. The C Factor: Cognitive dysfunction as a transdiagnostic dimension in psychopathology. Clin Psychol Rev. 2021 Jun;86:102007. pmid:33864968
- 72. Klaver CC, Wolfs RC, Vingerling JR, Hofman A, de Jong PT. Age-specific prevalence and causes of blindness and visual impairment in an older population: the Rotterdam Study. Arch Ophthalmol. 1998 May;116(5):653–8. pmid:9596502
- 73. Murman DL. The Impact of Age on Cognition. Semin Hear. 2015 Aug;36(3):111–21. pmid:27516712
- 74. Rashedi V, Rezaei M, Gharib M. Prevalence of cognitive impairment in community-dwelling older adults. Basic Clin Neurosci. 2014 Winter;5(1):28–30. pmid:25436081
- 75. Fiske A, Wetherell JL, Gatz M. Depression in older adults. Annu Rev Clin Psychol. 2009; 5:363–89. pmid:19327033
- 76. Xia Y, Yang Y. RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behav Res Methods. 2019 Feb;51(1):409–428. pmid:29869222
- 77. Savalei V. Improving Fit Indices in Structural Equation Modeling with Categorical Data. Multivariate Behav Res. 2021 May-Jun;56(3):390–407. pmid:32054327