Preliminary validation of a Urinary Symptom Questionnaire for individuals with Neuropathic Bladder using Intermittent Catheterization (USQNB-IC): A patient-centered patient reported outcome

Background We developed a Urinary Symptom Questionnaire for individuals with neurogenic bladder due to spinal cord injury (SCI) and spina bifida (SB) who manage their bladders with intermittent catheterization, the USQNB-IC. This project followed an approach to patient-centered patient reported outcomes development that we created and published in 2017, specifically to ensure the primacy of the patient’s perspective and experience. Participants Two sets of responses were collected from individuals with neurogenic bladder due to either SCI (n = 336) and SB (patients, n = 179; and caregivers of patients with NB, n = 66), and three sets of “controls”, individuals with neurogenic bladder who do not have a history of UTIs (n = 49) individuals with chronic mobility impairments (neither SCI nor SB) and without neurogenic bladder (n = 46), and those with no mobility impairment, no neurogenic bladder, and no history of UTIs (n = 64). Method Data were collected from all respondents to estimate these psychometric or measurement domains characterizing a health related PRO: Reliability (minimization of measurement error; internal consistency or interrelatedness of the items; and maximization of variability that is due to “true” difference between levels of the symptoms across patients), and validity (content, reflection of the construct to be measured; face, recognizability of the contents as representing the construct to be measured; structural, the extent to which the instrument captures recognizable dimensions of the construct to be measured; and criterion, association with a gold standard). Results Evidence from these five groups of respondents suggest the instrument has face, content, criterion, convergent, and divergent validity, as well as reliability. The items were all more descriptive of our patient (focus) groups and were only weakly endorsed by the control groups. Conclusions The instrument is unique in its emphasis on, and origination from, the lived experiences of patients with neurogenic bladder who use intermittent catheterization; this preliminary psychometric evidence suggests the instrument could be useful for research and in the clinic. These results justify further development of the instrument, including formal exploration of the scoring and estimation of responsivity of these items to clinical interventions as well as patient-directed self care.

Introduction Urinary tract infection (UTI) is a major chronic and recurrent health condition for people with neurogenic bladder, as well as being a major worldwide public health problem. [1][2][3][4] Neurogenic bladder is also associated with a disproportionately high risk of genitourinary complications, including bladder and kidney infections, [1,[5][6][7] calculi, [8,9] and bladder cancer, [5,10] among others. While neurogenic bladder can result from any trauma or disease of the brain or spinal cord, people with spinal cord injury (SCI) and spina bifida (SB) are nearly universally affected. Urinary tract infections (UTIs) and renal damage were historically the most common cause of death for people with SCI and SB [7], and while that early mortality has declined with improved genitourinary management, UTIs remain the most common cause of emergency department visits [8] and rehospitalization [11] among people with neurogenic bladder. [5,[12][13][14] Identification of a UTI is paramount in prevention and management efforts, and that is complicated in the patient with neurogenic bladder by several factors. One of the most important challenges in early detection of a UTI is that the patient needs to identify symptoms that may be indicative of UTI, and then those symptoms need to be understood by a health care professional to determine whether the symptoms represent an actual UTI, non-UTI urinary symptoms, or symptoms due to another cause. What constitutes symptoms indicative of UTI, which are often significantly altered in people who have sensory impairment relative to groups from whom neurologic function is intact, is not clearly defined for people with neurogenic bladder. These gaps are long-standing and have resulted in attempts to address diagnostic shortcomings by the National Institute of Disability and Rehabilitation Research (1992), the Agency for Healthcare Research and Quality (1999), and the Infectious Diseases Society of America (2009). Without exception, all of these guidelines note the lack of high quality supporting evidence for their use, and each calls for research addressing these challenges in people with neurogenic bladder. number: AD-1310-08215 to SLG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The first/corresponding author is a Section Editor with PLOS ONE, and neither this nor the employment of the other authors by a commercial company (MedStar) alters our adherence to PLOS ONE policies on sharing data and materials. There are no other actual or potential conflicts of interest.
To that end, our research team is developing a series of Urinary Symptom Questionnaires for people with neurogenic bladder according to bladder management method. This report specifically refers to the development of a Urinary Symptom Questionnaire (USQ) for individuals with neurogenic bladder (NB) due to SCI and SB, and who manage their bladders with intermittent catheterization (IC) (USQNB-IC), focusing on upper and lower urinary tract signs and symptoms. While historically, UTI diagnostic criteria and instrument development have been driven primarily by clinical perspectives [15], without accommodating the patient's lived experience, this instrument was developed following an approach that we developed [16] specifically to ensure the primacy of the patient's perspective and experience with neurogenic bladder. Our objective has been to create an instrument that can promote patients' self-management of urinary signs and symptoms, and which may be useful for preventative, comparative effectiveness, and diagnostic research around urinary signs and symptoms, or possibly of UTIs, in the future. As such, the patient's report is essential; however, rather than focusing on quality of life, as many "patient reported" perspectives on UTI have done [17][18][19], our overall approach has had a purposeful focus on signs and symptoms in the patients' experience that could potentially bear on interventions and their development and validation in the future.
Based on clinical experience, we suspect a urinary symptom instrument will differ depending on the degree of neurogenic impairment of the bladder, and because bladder management method (indwelling catheterization, intermittent catheterization, and voiding, in decreasing order of severity) is, to some extent, a proxy for degree of neurogenic bladder dysfunction. This initial effort is focused on people with neurogenic bladder who manage their bladders with intermittent catheters, and here we report on the preliminary validation evidence for the USQNB-IC for this population. This article follows and utilizes the recommendations and definitions of the international Consensus-based Standards for the selection of health Measurement Instruments, COSMIN [20] to document the measurement properties of the new USQNB-IC.

Methods
Approval for all parts of this study was received from the MedStar National Rehabilitation Hospital Institutional Review Board (IRB # 2013-187). Participants were recruited by advertising the need for responses on this new instrument through Facebook, and via email and otherwise with the assistance of the national (U.S.) advocacy networks in spinal cord injury and spina bifida. All of these outreach and recruitment efforts were advertisements seeking respondents with NB who use intermittent catheters to visit the URL we established for data collection. Within 10 months, nearly 600 responses had been obtained from individuals with spinal cord injury, spina bifida, and their caregivers. We then sought responses from participants in our validation (divergent/convergent) populations, described below. No personal or identifying data were collected from any respondent, and initiating responses on the survey was deemed sufficient consent by the IRB. We developed the USQNB-IC following a new model for the creation of patient-centered patient reported outcomes [16] shown in Fig 1.

Methods: Materials
An instrument with 29 items resulted from a total of 13 focus groups with eligible patients (with neurogenic bladder due to SCI or SB, and who use intermittent catheters) or their caregivers, from around the country, plus iterative focused discussions with a core of four clinicians with expertise in this domain (as described in Tractenberg et al. 2017). [16] As described below, a national sample of people with neurogenic bladder who manage their bladders with intermittent catheterization were recruited from patient populations of SCI (N = 336) and SB (N = 178), plus caregivers of people with SB (N = 66). Participants were recruited to complete the instrument online, using SurveyMonkey. Each item was presented as a query about whether the respondent had experienced it during the past year (yes/no), with three additional required responses: average frequency (0-365); average severity (usually not at all severe; usually somewhat severe; usually severe; always very severe); and average impact on, or importance in, daily life (rarely affects my actions or decisions to go about my daily life; sometimes affects my actions or decisions to go about my daily life; usually affects my actions or decisions to go about my daily life; always affects my actions or decisions to go about my daily life). The analyses described below were focused on the endorsement (yes/no) of items as sources of validity evidence for the instrument and its makeup. Once preliminary validity evidence is established, we plan to revisit the frequency, severity, and impact ratings to develop and refine a scoring algorithm in a future article.

Methods: Validation
Our analyses were planned to describe the validity of the USQNB-IC using classical psychometric methods. [21] The COSMIN report identifies four key psychometric or measurement domains on which a health related PRO should be characterized: Reliability (minimization of measurement error; internal consistency or interrelatedness of the items; and maximization of variability that is due to "true" difference between levels of the symptoms across patients), validity (content, reflection of the construct to be measured; face, recognizability of the contents as representing the construct to be measured; structural, the extent to which the instrument captures recognizable dimensions of the construct to be measured; and criterion, association with a gold standard), cross-cultural validity, and the interpretability of scoring. Cross-cultural validity would not be relevant if any of the other measurement properties are not met, so we did not address this type of validity in this study. Similarly, responsiveness, defined by the COSMIN criteria as the sensitivity of an instrument to change in the construct of interest, was also not evaluated in this study because like cross-cultural validity, that would be premature.
The development of this instrument was characterized by a focus on face and content validity from both the clinician and the patient perspectives (Fig 1, above), thus this report augments that content and face validity evidence. Because there is, by their origins, no diagnostic "gold standard" for what these signs and symptoms represent, our evidence on convergent and divergent validity was classified as supporting both "reliability" and "criterion validity", as described in the next section.
Importantly, the COSMIN requirements for "structural validity", which they defined as dimensionality, are not consistent with either the approach to patient-centered patient reported outcomes that we followed, nor with the early stage of our instrument development. Specifically, for formal structural dimensionality to be a feature of a PRO at this early stage, it would need to have been developed following procedures outlined by Bollen (1989), with a measurement model in mind. [15,21] Instead, our instrument was developed (as noted) with no measurement model in mind, but instead with the specific intention of capturing the patient experience with high fidelity. Thus, the dimensionality on which we can report relates to the aspects of urinary signs and symptoms, and not to specific causal or response-generating features of the items on the instrument. We estimated internal consistency of the items with Cronbach's alpha, which is a function of the correlation of each item with the total score that takes the total score variability into account, and provides an estimate of the shared covariance of items on the instrument-but does not relate to the instrument's dimensionality or response-generating mechanism. [22,23] In this report we describe new evidence of face and content validity as well as estimating internal consistency (COSMIN defined "reliability") with Cronbach's alpha, dimensionality (COSMIN defined "construct validity") with exploratory common factor analysis, modeling the covariance of items; and inferred causal model with Bayesian networks, modeling the shared Shannon Information. Given the early stage of this instrument, we demonstrate both construct and criterion validity (which COSMIN defined as including divergent and convergent validity) with distributions of scores (total and item endorsements) for the target population of individuals with neurogenic bladder who manage their bladders with intermittent catheterization, and also with data from individuals with neurogenic bladder who do not have a history of UTIs, individuals with chronic mobility impairments (neither SCI nor SB) and without neurogenic bladder, and those with no mobility impairment, no neurogenic bladder, and no history of UTIs. Table 1 below recapitulates the COSMIN measurement properties and describes the methods by which these were analyzed in this article.

Methods: Target patient population participants and recruitment
A nationally representative sample of individuals with neurogenic bladder due to either SCI or SB was recruited to complete the survey. We also obtained responses from individuals caring for people with SCI or SB. These samples represent the target populations as well as convergent validity data, and are described below.
Inclusion Criteria: Inclusion criteria for participants in the national sample with SCI: 1) age!18 years; 2) SCI at least 1-year duration; 3) neurogenic bladder, as reported by respondents; 4) utilizing intermittent catheterization for bladder management; 5) and a history of 2 or more UTIs in the past year. Inclusion criteria for participants in the national sample with SB were: 1) age! 18 years; 2) neurogenic bladder, as reported by respondents; 3) utilization of intermittent catheterization for bladder management; 4) and a history of 2 or more UTIs in the past year. The inclusion criterion for participants in the national sample who were caring for a dependent with SB was only that they were a caregiver for an individual with SB or SCI.
Our consumer partner, United Spinal Association, assisted in recruiting nationally-representative samples of adults with SCI (N = 336), adults with SB (SB; N = 179) caregivers of children with SB (CG; N = 66).

Methods: Divergent validity participant recruitment
Individuals with neurogenic bladder who did not self-report a history of UTIs (however the respondent defined this), individuals with chronic mobility impairments (neither SCI nor SB) and without neurogenic bladder, and individuals with no mobility impairment, no neurogenic bladder, and no history of UTIs were recruited largely from an inner city rehabilitation hospital to complete the same instrument.

Methods: Data analysis
Statistical analyses were carried out using SPSS v. 24 [24] for descriptive statistics and to estimate the variance explained by the exploratory factor analysis; exploratory factor analysis with tetrachoric correlations was carried out in MPlus v. 8 (Statsoft, Los Angeles, CA). BayesiaLab v. 6 (Bayesia S.A.S., Laval, France) was used for the Bayesian network modeling: unsupervised structural learning (searching for the overall data structure) using a Maximum Weight Spanning Tree (MWST) learning algorithm to constrain the structure so that only one "parent" per variable is identified. Minimum Description Length (MDL-information, not probability) was used for scoring because linearity in relationships cannot be safely assumed for these items. The resulting tree was visualized in automatic layout for interpretability (features discussed in Ch. 7 of Conrady & Jouffe, 2015). [25] The data analyzed in this manuscript are available as Supporting information files.

Results
The results are presented in two parts. First, descriptive results for each of our three responding diagnostic groups are presented, followed by the descriptive data from our three "control" groups. Then, we describe the validity and reliability results for the instrument based on these respondents.

Results: Descriptive statistics, SCI
Complete and valid (i.e., from eligible respondents) responses to the USQNB-IC were obtained from 336 individuals with spinal cord injury (SCI). On average, respondents with neurogenic bladder were 46.7 years old (sd: 13.64y; range 18-75) and had been living with their injury (and neurogenic bladder) for an average of 14.7 years (sd: 12.5; range 1-62 years). Of this sample, 45.5% were women and 30.1% reported that they reuse catheters; we did not obtain the level of injury but participants did report whether their injury was complete (39.9%), incomplete (53.9%), or unknown (6.8%).

Results: Descriptive statistics, SB
Complete and valid responses to the USQNB-IC were obtained from 179 individuals with spina bifida (SB). On average, respondents were 35.3 years old (sd: 11.7 y; range 18-65). Of this sample, 72% were women and 42.8% reported that they reuse catheters.

Results: Validity evidence, reliability
The approach to validation was derived from Mokkink et al. (2010)[20], as discussed above; content and face validity are conferred by the process by which the instrument was created (Fig 1). [16] Table 2 summarizes the COSMIN criteria and our results following the methods outlined in Table 1.

Results: Exploratory factor model and inferred causal network
Principal axis factoring (PAF) with oblimin rotation extracted eight factors from the tetrachoric correlation matrix of endorsement rates of the overall sample (N = 581; data not shown). This model fit the data well (root mean square of the approximation, RMSEA = 0.024, 95% CI: 0.016-0.031; confirmatory factor index, CFI = 0.985; Tucker's factor index, TFI = 0.971; standardized root mean square of the residuals 0.037). The 8-factor model fit significantly better than the 7 (chi square 52.8 with 22 degrees of freedom, p = 0.002) and all larger models up to 8 fit significantly better than models with fewer factors (data not shown but all p<0.001); however, we estimated that just 35.6% of the total variance in this system was explained by the 8factor model using the PAF method with oblimin rotation based on the Pearson Correlation matrix in SPSS. Oblimin was used because of the high likelihood of correlated factors (given the origins of the items within focus group sessions guided by a single common script); many of the eight extracted factors were correlated with p<0.05; these data are not shown because although the 8 factor model fit well according to the fit indices, a) an 8 factor model did not have much interpretability; b) the estimated variance explained was inadequate (35.6%)-yet this was greater than for any smaller model; and c) we did not hypothesize a meaningful measurement model given the time frame of our questions and the origins of these items (i.e., without a measurement model in mind).
The sub-samples were too small (apart from SCI) to re-run this extraction procedure with interpretable results; however it was not hypothesized that collapsing over diagnostic group was the cause of the limited explanatory power of the modeling. Given the origins of these items (i.e., not following Bollen 1989/no measurement model in mind) and the fact that all queries were formulated for "the past year", the lack of a coherent causal measurement model that explaining a majority of the variability among the three neurogenic bladder response groups was not surprising.
We next used inferred Bayesian networks (Bayesialab v. 6, Bayesia S.A.S., Laval, France) to discover evidence of underlying structure for these items. This method is not parametric, does not use or require inferences, and uses information instead of probability for estimating associations. The results are, therefore, associative and not specifically causal. This method allowed us to identify possible clusterings of symptoms, without sample sizes or parameter estimation precision being the issue that they can be in an inference-, or estimation-, oriented method like PAF. Like the PAF analysis above, the Bayesian network was inferred based on the yes/no endorsement of all items for the full sample using unsupervised learning and the maximum spanning tree option. This method was used to uncover basic structure, but limit the associations that appeared to a single-neighbor (i.e., a Markov Blanket approach). This method is similar to the employment of rotation in factor analysis solutions for "simple structure".
We discovered a network with a complex structure from the full sample of 581. The network is represented in Fig 2 below.
The network shown in Fig 2 highlights the complexity of the lived experience of urinary signs and symptoms in neurogenic bladder. Importantly, as anticipated (see comments above), the structure does not show a single causal or originating factor for all or even some of the symptoms; this has both treatment (clinician) and management (patient) implications, as well as highlighting critical characteristics of the patient's experience for clinical research and clinical trials of future interventions for UTI. For example, "incontinence" and "frequency of spasms" are directly or indirectly related to the majority of symptoms; therefore, if interventions address these two symptoms, they may have the most distributed effects. By contrast, the associations of "generally not feeling well" are more limited (i.e., only associated with fatigue, fever, headaches, etc.), and therefore, an intervention that addressed "generally not feeling well" might have much more limited effects than one addressing incontinence. At this preliminary stage of validation we asked whether each item had been experienced within one year. Our follow-up research questions will include whether each item was experienced within any one-week period, because that is the putative timeframe for use of the instrument in research or treatment. Because of the confounding effects of the timeframe of the questions, and because the measurement model structure (e.g., path weights, factor makeup) was not our purpose with this model, we did not estimate fit, factors, or strengths of associations between any of the items. That this network was readily interpretable is additional evidence that the USQ-NB-IC has sufficient structural validity for continuing the evaluation of its measurement properties. The interpretability of this network also contributes to the reliability evidence for the instrument.
The instrument demonstrates reliability and construct validity as discussed in the preceding section. Content and face validity are conferred by the process by which the instrument was created (Fig 1). Additional content and face validity evidence arises from an examination of endorsement rates across the three groups of nationally-representative patients with neurogenic bladder in our sample. Table 3 provides the presenting the endorsement rates (number of respondents identifying having experienced the symptom within the past year/total number of respondents in that group) by persons with neurogenic bladder for each symptom on the USQNB-IC in the preceding 12 months.
Endorsement rates are similar for individuals with SCI & SB, whereas endorsement by item was slightly different for the caregivers. The endorsement rates for each item on the instrument were at least 10% in every group, supporting a claim of face validity across these participants since all items are perceptible to these three patient groups. Table 3 shows that the endorsement rates for the items are generally similar across groups, except for rigor (far more frequently endorsed by SCI). We did not test for significant differences across groups due to a) a need for correction for 29 comparisons (reducing the likelihood that any would remain significant) and b) our assumption that the focus groups and focus group facilitators, who were consumers themselves (as described previously [16]), were able to capture the lived experience of neurogenic bladder across etiology. Four of the five most highly endorsed items were the same in each group: Darker urine; cloudy urine; bad smelling urine; and generally not feeling well. Fatigue was reported in the SB group with greater frequency than the other two groups. The fifth most frequently endorsed item was increased incontinence for SCI; increased leakage for SB and fever for CG.
The similarities of endorsement rates across groups offers content and face validity evidence, together with convergent validity and some reliability evidence. It also suggests that neurogenic bladder-which was the defining feature of the focus groups from which the symptoms were derived-does have a set of "signs and symptoms" that neurogenic bladder patients may experience. The focus groups convened neurogenic bladder patients from diverse etiologies and their input created a set of signs and symptoms that the national sample of neurogenic bladder respondents recognized. The similarities suggest the instrument is relevant/resonant for neurogenic bladder respondents who manage their bladders with intermittent catheters, also contributing evidence of face and content validity. Table 4 presents the endorsement rates across our control groups. These groups were recruited (as noted above) to evaluate the prevalence of these symptoms for other groups (not SCI or SB) with mobility impairments without neuropathic bladder, those with neuropathic bladder but who do not have frequent UTIs, and finally from those without mobility impairment, neuropathic bladder, or frequent UTIs. Importantly, in Table 4, endorsement of items was not simply interpretable as "yes/no" because for some of these participants the items had no possibility of endorsement. That is, for divergent validity participants, catheterization was very rare (just 16 of 160) whereas all participants in the USQNB-IC groups did use intermittent catheterization. Thus, the endorsement rates in Table 4 are given together with the number of respondents in any given group for whom the item was possible to endorse.
In Table 4, the endorsement rates for "increased catheterization" and "burning on catheterization" are low, but the number of individuals responding to the item is also quite low. This is highlighted because these endorsement rates (%s) cannot be directly compared to the rates from Table 3, as the values in Table 4 for these items, at least, are based on dramatically smaller samples.
The last row in the table shows that the average number of endorsed items was 5.84 of 29 for the controls overall; this is less than half of the average for the patient groups ( Table 3). The highest average number of endorsed items was in the group without mobility impairments and without neurogenic bladder (7.0, SD = 4.45). The lowest average number of endorsed items (4.55, SD = 4.27) was for those with mobility impairments who did not have neurogenic bladder.
There was agreement among the four most-frequently endorsed items among the control groups shown in Table 4. Increased urgency was among the top five most endorsed items in all three groups, while fatigue was in the top five for those with mobility impairment/no neurogenic bladder and those with neurogenic bladder but no UTIs; generally not feeling well and muscle aches were in the top five for both those with neurogenic bladder but no UTIs and those with neither UTIs, neurogenic bladder, nor mobility impairment. None of the endorsement rates for these items was as high as it was for any of the neurogenic bladder patient groups; the most striking difference between these control group endorsement rates and those from our national sample is for those with neurogenic bladder who have not experienced any (many) UTIs. Their rate of endorsement is the second lowest of the three control groups (with those who have neither mobility impairment nor neurogenic bladder having the highest average endorsement rate).
A final aspect of validity was explored by studying whether respondents tended to attribute the symptoms to a UTI. This feature of the instrument was challenging to conceptualize, not only because the frame of reference for endorsement was the previous year, but also because attribution, by the patient, of a symptom "to a UTI" could arise from a wide variety of ultimately unverifiable rationales. For example, a respondent could attribute a symptom to a UTI for any (one or more) of the following reasons: 1. Their clinician called it a UTI (whether or not this was confirmed); 2. Their clinician treated it with/prescribed antibiotics (whether or not this was consistent with clinical practice guidelines, which are noted to be based more on expert consensus than evidence) [26]; 3. They self-diagnosed a UTI; 4. Previous experience with that or a similar symptom was clinically-or self-diagnosed as a UTI; 5. The symptom represented a deviation from normal and was attributed to UTI.
Because it is (and was) impossible to verify whether any experience was actually a UTI, we instead present results based on respondents' assertions that the symptom was never associated with a UTI. This would therefore exclude all of the above possible associations with, or attributions to, a UTI. Therefore, Table 5 presents the proportions of respondents, collapsed over our national patients-with-neurogenic bladder samples (n = 581) and control (n = 160) groups. In each case, the proportions of those who endorsed each item and also never attributed it to a UTI are given. Table 5 shows that the controls attributed just one of the USQNB-IC items to UTI in any appreciable numbers: "burning on passing urine". "Blood in urine", "cloudy urine", and "abdominal pain" were the items with the lowest proportion of controls attributing the symptom to a UTI. By contrast, the majority of the patient groups attributed 16 of these 29 symptoms to UTIs-these are indicated with an asterisk ( Ã ). Because of how participants in both groups were recruited, it is supportive of face and content validity claims that most patients with experience with UTIs do attribute most items to having a UTI while participants who were recruited specifically because they have limited experience with UTIs do not attribute these symptoms to having a UTI. Also, members of our focus groups for developing the instrument originally were from the patient group, not from the divergent validity "control" groups. Thus the higher likelihood of attributing these symptoms to having a UTI in our target patient group also supports the reliability and validity of these items.
Taken together, these results suggest that our instrument possesses important validity evidence according to the COSMIN criteria. Specifically, the results present several lines and sources of evidence of face, content, criterion, convergent, and divergent validity for these items; reliability evidence also arises from the results considered all together.

Discussion
The purpose of this study was to describe the measurement properties of a new patient-centered patient reported outcome instrument for urinary signs and symptoms in people with neurogenic bladder and who use intermittent catheterization, the USQNB-IC. Overall, our validity evidence addresses the following COSMIN criteria: 1. Face, content, convergence validity, reliability: All items-which were generated by patients themselves and then integrated with clinician input (see Fig 1)-were recognized and endorsed by at least 10% (most were endorsed by at least 50%) of our target patient populations: those with neuropathic bladder who use intermittent catheterization, and who experience frequent UTIs (Table 3).
2. Divergent validity, reliability: Few items were endorsed by even 50% of any of the control groups ( Table 4), suggesting that the items are more descriptive for our target patient group and less descriptive of the experiences of those outside that group. This divergent validity evidence also supports claims of reliability, namely, that the items describe what they are intended to describe.
3. Convergent, divergent validity: The total number of items endorsed by patient groups were similar to each other, but were more than double, on average, the total number endorsed by control groups (Table 5).
4. Criterion validity: Of those endorsing any item, one item (burning on passing urine) was attributed to a UTI by 50% of the controls but every other item was not attributed to a UTI by 75% or more controls (most items were not attributed to a UTI by 100% of controls). Burning on passing urine is a classic symptom of a UTI, offering additional convergent validity evidence, as well as criterion validity evidence, from the control groups.
This instrument is substantially (and substantively) different from any other: it was developed using explicitly patient-centric methods, is specific to the lived experience of the neurogenic bladder population, and the focus of these items is on infection-related symptoms. Urinary symptoms tools and health quality of life scales generally fall under the domains of female/general urinary incontinence, male urinary symptoms, and specific urinary symptoms (nocturia, overactive bladder, urgency, patient perception of bladder condition, and intermittent catheterization). [15] A recent (2016) systematic review of patient reported outcome measures for neurogenic bladder and bowel revealed heterogeneity in the current patient reported outcome measures in this area and a clear focus on quality of life. While the Medical Outcomes Study SF-36 was the most frequently utilized tool, only three bladder or bowel specific measures were identified: the Qualiveen, FICQoL, and the QoL-BM-all quality of life, and not symptom, measures. [18] Another systematic review examined and compared validated questionnaires for people with neurogenic bladder due to SCI and multiple sclerosis. Of 18 questionnaires identified, 14 were for people with MS, 3 for people with SCI, and 1 was general. [17] The only peer reviewed, published symptom scale that we identified for people with neurogenic bladder is the NBSS (Neurogenic Bladder Symptom Scale). [15,27] In the development of this generic scale, 266 items were generated from a search of the existing/published (not specific to neurogenic bladder) measurement tools. Of these, only two were specific to urinary tract infection. A multi-disciplinary team reduced the original set of 266 items, and interviews were conducted with consumers with SCI and MS, who identified urinary incontinence, UTI, urgency, and bladder spasms to be dominant issues for them. [15] While our instrument is also specific to individuals with NB, our measure differs significantly from the NBSS in terms of the origins of our items (i.e., see Tractenberg et al. [16] and Fig 1) and their scope. As noted, the development of our measure was fundamentally different from that of the NBSS, as our core information/symptoms is based in patient report, as opposed to deriving from clinicianor evidence-defined reports. [16] The USQNB-IC has the potential to meaningfully advance the quality of health care in the population of people with neurogenic bladder, a population in which UTIs are common, and likely over-treated with antimicrobials. [28] This over treatment may be due, in part, to our lack of understanding of the different clinical components contributing to UTI diagnosis (symptoms, inflammatory markers, bacterial load). There is currently broad consensus that only symptomatic UTI should be treated, [28][29][30] however the term "symptomatic UTI" is not well defined. We expect that, deriving from the patients' experience, the USQNB-IC includes symptoms that may be attributable to infection, as well as those that precede the infection, possibly signaling an increase in susceptibility or vulnerability. Most scientists and clinicians agree that fever is a symptom [28,30,31] of an infection, and when no other source of infection can be identified, UTI is a common source. However, difficulties remain in differentiating other symptoms more likely to be indicative of UTI versus non-infectious or non-urinary symptoms. According to the International SCI UTI Basic Data Set [32], signs/symptoms of UTI include: fever, incontinence (onset or increase in episodes, including leaking around catheter), increased spasticity, malaise (lethargy or sense of unease), cloudy urine (with or without mucus or sediment) with increased odor, pyuria, discomfort or pain over the kidney or bladder or during micturition, autonomic dysreflexia, and 'other'. [32] The International SCI UTI Data Set is intended to be a standardized collection tool for the minimal amount of information related to a possible UTI. As such, these symptoms are derived from clinical consensus and are appropriately broad for the purpose of a standardized data set. These nine symptoms (which are a subset of the ones our focus groups and clinicians identified, except for autonomic dysreflexia), are not intended to be specific and comprehensive enough to be used in routine clinical practice. By contrast, the USQNB-IC has 29 urinary symptoms that encompass the authentic lived experience of urinary symptoms by the patient with neurogenic bladderas well as these data set items-so these might be useful for routine clinical practice as well as research and potentially, self-management by patients.
There is significant overlap between the International SCI UTI Data Set-identified signs/ symptoms and the USQNB-IC: "fever", "incontinence", "malaise", "increased spasticity", "discomfort", and "cloudy urine" are present in both the International SCI UTI Data Set and our patient-centric list; however "incontinence" is further elaborated on by patients with additional descriptive symptoms such as "increased frequency of spasms", "increased urgency", "increased catheterization", "increased incontinence", and "increased urine leakage." Similarly, in addition to "malaise", the USQNB-IC captures constitutional symptoms identified by our focus groups as distinct, such as "fatigue", "generally not feeling well", "muscle aches", "altered sleep patterns", "increased irritability", "dizziness", and "loss of appetite". We are in the process of deepening our understanding of how all of these patient-and clinician-focus groupidentified symptoms on the USQNB-IC can be used by patients and clinicians, as well as investigators studying potential intervention and self-management strategies.
Although we have multiple lines of evidence supporting claims that the instrument is valid, the study has some limitations and considerations. Foremost among these is that we did not include a control group that did not have neurogenic bladder but did have as many urinary problems as our neurogenic bladder patient groups did. However, a proxy for this evidence is the consensus-based guidelines for diagnosing urinary tract infection set out by the Infectious Diseases Society of America (IDSA). [26] These guidelines are derived from clinician experience with general UTI patients; those with neither mobility impairment nor neurogenic bladder and with UTIs. Testing the convergent and divergent validity of these items with an equally symptomatic group that has no mobility impairment is an important next step in our validation efforts, although our earlier work [16] shows that consensus-based guidelines, which are not based on individuals with neurogenic bladder and so do derive from an equally symptomatic group that has no mobility impairment, are not as frequently endorsed among persons with neurogenic bladder as those derived from our focus groups. Five of these IDSA guideline symptoms are among the lowest endorsed items of the USQN-B-IC in the patient population.
A second limitation is that we asked whether each item had been experienced within one year, although the intended use of the instrument is for a much tighter time window, such as one or two weeks. Because this was a preliminary validation study, we wanted to ensure that we captured patient experiences with these symptoms; we asked those in the national sample to estimate the frequencies of each item within the past year to get an idea of whether using this instrument with shorter time frames would ever be feasible. We also asked about the past year to capture evidence of any possible seasonality or other longer-term fluctuations in symptoms-as well as to ensure that we did not obtain misleading evidence about the relevance of these symptoms by artificially limiting the recall window. Other research questions our team is investigating, for which this instrument was specifically created, do involve administration of the USQNB-IC every week, so eventually we will have a sense of endorsement rates over shorter time frames once that study is completed. For these preliminary results, to increase the likelihood of capturing interpretable data on these signs and symptoms, the one-year timeframe was essential, even if these endorsement rates over that period preclude us from determining, for example, which of these items occur (or are most likely to occur) together.
A further consideration relates to our exclusion and inclusion criteria: while our focus groups were solely comprised of community-dwelling individuals, the national survey did not exclude respondents with any of the following conditions from participation: 1) known genitourinary pathology beyond neurogenic bladder (i.e., vesicoureteral reflux, bladder or kidney stones, etc.); 2) use of prophylactic antibiotics; 3) instillation of intravesicular agents to reduce UTI (i.e., gentamycin); 4) psychologic or psychiatric conditions influencing the ability to follow instructions; and 5) participation in another study in which results would be confounded.
Although these were all exclusion criteria for the focus groups, it is possible that our national sample included respondents with some of these potentially confounding factors. While this may enhance the generalizability of our national results, the relationships between these patient characteristics and responses on the USQNB-IC are unknown at this point.
Finally, these data suggest that having neurogenic bladder does not by itself confer the UTI/ urinary problems that all our SCI, SB, and CG reported, because we found a control sample of individuals who also have neurogenic bladder, but who endorsed the same signs and symptoms much less often than those who do have frequent UTIs. Thus, we can now incorporate this fact in the designs of future studies and interventions, knowing that an individual has neurogenic bladder alone is too broad to identify a target patient population; further, tools like the NBSS may not be sufficiently specific for work in this population. Our own work continues with efforts to develop new instruments (following the same methods outlined here) for patients with neurogenic bladder who manage their bladders with either indwelling catheters or by voiding.

Conclusions
This instrument was developed as a specifically patient-centered patient reported outcome, and so is consistent with principles of "valuing the patient perspective", and maintaining a "culture of patient centeredness in research". [4] However, this patient-centered approach is inconsistent with formal methods for instrument development (i.e., those laid out by Bollen, 1989) [21] that lead to the very measurement properties that are of greatest interest (as per Mokkink et al. 2010). [20] Our work in this area (urinary symptom detection and management for neurogenic bladder) has balanced these two perspectives, and our preliminary validity evidence suggests that both can be integrated to yield a valid PC-PRO instrument. Our next objective is to explore the best scoring approaches to this instrument, keeping the balance in mind, but also considering that we intend this instrument for use by patients in their own self-management, and by clinicians in their management of (collaboration with) patients who are also using the instrument.
Supporting information S1 Data. Data for these analyses. This work was supported by PCORI grant number: AD-1310-08215 to SLG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders did not have any role in the study design, data collection, data analysis, decision to publish, or preparation of the manuscript.
Competing Interests: The first/corresponding author is a Section Editor with PLOS ONE, and this does not alter our adherence to PLOS ONE policies on sharing data and materials. There are no other actual or potential conflicts of interest.
The first author had full access to all of the quantitative data in the study and takes responsibility for the integrity of the data and the accuracy of the quantitative data analysis. The quantitative data are available in the file S1 Data. The third author had full access to all of the qualitative data in the study and takes responsibility for the integrity of the data and the accuracy of the qualitative data analysis. The qualitative data are all embedded in the manuscript.