Psychometric properties of the Adult Primary Care Assessment Tool Short form (PCAT-S) among high-risk patients in Australian general practice

Chau M. Bui; Marijka J. Batterham; Judy Mullan; Gregory Peterson; Christine Metusela; Jan Radford; Simon Eckermann; Danielle Mazza; Grant Russell; Andrew Bonney

doi:10.1371/journal.pone.0341250

Abstract

Introduction

The Primary Care Assessment Tool (PCAT) is designed to assess a patient’s experience with primary care across various core and ancillary domains, including First contact – Utilization, First contact – Access, Ongoing Care, Coordination, Comprehensiveness (services provided), Family-centeredness, Community Orientation, and Cultural Competence. This study examined the psychometric properties of the Adult Primary Care Assessment Tool Short Form (PCAT-S) in the Australian general practice setting.

Method

Data included 715 participants from the EQuIP-GP study, a cluster randomized controlled trial (RCT) conducted with adults aged 18–65 years with a chronic illness or aged over 65 years, from 34 general practices across Australia. For each subscale we assessed internal consistency using Cronbach’s alpha. Factor structure of the PCAT-S instrument was assessed through confirmatory and exploratory factor analysis, using three samples with different methods for handling ‘don’t know/can’t remember’ responses.

Results

The findings were mixed. Specifically, the subscales related to First Contact – Utilization, Ongoing Care and Comprehensiveness, demonstrated satisfactory internal consistency. However, the remaining subscales showed weak internal consistency. Confirmatory factor analysis indicated potential model misspecification, while exploratory factor analysis generally supported the hypothesized factor structure, albeit with some observed deviations.

Conclusions

The findings indicate the PCAT-S shows promise as an instrument to evaluate primary care experiences in Australia. However, the observed variability in internal consistency, along with issues identified in confirmatory and exploratory factor analyses, highlight the need for further validation and refinement in this population. Further research is required to address the identified limitations and enhance the tool’s applicability within the Australian general practice context.

Citation: Bui CM, Batterham MJ, Mullan J, Peterson G, Metusela C, Radford J, et al. (2026) Psychometric properties of the Adult Primary Care Assessment Tool Short form (PCAT-S) among high-risk patients in Australian general practice. PLoS One 21(2): e0341250. https://doi.org/10.1371/journal.pone.0341250

Editor: Bojana Bukurov, University of Belgrade: Univerzitet u Beogradu, SERBIA

Received: October 24, 2024; Accepted: January 5, 2026; Published: February 6, 2026

Copyright: © 2026 Bui et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Requests for data may be made to the authors, or if unavailable, through the UOW Health and Medical Human Research Ethics Committee (uow-humanethics@uow.edu.au).

Funding: This study used funding from a larger project funded by the Australian Government Department of Health and Aged Care (https://www.health.gov.au/). This funding was provided via the Royal Australian College of General Practitioners (RACGP, https://www.racgp.org.au/). The funding bodies had no role in the design, data collection and analysis, decision to publish, or preparation of the manuscript. GR, JR and DM have received honoraria from the RACGP for expert committee roles. The other authors have no conflicts to declare.

Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: Prof Grant Russell, A/Prof Jan Radford, and Prof Danielle Mazza have received honoraria from the RACGP for expert committee roles. The other authors have no conflicts to declare. This does not alter our adherence to PLOS ONE policies on sharing data and materials. Requests for data may be made to the authors, or if unavailable, through the UOW Health and Medical Human Research Ethics Committee (uow-humanethics@uow.edu.au).

Introduction

Primary health care is widely recognized as fundamental to health systems for attaining universal health coverage, better health outcomes, and health equity [1,2]. Many countries have prioritized strengthening primary care as part of health system reforms [2]. Globally, there has been a growing emphasis on measuring patient experiences as a means to assess health system performance [3].

The Primary Care Assessment Tool (PCAT) was developed by the Johns Hopkins Primary Care Policy Centre [4]. The PCAT is designed to assess the extent and quality of primary care services in provider settings identified by consumers as their predominant source of health care.

The PCAT is based on a theoretical model of primary care formulated by Starfield [5], which incorporates the following four essential primary care attributes: first-contact accessibility and use, continuity, comprehensiveness, and coordination, as well as family-centeredness, community orientation, and cultural competence.

The development of the PCAT began with the Children and Adolescents version, which was initially validated in the United States by Cassady et al. (2000) [6]. This was followed by the development and validation of the Adult Edition and the Adult Short Form, with further validation and refinement documented by Shi et al. (2001) [7].

The PCAT has since been translated, validated, and utilized across many countries, including Canada, Brazil, Malta, Spain, South Korea, Japan, China, Hong Kong, Taiwan, Tibet, Vietnam, Malawi, Uganda, and South Africa [4–8]. As noted by Rocha and colleagues [8], while many tools assess specific dimensions of primary care quality, there are few like the PCAT that enable a comprehensive evaluation of primary care from the population perspective. Additionally, the Consumer/Client Surveys are designed for self-administration and require only a high school reading level, making the tool broadly accessible to the general population [7].

To date, there have been no studies investigating psychometric properties of the PCAT when applied in Australia. General practice plays a central role in Australia’s primary healthcare system, with general practitioners (GPs) typically serving as the first point of contact in the health system for an individual seeking health care [9]. While there are several tools which have been validated for measuring Australian patient experiences in general practice settings, such as the Doctors’ Interpersonal Skills Questionnaire (DISQ) [10], the Practice Accreditation and Improvement Survey (PAIS) [11] and the Patient Enablement and Satisfaction Survey (PESS) [12], these tools are not designed to provide a holistic evaluation of primary care. In contrast, the PCAT is a multidimensional scale developed to assess several key structural and process features of primary care, including four core (first contact care, person-focused care over time, comprehensiveness, and coordination) and three ancillary (family orientation, community orientation, and cultural competence) broad theoretical concepts of primary care [4,6].

Investigating the psychometric properties of the PCAT in Australia presents an opportunity to provide a comprehensive measure of patient experiences with primary care, facilitating international comparisons. Undertaking comparative analyses can offer valuable insights into the relative strengths and weaknesses of primary care across different countries. This study therefore aimed to evaluate the psychometric properties of the 28-item abridged version of the adult patient survey, referred to as the Primary Care Assessment Tool Short Form (PCAT-S), among patients aged 18–65 years with a chronic illness or aged over 65 years, in general practice settings in Australia. The aim of this study was to examine the internal consistency and factor structure of the PCAT-S.

Materials and methods

Participants and procedures

We used data from a clustered two-arm Randomised Control Trial (RCT) which ran from 1 August 2018 to 31 July 2019, evaluating an intervention involving patient enrolment and a funding model for higher-risk patients in metropolitan, regional, and rural Australia [13]. Details of patient recruitment and survey administration have been published elsewhere [13–15]. Briefly, the RCT recruited 774 adult participants from 34 general practices. Recruitment of practices was between 1 April 2018 and 31 August 2018. Recruitment of eligible patients of consenting practices was between 1 May 2018 and 31 December 2018. To be eligible, adult participants needed to have attended the practice three or more times in the last 2 years, and be either: (i) aged 18–65 years with a chronic condition (such as chronic obstructive pulmonary disease, diabetes, ischaemic heart disease, cardiac failure, or asthma), or (ii) aged over 65 years.

Ethics approval was provided by the University of Wollongong Human Research Ethics Committee, Monash University and the University of Tasmania (2017/417). The trial was registered on the Australian New Zealand Clinical Trials Registry (ACTRN12618000105246). All participants provided written informed consent.

The PCAT-S was included in a broader questionnaire administered to the study participants at trial entry and trial completion, either online or through paper-based self-completion or by telephone interview. The questionnaire also included questions on the patient’s socio-demographic background, health service utilisation within the past 12 months, and overall health-related quality of life measured using the EQ-5D-5L instrument [16,17]. To identify each participant’s primary care provider (PCP), the survey employed the same three questions and algorithm as the original PCAT: (i) Is there a doctor to whom you usually go if you are sick or need advice about your health? (usual source), (ii) Is there a doctor who knows you best as a person? (knows best) and (iii) Is there a doctor who is most responsible for your health care? (most responsible) [7]. Similar to the original PCAT, participants needed to have answered ‘yes’ to the question ‘Have you ever had a visit to any kind of specialist or special service (like a surgeon or kidney specialist)’ to be eligible to respond to items in the Coordination subscale. A small number of adjustments were made to the Adult PCAT-S provided by tool developers and are listed in Supplementary S1 Table. Note the alphabetical ordering of the subscales from the original PCAT-S has not been retained due to the exclusion of two subscales—Coordination (Information Systems) and Comprehensiveness (Services Available) (see Supplementary S1 Table).

Data analysis

This analysis focused only on the 28 items used to assess primary care attributes, organised into eight subscales: First contact – Utilization, First contact – Access, Ongoing Care, Coordination, Comprehensiveness (services provided), Family-centeredness, Community Orientation and Culturally Competent. Each subscale comprises 2–5 items. Response options for each item are measured on a 4-point Likert-type scale (1 = definitely not; 2 = probably not; 3 = probably yes; and 4 = definitely yes), with an additional response option ‘don’t know/can’t remember’, which were treated as missing data. All items are worded in a positive direction whereby higher scores equate to better experiences.

For this analysis, we used data from 715 adult participants who responded to any item/s of the PCAT-S at entry into the trial. Baseline responses from participants in both control and intervention arms were used for assessing factor structure and internal validity of the PCAT-S.

We report patient socio-demographic characteristics as well as an assessment of the data quality (proportion of missing data and ‘don’t know/can’t remember’ responses). Large ceiling or floor effects were defined as items where > 20% of respondents scored the highest (ceiling) or lowest (floor) possible score.

All analyses used R version 4.3.1 [18]. We used the package psych version 2.4.3 [19] to conduct sample adequacy tests, exploratory factor analyses and calculation of Cronbach’s alpha, and lavaan version 0.6.16 [20].

(i). Factor structure

Factor analysis examines the relationships between survey items to determine if responses from different subsets of items relate more closely to each other than to other subsets [21]. Confirmatory factor analysis (CFA) is used to confirm an existing theoretical model, or if the data do not fit the theoretical model, exploratory factor analysis (EFA) can be used to identify plausible underlying constructs for a set of items [21]. In the original PCAT validation study (conducted in two distinct subpopulations in the United States) Family-Centeredness emerged as a distinct factor only in the predominantly white, higher-income sample, while Cultural Competence emerged as a distinct factor only in the predominantly non-white, lower-income sample [7]. In several country-specific validation studies, factor analytic models did not exactly support the theoretical factor structure [22–26]. These examples highlight the need to validate the PCAT’s factor structure for different health system contexts.

We conducted the Kaiser-Meyer-Olkin (KMO) statistic and Bartlett’s Test for Sphericity to evaluate whether our sample was appropriate for carrying out factor analysis. We considered a KMO > 0.60 and significance of Bartlett’s test of sphericity value (p < 0.05) to indicate suitability [27,28]. Multicollinearity risk was assessed by visualising correlation matrix and examining for small determinant of the correlation matrix (< 0.00001) – results are and is presented in S2 Table [29].

Factor structure was examined using confirmatory factor analysis (CFA) with a diagonally weighted least squares (DWLS) estimator, as is appropriate for ordinal data [30]. The 28 items of the PCAT-S were specified to load onto the eight factors as per the original PCAT, with each item loading only on its respective factor. We determined model fit using standard criterion and accepted benchmarks: Comparative Fit Index (CFI ≥ 0.90), Standardized Root Mean square Residual (SRMR <0.08), and Root Mean Square Error of Approximation (RMSEA ≤0.06) [31–33]. We reported scaled statistics, which corrects for the sample statistics of latent response variables being estimated with less precision than for observed variables [34]. We considered factor loadings of above 0.5 as acceptable.

Since the data did not fit the hypothesised CFA model [7], additional exploratory factor analysis was conducted. When a hypothesized CFA model does not adequately fit the data, it is acceptable practice to use EFA to explore the underlying factor structure and identify alternative item loadings or configurations [35,36]. To confirm the number of factors, we examined a Cattell’s scree plot of eigenvalues. Response data were treated as continuous. We first used an oblique (oblimin) factor rotation method which assumes factors are non-independent. We assessed the correlation matrix describing relationships between factors to determine if correlations were low (defined as r < |0.30|) (S3–S5 Tables). Since we found most correlations to be low, we then used an orthogonal (varimax) factor rotation method which assumes factors are independent and not correlated. Items were considered loaded onto a factor if the factor loading was > 0.40 for that factor and <0.40 for all other factors [37].

Both exploratory and confirmatory factor analysis require samples with no missing data, meaning data could only be used from participants who responded to all 28 items with a valid response (either ‘1 = definitely not’, ‘2 = probably not’, ‘3 = probably yes’, or ‘4 = definitely yes’). These are called ‘complete observations’. Only 180 (of 715) participants had complete observations, which was too small for a meaningful factor analysis. Consequently, we used two data imputation methods to handle ‘don’t know/can’t remember’ responses and conducted separate analyses for each method. The first method, recommended by PCAT developers, involved recoding ‘don’t know/can’t remember’ responses to ‘2 (probably not)’ for items within a subscale where participants had valid responses to at least 50% of the items in that subscale. This method (applicable for all subscales excluding Comprehensiveness) interprets a patient’s uncertainty about an item as indicating a negative perception of the service [4]. Using this ‘developer-recommended’ imputation method, after excluding individuals with any missing responses, there were 373 observations available for factor analysis.

The second method involved recoding all ‘don’t know/can’t remember’ responses to a neutral value, regardless of how many items within the subscale had valid responses— a method used in other PCAT studies [8,22,24,38]. Using this ‘neutral value’ imputation method, after excluding individuals with any missing response, there were 606 observations available for factor analysis.

(ii). Internal consistency

Internal consistency evaluates the extent to which all the items in a test measure the same concept or construct [39]. In cultural adaptations of the PCAT, internal consistency was generally reported to be in an acceptable range [38,40,41]. However, in the original PCAT validation study, a lower than acceptable score was observed only for one subscale, First contact – Utilization (α = 0.64) [7]. In a subsequent Canadian study, the PCAT-S was found to have lower than acceptable internal consistency scores for First contact – Utilization (α = 0.68) and Community Orientation (α = 0.65), but acceptable scores for the remaining subscales (range 0.72–0.76) [26].

The internal consistency of each subscale of the PCAT-S was evaluated using Cronbach’s alpha. We considered a Cronbach’s alpha of >0.70 to indicate a minimally reliable subscale. Missing values were handled through pairwise deletion.

Results

Patient and health service utilisation characteristics of all 715 respondents who responded to any item are presented in Table 1. Most participants were female (60%), born in Australia (83%) and spoke English at home (99%). The mean age of the sample was 66.9 years. Around 50% resided in a higher socioeconomic area, and most participants owned or mortgaged their own home (82%). Over half (62%) of participants were retirees, and just under a quarter (23.8%) were employed in some form. Over one-quarter of participants (26.9%) held a vocational or university qualification. More than half of the participants had private health care coverage for the full 12 months prior to completing the survey (65.5%), and the majority (76.8%) had a relationship with their GP or practice for over five years.

Download:

Table 1. Demographic and health characteristics of total study participants (N = 715).

https://doi.org/10.1371/journal.pone.0341250.t001

Data quality

We present a summary of participant responses in Table 2. Overall, the item response rate was high. With the exception of items in the Coordination subscale, the proportion of missing responses was < 2% for each item. Most participants (n = 651, 91%) reported having visited a specialist or special service and were eligible to provide responses to items in the Coordination subscale. The proportions of the response category ‘not sure/don’t remember’ ranged widely from 0.1 to 37.8%. Particularly high proportions were found in all items of the Community Orientation subscale (I1, I2, and I3), as well as the last two items in the First Contact – Access subscale (D3 and D4), with over 20% of responses falling into this category. All subscales demonstrated the full range of possible scores. Based on the standard 20% criterion, 5 of the 28 items showed large floor effects and 23 items showed large ceiling effects. Additionally, very high ceiling effects (>80%) were noted for the two first-contact utilization items (C1 and C2) and for item J1.

Download:

Table 2. Descriptive statistics of PCAT-S items from sample (N = 715).

https://doi.org/10.1371/journal.pone.0341250.t002

Factor structure

Factor analysis was conducted on all 28 items. Three samples were used for factor analysis: (i) a sample of 180 participants with no imputation, (ii) a sample of 373 participants with developer-recommended imputation, and (iii) 606 participants with neutral value imputation.

(i). Sample with no imputation

Bartlett’s test of sphericity was significant (χ2 (378) = 1875.635, p < 0.001), indicating that it was appropriate to use the factor analytic model on this set of data. The Kaiser-Meyer-Olkin measure of sampling adequacy indicated that the strength of the relationships among variables was high (KMO = 0.77); thus, it was acceptable to proceed with the analysis. The determinant of the correlation matrix was close to 0.0001 (S2 Table), suggesting potential multicollinearity in the non-imputed sample.

The CFA model did not fit the data well, indicated by a higher than acceptable SRMR (χ2 (322) 487.551, p < .001, CFI 0.92, RMSEA 0.05, SRMR 0.10). All items exhibited standardised loadings > 0.50, with the exception of E1 (loading = 0.49), see S6 Table. In addition, the variance-covariance matrix of the estimated parameters was not positive definite, as the smallest eigenvalue was found to be negative (eigen = –1.16e – 16). We interpreted this eigenvalue to be effectively zero, and likely negative due to computational limitations. A zero eigenvalue may indicate model misspecification, or be an indication of linear dependency (redundancy). Furthermore, item J1 had a standardized loading of greater than 1.0 (loading = 1.19) along with a negative residual variance estimate (−0.42), indicating an improper solution, or “Heywood case”. This suggests that the Culturally Competent latent variable explained more than 100% of the variance in item J1, which is not logically possible.

Since the data did not fit the hypothesized model, exploratory factor analysis was performed to further examine the factor structure of the PCAT-S. A scree plot of eigenvalues suggested eight factors should be retained. Varimax rotated factor loadings for an eight-factor solution are shown in S7 Table. The following deviations from the expected factor structure were observed: (i) item I1 loaded inadequately (loadings < 0.40) and items G1 and J1 demonstrated cross-loading, (ii) item E2 from the Ongoing Care subscale loaded onto a factor together with all four items from the First-Contact Accessibility subscale, and (iii) item G5 from the Comprehensiveness subscale loaded onto a factor together with the remaining items (E1, E3, and E4) from the Ongoing Care subscale.

(ii). Sample with developer-recommended imputation

The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.84, and Bartlett’s test of sphericity was significant (χ2 (378) = 2721.387, p < 0.001), indicating the sample was appropriate for factor analysis. The CFA model demonstrated poor fit, with lower than acceptable CFI (χ2 (322) 701.816, p < .001, CFI 0.89, RMSEA 0.06, SRMR 0.08). All items exhibited standardised loadings > 0.50, with the exception of D4, E1 and F2 (range 0.41–0.45, see S6 Table). Similar to the sample with no imputation, item J1 had a standardized loading of greater than 1.0 (loading = 1.24) along with a negative residual variance estimate (−0.54).

Results of exploratory factor analysis using varimax rotation with an eight factor solution are shown in S8 Table. Items loaded onto factors as expected with the following exceptions: (i) item F4 and I1 loaded inadequately (loadings < 0.40), (ii) item E2 from the Ongoing Care subscale loaded onto a factor together with all four items from the First-Contact Accessibility subscale, and (iii) item G5 from the Comprehensiveness subscale, and item J1 from the Culturally Competent subscale, loaded onto a factor together with the remaining items (E1, E3, and E4) from the Ongoing Care subscale.

(iii). Sample with neutral-value imputation

The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.85, and Bartlett’s test of sphericity was significant (χ2 (378) = 4171.491, p < 0.001), indicating the sample was appropriate for factor analysis. The confirmatory factor analysis model demonstrated acceptable fit (χ2 (322) 972.528, p < .001, CFI = 0.90, RMSEA 0.06, SRMR 0.07). All items exhibited standardised loadings > 0.50, with the exception of D3, D4, E1 and F2 (range 0.36–0.50, see S6 Table). Similar to the sample with no imputation, the smallest eigenvalue was effectively zero (eigen = 1.5 e – 17) and item J1 had a standardized loading of greater than 1.0 (loading = 1.13) along with a negative residual variance estimate (−0.28).

CFA fit indices fell within an acceptable range; however, two of the fit indices indicated only borderline fit, and we therefore still conducted an EFA to further investigate factor structure. We conducted an EFA as an additional exploratory step to assess whether the data might indicate a more appropriate factor structure, and to provide a comparison across the three imputation samples. Results of exploratory factor analysis using varimax rotation with an eight factor solution are shown in S9 Table. Items loaded onto factors as expected with the following exceptions: (i) items D3, E1, F2, I1 and all three items from the Family Centeredness subscale (H1, H2, and H3) loaded inadequately (loadings < 0.40), (ii) item E2 from the Ongoing Care subscale loaded onto a factor together with the remaining three items from the First-Contact Accessibility subscale (D1, D2 and D3), (iii) item F4 from the Coordination subscale loaded onto a factor together with three items from the Comprehensiveness subscale (G1, G4 and G5), while items G2 and G3 loaded onto a separate factor altogether, and (iv) item J1 from the Culturally Competent subscale, loaded onto a factor together with the remaining items (E1, E3, and E4) from the Ongoing Care subscale.

Internal consistency

Cronbach’s alpha showed acceptable internal consistency for First-Contact Utilization (2 items, α = 0.71), Ongoing Care (4 items, α = 0.72), and Comprehensiveness (5 items, α = 0.77). Internal consistency for the remaining subscales were weak (range 0.61–0.67). Results are presented in Table 3.

Download:

Table 3. Internal consistency.

https://doi.org/10.1371/journal.pone.0341250.t003

Discussion

This study examined the reliability and validity of the PCAT-S as a measure of patient experiences within general practices in Australia. Our findings were mixed, with only three subscales demonstrating good internal consistency: Comprehensiveness, Ongoing Care and First Contact – Utilization. The remaining subscales exhibited weak internal consistency. Examination of factor structure using CFA identified possible model misspecification. Subsequent exploratory factor analysis through EFA indicated that, generally, the items supported the hypothesized factor structure, although some deviations were observed. For each subscale, we discuss the implications of our findings and propose potential explanations. Readers should note that we did not conduct formal face validity assessments. However, we attempted to relate our findings to contextual factors, which we believe have provided useful insights for interpreting the results and potential areas for future research.

In the First Contact – Utilisation subscale (items C1–2), both items exhibited a large ceiling effect, with greater than 90% of participants selecting the highest response for both items. This may be because most participants had attended their GP or practice for over 5 years, hence would naturally be more inclined to choose their regular practice for checkups or new health issues. Items with large ceiling effects can compromise statistical assumptions and suggest problems with the instrument (limited range) or response bias (unbalanced sample). Despite this limitation, the subscale showed adequate factor loading in both CFA and EFA, and acceptable internal consistency.

For the First Contact – Access subscale (items D1–4), participants scored generally positively for the first three items (mean range 3.2–3.28) and negatively for the last item (mean 1.67). Interestingly, for this last item ‘When your PCP is closed and you get sick during the night, would someone from the practice see you that night?’, almost 30% of participants chose ‘Don’t know/ don’t remember’. This may be due to the fact that participants would only know if they had experienced such an event. For items D3 and D4, factor loadings varied across EFA and CFA, and across samples, with some factor solutions showing lower than acceptable loadings. Coupled with weak internal consistency (α = 0.61) for this subscale, these results suggest items, particular D3 and D4, may need revision. These two items specifically address accessibility to after-hours care, which in Australia is often not provided by a patient’s regular GP but by services such as extended hours GP services, medical deputizing services (MDS), nurse-led walk-in clinics, or public emergency departments [42]. This suggests a need to reconsider these items to better capture the reality of after-hours care accessibility in the Australian healthcare context which now includes urgent care clinics [43].

Across all three imputation method samples, EFA of the Ongoing Care subscale (E1–4), consistently showed item E2 ‘If you have a question, can you call and talk to the doctor or nurse who knows you best?’ loaded onto the First Contact – Access subscale. This was not entirely unexpected as this question also relates to accessibility, as it implies a need to make a phone call from home. Construct overlap between First-Contact Accessibility and Ongoing Care has also been observed in other PCAT studies [23,26,44,45], with some researchers collapsing these into a single subscale [23,46]. Despite these issues with the factor analysis, the Ongoing Care subscale showed acceptable internal consistency (α = 0.72), suggesting items generally measure the same construct.

For the Coordination subscale (items F1–4), CFA and EFA generally found items F2 ‘Did your PCP (or someone working with your PCP) help you make the appointment?’ and F4 ‘After you went to the specialist or special service, did your PCP talk with you about what happened at the visit? loaded inadequately. The subscale exhibited weak internal consistency (α = 0.61), indicating potential issues with items aligning to a single construct. These findings suggest a need to further examine the wording or relevance of items F2 and F4 in particular.

The Comprehensiveness subscale performed relatively well, showing the highest internal consistency of all subscales (α = 0.77) and adequate factor loadings across all three CFA. However, EFA found deviations from the hypothesized data structure. In the no imputation and developer-recommended imputation samples, item G5 ‘Checking on, and discussing the medications you are taking’ loaded on the Ongoing Care subscale. G5 differs from other items in this subscale, as it relates to clinical quality (tasks that should be done) rather than comprehensive care, which encompasses a broader range of services (tasks that could be done). Medication management likely becomes more relevant as continuity and the patient-provider relationship improve [47]. In the neutral-value imputation sample, items G2 (home safety) and G3 (family conflicts) loaded onto another factor entirely. In the Australian context, discussions around safety and family conflict may not be common in general practice settings. In different country versions of the PCAT, items in the Comprehensiveness subscale differ widely, and are selected based on population demographics (for example, in our study which focuses on higher-risk patients, items from the original scale covering childhood immunisations were excluded) [40]. In light of this, Haggerty et al (2011) [48] argued that providers, rather than patients, are best positioned to assess this particular subscale, primarily because providers plan care for a broad range of patients, while patients can only assess services based on their personal experience.

Family-centeredness (items H1–3), showed weak internal consistency and mixed results for factor analysis, with neither item loading adequately in EFA of the neutral-value sample, and adequate loading in the other non-imputation and developer-recommended samples. Based on these findings it is evident that the imputation approach used influenced the factor solution, making subsequent conclusions about the validity of this subscale uncertain. Analytical issues arising from lack of clarity on how to score ‘Don’t know/don’t remember’ responses are a known problem with the PCAT. Researchers have used various methods, including neutral value [8,22], median value [38], or a mix of approaches [23,24,26]. This issue has led to calls for further work and international collaborations to refine response options to better reflect patient experiences in different contexts [23,26].

All Community Orientation items had high ‘Don’t know/don’t remember’ responses, suggesting that patients may not be the best information source for this subscale. Over one third of participants responded ‘Don’t know/don’t remember’ for item I1 ‘Does anyone at your PCP’s office ever make home visits?’ and item I2 ‘Does your PCP know about the important health problems of your neighbourhood?’. This could be because participants never found the need to inquire about whether their GP conducts home visits or is aware of community issues. Notably, in Australia, home visits have been on the decline [49,50]. Additionally, Australian GPs primarily focus on providing individual healthcare, while community healthcare is predominantly managed by public health agencies and community health centres [51]. Furthermore, in the EFA, item I1 showed near acceptable loadings (range 0.30–0.40) in both Factor 2 (First-Contact Accessibility) and Factor 7 (Community Orientation) across all three samples. This finding aligns with previous findings by Haggerty et al (2011) [26], which found a construct overlap between First-Contact Accessibility and Community Orientation.

CFA and EFA were consistent across the three imputation methods for the Culturally Competent (items J1–3) subscale. As mentioned earlier, in the original PCAT validation study Culturally Competent emerged as a distinct factor only in a predominantly non-white sample [7]. Items J1–J3 may be more challenging to interpret in populations with less diversity. In our study, we found item J1 ‘Would you recommend your PCP to a friend or relative?’ showed a high ceiling effect, with 85% of participants selecting the highest option. Again, this may be because of this sample of participants already having a long-term relationship with their GP (over 75% of participants had a relationship of 5 years or more) and selection bias (participants agreeing to participate in the study may have been more inclined to have positive experiences with the practice). Hence, participants would be more likely to recommend their regular practice to others. All EFAs consistently found item J1 as loading onto factor 3 (Ongoing Care). This is expected, as both liking and recommending a practice are indicative of good relational continuity.

Limitations relating to the sample of participants have been discussed previously [13,14]. Specific limitations regarding this analysis of the PCAT-S include the following: the results may not be generalizable to the wider Australian population due to the specific sample used. The sample consisted of individuals who (i) generally had a long-term relationship with their GP or practice, and (ii) were adult higher-risk patients due to chronic disease and/or age. Notably, our sample were predominantly older adults with a mean age of over 60 years. These factors may have led to higher scores in areas such as Ongoing Care, as participants likely had more established relationships with their healthcare providers. Older patients or patients with chronic disease also tend to have more complex or frequent interactions with healthcare services, which may result in different experiences with primary care compared to younger, healthier individuals who have fewer or less involved healthcare engagements. In addition, nearly all items exhibited floor or ceiling effects, likely reflecting the homogeneity of our sample.

For factor analysis, we excluded subjects with ‘Don’t know/can’t remember’ responses, which reduced statistical power and may have produced a biased sample. To address this, we conducted analyses with imputed samples, which altered our overall conclusions for some factors. Despite these limitations, this study provides a solid foundation for further work on the PCAT-S in Australia. Future research could explore the correlation of PCAT-S subscales with other validated Australian instruments that measure theoretically related constructs, such as the PAIS [11] or PESS [12].

Our CFA identified notable limitations, including zero eigenvalues and Heywood cases, which undermine the credibility of the results. A zero eigenvalue may indicate model misspecification, or be an indication of linear dependency or redundancy, and Heywood cases imply that unique factors have negative error variances. Negative error variances often indicate model misspecification, but there can be other reasons, such as random sampling fluctuations, small sample size, too few items for a given factor, or too many common factors to provide stable estimates with the available data. In Berra et al. (2011) [45], Heywood cases were observed in a factor solution examining the original factor structure, which included six original subscales. As is typical when model misspecification is suspected, the authors further examined several factor solutions and found a reduced five-factor model showed no Heywood cases and adequate fit. Exploring factor solutions with varying numbers of factors was beyond the scope of this study and would be more appropriate for face and content validity assessments, which involve item selection and refinement. We also acknowledge that this study does not assess convergent and discriminant validity, which would have been helpful for clarifying factor analyses. Instead, we further examined factor structure via EFA, which provides insights into the underlying factor structure without the constraints of a predefined model – this can be useful for understanding the causes of model misspecification observed in CFA and provides a more comprehensive understanding of the factor structure.

One subscale (First-Contact Utilisation) included only two items – while this subscale contained three items in the original PCAT-S, one item was omitted for the purpose of the larger RCT from which our study data is drawn from. We acknowledge that while we retained it to reflect how the instrument was used in practice, this may have contributed to poorer model fit and may have affected the overall CFA model.

We note that in studies which performed CFA for psychometric validation of country-specific adaptations of the PCAT, best fitting models demonstrated goodness-of-fit statistics well within acceptable ranges [23,44,45]. Such adaptations of the PCAT involve modifications to survey questions, rearranging of items into different subscales or the addition or removal of items [8,24,38,40,45,52]. In contrast, the version used in our study was pragmatically modified for use in a larger RCT, and these changes were not informed by formal psychometric procedures such as content validation. Our findings that certain subscales require revision to improve reliability and validity, suggest a need to develop an Australian version of the PCAT-S, which is consistent with international evidence emphasising the importance of adapting the PCAT to local healthcare contexts.

Results from our study can be compared to Canadian research on the PCAT-S [26,53]. Like our study, the instrument was evaluated without prior refinement; additionally, the study was conducted in Canada, which shares a similar healthcare context to Australia compared to other validation studies conducted in Asia, Africa and South America [22–25,38,40,44,45,52,54–56]. Internal consistency scores were similar to our study for several subscales: First-Contact – Utilization (0.68 [Canadian study] vs. 0.71 [our study]), Ongoing Care (0.73 vs. 0.72), Comprehensiveness (0.72 vs. 0.77) and Community Orientation (0.65 vs. 0.65) [26], however, the Canadian study reported higher scores for First contact – Access (0.72 vs. 0.61) and Coordination (0.76 vs. 0.61) [26]. These differences may reflect variations in healthcare delivery between Australian general practices and Canada – for example, out-of-hours access may be less common in Australia.

At present, there are no validated instruments specifically designed for evaluating primary care patient experiences in Australia (except for PAIS and PESS, which focus on different domains [57]), reinforcing the PCAT’s value as a tool for this purpose. Our initial validation of the PCAT-S in an Australian general practice sample showed mixed psychometric performance. While some subscales demonstrated acceptable reliability and model fit, others fell below recommended reliability thresholds. Hence survey results should be interpreted with nuance, and consideration of item relevance and interpretation in the Australian context.

Next steps should include the development of a country-specific Australian version of the PCAT. As in other countries, this process should be guided by formal face and content validity assessments, expert consultation, and engagement with patients and providers. Item refinement should include reviewing relevance of items which may not be relevant to how general practice is currently delivered in Australia (such as after-hours care, home visits, general practice involvement in community health services, discussion of home safety, and facility-related conflict). Consideration should also be given to how provider and specialist services operate within the Australian healthcare system. Future development work should draw on data collected specifically for psychometric purposes and aim for broader demographic representation. More advanced psychometric testing, including assessments of convergent and discriminant validity and exploration of alternative factor structures, would further strengthen the instrument.

A version of the PCAT-S validated for the Australian context would provide a robust tool for assessing primary care performance and facilitate meaningful national and international comparisons.

Supporting information

S1 Table. Adjustments to the PCAT-S for the EQuIP-GP trial.

https://doi.org/10.1371/journal.pone.0341250.s001

(DOCX)

S2 Table. Multicollinearity statistics.

The determinant of the correlation matrix that is smaller than 0.00001 suggests an issue with multicollinearity.

https://doi.org/10.1371/journal.pone.0341250.s002

(DOCX)

S3 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, complete observations only (n = 180).

Correlations where ) are indicated by an asterisk.

https://doi.org/10.1371/journal.pone.0341250.s003

(DOCX)

S4 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, developer-recommended imputation (n = 373).

Correlations where ) are indicated by an asterisk.

https://doi.org/10.1371/journal.pone.0341250.s004

(DOCX)

S5 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, neutral-value imputation (n = 606).

Correlations where ) are indicated by an asterisk.

https://doi.org/10.1371/journal.pone.0341250.s005

(DOCX)

S6 Table. Standardised factor loadings from CFA models by imputation method.

Confirmatory Factor Analysis (CFA). Problematic items are indicated by an asterisk. Items were considered to load onto a specific factor if the standardised factor loading was > 0.50. Heywood cases occur when the loading is greater than 1.0. Items are presented using wording from the administered survey.

https://doi.org/10.1371/journal.pone.0341250.s006

(DOCX)

S7 Table. Factor loadings and extracted factors using complete observations only (n = 180).

Exploratory factor analysis (EFA) using varimax rotation. The highest loadings for each item are bolded. Items were considered to load onto a specific factor if the factor loading was > 0.40 for that factor and <0.40 for all other factors. Items are presented using wording from the administered survey.

https://doi.org/10.1371/journal.pone.0341250.s007

(DOCX)

S8 Table. Factor loadings and extracted factors, developer-recommended imputation (n = 373).

Exploratory factor analysis (EFA) using varimax rotation. The highest loadings for each item are bolded. Items were considered to load onto a specific factor if the factor loading was > 0.40 for that factor and <0.40 for all other factors. Items are presented using wording from the administered survey.

https://doi.org/10.1371/journal.pone.0341250.s008

(DOCX)

S9 Table. Factor loadings and extracted factors using neutral-value imputation (n = 606).

Exploratory factor analysis (EFA) using varimax rotation. The highest loadings for each item are bolded. Items were considered to load onto a specific factor if the factor loading was > 0.40 for that factor and <0.40 for all other factors. Items are presented using wording from the administered survey.

https://doi.org/10.1371/journal.pone.0341250.s009

(DOCX)

Acknowledgments

We would like to thank the Royal Australian College of General Practitioners (RACGP), the Australian Government Department of Health, as well as the participating general practices and patients involved in the larger EQuIP-GP project which provided the data for this study.

References

1. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83(3):457–502. pmid:16202000
- View Article
- PubMed/NCBI
- Google Scholar
2. van Weel C, Kidd MR. Why strengthening primary health care is essential to achieving universal health coverage. CMAJ. 2018;190(15):E463–6. pmid:29661815
- View Article
- PubMed/NCBI
- Google Scholar
3. Jamieson Gilmore K, Corazza I, Coletta L, Allin S. The uses of Patient Reported Experience Measures in health systems: A systematic narrative review. Health Policy. 2023;128:1–10. pmid:35934546
- View Article
- PubMed/NCBI
- Google Scholar
4. Johns Hopkins Primary Care Policy Center. Primary Care Assessment Tools. Primary Care Policy Center: John Hopkins Bloomberg School of Public Health. 2024 [Cited 2024 Sep 10]. Available from: https://publichealth.jhu.edu/johns-hopkins-primary-care-policy-center/primary-care-assessment-tools
5. Starfield B. Primary Care: Balancing Health Needs, Services, and Technology. online edn ed. New York, NY: Oxford Academic; 1998 31 Oct. 2023.
6. Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. Pediatrics. 2000;105(4 Pt 2):998–1003. pmid:10742362
- View Article
- PubMed/NCBI
- Google Scholar
7. Shi L, Xu J, Starfield B. Validating the Adult Primary Care Assessment Tool. The Journal of Family Practice. 2001;50(2):161.
- View Article
- Google Scholar
8. Rocha KB, Rodríguez-Sanz M, Pasarín MI, Berra S, Gotsens M, Borrell C. Assessment of primary care in health surveys: a population perspective. Eur J Public Health. 2012;22(1):14–9. pmid:21470974
- View Article
- PubMed/NCBI
- Google Scholar
9. Australian Institute of Health and Welfare. General practice, allied health and other primary care services. Canberra: Australian Institute of Health and Welfare. 2024 Mar 07 [Cited 2024 Sep 10]. Available from: https://www.aihw.gov.au/reports/primary-health-care/general-practice-allied-health-primary-care
10. Greco M, Cavanagh M, Brownlea A, McGovern J. Validation studies of the doctors’ interpersonal skills questionnaire. Educ Gen Pract. 1999;10:256–64.
- View Article
- Google Scholar
11. Greco M, Sweeney K, Brownlea A, McGovern J. The practice accreditation and improvement survey (PAIS). What patients think. Aust Fam Physician. 2001;30(11):1096–100. pmid:11759463
- View Article
- PubMed/NCBI
- Google Scholar
12. Desborough J, Banfield M, Parker R. A tool to evaluate patients’ experiences of nursing care in Australian general practice: development of the Patient Enablement and Satisfaction Survey. Aust J Prim Health. 2014;20(2):209–15. pmid:23469805
- View Article
- PubMed/NCBI
- Google Scholar
13. Bonney A, Russell G, Radford J, Zwar N, Mullan J, Batterham M, et al. Effectiveness of Quality Incentive Payments in General Practice (EQuIP-GP) cluster randomized trial: impact on patient-reported experience. Fam Pract. 2022;39(3):373–80. pmid:35640205
- View Article
- PubMed/NCBI
- Google Scholar
14. Peterson GM, Radford J, Russell G, Zwar N, Mullan J, Batterham M, et al. Cluster-randomised trial of the Effectiveness of Quality Incentive Payments in General Practice (EQuIP-GP): Prescribing of medicines outcomes. Res Social Adm Pharm. 2023;19(5):836–40. pmid:36754667
- View Article
- PubMed/NCBI
- Google Scholar
15. Peterson GM, Russell G, Radford JC, Zwar N, Mazza D, Eckermann S, et al. Effectiveness of quality incentive payments in general practice (EQuIP-GP): a study protocol for a cluster-randomised trial of an outcomes-based funding model in Australian general practice to improve patient care. BMC Health Serv Res. 2019;19(1):529. pmid:31357999
- View Article
- PubMed/NCBI
- Google Scholar
16. Brooks R. EuroQol: the current state of play. Health Policy. 1996;37(1):53–72. pmid:10158943
- View Article
- PubMed/NCBI
- Google Scholar
17. EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. pmid:10109801
- View Article
- PubMed/NCBI
- Google Scholar
18. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013.
19. Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research [R package version 2.4.3]. Northwestern University, Evanston, Illinois; 2023 [Cited 2024 Sep 10]. Available from: https://CRAN.R-project.org/package=psych
20. Rosseel Y. lavaan: AnRPackage for Structural Equation Modeling. J Stat Soft. 2012;48(2).
- View Article
- Google Scholar
21. Knekta E, Runyon C, Eddy S. One Size Doesn’t Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research. CBE Life Sci Educ. 2019;18(1):rm1. pmid:30821600
- View Article
- PubMed/NCBI
- Google Scholar
22. Lee JH, Choi Y-J, Sung NJ, Kim SY, Chung SH, Kim J, et al. Development of the Korean primary care assessment tool--measuring user experience: tests of data quality and measurement performance. Int J Qual Health Care. 2009;21(2):103–11. pmid:19286829
- View Article
- PubMed/NCBI
- Google Scholar
23. Wang W, Shi L, Yin A, Lai Y, Maitland E, Nicholas S. Development and validation of the Tibetan primary care assessment tool. Biomed Res Int. 2014;2014:308739. pmid:24967349
- View Article
- PubMed/NCBI
- Google Scholar
24. Yang H, Shi L, Lebrun LA, Zhou X, Liu J, Wang H. Development of the Chinese primary care assessment tool: data quality and measurement properties. Int J Qual Health Care. 2013;25(1):92–105. pmid:23175535
- View Article
- PubMed/NCBI
- Google Scholar
25. Mei J, Liang Y, Shi L, Zhao J, Wang Y, Kuang L. The Development and Validation of a Rapid Assessment Tool of Primary Care in China. Biomed Res Int. 2016;2016:6019603. pmid:26885509
- View Article
- PubMed/NCBI
- Google Scholar
26. Haggerty JL, Burge F, Beaulieu M-D, Pineault R, Beaulieu C, Lévesque J-F, et al. Validation of instruments to evaluate primary healthcare from the patient perspective: overview of the method. Healthc Policy. 2011;7(Spec Issue):31–46. pmid:23205034
- View Article
- PubMed/NCBI
- Google Scholar
27. Dziuban CD, Shirkey EC. When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin. 1974;81(6):358–61.
- View Article
- Google Scholar
28. Bartlett MS. The effect of standardization on a χ2 approximation in factor analysis. Biometrika. 1951;38(3–4):337–44.
- View Article
- Google Scholar
29. Field A. Discovering statistics using IBM SPSS Statistics. 5th ed. London, UK: SAGE; 2017.
30. Forero CG, Maydeu-Olivares A, Gallardo-Pujol D. Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation. Structural Equation Modeling: A Multidisciplinary Journal. 2009;16(4):625–41.
- View Article
- Google Scholar
31. DiStefano C, Hess B. Using Confirmatory Factor Analysis for Construct Validation: An Empirical Review. Journal of Psychoeducational Assessment. 2005;23(3):225–41.
- View Article
- Google Scholar
32. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238–46. pmid:2320703
- View Article
- PubMed/NCBI
- Google Scholar
33. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55.
- View Article
- Google Scholar
34. Wirth RJ, Edwards MC. Item factor analysis: current approaches and future directions. Psychol Methods. 2007;12(1):58–79. pmid:17402812
- View Article
- PubMed/NCBI
- Google Scholar
35. Flora DB, Flake JK. The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement. 2017;49(2):78–88.
- View Article
- Google Scholar
36. Ryan J, Curtis R, Olds T, Edney S, Vandelanotte C, Plotnikoff R, et al. Psychometric properties of the PERMA Profiler for measuring wellbeing in Australian adults. PLoS One. 2019;14(12):e0225932. pmid:31869336
- View Article
- PubMed/NCBI
- Google Scholar
37. Stevens J. Applied multivariate statistics for the social sciences. 4th ed. Mahwah, N.J: L. Erlbaum; 2002.
38. Aoki T, Inoue M, Nakayama T. Development and validation of the Japanese version of Primary Care Assessment Tool. Fam Pract. 2016;33(1):112–7. pmid:26546033
- View Article
- PubMed/NCBI
- Google Scholar
39. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5. pmid:28029643
- View Article
- PubMed/NCBI
- Google Scholar
40. Hoa NT, Tam NM, Peersman W, Derese A, Markuns JF. Development and validation of the Vietnamese primary care assessment tool. PLoS One. 2018;13(1):e0191181. pmid:29324851
- View Article
- PubMed/NCBI
- Google Scholar
41. Jeon K-Y. Cross-cultural adaptation of the US consumer form of the short Primary Care Assessment Tool (PCAT): the Korean consumer form of the short PCAT (KC PCAT) and the Korean standard form of the short PCAT (KS PCAT). Qual Prim Care. 2011;19(2):85–103. pmid:21575331
- View Article
- PubMed/NCBI
- Google Scholar
42. Barnes K, Ceramidas D, Douglas K. Why patients attend after-hours medical services: a cross-sectional survey of patients across the Australian Capital Territory. Aust J Prim Health. 2022;28(6):549–55. pmid:36042687
- View Article
- PubMed/NCBI
- Google Scholar
43. Australian Government Department of Health DaAC. About Medicare Urgent Care Clinics. Canberra: Department of Health. 2025 [Cited 2025 Nov 22]. Available from: https://www.health.gov.au/our-work/medicare-urgent-care-clinics/about-medicare-urgent-care-clinics?language=en
44. Kijima T, Matsushita A, Akai K, Hamano T, Takahashi S, Fujiwara K, et al. Patient satisfaction and loyalty in Japanese primary care: a cross-sectional study. BMC Health Serv Res. 2021;21(1):274. pmid:33766027
- View Article
- PubMed/NCBI
- Google Scholar
45. Berra S, Rocha KB, Rodríguez-Sanz M, Pasarín MI, Rajmil L, Borrell C, et al. Properties of a short questionnaire for assessing primary care experiences for children in a population survey. BMC Public Health. 2011;11:285. pmid:21554717
- View Article
- PubMed/NCBI
- Google Scholar
46. Mead N, Bower P, Roland M. The General Practice Assessment Questionnaire (GPAQ) - development and psychometric characteristics. BMC Fam Pract. 2008;9:13. pmid:18289385
- View Article
- PubMed/NCBI
- Google Scholar
47. Lampe D, Grosser J, Gensorowsky D, Witte J, Muth C, van den Akker M, et al. The Relationship of Continuity of Care, Polypharmacy and Medication Appropriateness: A Systematic Review of Observational Studies. Drugs Aging. 2023;40(6):473–97. pmid:36972012
- View Article
- PubMed/NCBI
- Google Scholar
48. Haggerty JL, Beaulieu M-D, Pineault R, Burge F, Lévesque J-F, Santor DA, et al. Comprehensiveness of care from the patient perspective: comparison of primary healthcare evaluation instruments. Healthc Policy. 2011;7(Spec Issue):154–66. pmid:23205042
- View Article
- PubMed/NCBI
- Google Scholar
49. Attwooll J. ‘We are heading for a crisis’: Can GP home visits be revived?: The Royal Australian College of General Practitioners (RACGP). 2023 Nov 03 [Cited 2024 Sep 10]. Available from: https://www1.racgp.org.au/newsgp/professional/we-are-heading-for-a-crisis-can-gp-home-visits-be
50. Joyce C, Piterman L. Trends in GP home visits. Aust Fam Physician. 2008;37(12):1039–42. pmid:19142281
- View Article
- PubMed/NCBI
- Google Scholar
51. Porter G, Blashki G, Grills N. General practice and public health: who is my patient? Aust Fam Physician. 2014;43(7):483–6. pmid:25006614
- View Article
- PubMed/NCBI
- Google Scholar
52. Bresick GF, Sayed A-R, Le Grange C, Bhagwan S, Manga N, Hellenberg D. Western Cape Primary Care Assessment Tool (PCAT) study: Measuring primary care organisation and performance in the Western Cape Province, South Africa (2013). Afr J Prim Health Care Fam Med. 2016;8(1):e1–12. pmid:27247157
- View Article
- PubMed/NCBI
- Google Scholar
53. Haggerty JL. Measurement of primary healthcare attributes from the patient perspective. Healthc Policy. 2011;7(Spec Issue):13–20. pmid:23205032
- View Article
- PubMed/NCBI
- Google Scholar
54. Besigye IK, Mash R. Adaptation and validation of the Ugandan Primary Care Assessment Tool. Afr J Prim Health Care Fam Med. 2023;15(1):e1–7. pmid:36744453
- View Article
- PubMed/NCBI
- Google Scholar
55. Shieh C-F. Assessing the Quality of Primary Health Care in a Remote County of Taiwan: ProQuest Dissertations Publishing; 2020.
56. Owolabi O, Zhang Z, Wei X, Yang N, Li H, Wong SYS, et al. Patients’ socioeconomic status and their evaluations of primary care in Hong Kong. BMC Health Serv Res. 2013;13:487. pmid:24274660
- View Article
- PubMed/NCBI
- Google Scholar
57. ZEST Health Stategies. Patient experience measurement in primary health care. Sydney: Australian Commission on Safety and Quality in Health Care. 2023 April [Cited 2024 Sep 10]. Available from: https://www.safetyandquality.gov.au/publications-and-resources/resource-library/literature-review-patient-experience-primary-health-care

[ref1] 1. Starfield B, Shi L, Macinko J. Contribution of primary care to health systems and health. Milbank Q. 2005;83(3):457–502. pmid:16202000
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. van Weel C, Kidd MR. Why strengthening primary health care is essential to achieving universal health coverage. CMAJ. 2018;190(15):E463–6. pmid:29661815
View Article
PubMed/NCBI
Google Scholar

[6] View Article

[7] PubMed/NCBI

[8] Google Scholar

[ref3] 3. Jamieson Gilmore K, Corazza I, Coletta L, Allin S. The uses of Patient Reported Experience Measures in health systems: A systematic narrative review. Health Policy. 2023;128:1–10. pmid:35934546
View Article
PubMed/NCBI
Google Scholar

[10] View Article

[11] PubMed/NCBI

[12] Google Scholar

[ref4] 4. Johns Hopkins Primary Care Policy Center. Primary Care Assessment Tools. Primary Care Policy Center: John Hopkins Bloomberg School of Public Health. 2024 [Cited 2024 Sep 10]. Available from: https://publichealth.jhu.edu/johns-hopkins-primary-care-policy-center/primary-care-assessment-tools

[ref5] 5. Starfield B. Primary Care: Balancing Health Needs, Services, and Technology. online edn ed. New York, NY: Oxford Academic; 1998 31 Oct. 2023.

[ref6] 6. Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. Pediatrics. 2000;105(4 Pt 2):998–1003. pmid:10742362
View Article
PubMed/NCBI
Google Scholar

[16] View Article

[17] PubMed/NCBI

[18] Google Scholar

[ref7] 7. Shi L, Xu J, Starfield B. Validating the Adult Primary Care Assessment Tool. The Journal of Family Practice. 2001;50(2):161.
View Article
Google Scholar

[20] View Article

[21] Google Scholar

[ref8] 8. Rocha KB, Rodríguez-Sanz M, Pasarín MI, Berra S, Gotsens M, Borrell C. Assessment of primary care in health surveys: a population perspective. Eur J Public Health. 2012;22(1):14–9. pmid:21470974
View Article
PubMed/NCBI
Google Scholar

[23] View Article

[24] PubMed/NCBI

[25] Google Scholar

[ref9] 9. Australian Institute of Health and Welfare. General practice, allied health and other primary care services. Canberra: Australian Institute of Health and Welfare. 2024 Mar 07 [Cited 2024 Sep 10]. Available from: https://www.aihw.gov.au/reports/primary-health-care/general-practice-allied-health-primary-care

[ref10] 10. Greco M, Cavanagh M, Brownlea A, McGovern J. Validation studies of the doctors’ interpersonal skills questionnaire. Educ Gen Pract. 1999;10:256–64.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref11] 11. Greco M, Sweeney K, Brownlea A, McGovern J. The practice accreditation and improvement survey (PAIS). What patients think. Aust Fam Physician. 2001;30(11):1096–100. pmid:11759463
View Article
PubMed/NCBI
Google Scholar

[31] View Article

[32] PubMed/NCBI

[33] Google Scholar

[ref12] 12. Desborough J, Banfield M, Parker R. A tool to evaluate patients’ experiences of nursing care in Australian general practice: development of the Patient Enablement and Satisfaction Survey. Aust J Prim Health. 2014;20(2):209–15. pmid:23469805
View Article
PubMed/NCBI
Google Scholar

[35] View Article

[36] PubMed/NCBI

[37] Google Scholar

[ref13] 13. Bonney A, Russell G, Radford J, Zwar N, Mullan J, Batterham M, et al. Effectiveness of Quality Incentive Payments in General Practice (EQuIP-GP) cluster randomized trial: impact on patient-reported experience. Fam Pract. 2022;39(3):373–80. pmid:35640205
View Article
PubMed/NCBI
Google Scholar

[39] View Article

[40] PubMed/NCBI

[41] Google Scholar

[ref14] 14. Peterson GM, Radford J, Russell G, Zwar N, Mullan J, Batterham M, et al. Cluster-randomised trial of the Effectiveness of Quality Incentive Payments in General Practice (EQuIP-GP): Prescribing of medicines outcomes. Res Social Adm Pharm. 2023;19(5):836–40. pmid:36754667
View Article
PubMed/NCBI
Google Scholar

[43] View Article

[44] PubMed/NCBI

[45] Google Scholar

[ref15] 15. Peterson GM, Russell G, Radford JC, Zwar N, Mazza D, Eckermann S, et al. Effectiveness of quality incentive payments in general practice (EQuIP-GP): a study protocol for a cluster-randomised trial of an outcomes-based funding model in Australian general practice to improve patient care. BMC Health Serv Res. 2019;19(1):529. pmid:31357999
View Article
PubMed/NCBI
Google Scholar

[47] View Article

[48] PubMed/NCBI

[49] Google Scholar

[ref16] 16. Brooks R. EuroQol: the current state of play. Health Policy. 1996;37(1):53–72. pmid:10158943
View Article
PubMed/NCBI
Google Scholar

[51] View Article

[52] PubMed/NCBI

[53] Google Scholar

[ref17] 17. EuroQol Group. EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. pmid:10109801
View Article
PubMed/NCBI
Google Scholar

[55] View Article

[56] PubMed/NCBI

[57] Google Scholar

[ref18] 18. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013.

[ref19] 19. Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research [R package version 2.4.3]. Northwestern University, Evanston, Illinois; 2023 [Cited 2024 Sep 10]. Available from: https://CRAN.R-project.org/package=psych

[ref20] 20. Rosseel Y. lavaan: AnRPackage for Structural Equation Modeling. J Stat Soft. 2012;48(2).
View Article
Google Scholar

[61] View Article

[62] Google Scholar

[ref21] 21. Knekta E, Runyon C, Eddy S. One Size Doesn’t Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research. CBE Life Sci Educ. 2019;18(1):rm1. pmid:30821600
View Article
PubMed/NCBI
Google Scholar

[64] View Article

[65] PubMed/NCBI

[66] Google Scholar

[ref22] 22. Lee JH, Choi Y-J, Sung NJ, Kim SY, Chung SH, Kim J, et al. Development of the Korean primary care assessment tool--measuring user experience: tests of data quality and measurement performance. Int J Qual Health Care. 2009;21(2):103–11. pmid:19286829
View Article
PubMed/NCBI
Google Scholar

[68] View Article

[69] PubMed/NCBI

[70] Google Scholar

[ref23] 23. Wang W, Shi L, Yin A, Lai Y, Maitland E, Nicholas S. Development and validation of the Tibetan primary care assessment tool. Biomed Res Int. 2014;2014:308739. pmid:24967349
View Article
PubMed/NCBI
Google Scholar

[72] View Article

[73] PubMed/NCBI

[74] Google Scholar

[ref24] 24. Yang H, Shi L, Lebrun LA, Zhou X, Liu J, Wang H. Development of the Chinese primary care assessment tool: data quality and measurement properties. Int J Qual Health Care. 2013;25(1):92–105. pmid:23175535
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref25] 25. Mei J, Liang Y, Shi L, Zhao J, Wang Y, Kuang L. The Development and Validation of a Rapid Assessment Tool of Primary Care in China. Biomed Res Int. 2016;2016:6019603. pmid:26885509
View Article
PubMed/NCBI
Google Scholar

[80] View Article

[81] PubMed/NCBI

[82] Google Scholar

[ref26] 26. Haggerty JL, Burge F, Beaulieu M-D, Pineault R, Beaulieu C, Lévesque J-F, et al. Validation of instruments to evaluate primary healthcare from the patient perspective: overview of the method. Healthc Policy. 2011;7(Spec Issue):31–46. pmid:23205034
View Article
PubMed/NCBI
Google Scholar

[84] View Article

[85] PubMed/NCBI

[86] Google Scholar

[ref27] 27. Dziuban CD, Shirkey EC. When is a correlation matrix appropriate for factor analysis? Some decision rules. Psychological Bulletin. 1974;81(6):358–61.
View Article
Google Scholar

[88] View Article

[89] Google Scholar

[ref28] 28. Bartlett MS. The effect of standardization on a χ2 approximation in factor analysis. Biometrika. 1951;38(3–4):337–44.
View Article
Google Scholar

[91] View Article

[92] Google Scholar

[ref29] 29. Field A. Discovering statistics using IBM SPSS Statistics. 5th ed. London, UK: SAGE; 2017.

[ref30] 30. Forero CG, Maydeu-Olivares A, Gallardo-Pujol D. Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation. Structural Equation Modeling: A Multidisciplinary Journal. 2009;16(4):625–41.
View Article
Google Scholar

[95] View Article

[96] Google Scholar

[ref31] 31. DiStefano C, Hess B. Using Confirmatory Factor Analysis for Construct Validation: An Empirical Review. Journal of Psychoeducational Assessment. 2005;23(3):225–41.
View Article
Google Scholar

[98] View Article

[99] Google Scholar

[ref32] 32. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238–46. pmid:2320703
View Article
PubMed/NCBI
Google Scholar

[101] View Article

[102] PubMed/NCBI

[103] Google Scholar

[ref33] 33. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55.
View Article
Google Scholar

[105] View Article

[106] Google Scholar

[ref34] 34. Wirth RJ, Edwards MC. Item factor analysis: current approaches and future directions. Psychol Methods. 2007;12(1):58–79. pmid:17402812
View Article
PubMed/NCBI
Google Scholar

[108] View Article

[109] PubMed/NCBI

[110] Google Scholar

[ref35] 35. Flora DB, Flake JK. The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement. 2017;49(2):78–88.
View Article
Google Scholar

[112] View Article

[113] Google Scholar

[ref36] 36. Ryan J, Curtis R, Olds T, Edney S, Vandelanotte C, Plotnikoff R, et al. Psychometric properties of the PERMA Profiler for measuring wellbeing in Australian adults. PLoS One. 2019;14(12):e0225932. pmid:31869336
View Article
PubMed/NCBI
Google Scholar

[115] View Article

[116] PubMed/NCBI

[117] Google Scholar

[ref37] 37. Stevens J. Applied multivariate statistics for the social sciences. 4th ed. Mahwah, N.J: L. Erlbaum; 2002.

[ref38] 38. Aoki T, Inoue M, Nakayama T. Development and validation of the Japanese version of Primary Care Assessment Tool. Fam Pract. 2016;33(1):112–7. pmid:26546033
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

[ref39] 39. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5. pmid:28029643
View Article
PubMed/NCBI
Google Scholar

[124] View Article

[125] PubMed/NCBI

[126] Google Scholar

[ref40] 40. Hoa NT, Tam NM, Peersman W, Derese A, Markuns JF. Development and validation of the Vietnamese primary care assessment tool. PLoS One. 2018;13(1):e0191181. pmid:29324851
View Article
PubMed/NCBI
Google Scholar

[128] View Article

[129] PubMed/NCBI

[130] Google Scholar

[ref41] 41. Jeon K-Y. Cross-cultural adaptation of the US consumer form of the short Primary Care Assessment Tool (PCAT): the Korean consumer form of the short PCAT (KC PCAT) and the Korean standard form of the short PCAT (KS PCAT). Qual Prim Care. 2011;19(2):85–103. pmid:21575331
View Article
PubMed/NCBI
Google Scholar

[132] View Article

[133] PubMed/NCBI

[134] Google Scholar

[ref42] 42. Barnes K, Ceramidas D, Douglas K. Why patients attend after-hours medical services: a cross-sectional survey of patients across the Australian Capital Territory. Aust J Prim Health. 2022;28(6):549–55. pmid:36042687
View Article
PubMed/NCBI
Google Scholar

[136] View Article

[137] PubMed/NCBI

[138] Google Scholar

[ref43] 43. Australian Government Department of Health DaAC. About Medicare Urgent Care Clinics. Canberra: Department of Health. 2025 [Cited 2025 Nov 22]. Available from: https://www.health.gov.au/our-work/medicare-urgent-care-clinics/about-medicare-urgent-care-clinics?language=en

[ref44] 44. Kijima T, Matsushita A, Akai K, Hamano T, Takahashi S, Fujiwara K, et al. Patient satisfaction and loyalty in Japanese primary care: a cross-sectional study. BMC Health Serv Res. 2021;21(1):274. pmid:33766027
View Article
PubMed/NCBI
Google Scholar

[141] View Article

[142] PubMed/NCBI

[143] Google Scholar

[ref45] 45. Berra S, Rocha KB, Rodríguez-Sanz M, Pasarín MI, Rajmil L, Borrell C, et al. Properties of a short questionnaire for assessing primary care experiences for children in a population survey. BMC Public Health. 2011;11:285. pmid:21554717
View Article
PubMed/NCBI
Google Scholar

[145] View Article

[146] PubMed/NCBI

[147] Google Scholar

[ref46] 46. Mead N, Bower P, Roland M. The General Practice Assessment Questionnaire (GPAQ) - development and psychometric characteristics. BMC Fam Pract. 2008;9:13. pmid:18289385
View Article
PubMed/NCBI
Google Scholar

[149] View Article

[150] PubMed/NCBI

[151] Google Scholar

[ref47] 47. Lampe D, Grosser J, Gensorowsky D, Witte J, Muth C, van den Akker M, et al. The Relationship of Continuity of Care, Polypharmacy and Medication Appropriateness: A Systematic Review of Observational Studies. Drugs Aging. 2023;40(6):473–97. pmid:36972012
View Article
PubMed/NCBI
Google Scholar

[153] View Article

[154] PubMed/NCBI

[155] Google Scholar

[ref48] 48. Haggerty JL, Beaulieu M-D, Pineault R, Burge F, Lévesque J-F, Santor DA, et al. Comprehensiveness of care from the patient perspective: comparison of primary healthcare evaluation instruments. Healthc Policy. 2011;7(Spec Issue):154–66. pmid:23205042
View Article
PubMed/NCBI
Google Scholar

[157] View Article

[158] PubMed/NCBI

[159] Google Scholar

[ref49] 49. Attwooll J. ‘We are heading for a crisis’: Can GP home visits be revived?: The Royal Australian College of General Practitioners (RACGP). 2023 Nov 03 [Cited 2024 Sep 10]. Available from: https://www1.racgp.org.au/newsgp/professional/we-are-heading-for-a-crisis-can-gp-home-visits-be

[ref50] 50. Joyce C, Piterman L. Trends in GP home visits. Aust Fam Physician. 2008;37(12):1039–42. pmid:19142281
View Article
PubMed/NCBI
Google Scholar

[162] View Article

[163] PubMed/NCBI

[164] Google Scholar

[ref51] 51. Porter G, Blashki G, Grills N. General practice and public health: who is my patient? Aust Fam Physician. 2014;43(7):483–6. pmid:25006614
View Article
PubMed/NCBI
Google Scholar

[166] View Article

[167] PubMed/NCBI

[168] Google Scholar

[ref52] 52. Bresick GF, Sayed A-R, Le Grange C, Bhagwan S, Manga N, Hellenberg D. Western Cape Primary Care Assessment Tool (PCAT) study: Measuring primary care organisation and performance in the Western Cape Province, South Africa (2013). Afr J Prim Health Care Fam Med. 2016;8(1):e1–12. pmid:27247157
View Article
PubMed/NCBI
Google Scholar

[170] View Article

[171] PubMed/NCBI

[172] Google Scholar

[ref53] 53. Haggerty JL. Measurement of primary healthcare attributes from the patient perspective. Healthc Policy. 2011;7(Spec Issue):13–20. pmid:23205032
View Article
PubMed/NCBI
Google Scholar

[174] View Article

[175] PubMed/NCBI

[176] Google Scholar

[ref54] 54. Besigye IK, Mash R. Adaptation and validation of the Ugandan Primary Care Assessment Tool. Afr J Prim Health Care Fam Med. 2023;15(1):e1–7. pmid:36744453
View Article
PubMed/NCBI
Google Scholar

[178] View Article

[179] PubMed/NCBI

[180] Google Scholar

[ref55] 55. Shieh C-F. Assessing the Quality of Primary Health Care in a Remote County of Taiwan: ProQuest Dissertations Publishing; 2020.

[ref56] 56. Owolabi O, Zhang Z, Wei X, Yang N, Li H, Wong SYS, et al. Patients’ socioeconomic status and their evaluations of primary care in Hong Kong. BMC Health Serv Res. 2013;13:487. pmid:24274660
View Article
PubMed/NCBI
Google Scholar

[183] View Article

[184] PubMed/NCBI

[185] Google Scholar

[ref57] 57. ZEST Health Stategies. Patient experience measurement in primary health care. Sydney: Australian Commission on Safety and Quality in Health Care. 2023 April [Cited 2024 Sep 10]. Available from: https://www.safetyandquality.gov.au/publications-and-resources/resource-library/literature-review-patient-experience-primary-health-care

Figures

Abstract

Introduction

Method

Results

Conclusions

Introduction

Materials and methods

Participants and procedures

Data analysis

Results

Data quality

Factor structure

Internal consistency

Discussion

Supporting information

S1 Table. Adjustments to the PCAT-S for the EQuIP-GP trial.

S2 Table. Multicollinearity statistics.

S3 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, complete observations only (n = 180).

S4 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, developer-recommended imputation (n = 373).

S5 Table. Factor correlations of extracted factors from Exploratory Factor Analysis (EFA) using oblimin rotation, neutral-value imputation (n = 606).

S6 Table. Standardised factor loadings from CFA models by imputation method.

S7 Table. Factor loadings and extracted factors using complete observations only (n = 180).

S8 Table. Factor loadings and extracted factors, developer-recommended imputation (n = 373).

S9 Table. Factor loadings and extracted factors using neutral-value imputation (n = 606).

Acknowledgments

References